![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1342094%2F394e4da786137695529c33b72479b240%2Ftop-lb.png?generation=1734715479315940&alt=media\")
If I didn't make any calculation mistakes, this is what the top LB predictions look like.
\n
If so, the resulting model will be pretty much useless after the end of the challenge.
", "messages": [ { "id": 3077231, "postDate": "2024-12-20T17:37:17.270Z", "content": "How did you get the top LB model to generate the predicts?
", "rawMarkdown": "How did you get the top LB model to generate the predicts? ", "votes": 1, "replies": [ { "id": 3077236, "postDate": "2024-12-20T17:41:45.467Z", "content": "I am assuming they will be similar because of the weighted R2 score.
", "rawMarkdown": " I am assuming they will be similar because of the weighted R2 score.", "replies": [ { "id": 3077265, "postDate": "2024-12-20T18:10:08.040Z", "content": "Ok. What if you plot the predicted values on the right axis? It is expected that the predictions have much smaller value range than the real value.It is not ideal to plot them together on the same axis.
", "rawMarkdown": "Ok. What if you plot the predicted values on the right axis? It is expected that the predictions have much smaller value range than the real value.It is not ideal to plot them together on the same axis." }, { "id": 3077269, "postDate": "2024-12-20T18:15:53.573Z", "content": "Why is it expected to be in different range from the real value?
", "rawMarkdown": "Why is it expected to be in different range from the real value?" }, { "id": 3077397, "postDate": "2024-12-20T23:05:11.187Z", "content": "Bc of the long tail distribution of targets. It would be unrealistic to expect the model to have equivalent predictive capabilities in the full +/- 5 range. The model’s predictions are more likely to be within +/- 1 or +/- 0.5.
\nNote that not being able to cover the exact full dynamic value range does not mean the model is useless. A good correlation (which also relates to higher R2) between the predictions and the targets can also be very useful. This is very common in quant finance, i.e a high IC (information content) model.
\nSo, I would suggest visualizing your plot with two different axis and check if there’s any correlation (visually and quantitatively) when R2 is high (there should be).
", "rawMarkdown": "Bc of the long tail distribution of targets. It would be unrealistic to expect the model to have equivalent predictive capabilities in the full +/- 5 range. The model’s predictions are more likely to be within +/- 1 or +/- 0.5.\n\nNote that not being able to cover the exact full dynamic value range does not mean the model is useless. A good correlation (which also relates to higher R2) between the predictions and the targets can also be very useful. This is very common in quant finance, i.e a high IC (information content) model.\n\nSo, I would suggest visualizing your plot with two different axis and check if there’s any correlation (visually and quantitatively) when R2 is high (there should be).", "votes": 2 }, { "id": 3077411, "postDate": "2024-12-20T23:51:06.477Z", "content": "Good point. Thanks for the explanation.
", "rawMarkdown": "Good point. Thanks for the explanation." } ] } ] }, { "id": 3077640, "postDate": "2024-12-21T08:20:46.220Z", "content": "I think it's trivial to discuss this at this point. Models being how much useful is up to traders at Jane Street.
", "rawMarkdown": "I think it's trivial to discuss this at this point. Models being how much useful is up to traders at Jane Street." }, { "id": 3077222, "postDate": "2024-12-20T17:33:40.423Z", "content": "If I didn't make any calculation mistakes, this is what the top LB predictions look like.
\n
If so, the resulting model will be pretty much useless after the end of the challenge.
", "rawMarkdown": "If I didn't make any calculation mistakes, this is what the top LB predictions look like.\n\n\n\nIf so, the resulting model will be pretty much useless after the end of the challenge.\n", "votes": -3 }, { "id": 3082387, "postDate": "2024-12-28T05:16:47.157Z", "content": "@serjhenrique I agree. The current prediction is not meaningful at all. We probably have not figured out the right way to build the prediction model using the data JS provided
", "rawMarkdown": "@serjhenrique I agree. The current prediction is not meaningful at all. We probably have not figured out the right way to build the prediction model using the data JS provided" }, { "id": 3077569, "postDate": "2024-12-21T05:54:09.727Z", "content": "interesting…. I dont know what to say…
", "rawMarkdown": "interesting.... I dont know what to say..." }, { "id": 3082279, "postDate": "2024-12-27T22:56:55.560Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 3077231, "author_name": "SLi", "author_url": "", "post_date": "2024-12-20T17:37:17.270000", "content": "How did you get the top LB model to generate the predicts?
", "votes": 1, "replies": [ { "id": 3077236, "author_name": "Sergio Henrique", "author_url": "", "post_date": "2024-12-20T17:41:45.467000", "content": "I am assuming they will be similar because of the weighted R2 score.
", "votes": 0, "replies": [ { "id": 3077265, "author_name": "SLi", "author_url": "", "post_date": "2024-12-20T18:10:08.040000", "content": "Ok. What if you plot the predicted values on the right axis? It is expected that the predictions have much smaller value range than the real value.It is not ideal to plot them together on the same axis.
", "votes": 0, "replies": [] }, { "id": 3077269, "author_name": "Sergio Henrique", "author_url": "", "post_date": "2024-12-20T18:15:53.573000", "content": "Why is it expected to be in different range from the real value?
", "votes": 0, "replies": [] }, { "id": 3077397, "author_name": "SLi", "author_url": "", "post_date": "2024-12-20T23:05:11.187000", "content": "Bc of the long tail distribution of targets. It would be unrealistic to expect the model to have equivalent predictive capabilities in the full +/- 5 range. The model’s predictions are more likely to be within +/- 1 or +/- 0.5.
\nNote that not being able to cover the exact full dynamic value range does not mean the model is useless. A good correlation (which also relates to higher R2) between the predictions and the targets can also be very useful. This is very common in quant finance, i.e a high IC (information content) model.
\nSo, I would suggest visualizing your plot with two different axis and check if there’s any correlation (visually and quantitatively) when R2 is high (there should be).
", "votes": 2, "replies": [] }, { "id": 3077411, "author_name": "Sergio Henrique", "author_url": "", "post_date": "2024-12-20T23:51:06.477000", "content": "Good point. Thanks for the explanation.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3077640, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2024-12-21T08:20:46.220000", "content": "I think it's trivial to discuss this at this point. Models being how much useful is up to traders at Jane Street.
", "votes": 0, "replies": [] }, { "id": 3082387, "author_name": "Leon Jiang", "author_url": "", "post_date": "2024-12-28T05:16:47.157000", "content": "@serjhenrique I agree. The current prediction is not meaningful at all. We probably have not figured out the right way to build the prediction model using the data JS provided
", "votes": 0, "replies": [] }, { "id": 3077569, "author_name": "ZT", "author_url": "", "post_date": "2024-12-21T05:54:09.727000", "content": "interesting…. I dont know what to say…
", "votes": 0, "replies": [] }, { "id": 3082279, "author_name": "", "author_url": "", "post_date": "2024-12-27T22:56:55.560000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3077231": "How did you get the top LB model to generate the predicts? ", "3077640": "I think it's trivial to discuss this at this point. Models being how much useful is up to traders at Jane Street.", "3077222": "If I didn't make any calculation mistakes, this is what the top LB predictions look like.\n\n\n\nIf so, the resulting model will be pretty much useless after the end of the challenge.\n", "3082387": "@serjhenrique I agree. The current prediction is not meaningful at all. We probably have not figured out the right way to build the prediction model using the data JS provided", "3077569": "interesting.... I dont know what to say...", "3082279": "" } }