{
  "id": 587902,
  "title": "Covariate Shift in the data between train and test",
  "url": "/competitions/drw-crypto-market-prediction/discussion/587902",
  "author_name": "",
  "post_date": "2025-07-03T10:58:03.661260300Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I have train a 5-fold Xgboost regressor with the train dataframe an got the following performances on the differents validation folders:<br>\nFold 1 Pearson: 0.52704<br>\nFold 2 Pearson: 0.52433<br>\nFold 3 Pearson: 0.51880<br>\nFold 4 Pearson: 0.52876<br>\nFold 5 Pearson: 0.52380</p>\n<p>That sounds good, but when I predict on the test set I got a leaderboard score of: Pearson: 0.087.</p>\n<p>I suspected a covariate shift between the train and the test set, and tried to see if I was right. Then I plot the means of the different columns that I used for training and got this:</p>\n<p>X863 | Train mean: -0.0094 | Test mean: -0.0058<br>\nX294 | Train mean: 0.1059 | Test mean: -0.1811<br>\nsell_qty | Train mean: 132.6739 | Test mean: 89.7594<br>\nX168 | Train mean: 0.1474 | Test mean: 0.5112<br>\nX674 | Train mean: 0.3890 | Test mean: 1.1230<br>\nX299 | Train mean: 0.1955 | Test mean: -0.1873<br>\nX288 | Train mean: 0.0286 | Test mean: -0.1606<br>\nX29 | Train mean: -0.0006 | Test mean: 0.0025<br>\nX198 | Train mean: -0.5492 | Test mean: -0.8954<br>\nX137 | Train mean: 0.0558 | Test mean: 0.1624<br>\nX218 | Train mean: -0.1070 | Test mean: -0.2142<br>\nX179 | Train mean: 0.1142 | Test mean: 0.2454<br>\nX421 | Train mean: 0.1799 | Test mean: -0.0129<br>\nX173 | Train mean: 0.0569 | Test mean: 0.1233<br>\nX852 | Train mean: 0.8138 | Test mean: 0.7686<br>\nX197 | Train mean: -0.2727 | Test mean: -0.4493<br>\nX524 | Train mean: 0.0157 | Test mean: -0.0031<br>\nX860 | Train mean: -0.0114 | Test mean: -0.0196<br>\nX531 | Train mean: 0.0132 | Test mean: 0.0016<br>\nX302 | Train mean: 0.2316 | Test mean: -0.1789<br>\nX301 | Train mean: 0.2015 | Test mean: -0.1832<br>\nX466 | Train mean: -0.4203 | Test mean: -0.4882<br>\nX855 | Train mean: 0.7103 | Test mean: 0.6917<br>\nask_qty | Train mean: 10.1742 | Test mean: 6.5443<br>\nX297 | Train mean: 0.0925 | Test mean: -0.1655<br>\nX292 | Train mean: 0.0994 | Test mean: -0.1840<br>\nX225 | Train mean: -0.1468 | Test mean: -0.2191<br>\nX26 | Train mean: -0.0003 | Test mean: 0.0005<br>\nX174 | Train mean: 0.1473 | Test mean: 0.5108<br>\nX20 | Train mean: -0.0004 | Test mean: 0.0005<br>\nX888 | Train mean: 0.4975 | Test mean: 0.5048<br>\nX27 | Train mean: -0.0002 | Test mean: 0.0007<br>\nX23 | Train mean: -0.0019 | Test mean: 0.0021<br>\nX295 | Train mean: 0.0819 | Test mean: -0.1693<br>\nX612 | Train mean: 0.5279 | Test mean: -0.0059<br>\nbuy_qty | Train mean: 131.7267 | Test mean: 88.2782<br>\nvolume | Train mean: 264.4006 | Test mean: 178.0376<br>\nX175 | Train mean: -0.0355 | Test mean: -0.1932<br>\nX30 | Train mean: -0.0010 | Test mean: 0.0030<br>\nX603 | Train mean: 0.1570 | Test mean: 0.1178<br>\nX281 | Train mean: -0.0549 | Test mean: -0.1502<br>\nX19 | Train mean: -0.0005 | Test mean: 0.0006<br>\nX178 | Train mean: 0.0331 | Test mean: 0.0360<br>\nX287 | Train mean: 0.0052 | Test mean: -0.1423<br>\nX857 | Train mean: 0.0314 | Test mean: 0.1251<br>\nX285 | Train mean: -0.0557 | Test mean: -0.1701<br>\nX298 | Train mean: 0.2184 | Test mean: -0.1931<br>\nX861 | Train mean: 0.0381 | Test mean: 0.0587<br>\nX598 | Train mean: 0.0299 | Test mean: -0.0089<br>\nX28 | Train mean: -0.0004 | Test mean: 0.0013<br>\nbid_qty | Train mean: 9.9680 | Test mean: 6.3980<br>\nX293 | Train mean: 0.0752 | Test mean: -0.1710<br>\nX169 | Train mean: -0.0355 | Test mean: -0.1936<br>\nX21 | Train mean: -0.0006 | Test mean: 0.0009<br>\nX300 | Train mean: 0.2237 | Test mean: -0.1878<br>\nX40 | Train mean: 0.3845 | Test mean: 0.7900<br>\nX217 | Train mean: -0.1054 | Test mean: -0.2166<br>\nX858 | Train mean: 0.0241 | Test mean: 0.0630<br>\nX22 | Train mean: -0.0008 | Test mean: 0.0012<br>\nX291 | Train mean: 0.0175 | Test mean: -0.1435<br>\nX289 | Train mean: 0.0098 | Test mean: -0.1433<br>\nX385 | Train mean: 0.0703 | Test mean: -0.0607<br>\nX862 | Train mean: -0.5205 | Test mean: -0.5671<br>\nX345 | Train mean: 0.0235 | Test mean: 0.1589<br>\nX415 | Train mean: 0.1797 | Test mean: -0.0124<br>\nX286 | Train mean: 0.0239 | Test mean: -0.1602<br>\nX465 | Train mean: -0.2071 | Test mean: -0.2488<br>\nX303 | Train mean: 0.2105 | Test mean: -0.1762<br>\nX296 | Train mean: 0.1161 | Test mean: -0.1750<br>\nX856 | Train mean: -0.0129 | Test mean: 0.0532<br>\nX226 | Train mean: -0.1482 | Test mean: -0.2253<br>\nX333 | Train mean: 0.0123 | Test mean: 0.0793<br>\nX283 | Train mean: -0.0554 | Test mean: -0.1586<br>\nX219 | Train mean: -0.1104 | Test mean: -0.2099<br>\nX181 | Train mean: -0.0356 | Test mean: -0.1925<br>\nX290 | Train mean: 0.0363 | Test mean: -0.1593<br>\nX18 | Train mean: -0.0002 | Test mean: 0.0002<br>\nvolume_weighted_sell | Train mean: 209097.1370 | Test mean: 67871.4785<br>\nbuy_sell_ratio | Train mean: 1.5021 | Test mean: 1.4390<br>\nselling_pressure | Train mean: 0.5033 | Test mean: 0.5052<br>\neffective_spread_proxy | Train mean: 0.3118 | Test mean: 0.3084</p>\n<p>This statistics are showing clearly the covariate shift problem in this competition.</p>\n<p>Does someone has an idea on how to deal with that ?</p>",
  "messages": [
    {
      "id": "3239989",
      "postDate": "07/03/2025 10:58:03",
      "content": "<p>I have train a 5-fold Xgboost regressor with the train dataframe an got the following performances on the differents validation folders:<br>\nFold 1 Pearson: 0.52704<br>\nFold 2 Pearson: 0.52433<br>\nFold 3 Pearson: 0.51880<br>\nFold 4 Pearson: 0.52876<br>\nFold 5 Pearson: 0.52380</p>\n<p>That sounds good, but when I predict on the test set I got a leaderboard score of: Pearson: 0.087.</p>\n<p>I suspected a covariate shift between the train and the test set, and tried to see if I was right. Then I plot the means of the different columns that I used for training and got this:</p>\n<p>X863 | Train mean: -0.0094 | Test mean: -0.0058<br>\nX294 | Train mean: 0.1059 | Test mean: -0.1811<br>\nsell_qty | Train mean: 132.6739 | Test mean: 89.7594<br>\nX168 | Train mean: 0.1474 | Test mean: 0.5112<br>\nX674 | Train mean: 0.3890 | Test mean: 1.1230<br>\nX299 | Train mean: 0.1955 | Test mean: -0.1873<br>\nX288 | Train mean: 0.0286 | Test mean: -0.1606<br>\nX29 | Train mean: -0.0006 | Test mean: 0.0025<br>\nX198 | Train mean: -0.5492 | Test mean: -0.8954<br>\nX137 | Train mean: 0.0558 | Test mean: 0.1624<br>\nX218 | Train mean: -0.1070 | Test mean: -0.2142<br>\nX179 | Train mean: 0.1142 | Test mean: 0.2454<br>\nX421 | Train mean: 0.1799 | Test mean: -0.0129<br>\nX173 | Train mean: 0.0569 | Test mean: 0.1233<br>\nX852 | Train mean: 0.8138 | Test mean: 0.7686<br>\nX197 | Train mean: -0.2727 | Test mean: -0.4493<br>\nX524 | Train mean: 0.0157 | Test mean: -0.0031<br>\nX860 | Train mean: -0.0114 | Test mean: -0.0196<br>\nX531 | Train mean: 0.0132 | Test mean: 0.0016<br>\nX302 | Train mean: 0.2316 | Test mean: -0.1789<br>\nX301 | Train mean: 0.2015 | Test mean: -0.1832<br>\nX466 | Train mean: -0.4203 | Test mean: -0.4882<br>\nX855 | Train mean: 0.7103 | Test mean: 0.6917<br>\nask_qty | Train mean: 10.1742 | Test mean: 6.5443<br>\nX297 | Train mean: 0.0925 | Test mean: -0.1655<br>\nX292 | Train mean: 0.0994 | Test mean: -0.1840<br>\nX225 | Train mean: -0.1468 | Test mean: -0.2191<br>\nX26 | Train mean: -0.0003 | Test mean: 0.0005<br>\nX174 | Train mean: 0.1473 | Test mean: 0.5108<br>\nX20 | Train mean: -0.0004 | Test mean: 0.0005<br>\nX888 | Train mean: 0.4975 | Test mean: 0.5048<br>\nX27 | Train mean: -0.0002 | Test mean: 0.0007<br>\nX23 | Train mean: -0.0019 | Test mean: 0.0021<br>\nX295 | Train mean: 0.0819 | Test mean: -0.1693<br>\nX612 | Train mean: 0.5279 | Test mean: -0.0059<br>\nbuy_qty | Train mean: 131.7267 | Test mean: 88.2782<br>\nvolume | Train mean: 264.4006 | Test mean: 178.0376<br>\nX175 | Train mean: -0.0355 | Test mean: -0.1932<br>\nX30 | Train mean: -0.0010 | Test mean: 0.0030<br>\nX603 | Train mean: 0.1570 | Test mean: 0.1178<br>\nX281 | Train mean: -0.0549 | Test mean: -0.1502<br>\nX19 | Train mean: -0.0005 | Test mean: 0.0006<br>\nX178 | Train mean: 0.0331 | Test mean: 0.0360<br>\nX287 | Train mean: 0.0052 | Test mean: -0.1423<br>\nX857 | Train mean: 0.0314 | Test mean: 0.1251<br>\nX285 | Train mean: -0.0557 | Test mean: -0.1701<br>\nX298 | Train mean: 0.2184 | Test mean: -0.1931<br>\nX861 | Train mean: 0.0381 | Test mean: 0.0587<br>\nX598 | Train mean: 0.0299 | Test mean: -0.0089<br>\nX28 | Train mean: -0.0004 | Test mean: 0.0013<br>\nbid_qty | Train mean: 9.9680 | Test mean: 6.3980<br>\nX293 | Train mean: 0.0752 | Test mean: -0.1710<br>\nX169 | Train mean: -0.0355 | Test mean: -0.1936<br>\nX21 | Train mean: -0.0006 | Test mean: 0.0009<br>\nX300 | Train mean: 0.2237 | Test mean: -0.1878<br>\nX40 | Train mean: 0.3845 | Test mean: 0.7900<br>\nX217 | Train mean: -0.1054 | Test mean: -0.2166<br>\nX858 | Train mean: 0.0241 | Test mean: 0.0630<br>\nX22 | Train mean: -0.0008 | Test mean: 0.0012<br>\nX291 | Train mean: 0.0175 | Test mean: -0.1435<br>\nX289 | Train mean: 0.0098 | Test mean: -0.1433<br>\nX385 | Train mean: 0.0703 | Test mean: -0.0607<br>\nX862 | Train mean: -0.5205 | Test mean: -0.5671<br>\nX345 | Train mean: 0.0235 | Test mean: 0.1589<br>\nX415 | Train mean: 0.1797 | Test mean: -0.0124<br>\nX286 | Train mean: 0.0239 | Test mean: -0.1602<br>\nX465 | Train mean: -0.2071 | Test mean: -0.2488<br>\nX303 | Train mean: 0.2105 | Test mean: -0.1762<br>\nX296 | Train mean: 0.1161 | Test mean: -0.1750<br>\nX856 | Train mean: -0.0129 | Test mean: 0.0532<br>\nX226 | Train mean: -0.1482 | Test mean: -0.2253<br>\nX333 | Train mean: 0.0123 | Test mean: 0.0793<br>\nX283 | Train mean: -0.0554 | Test mean: -0.1586<br>\nX219 | Train mean: -0.1104 | Test mean: -0.2099<br>\nX181 | Train mean: -0.0356 | Test mean: -0.1925<br>\nX290 | Train mean: 0.0363 | Test mean: -0.1593<br>\nX18 | Train mean: -0.0002 | Test mean: 0.0002<br>\nvolume_weighted_sell | Train mean: 209097.1370 | Test mean: 67871.4785<br>\nbuy_sell_ratio | Train mean: 1.5021 | Test mean: 1.4390<br>\nselling_pressure | Train mean: 0.5033 | Test mean: 0.5052<br>\neffective_spread_proxy | Train mean: 0.3118 | Test mean: 0.3084</p>\n<p>This statistics are showing clearly the covariate shift problem in this competition.</p>\n<p>Does someone has an idea on how to deal with that ?</p>",
      "rawMarkdown": "I have train a 5-fold Xgboost regressor with the train dataframe an got the following performances on the differents validation folders:\nFold 1 Pearson: 0.52704\nFold 2 Pearson: 0.52433\nFold 3 Pearson: 0.51880\nFold 4 Pearson: 0.52876\nFold 5 Pearson: 0.52380\n\nThat sounds good, but when I predict on the test set I got a leaderboard score of: Pearson: 0.087.\n\nI suspected a covariate shift between the train and the test set, and tried to see if I was right. Then I plot the means of the different columns that I used for training and got this:\n\nX863 | Train mean: -0.0094 | Test mean: -0.0058\nX294 | Train mean: 0.1059 | Test mean: -0.1811\nsell_qty | Train mean: 132.6739 | Test mean: 89.7594\nX168 | Train mean: 0.1474 | Test mean: 0.5112\nX674 | Train mean: 0.3890 | Test mean: 1.1230\nX299 | Train mean: 0.1955 | Test mean: -0.1873\nX288 | Train mean: 0.0286 | Test mean: -0.1606\nX29 | Train mean: -0.0006 | Test mean: 0.0025\nX198 | Train mean: -0.5492 | Test mean: -0.8954\nX137 | Train mean: 0.0558 | Test mean: 0.1624\nX218 | Train mean: -0.1070 | Test mean: -0.2142\nX179 | Train mean: 0.1142 | Test mean: 0.2454\nX421 | Train mean: 0.1799 | Test mean: -0.0129\nX173 | Train mean: 0.0569 | Test mean: 0.1233\nX852 | Train mean: 0.8138 | Test mean: 0.7686\nX197 | Train mean: -0.2727 | Test mean: -0.4493\nX524 | Train mean: 0.0157 | Test mean: -0.0031\nX860 | Train mean: -0.0114 | Test mean: -0.0196\nX531 | Train mean: 0.0132 | Test mean: 0.0016\nX302 | Train mean: 0.2316 | Test mean: -0.1789\nX301 | Train mean: 0.2015 | Test mean: -0.1832\nX466 | Train mean: -0.4203 | Test mean: -0.4882\nX855 | Train mean: 0.7103 | Test mean: 0.6917\nask_qty | Train mean: 10.1742 | Test mean: 6.5443\nX297 | Train mean: 0.0925 | Test mean: -0.1655\nX292 | Train mean: 0.0994 | Test mean: -0.1840\nX225 | Train mean: -0.1468 | Test mean: -0.2191\nX26 | Train mean: -0.0003 | Test mean: 0.0005\nX174 | Train mean: 0.1473 | Test mean: 0.5108\nX20 | Train mean: -0.0004 | Test mean: 0.0005\nX888 | Train mean: 0.4975 | Test mean: 0.5048\nX27 | Train mean: -0.0002 | Test mean: 0.0007\nX23 | Train mean: -0.0019 | Test mean: 0.0021\nX295 | Train mean: 0.0819 | Test mean: -0.1693\nX612 | Train mean: 0.5279 | Test mean: -0.0059\nbuy_qty | Train mean: 131.7267 | Test mean: 88.2782\nvolume | Train mean: 264.4006 | Test mean: 178.0376\nX175 | Train mean: -0.0355 | Test mean: -0.1932\nX30 | Train mean: -0.0010 | Test mean: 0.0030\nX603 | Train mean: 0.1570 | Test mean: 0.1178\nX281 | Train mean: -0.0549 | Test mean: -0.1502\nX19 | Train mean: -0.0005 | Test mean: 0.0006\nX178 | Train mean: 0.0331 | Test mean: 0.0360\nX287 | Train mean: 0.0052 | Test mean: -0.1423\nX857 | Train mean: 0.0314 | Test mean: 0.1251\nX285 | Train mean: -0.0557 | Test mean: -0.1701\nX298 | Train mean: 0.2184 | Test mean: -0.1931\nX861 | Train mean: 0.0381 | Test mean: 0.0587\nX598 | Train mean: 0.0299 | Test mean: -0.0089\nX28 | Train mean: -0.0004 | Test mean: 0.0013\nbid_qty | Train mean: 9.9680 | Test mean: 6.3980\nX293 | Train mean: 0.0752 | Test mean: -0.1710\nX169 | Train mean: -0.0355 | Test mean: -0.1936\nX21 | Train mean: -0.0006 | Test mean: 0.0009\nX300 | Train mean: 0.2237 | Test mean: -0.1878\nX40 | Train mean: 0.3845 | Test mean: 0.7900\nX217 | Train mean: -0.1054 | Test mean: -0.2166\nX858 | Train mean: 0.0241 | Test mean: 0.0630\nX22 | Train mean: -0.0008 | Test mean: 0.0012\nX291 | Train mean: 0.0175 | Test mean: -0.1435\nX289 | Train mean: 0.0098 | Test mean: -0.1433\nX385 | Train mean: 0.0703 | Test mean: -0.0607\nX862 | Train mean: -0.5205 | Test mean: -0.5671\nX345 | Train mean: 0.0235 | Test mean: 0.1589\nX415 | Train mean: 0.1797 | Test mean: -0.0124\nX286 | Train mean: 0.0239 | Test mean: -0.1602\nX465 | Train mean: -0.2071 | Test mean: -0.2488\nX303 | Train mean: 0.2105 | Test mean: -0.1762\nX296 | Train mean: 0.1161 | Test mean: -0.1750\nX856 | Train mean: -0.0129 | Test mean: 0.0532\nX226 | Train mean: -0.1482 | Test mean: -0.2253\nX333 | Train mean: 0.0123 | Test mean: 0.0793\nX283 | Train mean: -0.0554 | Test mean: -0.1586\nX219 | Train mean: -0.1104 | Test mean: -0.2099\nX181 | Train mean: -0.0356 | Test mean: -0.1925\nX290 | Train mean: 0.0363 | Test mean: -0.1593\nX18 | Train mean: -0.0002 | Test mean: 0.0002\nvolume_weighted_sell | Train mean: 209097.1370 | Test mean: 67871.4785\nbuy_sell_ratio | Train mean: 1.5021 | Test mean: 1.4390\nselling_pressure | Train mean: 0.5033 | Test mean: 0.5052\neffective_spread_proxy | Train mean: 0.3118 | Test mean: 0.3084\n\nThis statistics are showing clearly the covariate shift problem in this competition.\n\nDoes someone has an idea on how to deal with that ?",
      "votes": null
    },
    {
      "id": "3240005",
      "postDate": "07/03/2025 11:26:08",
      "content": "<p>Your very high correlation results on the test data are most likelly due to the fact that you are using KFold validation. So there is a large degree of temporal leakage, meaning that your model is training on the future to predict the past which is many times easier than the reverse. This is because it can see which pattern remain stable in the future and which do not. You should use a temporal cross validation technique and you will see that the results from it match the public ones a lot better. Something like WalkForwardCV or TimeSeriesSplit</p>",
      "rawMarkdown": "Your very high correlation results on the test data are most likelly due to the fact that you are using KFold validation. So there is a large degree of temporal leakage, meaning that your model is training on the future to predict the past which is many times easier than the reverse. This is because it can see which pattern remain stable in the future and which do not. You should use a temporal cross validation technique and you will see that the results from it match the public ones a lot better. Something like WalkForwardCV or TimeSeriesSplit",
      "votes": null
    },
    {
      "id": "3240138",
      "postDate": "07/03/2025 13:48:48",
      "content": "<p>is shuffle=true in your KFold method? change it to False then cv and lb score will be much aligned.</p>",
      "rawMarkdown": "is shuffle=true in your KFold method? change it to False then cv and lb score will be much aligned.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3240005,
      "author_name": "stefanconstantin",
      "author_url": "",
      "post_date": "07/03/2025 11:26:08",
      "content": "<p>Your very high correlation results on the test data are most likelly due to the fact that you are using KFold validation. So there is a large degree of temporal leakage, meaning that your model is training on the future to predict the past which is many times easier than the reverse. This is because it can see which pattern remain stable in the future and which do not. You should use a temporal cross validation technique and you will see that the results from it match the public ones a lot better. Something like WalkForwardCV or TimeSeriesSplit</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3240138,
      "author_name": "yuhengjia",
      "author_url": "",
      "post_date": "07/03/2025 13:48:48",
      "content": "<p>is shuffle=true in your KFold method? change it to False then cv and lb score will be much aligned.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3239989": "I have train a 5-fold Xgboost regressor with the train dataframe an got the following performances on the differents validation folders:\nFold 1 Pearson: 0.52704\nFold 2 Pearson: 0.52433\nFold 3 Pearson: 0.51880\nFold 4 Pearson: 0.52876\nFold 5 Pearson: 0.52380\n\nThat sounds good, but when I predict on the test set I got a leaderboard score of: Pearson: 0.087.\n\nI suspected a covariate shift between the train and the test set, and tried to see if I was right. Then I plot the means of the different columns that I used for training and got this:\n\nX863 | Train mean: -0.0094 | Test mean: -0.0058\nX294 | Train mean: 0.1059 | Test mean: -0.1811\nsell_qty | Train mean: 132.6739 | Test mean: 89.7594\nX168 | Train mean: 0.1474 | Test mean: 0.5112\nX674 | Train mean: 0.3890 | Test mean: 1.1230\nX299 | Train mean: 0.1955 | Test mean: -0.1873\nX288 | Train mean: 0.0286 | Test mean: -0.1606\nX29 | Train mean: -0.0006 | Test mean: 0.0025\nX198 | Train mean: -0.5492 | Test mean: -0.8954\nX137 | Train mean: 0.0558 | Test mean: 0.1624\nX218 | Train mean: -0.1070 | Test mean: -0.2142\nX179 | Train mean: 0.1142 | Test mean: 0.2454\nX421 | Train mean: 0.1799 | Test mean: -0.0129\nX173 | Train mean: 0.0569 | Test mean: 0.1233\nX852 | Train mean: 0.8138 | Test mean: 0.7686\nX197 | Train mean: -0.2727 | Test mean: -0.4493\nX524 | Train mean: 0.0157 | Test mean: -0.0031\nX860 | Train mean: -0.0114 | Test mean: -0.0196\nX531 | Train mean: 0.0132 | Test mean: 0.0016\nX302 | Train mean: 0.2316 | Test mean: -0.1789\nX301 | Train mean: 0.2015 | Test mean: -0.1832\nX466 | Train mean: -0.4203 | Test mean: -0.4882\nX855 | Train mean: 0.7103 | Test mean: 0.6917\nask_qty | Train mean: 10.1742 | Test mean: 6.5443\nX297 | Train mean: 0.0925 | Test mean: -0.1655\nX292 | Train mean: 0.0994 | Test mean: -0.1840\nX225 | Train mean: -0.1468 | Test mean: -0.2191\nX26 | Train mean: -0.0003 | Test mean: 0.0005\nX174 | Train mean: 0.1473 | Test mean: 0.5108\nX20 | Train mean: -0.0004 | Test mean: 0.0005\nX888 | Train mean: 0.4975 | Test mean: 0.5048\nX27 | Train mean: -0.0002 | Test mean: 0.0007\nX23 | Train mean: -0.0019 | Test mean: 0.0021\nX295 | Train mean: 0.0819 | Test mean: -0.1693\nX612 | Train mean: 0.5279 | Test mean: -0.0059\nbuy_qty | Train mean: 131.7267 | Test mean: 88.2782\nvolume | Train mean: 264.4006 | Test mean: 178.0376\nX175 | Train mean: -0.0355 | Test mean: -0.1932\nX30 | Train mean: -0.0010 | Test mean: 0.0030\nX603 | Train mean: 0.1570 | Test mean: 0.1178\nX281 | Train mean: -0.0549 | Test mean: -0.1502\nX19 | Train mean: -0.0005 | Test mean: 0.0006\nX178 | Train mean: 0.0331 | Test mean: 0.0360\nX287 | Train mean: 0.0052 | Test mean: -0.1423\nX857 | Train mean: 0.0314 | Test mean: 0.1251\nX285 | Train mean: -0.0557 | Test mean: -0.1701\nX298 | Train mean: 0.2184 | Test mean: -0.1931\nX861 | Train mean: 0.0381 | Test mean: 0.0587\nX598 | Train mean: 0.0299 | Test mean: -0.0089\nX28 | Train mean: -0.0004 | Test mean: 0.0013\nbid_qty | Train mean: 9.9680 | Test mean: 6.3980\nX293 | Train mean: 0.0752 | Test mean: -0.1710\nX169 | Train mean: -0.0355 | Test mean: -0.1936\nX21 | Train mean: -0.0006 | Test mean: 0.0009\nX300 | Train mean: 0.2237 | Test mean: -0.1878\nX40 | Train mean: 0.3845 | Test mean: 0.7900\nX217 | Train mean: -0.1054 | Test mean: -0.2166\nX858 | Train mean: 0.0241 | Test mean: 0.0630\nX22 | Train mean: -0.0008 | Test mean: 0.0012\nX291 | Train mean: 0.0175 | Test mean: -0.1435\nX289 | Train mean: 0.0098 | Test mean: -0.1433\nX385 | Train mean: 0.0703 | Test mean: -0.0607\nX862 | Train mean: -0.5205 | Test mean: -0.5671\nX345 | Train mean: 0.0235 | Test mean: 0.1589\nX415 | Train mean: 0.1797 | Test mean: -0.0124\nX286 | Train mean: 0.0239 | Test mean: -0.1602\nX465 | Train mean: -0.2071 | Test mean: -0.2488\nX303 | Train mean: 0.2105 | Test mean: -0.1762\nX296 | Train mean: 0.1161 | Test mean: -0.1750\nX856 | Train mean: -0.0129 | Test mean: 0.0532\nX226 | Train mean: -0.1482 | Test mean: -0.2253\nX333 | Train mean: 0.0123 | Test mean: 0.0793\nX283 | Train mean: -0.0554 | Test mean: -0.1586\nX219 | Train mean: -0.1104 | Test mean: -0.2099\nX181 | Train mean: -0.0356 | Test mean: -0.1925\nX290 | Train mean: 0.0363 | Test mean: -0.1593\nX18 | Train mean: -0.0002 | Test mean: 0.0002\nvolume_weighted_sell | Train mean: 209097.1370 | Test mean: 67871.4785\nbuy_sell_ratio | Train mean: 1.5021 | Test mean: 1.4390\nselling_pressure | Train mean: 0.5033 | Test mean: 0.5052\neffective_spread_proxy | Train mean: 0.3118 | Test mean: 0.3084\n\nThis statistics are showing clearly the covariate shift problem in this competition.\n\nDoes someone has an idea on how to deal with that ?",
    "3240005": "Your very high correlation results on the test data are most likelly due to the fact that you are using KFold validation. So there is a large degree of temporal leakage, meaning that your model is training on the future to predict the past which is many times easier than the reverse. This is because it can see which pattern remain stable in the future and which do not. You should use a temporal cross validation technique and you will see that the results from it match the public ones a lot better. Something like WalkForwardCV or TimeSeriesSplit",
    "3240138": "is shuffle=true in your KFold method? change it to False then cv and lb score will be much aligned."
  },
  "source": "meta"
}