![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4172517%2Ff206215699c522a7197223c5645ef485%2FIrrrMG_20231127_141535.jpg?generation=1701083967056127&alt=media\")
As the title suggests…
", "messages": [ { "id": "2539606", "postDate": "11/27/2023 07:34:58", "content": "As the title suggests…
", "rawMarkdown": "As the title suggests...", "votes": null }, { "id": "2539622", "postDate": "11/27/2023 07:57:01", "content": "Pseudolabelling makes sense if you could know which part is public. Otherwise your model may predict something similar to the public kernel for the private part, which you may want to avoid.
", "rawMarkdown": "Pseudolabelling makes sense if you could know which part is public. Otherwise your model may predict something similar to the public kernel for the private part, which you may want to avoid.", "votes": null }, { "id": "2539673", "postDate": "11/27/2023 08:56:43", "content": "Exactly the thing I am asking myself !
\nI think - NO (but not sure), because:
\nalthough the rate is impressive the relative change is small - \"normal\" publics are about 0.565+, now we have 0.531,
\nso the error is still quite big.
So we can make such a local experiment:
\nsplit local train - test:
\ntake some labels from the local train, and distort them such that error would be about 0.565 and 0.531
\ntrain on such two variatns of labels
\nand compare the results on local test
\nI guess the difference will be very small
If I have more time - I will try to do that, but not sure.
\nPS
\n@aerdem4
\nthe split to public and private is given to us - see the image on the \"overview\" page.
\nThese public notebooks seems to work only on public LB and most probably fall down on private,
\nalthouth there is small chance that some ideas from them might be useful - have not thought on that much…
input: (1,18116)
\noutput: (1,1) the \"constant_value\".
The constant that minimizes the RMSE for SM_name.
\nThe constant_value is the value of x that minimizes the function Y.
\nInput data:
\n The constant_value train 600 rows: rmse_LB_sort_TRAIN_whide2_lb.csv
\n The constant_value submit 100 rows (remove constant_value submit 77*2 rows - forecast is weak.).
https://www.kaggle.com/code/olegpush/op2-eda-lb?scriptVersionId=152467205
\nthe output of version 38.
It is important to ensure that model has not been trained on LB sm_names only.
\nI calculate the loss for each iteration by:
\nI used Model 1 to \"move up or move down\" Model 2 prediction for sm_name:
\nwhere C0 -C18116 - prediction of Model2.
\nCalculate using Algorithm1 constant_value for rows 0-255 submition.csv of best public blend.
\nFor each row in submition:
\n Y_pred = constant_value_Model1 \n + Y_pred(:)_Model2 \n - constant_value_Model2 \n
I also think that these blends are likely to be useless for the private LB.
\nThe last two weeks of this competition look like a crowdsourced overfitting project, in places down to the granularity of choosing whichever model gets the lowest score on each item of the public LB data.
\nObviously enough, each of us needs to decide whether to completely ignore these \"blends of blends\", or whether we think there might somehow be a way of using them to improve our private LB score.
\nMy own opinion is that there was probably a point in the history of this competition up to which the blends had some value, but that we are now well past that and the new developments are just overfitting.
\nOther opinions are available.
", "rawMarkdown": "I also think that these blends are likely to be useless for the private LB.\n\nThe last two weeks of this competition look like a crowdsourced overfitting project, in places down to the granularity of choosing whichever model gets the lowest score on each item of the public LB data.\n\nObviously enough, each of us needs to decide whether to completely ignore these \"blends of blends\", or whether we think there might somehow be a way of using them to improve our private LB score.\n\nMy own opinion is that there was probably a point in the history of this competition up to which the blends had some value, but that we are now well past that and the new developments are just overfitting.\n\nOther opinions are available.", "votes": null } ], "comments": [ { "id": 2539622, "author_name": "aerdem4", "author_url": "", "post_date": "11/27/2023 07:57:01", "content": "Pseudolabelling makes sense if you could know which part is public. Otherwise your model may predict something similar to the public kernel for the private part, which you may want to avoid.
", "votes": null, "replies": [] }, { "id": 2539673, "author_name": "alexandervc", "author_url": "", "post_date": "11/27/2023 08:56:43", "content": "Exactly the thing I am asking myself !
\nI think - NO (but not sure), because:
\nalthough the rate is impressive the relative change is small - \"normal\" publics are about 0.565+, now we have 0.531,
\nso the error is still quite big.
So we can make such a local experiment:
\nsplit local train - test:
\ntake some labels from the local train, and distort them such that error would be about 0.565 and 0.531
\ntrain on such two variatns of labels
\nand compare the results on local test
\nI guess the difference will be very small
If I have more time - I will try to do that, but not sure.
\nPS
\n@aerdem4
\nthe split to public and private is given to us - see the image on the \"overview\" page.
\nThese public notebooks seems to work only on public LB and most probably fall down on private,
\nalthouth there is small chance that some ideas from them might be useful - have not thought on that much…
I also think that these blends are likely to be useless for the private LB.
\nThe last two weeks of this competition look like a crowdsourced overfitting project, in places down to the granularity of choosing whichever model gets the lowest score on each item of the public LB data.
\nObviously enough, each of us needs to decide whether to completely ignore these \"blends of blends\", or whether we think there might somehow be a way of using them to improve our private LB score.
\nMy own opinion is that there was probably a point in the history of this competition up to which the blends had some value, but that we are now well past that and the new developments are just overfitting.
\nOther opinions are available.
", "votes": null, "replies": [] } ] }, { "id": 2539684, "author_name": "", "author_url": "", "post_date": "11/27/2023 09:06:28", "content": "input: (1,18116)
\noutput: (1,1) the \"constant_value\".
The constant that minimizes the RMSE for SM_name.
\nThe constant_value is the value of x that minimizes the function Y.
\nInput data:
\n The constant_value train 600 rows: rmse_LB_sort_TRAIN_whide2_lb.csv
\n The constant_value submit 100 rows (remove constant_value submit 77*2 rows - forecast is weak.).
https://www.kaggle.com/code/olegpush/op2-eda-lb?scriptVersionId=152467205
\nthe output of version 38.
It is important to ensure that model has not been trained on LB sm_names only.
\nI calculate the loss for each iteration by:
\nI used Model 1 to \"move up or move down\" Model 2 prediction for sm_name:
\nwhere C0 -C18116 - prediction of Model2.
\nCalculate using Algorithm1 constant_value for rows 0-255 submition.csv of best public blend.
\nFor each row in submition:
\n Y_pred = constant_value_Model1 \n + Y_pred(:)_Model2 \n - constant_value_Model2 \n