PseudoLabeling. \n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1528571%2Fb2c58b100fb88e3a981fcf3673b18e17%2Fpsuedo.png?generation=1567008187143187&alt=media\")
The difference in LB and CV has already been mentioned many times like here and here. This could be due to difference in train and test(public+private) distribution as shown here and discussed here
\n\nAccording to this lecture by Andrew Ng, one way to solve train-test distribution difference would be to add portion of test set(which come from different distribution than train) to the train set and then training the model.
In our case, we can do this by PseudoLabeling. \n
Train ur model using train(+external data) and then make predictions on (public) test set; take the most confident predictions and add them to the original training set and train again. You can repeat this process for K rounds where K depends on u.
My way of measuring confidence would be to take two(or more) individual models(regression or classification) and make predictions on the public test set. Add the samples which have same predictions from both these models.
I'm pretty sure there are other smart ways to do pseudolabeling. please share ur way :)
The difference in LB and CV has already been mentioned many times like here and here. This could be due to difference in train and test(public+private) distribution as shown here and discussed here
\n\nAccording to this lecture by Andrew Ng, one way to solve train-test distribution difference would be to add portion of test set(which come from different distribution than train) to the train set and then training the model.
In our case, we can do this by PseudoLabeling. \n
Train ur model using train(+external data) and then make predictions on (public) test set; take the most confident predictions and add them to the original training set and train again. You can repeat this process for K rounds where K depends on u.
My way of measuring confidence would be to take two(or more) individual models(regression or classification) and make predictions on the public test set. Add the samples which have same predictions from both these models.
I'm pretty sure there are other smart ways to do pseudolabeling. please share ur way :)
[Reference] Pic for this post was taken from this paper
", "rawMarkdown": "[Reference] Pic for this post was taken from this [paper](https://arxiv.org/pdf/1904.04445.pdf)" }, { "id": 621666, "postDate": "2019-09-08T20:16:14.917Z", "rawMarkdown": "", "isDeleted": true }, { "id": 610648, "postDate": "2019-08-28T23:42:16.230Z", "content": "thanks @bibek777
", "rawMarkdown": "thanks @bibek777 \n" } ], "comments": [ { "id": 610295, "author_name": "Bibek", "author_url": "", "post_date": "2019-08-28T16:16:49.030000", "content": "[Reference] Pic for this post was taken from this paper
", "votes": 0, "replies": [] }, { "id": 621666, "author_name": "", "author_url": "", "post_date": "2019-09-08T20:16:14.917000", "content": "", "votes": 0, "replies": [] }, { "id": 610648, "author_name": "Mobassir", "author_url": "", "post_date": "2019-08-28T23:42:16.230000", "content": "thanks @bibek777
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "610294": "The difference in LB and CV has already been mentioned many times like [here](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#latest-607462) and [here](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815#latest-60). This could be due to difference in train and test(public+private) distribution as shown [here](https://www.kaggle.com/currypurin/image-shape-distribution-previous-and-present) and discussed [here](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/104692#latest-603729)\n\nAccording to this [lecture](https://www.youtube.com/watch?v=sfk5h0yC67o) by Andrew Ng, one way to solve `train-test distribution difference` would be to add portion of `test set`(which come from different distribution than train) to the `train set` and then training the model. \n\nIn our case, we can do this by `PseudoLabeling`. \n\n\n\nTrain ur model using train(+external data) and then make predictions on (public) test set; take the most confident predictions and add them to the original training set and train again. You can repeat this process for `K` rounds where `K` depends on u.\n\n## But how to know confident predictions? \nMy way of measuring confidence would be to take two(or more) individual models(regression or classification) and make predictions on the public test set. `Add the samples which have same predictions from both these models.`\n\nI'm pretty sure there are other smart ways to do `pseudolabeling`. please share ur way :) ", "610295": "[Reference] Pic for this post was taken from this [paper](https://arxiv.org/pdf/1904.04445.pdf)", "621666": "", "610648": "thanks @bibek777 \n" } }