really wondering. any educated guesses ;) ?
", "messages": [ { "id": "892493", "postDate": "06/18/2020 23:35:01", "content": "really wondering. any educated guesses ;) ?
", "rawMarkdown": "really wondering. any educated guesses ;) ?", "votes": null }, { "id": "892547", "postDate": "06/19/2020 02:04:30", "content": "Saving it for later maybe?
", "rawMarkdown": "Saving it for later maybe?", "votes": null }, { "id": "892569", "postDate": "06/19/2020 02:33:31", "content": "because we dont have unlabelled data?
", "rawMarkdown": "because we dont have unlabelled data?", "votes": null }, { "id": "892578", "postDate": "06/19/2020 02:48:37", "content": "I think the idea of pseudo labeling is to augment the train set with a subset of the test data + predicted labels. The subset only includes instances on which our learning algorithm makes predictions with a high level of confidence. So, our unlabeled data come from the test set.
", "rawMarkdown": "I think the idea of pseudo labeling is to augment the train set with a subset of the test data + predicted labels. The subset only includes instances on which our learning algorithm makes predictions with a high level of confidence. So, our unlabeled data come from the test set.", "votes": null }, { "id": "892588", "postDate": "06/19/2020 03:03:14", "content": "It's too early. We need good enough models to generate pseudo labels.
", "rawMarkdown": "It's too early. We need good enough models to generate pseudo labels.", "votes": null }, { "id": "892738", "postDate": "06/19/2020 05:58:54", "content": "😂
", "rawMarkdown": "😂", "votes": null }, { "id": "892976", "postDate": "06/19/2020 09:44:01", "content": "@zzy990106 @graf10a \n probably yes - but as far as I know (not much) its best to take only the very likely examples and add them to the training data - so there should not be much difference between the 0.94 models now and the 0.96 models later - hmm
", "rawMarkdown": "zzy990106 @graf10a \n probably yes - but as far as I know (not much) its best to take only the very likely examples and add them to the training data - so there should not be much difference between the 0.94 models now and the 0.96 models later - hmm", "votes": null }, { "id": "893016", "postDate": "06/19/2020 10:21:50", "content": "Since there is such a huge amount of external labeled data available, it seems risky to be using uncertain labels inferred from the test set IMO. Nevertheless, the images taken from the test set definitely are more similar to the test set than external images :)...
", "rawMarkdown": "Since there is such a huge amount of external labeled data available, it seems risky to be using uncertain labels inferred from the test set IMO. Nevertheless, the images taken from the test set definitely are more similar to the test set than external images :)...", "votes": null }, { "id": "894266", "postDate": "06/20/2020 09:27:12", "content": "I tried but not help. May be need to do for more models
", "rawMarkdown": "I tried but not help. May be need to do for more models", "votes": null }, { "id": "896941", "postDate": "06/22/2020 14:32:08", "content": "Usually it is a bit early to start using pseudo-labelling. You can use pseudo-labelling when we are sure your best-performing model is not overfitting while performing quite high on LB (my honest guess is around 0.95). Also if you want to do pseudo-labelling anyway, make sure to use very confident predictions usually for positive targets AUC > 0.99 and for negative ones AUC < 0.02, otherwise you will introduce a strong confirmation bias.
", "rawMarkdown": "Usually it is a bit early to start using pseudo-labelling. You can use pseudo-labelling when we are sure your best-performing model is not overfitting while performing quite high on LB (my honest guess is around 0.95). Also if you want to do pseudo-labelling anyway, make sure to use very confident predictions usually for positive targets AUC > 0.99 and for negative ones AUC < 0.02, otherwise you will introduce a strong confirmation bias.", "votes": null }, { "id": "900125", "postDate": "06/24/2020 16:42:10", "content": "Where are you planning to gather those unlabeled images from, Test data ?
", "rawMarkdown": "Where are you planning to gather those unlabeled images from, Test data ?", "votes": null }, { "id": "907358", "postDate": "06/29/2020 22:17:12", "content": "Yeah I tried that, there is not much improvement.
", "rawMarkdown": "Yeah I tried that, there is not much improvement.", "votes": null }, { "id": "907395", "postDate": "06/29/2020 23:54:11", "content": "can you specify the amount of improvement and can you tell the score of the model you used to predict the target values?
", "rawMarkdown": "can you specify the amount of improvement and can you tell the score of the model you used to predict the target values?", "votes": null } ], "comments": [ { "id": 892547, "author_name": "graf10a", "author_url": "", "post_date": "06/19/2020 02:04:30", "content": "Saving it for later maybe?
", "votes": null, "replies": [] }, { "id": 892569, "author_name": "moewie94", "author_url": "", "post_date": "06/19/2020 02:33:31", "content": "because we dont have unlabelled data?
", "votes": null, "replies": [ { "id": 892578, "author_name": "graf10a", "author_url": "", "post_date": "06/19/2020 02:48:37", "content": "I think the idea of pseudo labeling is to augment the train set with a subset of the test data + predicted labels. The subset only includes instances on which our learning algorithm makes predictions with a high level of confidence. So, our unlabeled data come from the test set.
", "votes": null, "replies": [] }, { "id": 892738, "author_name": "changewow", "author_url": "", "post_date": "06/19/2020 05:58:54", "content": "😂
", "votes": null, "replies": [] } ] }, { "id": 892588, "author_name": "zzy990106", "author_url": "", "post_date": "06/19/2020 03:03:14", "content": "It's too early. We need good enough models to generate pseudo labels.
", "votes": null, "replies": [ { "id": 892976, "author_name": "romanweilguny", "author_url": "", "post_date": "06/19/2020 09:44:01", "content": "@zzy990106 @graf10a \n probably yes - but as far as I know (not much) its best to take only the very likely examples and add them to the training data - so there should not be much difference between the 0.94 models now and the 0.96 models later - hmm
", "votes": null, "replies": [] } ] }, { "id": 893016, "author_name": "group16", "author_url": "", "post_date": "06/19/2020 10:21:50", "content": "Since there is such a huge amount of external labeled data available, it seems risky to be using uncertain labels inferred from the test set IMO. Nevertheless, the images taken from the test set definitely are more similar to the test set than external images :)...
", "votes": null, "replies": [] }, { "id": 894266, "author_name": "dinhthilan", "author_url": "", "post_date": "06/20/2020 09:27:12", "content": "I tried but not help. May be need to do for more models
", "votes": null, "replies": [] }, { "id": 896941, "author_name": "rftexas", "author_url": "", "post_date": "06/22/2020 14:32:08", "content": "Usually it is a bit early to start using pseudo-labelling. You can use pseudo-labelling when we are sure your best-performing model is not overfitting while performing quite high on LB (my honest guess is around 0.95). Also if you want to do pseudo-labelling anyway, make sure to use very confident predictions usually for positive targets AUC > 0.99 and for negative ones AUC < 0.02, otherwise you will introduce a strong confirmation bias.
", "votes": null, "replies": [] }, { "id": 900125, "author_name": "niteshx2", "author_url": "", "post_date": "06/24/2020 16:42:10", "content": "Where are you planning to gather those unlabeled images from, Test data ?
", "votes": null, "replies": [] }, { "id": 907358, "author_name": "abhishek60", "author_url": "", "post_date": "06/29/2020 22:17:12", "content": "Yeah I tried that, there is not much improvement.
", "votes": null, "replies": [ { "id": 907395, "author_name": "romanweilguny", "author_url": "", "post_date": "06/29/2020 23:54:11", "content": "can you specify the amount of improvement and can you tell the score of the model you used to predict the target values?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "892493": "really wondering. any educated guesses ;) ?", "892547": "Saving it for later maybe?", "892569": "because we dont have unlabelled data?", "892578": "I think the idea of pseudo labeling is to augment the train set with a subset of the test data + predicted labels. The subset only includes instances on which our learning algorithm makes predictions with a high level of confidence. So, our unlabeled data come from the test set.", "892588": "It's too early. We need good enough models to generate pseudo labels.", "892738": "😂", "892976": "zzy990106 @graf10a \n probably yes - but as far as I know (not much) its best to take only the very likely examples and add them to the training data - so there should not be much difference between the 0.94 models now and the 0.96 models later - hmm", "893016": "Since there is such a huge amount of external labeled data available, it seems risky to be using uncertain labels inferred from the test set IMO. Nevertheless, the images taken from the test set definitely are more similar to the test set than external images :)...", "894266": "I tried but not help. May be need to do for more models", "896941": "Usually it is a bit early to start using pseudo-labelling. You can use pseudo-labelling when we are sure your best-performing model is not overfitting while performing quite high on LB (my honest guess is around 0.95). Also if you want to do pseudo-labelling anyway, make sure to use very confident predictions usually for positive targets AUC > 0.99 and for negative ones AUC < 0.02, otherwise you will introduce a strong confirmation bias.", "900125": "Where are you planning to gather those unlabeled images from, Test data ?", "907358": "Yeah I tried that, there is not much improvement.", "907395": "can you specify the amount of improvement and can you tell the score of the model you used to predict the target values?" }, "source": "meta" }