In this competition, we tried pseudo label and it increased our public score from 0.670 to 0.673. The private score was also improved accordingly.
", "messages": [ { "id": 676195, "postDate": "2019-11-19T02:34:35.270Z", "content": "In this competition, we tried pseudo label and it increased our public score from 0.670 to 0.673. The private score was also improved accordingly.
", "rawMarkdown": "In this competition, we tried pseudo label and it increased our public score from 0.670 to 0.673. The private score was also improved accordingly.", "votes": 4 }, { "id": 677038, "postDate": "2019-11-19T18:15:33.937Z", "content": "It worked well for Kha increasing his CV and LB by 0.003 described here
", "rawMarkdown": "It worked well for Kha increasing his CV and LB by 0.003 described [here][1]\n\n[1]: https://www.kaggle.com/c/understanding_cloud_organization/discussion/118065", "votes": 1 }, { "id": 676440, "postDate": "2019-11-19T07:23:38.027Z", "content": "Definitely, it's helped me get additional 0.01 on my CV (which one correlate with LB).
", "rawMarkdown": "Definitely, it's helped me get additional 0.01 on my CV (which one correlate with LB). ", "votes": 1 }, { "id": 676429, "postDate": "2019-11-19T07:08:11.537Z", "content": "I failed. I scrapped 28730 images using modified code of https://www.kaggle.com/c/understanding_cloud_organization/discussion/116892#671521 , pseudo labeled them, concatenated them with test data, selected ~13000 most confident samples, added them to train set, and trained with more noise like in https://arxiv.org/abs/1911.04252 with bce loss. It improved my classifier and Unet cv to about 0.008. However, cv after postprocessing was a bit lower than before I introduced pseudo label.\nThere might have been a bug in my code, or something with postprocessing that screwed up. Leaderboard scores were higher when I didn't use pseudo label.
", "rawMarkdown": "I failed. I scrapped 28730 images using modified code of https://www.kaggle.com/c/understanding_cloud_organization/discussion/116892#671521 , pseudo labeled them, concatenated them with test data, selected ~13000 most confident samples, added them to train set, and trained with more noise like in https://arxiv.org/abs/1911.04252 with bce loss. It improved my classifier and Unet cv to about 0.008. However, cv after postprocessing was a bit lower than before I introduced pseudo label.\nThere might have been a bug in my code, or something with postprocessing that screwed up. Leaderboard scores were higher when I didn't use pseudo label.", "votes": 1, "replies": [ { "id": 676442, "postDate": "2019-11-19T07:29:46.813Z", "content": "@gogo827jz, @valyukov, did you use soft labels? If yes, what loss function did you use when training with soft labels? bce-dice loss didn't work for me.\nEdit: soft labels for pseudo labeling
", "rawMarkdown": "@gogo827jz, @valyukov, did you use soft labels? If yes, what loss function did you use when training with soft labels? bce-dice loss didn't work for me.\n*Edit: soft labels for pseudo labeling*" }, { "id": 676469, "postDate": "2019-11-19T07:59:52.580Z", "content": "what I've done:\n1. select best min size for each threshold \n2. use different threshold for latest stage of training from 0.8 to 0.95\n3. build pseudo labels on single fold ensemble models that's get me best local CV score\n4. find best proportion of labeled/unlabelled data
", "rawMarkdown": "what I've done:\n1. select best min size for each threshold \n2. use different threshold for latest stage of training from 0.8 to 0.95\n3. build pseudo labels on single fold ensemble models that's get me best local CV score\n4. find best proportion of labeled/unlabelled data" }, { "id": 676614, "postDate": "2019-11-19T10:55:16.413Z", "content": "@YoonSoo
\n\nI suggest you can create a new kaggle dataset and upload your download cloud images.
", "rawMarkdown": "@YoonSoo\n\nI suggest you can create a new kaggle dataset and upload your download cloud images.", "votes": 1 }, { "id": 676757, "postDate": "2019-11-19T13:36:51.297Z", "content": "No, I didn't use soft labels for pseudo labelling. I pretrained the model on all pseudo labels (best submission) and then trained with KFold CV on training data.
", "rawMarkdown": "No, I didn't use soft labels for pseudo labelling. I pretrained the model on all pseudo labels (best submission) and then trained with KFold CV on training data.", "votes": 1 } ] }, { "id": 676409, "postDate": "2019-11-19T06:31:16.177Z", "content": "I failed using pseudo label. It even doesn't work for public score.
", "rawMarkdown": "I failed using pseudo label. It even doesn't work for public score.", "votes": 1 }, { "id": 676479, "postDate": "2019-11-19T08:12:47.923Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ], "comments": [ { "id": 677038, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-11-19T18:15:33.937000", "content": "It worked well for Kha increasing his CV and LB by 0.003 described here
", "votes": 1, "replies": [] }, { "id": 676440, "author_name": "Vlad A", "author_url": "", "post_date": "2019-11-19T07:23:38.027000", "content": "Definitely, it's helped me get additional 0.01 on my CV (which one correlate with LB).
", "votes": 1, "replies": [] }, { "id": 676429, "author_name": "YoonSoo", "author_url": "", "post_date": "2019-11-19T07:08:11.537000", "content": "I failed. I scrapped 28730 images using modified code of https://www.kaggle.com/c/understanding_cloud_organization/discussion/116892#671521 , pseudo labeled them, concatenated them with test data, selected ~13000 most confident samples, added them to train set, and trained with more noise like in https://arxiv.org/abs/1911.04252 with bce loss. It improved my classifier and Unet cv to about 0.008. However, cv after postprocessing was a bit lower than before I introduced pseudo label.\nThere might have been a bug in my code, or something with postprocessing that screwed up. Leaderboard scores were higher when I didn't use pseudo label.
", "votes": 1, "replies": [ { "id": 676442, "author_name": "YoonSoo", "author_url": "", "post_date": "2019-11-19T07:29:46.813000", "content": "@gogo827jz, @valyukov, did you use soft labels? If yes, what loss function did you use when training with soft labels? bce-dice loss didn't work for me.\nEdit: soft labels for pseudo labeling
", "votes": 0, "replies": [] }, { "id": 676469, "author_name": "Vlad A", "author_url": "", "post_date": "2019-11-19T07:59:52.580000", "content": "what I've done:\n1. select best min size for each threshold \n2. use different threshold for latest stage of training from 0.8 to 0.95\n3. build pseudo labels on single fold ensemble models that's get me best local CV score\n4. find best proportion of labeled/unlabelled data
", "votes": 0, "replies": [] }, { "id": 676614, "author_name": "hengck23", "author_url": "", "post_date": "2019-11-19T10:55:16.413000", "content": "@YoonSoo
\n\nI suggest you can create a new kaggle dataset and upload your download cloud images.
", "votes": 1, "replies": [] }, { "id": 676757, "author_name": "Yirun Zhang", "author_url": "", "post_date": "2019-11-19T13:36:51.297000", "content": "No, I didn't use soft labels for pseudo labelling. I pretrained the model on all pseudo labels (best submission) and then trained with KFold CV on training data.
", "votes": 1, "replies": [] } ] }, { "id": 676409, "author_name": "llh1818", "author_url": "", "post_date": "2019-11-19T06:31:16.177000", "content": "I failed using pseudo label. It even doesn't work for public score.
", "votes": 1, "replies": [] }, { "id": 676479, "author_name": "", "author_url": "", "post_date": "2019-11-19T08:12:47.923000", "content": "", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "676195": "In this competition, we tried pseudo label and it increased our public score from 0.670 to 0.673. The private score was also improved accordingly.", "677038": "It worked well for Kha increasing his CV and LB by 0.003 described [here][1]\n\n[1]: https://www.kaggle.com/c/understanding_cloud_organization/discussion/118065", "676440": "Definitely, it's helped me get additional 0.01 on my CV (which one correlate with LB). ", "676429": "I failed. I scrapped 28730 images using modified code of https://www.kaggle.com/c/understanding_cloud_organization/discussion/116892#671521 , pseudo labeled them, concatenated them with test data, selected ~13000 most confident samples, added them to train set, and trained with more noise like in https://arxiv.org/abs/1911.04252 with bce loss. It improved my classifier and Unet cv to about 0.008. However, cv after postprocessing was a bit lower than before I introduced pseudo label.\nThere might have been a bug in my code, or something with postprocessing that screwed up. Leaderboard scores were higher when I didn't use pseudo label.", "676409": "I failed using pseudo label. It even doesn't work for public score.", "676479": "" } }