I am thinking about trying MixMatch or related methods (UDA) with the noisy dataset but wanted to know whether other people have gotten promising results. In addition, it seems using the full noisy dataset will take up too much memory according to @daisukelab's kernel so how have people been able to overcome that?
", "messages": [ { "id": "537025", "postDate": "05/26/2019 02:57:54", "content": "I am thinking about trying MixMatch or related methods (UDA) with the noisy dataset but wanted to know whether other people have gotten promising results. In addition, it seems using the full noisy dataset will take up too much memory according to @daisukelab's kernel so how have people been able to overcome that?
", "rawMarkdown": "I am thinking about trying MixMatch or related methods (UDA) with the noisy dataset but wanted to know whether other people have gotten promising results. In addition, it seems using the full noisy dataset will take up too much memory according to @daisukelab's kernel so how have people been able to overcome that?", "votes": null }, { "id": "537137", "postDate": "05/26/2019 09:14:23", "content": "For me, mixmatch has not shown its power. For the full noisy dataset, I think you can train the model locally and upload the trained model for the final kernel.
", "rawMarkdown": "For me, mixmatch has not shown its power. For the full noisy dataset, I think you can train the model locally and upload the trained model for the final kernel.", "votes": null }, { "id": "537139", "postDate": "05/26/2019 09:20:03", "content": "What other approaches have you used for noisy labels? (if you are willing to share)
", "rawMarkdown": "What other approaches have you used for noisy labels? (if you are willing to share)", "votes": null }, { "id": "537295", "postDate": "05/26/2019 16:33:14", "content": "I tried various approaches with little or no success. It seems that the main problem is not the label noise, but rather a different data distribution of the noisy subset.
\n\nA model trained only on noisy data with a local CV score of almost 0.8 (computed on the noisy subset) gives only 0.22 on LB and only about 0.25 on the curated train data. So, the label quality of the noisy subset is not that bad, but the distribution shift is what makes this data very difficult to use.
", "rawMarkdown": "I tried various approaches with little or no success. It seems that the main problem is not the label noise, but rather a different data distribution of the noisy subset.\n\nA model trained only on noisy data with a local CV score of almost 0.8 (computed on the noisy subset) gives only 0.22 on LB and only about 0.25 on the curated train data. So, the label quality of the noisy subset is not that bad, but the distribution shift is what makes this data very difficult to use.", "votes": null }, { "id": "537371", "postDate": "05/26/2019 21:15:57", "content": "Ah i see... How about mixing the curated and noisy label data and training on that? If I understand correctly, MixMatch augments the unlabeled data and shuffles it with labeled data and trains on that.
", "rawMarkdown": "Ah i see... How about mixing the curated and noisy label data and training on that? If I understand correctly, MixMatch augments the unlabeled data and shuffles it with labeled data and trains on that.", "votes": null } ], "comments": [ { "id": 537137, "author_name": "sailorwei", "author_url": "", "post_date": "05/26/2019 09:14:23", "content": "For me, mixmatch has not shown its power. For the full noisy dataset, I think you can train the model locally and upload the trained model for the final kernel.
", "votes": null, "replies": [ { "id": 537139, "author_name": "tanlikesmath", "author_url": "", "post_date": "05/26/2019 09:20:03", "content": "What other approaches have you used for noisy labels? (if you are willing to share)
", "votes": null, "replies": [] }, { "id": 537295, "author_name": "ddanevskyi", "author_url": "", "post_date": "05/26/2019 16:33:14", "content": "I tried various approaches with little or no success. It seems that the main problem is not the label noise, but rather a different data distribution of the noisy subset.
\n\nA model trained only on noisy data with a local CV score of almost 0.8 (computed on the noisy subset) gives only 0.22 on LB and only about 0.25 on the curated train data. So, the label quality of the noisy subset is not that bad, but the distribution shift is what makes this data very difficult to use.
", "votes": null, "replies": [] }, { "id": 537371, "author_name": "tanlikesmath", "author_url": "", "post_date": "05/26/2019 21:15:57", "content": "Ah i see... How about mixing the curated and noisy label data and training on that? If I understand correctly, MixMatch augments the unlabeled data and shuffles it with labeled data and trains on that.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "537025": "I am thinking about trying MixMatch or related methods (UDA) with the noisy dataset but wanted to know whether other people have gotten promising results. In addition, it seems using the full noisy dataset will take up too much memory according to @daisukelab's kernel so how have people been able to overcome that?", "537137": "For me, mixmatch has not shown its power. For the full noisy dataset, I think you can train the model locally and upload the trained model for the final kernel.", "537139": "What other approaches have you used for noisy labels? (if you are willing to share)", "537295": "I tried various approaches with little or no success. It seems that the main problem is not the label noise, but rather a different data distribution of the noisy subset.\n\nA model trained only on noisy data with a local CV score of almost 0.8 (computed on the noisy subset) gives only 0.22 on LB and only about 0.25 on the curated train data. So, the label quality of the noisy subset is not that bad, but the distribution shift is what makes this data very difficult to use.", "537371": "Ah i see... How about mixing the curated and noisy label data and training on that? If I understand correctly, MixMatch augments the unlabeled data and shuffles it with labeled data and trains on that." }, "source": "meta" }