I am currently focusing on analyzing the given datasets, and have found some mislabeled datapoints, for example row 38, Why do Christian-conservative Americans call abortion murder? which label as sincere. The idea I have now is to drop them; however, it is difficult to find them all. What technique/method do you apply for this problem?
\n\nMany thanks!
", "messages": [ { "id": "419559", "postDate": "11/12/2018 07:36:57", "content": "I am currently focusing on analyzing the given datasets, and have found some mislabeled datapoints, for example row 38, Why do Christian-conservative Americans call abortion murder? which label as sincere. The idea I have now is to drop them; however, it is difficult to find them all. What technique/method do you apply for this problem?
\n\nMany thanks!
", "rawMarkdown": "I am currently focusing on analyzing the given datasets, and have found some mislabeled datapoints, for example row 38, Why do Christian-conservative Americans call abortion murder? which label as sincere. The idea I have now is to drop them; however, it is difficult to find them all. What technique/method do you apply for this problem? \n\nMany thanks!", "votes": null }, { "id": "419576", "postDate": "11/12/2018 08:30:47", "content": "Well, to begin with, how are you sure that this (or any other sentence) is miss-labeled? I mean it can certainly be insincere as you suggest but it is highly subjective and there exists no ground-truth...
", "rawMarkdown": "Well, to begin with, how are you sure that this (or any other sentence) is miss-labeled? I mean it can certainly be insincere as you suggest but it is highly subjective and there exists no ground-truth...", "votes": null }, { "id": "420346", "postDate": "11/13/2018 13:34:25", "content": "This example is at least potentially sincere... since the same labelling strategy is applied to the train and test set, I would imagine there is little to be gained by dropping it
", "rawMarkdown": "This example is at least potentially sincere... since the same labelling strategy is applied to the train and test set, I would imagine there is little to be gained by dropping it", "votes": null }, { "id": "420748", "postDate": "11/14/2018 04:16:56", "content": "I think there a lot of mislabeled data as I have make a detailed analysis here:\nhttps://www.kaggle.com/ratthachat/explore-limits-error-analysis-of-srk-s-glove-gru
", "rawMarkdown": "I think there a lot of mislabeled data as I have make a detailed analysis here:\nhttps://www.kaggle.com/ratthachat/explore-limits-error-analysis-of-srk-s-glove-gru", "votes": null }, { "id": "420802", "postDate": "11/14/2018 06:31:17", "content": "Indeed, I observed a math question been assigned as insincere on my t-SNE plot. My current strategy for mislabeled data is using large batch size on nn model.
", "rawMarkdown": "Indeed, I observed a math question been assigned as insincere on my t-SNE plot. My current strategy for mislabeled data is using large batch size on nn model.", "votes": null }, { "id": "421012", "postDate": "11/14/2018 13:19:22", "content": "How large is it, could you give some idea?
\n\nReading your post above just see you mentioning about removing/correcting the mislabeled data. At first, I also agree. I think correcting them (if possible) will make the training easier.
\n\nHowever, just realize that there is also a downside as the test dataset will also have noise. So, if we correcting the validation data, it will not have the same distribution as the test data, and so it will be difficult to judge our performance before submission.
", "rawMarkdown": "How large is it, could you give some idea? \n\nReading your post above just see you mentioning about removing/correcting the mislabeled data. At first, I also agree. I think correcting them (if possible) will make the training easier. \n\nHowever, just realize that there is also a downside as the test dataset will also have noise. So, if we correcting the validation data, it will not have the same distribution as the test data, and so it will be difficult to judge our performance before submission.", "votes": null }, { "id": "422247", "postDate": "11/16/2018 00:46:14", "content": "I am using 256 to 1024 batch size.\nI also noticed that problem because the training set and the testing set both might gather in the same period.
\n\nI am currently working on finding those (potential) mislabeled data and see why they get mislabeled.
", "rawMarkdown": "I am using 256 to 1024 batch size.\nI also noticed that problem because the training set and the testing set both might gather in the same period. \n\nI am currently working on finding those (potential) mislabeled data and see why they get mislabeled.", "votes": null }, { "id": "422326", "postDate": "11/16/2018 04:22:11", "content": "There are many mislabeled data points. But Why do Christian-conservative Americans call abortion murder? is a perfectly legit question. Abortion is a highly controversial topic in the U.S and this could be a highlight questions on Quora for many.
Great! Looking forward to see that.
", "rawMarkdown": "Great! Looking forward to see that.", "votes": null } ], "comments": [ { "id": 419576, "author_name": "georsara1", "author_url": "", "post_date": "11/12/2018 08:30:47", "content": "Well, to begin with, how are you sure that this (or any other sentence) is miss-labeled? I mean it can certainly be insincere as you suggest but it is highly subjective and there exists no ground-truth...
", "votes": null, "replies": [] }, { "id": 420346, "author_name": "rdboyes", "author_url": "", "post_date": "11/13/2018 13:34:25", "content": "This example is at least potentially sincere... since the same labelling strategy is applied to the train and test set, I would imagine there is little to be gained by dropping it
", "votes": null, "replies": [] }, { "id": 420748, "author_name": "ratthachat", "author_url": "", "post_date": "11/14/2018 04:16:56", "content": "I think there a lot of mislabeled data as I have make a detailed analysis here:\nhttps://www.kaggle.com/ratthachat/explore-limits-error-analysis-of-srk-s-glove-gru
", "votes": null, "replies": [ { "id": 420802, "author_name": "konohayui", "author_url": "", "post_date": "11/14/2018 06:31:17", "content": "Indeed, I observed a math question been assigned as insincere on my t-SNE plot. My current strategy for mislabeled data is using large batch size on nn model.
", "votes": null, "replies": [] }, { "id": 421012, "author_name": "ratthachat", "author_url": "", "post_date": "11/14/2018 13:19:22", "content": "How large is it, could you give some idea?
\n\nReading your post above just see you mentioning about removing/correcting the mislabeled data. At first, I also agree. I think correcting them (if possible) will make the training easier.
\n\nHowever, just realize that there is also a downside as the test dataset will also have noise. So, if we correcting the validation data, it will not have the same distribution as the test data, and so it will be difficult to judge our performance before submission.
", "votes": null, "replies": [] }, { "id": 422247, "author_name": "konohayui", "author_url": "", "post_date": "11/16/2018 00:46:14", "content": "I am using 256 to 1024 batch size.\nI also noticed that problem because the training set and the testing set both might gather in the same period.
\n\nI am currently working on finding those (potential) mislabeled data and see why they get mislabeled.
", "votes": null, "replies": [] }, { "id": 422361, "author_name": "ratthachat", "author_url": "", "post_date": "11/16/2018 05:46:02", "content": "Great! Looking forward to see that.
", "votes": null, "replies": [] } ] }, { "id": 422326, "author_name": "sijunhe9248", "author_url": "", "post_date": "11/16/2018 04:22:11", "content": "There are many mislabeled data points. But Why do Christian-conservative Americans call abortion murder? is a perfectly legit question. Abortion is a highly controversial topic in the U.S and this could be a highlight questions on Quora for many.