Hi guys, I'm new at this so this may be a dumb question:
\n\nI noticed only 6% of the training samples are labeled \"insincere\". So as a baseline, I made a predictions full of 0s. After I submitted it, I got 0 f1 score.
\n\nI know this is a stupid predictions, but I didn't expect such a low score. There must be at least some sincere questions inside the test set, right? What did I miss?
", "messages": [ { "id": 435514, "postDate": "2018-12-08T06:23:07.253Z", "content": "Hi guys, I'm new at this so this may be a dumb question:
\n\nI noticed only 6% of the training samples are labeled \"insincere\". So as a baseline, I made a predictions full of 0s. After I submitted it, I got 0 f1 score.
\n\nI know this is a stupid predictions, but I didn't expect such a low score. There must be at least some sincere questions inside the test set, right? What did I miss?
", "rawMarkdown": "Hi guys, I'm new at this so this may be a dumb question:\n\nI noticed only 6% of the training samples are labeled \"insincere\". So as a baseline, I made a predictions full of 0s. After I submitted it, I got 0 f1 score.\n\nI know this is a stupid predictions, but I didn't expect such a low score. There must be at least **some sincere questions** inside the test set, right? What did I miss?", "votes": 1 }, { "id": 435558, "postDate": "2018-12-08T08:44:00.790Z", "content": "The metric chosen for this competition was actually chosen to avoid the trick you want to use. F1 score depends on which label is chosen as the \"positive\" one, which is the most relevant one (here the \"1\" label). If you have zero true positives, you have 0 precision and recall.
\n\nhttps://en.wikipedia.org/wiki/Precision_and_recall
\n\nProbably you are choosing in scikit-learn a bad labelling.
", "rawMarkdown": "The metric chosen for this competition was actually chosen to avoid the trick you want to use. F1 score depends on which label is chosen as the \"positive\" one, which is the most relevant one (here the \"1\" label). If you have zero true positives, you have 0 precision and recall.\n\nhttps://en.wikipedia.org/wiki/Precision_and_recall\n\nProbably you are choosing in scikit-learn a bad labelling.", "replies": [ { "id": 435909, "postDate": "2018-12-09T03:40:55.243Z", "content": "Thank you for your reply! Now I see it, you are absolutely right!
", "rawMarkdown": "Thank you for your reply! Now I see it, you are absolutely right! " } ] }, { "id": 435517, "postDate": "2018-12-08T06:27:22.373Z", "content": "Before I submitted the all 0s prediction, I split 15% out of the training data as validation set. And the all 0s prediction got me a 0.9 f1 score by f1_score method from sklearn. So I must made a stupid mistake somewhere...
The metric chosen for this competition was actually chosen to avoid the trick you want to use. F1 score depends on which label is chosen as the \"positive\" one, which is the most relevant one (here the \"1\" label). If you have zero true positives, you have 0 precision and recall.
\n\nhttps://en.wikipedia.org/wiki/Precision_and_recall
\n\nProbably you are choosing in scikit-learn a bad labelling.
", "votes": 0, "replies": [ { "id": 435909, "author_name": "Junlin Shang", "author_url": "", "post_date": "2018-12-09T03:40:55.243000", "content": "Thank you for your reply! Now I see it, you are absolutely right!
", "votes": 0, "replies": [] } ] }, { "id": 435517, "author_name": "Junlin Shang", "author_url": "", "post_date": "2018-12-08T06:27:22.373000", "content": "Before I submitted the all 0s prediction, I split 15% out of the training data as validation set. And the all 0s prediction got me a 0.9 f1 score by f1_score method from sklearn. So I must made a stupid mistake somewhere...