whatever to choose the threshold, the predict result will have some null result, which mean it don't belong any class from prediction, so far I set them to \"0\" class, but I think it is not best practice , how about your solution in such case?
", "messages": [ { "id": "433693", "postDate": "12/05/2018 11:00:48", "content": "whatever to choose the threshold, the predict result will have some null result, which mean it don't belong any class from prediction, so far I set them to \"0\" class, but I think it is not best practice , how about your solution in such case?
", "rawMarkdown": "whatever to choose the threshold, the predict result will have some null result, which mean it don't belong any class from prediction, so far I set them to \"0\" class, but I think it is not best practice , how about your solution in such case?", "votes": null }, { "id": "434160", "postDate": "12/06/2018 01:24:20", "content": "I lower the threshold by 50% and try again. If there are still no matches I set it to the class that scored highest.
", "rawMarkdown": "I lower the threshold by 50% and try again. If there are still no matches I set it to the class that scored highest.", "votes": null }, { "id": "434161", "postDate": "12/06/2018 01:28:55", "content": "I choose a class with the highest score
", "rawMarkdown": "I choose a class with the highest score", "votes": null }, { "id": "434173", "postDate": "12/06/2018 02:18:43", "content": "thanks , let me try the max value.
", "rawMarkdown": "thanks , let me try the max value.", "votes": null }, { "id": "434387", "postDate": "12/06/2018 10:00:30", "content": "There might be secret 28th,29th classes! ;\nEmma's paper:bottom of this page https://www.proteinatlas.org/news/2018-08-20/mapping-of-cells-and-proteins-improved-with-combination-of-multiplayer-crowdsourcing-and-ai\nFigure 1 d) 28:Negative 29:Unspecific
\n\nInteresting point of view...\nPersonally, I have about 200 null predictions in my best submission, but setting them to the most frequent class (0) made the LB score a worse (-0.001).\nI will try Brian suggestion too
", "rawMarkdown": "Interesting point of view...\nPersonally, I have about 200 null predictions in my best submission, but setting them to the most frequent class (0) made the LB score a worse (-0.001).\nI will try Brian suggestion too", "votes": null }, { "id": "434432", "postDate": "12/06/2018 11:41:02", "content": "Here's the subnails of my Nan predicted images,\nsome of them are clean, so my model fail to predict right class\nbut some of them have strange taste.
", "rawMarkdown": "Here's the subnails of my Nan predicted images,\nsome of them are clean, so my model fail to predict right class\nbut some of them have strange taste.", "votes": null }, { "id": "435459", "postDate": "12/08/2018 04:13:31", "content": "Interesting point.\nbut I still believe all testing case belong some of 28 class.
", "rawMarkdown": "Interesting point.\nbut I still believe all testing case belong some of 28 class.", "votes": null }, { "id": "435479", "postDate": "12/08/2018 04:44:25", "content": "Hi, @Qaing\nYou are right. \nI predicted train-set-samples and checked Nan-predicted, \nsome of them look strange, but still all of them belongs to some classes.\nI was wrong...sorry.
\n\nThere are 2 different task setting:
\n\n1) Classify robustly even if difficult-situation \n--> this competition expect us to do so, as you said.
\n\n2) Refrain from judging and alert human there's something with the sample!
\n--> In the production setting, it is more preferable, I think.
I read related paper,recently and confused the tasks.\n[Knows When it Doesn’t Know: Deep Abstaining Classifiers]\nhttps://openreview.net/forum?id=rJxF73R9tX
", "rawMarkdown": "Hi, @Qaing\nYou are right. \nI predicted train-set-samples and checked Nan-predicted, \nsome of them look strange, but still all of them belongs to some classes.\nI was wrong...sorry.\n\nThere are 2 different task setting:\n\n1) Classify robustly even if difficult-situation \n--> this competition expect us to do so, as you said.\n\n2) Refrain from judging and alert human there's something with the sample! \n--> In the production setting, it is more preferable, I think.\n\nI read related paper,recently and confused the tasks.\n[Knows When it Doesn’t Know: Deep Abstaining Classifiers]\nhttps://openreview.net/forum?id=rJxF73R9tX", "votes": null }, { "id": "436296", "postDate": "12/10/2018 04:46:59", "content": "TomomiMoriyama,
\n\nyou got the point.\nthe LB F1 score is NOT pure F1 score, it is with weight F1 score, it mean rare class prediction (like 27) will get a higher score than a common class(like 0), I learned the point from Leak discussion.
", "rawMarkdown": "TomomiMoriyama,\n\nyou got the point.\nthe LB F1 score is NOT pure F1 score, it is with weight F1 score, it mean rare class prediction (like 27) will get a higher score than a common class(like 0), I learned the point from Leak discussion.", "votes": null }, { "id": "436308", "postDate": "12/10/2018 05:30:08", "content": "I wonder if the number of null predictions correlates with the LB score. If not, then I begin to think about having > 28 classes.
", "rawMarkdown": "I wonder if the number of null predictions correlates with the LB score. If not, then I begin to think about having > 28 classes.", "votes": null } ], "comments": [ { "id": 434160, "author_name": "ldm314", "author_url": "", "post_date": "12/06/2018 01:24:20", "content": "I lower the threshold by 50% and try again. If there are still no matches I set it to the class that scored highest.
", "votes": null, "replies": [] }, { "id": 434161, "author_name": "maudung164", "author_url": "", "post_date": "12/06/2018 01:28:55", "content": "I choose a class with the highest score
", "votes": null, "replies": [] }, { "id": 434173, "author_name": "crilinux", "author_url": "", "post_date": "12/06/2018 02:18:43", "content": "thanks , let me try the max value.
", "votes": null, "replies": [] }, { "id": 434387, "author_name": "tomomimoriyama", "author_url": "", "post_date": "12/06/2018 10:00:30", "content": "There might be secret 28th,29th classes! ;\nEmma's paper:bottom of this page https://www.proteinatlas.org/news/2018-08-20/mapping-of-cells-and-proteins-improved-with-combination-of-multiplayer-crowdsourcing-and-ai\nFigure 1 d) 28:Negative 29:Unspecific
\n\nInteresting point of view...\nPersonally, I have about 200 null predictions in my best submission, but setting them to the most frequent class (0) made the LB score a worse (-0.001).\nI will try Brian suggestion too
", "votes": null, "replies": [] }, { "id": 435459, "author_name": "crilinux", "author_url": "", "post_date": "12/08/2018 04:13:31", "content": "Interesting point.\nbut I still believe all testing case belong some of 28 class.
", "votes": null, "replies": [] }, { "id": 435479, "author_name": "tomomimoriyama", "author_url": "", "post_date": "12/08/2018 04:44:25", "content": "Hi, @Qaing\nYou are right. \nI predicted train-set-samples and checked Nan-predicted, \nsome of them look strange, but still all of them belongs to some classes.\nI was wrong...sorry.
\n\nThere are 2 different task setting:
\n\n1) Classify robustly even if difficult-situation \n--> this competition expect us to do so, as you said.
\n\n2) Refrain from judging and alert human there's something with the sample!
\n--> In the production setting, it is more preferable, I think.
I read related paper,recently and confused the tasks.\n[Knows When it Doesn’t Know: Deep Abstaining Classifiers]\nhttps://openreview.net/forum?id=rJxF73R9tX
", "votes": null, "replies": [] }, { "id": 436296, "author_name": "crilinux", "author_url": "", "post_date": "12/10/2018 04:46:59", "content": "TomomiMoriyama,
\n\nyou got the point.\nthe LB F1 score is NOT pure F1 score, it is with weight F1 score, it mean rare class prediction (like 27) will get a higher score than a common class(like 0), I learned the point from Leak discussion.
", "votes": null, "replies": [] } ] }, { "id": 434432, "author_name": "tomomimoriyama", "author_url": "", "post_date": "12/06/2018 11:41:02", "content": "Here's the subnails of my Nan predicted images,\nsome of them are clean, so my model fail to predict right class\nbut some of them have strange taste.
", "votes": null, "replies": [] }, { "id": 436308, "author_name": "petewills", "author_url": "", "post_date": "12/10/2018 05:30:08", "content": "I wonder if the number of null predictions correlates with the LB score. If not, then I begin to think about having > 28 classes.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "433693": "whatever to choose the threshold, the predict result will have some null result, which mean it don't belong any class from prediction, so far I set them to \"0\" class, but I think it is not best practice , how about your solution in such case?", "434160": "I lower the threshold by 50% and try again. If there are still no matches I set it to the class that scored highest.", "434161": "I choose a class with the highest score", "434173": "thanks , let me try the max value.", "434387": "1. Why don't you trust your model and set it \"\"?\n\nThere might be secret 28th,29th classes! ;\nEmma's paper:bottom of this page https://www.proteinatlas.org/news/2018-08-20/mapping-of-cells-and-proteins-improved-with-combination-of-multiplayer-crowdsourcing-and-ai\nFigure 1 d) 28:Negative 29:Unspecific\n\n2. Have you looked into null-resulted-image? Are they have any common patterns?", "434414": "Interesting point of view...\nPersonally, I have about 200 null predictions in my best submission, but setting them to the most frequent class (0) made the LB score a worse (-0.001).\nI will try Brian suggestion too", "434432": "Here's the subnails of my Nan predicted images,\nsome of them are clean, so my model fail to predict right class\nbut some of them have strange taste.", "435459": "Interesting point.\nbut I still believe all testing case belong some of 28 class.", "435479": "Hi, @Qaing\nYou are right. \nI predicted train-set-samples and checked Nan-predicted, \nsome of them look strange, but still all of them belongs to some classes.\nI was wrong...sorry.\n\nThere are 2 different task setting:\n\n1) Classify robustly even if difficult-situation \n--> this competition expect us to do so, as you said.\n\n2) Refrain from judging and alert human there's something with the sample! \n--> In the production setting, it is more preferable, I think.\n\nI read related paper,recently and confused the tasks.\n[Knows When it Doesn’t Know: Deep Abstaining Classifiers]\nhttps://openreview.net/forum?id=rJxF73R9tX", "436296": "TomomiMoriyama,\n\nyou got the point.\nthe LB F1 score is NOT pure F1 score, it is with weight F1 score, it mean rare class prediction (like 27) will get a higher score than a common class(like 0), I learned the point from Leak discussion.", "436308": "I wonder if the number of null predictions correlates with the LB score. If not, then I begin to think about having > 28 classes." }, "source": "meta" }