I got private stage 2 LB .180 for my retrained model and .203 for my stage 1 model without retraining. Did other people experience similar results? What can we learn from this?
", "messages": [ { "id": "413858", "postDate": "11/01/2018 16:49:44", "content": "I got private stage 2 LB .180 for my retrained model and .203 for my stage 1 model without retraining. Did other people experience similar results? What can we learn from this?
", "rawMarkdown": "I got private stage 2 LB .180 for my retrained model and .203 for my stage 1 model without retraining. Did other people experience similar results? What can we learn from this?", "votes": null }, { "id": "413865", "postDate": "11/01/2018 17:03:07", "content": "We scored 0.200 with retraining our model with stage 2 data.
\n\nAnd 0.208 on stage 2 with our previous stage1 trained model.
\n\nUnfortunately for us the retrained model performed better on the 1% sample and we ended up going with it. That costed us a silver medal.
", "rawMarkdown": "We scored 0.200 with retraining our model with stage 2 data. \n\nAnd 0.208 on stage 2 with our previous stage1 trained model. \n\nUnfortunately for us the retrained model performed better on the 1% sample and we ended up going with it. That costed us a silver medal.", "votes": null }, { "id": "413893", "postDate": "11/01/2018 18:15:55", "content": "There was not enough data in the state 2 public LB. It was just 30 images. And the addition of 1000 of stage 1 represented less than 4% of data increase. It means that retraining would be completly blind, since we couldn't validate the new model with only 30 images. So, it was high risk low reward. For that reason, my team choosen to not take the risk.
", "rawMarkdown": "There was not enough data in the state 2 public LB. It was just 30 images. And the addition of 1000 of stage 1 represented less than 4% of data increase. It means that retraining would be completly blind, since we couldn't validate the new model with only 30 images. So, it was high risk low reward. For that reason, my team choosen to not take the risk.", "votes": null }, { "id": "413904", "postDate": "11/01/2018 18:39:28", "content": "Hi James, According to this discussion https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/69745 medal rankings will be based on stage 1 participants... so with 1400+ (I believe...) in stage 1 that would mean Silver for Top 5%. That is the first 70 something if I understood correctly....so 52nd place ==> Silver.
\n\nThe leaderbord does not yet reflect those statements.
", "rawMarkdown": "Hi James, According to this discussion https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/69745 medal rankings will be based on stage 1 participants... so with 1400+ (I believe...) in stage 1 that would mean Silver for Top 5%. That is the first 70 something if I understood correctly....so 52nd place ==> Silver.\n\nThe leaderbord does not yet reflect those statements.", "votes": null }, { "id": "413923", "postDate": "11/01/2018 19:34:30", "content": ": )
", "rawMarkdown": ": )", "votes": null }, { "id": "414007", "postDate": "11/02/2018 00:19:02", "content": "I observed the same hence posted my thoughts to caution people in https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/69998 and followed up with the stage-2 public LB post https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70216
", "rawMarkdown": "I observed the same hence posted my thoughts to caution people in https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/69998 and followed up with the stage-2 public LB post https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70216", "votes": null } ], "comments": [ { "id": 413865, "author_name": "jryanmason", "author_url": "", "post_date": "11/01/2018 17:03:07", "content": "We scored 0.200 with retraining our model with stage 2 data.
\n\nAnd 0.208 on stage 2 with our previous stage1 trained model.
\n\nUnfortunately for us the retrained model performed better on the 1% sample and we ended up going with it. That costed us a silver medal.
", "votes": null, "replies": [ { "id": 413904, "author_name": "rsmits", "author_url": "", "post_date": "11/01/2018 18:39:28", "content": "Hi James, According to this discussion https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/69745 medal rankings will be based on stage 1 participants... so with 1400+ (I believe...) in stage 1 that would mean Silver for Top 5%. That is the first 70 something if I understood correctly....so 52nd place ==> Silver.
\n\nThe leaderbord does not yet reflect those statements.
", "votes": null, "replies": [] }, { "id": 413923, "author_name": "jryanmason", "author_url": "", "post_date": "11/01/2018 19:34:30", "content": ": )
", "votes": null, "replies": [] } ] }, { "id": 413893, "author_name": "fernandocamargo", "author_url": "", "post_date": "11/01/2018 18:15:55", "content": "There was not enough data in the state 2 public LB. It was just 30 images. And the addition of 1000 of stage 1 represented less than 4% of data increase. It means that retraining would be completly blind, since we couldn't validate the new model with only 30 images. So, it was high risk low reward. For that reason, my team choosen to not take the risk.
", "votes": null, "replies": [] }, { "id": 414007, "author_name": "sheriytm", "author_url": "", "post_date": "11/02/2018 00:19:02", "content": "I observed the same hence posted my thoughts to caution people in https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/69998 and followed up with the stage-2 public LB post https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70216
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "413858": "I got private stage 2 LB .180 for my retrained model and .203 for my stage 1 model without retraining. Did other people experience similar results? What can we learn from this?", "413865": "We scored 0.200 with retraining our model with stage 2 data. \n\nAnd 0.208 on stage 2 with our previous stage1 trained model. \n\nUnfortunately for us the retrained model performed better on the 1% sample and we ended up going with it. That costed us a silver medal.", "413893": "There was not enough data in the state 2 public LB. It was just 30 images. And the addition of 1000 of stage 1 represented less than 4% of data increase. It means that retraining would be completly blind, since we couldn't validate the new model with only 30 images. So, it was high risk low reward. For that reason, my team choosen to not take the risk.", "413904": "Hi James, According to this discussion https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/69745 medal rankings will be based on stage 1 participants... so with 1400+ (I believe...) in stage 1 that would mean Silver for Top 5%. That is the first 70 something if I understood correctly....so 52nd place ==> Silver.\n\nThe leaderbord does not yet reflect those statements.", "413923": ": )", "414007": "I observed the same hence posted my thoughts to caution people in https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/69998 and followed up with the stage-2 public LB post https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/70216" }, "source": "meta" }