It is clearly stated in the \"Rules\" tab about the model upload requirement:
\n\nMODEL UPLOAD REQUIREMENT (under A. SPECIFIC COMPETITION RULES)
\n\n> In order to be eligible for Stage 2, each team's Stage 1 submission must include the model uploaded, via Team -> Your Model. This model should match that which was used to generate the 1 final submission selected for scoring.
\n\nWhich is confirmed again in the Timeline subheading of Overview tab:\n> * In order to be eligible for Stage 2, each team's Stage 1 submission must include the model uploaded, via Team -> Your Model, per the Competition Rules. This model should match that which was used to generate the 1 final submission selected for scoring.
\n\nThere are many ways to improve model performance when you have ground truth label of stage 1 test set. Hence, it was fair enough to lock the model so that no participant might be in a situation of advantages beyond the competition rules. It is very unlikely that you will get same result for two different run due to random initialization of the network.
\n\nOnce again this is explained clearly \n> This requirement is in place for the host team to verify the performance of the uploaded models matches the Stage 2 submission file. Compliance with the above will be verified by the host team. Submitters who fail to upload their model by the Stage 1 deadline, or are found not to be in compliance, may be disqualified from Stage 2 and removed from the final leaderboard.
\n\nAnd finally, if you look at the part \"10. WINNERS OBLIGATIONS.\" of the official competition \"Rules\", it is said that
\n\n> As a condition to being awarded a Prize, a Prize winner must fulfill the following obligations:
\n\n> (a) deliver to the Competition Sponsor the final model's software code as used to generate the winning Submission and associated documentation. The delivered software code must be capable of generating the winning Submission and contain a description of resources required to build and/or run the executable code successfully;
\n\nSo, \"Model\" and \"model's software code\" are definitely defined as differently and we would not able to change anything after stage 1 except generating prediction for new test set of stage 2 using the stage 1 model (by which stage 1 test set was generated), at least for this competition.
\n\nHowever, all of a sudden we get STAGE 2 UPDATE \n> STAGE 2 UPDATE
\n\n> Note that new files are available to download! The training set now contains both the train and test set from stage 1. The test set is comprised of new, unseen images. The metric and file formats remain the same, but you'll now be making predictions using the updated train and test sets. We have an FAQ about two-stage competitions that provides some context for how this all works. Please give it a read.
\n\n> Good luck!
\n\nA lot of folks have really invested very hard and intensive efforts to solve a real-life problem. It will be really a frustrating situation for all of them for a sudden and unexpected change of competition rules which may make their strategy effected. This isn't the matter of winning the competition, this is the matter of advancing an effective and robust solution for the community.
\n\nHow other folks are thinking the situation?
", "messages": [ { "id": 411006, "postDate": "2018-10-27T06:40:57.737Z", "content": "It is clearly stated in the \"Rules\" tab about the model upload requirement:
\n\nMODEL UPLOAD REQUIREMENT (under A. SPECIFIC COMPETITION RULES)
\n\n> In order to be eligible for Stage 2, each team's Stage 1 submission must include the model uploaded, via Team -> Your Model. This model should match that which was used to generate the 1 final submission selected for scoring.
\n\nWhich is confirmed again in the Timeline subheading of Overview tab:\n> * In order to be eligible for Stage 2, each team's Stage 1 submission must include the model uploaded, via Team -> Your Model, per the Competition Rules. This model should match that which was used to generate the 1 final submission selected for scoring.
\n\nThere are many ways to improve model performance when you have ground truth label of stage 1 test set. Hence, it was fair enough to lock the model so that no participant might be in a situation of advantages beyond the competition rules. It is very unlikely that you will get same result for two different run due to random initialization of the network.
\n\nOnce again this is explained clearly \n> This requirement is in place for the host team to verify the performance of the uploaded models matches the Stage 2 submission file. Compliance with the above will be verified by the host team. Submitters who fail to upload their model by the Stage 1 deadline, or are found not to be in compliance, may be disqualified from Stage 2 and removed from the final leaderboard.
\n\nAnd finally, if you look at the part \"10. WINNERS OBLIGATIONS.\" of the official competition \"Rules\", it is said that
\n\n> As a condition to being awarded a Prize, a Prize winner must fulfill the following obligations:
\n\n> (a) deliver to the Competition Sponsor the final model's software code as used to generate the winning Submission and associated documentation. The delivered software code must be capable of generating the winning Submission and contain a description of resources required to build and/or run the executable code successfully;
\n\nSo, \"Model\" and \"model's software code\" are definitely defined as differently and we would not able to change anything after stage 1 except generating prediction for new test set of stage 2 using the stage 1 model (by which stage 1 test set was generated), at least for this competition.
\n\nHowever, all of a sudden we get STAGE 2 UPDATE \n> STAGE 2 UPDATE
\n\n> Note that new files are available to download! The training set now contains both the train and test set from stage 1. The test set is comprised of new, unseen images. The metric and file formats remain the same, but you'll now be making predictions using the updated train and test sets. We have an FAQ about two-stage competitions that provides some context for how this all works. Please give it a read.
\n\n> Good luck!
\n\nA lot of folks have really invested very hard and intensive efforts to solve a real-life problem. It will be really a frustrating situation for all of them for a sudden and unexpected change of competition rules which may make their strategy effected. This isn't the matter of winning the competition, this is the matter of advancing an effective and robust solution for the community.
\n\nHow other folks are thinking the situation?
", "rawMarkdown": "It is clearly stated in the \"[Rules][1]\" tab about the model upload requirement:\n\nMODEL UPLOAD REQUIREMENT (under A. SPECIFIC COMPETITION RULES)\n\n> In order to be eligible for Stage 2, each team's Stage 1 submission must include the model uploaded, via Team -> Your Model. **This model should match that which was used to generate the 1 final submission selected for scoring.**\n\nWhich is confirmed again in the [Timeline][2] subheading of Overview tab:\n> * In order to be eligible for Stage 2, each team's Stage 1 submission must include the model uploaded, via Team -> Your Model, per the Competition Rules. This model should match that which was used to generate the 1 final submission selected for scoring.\n\nThere are many ways to improve model performance when you have ground truth label of stage 1 test set. Hence, it was fair enough to lock the model so that no participant might be in a situation of advantages beyond the competition rules. It is very unlikely that you will get same result for two different run due to random initialization of the network.\n\nOnce again this is explained clearly \n> This requirement is in place for the host team to verify the performance of the uploaded models matches the Stage 2 submission file. Compliance with the above will be verified by the host team. Submitters who fail to upload their model by the Stage 1 deadline, or are found not to be in compliance, may be disqualified from Stage 2 and removed from the final leaderboard.\n\nAnd finally, if you look at the part \"10. WINNERS OBLIGATIONS.\" of the official competition \"[Rules][1]\", it is said that\n\n> As a condition to being awarded a Prize, a Prize winner must fulfill the following obligations:\n\n> (a) deliver to the Competition Sponsor the final **model's software code** as used to generate the winning Submission and associated documentation. The delivered software code must be capable of generating the winning Submission and contain a description of resources required to build and/or run the executable code successfully;\n\nSo, **\"Model\"** and **\"model's software code\"** are definitely defined as differently and we would not able to change anything after stage 1 except generating prediction for new test set of stage 2 using the stage 1 model (by which stage 1 test set was generated), at least for this competition.\n\nHowever, all of a sudden we get STAGE 2 UPDATE \n> STAGE 2 UPDATE\n\n> Note that new files are available to download! The training set now contains both the train and test set from stage 1. The test set is comprised of new, unseen images. The metric and file formats remain the same, but you'll now be making predictions using the updated train and test sets. We have an FAQ about two-stage competitions that provides some context for how this all works. Please give it a read.\n\n> Good luck!\n\nA lot of folks have really invested very hard and intensive efforts to solve a real-life problem. It will be really a frustrating situation for all of them for a sudden and unexpected change of competition rules which may make their strategy effected. This isn't the matter of winning the competition, this is the matter of advancing an effective and robust solution for the community.\n\n**How other folks are thinking the situation?**\n\n [1]: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/rules\n [2]: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge#Timelined\n\n", "votes": 4 } ], "comments": [], "raw_markdown_by_id": { "411006": "It is clearly stated in the \"[Rules][1]\" tab about the model upload requirement:\n\nMODEL UPLOAD REQUIREMENT (under A. SPECIFIC COMPETITION RULES)\n\n> In order to be eligible for Stage 2, each team's Stage 1 submission must include the model uploaded, via Team -> Your Model. **This model should match that which was used to generate the 1 final submission selected for scoring.**\n\nWhich is confirmed again in the [Timeline][2] subheading of Overview tab:\n> * In order to be eligible for Stage 2, each team's Stage 1 submission must include the model uploaded, via Team -> Your Model, per the Competition Rules. This model should match that which was used to generate the 1 final submission selected for scoring.\n\nThere are many ways to improve model performance when you have ground truth label of stage 1 test set. Hence, it was fair enough to lock the model so that no participant might be in a situation of advantages beyond the competition rules. It is very unlikely that you will get same result for two different run due to random initialization of the network.\n\nOnce again this is explained clearly \n> This requirement is in place for the host team to verify the performance of the uploaded models matches the Stage 2 submission file. Compliance with the above will be verified by the host team. Submitters who fail to upload their model by the Stage 1 deadline, or are found not to be in compliance, may be disqualified from Stage 2 and removed from the final leaderboard.\n\nAnd finally, if you look at the part \"10. WINNERS OBLIGATIONS.\" of the official competition \"[Rules][1]\", it is said that\n\n> As a condition to being awarded a Prize, a Prize winner must fulfill the following obligations:\n\n> (a) deliver to the Competition Sponsor the final **model's software code** as used to generate the winning Submission and associated documentation. The delivered software code must be capable of generating the winning Submission and contain a description of resources required to build and/or run the executable code successfully;\n\nSo, **\"Model\"** and **\"model's software code\"** are definitely defined as differently and we would not able to change anything after stage 1 except generating prediction for new test set of stage 2 using the stage 1 model (by which stage 1 test set was generated), at least for this competition.\n\nHowever, all of a sudden we get STAGE 2 UPDATE \n> STAGE 2 UPDATE\n\n> Note that new files are available to download! The training set now contains both the train and test set from stage 1. The test set is comprised of new, unseen images. The metric and file formats remain the same, but you'll now be making predictions using the updated train and test sets. We have an FAQ about two-stage competitions that provides some context for how this all works. Please give it a read.\n\n> Good luck!\n\nA lot of folks have really invested very hard and intensive efforts to solve a real-life problem. It will be really a frustrating situation for all of them for a sudden and unexpected change of competition rules which may make their strategy effected. This isn't the matter of winning the competition, this is the matter of advancing an effective and robust solution for the community.\n\n**How other folks are thinking the situation?**\n\n [1]: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/rules\n [2]: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge#Timelined\n\n" } }