Hello all,\nafter reading the competiton rules, it is not clear to me what is going on whith the stages.\n- Due to the fact that on the timeline there are only the stage 1 deadlines, will the prizes and award points be awarded based on the stage 1?\n-The stage 2 will be a 2nd competition?\n- Only the teams that will submit their model, will be awarded the ranking points and the prizes?
\n\nthank you
", "messages": [ { "id": "383756", "postDate": "09/09/2018 14:39:25", "content": "Hello all,\nafter reading the competiton rules, it is not clear to me what is going on whith the stages.\n- Due to the fact that on the timeline there are only the stage 1 deadlines, will the prizes and award points be awarded based on the stage 1?\n-The stage 2 will be a 2nd competition?\n- Only the teams that will submit their model, will be awarded the ranking points and the prizes?
\n\nthank you
", "rawMarkdown": "Hello all,\nafter reading the competiton rules, it is not clear to me what is going on whith the stages.\n- Due to the fact that on the timeline there are only the stage 1 deadlines, will the prizes and award points be awarded based on the stage 1?\n-The stage 2 will be a 2nd competition?\n- Only the teams that will submit their model, will be awarded the ranking points and the prizes?\n\nthank you", "votes": null }, { "id": "383775", "postDate": "09/09/2018 15:34:30", "content": "The simple (non-Administer answer):
\n\n[answer is based on my experience with prior contests, not an official answer from Kaggle]
\n\nSome Kaggle contests have two stages. This is done when you could look at the Stage 1 and Stage 2 data and figure out for yourself the \"truth\". Since we are dealing with images, one can \"game\" the system by manually looking at each Stage 1 image and tune your model to do very well on Stage 1, but only because you can look at the answers. (I also note that annotating the test data is specifically prohibited in this contest.)
\n\nYou could also then look at the Stage 2 data and tune your model better, simply because you can see where the pneumonia is on the pictures (okay, easier for a radiologist than a programmer :^)
\n\nYou could also try and use public leaderboard results on Stage 2 to manually tweak parameters or blending ratios based on leaderboard results.
\n\nTo avoid this, you \"lock\" your model at the end of stage 1. Then, you have a week during stage 2 to run your existing model against the new data and make a submission.
\n\nNo prizes/ranks are based on Stage 1 data. All prizes/ranks/private leaderboard standings are based on Stage 2.
\n\nThe \"Timelines\" page has dates for both Stage 1 and Stage 2.
\n\nYou must make a submission and upload your model by the Stage 1 deadline. Your model should be able to run in an appropriate environment, but it does not specifically have to run in a Kaggle Kernal. It could take days to run and tons of memory. But it should represent the model you are going to run on Stage 2.
\n\nUpload (on the \"Team\" page) your program and any supporting files that would be needed to prove it all existed before Stage 2 started. If in doubt, upload your supporting files.
\n\nFor Stage 2 you must make a submission by the deadline.
\n\nTo get ranking points you must make a Stage 1 submission, upload your model by the end of Stage 1, and make a Stage 2 submission. Otherwise, you will be removed from the final private leaderboad (and probably public also).
\n\nFrom experience (in one prior contest with two stages), if you don't win a prize, they do not try and run your model. Clearly for hundreds of teams, it would be unrealistic to try and run all the models.
\n\nIf you qualify for a prize, you have to show that your Stage 2 submission is based on your Stage 1 model. My sense is that something trivial like updating a directory name is allowed. But you cannot manually change your number of layers, number of epochs, learning rate, etc because it works better on stage 2 data. It is up to the sponsoring host (I presume RSNA) to determine their validation procedures.
\n\nGood luck!
", "rawMarkdown": "The simple (non-Administer answer):\n\n[answer is based on my experience with prior contests, not an official answer from Kaggle]\n\nSome Kaggle contests have two stages. This is done when you could look at the Stage 1 and Stage 2 data and figure out for yourself the \"truth\". Since we are dealing with images, one can \"game\" the system by manually looking at each Stage 1 image and tune your model to do very well on Stage 1, but only because you can look at the answers. (I also note that annotating the test data is specifically prohibited in this contest.)\n\nYou could also then look at the Stage 2 data and tune your model better, simply because you can see where the pneumonia is on the pictures (okay, easier for a radiologist than a programmer :^)\n\nYou could also try and use public leaderboard results on Stage 2 to manually tweak parameters or blending ratios based on leaderboard results.\n\nTo avoid this, you \"lock\" your model at the end of stage 1. Then, you have a week during stage 2 to run your existing model against the new data and make a submission. \n\nNo prizes/ranks are based on Stage 1 data. All prizes/ranks/private leaderboard standings are based on Stage 2.\n\nThe \"Timelines\" page has dates for both Stage 1 and Stage 2.\n\nYou must make a submission and upload your model by the Stage 1 deadline. Your model should be able to run in an appropriate environment, but it does not specifically have to run in a Kaggle Kernal. It could take days to run and tons of memory. But it should represent the model you are going to run on Stage 2.\n\nUpload (on the \"Team\" page) your program and any supporting files that would be needed to prove it all existed before Stage 2 started. If in doubt, upload your supporting files.\n\nFor Stage 2 you must make a submission by the deadline.\n\nTo get ranking points you must make a Stage 1 submission, upload your model by the end of Stage 1, and make a Stage 2 submission. Otherwise, you will be removed from the final private leaderboad (and probably public also).\n\nFrom experience (in one prior contest with two stages), if you don't win a prize, they do not try and run your model. Clearly for hundreds of teams, it would be unrealistic to try and run all the models.\n\nIf you qualify for a prize, you have to show that your Stage 2 submission is based on your Stage 1 model. My sense is that something trivial like updating a directory name is allowed. But you cannot manually change your number of layers, number of epochs, learning rate, etc because it works better on stage 2 data. It is up to the sponsoring host (I presume RSNA) to determine their validation procedures.\n\nGood luck!", "votes": null }, { "id": "383866", "postDate": "09/09/2018 19:46:36", "content": "thank you very much for your clear eplaination!!\nthat makes sense now!!\nGood luck through the competition
", "rawMarkdown": "thank you very much for your clear eplaination!!\nthat makes sense now!!\nGood luck through the competition", "votes": null }, { "id": "383930", "postDate": "09/10/2018 01:30:37", "content": "Does the \"model\" mean a program script or a model file just like *.h5 in keras ?
", "rawMarkdown": "Does the \"model\" mean a program script or a model file just like *.h5 in keras ?", "votes": null }, { "id": "385157", "postDate": "09/10/2018 13:00:12", "content": "Hi Teng-Yi, see Phil Culliton's answer here: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/64270#377149.
", "rawMarkdown": "Hi Teng-Yi, see Phil Culliton's answer here: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/64270#377149.", "votes": null }, { "id": "385331", "postDate": "09/10/2018 19:52:35", "content": "I got it, thank you.
", "rawMarkdown": "I got it, thank you.", "votes": null } ], "comments": [ { "id": 383775, "author_name": "richardepstein", "author_url": "", "post_date": "09/09/2018 15:34:30", "content": "The simple (non-Administer answer):
\n\n[answer is based on my experience with prior contests, not an official answer from Kaggle]
\n\nSome Kaggle contests have two stages. This is done when you could look at the Stage 1 and Stage 2 data and figure out for yourself the \"truth\". Since we are dealing with images, one can \"game\" the system by manually looking at each Stage 1 image and tune your model to do very well on Stage 1, but only because you can look at the answers. (I also note that annotating the test data is specifically prohibited in this contest.)
\n\nYou could also then look at the Stage 2 data and tune your model better, simply because you can see where the pneumonia is on the pictures (okay, easier for a radiologist than a programmer :^)
\n\nYou could also try and use public leaderboard results on Stage 2 to manually tweak parameters or blending ratios based on leaderboard results.
\n\nTo avoid this, you \"lock\" your model at the end of stage 1. Then, you have a week during stage 2 to run your existing model against the new data and make a submission.
\n\nNo prizes/ranks are based on Stage 1 data. All prizes/ranks/private leaderboard standings are based on Stage 2.
\n\nThe \"Timelines\" page has dates for both Stage 1 and Stage 2.
\n\nYou must make a submission and upload your model by the Stage 1 deadline. Your model should be able to run in an appropriate environment, but it does not specifically have to run in a Kaggle Kernal. It could take days to run and tons of memory. But it should represent the model you are going to run on Stage 2.
\n\nUpload (on the \"Team\" page) your program and any supporting files that would be needed to prove it all existed before Stage 2 started. If in doubt, upload your supporting files.
\n\nFor Stage 2 you must make a submission by the deadline.
\n\nTo get ranking points you must make a Stage 1 submission, upload your model by the end of Stage 1, and make a Stage 2 submission. Otherwise, you will be removed from the final private leaderboad (and probably public also).
\n\nFrom experience (in one prior contest with two stages), if you don't win a prize, they do not try and run your model. Clearly for hundreds of teams, it would be unrealistic to try and run all the models.
\n\nIf you qualify for a prize, you have to show that your Stage 2 submission is based on your Stage 1 model. My sense is that something trivial like updating a directory name is allowed. But you cannot manually change your number of layers, number of epochs, learning rate, etc because it works better on stage 2 data. It is up to the sponsoring host (I presume RSNA) to determine their validation procedures.
\n\nGood luck!
", "votes": null, "replies": [ { "id": 383866, "author_name": "dkaraflos", "author_url": "", "post_date": "09/09/2018 19:46:36", "content": "thank you very much for your clear eplaination!!\nthat makes sense now!!\nGood luck through the competition
", "votes": null, "replies": [] } ] }, { "id": 383930, "author_name": "tyhuang", "author_url": "", "post_date": "09/10/2018 01:30:37", "content": "Does the \"model\" mean a program script or a model file just like *.h5 in keras ?
", "votes": null, "replies": [ { "id": 385157, "author_name": "mrbarbasa", "author_url": "", "post_date": "09/10/2018 13:00:12", "content": "Hi Teng-Yi, see Phil Culliton's answer here: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/64270#377149.
", "votes": null, "replies": [] } ] }, { "id": 385331, "author_name": "tyhuang", "author_url": "", "post_date": "09/10/2018 19:52:35", "content": "I got it, thank you.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "383756": "Hello all,\nafter reading the competiton rules, it is not clear to me what is going on whith the stages.\n- Due to the fact that on the timeline there are only the stage 1 deadlines, will the prizes and award points be awarded based on the stage 1?\n-The stage 2 will be a 2nd competition?\n- Only the teams that will submit their model, will be awarded the ranking points and the prizes?\n\nthank you", "383775": "The simple (non-Administer answer):\n\n[answer is based on my experience with prior contests, not an official answer from Kaggle]\n\nSome Kaggle contests have two stages. This is done when you could look at the Stage 1 and Stage 2 data and figure out for yourself the \"truth\". Since we are dealing with images, one can \"game\" the system by manually looking at each Stage 1 image and tune your model to do very well on Stage 1, but only because you can look at the answers. (I also note that annotating the test data is specifically prohibited in this contest.)\n\nYou could also then look at the Stage 2 data and tune your model better, simply because you can see where the pneumonia is on the pictures (okay, easier for a radiologist than a programmer :^)\n\nYou could also try and use public leaderboard results on Stage 2 to manually tweak parameters or blending ratios based on leaderboard results.\n\nTo avoid this, you \"lock\" your model at the end of stage 1. Then, you have a week during stage 2 to run your existing model against the new data and make a submission. \n\nNo prizes/ranks are based on Stage 1 data. All prizes/ranks/private leaderboard standings are based on Stage 2.\n\nThe \"Timelines\" page has dates for both Stage 1 and Stage 2.\n\nYou must make a submission and upload your model by the Stage 1 deadline. Your model should be able to run in an appropriate environment, but it does not specifically have to run in a Kaggle Kernal. It could take days to run and tons of memory. But it should represent the model you are going to run on Stage 2.\n\nUpload (on the \"Team\" page) your program and any supporting files that would be needed to prove it all existed before Stage 2 started. If in doubt, upload your supporting files.\n\nFor Stage 2 you must make a submission by the deadline.\n\nTo get ranking points you must make a Stage 1 submission, upload your model by the end of Stage 1, and make a Stage 2 submission. Otherwise, you will be removed from the final private leaderboad (and probably public also).\n\nFrom experience (in one prior contest with two stages), if you don't win a prize, they do not try and run your model. Clearly for hundreds of teams, it would be unrealistic to try and run all the models.\n\nIf you qualify for a prize, you have to show that your Stage 2 submission is based on your Stage 1 model. My sense is that something trivial like updating a directory name is allowed. But you cannot manually change your number of layers, number of epochs, learning rate, etc because it works better on stage 2 data. It is up to the sponsoring host (I presume RSNA) to determine their validation procedures.\n\nGood luck!", "383866": "thank you very much for your clear eplaination!!\nthat makes sense now!!\nGood luck through the competition", "383930": "Does the \"model\" mean a program script or a model file just like *.h5 in keras ?", "385157": "Hi Teng-Yi, see Phil Culliton's answer here: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/discussion/64270#377149.", "385331": "I got it, thank you." }, "source": "meta" }