This is your reminder that October 24th (next week) is the close of stage 1 and deadline to both make your 1 final submission selection and upload the corresponding model. There will be no exceptions to this deadline, and prize eligibility will be contingent on compliance with this requirement.
\n\nUpon the close of stage 1, stage 2 data will be released, with only 6 days to submit your predictions on that test set.
\n\nSee the timeline page for full details.
", "messages": [ { "id": 407146, "postDate": "2018-10-20T13:24:15.860Z", "content": "Can we include an MD5 checksum for our model archive so we don't need to upload all of the models we used (unless we win)? The weight files are pretty large, worried about spotty internet connections interrupting the upload...
", "rawMarkdown": "Can we include an MD5 checksum for our model archive so we don't need to upload all of the models we used (unless we win)? The weight files are pretty large, worried about spotty internet connections interrupting the upload...", "votes": 8, "replies": [ { "id": 407925, "postDate": "2018-10-22T02:04:06.620Z", "content": "This would be really great! This is a real problem for us in Australia, where the upload speeds are abysmal.
\n\nAnother possibility would be if I could provide a script for downloading the models from S3 or something. At least then I don't have to worry about my connection dropping out while uploading the model via the UI. Is that acceptable?
", "rawMarkdown": "This would be really great! This is a real problem for us in Australia, where the upload speeds are abysmal. \n\nAnother possibility would be if I could provide a script for downloading the models from S3 or something. At least then I don't have to worry about my connection dropping out while uploading the model via the UI. Is that acceptable?", "votes": 1 }, { "id": 409037, "postDate": "2018-10-23T18:40:56.203Z", "content": "The MD5 checksum would be beneficial for the challenge participants so I don't see any conflicts. Can you make this available before the upcoming deadlines?
", "rawMarkdown": "The MD5 checksum would be beneficial for the challenge participants so I don't see any conflicts. Can you make this available before the upcoming deadlines?", "votes": 2 } ] }, { "id": 407505, "postDate": "2018-10-21T10:52:53.067Z", "content": "Will you be releasing the stage 1 labels when you release the stage 2 dataset?
", "rawMarkdown": "Will you be releasing the stage 1 labels when you release the stage 2 dataset?", "votes": 5 }, { "id": 407224, "postDate": "2018-10-20T16:42:23.380Z", "content": "Julia, what is the reason behind the stage 1 final submission selection? As I understand the final score will be based on the second stage results only or have I missed something?
", "rawMarkdown": "Julia, what is the reason behind the stage 1 final submission selection? As I understand the final score will be based on the second stage results only or have I missed something?", "votes": 5 }, { "id": 405564, "postDate": "2018-10-17T18:03:28.600Z", "content": "This is your reminder that October 24th (next week) is the close of stage 1 and deadline to both make your 1 final submission selection and upload the corresponding model. There will be no exceptions to this deadline, and prize eligibility will be contingent on compliance with this requirement.
\n\nUpon the close of stage 1, stage 2 data will be released, with only 6 days to submit your predictions on that test set.
\n\nSee the timeline page for full details.
", "rawMarkdown": "This is your reminder that October 24th (next week) is the close of stage 1 and deadline to both make your 1 final submission selection and upload the corresponding model. There will be no exceptions to this deadline, and prize eligibility will be contingent on compliance with this requirement.\n\nUpon the close of stage 1, stage 2 data will be released, with only 6 days to submit your predictions on that test set.\n\nSee the [timeline page][1] for full details. \n\n\n[1]: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge#Timeline", "votes": 5 }, { "id": 410396, "postDate": "2018-10-26T01:43:18.220Z", "content": "Sorry, I made a mistake in uploading file... I only upload my weights of my model, will I be permitted into stage2 :( ?
", "rawMarkdown": "Sorry, I made a mistake in uploading file... I only upload my weights of my model, will I be permitted into stage2 :( ? ", "votes": 3 }, { "id": 410257, "postDate": "2018-10-25T17:27:00.473Z", "content": "This is all very confusing, just adding my voice to the choir asking for some clarification please Kaggle.
", "rawMarkdown": "This is all very confusing, just adding my voice to the choir asking for some clarification please Kaggle.", "votes": 3 }, { "id": 410425, "postDate": "2018-10-26T03:20:30.780Z", "content": "We just uploaded the model and the code for test。。。。。did I misunderstanding what is 'model' mean?
", "rawMarkdown": "We just uploaded the model and the code for test。。。。。did I misunderstanding what is 'model' mean?", "votes": 4, "replies": [ { "id": 410758, "postDate": "2018-10-26T16:09:13.710Z", "rawMarkdown": "", "isDeleted": true }, { "id": 410857, "postDate": "2018-10-26T19:07:47.297Z", "content": "yes..but there only said to upload the files to get my submission...I thought the weights or MD5 is the most important. Now I won't retrain my model ,and keep the weights I have uploaded...maybe this can avoid the problem.
", "rawMarkdown": "yes..but there only said to upload the files to get my submission...I thought the weights or MD5 is the most important. Now I won't retrain my model ,and keep the weights I have uploaded...maybe this can avoid the problem." } ] }, { "id": 406350, "postDate": "2018-10-19T05:29:11.117Z", "content": "Are there any guidelines regarding the model upload?
\n\nI've found this FAQs very helpful.\nhttps://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/32580
\n\nIs the model upload rule of this competition is as same as described in above FAQs?
", "rawMarkdown": "Are there any guidelines regarding the model upload?\n\nI've found this FAQs very helpful.\nhttps://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/32580\n\nIs the model upload rule of this competition is as same as described in above FAQs?", "votes": 4 }, { "id": 414072, "postDate": "2018-11-02T03:44:05.453Z", "content": "My score has disappeared from the leaderboard ,could you give me a reason?I upload my model ,and I didn\n't change anything in stage 2......
", "rawMarkdown": "My score has disappeared from the leaderboard ,could you give me a reason?I upload my model ,and I didn\n't change anything in stage 2......", "votes": 1 }, { "id": 410783, "postDate": "2018-10-26T16:51:36.147Z", "content": "From the requirement to make the stage 1 final submission selection and uploaded model to match this submission, I expected any re-training with the released stage 1 labels would violate the second requirement and so not be allowed.
", "rawMarkdown": "From the requirement to make the stage 1 final submission selection and uploaded model to match this submission, I expected any re-training with the released stage 1 labels would violate the second requirement and so not be allowed.", "votes": 1, "replies": [ { "id": 410845, "postDate": "2018-10-26T18:42:29.130Z", "content": "It seems that retraining is ok, IF no hyperparameter is changed.
", "rawMarkdown": "It seems that retraining is ok, IF no hyperparameter is changed." } ] }, { "id": 410351, "postDate": "2018-10-25T22:17:51.993Z", "content": "It might be clearer to call it a \"learner\"/\"learning algorithm\" upload deadline in the future.
", "rawMarkdown": "It might be clearer to call it a \"learner\"/\"learning algorithm\" upload deadline in the future.", "votes": 1 }, { "id": 406781, "postDate": "2018-10-19T21:26:17.450Z", "content": "Hi, some questions:
\n\nreference:
\n\n\n", "rawMarkdown": "Hi, some questions:\n\n 1. are you guys executing each team's inference routine on the stage 2 data to match the stage 2 submission?\n 2. does this mean we are not allowed to change anything in the model (hyperparameters, arch, etc..) after the model upload, while stage 2 data is out? How do you match model to submission? how is the auto-select based on stage1 done?\n\nreference:\n\n> Be aware that if you do not select a final submission (via 'My\n> Submissions'), the platform will auto-select your best-scoring model\n> on the Stage 1 public leaderboard. The deadline for model upload is\n> firmly the end of Stage 1.\n> \n> This requirement is in place for the host team to verify the\n> performance of the uploaded models matches the Stage 2 submission\n> file. Compliance with the above will be verified by the host team.\n\n", "votes": 1, "replies": [ { "id": 407008, "postDate": "2018-10-20T07:08:20.827Z", "content": "Be aware that if you do not select a final submission (via 'My\n Submissions'), the platform will auto-select your best-scoring model\n on the Stage 1 public leaderboard. The deadline for model upload is\n firmly the end of Stage 1.
\n \nThis requirement is in place for the host team to verify the\n performance of the uploaded models matches the Stage 2 submission\n file. Compliance with the above will be verified by the host team.
\n
At least from my previous experience, you need to upload the model and instructions how to reproduce the exact stage 2 submission, so you can't and not supposed to change any hyper parameters etc. I'd expect Kaggle to check at least top teams solutions.
", "rawMarkdown": "At least from my previous experience, you need to upload the model and instructions how to reproduce the exact stage 2 submission, so you can't and not supposed to change any hyper parameters etc. I'd expect Kaggle to check at least top teams solutions.", "votes": 1 }, { "id": 407075, "postDate": "2018-10-20T10:42:00.120Z", "content": "Hi Dmytro, does it mean that we need to upload our script/s via kaggle kernel that produce a result when we do result submission with scoring in leaderboard? how about the scenario that scripts can't reproduce result submission due to computational constraints? Sorry, this is my first time on this type of competition.
", "rawMarkdown": "Hi Dmytro, does it mean that we need to upload our script/s via kaggle kernel that produce a result when we do result submission with scoring in leaderboard? how about the scenario that scripts can't reproduce result submission due to computational constraints? Sorry, this is my first time on this type of competition.", "votes": 1 }, { "id": 407222, "postDate": "2018-10-20T16:35:04.780Z", "content": "It's in the \"Team\" tab, you can upload your model source code and instructions how to run it, you need to upload it before the second stage starts.
", "rawMarkdown": "It's in the \"Team\" tab, you can upload your model source code and instructions how to run it, you need to upload it before the second stage starts." } ] }, { "id": 410052, "postDate": "2018-10-25T09:34:38.290Z", "content": "Can submission window for stage1 could be opened again. I missed the deadline.
", "rawMarkdown": "Can submission window for stage1 could be opened again. I missed the deadline." }, { "id": 408333, "postDate": "2018-10-22T17:38:50.107Z", "content": "NOTE: There was an update made to bullet 6 in my original response. I'd misspoken and made a typo in that response. The Stage 1 labels WILL be provided when Stage 2 is updated. Apologies!
\n\nA few clarifications on the model upload:
\n\nSo, basically, checksums of large files are enough? This will allow you to perform the verification if needed if I under the procedure
", "rawMarkdown": "So, basically, checksums of large files are enough? This will allow you to perform the verification if needed if I under the procedure" }, { "id": 409011, "postDate": "2018-10-23T18:03:06.437Z", "content": "One model per team? Not one model per team member?
", "rawMarkdown": "One model per team? Not one model per team member?" }, { "id": 409742, "postDate": "2018-10-24T19:30:05.570Z", "content": "Yes, one model per team.
", "rawMarkdown": "Yes, one model per team." }, { "id": 409743, "postDate": "2018-10-24T19:33:41.120Z", "content": "@Julia For clarification, there will be only be a test dataset provided for stage2? Not a training set as well?
", "rawMarkdown": "@Julia For clarification, there will be only be a test dataset provided for stage2? Not a training set as well? " }, { "id": 409754, "postDate": "2018-10-24T19:51:49.460Z", "content": "@William - good question. When Stage 2 is updated, we will wipe clean the leaderboards. The new Stage 2 train set will contain a combination of the original Stage 1 training data, as well as Stage 1's test data. All of that will be provided with its labels.
\n\nThe new Stage 2 test set (with labels withheld) will contain a few rows to be used for the public leaderboard, but mostly used towards the private leaderboard. This is intended to provide competitors a tiny bit of signal as to where they are roughly scoring.
", "rawMarkdown": "@William - good question. When Stage 2 is updated, we will wipe clean the leaderboards. The new Stage 2 train set will contain a combination of the original Stage 1 training data, as well as Stage 1's test data. All of that will be provided with its labels. \n\nThe new Stage 2 test set (with labels withheld) will contain a few rows to be used for the public leaderboard, but mostly used towards the private leaderboard. This is intended to provide competitors a tiny bit of signal as to where they are roughly scoring." }, { "id": 409781, "postDate": "2018-10-24T20:28:45.680Z", "content": "Does that mean only the portion of the stage 2 test set that is held out for the private leaderboard will be used to determine the final ranking?
", "rawMarkdown": "Does that mean only the portion of the stage 2 test set that is held out for the private leaderboard will be used to determine the final ranking? " }, { "id": 409786, "postDate": "2018-10-24T20:49:18.793Z", "content": "@Ian - Yes.
", "rawMarkdown": "@Ian - Yes." }, { "id": 409821, "postDate": "2018-10-24T22:40:43.937Z", "content": "@Julia
\n\nYou are referring to a new Stage 2 train set that partially includes Stage 1 test data.
\n\nQ1 : If it is a train set, does it mean we can retrain/finetune our(s) model(s) on this new available train set ?
\n\nFrom the available information, we thought the uploaded model needed to be exactly the same for stage 1 and stage 2 with the same inference parameters. That was the format used in DSB 2018 which makes sense to prevent any kind of overfitting of Stage 2 test data.
\n\nSo, if we can retrain/finetune our models with Stage 2 train set, then the model(s) will not be the same as the one(s) uploaded.
\n\nQ2 : And, If we can't retrain (as we thought), then why calling the new available labels Stage 2 train set ?
", "rawMarkdown": "@Julia\n\nYou are referring to a new Stage 2 train set that partially includes Stage 1 test data. \n\nQ1 : If it is a train set, does it mean we can retrain/finetune our(s) model(s) on this new available train set ?\n\nFrom the available information, we thought the uploaded model needed to be exactly the same for stage 1 and stage 2 with the same inference parameters. That was the format used in DSB 2018 which makes sense to prevent any kind of overfitting of Stage 2 test data.\n\nSo, if we can retrain/finetune our models with Stage 2 train set, then the model(s) will not be the same as the one(s) uploaded.\n\nQ2 : And, If we can't retrain (as we thought), then why calling the new available labels Stage 2 train set ?", "votes": 18 }, { "id": 409975, "postDate": "2018-10-25T06:28:27.183Z", "content": "@Alexandre Cadrin-Chênevert
\n\nI agree with you. According to the logic of the competition, the Stage 1 labels do not need to be published until the competition is over.
", "rawMarkdown": "@Alexandre Cadrin-Chênevert\n\nI agree with you. According to the logic of the competition, the Stage 1 labels do not need to be published until the competition is over.", "votes": 3 }, { "id": 409982, "postDate": "2018-10-25T06:42:15.673Z", "content": "Agree~
", "rawMarkdown": "Agree~" }, { "id": 410081, "postDate": "2018-10-25T10:42:58.047Z", "rawMarkdown": "", "votes": 4, "isDeleted": true }, { "id": 410093, "postDate": "2018-10-25T11:57:25.620Z", "content": "I have the same questions as @Alexandre Cadrin-Chênevert.
", "rawMarkdown": "I have the same questions as @Alexandre Cadrin-Chênevert." }, { "id": 410168, "postDate": "2018-10-25T14:52:20.457Z", "content": "@statsintuition Indeed, changing the rules within less than one day of the deadline, since Julia's correction to her response above could change the appropriate strategy. Given the difficulties of the model selection and upload process, that's likely too late for most people to do anything about it, even if they got the information immediately. And even after stage 1, the rules are still unclear: what aspects of the model (nothing at all? just the fitted weights? certain kinds of hyperparameters? all of the hyperparameters?) can we change, after receiving the stage 1 test labels, without being considered in violation of the rules?
", "rawMarkdown": "@statsintuition Indeed, changing the rules within *less than one* day of the deadline, since Julia's correction to her response above could change the appropriate strategy. Given the difficulties of the model selection and upload process, that's likely too late for most people to do anything about it, even if they got the information immediately. And even after stage 1, the rules are still unclear: what aspects of the model (nothing at all? just the fitted weights? certain kinds of hyperparameters? all of the hyperparameters?) can we change, after receiving the stage 1 test labels, without being considered in violation of the rules?", "votes": 4 }, { "id": 410173, "postDate": "2018-10-25T15:02:43.140Z", "content": "@Andy We ran into the same situation with DSB 2018. I think going forward:
\n\nAnd even after stage 1, the rules are still unclear: what aspects of the model (nothing at all? just the fitted weights? certain kinds of hyperparameters? all of the hyperparameters?) \nneeds be clearly define in the competitions rules. If there are a lot of questions being asked, then that is a good indicator that the rules or portion of the rules are not clearly defined. In some case, the rules need to be really spelled out so everyone has a clear understanding.
", "rawMarkdown": "@Andy We ran into the same situation with DSB 2018. I think going forward:\n\n And even after stage 1, the rules are still unclear: what aspects of the model (nothing at all? just the fitted weights? certain kinds of hyperparameters? all of the hyperparameters?) \n\nneeds be clearly define in the competitions rules. If there are a lot of questions being asked, then that is a good indicator that the rules or portion of the rules are not clearly defined. In some case, the rules need to be really spelled out so everyone has a clear understanding. \n\n", "votes": 2 }, { "id": 410264, "postDate": "2018-10-25T17:41:59.123Z", "content": "To be clear: the rules have not changed.
\n\nI realize confusion may have been caused in attempting to clarify that the stage 2 dataset will include stage 1's labels. We have resurrected a \"Two-stage competition FAQ\" page that had been inadvertently deprecated in a recent migration. These should address many of the questions about what you can and cannot do to your models during Stage 2. This is the identical design and approach taken in prior 2-stage competitions, including DSB 2018.
\n\nAs stated there: \"We expect you may need to make some \"non scientific\" alterations, such as changes to path names, in order to create your submissions for the second stage. You are allowed to re-train your model (including the stage one data), but your code should not change. You should not be doing any hyper parameter tuning in the second stage. Parameter tuning is permitted as long as it is fully automated.\"
\n\nLastly, the Stage 2 data will be available today (October 25th) no later than 11:59pm UTC, in line with all of the other stated deadlines on the Timeline page.
", "rawMarkdown": "To be clear: **the rules have not changed**.\n\nI realize confusion may have been caused in attempting to clarify that the stage 2 dataset will include stage 1's labels. We have resurrected a \"[Two-stage competition FAQ][1]\" page that had been inadvertently deprecated in a recent migration. These should address many of the questions about what you can and cannot do to your models during Stage 2. This is the identical design and approach taken in prior 2-stage competitions, including DSB 2018.\n\nAs stated there: *\"We expect you may need to make some \"non scientific\" alterations, such as changes to path names, in order to create your submissions for the second stage. You are allowed to re-train your model (including the stage one data), but your code should not change. You should not be doing any hyper parameter tuning in the second stage. Parameter tuning is permitted as long as it is fully automated.\"*\n\nLastly, the Stage 2 data will be available today (October 25th) no later than 11:59pm UTC, in line with all of the other stated deadlines on the Timeline page.\n\n\n[1]: https://www.kaggle.com/two-stage-frequently-asked-questions" }, { "id": 410269, "postDate": "2018-10-25T17:53:37.517Z", "content": "Thanks for the update. It's a shame we didn't have access to the \"Two-stage competition FAQ\" page before submission, that makes things a lot clearer. Hopefully the requirements don't catch too many people out.
", "rawMarkdown": "Thanks for the update. It's a shame we didn't have access to the \"Two-stage competition FAQ\" page before submission, that makes things a lot clearer. Hopefully the requirements don't catch too many people out.", "votes": 5 }, { "id": 410287, "postDate": "2018-10-25T19:00:18.910Z", "content": "If we are using an ensemble of models,how could we tuning? Tuning part of weight? Or we can even add a new model with the same hyper parameter? I think lock the weights we uploaded is much better.......
", "rawMarkdown": "If we are using an ensemble of models,how could we tuning? Tuning part of weight? Or we can even add a new model with the same hyper parameter? I think lock the weights we uploaded is much better.......", "votes": 5 }, { "id": 410345, "postDate": "2018-10-25T21:53:00.497Z", "content": "Thanks Julia for the answer. The ressurected FAQ is clear.
\n\nI didn`t know before model upload deadline that we could retrain for stage 2 using stage 1 test labels.
\n\nI will be aware next time.
", "rawMarkdown": "Thanks Julia for the answer. The ressurected FAQ is clear.\n\nI didn`t know before model upload deadline that we could retrain for stage 2 using stage 1 test labels.\n\nI will be aware next time.", "votes": 2 }, { "id": 410613, "postDate": "2018-10-26T10:49:51.127Z", "content": "Thank you for your explanation.\nHowever, the \"Two-stage competition FAQ\" page did not supply participants at the end of the stage 1. \nIf the team only submits the training ”model“(model weight) in the stage 1 instead of the training code, can they retrain the data using the same code.
", "rawMarkdown": "Thank you for your explanation.\nHowever, the \"Two-stage competition FAQ\" page did not supply participants at the end of the stage 1. \nIf the team only submits the training ”model“(model weight) in the stage 1 instead of the training code, can they retrain the data using the same code.", "votes": 2 }, { "id": 410656, "postDate": "2018-10-26T12:16:06.697Z", "content": "@Julia
\n\nThanks for the clarification. In the quote you provided, it says hyperparameter tuning is not allowed, but parameter tuning is allowed if it is fully-automated. Would fully-automated hyperparameter tuning be allowed?
", "rawMarkdown": "@Julia\n\nThanks for the clarification. In the quote you provided, it says hyperparameter tuning is not allowed, but parameter tuning is allowed if it is fully-automated. Would fully-automated hyperparameter tuning be allowed?" }, { "id": 410753, "postDate": "2018-10-26T15:51:10.370Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 409745, "postDate": "2018-10-24T19:36:08.800Z", "content": "NOTE: There was an update made to bullet 6 in my clarification response. I'd misspoken and made a typo in that response. The Stage 1 labels WILL be provided when Stage 2 is updated. Apologies!
", "rawMarkdown": "NOTE: There was an update made to bullet 6 in my clarification response. I'd misspoken and made a typo in that response. The Stage 1 labels WILL be provided when Stage 2 is updated. Apologies!", "votes": -2, "replies": [ { "id": 409947, "postDate": "2018-10-25T05:28:02.477Z", "content": "But we have to use the same model we uploaded, and are not allowed to use the stage 1 data and label to train a new model, right?
", "rawMarkdown": "But we have to use the same model we uploaded, and are not allowed to use the stage 1 data and label to train a new model, right?\n", "votes": 1 }, { "id": 409963, "postDate": "2018-10-25T05:56:02.527Z", "content": "Excuse me but I do not see what you mean. The model and code we uploaded do not include any training code, as a consequence of the previous rules that there will not be any new labels and no further training will be possible. So We cannot use the new labels without changing completely the code and model that we have uploaded. Changing the rules of the competition just after it ends is a bit weird, I have to say...
", "rawMarkdown": "Excuse me but I do not see what you mean. The model and code we uploaded do not include any training code, as a consequence of the previous rules that there will not be any new labels and no further training will be possible. So We cannot use the new labels without changing completely the code and model that we have uploaded. Changing the rules of the competition just after it ends is a bit weird, I have to say...", "votes": 7 }, { "id": 410330, "postDate": "2018-10-25T21:09:05.440Z", "content": "yeah and also the rules of the competition should not be changed in comments under a thread called Reminder: Model Upload deadline that doesn't trigger notifications. (Is there a way to turn on email notifications?). The name of the thread suggests that if you uploaded your model you don't need to read the thread which is roughly the opposite of the truth.
humor on
\n\nYay, our model is the darknet (aka yolov3) and the executable will not change. We will train with Stage 2 (er Stage 1) training set and test with Stage 2 test set. It would be very helpful if we could now have ground truth labels for Stage 1 test data too. ;)
\n\nOnly upside for team Yolopacity Beta! We just do garden variety transfer learning. But we are so far down every change in rule helps :)
\n\nhumor off
", "rawMarkdown": "humor on\n\nYay, our model is the darknet (aka yolov3) and the executable will not change. We will train with Stage 2 (er Stage 1) training set and test with Stage 2 test set. It would be very helpful if we could now have ground truth labels for Stage 1 test data too. ;)\n\nOnly upside for team Yolopacity Beta! We just do garden variety transfer learning. But we are so far down every change in rule helps :)\n\nhumor off" }, { "id": 410223, "postDate": "2018-10-25T16:16:32.940Z", "content": "@Julia - Hi Julia, our team tried to upload our models yesterday but the connection was very poor. We were uploading before the deadline but when we finished uploading, it passed the deadline by 30min... The system automatically cancelled our uploaded models.. It was our first time uploading models so we didn't expect uploading required such a long time... I know we may not win any prizes but we hope we could at least attend the stage 2...
", "rawMarkdown": "@Julia - Hi Julia, our team tried to upload our models yesterday but the connection was very poor. We were uploading before the deadline but when we finished uploading, it passed the deadline by 30min... The system automatically cancelled our uploaded models.. It was our first time uploading models so we didn't expect uploading required such a long time... I know we may not win any prizes but we hope we could at least attend the stage 2...", "replies": [ { "id": 410358, "postDate": "2018-10-25T22:30:42.733Z", "content": "Per the two-stage competition FAQ:
\n\n\n\n\nWill I still be on the leaderboard if I don't upload a model but do\n submit in the second stage? Yes. However, if you do not upload a model\n and finish in a prize position, your team will be removed from the\n competition standings entirely.
\n
So as long as you don't finish in a prize position, you should still be on the leaderboard.
", "rawMarkdown": "Per the two-stage competition FAQ:\n\n> Will I still be on the leaderboard if I don't upload a model but do\n> submit in the second stage? Yes. However, if you do not upload a model\n> and finish in a prize position, your team will be removed from the\n> competition standings entirely.\n\nSo as long as you don't finish in a prize position, you should still be on the leaderboard.\n" } ] }, { "id": 410025, "postDate": "2018-10-25T08:28:18.860Z", "content": "This is sad but it was my fault. Didn't realise that I have to select 1 submission!!! This is not like other competitions :(
", "rawMarkdown": "This is sad but it was my fault. Didn't realise that I have to select 1 submission!!! This is not like other competitions :(", "replies": [ { "id": 410046, "postDate": "2018-10-25T09:16:35.927Z", "content": "@Kagoo you may still be okay if you uploaded your best scoring model based on the stage-1 public LB. Because the system will automatically select your best submission for stage-1 data as your 1 final submission.
", "rawMarkdown": "@Kagoo you may still be okay if you uploaded your best scoring model based on the stage-1 public LB. Because the system will automatically select your best submission for stage-1 data as your 1 final submission." } ] }, { "id": 409904, "postDate": "2018-10-25T03:11:51.390Z", "content": "Hi,
\n\nStage 1 is closed. Where can I get stage 2 data?
\n\nThanks,
", "rawMarkdown": "Hi,\n\nStage 1 is closed. Where can I get stage 2 data?\n\nThanks,", "replies": [ { "id": 410048, "postDate": "2018-10-25T09:17:43.913Z", "content": "According to the timeline, stage-2 test data will be available on October 25th.
", "rawMarkdown": "According to the timeline, stage-2 test data will be available on October 25th." } ] }, { "id": 409877, "postDate": "2018-10-25T02:01:46.207Z", "content": "Hi, Unfortunately I missed the model submission deadline. But my team want to keep going on.
\n\nIs it possible to get score from late submission with Stage 1 data??\nWe don't have to be in the leaderboard, just want to make sure that we're developing model better than before.
", "rawMarkdown": "Hi, Unfortunately I missed the model submission deadline. But my team want to keep going on. \n\nIs it possible to get score from late submission with Stage 1 data??\nWe don't have to be in the leaderboard, just want to make sure that we're developing model better than before.", "replies": [ { "id": 410266, "postDate": "2018-10-25T17:43:46.173Z", "content": "You can submit without prize eligibility after the competition is finalized and closed, after October 31, 2018.
", "rawMarkdown": "You can submit without prize eligibility after the competition is finalized and closed, after October 31, 2018." } ] }, { "id": 409329, "postDate": "2018-10-24T06:11:01.197Z", "content": "If we are using an ensemble of models, how would we submit it?
", "rawMarkdown": "If we are using an ensemble of models, how would we submit it?", "replies": [ { "id": 409629, "postDate": "2018-10-24T16:06:27.300Z", "content": "Not official response. I think you can put the code and the weights in a zip file. Regarding the weights, instead of the files, you can include a .txt file with a hash of each file.
", "rawMarkdown": "Not official response. I think you can put the code and the weights in a zip file. Regarding the weights, instead of the files, you can include a .txt file with a hash of each file. ", "votes": 1 }, { "id": 409740, "postDate": "2018-10-24T19:29:55.163Z", "content": "The recommendation provided by Virilo will be adequate.
", "rawMarkdown": "The recommendation provided by Virilo will be adequate." }, { "id": 410748, "postDate": "2018-10-26T15:40:43.830Z", "content": "If check sums are enough then there can be no further training in stage 2, isn't it?
", "rawMarkdown": "If check sums are enough then there can be no further training in stage 2, isn't it?" } ] }, { "id": 410071, "postDate": "2018-10-25T10:24:58.517Z", "rawMarkdown": "", "isDeleted": true }, { "id": 409917, "postDate": "2018-10-25T03:45:18.573Z", "rawMarkdown": "", "isDeleted": true }, { "id": 409901, "postDate": "2018-10-25T03:05:18.467Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 407146, "author_name": "Ian Pan", "author_url": "", "post_date": "2018-10-20T13:24:15.860000", "content": "Can we include an MD5 checksum for our model archive so we don't need to upload all of the models we used (unless we win)? The weight files are pretty large, worried about spotty internet connections interrupting the upload...
", "votes": 8, "replies": [ { "id": 407925, "author_name": "Lex Toumbourou", "author_url": "", "post_date": "2018-10-22T02:04:06.620000", "content": "This would be really great! This is a real problem for us in Australia, where the upload speeds are abysmal.
\n\nAnother possibility would be if I could provide a script for downloading the models from S3 or something. At least then I don't have to worry about my connection dropping out while uploading the model via the UI. Is that acceptable?
", "votes": 1, "replies": [] }, { "id": 409037, "author_name": "Safwan Halabi MD", "author_url": "", "post_date": "2018-10-23T18:40:56.203000", "content": "The MD5 checksum would be beneficial for the challenge participants so I don't see any conflicts. Can you make this available before the upcoming deadlines?
", "votes": 2, "replies": [] } ] }, { "id": 407505, "author_name": "GuyE", "author_url": "", "post_date": "2018-10-21T10:52:53.067000", "content": "Will you be releasing the stage 1 labels when you release the stage 2 dataset?
", "votes": 5, "replies": [] }, { "id": 407224, "author_name": "Dmytro Poplavskiy", "author_url": "", "post_date": "2018-10-20T16:42:23.380000", "content": "Julia, what is the reason behind the stage 1 final submission selection? As I understand the final score will be based on the second stage results only or have I missed something?
", "votes": 5, "replies": [] }, { "id": 410396, "author_name": "building11_301", "author_url": "", "post_date": "2018-10-26T01:43:18.220000", "content": "Sorry, I made a mistake in uploading file... I only upload my weights of my model, will I be permitted into stage2 :( ?
", "votes": 3, "replies": [] }, { "id": 410257, "author_name": "Tom Aindow", "author_url": "", "post_date": "2018-10-25T17:27:00.473000", "content": "This is all very confusing, just adding my voice to the choir asking for some clarification please Kaggle.
", "votes": 3, "replies": [] }, { "id": 410425, "author_name": "zzz333", "author_url": "", "post_date": "2018-10-26T03:20:30.780000", "content": "We just uploaded the model and the code for test。。。。。did I misunderstanding what is 'model' mean?
", "votes": 4, "replies": [ { "id": 410758, "author_name": "", "author_url": "", "post_date": "2018-10-26T16:09:13.710000", "content": "", "votes": 0, "replies": [] }, { "id": 410857, "author_name": "zzz333", "author_url": "", "post_date": "2018-10-26T19:07:47.297000", "content": "yes..but there only said to upload the files to get my submission...I thought the weights or MD5 is the most important. Now I won't retrain my model ,and keep the weights I have uploaded...maybe this can avoid the problem.
", "votes": 0, "replies": [] } ] }, { "id": 406350, "author_name": "Appian", "author_url": "", "post_date": "2018-10-19T05:29:11.117000", "content": "Are there any guidelines regarding the model upload?
\n\nI've found this FAQs very helpful.\nhttps://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/32580
\n\nIs the model upload rule of this competition is as same as described in above FAQs?
", "votes": 4, "replies": [] }, { "id": 414072, "author_name": "zzz333", "author_url": "", "post_date": "2018-11-02T03:44:05.453000", "content": "My score has disappeared from the leaderboard ,could you give me a reason?I upload my model ,and I didn\n't change anything in stage 2......
", "votes": 1, "replies": [] }, { "id": 410783, "author_name": "Dmytro Poplavskiy", "author_url": "", "post_date": "2018-10-26T16:51:36.147000", "content": "From the requirement to make the stage 1 final submission selection and uploaded model to match this submission, I expected any re-training with the released stage 1 labels would violate the second requirement and so not be allowed.
", "votes": 1, "replies": [ { "id": 410845, "author_name": "Fernando Camargo", "author_url": "", "post_date": "2018-10-26T18:42:29.130000", "content": "It seems that retraining is ok, IF no hyperparameter is changed.
", "votes": 0, "replies": [] } ] }, { "id": 410351, "author_name": "Valentin Tiriac", "author_url": "", "post_date": "2018-10-25T22:17:51.993000", "content": "It might be clearer to call it a \"learner\"/\"learning algorithm\" upload deadline in the future.
", "votes": 1, "replies": [] }, { "id": 406781, "author_name": "guang.ster", "author_url": "", "post_date": "2018-10-19T21:26:17.450000", "content": "Hi, some questions:
\n\nreference:
\n\n\n", "votes": 1, "replies": [ { "id": 407008, "author_name": "Dmytro Poplavskiy", "author_url": "", "post_date": "2018-10-20T07:08:20.827000", "content": "Be aware that if you do not select a final submission (via 'My\n Submissions'), the platform will auto-select your best-scoring model\n on the Stage 1 public leaderboard. The deadline for model upload is\n firmly the end of Stage 1.
\n \nThis requirement is in place for the host team to verify the\n performance of the uploaded models matches the Stage 2 submission\n file. Compliance with the above will be verified by the host team.
\n
At least from my previous experience, you need to upload the model and instructions how to reproduce the exact stage 2 submission, so you can't and not supposed to change any hyper parameters etc. I'd expect Kaggle to check at least top teams solutions.
", "votes": 1, "replies": [] }, { "id": 407075, "author_name": "", "author_url": "", "post_date": "2018-10-20T10:42:00.120000", "content": "Hi Dmytro, does it mean that we need to upload our script/s via kaggle kernel that produce a result when we do result submission with scoring in leaderboard? how about the scenario that scripts can't reproduce result submission due to computational constraints? Sorry, this is my first time on this type of competition.
", "votes": 1, "replies": [] }, { "id": 407222, "author_name": "Dmytro Poplavskiy", "author_url": "", "post_date": "2018-10-20T16:35:04.780000", "content": "It's in the \"Team\" tab, you can upload your model source code and instructions how to run it, you need to upload it before the second stage starts.
", "votes": 0, "replies": [] } ] }, { "id": 410052, "author_name": "Ronak Dedhiya", "author_url": "", "post_date": "2018-10-25T09:34:38.290000", "content": "Can submission window for stage1 could be opened again. I missed the deadline.
", "votes": 0, "replies": [] }, { "id": 408333, "author_name": "Julia Elliott", "author_url": "", "post_date": "2018-10-22T17:38:50.107000", "content": "NOTE: There was an update made to bullet 6 in my original response. I'd misspoken and made a typo in that response. The Stage 1 labels WILL be provided when Stage 2 is updated. Apologies!
\n\nA few clarifications on the model upload:
\n\nSo, basically, checksums of large files are enough? This will allow you to perform the verification if needed if I under the procedure
", "votes": 0, "replies": [] }, { "id": 409011, "author_name": "William Green", "author_url": "", "post_date": "2018-10-23T18:03:06.437000", "content": "One model per team? Not one model per team member?
", "votes": 0, "replies": [] }, { "id": 409742, "author_name": "Julia Elliott", "author_url": "", "post_date": "2018-10-24T19:30:05.570000", "content": "Yes, one model per team.
", "votes": 0, "replies": [] }, { "id": 409743, "author_name": "William Green", "author_url": "", "post_date": "2018-10-24T19:33:41.120000", "content": "@Julia For clarification, there will be only be a test dataset provided for stage2? Not a training set as well?
", "votes": 0, "replies": [] }, { "id": 409754, "author_name": "Julia Elliott", "author_url": "", "post_date": "2018-10-24T19:51:49.460000", "content": "@William - good question. When Stage 2 is updated, we will wipe clean the leaderboards. The new Stage 2 train set will contain a combination of the original Stage 1 training data, as well as Stage 1's test data. All of that will be provided with its labels.
\n\nThe new Stage 2 test set (with labels withheld) will contain a few rows to be used for the public leaderboard, but mostly used towards the private leaderboard. This is intended to provide competitors a tiny bit of signal as to where they are roughly scoring.
", "votes": 0, "replies": [] }, { "id": 409781, "author_name": "Ian Pan", "author_url": "", "post_date": "2018-10-24T20:28:45.680000", "content": "Does that mean only the portion of the stage 2 test set that is held out for the private leaderboard will be used to determine the final ranking?
", "votes": 0, "replies": [] }, { "id": 409786, "author_name": "Julia Elliott", "author_url": "", "post_date": "2018-10-24T20:49:18.793000", "content": "@Ian - Yes.
", "votes": 0, "replies": [] }, { "id": 409821, "author_name": "Alexandre Cadrin-Chênevert", "author_url": "", "post_date": "2018-10-24T22:40:43.937000", "content": "@Julia
\n\nYou are referring to a new Stage 2 train set that partially includes Stage 1 test data.
\n\nQ1 : If it is a train set, does it mean we can retrain/finetune our(s) model(s) on this new available train set ?
\n\nFrom the available information, we thought the uploaded model needed to be exactly the same for stage 1 and stage 2 with the same inference parameters. That was the format used in DSB 2018 which makes sense to prevent any kind of overfitting of Stage 2 test data.
\n\nSo, if we can retrain/finetune our models with Stage 2 train set, then the model(s) will not be the same as the one(s) uploaded.
\n\nQ2 : And, If we can't retrain (as we thought), then why calling the new available labels Stage 2 train set ?
", "votes": 18, "replies": [] }, { "id": 409975, "author_name": "Sergey Zlobin", "author_url": "", "post_date": "2018-10-25T06:28:27.183000", "content": "@Alexandre Cadrin-Chênevert
\n\nI agree with you. According to the logic of the competition, the Stage 1 labels do not need to be published until the competition is over.
", "votes": 3, "replies": [] }, { "id": 409982, "author_name": "TTHJ", "author_url": "", "post_date": "2018-10-25T06:42:15.673000", "content": "Agree~
", "votes": 0, "replies": [] }, { "id": 410081, "author_name": "", "author_url": "", "post_date": "2018-10-25T10:42:58.047000", "content": "", "votes": 4, "replies": [] }, { "id": 410093, "author_name": "FelipeKitamura, MD, PhD", "author_url": "", "post_date": "2018-10-25T11:57:25.620000", "content": "I have the same questions as @Alexandre Cadrin-Chênevert.
", "votes": 0, "replies": [] }, { "id": 410168, "author_name": "Andy Harless", "author_url": "", "post_date": "2018-10-25T14:52:20.457000", "content": "@statsintuition Indeed, changing the rules within less than one day of the deadline, since Julia's correction to her response above could change the appropriate strategy. Given the difficulties of the model selection and upload process, that's likely too late for most people to do anything about it, even if they got the information immediately. And even after stage 1, the rules are still unclear: what aspects of the model (nothing at all? just the fitted weights? certain kinds of hyperparameters? all of the hyperparameters?) can we change, after receiving the stage 1 test labels, without being considered in violation of the rules?
", "votes": 4, "replies": [] }, { "id": 410173, "author_name": "William Green", "author_url": "", "post_date": "2018-10-25T15:02:43.140000", "content": "@Andy We ran into the same situation with DSB 2018. I think going forward:
\n\nAnd even after stage 1, the rules are still unclear: what aspects of the model (nothing at all? just the fitted weights? certain kinds of hyperparameters? all of the hyperparameters?) \nneeds be clearly define in the competitions rules. If there are a lot of questions being asked, then that is a good indicator that the rules or portion of the rules are not clearly defined. In some case, the rules need to be really spelled out so everyone has a clear understanding.
", "votes": 2, "replies": [] }, { "id": 410264, "author_name": "Julia Elliott", "author_url": "", "post_date": "2018-10-25T17:41:59.123000", "content": "To be clear: the rules have not changed.
\n\nI realize confusion may have been caused in attempting to clarify that the stage 2 dataset will include stage 1's labels. We have resurrected a \"Two-stage competition FAQ\" page that had been inadvertently deprecated in a recent migration. These should address many of the questions about what you can and cannot do to your models during Stage 2. This is the identical design and approach taken in prior 2-stage competitions, including DSB 2018.
\n\nAs stated there: \"We expect you may need to make some \"non scientific\" alterations, such as changes to path names, in order to create your submissions for the second stage. You are allowed to re-train your model (including the stage one data), but your code should not change. You should not be doing any hyper parameter tuning in the second stage. Parameter tuning is permitted as long as it is fully automated.\"
\n\nLastly, the Stage 2 data will be available today (October 25th) no later than 11:59pm UTC, in line with all of the other stated deadlines on the Timeline page.
", "votes": 0, "replies": [] }, { "id": 410269, "author_name": "Tom Aindow", "author_url": "", "post_date": "2018-10-25T17:53:37.517000", "content": "Thanks for the update. It's a shame we didn't have access to the \"Two-stage competition FAQ\" page before submission, that makes things a lot clearer. Hopefully the requirements don't catch too many people out.
", "votes": 5, "replies": [] }, { "id": 410287, "author_name": "double", "author_url": "", "post_date": "2018-10-25T19:00:18.910000", "content": "If we are using an ensemble of models,how could we tuning? Tuning part of weight? Or we can even add a new model with the same hyper parameter? I think lock the weights we uploaded is much better.......
", "votes": 5, "replies": [] }, { "id": 410345, "author_name": "Alexandre Cadrin-Chênevert", "author_url": "", "post_date": "2018-10-25T21:53:00.497000", "content": "Thanks Julia for the answer. The ressurected FAQ is clear.
\n\nI didn`t know before model upload deadline that we could retrain for stage 2 using stage 1 test labels.
\n\nI will be aware next time.
", "votes": 2, "replies": [] }, { "id": 410613, "author_name": "Glacier_Li", "author_url": "", "post_date": "2018-10-26T10:49:51.127000", "content": "Thank you for your explanation.\nHowever, the \"Two-stage competition FAQ\" page did not supply participants at the end of the stage 1. \nIf the team only submits the training ”model“(model weight) in the stage 1 instead of the training code, can they retrain the data using the same code.
", "votes": 2, "replies": [] }, { "id": 410656, "author_name": "Ian Pan", "author_url": "", "post_date": "2018-10-26T12:16:06.697000", "content": "@Julia
\n\nThanks for the clarification. In the quote you provided, it says hyperparameter tuning is not allowed, but parameter tuning is allowed if it is fully-automated. Would fully-automated hyperparameter tuning be allowed?
", "votes": 0, "replies": [] }, { "id": 410753, "author_name": "", "author_url": "", "post_date": "2018-10-26T15:51:10.370000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 409745, "author_name": "Julia Elliott", "author_url": "", "post_date": "2018-10-24T19:36:08.800000", "content": "NOTE: There was an update made to bullet 6 in my clarification response. I'd misspoken and made a typo in that response. The Stage 1 labels WILL be provided when Stage 2 is updated. Apologies!
", "votes": -2, "replies": [ { "id": 409947, "author_name": "TTHJ", "author_url": "", "post_date": "2018-10-25T05:28:02.477000", "content": "But we have to use the same model we uploaded, and are not allowed to use the stage 1 data and label to train a new model, right?
", "votes": 1, "replies": [] }, { "id": 409963, "author_name": "Hadar", "author_url": "", "post_date": "2018-10-25T05:56:02.527000", "content": "Excuse me but I do not see what you mean. The model and code we uploaded do not include any training code, as a consequence of the previous rules that there will not be any new labels and no further training will be possible. So We cannot use the new labels without changing completely the code and model that we have uploaded. Changing the rules of the competition just after it ends is a bit weird, I have to say...
", "votes": 7, "replies": [] }, { "id": 410330, "author_name": "Sam Shleifer", "author_url": "", "post_date": "2018-10-25T21:09:05.440000", "content": "yeah and also the rules of the competition should not be changed in comments under a thread called Reminder: Model Upload deadline that doesn't trigger notifications. (Is there a way to turn on email notifications?). The name of the thread suggests that if you uploaded your model you don't need to read the thread which is roughly the opposite of the truth.
humor on
\n\nYay, our model is the darknet (aka yolov3) and the executable will not change. We will train with Stage 2 (er Stage 1) training set and test with Stage 2 test set. It would be very helpful if we could now have ground truth labels for Stage 1 test data too. ;)
\n\nOnly upside for team Yolopacity Beta! We just do garden variety transfer learning. But we are so far down every change in rule helps :)
\n\nhumor off
", "votes": 0, "replies": [] }, { "id": 410223, "author_name": "Han Liu", "author_url": "", "post_date": "2018-10-25T16:16:32.940000", "content": "@Julia - Hi Julia, our team tried to upload our models yesterday but the connection was very poor. We were uploading before the deadline but when we finished uploading, it passed the deadline by 30min... The system automatically cancelled our uploaded models.. It was our first time uploading models so we didn't expect uploading required such a long time... I know we may not win any prizes but we hope we could at least attend the stage 2...
", "votes": 0, "replies": [ { "id": 410358, "author_name": "Ian Pan", "author_url": "", "post_date": "2018-10-25T22:30:42.733000", "content": "Per the two-stage competition FAQ:
\n\n\n\n\nWill I still be on the leaderboard if I don't upload a model but do\n submit in the second stage? Yes. However, if you do not upload a model\n and finish in a prize position, your team will be removed from the\n competition standings entirely.
\n
So as long as you don't finish in a prize position, you should still be on the leaderboard.
", "votes": 0, "replies": [] } ] }, { "id": 410025, "author_name": "Kaggoo", "author_url": "", "post_date": "2018-10-25T08:28:18.860000", "content": "This is sad but it was my fault. Didn't realise that I have to select 1 submission!!! This is not like other competitions :(
", "votes": 0, "replies": [ { "id": 410046, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2018-10-25T09:16:35.927000", "content": "@Kagoo you may still be okay if you uploaded your best scoring model based on the stage-1 public LB. Because the system will automatically select your best submission for stage-1 data as your 1 final submission.
", "votes": 0, "replies": [] } ] }, { "id": 409904, "author_name": "AkiraSosa", "author_url": "", "post_date": "2018-10-25T03:11:51.390000", "content": "Hi,
\n\nStage 1 is closed. Where can I get stage 2 data?
\n\nThanks,
", "votes": 0, "replies": [ { "id": 410048, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2018-10-25T09:17:43.913000", "content": "According to the timeline, stage-2 test data will be available on October 25th.
", "votes": 0, "replies": [] } ] }, { "id": 409877, "author_name": "Taegyun Um", "author_url": "", "post_date": "2018-10-25T02:01:46.207000", "content": "Hi, Unfortunately I missed the model submission deadline. But my team want to keep going on.
\n\nIs it possible to get score from late submission with Stage 1 data??\nWe don't have to be in the leaderboard, just want to make sure that we're developing model better than before.
", "votes": 0, "replies": [ { "id": 410266, "author_name": "Julia Elliott", "author_url": "", "post_date": "2018-10-25T17:43:46.173000", "content": "You can submit without prize eligibility after the competition is finalized and closed, after October 31, 2018.
", "votes": 0, "replies": [] } ] }, { "id": 409329, "author_name": "Adithya Seshasayee", "author_url": "", "post_date": "2018-10-24T06:11:01.197000", "content": "If we are using an ensemble of models, how would we submit it?
", "votes": 0, "replies": [ { "id": 409629, "author_name": "Virilo Tejedor Aguilera", "author_url": "", "post_date": "2018-10-24T16:06:27.300000", "content": "Not official response. I think you can put the code and the weights in a zip file. Regarding the weights, instead of the files, you can include a .txt file with a hash of each file.
", "votes": 1, "replies": [] }, { "id": 409740, "author_name": "Julia Elliott", "author_url": "", "post_date": "2018-10-24T19:29:55.163000", "content": "The recommendation provided by Virilo will be adequate.
", "votes": 0, "replies": [] }, { "id": 410748, "author_name": "Hadar", "author_url": "", "post_date": "2018-10-26T15:40:43.830000", "content": "If check sums are enough then there can be no further training in stage 2, isn't it?
", "votes": 0, "replies": [] } ] }, { "id": 410071, "author_name": "", "author_url": "", "post_date": "2018-10-25T10:24:58.517000", "content": "", "votes": 0, "replies": [] }, { "id": 409917, "author_name": "", "author_url": "", "post_date": "2018-10-25T03:45:18.573000", "content": "", "votes": 0, "replies": [] }, { "id": 409901, "author_name": "", "author_url": "", "post_date": "2018-10-25T03:05:18.467000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "407146": "Can we include an MD5 checksum for our model archive so we don't need to upload all of the models we used (unless we win)? The weight files are pretty large, worried about spotty internet connections interrupting the upload...", "407505": "Will you be releasing the stage 1 labels when you release the stage 2 dataset?", "407224": "Julia, what is the reason behind the stage 1 final submission selection? As I understand the final score will be based on the second stage results only or have I missed something?", "405564": "This is your reminder that October 24th (next week) is the close of stage 1 and deadline to both make your 1 final submission selection and upload the corresponding model. There will be no exceptions to this deadline, and prize eligibility will be contingent on compliance with this requirement.\n\nUpon the close of stage 1, stage 2 data will be released, with only 6 days to submit your predictions on that test set.\n\nSee the [timeline page][1] for full details. \n\n\n[1]: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge#Timeline", "410396": "Sorry, I made a mistake in uploading file... I only upload my weights of my model, will I be permitted into stage2 :( ? ", "410257": "This is all very confusing, just adding my voice to the choir asking for some clarification please Kaggle.", "410425": "We just uploaded the model and the code for test。。。。。did I misunderstanding what is 'model' mean?", "406350": "Are there any guidelines regarding the model upload?\n\nI've found this FAQs very helpful.\nhttps://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/32580\n\nIs the model upload rule of this competition is as same as described in above FAQs?", "414072": "My score has disappeared from the leaderboard ,could you give me a reason?I upload my model ,and I didn\n't change anything in stage 2......", "410783": "From the requirement to make the stage 1 final submission selection and uploaded model to match this submission, I expected any re-training with the released stage 1 labels would violate the second requirement and so not be allowed.", "410351": "It might be clearer to call it a \"learner\"/\"learning algorithm\" upload deadline in the future.", "406781": "Hi, some questions:\n\n 1. are you guys executing each team's inference routine on the stage 2 data to match the stage 2 submission?\n 2. does this mean we are not allowed to change anything in the model (hyperparameters, arch, etc..) after the model upload, while stage 2 data is out? How do you match model to submission? how is the auto-select based on stage1 done?\n\nreference:\n\n> Be aware that if you do not select a final submission (via 'My\n> Submissions'), the platform will auto-select your best-scoring model\n> on the Stage 1 public leaderboard. The deadline for model upload is\n> firmly the end of Stage 1.\n> \n> This requirement is in place for the host team to verify the\n> performance of the uploaded models matches the Stage 2 submission\n> file. Compliance with the above will be verified by the host team.\n\n", "410052": "Can submission window for stage1 could be opened again. I missed the deadline.", "408333": "NOTE: There was an update made to bullet 6 in my original response. I'd misspoken and made a typo in that response. The Stage 1 labels WILL be provided when Stage 2 is updated. Apologies!\n\nA few clarifications on the model upload:\n\n1. Model upload can be completed by navigating to the [Team tab][1], and scrolling down to \"Your Model.\" It's not necessary that your model be created in kernels.\n2. The purpose of the model upload is to \"lock in\" the model your team selected based on the stage 1 data and to verify that you have not made changes in violation of the rules (including performing any hand-labeling or annotations to your model as a result of having seen the stage 2 dataset).\n3. As a result of #2 above, you can expect that the host team will verify you are not in violation by re-running models eligible for prize.\n4. For context, winning teams will be expected to also submit [full documentation of their solutions][2], a presentation summarizing that documentation & approach, and a short (3 min) video to share their approach.\n5. It is advised that you select your final submission before Wednesday's deadline. If you do not do so, the platform will auto-select your submission based on that which performed best on the public leaderboard. Your uploaded model will be expected to match the final submission selected & scored.\n6. The stage 1 labels *will* be released with the stage 2 dataset.\n\n\n[1]: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/team\n[2]: https://www.kaggle.com/WinningModelDocumentationGuidelines", "409745": "NOTE: There was an update made to bullet 6 in my clarification response. I'd misspoken and made a typo in that response. The Stage 1 labels WILL be provided when Stage 2 is updated. Apologies!", "410240": "humor on\n\nYay, our model is the darknet (aka yolov3) and the executable will not change. We will train with Stage 2 (er Stage 1) training set and test with Stage 2 test set. It would be very helpful if we could now have ground truth labels for Stage 1 test data too. ;)\n\nOnly upside for team Yolopacity Beta! We just do garden variety transfer learning. But we are so far down every change in rule helps :)\n\nhumor off", "410223": "@Julia - Hi Julia, our team tried to upload our models yesterday but the connection was very poor. We were uploading before the deadline but when we finished uploading, it passed the deadline by 30min... The system automatically cancelled our uploaded models.. It was our first time uploading models so we didn't expect uploading required such a long time... I know we may not win any prizes but we hope we could at least attend the stage 2...", "410025": "This is sad but it was my fault. Didn't realise that I have to select 1 submission!!! This is not like other competitions :(", "409904": "Hi,\n\nStage 1 is closed. Where can I get stage 2 data?\n\nThanks,", "409877": "Hi, Unfortunately I missed the model submission deadline. But my team want to keep going on. \n\nIs it possible to get score from late submission with Stage 1 data??\nWe don't have to be in the leaderboard, just want to make sure that we're developing model better than before.", "409329": "If we are using an ensemble of models, how would we submit it?", "410071": "", "409917": "", "409901": "" } }