How are some notebooks (such as https://www.kaggle.com/h053473666/siim-cov19-efnb7-yolov5-infer/output and https://www.kaggle.com/anima890/siim-covid-19-detection-using-yolov5/output) able to submit and get a valid score even if they have only 4 rows in the final submission.csv? I thought this would throw an error since there are more than a thousand images in the test set. What am I missing here? I also tried to submit predictions for all the 1263 images in the test set, but the notebook returned a score of 0.104
", "messages": [ { "id": "1388587", "postDate": "07/15/2021 05:10:03", "content": "How are some notebooks (such as https://www.kaggle.com/h053473666/siim-cov19-efnb7-yolov5-infer/output and https://www.kaggle.com/anima890/siim-covid-19-detection-using-yolov5/output) able to submit and get a valid score even if they have only 4 rows in the final submission.csv? I thought this would throw an error since there are more than a thousand images in the test set. What am I missing here? I also tried to submit predictions for all the 1263 images in the test set, but the notebook returned a score of 0.104
", "rawMarkdown": "How are some notebooks (such as https://www.kaggle.com/h053473666/siim-cov19-efnb7-yolov5-infer/output and https://www.kaggle.com/anima890/siim-covid-19-detection-using-yolov5/output) able to submit and get a valid score even if they have only 4 rows in the final submission.csv? I thought this would throw an error since there are more than a thousand images in the test set. What am I missing here? I also tried to submit predictions for all the 1263 images in the test set, but the notebook returned a score of 0.104", "votes": null }, { "id": "1389532", "postDate": "07/15/2021 19:39:00", "content": "There is an if statement in beginning to take just two samples if the size of the submission file is equal to the public submission csv but it doesn't get triggered when the size is different and in the submission phase the size will be much larger then the public test set (public+private) and the condition will no get trigerred.
", "rawMarkdown": "There is an if statement in beginning to take just two samples if the size of the submission file is equal to the public submission csv but it doesn't get triggered when the size is different and in the submission phase the size will be much larger then the public test set (public+private) and the condition will no get trigerred.", "votes": null }, { "id": "1389762", "postDate": "07/16/2021 03:55:32", "content": "Yes, but there are more than just 4 images in the public test set right? Dont we have to submit predictions for all of them?
", "rawMarkdown": "Yes, but there are more than just 4 images in the public test set right? Dont we have to submit predictions for all of them?", "votes": null }, { "id": "1389777", "postDate": "07/16/2021 04:40:22", "content": "In the submission process, the df has different length (df.shape[0] != 2477)
", "rawMarkdown": "In the submission process, the df has different length (df.shape[0] != 2477)", "votes": null }, { "id": "1390131", "postDate": "07/16/2021 11:31:31", "content": "Yes, the df will have different length for private test set. But, for the public test set, the df has 2477 rows. So, why does submission.csv not have 2477 rows as well? How is it possible to get a score on the public leaderboard if we only submit 4 rows?
", "rawMarkdown": "Yes, the df will have different length for private test set. But, for the public test set, the df has 2477 rows. So, why does submission.csv not have 2477 rows as well? How is it possible to get a score on the public leaderboard if we only submit 4 rows?", "votes": null }, { "id": "1390142", "postDate": "07/16/2021 11:44:51", "content": "don't confuse the save mode with the submit mode - when you press submit the sample_submission.csv gets replaced with the image ids of public + private set and gets score for both. We can only see of course the public only score for 2477 rows (public LB score) [eDITED]
in short, doesn't matter if you save your inference nb with df of 4 rows or 2477 rows, when you press submit it will be replaced anyway. (also running and saving with small df saves you gpu quota)
", "rawMarkdown": "don't confuse the `save` mode with the `submit` mode - when you press submit the sample_submission.csv gets replaced with the image ids of public + private set and gets score for both. We can only see of course the public only score for 2477 rows (public LB score) [eDITED]\n\nin short, doesn't matter if you save your inference nb with df of 4 rows or 2477 rows, when you press submit it will be replaced anyway. (also running and saving with small df saves you gpu quota)", "votes": null }, { "id": "1390256", "postDate": "07/16/2021 13:49:19", "content": "Ohhh, I understand now. I always assumed that the 4 rows were the final submission to the leaderboard. Thank you so much @imeintanis @daiquoctran @varundutt9213 😄
", "rawMarkdown": "Ohhh, I understand now. I always assumed that the 4 rows were the final submission to the leaderboard. Thank you so much @imeintanis @daiquoctran @varundutt9213 😄", "votes": null } ], "comments": [ { "id": 1389532, "author_name": "varundutt9213", "author_url": "", "post_date": "07/15/2021 19:39:00", "content": "There is an if statement in beginning to take just two samples if the size of the submission file is equal to the public submission csv but it doesn't get triggered when the size is different and in the submission phase the size will be much larger then the public test set (public+private) and the condition will no get trigerred.
", "votes": null, "replies": [ { "id": 1389762, "author_name": "novice03", "author_url": "", "post_date": "07/16/2021 03:55:32", "content": "Yes, but there are more than just 4 images in the public test set right? Dont we have to submit predictions for all of them?
", "votes": null, "replies": [] }, { "id": 1389777, "author_name": "daiquoctran", "author_url": "", "post_date": "07/16/2021 04:40:22", "content": "In the submission process, the df has different length (df.shape[0] != 2477)
", "votes": null, "replies": [] }, { "id": 1390131, "author_name": "novice03", "author_url": "", "post_date": "07/16/2021 11:31:31", "content": "Yes, the df will have different length for private test set. But, for the public test set, the df has 2477 rows. So, why does submission.csv not have 2477 rows as well? How is it possible to get a score on the public leaderboard if we only submit 4 rows?
", "votes": null, "replies": [] }, { "id": 1390142, "author_name": "imeintanis", "author_url": "", "post_date": "07/16/2021 11:44:51", "content": "don't confuse the save mode with the submit mode - when you press submit the sample_submission.csv gets replaced with the image ids of public + private set and gets score for both. We can only see of course the public only score for 2477 rows (public LB score) [eDITED]
in short, doesn't matter if you save your inference nb with df of 4 rows or 2477 rows, when you press submit it will be replaced anyway. (also running and saving with small df saves you gpu quota)
", "votes": null, "replies": [] }, { "id": 1390256, "author_name": "novice03", "author_url": "", "post_date": "07/16/2021 13:49:19", "content": "Ohhh, I understand now. I always assumed that the 4 rows were the final submission to the leaderboard. Thank you so much @imeintanis @daiquoctran @varundutt9213 😄
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1388587": "How are some notebooks (such as https://www.kaggle.com/h053473666/siim-cov19-efnb7-yolov5-infer/output and https://www.kaggle.com/anima890/siim-covid-19-detection-using-yolov5/output) able to submit and get a valid score even if they have only 4 rows in the final submission.csv? I thought this would throw an error since there are more than a thousand images in the test set. What am I missing here? I also tried to submit predictions for all the 1263 images in the test set, but the notebook returned a score of 0.104", "1389532": "There is an if statement in beginning to take just two samples if the size of the submission file is equal to the public submission csv but it doesn't get triggered when the size is different and in the submission phase the size will be much larger then the public test set (public+private) and the condition will no get trigerred.", "1389762": "Yes, but there are more than just 4 images in the public test set right? Dont we have to submit predictions for all of them?", "1389777": "In the submission process, the df has different length (df.shape[0] != 2477)", "1390131": "Yes, the df will have different length for private test set. But, for the public test set, the df has 2477 rows. So, why does submission.csv not have 2477 rows as well? How is it possible to get a score on the public leaderboard if we only submit 4 rows?", "1390142": "don't confuse the `save` mode with the `submit` mode - when you press submit the sample_submission.csv gets replaced with the image ids of public + private set and gets score for both. We can only see of course the public only score for 2477 rows (public LB score) [eDITED]\n\nin short, doesn't matter if you save your inference nb with df of 4 rows or 2477 rows, when you press submit it will be replaced anyway. (also running and saving with small df saves you gpu quota)", "1390256": "Ohhh, I understand now. I always assumed that the 4 rows were the final submission to the leaderboard. Thank you so much @imeintanis @daiquoctran @varundutt9213 😄" }, "source": "meta" }