Whatever I do in this competition, I get the same score 0.177 (except once, no idea what happened). I now made a minimal notebook which just assigns the label \"healthy\" to all test images. I still get the same score, while other people seem to get 0.251 in this case. 0.177 is supposed to be the score of the sample_submission.csv file, which contains just 3 entries. So I think what is happening in my case is that somehow the file generated during the commit, with only 3 entries, is used in the competition. Well actually, I have no idea what I'm doing wrong, this is my first code competition. Can someone more experienced check my notebook and tell me why I get 0.177 rather than 0.251? It must be something trivial but I just can't see it.
\nNotebook link: https://www.kaggle.com/ulrichebling/what-is-wrong-with-my-submission-file
", "messages": [ { "id": "1304874", "postDate": "05/13/2021 00:44:25", "content": "Whatever I do in this competition, I get the same score 0.177 (except once, no idea what happened). I now made a minimal notebook which just assigns the label \"healthy\" to all test images. I still get the same score, while other people seem to get 0.251 in this case. 0.177 is supposed to be the score of the sample_submission.csv file, which contains just 3 entries. So I think what is happening in my case is that somehow the file generated during the commit, with only 3 entries, is used in the competition. Well actually, I have no idea what I'm doing wrong, this is my first code competition. Can someone more experienced check my notebook and tell me why I get 0.177 rather than 0.251? It must be something trivial but I just can't see it.
\nNotebook link: https://www.kaggle.com/ulrichebling/what-is-wrong-with-my-submission-file
", "rawMarkdown": "Whatever I do in this competition, I get the same score 0.177 (except once, no idea what happened). I now made a minimal notebook which just assigns the label \"healthy\" to all test images. I still get the same score, while other people seem to get 0.251 in this case. 0.177 is supposed to be the score of the sample_submission.csv file, which contains just 3 entries. So I think what is happening in my case is that somehow the file generated during the commit, with only 3 entries, is used in the competition. Well actually, I have no idea what I'm doing wrong, this is my first code competition. Can someone more experienced check my notebook and tell me why I get 0.177 rather than 0.251? It must be something trivial but I just can't see it.\n\nNotebook link: [https://www.kaggle.com/ulrichebling/what-is-wrong-with-my-submission-file](https://www.kaggle.com/ulrichebling/what-is-wrong-with-my-submission-file)", "votes": null }, { "id": "1305105", "postDate": "05/13/2021 05:17:44", "content": "I am also a rookie, but it all seems alright for me. I changed test_folder with train_folder (just for sanity check) and the .csv file had all the entries.
", "rawMarkdown": "I am also a rookie, but it all seems alright for me. I changed test_folder with train_folder (just for sanity check) and the .csv file had all the entries.", "votes": null }, { "id": "1305116", "postDate": "05/13/2021 05:23:52", "content": "Just for the sake of curiosity, I submitted your notebook and also got graded with 0.177. However, when I run my own submission pipeline (the exact same I used for my 0.798 score) I also got graded with 0.177, so I guess your code is indeed correct:
\nfrom fastai.vision.all import *\nfiles = get_image_files('../input/plant-pathology-2021-fgvc8/test_images')\nsubmission = pd.DataFrame({\n 'image': [f.name for f in files],\n 'labels': 'healthy'\n})\nsubmission.to_csv('submission.csv', index=False)\nThanks for sacrificing one of your 5 submissions for my notebook. What confuses me is that there are people who were probing the hidden test set by submitting single labels and they got different scores. Check e.g. here where they have 0.251 for an all \"healthy\" prediction.
", "rawMarkdown": "Thanks for sacrificing one of your 5 submissions for my notebook. What confuses me is that there are people who were probing the hidden test set by submitting single labels and they got different scores. Check e.g. [here](https://www.kaggle.com/c/plant-pathology-2021-fgvc8/discussion/232200) where they have 0.251 for an all \"healthy\" prediction.", "votes": null }, { "id": "1305171", "postDate": "05/13/2021 06:08:23", "content": "Glad to help you, well I know that they changed the hidden test set a few weeks/months ago due to having the same stuff of the 2020 competition or something. Maybe this was before that change?
\nLet me know if you find that out. Cheers
Right, that might be the case. I guess the best way for me to find out is to submit all \"scab\" or some other label and see if I get a different score.
", "rawMarkdown": "Right, that might be the case. I guess the best way for me to find out is to submit all \"scab\" or some other label and see if I get a different score.", "votes": null }, { "id": "1306627", "postDate": "05/14/2021 00:07:49", "content": "My daily submissions just restocked. I submitted 100% \"scab\" and got a different score (0.076) so it looks like the submission file is not the problem and the values posted by others were obtained before the test data was changed. Apparently when I was submitting a real notebook my results just happened to be all \"healthy\", probably due to some bug in there.
", "rawMarkdown": "My daily submissions just restocked. I submitted 100% \"scab\" and got a different score (0.076) so it looks like the submission file is not the problem and the values posted by others were obtained before the test data was changed. Apparently when I was submitting a real notebook my results just happened to be all \"healthy\", probably due to some bug in there.", "votes": null }, { "id": "1307699", "postDate": "05/14/2021 16:00:51", "content": "Tnx for the info, appreciate it
", "rawMarkdown": "Tnx for the info, appreciate it", "votes": null } ], "comments": [ { "id": 1305105, "author_name": "coldfir3", "author_url": "", "post_date": "05/13/2021 05:17:44", "content": "I am also a rookie, but it all seems alright for me. I changed test_folder with train_folder (just for sanity check) and the .csv file had all the entries.
", "votes": null, "replies": [] }, { "id": 1305116, "author_name": "coldfir3", "author_url": "", "post_date": "05/13/2021 05:23:52", "content": "Just for the sake of curiosity, I submitted your notebook and also got graded with 0.177. However, when I run my own submission pipeline (the exact same I used for my 0.798 score) I also got graded with 0.177, so I guess your code is indeed correct:
\nfrom fastai.vision.all import *\nfiles = get_image_files('../input/plant-pathology-2021-fgvc8/test_images')\nsubmission = pd.DataFrame({\n 'image': [f.name for f in files],\n 'labels': 'healthy'\n})\nsubmission.to_csv('submission.csv', index=False)\nThanks for sacrificing one of your 5 submissions for my notebook. What confuses me is that there are people who were probing the hidden test set by submitting single labels and they got different scores. Check e.g. here where they have 0.251 for an all \"healthy\" prediction.
", "votes": null, "replies": [] }, { "id": 1305171, "author_name": "coldfir3", "author_url": "", "post_date": "05/13/2021 06:08:23", "content": "Glad to help you, well I know that they changed the hidden test set a few weeks/months ago due to having the same stuff of the 2020 competition or something. Maybe this was before that change?
\nLet me know if you find that out. Cheers
Right, that might be the case. I guess the best way for me to find out is to submit all \"scab\" or some other label and see if I get a different score.
", "votes": null, "replies": [] }, { "id": 1306627, "author_name": "ulrichebling", "author_url": "", "post_date": "05/14/2021 00:07:49", "content": "My daily submissions just restocked. I submitted 100% \"scab\" and got a different score (0.076) so it looks like the submission file is not the problem and the values posted by others were obtained before the test data was changed. Apparently when I was submitting a real notebook my results just happened to be all \"healthy\", probably due to some bug in there.
", "votes": null, "replies": [] }, { "id": 1307699, "author_name": "coldfir3", "author_url": "", "post_date": "05/14/2021 16:00:51", "content": "Tnx for the info, appreciate it
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1304874": "Whatever I do in this competition, I get the same score 0.177 (except once, no idea what happened). I now made a minimal notebook which just assigns the label \"healthy\" to all test images. I still get the same score, while other people seem to get 0.251 in this case. 0.177 is supposed to be the score of the sample_submission.csv file, which contains just 3 entries. So I think what is happening in my case is that somehow the file generated during the commit, with only 3 entries, is used in the competition. Well actually, I have no idea what I'm doing wrong, this is my first code competition. Can someone more experienced check my notebook and tell me why I get 0.177 rather than 0.251? It must be something trivial but I just can't see it.\n\nNotebook link: [https://www.kaggle.com/ulrichebling/what-is-wrong-with-my-submission-file](https://www.kaggle.com/ulrichebling/what-is-wrong-with-my-submission-file)", "1305105": "I am also a rookie, but it all seems alright for me. I changed test_folder with train_folder (just for sanity check) and the .csv file had all the entries.", "1305116": "Just for the sake of curiosity, I submitted your notebook and also got graded with 0.177. However, when I run my own submission pipeline (the exact same I used for my 0.798 score) I also got graded with 0.177, so I guess your code is indeed correct: \n\n```\nfrom fastai.vision.all import *\nfiles = get_image_files('../input/plant-pathology-2021-fgvc8/test_images')\nsubmission = pd.DataFrame({\n 'image': [f.name for f in files],\n 'labels': 'healthy'\n})\nsubmission.to_csv('submission.csv', index=False)\n```", "1305130": "Thanks for sacrificing one of your 5 submissions for my notebook. What confuses me is that there are people who were probing the hidden test set by submitting single labels and they got different scores. Check e.g. [here](https://www.kaggle.com/c/plant-pathology-2021-fgvc8/discussion/232200) where they have 0.251 for an all \"healthy\" prediction.", "1305171": "Glad to help you, well I know that they changed the hidden test set a few weeks/months ago due to having the same stuff of the 2020 competition or something. Maybe this was before that change?\nLet me know if you find that out. Cheers", "1305180": "Right, that might be the case. I guess the best way for me to find out is to submit all \"scab\" or some other label and see if I get a different score.", "1306627": "My daily submissions just restocked. I submitted 100% \"scab\" and got a different score (0.076) so it looks like the submission file is not the problem and the values posted by others were obtained before the test data was changed. Apparently when I was submitting a real notebook my results just happened to be all \"healthy\", probably due to some bug in there.", "1307699": "Tnx for the info, appreciate it" }, "source": "meta" }