\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1133510%2Fc429fd3f308640575a4772fa3e5226f2%2FDeepinScreenshot_select-area_20231116182339.png?generation=1700132332965551&alt=media\")
Hi Kagglers,
\nI've encountered some issues while submitting my results. I've tried various troubleshooting methods, but I'm still unable to obtain the correct outcome. Could someone please assist me with this? I would greatly appreciate it!
\n
Initially, I submitted a locally validated model that seemed to perform well, following the instructions provided by the official guidelines. However, I received a 'Submission Scoring Error' feedback. Upon browsing the Discussion section, I noticed many people discussing issues with result submission. I referred to YumeNeko's discussion and made modifications to the RLE function before resubmitting the results. However, this time I received a score of 0. I then double-checked the model's output locally, and everything appeared to be normal. Based on this, I believe I should not have received a score of 0.
\nHere are my local test results, from left to right: the original image, the ground truth, and the predicted results.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1133510%2F4a91ab8239f4e6d53603a4b0aaa346e7%2F819.jpg?generation=1700131955691668&alt=media\")
To rule out any issues with my code, I submitted YumeNeko's public notebook with a score of 0.147, but once again received the 'Submission Scoring Error' feedback. This has left me confused and unsure about where the problem lies. I noticed that there have been successful submissions on the Leaderboard within the last hour, which makes me curious about how they managed to achieve it. Could someone please assist me with this? I would greatly appreciate it!
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1133510%2Fa28d85238c45c957d2e35eedf3781d0d%2FDeepinScreenshot_select-area_20231116184917.png?generation=1700131803929036&alt=media\")
Hi Kagglers,
\nI've encountered some issues while submitting my results. I've tried various troubleshooting methods, but I'm still unable to obtain the correct outcome. Could someone please assist me with this? I would greatly appreciate it!
\n
Initially, I submitted a locally validated model that seemed to perform well, following the instructions provided by the official guidelines. However, I received a 'Submission Scoring Error' feedback. Upon browsing the Discussion section, I noticed many people discussing issues with result submission. I referred to YumeNeko's discussion and made modifications to the RLE function before resubmitting the results. However, this time I received a score of 0. I then double-checked the model's output locally, and everything appeared to be normal. Based on this, I believe I should not have received a score of 0.
\nHere are my local test results, from left to right: the original image, the ground truth, and the predicted results.
\n
To rule out any issues with my code, I submitted YumeNeko's public notebook with a score of 0.147, but once again received the 'Submission Scoring Error' feedback. This has left me confused and unsure about where the problem lies. I noticed that there have been successful submissions on the Leaderboard within the last hour, which makes me curious about how they managed to achieve it. Could someone please assist me with this? I would greatly appreciate it!
\n
Hey @chrisluu I noticed that you are now on the leaderboard. Did you figure out what was causing the 0 score on the submissions? I am currently having the same problem with a model that seems to be performing fairly well locally on test data.
\nThanks!
Yes, I have now obtained Leaderboard results that appear to be more reasonable. For me, the solution was to modify the threshold for the output results. Initially, I set the threshold very low, which appeared to be normal in my local tests. However, the resulting Leaderboard score was consistently 0. I have now increased the threshold, and I am getting more reasonable scores. I think this may be related to the metric used in the competition, so it's important to be cautious when setting the threshold.
", "rawMarkdown": "Yes, I have now obtained Leaderboard results that appear to be more reasonable. For me, the solution was to modify the threshold for the output results. Initially, I set the threshold very low, which appeared to be normal in my local tests. However, the resulting Leaderboard score was consistently 0. I have now increased the threshold, and I am getting more reasonable scores. I think this may be related to the metric used in the competition, so it's important to be cautious when setting the threshold.", "votes": 2, "replies": [ { "id": 2530435, "postDate": "2023-11-19T06:31:34.163Z", "content": "Awesome! Thank you for your reply. I have gained a lot from your recent thread! 😁
", "rawMarkdown": "Awesome! Thank you for your reply. I have gained a lot from your recent thread! 😁" } ] } ] }, { "id": 2527334, "postDate": "2023-11-16T13:02:31.370Z", "content": "I had a similar experience with a notebook that previously succeeded, then errored out with no changes. I wonder if it was related to the notebook issue in this thread -> https://www.kaggle.com/discussions/product-feedback/455659
", "rawMarkdown": "I had a similar experience with a notebook that previously succeeded, then errored out with no changes. I wonder if it was related to the notebook issue in this thread -> https://www.kaggle.com/discussions/product-feedback/455659", "replies": [ { "id": 2527894, "postDate": "2023-11-17T00:06:28.120Z", "content": "I'm not sure if this is the reason, but I have made 11 submissions since two days ago, and none of them have been successful. During this time, I noticed that some people on the Leaderboard have been updating their results, which makes me curious about how they are able to do so.
", "rawMarkdown": "I'm not sure if this is the reason, but I have made 11 submissions since two days ago, and none of them have been successful. During this time, I noticed that some people on the Leaderboard have been updating their results, which makes me curious about how they are able to do so.", "votes": 1, "replies": [ { "id": 2527920, "postDate": "2023-11-17T00:35:52.517Z", "content": "Welcome to the club. I've made nearly 30 submissions so far. One of them initially worked and resulted in a score of 0.003, but when I tried to submit it again, it failed. Currently, I'm trying out YumeNeko's notebook to see if that makes any difference.
\nIn addition to this, I've conducted various checks: ensuring my RLEs are in pairs, inspecting all my tensors for NaN and infinite values, and writing custom code to identify any missing IDs, appending them with \"1 1\" as the RLE. Despite all these efforts, I still have erros. The situation is quite perplexing.
", "rawMarkdown": "Welcome to the club. I've made nearly 30 submissions so far. One of them initially worked and resulted in a score of 0.003, but when I tried to submit it again, it failed. Currently, I'm trying out YumeNeko's notebook to see if that makes any difference.\n\nIn addition to this, I've conducted various checks: ensuring my RLEs are in pairs, inspecting all my tensors for NaN and infinite values, and writing custom code to identify any missing IDs, appending them with \"1 1\" as the RLE. Despite all these efforts, I still have erros. The situation is quite perplexing.", "votes": 1 }, { "id": 2528058, "postDate": "2023-11-17T05:09:28.643Z", "content": "The same situation happened to me. I only got 0.005 in the early simple model, and then all got 0 after using a more confident model.
", "rawMarkdown": "The same situation happened to me. I only got 0.005 in the early simple model, and then all got 0 after using a more confident model.\n" }, { "id": 2529269, "postDate": "2023-11-18T04:57:58.383Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2529270, "postDate": "2023-11-18T04:58:26.653Z", "content": "This is extremely strange. I wonder if the organizers of the competition have a reasonable explanation. I noticed that almost half of the results on the Leaderboard are 0, which means that nearly half of the participants are facing the same situation.
", "rawMarkdown": "This is extremely strange. I wonder if the organizers of the competition have a reasonable explanation. I noticed that almost half of the results on the Leaderboard are 0, which means that nearly half of the participants are facing the same situation.", "votes": 1 }, { "id": 2530237, "postDate": "2023-11-18T23:30:49.423Z", "content": "I don't know if this is a problem, but the scores no longer show up. Moreover, the notebook that previously had a score of 0 now become a Submission Scoring Error.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1947614%2F2342c9bf0bd8f8bfa1741e7a10599645%2FNB_score_log.jpg?generation=1700350242828044&alt=media\")
I've observed the same issue and initially considered it to be the problem. However, it turns out that the experience varies depending on your starting point. If you are using a notebook that already has a scored submission, you'll see the best score displayed. On the other hand, if you start with a new notebook, you'll encounter the situation you just mentioned.
", "rawMarkdown": "I've observed the same issue and initially considered it to be the problem. However, it turns out that the experience varies depending on your starting point. If you are using a notebook that already has a scored submission, you'll see the best score displayed. On the other hand, if you start with a new notebook, you'll encounter the situation you just mentioned.", "votes": 1 } ] } ] } ], "comments": [ { "id": 2530189, "author_name": "Miltiades general", "author_url": "", "post_date": "2023-11-18T21:41:47.203000", "content": "Hey @chrisluu I noticed that you are now on the leaderboard. Did you figure out what was causing the 0 score on the submissions? I am currently having the same problem with a model that seems to be performing fairly well locally on test data.
\nThanks!
Yes, I have now obtained Leaderboard results that appear to be more reasonable. For me, the solution was to modify the threshold for the output results. Initially, I set the threshold very low, which appeared to be normal in my local tests. However, the resulting Leaderboard score was consistently 0. I have now increased the threshold, and I am getting more reasonable scores. I think this may be related to the metric used in the competition, so it's important to be cautious when setting the threshold.
", "votes": 2, "replies": [ { "id": 2530435, "author_name": "Miltiades general", "author_url": "", "post_date": "2023-11-19T06:31:34.163000", "content": "Awesome! Thank you for your reply. I have gained a lot from your recent thread! 😁
", "votes": 0, "replies": [] } ] } ] }, { "id": 2527334, "author_name": "David Roberts", "author_url": "", "post_date": "2023-11-16T13:02:31.370000", "content": "I had a similar experience with a notebook that previously succeeded, then errored out with no changes. I wonder if it was related to the notebook issue in this thread -> https://www.kaggle.com/discussions/product-feedback/455659
", "votes": 0, "replies": [ { "id": 2527894, "author_name": "Jun Liu", "author_url": "", "post_date": "2023-11-17T00:06:28.120000", "content": "I'm not sure if this is the reason, but I have made 11 submissions since two days ago, and none of them have been successful. During this time, I noticed that some people on the Leaderboard have been updating their results, which makes me curious about how they are able to do so.
", "votes": 1, "replies": [ { "id": 2527920, "author_name": "Parabellum", "author_url": "", "post_date": "2023-11-17T00:35:52.517000", "content": "Welcome to the club. I've made nearly 30 submissions so far. One of them initially worked and resulted in a score of 0.003, but when I tried to submit it again, it failed. Currently, I'm trying out YumeNeko's notebook to see if that makes any difference.
\nIn addition to this, I've conducted various checks: ensuring my RLEs are in pairs, inspecting all my tensors for NaN and infinite values, and writing custom code to identify any missing IDs, appending them with \"1 1\" as the RLE. Despite all these efforts, I still have erros. The situation is quite perplexing.
", "votes": 1, "replies": [] }, { "id": 2528058, "author_name": "JamesLearnsToCode", "author_url": "", "post_date": "2023-11-17T05:09:28.643000", "content": "The same situation happened to me. I only got 0.005 in the early simple model, and then all got 0 after using a more confident model.
", "votes": 0, "replies": [] }, { "id": 2529269, "author_name": "", "author_url": "", "post_date": "2023-11-18T04:57:58.383000", "content": "", "votes": 0, "replies": [] }, { "id": 2529270, "author_name": "Jun Liu", "author_url": "", "post_date": "2023-11-18T04:58:26.653000", "content": "This is extremely strange. I wonder if the organizers of the competition have a reasonable explanation. I noticed that almost half of the results on the Leaderboard are 0, which means that nearly half of the participants are facing the same situation.
", "votes": 1, "replies": [] }, { "id": 2530237, "author_name": "JamesLearnsToCode", "author_url": "", "post_date": "2023-11-18T23:30:49.423000", "content": "I don't know if this is a problem, but the scores no longer show up. Moreover, the notebook that previously had a score of 0 now become a Submission Scoring Error.
I've observed the same issue and initially considered it to be the problem. However, it turns out that the experience varies depending on your starting point. If you are using a notebook that already has a scored submission, you'll see the best score displayed. On the other hand, if you start with a new notebook, you'll encounter the situation you just mentioned.
", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2527180": "Hi Kagglers,\n\nI've encountered some issues while submitting my results. I've tried various troubleshooting methods, but I'm still unable to obtain the correct outcome. Could someone please assist me with this? I would greatly appreciate it!\n\n\nInitially, I submitted a locally validated model that seemed to perform well, following the instructions provided by the official guidelines. However, I received a 'Submission Scoring Error' feedback. Upon browsing the Discussion section, I noticed many people discussing issues with result submission. I referred to YumeNeko's discussion and made modifications to the RLE function before resubmitting the results. However, this time I received a score of 0. I then double-checked the model's output locally, and everything appeared to be normal. Based on this, I believe I should not have received a score of 0.\n\nHere are my local test results, from left to right: the original image, the ground truth, and the predicted results.\n\n\nRef: [YumeNeko's discussion](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/455001#2522801)\n\nTo rule out any issues with my code, I submitted YumeNeko's public notebook with a score of 0.147, but once again received the 'Submission Scoring Error' feedback. This has left me confused and unsure about where the problem lies. I noticed that there have been successful submissions on the Leaderboard within the last hour, which makes me curious about how they managed to achieve it. Could someone please assist me with this? I would greatly appreciate it!\n", "2530189": "Hey @chrisluu I noticed that you are now on the leaderboard. Did you figure out what was causing the 0 score on the submissions? I am currently having the same problem with a model that seems to be performing fairly well locally on test data.\nThanks! ", "2527334": "I had a similar experience with a notebook that previously succeeded, then errored out with no changes. I wonder if it was related to the notebook issue in this thread -> https://www.kaggle.com/discussions/product-feedback/455659" } }