I noticed the description on the homepage regarding the metric calculation: \"this metric takes a relatively long time to run; you may need to wait several hours for scoring to complete.\" I submitted two separate models. For one model, the inference time for a single TIFF was 8 seconds, and the total run time was 5 hours. The other model had an inference time of 64 seconds per TIFF, which is 8 times longer than the first model. However, it did not time out on the leaderboard but ran for 8 hours instead. Based on calculations, the metric computation seemed to take about 4 hours. However, during the submission run, it consistently showed \"submission running\" instead of \"scoring,\" which led to this question.
", "messages": [ { "id": 3326157, "postDate": "2025-11-15T09:02:43.897Z", "content": "I noticed the description on the homepage regarding the metric calculation: \"this metric takes a relatively long time to run; you may need to wait several hours for scoring to complete.\" I submitted two separate models. For one model, the inference time for a single TIFF was 8 seconds, and the total run time was 5 hours. The other model had an inference time of 64 seconds per TIFF, which is 8 times longer than the first model. However, it did not time out on the leaderboard but ran for 8 hours instead. Based on calculations, the metric computation seemed to take about 4 hours. However, during the submission run, it consistently showed \"submission running\" instead of \"scoring,\" which led to this question.
", "rawMarkdown": "I noticed the description on the homepage regarding the metric calculation: \"this metric takes a relatively long time to run; you may need to wait several hours for scoring to complete.\" I submitted two separate models. For one model, the inference time for a single TIFF was 8 seconds, and the total run time was 5 hours. The other model had an inference time of 64 seconds per TIFF, which is 8 times longer than the first model. However, it did not time out on the leaderboard but ran for 8 hours instead. Based on calculations, the metric computation seemed to take about 4 hours. However, during the submission run, it consistently showed \"submission running\" instead of \"scoring,\" which led to this question.", "votes": 8 }, { "id": 3328579, "postDate": "2025-11-15T21:14:40.667Z", "content": "The 9 hour limit is for your inference only. The metric runtime is not included.
", "rawMarkdown": "The 9 hour limit is for your inference only. The metric runtime is not included.", "votes": 4, "replies": [ { "id": 3330008, "postDate": "2025-11-16T07:19:54.057Z", "content": "Thank you very much for your explanation. So, if I understand correctly, if the metric calculation took 4 hours, is it possible that the \"submission running\" status could last for 13 hours in total?
", "rawMarkdown": "Thank you very much for your explanation. So, if I understand correctly, if the metric calculation took 4 hours, is it possible that the \"submission running\" status could last for 13 hours in total?", "replies": [ { "id": 3331470, "postDate": "2025-11-16T13:07:30.373Z", "content": "I think when your code is running , it shows \"Notebook Running\" , and when metric calculation is running, it shows \"Submission Running\". That's what it showed me during my submission. So you should see \"Submission Running\" for only 2-4 hours, rest would be \"Notebook Running\".
", "rawMarkdown": "I think when your code is running , it shows \"Notebook Running\" , and when metric calculation is running, it shows \"Submission Running\". That's what it showed me during my submission. So you should see \"Submission Running\" for only 2-4 hours, rest would be \"Notebook Running\".", "votes": 3 }, { "id": 3333113, "postDate": "2025-11-17T04:52:51.210Z", "content": "You're right. Thanks!
", "rawMarkdown": "You're right. Thanks!" } ] }, { "id": 3410772, "postDate": "2026-02-23T16:16:11.527Z", "content": "Just submitted 3 baselines notebooks (51s, 22min and 23 min, as per the logs) yet only the 51s didn't exceed time limits; the other two submissions didn't go through with the message 'Notebook Timeout'. How is that possible? The only thing running after the notebooks finished and created the submission.zip file was the scoring process. I don't understand. What have I done wrong?
", "rawMarkdown": "Just submitted 3 baselines notebooks (51s, 22min and 23 min, as per the logs) yet only the 51s didn't exceed time limits; the other two submissions didn't go through with the message 'Notebook Timeout'. How is that possible? The only thing running after the notebooks finished and created the submission.zip file was the scoring process. I don't understand. What have I done wrong?" } ] }, { "id": 3327173, "postDate": "2025-11-15T13:28:35.237Z", "content": "The metric calculation time can vary depending on the topology of the predictions, so what you observe does indeed make sense. @sohier may know better regarding the technical Kaggle-ish details.
", "rawMarkdown": "The metric calculation time can vary depending on the topology of the predictions, so what you observe does indeed make sense. @sohier may know better regarding the technical Kaggle-ish details." }, { "id": 3401847, "postDate": "2026-02-04T17:23:24.077Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 3328579, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-11-15T21:14:40.667000", "content": "The 9 hour limit is for your inference only. The metric runtime is not included.
", "votes": 4, "replies": [ { "id": 3330008, "author_name": "Boredom", "author_url": "", "post_date": "2025-11-16T07:19:54.057000", "content": "Thank you very much for your explanation. So, if I understand correctly, if the metric calculation took 4 hours, is it possible that the \"submission running\" status could last for 13 hours in total?
", "votes": 0, "replies": [ { "id": 3331470, "author_name": "Satwik", "author_url": "", "post_date": "2025-11-16T13:07:30.373000", "content": "I think when your code is running , it shows \"Notebook Running\" , and when metric calculation is running, it shows \"Submission Running\". That's what it showed me during my submission. So you should see \"Submission Running\" for only 2-4 hours, rest would be \"Notebook Running\".
", "votes": 3, "replies": [] }, { "id": 3333113, "author_name": "Boredom", "author_url": "", "post_date": "2025-11-17T04:52:51.210000", "content": "You're right. Thanks!
", "votes": 0, "replies": [] } ] }, { "id": 3410772, "author_name": "David Barbera", "author_url": "", "post_date": "2026-02-23T16:16:11.527000", "content": "Just submitted 3 baselines notebooks (51s, 22min and 23 min, as per the logs) yet only the 51s didn't exceed time limits; the other two submissions didn't go through with the message 'Notebook Timeout'. How is that possible? The only thing running after the notebooks finished and created the submission.zip file was the scoring process. I don't understand. What have I done wrong?
", "votes": 0, "replies": [] } ] }, { "id": 3327173, "author_name": "Giorgio Angelotti", "author_url": "", "post_date": "2025-11-15T13:28:35.237000", "content": "The metric calculation time can vary depending on the topology of the predictions, so what you observe does indeed make sense. @sohier may know better regarding the technical Kaggle-ish details.
", "votes": 0, "replies": [] }, { "id": 3401847, "author_name": "", "author_url": "", "post_date": "2026-02-04T17:23:24.077000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3326157": "I noticed the description on the homepage regarding the metric calculation: \"this metric takes a relatively long time to run; you may need to wait several hours for scoring to complete.\" I submitted two separate models. For one model, the inference time for a single TIFF was 8 seconds, and the total run time was 5 hours. The other model had an inference time of 64 seconds per TIFF, which is 8 times longer than the first model. However, it did not time out on the leaderboard but ran for 8 hours instead. Based on calculations, the metric computation seemed to take about 4 hours. However, during the submission run, it consistently showed \"submission running\" instead of \"scoring,\" which led to this question.", "3328579": "The 9 hour limit is for your inference only. The metric runtime is not included.", "3327173": "The metric calculation time can vary depending on the topology of the predictions, so what you observe does indeed make sense. @sohier may know better regarding the technical Kaggle-ish details.", "3401847": "" } }