{ "id": 672397, "title": "Debugging submission errors", "url": "/competitions/vesuvius-challenge-surface-detection/discussion/672397", "author_name": "Will Stevens", "post_date": "2026-02-08T02:16:02.135000", "votes": -3, "comment_count": 5, "views": 0, "content": "

I'm new to kaggle, when I submit it fails at the scoring stage. I'm unable to distinguish between the following possibilities:

\n\n

I think I read somewhere that when you submit you don't get to see the log file (to prevent getting information about the test data) - is that true? If so then how do people debug submissions?

\n

Is there a template 'do nothing' submission that just produces correctly formatted masks in submissions.zip and gets through scoring, from which I could incrementally add back in my code?

", "messages": [ { "id": 3403243, "postDate": "2026-02-08T02:37:13.953Z", "content": "

In a code competition, scoring almost never fails because your predictions are “bad.” If your submission file is valid, Kaggle will compute a score even if you submit random noise or all-zeros (it’ll just score poorly). Scoring failures are usually runtime errors / timeouts / OOM or an invalid/missing submission artifact.

\n

1) Submission format for this competition

\n

For Vesuvius Challenge – Surface Detection, the required artifact is:

\n

a .zip file

\n

containing one .tif mask per test image

\n

each mask named [image_id].tif

\n

with the same dimensions as the source test volume and the same dtype as the train mask

\n

If any file is missing, misnamed, wrong shape, or wrong dtype, you can get a scoring error.

\n

2) “Submit” vs “Submit Prediction”

\n

You’re doing the right thing: for code competitions, you submit a Notebook and Kaggle reruns it for scoring .“Submit” vs “Submit Prediction” are same thing

\n

3) Logs / debugging

\n

Kaggle intentionally provides limited debugging detail for code competitions to prevent probing the hidden evaluation setup.\nPractically, people debug by:

\n

Commit first (not Submit) and confirm the notebook runs end-to-end and produces the expected output artifact.

\n

Explicitly print and assert:

\n

the output zip path exists

\n

zip contents (names/count)

\n

each tif’s shape + dtype

\n

Then Submit .

", "rawMarkdown": "In a code competition, scoring almost never fails because your predictions are “bad.” If your submission file is valid, Kaggle will compute a score even if you submit random noise or all-zeros (it’ll just score poorly). Scoring failures are usually runtime errors / timeouts / OOM or an invalid/missing submission artifact.\n\n1) Submission format for this competition\n\nFor Vesuvius Challenge – Surface Detection, the required artifact is:\n\na .zip file\n\ncontaining one .tif mask per test image\n\neach mask named [image_id].tif\n\nwith the same dimensions as the source test volume and the same dtype as the train mask\n\nIf any file is missing, misnamed, wrong shape, or wrong dtype, you can get a scoring error.\n\n2) “Submit” vs “Submit Prediction”\n\nYou’re doing the right thing: for code competitions, you submit a Notebook and Kaggle reruns it for scoring .“Submit” vs “Submit Prediction” are same thing\n\n3) Logs / debugging\n\nKaggle intentionally provides limited debugging detail for code competitions to prevent probing the hidden evaluation setup.\nPractically, people debug by:\n\nCommit first (not Submit) and confirm the notebook runs end-to-end and produces the expected output artifact.\n\nExplicitly print and assert:\n\nthe output zip path exists\n\nzip contents (names/count)\n\neach tif’s shape + dtype\n\nThen Submit .", "votes": 3, "replies": [ { "id": 3403314, "postDate": "2026-02-08T08:17:07.540Z", "content": "

Thanks, that has helped be identify one problem (file names within zip were wrong). I’ve just done another submission attempt and I’ll see how it goes. Another thing that puzzles me is that the submission runs in only 3 mins. I don’t know how many test tifs there are, but that seems quick. Maybe that’s okay if there are not many test tifs. Also when I click on “Submit”, in the active events list in Kaggle it says that it’s both running the submission and scoring at the same time - two different events. Maybe that’s normal, but it makes me wonder maybe it’s picking up the dummy submission.zip with only one tif in - I’m hoping that’s impossible because my understanding is that it effectively gets a fresh environment when it runs a new version, with all previous output deleted.

", "rawMarkdown": "Thanks, that has helped be identify one problem (file names within zip were wrong). I’ve just done another submission attempt and I’ll see how it goes. Another thing that puzzles me is that the submission runs in only 3 mins. I don’t know how many test tifs there are, but that seems quick. Maybe that’s okay if there are not many test tifs. Also when I click on “Submit”, in the active events list in Kaggle it says that it’s both running the submission and scoring at the same time - two different events. Maybe that’s normal, but it makes me wonder maybe it’s picking up the dummy submission.zip with only one tif in - I’m hoping that’s impossible because my understanding is that it effectively gets a fresh environment when it runs a new version, with all previous output deleted.", "replies": [ { "id": 3403344, "postDate": "2026-02-08T09:57:37.350Z", "content": "

Hi Will, under the \"data\" tab, competition hosts say to expect roughly around 120 volumes, so you can think about the size vs time, however, my experience is that our run time on interactive notebook versus test set is quite different. In another chat I mentioned about 5 hours for a run to complete, while my code currently gets one tiff done in a 1-2 minutes window which includes imports and model probes, etc which wont happen in test loop.

", "rawMarkdown": "Hi Will, under the \"data\" tab, competition hosts say to expect roughly around 120 volumes, so you can think about the size vs time, however, my experience is that our run time on interactive notebook versus test set is quite different. In another chat I mentioned about 5 hours for a run to complete, while my code currently gets one tiff done in a 1-2 minutes window which includes imports and model probes, etc which wont happen in test loop. ", "votes": 1 } ] } ] }, { "id": 3403236, "postDate": "2026-02-08T02:16:02.137Z", "content": "

I'm new to kaggle, when I submit it fails at the scoring stage. I'm unable to distinguish between the following possibilities:

\n\n

I think I read somewhere that when you submit you don't get to see the log file (to prevent getting information about the test data) - is that true? If so then how do people debug submissions?

\n

Is there a template 'do nothing' submission that just produces correctly formatted masks in submissions.zip and gets through scoring, from which I could incrementally add back in my code?

", "rawMarkdown": "I'm new to kaggle, when I submit it fails at the scoring stage. I'm unable to distinguish between the following possibilities:\n\n- Perhaps scoring is failing because the prediction is poor (I don't know whether scoring will always give a number, even if fed random noise masks as input). \n- Perhaps I have somehow misunderstood how to submit (I am using the 'Submit' button, not the 'Submit Prediction' button) and it's not submitting and running the notebook properly. It seems to me that the 'Code Cometition FAQ' linked to from the Overview tab is just a generic description that doesn't apply to this competition.\n\nI think I read somewhere that when you submit you don't get to see the log file (to prevent getting information about the test data) - is that true? If so then how do people debug submissions?\n\nIs there a template 'do nothing' submission that just produces correctly formatted masks in submissions.zip and gets through scoring, from which I could incrementally add back in my code? ", "votes": -3 }, { "id": 3403244, "postDate": "2026-02-08T02:38:16.453Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3403457, "postDate": "2026-02-08T14:26:21.187Z", "content": "

Works now, thanks all.

", "rawMarkdown": "Works now, thanks all." } ], "comments": [ { "id": 3403243, "author_name": "Tony Li", "author_url": "", "post_date": "2026-02-08T02:37:13.953000", "content": "

In a code competition, scoring almost never fails because your predictions are “bad.” If your submission file is valid, Kaggle will compute a score even if you submit random noise or all-zeros (it’ll just score poorly). Scoring failures are usually runtime errors / timeouts / OOM or an invalid/missing submission artifact.

\n

1) Submission format for this competition

\n

For Vesuvius Challenge – Surface Detection, the required artifact is:

\n

a .zip file

\n

containing one .tif mask per test image

\n

each mask named [image_id].tif

\n

with the same dimensions as the source test volume and the same dtype as the train mask

\n

If any file is missing, misnamed, wrong shape, or wrong dtype, you can get a scoring error.

\n

2) “Submit” vs “Submit Prediction”

\n

You’re doing the right thing: for code competitions, you submit a Notebook and Kaggle reruns it for scoring .“Submit” vs “Submit Prediction” are same thing

\n

3) Logs / debugging

\n

Kaggle intentionally provides limited debugging detail for code competitions to prevent probing the hidden evaluation setup.\nPractically, people debug by:

\n

Commit first (not Submit) and confirm the notebook runs end-to-end and produces the expected output artifact.

\n

Explicitly print and assert:

\n

the output zip path exists

\n

zip contents (names/count)

\n

each tif’s shape + dtype

\n

Then Submit .

", "votes": 3, "replies": [ { "id": 3403314, "author_name": "Will Stevens", "author_url": "", "post_date": "2026-02-08T08:17:07.540000", "content": "

Thanks, that has helped be identify one problem (file names within zip were wrong). I’ve just done another submission attempt and I’ll see how it goes. Another thing that puzzles me is that the submission runs in only 3 mins. I don’t know how many test tifs there are, but that seems quick. Maybe that’s okay if there are not many test tifs. Also when I click on “Submit”, in the active events list in Kaggle it says that it’s both running the submission and scoring at the same time - two different events. Maybe that’s normal, but it makes me wonder maybe it’s picking up the dummy submission.zip with only one tif in - I’m hoping that’s impossible because my understanding is that it effectively gets a fresh environment when it runs a new version, with all previous output deleted.

", "votes": 0, "replies": [ { "id": 3403344, "author_name": "OpPrime", "author_url": "", "post_date": "2026-02-08T09:57:37.350000", "content": "

Hi Will, under the \"data\" tab, competition hosts say to expect roughly around 120 volumes, so you can think about the size vs time, however, my experience is that our run time on interactive notebook versus test set is quite different. In another chat I mentioned about 5 hours for a run to complete, while my code currently gets one tiff done in a 1-2 minutes window which includes imports and model probes, etc which wont happen in test loop.

", "votes": 1, "replies": [] } ] } ] }, { "id": 3403244, "author_name": "", "author_url": "", "post_date": "2026-02-08T02:38:16.453000", "content": "", "votes": 0, "replies": [] }, { "id": 3403457, "author_name": "Will Stevens", "author_url": "", "post_date": "2026-02-08T14:26:21.187000", "content": "

Works now, thanks all.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3403243": "In a code competition, scoring almost never fails because your predictions are “bad.” If your submission file is valid, Kaggle will compute a score even if you submit random noise or all-zeros (it’ll just score poorly). Scoring failures are usually runtime errors / timeouts / OOM or an invalid/missing submission artifact.\n\n1) Submission format for this competition\n\nFor Vesuvius Challenge – Surface Detection, the required artifact is:\n\na .zip file\n\ncontaining one .tif mask per test image\n\neach mask named [image_id].tif\n\nwith the same dimensions as the source test volume and the same dtype as the train mask\n\nIf any file is missing, misnamed, wrong shape, or wrong dtype, you can get a scoring error.\n\n2) “Submit” vs “Submit Prediction”\n\nYou’re doing the right thing: for code competitions, you submit a Notebook and Kaggle reruns it for scoring .“Submit” vs “Submit Prediction” are same thing\n\n3) Logs / debugging\n\nKaggle intentionally provides limited debugging detail for code competitions to prevent probing the hidden evaluation setup.\nPractically, people debug by:\n\nCommit first (not Submit) and confirm the notebook runs end-to-end and produces the expected output artifact.\n\nExplicitly print and assert:\n\nthe output zip path exists\n\nzip contents (names/count)\n\neach tif’s shape + dtype\n\nThen Submit .", "3403236": "I'm new to kaggle, when I submit it fails at the scoring stage. I'm unable to distinguish between the following possibilities:\n\n- Perhaps scoring is failing because the prediction is poor (I don't know whether scoring will always give a number, even if fed random noise masks as input). \n- Perhaps I have somehow misunderstood how to submit (I am using the 'Submit' button, not the 'Submit Prediction' button) and it's not submitting and running the notebook properly. It seems to me that the 'Code Cometition FAQ' linked to from the Overview tab is just a generic description that doesn't apply to this competition.\n\nI think I read somewhere that when you submit you don't get to see the log file (to prevent getting information about the test data) - is that true? If so then how do people debug submissions?\n\nIs there a template 'do nothing' submission that just produces correctly formatted masks in submissions.zip and gets through scoring, from which I could incrementally add back in my code? ", "3403244": "", "3403457": "Works now, thanks all." } }