Hi, I am new to Kaggle, and have a quick question - the notebook runtime is limited to 12 hours, however the final hidden test will have more samples than the current test. Does this mean that potentially our notebook may fail on the final test? e.g. it may take 6 hrs to run on public test, but 13 hrs on hidden test.
", "messages": [ { "id": "3109911", "postDate": "01/29/2025 12:35:39", "content": "Hi, I am new to Kaggle, and have a quick question - the notebook runtime is limited to 12 hours, however the final hidden test will have more samples than the current test. Does this mean that potentially our notebook may fail on the final test? e.g. it may take 6 hrs to run on public test, but 13 hrs on hidden test.
", "rawMarkdown": "Hi, I am new to Kaggle, and have a quick question - the notebook runtime is limited to 12 hours, however the final hidden test will have more samples than the current test. Does this mean that potentially our notebook may fail on the final test? e.g. it may take 6 hrs to run on public test, but 13 hrs on hidden test.", "votes": null }, { "id": "3110189", "postDate": "01/29/2025 18:37:31", "content": "If you run a submission, the scoring time (how long until you see your public score) is how long your model took on all 500 samples. You just see your public score at the end.
", "rawMarkdown": "If you run a submission, the scoring time (how long until you see your public score) is how long your model took on all 500 samples. You just see your public score at the end.", "votes": null }, { "id": "3110254", "postDate": "01/29/2025 19:46:33", "content": "Correct, it needs to run your notebook from start to finish in under 12 hours using the ~500 sample test set that is hidden. To estimate the total runtime just take how long it takes to run 3 samples and multiply it out.
", "rawMarkdown": "Correct, it needs to run your notebook from start to finish in under 12 hours using the ~500 sample test set that is hidden. To estimate the total runtime just take how long it takes to run 3 samples and multiply it out.", "votes": null }, { "id": "3110311", "postDate": "01/29/2025 21:12:00", "content": "Got it, thank you.
", "rawMarkdown": "Got it, thank you.", "votes": null }, { "id": "3110369", "postDate": "01/29/2025 22:39:42", "content": "@jerry34 Note, that all 500 samples are run every time, but they only report on a subset for now. If your submission succeeds now it has already succeeded for the final score.
", "rawMarkdown": "jerry34 Note, that all 500 samples are run every time, but they only report on a subset for now. If your submission succeeds now it has already succeeded for the final score.", "votes": null }, { "id": "3110964", "postDate": "01/30/2025 17:34:49", "content": "@davidlist ohhh that makes a lot of sense, thank you!
", "rawMarkdown": "davidlist ohhh that makes a lot of sense, thank you!", "votes": null } ], "comments": [ { "id": 3110189, "author_name": "chemdatafarmer", "author_url": "", "post_date": "01/29/2025 18:37:31", "content": "If you run a submission, the scoring time (how long until you see your public score) is how long your model took on all 500 samples. You just see your public score at the end.
", "votes": null, "replies": [] }, { "id": 3110254, "author_name": "connorjd", "author_url": "", "post_date": "01/29/2025 19:46:33", "content": "Correct, it needs to run your notebook from start to finish in under 12 hours using the ~500 sample test set that is hidden. To estimate the total runtime just take how long it takes to run 3 samples and multiply it out.
", "votes": null, "replies": [ { "id": 3110311, "author_name": "jerry34", "author_url": "", "post_date": "01/29/2025 21:12:00", "content": "Got it, thank you.
", "votes": null, "replies": [ { "id": 3110369, "author_name": "davidlist", "author_url": "", "post_date": "01/29/2025 22:39:42", "content": "@jerry34 Note, that all 500 samples are run every time, but they only report on a subset for now. If your submission succeeds now it has already succeeded for the final score.
", "votes": null, "replies": [ { "id": 3110964, "author_name": "jerry34", "author_url": "", "post_date": "01/30/2025 17:34:49", "content": "@davidlist ohhh that makes a lot of sense, thank you!
", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "3109911": "Hi, I am new to Kaggle, and have a quick question - the notebook runtime is limited to 12 hours, however the final hidden test will have more samples than the current test. Does this mean that potentially our notebook may fail on the final test? e.g. it may take 6 hrs to run on public test, but 13 hrs on hidden test.", "3110189": "If you run a submission, the scoring time (how long until you see your public score) is how long your model took on all 500 samples. You just see your public score at the end.", "3110254": "Correct, it needs to run your notebook from start to finish in under 12 hours using the ~500 sample test set that is hidden. To estimate the total runtime just take how long it takes to run 3 samples and multiply it out.", "3110311": "Got it, thank you.", "3110369": "jerry34 Note, that all 500 samples are run every time, but they only report on a subset for now. If your submission succeeds now it has already succeeded for the final score.", "3110964": "davidlist ohhh that makes a lot of sense, thank you!" }, "source": "meta" }