![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7891016%2F28ec9cff605b59fe5b5951f8573d5b23%2F1f2293dea277cd4778863e620bcb627.png?generation=1747411648356536&alt=media\")
On Kaggle, competitions evaluate submissions using a randomly sampled subset of the leaderboard test data. This means that even if you submit the same submission.csv, your score can vary due to changes in the evaluation slice.
\nSeeding helps ensure consistent training outputs, such as model weights, prediction behavior, etc. However, you cannot control or seed Kaggle's evaluation sampling, so unfortunately, small fluctuations in score are common even when utilizing a fixed seed.
", "rawMarkdown": "On Kaggle, competitions evaluate submissions using a randomly sampled subset of the leaderboard test data. This means that even if you submit the same submission.csv, your score can vary due to changes in the evaluation slice.\n\nSeeding helps ensure consistent training outputs, such as model weights, prediction behavior, etc. However, you cannot control or seed Kaggle's evaluation sampling, so unfortunately, small fluctuations in score are common even when utilizing a fixed seed.", "votes": null }, { "id": "3203538", "postDate": "05/16/2025 22:24:50", "content": "please share submission file?
", "rawMarkdown": "please share submission file?", "votes": null }, { "id": "3203641", "postDate": "05/17/2025 04:23:08", "content": "\n", "rawMarkdown": "> please share submission file?\nThank you very much for your reply! I am running the protenix code on the list and get submission. Although the problem has been solved by reply below, thank you again for your concern!", "votes": null }, { "id": "3203642", "postDate": "05/17/2025 04:24:03", "content": "please share submission file?
\n
\n Thank you very much for your reply! I am running the protenix code on the list and get submission. Although the problem has been solved by reply below, thank you again for your concern!
\n", "rawMarkdown": "> On Kaggle, competitions evaluate submissions using a randomly sampled subset of the leaderboard test data. This means that even if you submit the same submission.csv, your score can vary due to changes in the evaluation slice.\n> \n> Seeding helps ensure consistent training outputs, such as model weights, prediction behavior, etc. However, you cannot control or seed Kaggle's evaluation sampling, so unfortunately, small fluctuations in score are common even when utilizing a fixed seed.\nThank you very much for your patient and thoughtful response, which really helped a lot. Thank you again!", "votes": null }, { "id": "3207687", "postDate": "05/23/2025 06:07:10", "content": "On Kaggle, competitions evaluate submissions using a randomly sampled subset of the leaderboard test data. This means that even if you submit the same submission.csv, your score can vary due to changes in the evaluation slice.
\nSeeding helps ensure consistent training outputs, such as model weights, prediction behavior, etc. However, you cannot control or seed Kaggle's evaluation sampling, so unfortunately, small fluctuations in score are common even when utilizing a fixed seed.
\n
\n Thank you very much for your patient and thoughtful response, which really helped a lot. Thank you again!
I met the same issues before, didn't figure out yet.
", "rawMarkdown": "I met the same issues before, didn't figure out yet.", "votes": null }, { "id": "3207960", "postDate": "05/23/2025 13:27:58", "content": "That is not true. The submissions are always evaluated on the same test set, otherwise, the public LB would be meaningless. If you submit a deterministic submission you will always get the same score. The difference in scores is due to randomness in the seeds of different libraries. You must seed all the underlying modules to guarantee reproducibility. You can check that many competitors use a seed function that seeds many underlying libraries such as numpy, torch, random, etc.
In any case, there is no point in seeding submissions. The best approach is to submit the exact same code unseeded as many times as you can to check the mean and variance of the model. Then submit the model with the highest mean and lowest variance.
", "rawMarkdown": "That is not true. The submissions are always evaluated on the same test set, otherwise, the public LB would be meaningless. If you submit a deterministic submission you will always get the same score. The difference in scores is due to randomness in the seeds of different libraries. You must seed all the underlying modules to guarantee reproducibility. You can check that many competitors use a seed function that seeds many underlying libraries such as `numpy, torch, random`, etc.\n\nIn any case, there is no point in seeding submissions. The best approach is to submit the exact same code unseeded as many times as you can to check the mean and variance of the model. Then submit the model with the highest mean and lowest variance.", "votes": null }, { "id": "3208369", "postDate": "05/24/2025 02:15:41", "content": "\n\nThat is not true. The submissions are always evaluated on the same test set, otherwise, the public LB would be meaningless. If you submit a deterministic submission you will always get the same score. The difference in scores is due to randomness in the seeds of different libraries. You must seed all the underlying modules to guarantee reproducibility. You can check that many competitors use a seed function that seeds many underlying libraries such as
\nnumpy, torch, random, etc.In any case, there is no point in seeding submissions. The best approach is to submit the exact same code unseeded as many times as you can to check the mean and variance of the model. Then submit the model with the highest mean and lowest variance.
\n
Thank you very much for your reply, it is really helpful!!
", "rawMarkdown": "> That is not true. The submissions are always evaluated on the same test set, otherwise, the public LB would be meaningless. If you submit a deterministic submission you will always get the same score. The difference in scores is due to randomness in the seeds of different libraries. You must seed all the underlying modules to guarantee reproducibility. You can check that many competitors use a seed function that seeds many underlying libraries such as `numpy, torch, random`, etc.\n> \n> In any case, there is no point in seeding submissions. The best approach is to submit the exact same code unseeded as many times as you can to check the mean and variance of the model. Then submit the model with the highest mean and lowest variance.\n\nThank you very much for your reply, it is really helpful!!", "votes": null }, { "id": "3210603", "postDate": "05/27/2025 12:42:45", "content": "I am experiencing a recurring problem where the submission process fails after four minutes. I tried to find out the reason but I couldn’t. Can you provide me with a solution to this problem?
", "rawMarkdown": "I am experiencing a recurring problem where the submission process fails after four minutes. I tried to find out the reason but I couldn’t. Can you provide me with a solution to this problem?", "votes": null } ], "comments": [ { "id": 3203419, "author_name": "trevorhorner", "author_url": "", "post_date": "05/16/2025 17:56:56", "content": "On Kaggle, competitions evaluate submissions using a randomly sampled subset of the leaderboard test data. This means that even if you submit the same submission.csv, your score can vary due to changes in the evaluation slice.
\nSeeding helps ensure consistent training outputs, such as model weights, prediction behavior, etc. However, you cannot control or seed Kaggle's evaluation sampling, so unfortunately, small fluctuations in score are common even when utilizing a fixed seed.
", "votes": null, "replies": [ { "id": 3203642, "author_name": "cliff376", "author_url": "", "post_date": "05/17/2025 04:24:03", "content": "\n", "votes": null, "replies": [] }, { "id": 3207960, "author_name": "alejopaullier", "author_url": "", "post_date": "05/23/2025 13:27:58", "content": "On Kaggle, competitions evaluate submissions using a randomly sampled subset of the leaderboard test data. This means that even if you submit the same submission.csv, your score can vary due to changes in the evaluation slice.
\nSeeding helps ensure consistent training outputs, such as model weights, prediction behavior, etc. However, you cannot control or seed Kaggle's evaluation sampling, so unfortunately, small fluctuations in score are common even when utilizing a fixed seed.
\n
\n Thank you very much for your patient and thoughtful response, which really helped a lot. Thank you again!
That is not true. The submissions are always evaluated on the same test set, otherwise, the public LB would be meaningless. If you submit a deterministic submission you will always get the same score. The difference in scores is due to randomness in the seeds of different libraries. You must seed all the underlying modules to guarantee reproducibility. You can check that many competitors use a seed function that seeds many underlying libraries such as numpy, torch, random, etc.
In any case, there is no point in seeding submissions. The best approach is to submit the exact same code unseeded as many times as you can to check the mean and variance of the model. Then submit the model with the highest mean and lowest variance.
", "votes": null, "replies": [ { "id": 3208369, "author_name": "cliff376", "author_url": "", "post_date": "05/24/2025 02:15:41", "content": "\n\nThat is not true. The submissions are always evaluated on the same test set, otherwise, the public LB would be meaningless. If you submit a deterministic submission you will always get the same score. The difference in scores is due to randomness in the seeds of different libraries. You must seed all the underlying modules to guarantee reproducibility. You can check that many competitors use a seed function that seeds many underlying libraries such as
\nnumpy, torch, random, etc.In any case, there is no point in seeding submissions. The best approach is to submit the exact same code unseeded as many times as you can to check the mean and variance of the model. Then submit the model with the highest mean and lowest variance.
\n
Thank you very much for your reply, it is really helpful!!
", "votes": null, "replies": [ { "id": 3210603, "author_name": "sabreenelkamash", "author_url": "", "post_date": "05/27/2025 12:42:45", "content": "I am experiencing a recurring problem where the submission process fails after four minutes. I tried to find out the reason but I couldn’t. Can you provide me with a solution to this problem?
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3203538, "author_name": "leonblackd", "author_url": "", "post_date": "05/16/2025 22:24:50", "content": "please share submission file?
", "votes": null, "replies": [ { "id": 3203641, "author_name": "cliff376", "author_url": "", "post_date": "05/17/2025 04:23:08", "content": "\n", "votes": null, "replies": [ { "id": 3207687, "author_name": "", "author_url": "", "post_date": "05/23/2025 06:07:10", "content": "please share submission file?
\n
\n Thank you very much for your reply! I am running the protenix code on the list and get submission. Although the problem has been solved by reply below, thank you again for your concern!
I met the same issues before, didn't figure out yet.
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3203363": "", "3203419": "On Kaggle, competitions evaluate submissions using a randomly sampled subset of the leaderboard test data. This means that even if you submit the same submission.csv, your score can vary due to changes in the evaluation slice.\n\nSeeding helps ensure consistent training outputs, such as model weights, prediction behavior, etc. However, you cannot control or seed Kaggle's evaluation sampling, so unfortunately, small fluctuations in score are common even when utilizing a fixed seed.", "3203538": "please share submission file?", "3203641": "> please share submission file?\nThank you very much for your reply! I am running the protenix code on the list and get submission. Although the problem has been solved by reply below, thank you again for your concern!", "3203642": "> On Kaggle, competitions evaluate submissions using a randomly sampled subset of the leaderboard test data. This means that even if you submit the same submission.csv, your score can vary due to changes in the evaluation slice.\n> \n> Seeding helps ensure consistent training outputs, such as model weights, prediction behavior, etc. However, you cannot control or seed Kaggle's evaluation sampling, so unfortunately, small fluctuations in score are common even when utilizing a fixed seed.\nThank you very much for your patient and thoughtful response, which really helped a lot. Thank you again!", "3207687": "I met the same issues before, didn't figure out yet.", "3207960": "That is not true. The submissions are always evaluated on the same test set, otherwise, the public LB would be meaningless. If you submit a deterministic submission you will always get the same score. The difference in scores is due to randomness in the seeds of different libraries. You must seed all the underlying modules to guarantee reproducibility. You can check that many competitors use a seed function that seeds many underlying libraries such as `numpy, torch, random`, etc.\n\nIn any case, there is no point in seeding submissions. The best approach is to submit the exact same code unseeded as many times as you can to check the mean and variance of the model. Then submit the model with the highest mean and lowest variance.", "3208369": "> That is not true. The submissions are always evaluated on the same test set, otherwise, the public LB would be meaningless. If you submit a deterministic submission you will always get the same score. The difference in scores is due to randomness in the seeds of different libraries. You must seed all the underlying modules to guarantee reproducibility. You can check that many competitors use a seed function that seeds many underlying libraries such as `numpy, torch, random`, etc.\n> \n> In any case, there is no point in seeding submissions. The best approach is to submit the exact same code unseeded as many times as you can to check the mean and variance of the model. Then submit the model with the highest mean and lowest variance.\n\nThank you very much for your reply, it is really helpful!!", "3210603": "I am experiencing a recurring problem where the submission process fails after four minutes. I tried to find out the reason but I couldn’t. Can you provide me with a solution to this problem?" }, "source": "meta" }