When i use the same notebook, only the weights are changed.
\nHowever, some notebooks can get the results normally, but some will threw \"time out error\".
\nSo during the submission process, are there any other factors that will affect the overall running time?
\n**For example, **
\nDoes the time to install the environment fluctuate greatly each time?
\nWill changes in input data size in /kaggle/input/ affect notebook running time?
\nWill the process of calculating the csv result be calculated by kaggle in the submisson time?
\nIt’s really really really strange!!!😵
When i use the same notebook, only the weights are changed.
\nHowever, some notebooks can get the results normally, but some will threw \"time out error\".
\nSo during the submission process, are there any other factors that will affect the overall running time?
\n**For example, **
\nDoes the time to install the environment fluctuate greatly each time?
\nWill changes in input data size in /kaggle/input/ affect notebook running time?
\nWill the process of calculating the csv result be calculated by kaggle in the submisson time?
\nIt’s really really really strange!!!😵
The timeout issue may be caused by the memory usage. It helps improve the speed and avoid out of time by clearing memory after each image inference.
", "rawMarkdown": "The timeout issue may be caused by the memory usage. It helps improve the speed and avoid out of time by clearing memory after each image inference." }, { "id": 2635587, "postDate": "2024-02-04T13:49:44.893Z", "content": "The code implements the exact same result, but I've found that the time varies quite a bit depending on how I type.
", "rawMarkdown": "The code implements the exact same result, but I've found that the time varies quite a bit depending on how I type.", "replies": [ { "id": 2635606, "postDate": "2024-02-04T13:56:37.583Z", "content": "I simply modified the weight path, thus it timed out, which was really a strange thing.
", "rawMarkdown": "I simply modified the weight path, thus it timed out, which was really a strange thing." } ] }, { "id": 2635569, "postDate": "2024-02-04T13:45:05.653Z", "content": "to verify, you can waste a few submission:
\ncompare timming for
make\n\n\nI tried twice here and it takes about 14 minutes during submission if all results are '1 0'.
", "rawMarkdown": "I tried twice here and it takes about 14 minutes during submission if all results are '1 0'." } ] }, { "id": 2635449, "postDate": "2024-02-04T12:15:12.253Z", "content": "Absolutely the same happens to me as well. It's just the nature on Kaggle Kernels, unfortunately
", "rawMarkdown": "Absolutely the same happens to me as well. It's just the nature on Kaggle Kernels, unfortunately ", "replies": [ { "id": 2635590, "postDate": "2024-02-04T13:51:13.533Z", "content": "Hahaha, this is indeed a headache for us
", "rawMarkdown": "Hahaha, this is indeed a headache for us" } ] }, { "id": 2635143, "postDate": "2024-02-04T07:26:01.397Z", "content": "Even if the model size is the same and the model inference time is the same, if the weight is different, the predicton will change and the score calculation time will change, so I think this is the effect.
", "rawMarkdown": "Even if the model size is the same and the model inference time is the same, if the weight is different, the predicton will change and the score calculation time will change, so I think this is the effect.", "replies": [ { "id": 2635146, "postDate": "2024-02-04T07:28:21.357Z", "content": "Score calculation is indeed not used in my code, so do you mean that for different prediction results by different weights, there is a certain difference in the submission time calculated by kaggle?
", "rawMarkdown": "**Score calculation** is indeed not used in my code, so do you mean that for different prediction results by different weights, there is a certain difference in the submission time calculated by kaggle?", "replies": [ { "id": 2635156, "postDate": "2024-02-04T07:36:22.470Z", "content": "\n\ndifferent prediction results by different weights, there is a certain difference in the submission time calculated by kaggle?
\n
yes
", "rawMarkdown": ">different prediction results by different weights, there is a certain difference in the submission time calculated by kaggle?\n\nyes" }, { "id": 2635171, "postDate": "2024-02-04T07:41:04.210Z", "content": "You could find similar topics in the past thread like below
\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/459969
\n", "rawMarkdown": "You could find similar topics in the past thread like below\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/459969\n\n>For this metric, I extended the scoring time to 90 minutes. I haven't seen any submissions that need longer than that, but I can extend it further if necessary. Let me know" }, { "id": 2635183, "postDate": "2024-02-04T07:45:07.017Z", "content": "For this metric, I extended the scoring time to 90 minutes. I haven't seen any submissions that need longer than that, but I can extend it further if necessary. Let me know
\n
Thanks a lot, I feel that the metric calculation will be included in the overall submission inference time seems a bit unreasonable.😓
", "rawMarkdown": "Thanks a lot, I feel that the metric calculation will be included in the overall submission inference time seems a bit unreasonable.😓" }, { "id": 2635540, "postDate": "2024-02-04T13:17:43.867Z", "content": "Although I have not performed strict verification, it seems that the score calculation time depends on the inference result. It may not actually be included, so please forgive me if I'm wrong. Also, there seems to be some variation in the execution times of Kaggle's kernels, even though the code is exactly the same.
", "rawMarkdown": "Although I have not performed strict verification, it seems that the score calculation time depends on the inference result. It may not actually be included, so please forgive me if I'm wrong. Also, there seems to be some variation in the execution times of Kaggle's kernels, even though the code is exactly the same." }, { "id": 2635583, "postDate": "2024-02-04T13:49:19.087Z", "content": "Thanks，I'll try to reduce some inference time
", "rawMarkdown": "Thanks，I'll try to reduce some inference time", "votes": 1 }, { "id": 2636204, "postDate": "2024-02-04T22:08:26.030Z", "content": "It seems like scoring is done while notebook execution time. Bad solution make scoring long and this increases the notebook's runtime.
\nThis is exactly the problem that everyone had with the metric at the begging of competition, the metric was calculated for too long until it exceeded the available time of the GPU or got very small metric values, this problem disappeared when kagglers started to submitted +- good solutions.
Thanks, I'll check again😀
", "rawMarkdown": "Thanks, I'll check again😀\n\n\n" }, { "id": 2636862, "postDate": "2024-02-05T10:39:51.557Z", "content": "@viktoriaskorik no, the code has been changed that is why the bug disappeared. This has nothing to do with the quality of the submission.
", "rawMarkdown": "@viktoriaskorik no, the code has been changed that is why the bug disappeared. This has nothing to do with the quality of the submission." } ] } ] }, { "id": 2635140, "postDate": "2024-02-04T07:23:36.717Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2636356, "author_name": "Min-Hsien Weng", "author_url": "", "post_date": "2024-02-05T03:19:48.963000", "content": "The timeout issue may be caused by the memory usage. It helps improve the speed and avoid out of time by clearing memory after each image inference.
", "votes": 0, "replies": [] }, { "id": 2635587, "author_name": "HB", "author_url": "", "post_date": "2024-02-04T13:49:44.893000", "content": "The code implements the exact same result, but I've found that the time varies quite a bit depending on how I type.
", "votes": 0, "replies": [ { "id": 2635606, "author_name": "tanxxx", "author_url": "", "post_date": "2024-02-04T13:56:37.583000", "content": "I simply modified the weight path, thus it timed out, which was really a strange thing.
", "votes": 0, "replies": [] } ] }, { "id": 2635569, "author_name": "hengck23", "author_url": "", "post_date": "2024-02-04T13:45:05.653000", "content": "to verify, you can waste a few submission:
\ncompare timming for
make\n\n\nI tried twice here and it takes about 14 minutes during submission if all results are '1 0'.
", "votes": 0, "replies": [] } ] }, { "id": 2635449, "author_name": "Ivan Panshin", "author_url": "", "post_date": "2024-02-04T12:15:12.253000", "content": "Absolutely the same happens to me as well. It's just the nature on Kaggle Kernels, unfortunately
", "votes": 0, "replies": [ { "id": 2635590, "author_name": "tanxxx", "author_url": "", "post_date": "2024-02-04T13:51:13.533000", "content": "Hahaha, this is indeed a headache for us
", "votes": 0, "replies": [] } ] }, { "id": 2635143, "author_name": "ts", "author_url": "", "post_date": "2024-02-04T07:26:01.397000", "content": "Even if the model size is the same and the model inference time is the same, if the weight is different, the predicton will change and the score calculation time will change, so I think this is the effect.
", "votes": 0, "replies": [ { "id": 2635146, "author_name": "tanxxx", "author_url": "", "post_date": "2024-02-04T07:28:21.357000", "content": "Score calculation is indeed not used in my code, so do you mean that for different prediction results by different weights, there is a certain difference in the submission time calculated by kaggle?
", "votes": 0, "replies": [ { "id": 2635156, "author_name": "ts", "author_url": "", "post_date": "2024-02-04T07:36:22.470000", "content": "\n\ndifferent prediction results by different weights, there is a certain difference in the submission time calculated by kaggle?
\n
yes
", "votes": 0, "replies": [] }, { "id": 2635171, "author_name": "ts", "author_url": "", "post_date": "2024-02-04T07:41:04.210000", "content": "You could find similar topics in the past thread like below
\nhttps://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/459969
\n", "votes": 0, "replies": [] }, { "id": 2635183, "author_name": "tanxxx", "author_url": "", "post_date": "2024-02-04T07:45:07.017000", "content": "For this metric, I extended the scoring time to 90 minutes. I haven't seen any submissions that need longer than that, but I can extend it further if necessary. Let me know
\n
Thanks a lot, I feel that the metric calculation will be included in the overall submission inference time seems a bit unreasonable.😓
", "votes": 0, "replies": [] }, { "id": 2635540, "author_name": "ts", "author_url": "", "post_date": "2024-02-04T13:17:43.867000", "content": "Although I have not performed strict verification, it seems that the score calculation time depends on the inference result. It may not actually be included, so please forgive me if I'm wrong. Also, there seems to be some variation in the execution times of Kaggle's kernels, even though the code is exactly the same.
", "votes": 0, "replies": [] }, { "id": 2635583, "author_name": "tanxxx", "author_url": "", "post_date": "2024-02-04T13:49:19.087000", "content": "Thanks，I'll try to reduce some inference time
", "votes": 1, "replies": [] }, { "id": 2636204, "author_name": "Viktoria", "author_url": "", "post_date": "2024-02-04T22:08:26.030000", "content": "It seems like scoring is done while notebook execution time. Bad solution make scoring long and this increases the notebook's runtime.
\nThis is exactly the problem that everyone had with the metric at the begging of competition, the metric was calculated for too long until it exceeded the available time of the GPU or got very small metric values, this problem disappeared when kagglers started to submitted +- good solutions.
Thanks, I'll check again😀
", "votes": 0, "replies": [] }, { "id": 2636862, "author_name": "Optimo", "author_url": "", "post_date": "2024-02-05T10:39:51.557000", "content": "@viktoriaskorik no, the code has been changed that is why the bug disappeared. This has nothing to do with the quality of the submission.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2635140, "author_name": "", "author_url": "", "post_date": "2024-02-04T07:23:36.717000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2635041": "When i use the same notebook, only the weights are changed. \nHowever, **some notebooks can get the results normally, but some will threw \"time out error\".** \nSo during the submission process, are there any other factors that will affect the overall running time? \n**For example, **\nDoes the time to install the environment fluctuate greatly each time? \nWill changes in input data size in /kaggle/input/ affect notebook running time?\nWill the process of calculating the csv result be calculated by kaggle in the submisson time?\n**It’s really really really strange**!!!😵", "2636356": "The timeout issue may be caused by the memory usage. It helps improve the speed and avoid out of time by clearing memory after each image inference.", "2635587": "The code implements the exact same result, but I've found that the time varies quite a bit depending on how I type.", "2635569": "to verify, you can waste a few submission:\ncompare timming for\n\n```\nmake_dummy_submission(random objects)\n```\n\n```\nsleep(9hr)\nmake_dummy_submission(random objects)\n```", "2635449": "Absolutely the same happens to me as well. It's just the nature on Kaggle Kernels, unfortunately ", "2635143": "Even if the model size is the same and the model inference time is the same, if the weight is different, the predicton will change and the score calculation time will change, so I think this is the effect.", "2635140": "" } }