I submitted an inference notebook, and it is successfully run within 180 sec by watching the log.
\nBut the scoring time is at least longer than 1h.
Is there anyone who have the same issue?
", "messages": [ { "id": "2770680", "postDate": "04/24/2024 01:31:10", "content": "I submitted an inference notebook, and it is successfully run within 180 sec by watching the log.
\nBut the scoring time is at least longer than 1h.
Is there anyone who have the same issue?
", "rawMarkdown": "I submitted an inference notebook, and it is successfully run within 180 sec by watching the log.\nBut the scoring time is at least longer than 1h.\n\nIs there anyone who have the same issue?", "votes": null }, { "id": "2770706", "postDate": "04/24/2024 01:49:16", "content": "See https://www.kaggle.com/competitions/birdclef-2024/discussion/496684#2767134
", "rawMarkdown": "See https://www.kaggle.com/competitions/birdclef-2024/discussion/496684#2767134", "votes": null }, { "id": "2770749", "postDate": "04/24/2024 02:37:19", "content": "Thanks for the reply!
\nI've specified the cause of the problem.
In my case, 5-fold ensemble inference takes 12 sec per one test_data.
\nThus, total time for 1100 data takes 12 sec x 1100 = 13200 sec ~ 3.67 hours, which is beyond the inference time in this competition.
So I think this problem would be resolved by using parallelization of inference.
", "rawMarkdown": "Thanks for the reply! \nI've specified the cause of the problem.\n\nIn my case, 5-fold ensemble inference takes 12 sec per one test_data.\nThus, total time for 1100 data takes 12 sec x 1100 = 13200 sec ~ 3.67 hours, which is beyond the inference time in this competition.\n\nSo I think this problem would be resolved by using parallelization of inference.", "votes": null }, { "id": "2770844", "postDate": "04/24/2024 03:58:47", "content": "have you looked at cpu use? I don't know what you use, but code is probably already running with multi threading on cpu.
", "rawMarkdown": "have you looked at cpu use? I don't know what you use, but code is probably already running with multi threading on cpu.", "votes": null }, { "id": "2770864", "postDate": "04/24/2024 04:08:04", "content": "That's right. Thank you!
", "rawMarkdown": "That's right. Thank you!", "votes": null }, { "id": "2770875", "postDate": "04/24/2024 04:11:46", "content": "Hi, Hope you doing good!
\nHope my approaches help you in some way.
\nhttps://www.kaggle.com/competitions/birdclef-2024/discussion/494665
Thanks for sharing!
\nI tried your method, and it makes the inference speed faster (7700sec => 6600 sec).
Great, Hope that enough for submission
", "rawMarkdown": "Great, Hope that enough for submission", "votes": null }, { "id": "2771532", "postDate": "04/24/2024 09:25:35", "content": "I had the same issue. Are you loading all spectrograms before training or are you reading them from files on the fly during training? I was doing the latter previously and it was the cause of my slow inference speed
", "rawMarkdown": "I had the same issue. Are you loading all spectrograms before training or are you reading them from files on the fly during training? I was doing the latter previously and it was the cause of my slow inference speed", "votes": null }, { "id": "2771571", "postDate": "04/24/2024 09:59:28", "content": "I'm loading all spectrograms before training…
", "rawMarkdown": "I'm loading all spectrograms before training...", "votes": null }, { "id": "2771573", "postDate": "04/24/2024 10:00:48", "content": "Do you get the same result with openvino? I consistently lose 0.01 on the LB when I use it compared to ONNX models.
", "rawMarkdown": "Do you get the same result with openvino? I consistently lose 0.01 on the LB when I use it compared to ONNX models.", "votes": null }, { "id": "2771854", "postDate": "04/24/2024 12:33:42", "content": "I submitted a notebook to measure the time to create stft spectrograms under the following configuration:
\n # == stfft config ==
\n FS = 32000 # sample rate
\n MIN_FREQ = 40 # min frequency
\n MAX_FREQ = 15000 # max frequency
\n N_FFT = 1095 # n FFT of Spec.
\n N_MAX = 10 # max number of samples
\n WIN_SIZE = 412 # WIN_SIZE of Spec.
\n WIN_LAP = 100 # overlap of Spec.
From this experimental submission, I found that pure time to create the spectrograms is about 25 minutes.
", "rawMarkdown": "I submitted a notebook to measure the time to create stft spectrograms under the following configuration:\n # == stfft config ==\n FS = 32000 # sample rate\n MIN_FREQ = 40 # min frequency\n MAX_FREQ = 15000 # max frequency\n N_FFT = 1095 # n FFT of Spec.\n N_MAX = 10 # max number of samples\n WIN_SIZE = 412 # WIN_SIZE of Spec.\n WIN_LAP = 100 # overlap of Spec.\n\nFrom this experimental submission, I found that pure time to create the spectrograms is about 25 minutes.", "votes": null }, { "id": "2773909", "postDate": "04/25/2024 00:46:22", "content": "Sorry for late response.
\nI have just submitted an inference notebook to verify the difference, but got runtime error.
See https://www.kaggle.com/competitions/birdclef-2024/discussion/496684#2767134
", "votes": null, "replies": [ { "id": 2770749, "author_name": "johnlemon3", "author_url": "", "post_date": "04/24/2024 02:37:19", "content": "Thanks for the reply!
\nI've specified the cause of the problem.
In my case, 5-fold ensemble inference takes 12 sec per one test_data.
\nThus, total time for 1100 data takes 12 sec x 1100 = 13200 sec ~ 3.67 hours, which is beyond the inference time in this competition.
So I think this problem would be resolved by using parallelization of inference.
", "votes": null, "replies": [ { "id": 2770844, "author_name": "cpmpml", "author_url": "", "post_date": "04/24/2024 03:58:47", "content": "have you looked at cpu use? I don't know what you use, but code is probably already running with multi threading on cpu.
", "votes": null, "replies": [ { "id": 2770864, "author_name": "johnlemon3", "author_url": "", "post_date": "04/24/2024 04:08:04", "content": "That's right. Thank you!
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 2770875, "author_name": "lmhongkhnh", "author_url": "", "post_date": "04/24/2024 04:11:46", "content": "Hi, Hope you doing good!
\nHope my approaches help you in some way.
\nhttps://www.kaggle.com/competitions/birdclef-2024/discussion/494665
Thanks for sharing!
\nI tried your method, and it makes the inference speed faster (7700sec => 6600 sec).
Great, Hope that enough for submission
", "votes": null, "replies": [] }, { "id": 2771573, "author_name": "cpmpml", "author_url": "", "post_date": "04/24/2024 10:00:48", "content": "Do you get the same result with openvino? I consistently lose 0.01 on the LB when I use it compared to ONNX models.
", "votes": null, "replies": [ { "id": 2773909, "author_name": "johnlemon3", "author_url": "", "post_date": "04/25/2024 00:46:22", "content": "Sorry for late response.
\nI have just submitted an inference notebook to verify the difference, but got runtime error.
I had the same issue. Are you loading all spectrograms before training or are you reading them from files on the fly during training? I was doing the latter previously and it was the cause of my slow inference speed
", "votes": null, "replies": [ { "id": 2771571, "author_name": "johnlemon3", "author_url": "", "post_date": "04/24/2024 09:59:28", "content": "I'm loading all spectrograms before training…
", "votes": null, "replies": [] } ] }, { "id": 2771854, "author_name": "johnlemon3", "author_url": "", "post_date": "04/24/2024 12:33:42", "content": "I submitted a notebook to measure the time to create stft spectrograms under the following configuration:
\n # == stfft config ==
\n FS = 32000 # sample rate
\n MIN_FREQ = 40 # min frequency
\n MAX_FREQ = 15000 # max frequency
\n N_FFT = 1095 # n FFT of Spec.
\n N_MAX = 10 # max number of samples
\n WIN_SIZE = 412 # WIN_SIZE of Spec.
\n WIN_LAP = 100 # overlap of Spec.
From this experimental submission, I found that pure time to create the spectrograms is about 25 minutes.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2770680": "I submitted an inference notebook, and it is successfully run within 180 sec by watching the log.\nBut the scoring time is at least longer than 1h.\n\nIs there anyone who have the same issue?", "2770706": "See https://www.kaggle.com/competitions/birdclef-2024/discussion/496684#2767134", "2770749": "Thanks for the reply! \nI've specified the cause of the problem.\n\nIn my case, 5-fold ensemble inference takes 12 sec per one test_data.\nThus, total time for 1100 data takes 12 sec x 1100 = 13200 sec ~ 3.67 hours, which is beyond the inference time in this competition.\n\nSo I think this problem would be resolved by using parallelization of inference.", "2770844": "have you looked at cpu use? I don't know what you use, but code is probably already running with multi threading on cpu.", "2770864": "That's right. Thank you!", "2770875": "Hi, Hope you doing good!\nHope my approaches help you in some way. \nhttps://www.kaggle.com/competitions/birdclef-2024/discussion/494665", "2771103": "Thanks for sharing!\nI tried your method, and it makes the inference speed faster (7700sec => 6600 sec).", "2771163": "Great, Hope that enough for submission", "2771532": "I had the same issue. Are you loading all spectrograms before training or are you reading them from files on the fly during training? I was doing the latter previously and it was the cause of my slow inference speed", "2771571": "I'm loading all spectrograms before training...", "2771573": "Do you get the same result with openvino? I consistently lose 0.01 on the LB when I use it compared to ONNX models.", "2771854": "I submitted a notebook to measure the time to create stft spectrograms under the following configuration:\n # == stfft config ==\n FS = 32000 # sample rate\n MIN_FREQ = 40 # min frequency\n MAX_FREQ = 15000 # max frequency\n N_FFT = 1095 # n FFT of Spec.\n N_MAX = 10 # max number of samples\n WIN_SIZE = 412 # WIN_SIZE of Spec.\n WIN_LAP = 100 # overlap of Spec.\n\nFrom this experimental submission, I found that pure time to create the spectrograms is about 25 minutes.", "2773909": "Sorry for late response. \nI have just submitted an inference notebook to verify the difference, but got runtime error." }, "source": "meta" }