To estimate the time consumption, I simply time the num of audios to 200 and run a notebook. However, with the same code the running time difference is huge (between 1h56min to 3h5min).
\nAs indicated by @herbison in this discussion, which says I believe the Intel CPU you'll get in CPU interactive sessions will usually be an Intel Broadwell, Skylake is a newer model so it should be at least as good on average. Can I assume that when submitting the notebook, the CPU time is at least comparable to the shorter one (1h56min)? However, that notebook got a timeout error when I made the submission. Any idea? @herbison Thanks in advance!
update: with the same code (just forked one success submission), the forked one failed with timeout error when submit.
\nupdate 2: I have exactly the same code running for two versions, one finished in 2h55min, and the other is over 9hrs and I killed it.
", "messages": [ { "id": "2248480", "postDate": "05/06/2023 22:17:08", "content": "To estimate the time consumption, I simply time the num of audios to 200 and run a notebook. However, with the same code the running time difference is huge (between 1h56min to 3h5min).
\nAs indicated by @herbison in this discussion, which says I believe the Intel CPU you'll get in CPU interactive sessions will usually be an Intel Broadwell, Skylake is a newer model so it should be at least as good on average. Can I assume that when submitting the notebook, the CPU time is at least comparable to the shorter one (1h56min)? However, that notebook got a timeout error when I made the submission. Any idea? @herbison Thanks in advance!
update: with the same code (just forked one success submission), the forked one failed with timeout error when submit.
\nupdate 2: I have exactly the same code running for two versions, one finished in 2h55min, and the other is over 9hrs and I killed it.
", "rawMarkdown": "To estimate the time consumption, I simply time the num of audios to 200 and run a notebook. However, with the same code the running time difference is huge (between 1h56min to 3h5min). \n\nAs indicated by @herbison in [this discussion](https://www.kaggle.com/competitions/birdclef-2023/discussion/396546), which says `I believe the Intel CPU you'll get in CPU interactive sessions will usually be an Intel Broadwell, Skylake is a newer model so it should be at least as good on average`. Can I assume that when submitting the notebook, the CPU time is at least comparable to the shorter one (1h56min)? However, that notebook got a timeout error when I made the submission. Any idea? @herbison Thanks in advance!\n\n------------------------------------------\nupdate: with the same code (just forked one success submission), the forked one failed with timeout error when submit.\n\n--------------------------------------------\nupdate 2: I have exactly the same code running for two versions, one finished in 2h55min, and the other is over 9hrs and I killed it.", "votes": null }, { "id": "2248789", "postDate": "05/07/2023 07:36:17", "content": "Even with the same machine, it's a lottery ticket, see here.
", "rawMarkdown": "Even with the same machine, it's a lottery ticket, [see here](https://www.kaggle.com/competitions/asl-signs/discussion/405736).", "votes": null }, { "id": "2250441", "postDate": "05/08/2023 15:03:47", "content": "@leonshangguan Looking at your recent runs, the vast majority are running on Intel Broadwell CPUs, maybe 1 in 10 are AMD Rome.
\nIntel Skylake should be at least as good as Intel Broadwell runs.
\nThe AMD Rome ones are harder to compare with Skylake since its AMD vs. Intel.
\nThose runs appear to be the significantly faster ones, so I wouldn't rely on the Intel Skylake run being faster than those runs.
Thanks! I think that answers my question.
", "rawMarkdown": "Thanks! I think that answers my question.", "votes": null }, { "id": "2259242", "postDate": "05/14/2023 19:29:24", "content": "Hi @herbison , I got one more question: if one notebook passed the submission time restriction, then I think it should pass if I have an identity code (fork one), right? However this is not my case, I failed in the past 20 submissions with a timeout, and with the same code that passed the time restriction. Did kaggle make any changes in the past week?
", "rawMarkdown": "Hi @herbison , I got one more question: if one notebook passed the submission time restriction, then I think it should pass if I have an identity code (fork one), right? However this is not my case, I failed in the past 20 submissions with a timeout, and with the same code that passed the time restriction. Did kaggle make any changes in the past week?", "votes": null }, { "id": "2269856", "postDate": "05/22/2023 18:33:30", "content": "@leonshangguan Sorry for the slow response, I was at a conference last week.
\nIf the code is truly identical then my first guess would be that your code could be very close to the runtime limit.
\nIf that's the case, then just a little variance in runtime could cause it to tip over.
I did a cursory glance and all the submissions I see ran on the same hardware (Intel Skylake) so I don't think our infrastructure is the source of variance.
", "rawMarkdown": "leonshangguan Sorry for the slow response, I was at a conference last week.\n\nIf the code is truly identical then my first guess would be that your code could be very close to the runtime limit.\nIf that's the case, then just a little variance in runtime could cause it to tip over.\n\nI did a cursory glance and all the submissions I see ran on the same hardware (Intel Skylake) so I don't think our infrastructure is the source of variance.", "votes": null } ], "comments": [ { "id": 2248789, "author_name": "aphysict", "author_url": "", "post_date": "05/07/2023 07:36:17", "content": "Even with the same machine, it's a lottery ticket, see here.
", "votes": null, "replies": [] }, { "id": 2250441, "author_name": "herbison", "author_url": "", "post_date": "05/08/2023 15:03:47", "content": "@leonshangguan Looking at your recent runs, the vast majority are running on Intel Broadwell CPUs, maybe 1 in 10 are AMD Rome.
\nIntel Skylake should be at least as good as Intel Broadwell runs.
\nThe AMD Rome ones are harder to compare with Skylake since its AMD vs. Intel.
\nThose runs appear to be the significantly faster ones, so I wouldn't rely on the Intel Skylake run being faster than those runs.
Thanks! I think that answers my question.
", "votes": null, "replies": [] }, { "id": 2259242, "author_name": "leonshangguan", "author_url": "", "post_date": "05/14/2023 19:29:24", "content": "Hi @herbison , I got one more question: if one notebook passed the submission time restriction, then I think it should pass if I have an identity code (fork one), right? However this is not my case, I failed in the past 20 submissions with a timeout, and with the same code that passed the time restriction. Did kaggle make any changes in the past week?
", "votes": null, "replies": [ { "id": 2269856, "author_name": "herbison", "author_url": "", "post_date": "05/22/2023 18:33:30", "content": "@leonshangguan Sorry for the slow response, I was at a conference last week.
\nIf the code is truly identical then my first guess would be that your code could be very close to the runtime limit.
\nIf that's the case, then just a little variance in runtime could cause it to tip over.
I did a cursory glance and all the submissions I see ran on the same hardware (Intel Skylake) so I don't think our infrastructure is the source of variance.
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2248480": "To estimate the time consumption, I simply time the num of audios to 200 and run a notebook. However, with the same code the running time difference is huge (between 1h56min to 3h5min). \n\nAs indicated by @herbison in [this discussion](https://www.kaggle.com/competitions/birdclef-2023/discussion/396546), which says `I believe the Intel CPU you'll get in CPU interactive sessions will usually be an Intel Broadwell, Skylake is a newer model so it should be at least as good on average`. Can I assume that when submitting the notebook, the CPU time is at least comparable to the shorter one (1h56min)? However, that notebook got a timeout error when I made the submission. Any idea? @herbison Thanks in advance!\n\n------------------------------------------\nupdate: with the same code (just forked one success submission), the forked one failed with timeout error when submit.\n\n--------------------------------------------\nupdate 2: I have exactly the same code running for two versions, one finished in 2h55min, and the other is over 9hrs and I killed it.", "2248789": "Even with the same machine, it's a lottery ticket, [see here](https://www.kaggle.com/competitions/asl-signs/discussion/405736).", "2250441": "leonshangguan Looking at your recent runs, the vast majority are running on Intel Broadwell CPUs, maybe 1 in 10 are AMD Rome.\nIntel Skylake should be at least as good as Intel Broadwell runs.\nThe AMD Rome ones are harder to compare with Skylake since its AMD vs. Intel. \nThose runs appear to be the significantly faster ones, so I wouldn't rely on the Intel Skylake run being faster than those runs.", "2250517": "Thanks! I think that answers my question.", "2259242": "Hi @herbison , I got one more question: if one notebook passed the submission time restriction, then I think it should pass if I have an identity code (fork one), right? However this is not my case, I failed in the past 20 submissions with a timeout, and with the same code that passed the time restriction. Did kaggle make any changes in the past week?", "2269856": "leonshangguan Sorry for the slow response, I was at a conference last week.\n\nIf the code is truly identical then my first guess would be that your code could be very close to the runtime limit.\nIf that's the case, then just a little variance in runtime could cause it to tip over.\n\nI did a cursory glance and all the submissions I see ran on the same hardware (Intel Skylake) so I don't think our infrastructure is the source of variance." }, "source": "meta" }