![\"Kaggle](\"https://nsa40.casimages.com/img/2021/05/09/210509064255229230.png\")
Hi all,
\nI'm not able to commit any kernel since more than 1h. Everything queued. Even interactive kernel got queued (but worked after a few minutes).
\nDo you guys also experimenting this issue?
\nI also got this message:
\n
Hi all,
\nI'm not able to commit any kernel since more than 1h. Everything queued. Even interactive kernel got queued (but worked after a few minutes).
\nDo you guys also experimenting this issue?
\nI also got this message:
\n
Making 3 code competitions end in two consecutive days was probably not a good idea.
", "rawMarkdown": "Making 3 code competitions end in two consecutive days was probably not a good idea.", "votes": 13 }, { "id": 1299490, "postDate": "2021-05-09T17:40:02.520Z", "content": "can't even imagine what would happen tommorow 😨
", "rawMarkdown": "can't even imagine what would happen tommorow 😨", "votes": 3, "replies": [ { "id": 1299502, "postDate": "2021-05-09T17:52:06.553Z", "content": "Last-day issues already happened in the past and they extended deadline by one day.
\nBut currently it looks better, queue time decreases…
Nearly always get queued for TPU but today is the first to see GPU queued every time.
\nWondered if there is some rescore going on somewhere? Someone mentioned in another post the drop in number of teams had to do with a rescore for the new data, but new data happened ages ago. Maybe another rescore is in progress and teams will reappear.
True, I did not notice that the re-rescored happened. All of our previous re-ran submissions failed due to one (or both) of the following reasons:
\nI know (1) is not a good practice, I will avoid it for next competitions.
\nIf some teams have left since data updates, there is low probability to have their old submissions rescored correctly.
", "rawMarkdown": "True, I did not notice that the re-rescored happened. All of our previous re-ran submissions failed due to one (or both) of the following reasons:\n1. Use \"last environment\" option enabled (GDAL not compatible anymore in new environment)\n2. Fast commit option with hardcoded conditions/filename\n\nI know (1) is not a good practice, I will avoid it for next competitions.\n\nIf some teams have left since data updates, there is low probability to have their old submissions rescored correctly. " }, { "id": 1299481, "postDate": "2021-05-09T17:27:54.837Z", "content": "Think also an issue with new tiffs and layers and if using rasterio and not dealing with that previously.
", "rawMarkdown": "Think also an issue with new tiffs and layers and if using rasterio and not dealing with that previously. ", "votes": 1 } ] }, { "id": 1300693, "postDate": "2021-05-10T16:29:55.877Z", "content": "No movement for the next half hour, Our final submission is no longer on time for the deadline…
", "rawMarkdown": "No movement for the next half hour, Our final submission is no longer on time for the deadline...", "votes": 2 }, { "id": 1300598, "postDate": "2021-05-10T15:22:30.250Z", "content": "Same here, commits are stuck … :-(
\nhubmap-inference\nVersion #1 with GPU\nQueued\n10 minutes\n\nhubmap-inference\nVersion #1 with GPU\nQueued\n24 minutes\nwait like forever, be there for 1 hour, should be finished within 10 mins, please extend the deadline
", "rawMarkdown": "wait like forever, be there for 1 hour, should be finished within 10 mins, please extend the deadline " }, { "id": 1300656, "postDate": "2021-05-10T16:03:13.313Z", "content": "same here, now submitting is in queue
", "rawMarkdown": "same here, now submitting is in queue" }, { "id": 1300537, "postDate": "2021-05-10T14:45:33.880Z", "content": "with 3 competitions ending by tomorrow (HPA, Hubmap & shopee), looks like Kaggle servers are really stressed out
\nmy files are taking double the time…
in some ways poor timing would be giving a lot of Kagglers who planned just-in-time submissions the jitters !!
", "rawMarkdown": "with 3 competitions ending by tomorrow (HPA, Hubmap & shopee), looks like Kaggle servers are really stressed out\nmy files are taking double the time... \n\nin some ways poor timing would be giving a lot of Kagglers who planned just-in-time submissions the jitters !!" }, { "id": 1300446, "postDate": "2021-05-10T13:11:06.633Z", "content": "Looks like it's back again.
", "rawMarkdown": "Looks like it's back again. " }, { "id": 1299872, "postDate": "2021-05-10T05:25:12.167Z", "content": "I'm getting this but just for GPU and TPU commits.
", "rawMarkdown": "I'm getting this but just for GPU and TPU commits." }, { "id": 1300581, "postDate": "2021-05-10T15:14:37.120Z", "content": "I'm facing the same problem and have not been able to commit for the last few hours.
\nAlso, I get 'Submission CSV Not Found' error in testing after submission, is this the same issue and not my fault?
Around 20 minutes in the queue now. We won't be able to run the 5 submissions today.
", "rawMarkdown": "Around 20 minutes in the queue now. We won't be able to run the 5 submissions today.", "votes": 1 }, { "id": 1300627, "postDate": "2021-05-10T15:45:01.780Z", "content": "Same here, my submissions won't finish in time due to the queue.
\nLet's hope @addisonhoward watches this thread and the deadline can be extended for a day to resolve this.
\nEdited: Realistically, runs get scheduled within (max) ~1 hour, so it's not that bad, probably missed one submission due to that
", "rawMarkdown": "Same here, my submissions won't finish in time due to the queue. \n\nLet's hope @addisonhoward watches this thread and the deadline can be extended for a day to resolve this. \n\n---\n\nEdited: Realistically, runs get scheduled within (max) ~1 hour, so it's not that bad, probably missed one submission due to that" }, { "id": 1300644, "postDate": "2021-05-10T15:56:10.480Z", "content": "I think so too.
\nJust give us the additional submissions…..
Making 3 code competitions end in two consecutive days was probably not a good idea.
", "votes": 13, "replies": [] }, { "id": 1299490, "author_name": "Awsaf", "author_url": "", "post_date": "2021-05-09T17:40:02.520000", "content": "can't even imagine what would happen tommorow 😨
", "votes": 3, "replies": [ { "id": 1299502, "author_name": "MPWARE", "author_url": "", "post_date": "2021-05-09T17:52:06.553000", "content": "Last-day issues already happened in the past and they extended deadline by one day.
\nBut currently it looks better, queue time decreases…
Nearly always get queued for TPU but today is the first to see GPU queued every time.
\nWondered if there is some rescore going on somewhere? Someone mentioned in another post the drop in number of teams had to do with a rescore for the new data, but new data happened ages ago. Maybe another rescore is in progress and teams will reappear.
True, I did not notice that the re-rescored happened. All of our previous re-ran submissions failed due to one (or both) of the following reasons:
\nI know (1) is not a good practice, I will avoid it for next competitions.
\nIf some teams have left since data updates, there is low probability to have their old submissions rescored correctly.
", "votes": 0, "replies": [] }, { "id": 1299481, "author_name": "something4kag", "author_url": "", "post_date": "2021-05-09T17:27:54.837000", "content": "Think also an issue with new tiffs and layers and if using rasterio and not dealing with that previously.
", "votes": 1, "replies": [] } ] }, { "id": 1300693, "author_name": "ITK8191", "author_url": "", "post_date": "2021-05-10T16:29:55.877000", "content": "No movement for the next half hour, Our final submission is no longer on time for the deadline…
", "votes": 2, "replies": [] }, { "id": 1300598, "author_name": "Gena", "author_url": "", "post_date": "2021-05-10T15:22:30.250000", "content": "Same here, commits are stuck … :-(
\nhubmap-inference\nVersion #1 with GPU\nQueued\n10 minutes\n\nhubmap-inference\nVersion #1 with GPU\nQueued\n24 minutes\nwait like forever, be there for 1 hour, should be finished within 10 mins, please extend the deadline
", "votes": 0, "replies": [] }, { "id": 1300656, "author_name": "cswwp", "author_url": "", "post_date": "2021-05-10T16:03:13.313000", "content": "same here, now submitting is in queue
", "votes": 0, "replies": [] }, { "id": 1300537, "author_name": "Kamal Das", "author_url": "", "post_date": "2021-05-10T14:45:33.880000", "content": "with 3 competitions ending by tomorrow (HPA, Hubmap & shopee), looks like Kaggle servers are really stressed out
\nmy files are taking double the time…
in some ways poor timing would be giving a lot of Kagglers who planned just-in-time submissions the jitters !!
", "votes": 0, "replies": [] }, { "id": 1300446, "author_name": "something4kag", "author_url": "", "post_date": "2021-05-10T13:11:06.633000", "content": "Looks like it's back again.
", "votes": 0, "replies": [] }, { "id": 1299872, "author_name": "Naman Manchanda", "author_url": "", "post_date": "2021-05-10T05:25:12.167000", "content": "I'm getting this but just for GPU and TPU commits.
", "votes": 0, "replies": [] }, { "id": 1300581, "author_name": "", "author_url": "", "post_date": "2021-05-10T15:14:37.120000", "content": "I'm facing the same problem and have not been able to commit for the last few hours.
\nAlso, I get 'Submission CSV Not Found' error in testing after submission, is this the same issue and not my fault?
Around 20 minutes in the queue now. We won't be able to run the 5 submissions today.
", "votes": 1, "replies": [] }, { "id": 1300627, "author_name": "Gena", "author_url": "", "post_date": "2021-05-10T15:45:01.780000", "content": "Same here, my submissions won't finish in time due to the queue.
\nLet's hope @addisonhoward watches this thread and the deadline can be extended for a day to resolve this.
\nEdited: Realistically, runs get scheduled within (max) ~1 hour, so it's not that bad, probably missed one submission due to that
", "votes": 0, "replies": [] }, { "id": 1300644, "author_name": "", "author_url": "", "post_date": "2021-05-10T15:56:10.480000", "content": "I think so too.
\nJust give us the additional submissions…..