{ "id": 107165, "title": "Submission error ", "url": "/competitions/aptos2019-blindness-detection/discussion/107165", "author_name": "", "post_date": "2019-09-02T17:29:31.528371800Z", "votes": 19, "comment_count": 27, "views": 0, "content": "

All submission fail today (run for 9 hours and then error)\nAnyone else has the same issue?

", "messages": [ { "id": "616097", "postDate": "09/02/2019 17:29:31", "content": "

All submission fail today (run for 9 hours and then error)\nAnyone else has the same issue?

", "rawMarkdown": "All submission fail today (run for 9 hours and then error)\nAnyone else has the same issue?", "votes": null }, { "id": "616114", "postDate": "09/02/2019 17:46:21", "content": "

Same for me. The same kernel was submitted with no problem yesterday (it took ~8 hours)

", "rawMarkdown": "Same for me. The same kernel was submitted with no problem yesterday (it took ~8 hours)", "votes": null }, { "id": "616315", "postDate": "09/03/2019 02:10:45", "content": "

Yes. I had faced the same issue. The total commit for the kernel was around 40 mins. But on submission, the kernel ran for 9 hours and threw kernel error..

", "rawMarkdown": "Yes. I had faced the same issue. The total commit for the kernel was around 40 mins. But on submission, the kernel ran for 9 hours and threw kernel error..", "votes": null }, { "id": "616395", "postDate": "09/03/2019 05:16:23", "content": "

I believe Kaggle kernels have gone rogue...this is happening in other competitions as well :(

", "rawMarkdown": "I believe Kaggle kernels have gone rogue...this is happening in other competitions as well :(", "votes": null }, { "id": "616398", "postDate": "09/03/2019 05:29:23", "content": "

I've encountered the same situation.\nMy Kernel ends within 3 hours so far, but it still has not end after 5 hours.

", "rawMarkdown": "I've encountered the same situation.\nMy Kernel ends within 3 hours so far, but it still has not end after 5 hours.", "votes": null }, { "id": "616403", "postDate": "09/03/2019 05:35:18", "content": "

Play safe and make it <35 minutes

", "rawMarkdown": "Play safe and make it <35 minutes", "votes": null }, { "id": "616521", "postDate": "09/03/2019 07:55:23", "content": "

I had the same situation yesterday. The commit took 45 minutes and and submission ended after 9 hours with an error. Very disappointing...

", "rawMarkdown": "I had the same situation yesterday. The commit took 45 minutes and and submission ended after 9 hours with an error. Very disappointing...", "votes": null }, { "id": "616577", "postDate": "09/03/2019 08:56:58", "content": "

So... I lost two submissions.

\n\n

\"\"

", "rawMarkdown": "So... I lost two submissions. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2033774%2Fc4d7d3abd6e0c713f17d3044cc7bcc76%2F123.PNG?generation=1567500932622204&alt=media)", "votes": null }, { "id": "616599", "postDate": "09/03/2019 09:18:49", "content": "

This is happening to me as well. lost 3 submissions.

", "rawMarkdown": "This is happening to me as well. lost 3 submissions.", "votes": null }, { "id": "616611", "postDate": "09/03/2019 09:34:27", "content": "

@sohier what's going on? :)

", "rawMarkdown": "sohier what's going on? :)", "votes": null }, { "id": "616778", "postDate": "09/03/2019 13:10:14", "content": "

I lost a number of submissions.

", "rawMarkdown": "I lost a number of submissions.", "votes": null }, { "id": "616804", "postDate": "09/03/2019 13:46:25", "content": "

same here.

", "rawMarkdown": "same here.", "votes": null }, { "id": "616897", "postDate": "09/03/2019 15:07:36", "content": "

well, another two submissions are wasted. Perfect 👍

", "rawMarkdown": "well, another two submissions are wasted. Perfect 👍", "votes": null }, { "id": "616904", "postDate": "09/03/2019 15:15:36", "content": "

I may lost 1 sub (still running). I also lost the power to throw the remaining 3 subs.

", "rawMarkdown": "I may lost 1 sub (still running). I also lost the power to throw the remaining 3 subs.", "votes": null }, { "id": "616906", "postDate": "09/03/2019 15:19:12", "content": "

Why?

", "rawMarkdown": "Why?", "votes": null }, { "id": "616912", "postDate": "09/03/2019 15:27:12", "content": "

he gave up 🙌 :D

", "rawMarkdown": "he gave up 🙌 :D", "votes": null }, { "id": "616922", "postDate": "09/03/2019 15:36:50", "content": "

I have weakened in spirit. I will be fine after sleeping. (Maybe kerneler too)

", "rawMarkdown": "I have weakened in spirit. I will be fine after sleeping. (Maybe kerneler too)", "votes": null }, { "id": "616925", "postDate": "09/03/2019 15:40:02", "content": "

Update: It looks like kernels below 35 minutes (during commit) manage to pass submission. \nI have another kernel which took 42 minutes (during commit), and it is still running for 8 hours (submission). I hardly believe it can make it.

", "rawMarkdown": "Update: It looks like kernels below 35 minutes (during commit) manage to pass submission. \nI have another kernel which took 42 minutes (during commit), and it is still running for 8 hours (submission). I hardly believe it can make it.", "votes": null }, { "id": "616933", "postDate": "09/03/2019 15:51:58", "content": "

With TTA quite hard to get under 35 min, why has this suddenly become an issue?

", "rawMarkdown": "With TTA quite hard to get under 35 min, why has this suddenly become an issue?", "votes": null }, { "id": "616941", "postDate": "09/03/2019 15:57:35", "content": "

I suspect like mentioned below, there is a large overload on Kaggle site, as there are two completions ending this week (APTOS and SIM-ACR). Probably a lot of teams committing and submitting.

", "rawMarkdown": "I suspect like mentioned below, there is a large overload on Kaggle site, as there are two completions ending this week (APTOS and SIM-ACR). Probably a lot of teams committing and submitting.", "votes": null }, { "id": "616949", "postDate": "09/03/2019 16:07:49", "content": "

You are not alone, that's why I hope it will end soon :)

", "rawMarkdown": "You are not alone, that's why I hope it will end soon :)", "votes": null }, { "id": "616977", "postDate": "09/03/2019 16:28:37", "content": "

C´mon Kaggle! \nSo much time lost in this competition due to Kaggle problems..\nIt has been disappointing :(

", "rawMarkdown": "C´mon Kaggle! \nSo much time lost in this competition due to Kaggle problems..\nIt has been disappointing :(", "votes": null }, { "id": "617095", "postDate": "09/03/2019 18:35:30", "content": "

I also lost 2 submission. I resubmitted one without making changes and the second time it worked.

", "rawMarkdown": "I also lost 2 submission. I resubmitted one without making changes and the second time it worked.", "votes": null }, { "id": "617097", "postDate": "09/03/2019 18:37:25", "content": "

@wcukierski , @sohier, this has become ridiculous at this point. My kernel completed running in only 37 minutes yet my submission timed out with an error after 9hrs? And now I have lost a submission. To add salt to injury, no one from Kaggle is telling us why is all this happening? This is more than disappointing with only 2 days left in this competition.

\n\n

How do we even select a final submission when a previously working kernel cannot be submitted successfully after a minor change?

\n\n

We need answers don't you think ... Kaggle team?

", "rawMarkdown": "wcukierski , @sohier, this has become ridiculous at this point. My kernel completed running in only 37 minutes yet my submission timed out with an error after 9hrs? And now I have lost a submission. To add salt to injury, no one from Kaggle is telling us why is all this happening? This is more than disappointing with only 2 days left in this competition.\n\nHow do we even select a final submission when a previously working kernel cannot be submitted successfully after a minor change?\n\nWe need answers don't you think ... Kaggle team?", "votes": null }, { "id": "617120", "postDate": "09/03/2019 18:59:04", "content": "

There is a whole thread from kaggle admin on this at the top.

", "rawMarkdown": "There is a whole thread from kaggle admin on this at the top.", "votes": null }, { "id": "617213", "postDate": "09/03/2019 22:01:30", "content": "

> As several of you have noticed, some kernels are running more slowly than usual. Our engineers are looking into the issue now. I'll post updates once we have more information.

\n\n

@philippsinger , above is all they said in that thread. It did not answer anything about the submission errors. How are we supposed to make our last submissions and then make final selections when most submissions are wasted?

", "rawMarkdown": "> As several of you have noticed, some kernels are running more slowly than usual. Our engineers are looking into the issue now. I'll post updates once we have more information.\n\n@philippsinger , above is all they said in [that thread](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/107323#latest-617204). It did not answer anything about the submission errors. How are we supposed to make our last submissions and then make final selections when most submissions are wasted?", "votes": null }, { "id": "619116", "postDate": "09/05/2019 20:54:30", "content": "

Recent update from the Kaggle team:\n> Update 3 (13:45 PM PST September 5th):

\n\n

> Our engineers located and alleviated an I/O bottleneck that was affecting this competition. New submissions should run at the previously normal speeds, but you may see unexpected timeout errors on currently running submissions as we can't move them off of the congested resource without canceling the run entirely.

", "rawMarkdown": "Recent [update](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/107323#latest-619109) from the Kaggle team:\n> Update 3 (13:45 PM PST September 5th):\n\n> *Our engineers located and alleviated an I/O bottleneck that was affecting this competition. New submissions should run at the previously normal speeds, but you may see unexpected timeout errors on currently running submissions as we can't move them off of the congested resource without canceling the run entirely.*", "votes": null }, { "id": "619346", "postDate": "09/06/2019 05:26:04", "content": "

I am still facing the same issue. Kernel dies after just 2 hours. Very disappointing kaggle !! Lot of our time goes in participating and submitting the solutions. Please fix the issue asap.

", "rawMarkdown": "I am still facing the same issue. Kernel dies after just 2 hours. Very disappointing kaggle !! Lot of our time goes in participating and submitting the solutions. Please fix the issue asap.", "votes": null } ], "comments": [ { "id": 616114, "author_name": "kvlmll", "author_url": "", "post_date": "09/02/2019 17:46:21", "content": "

Same for me. The same kernel was submitted with no problem yesterday (it took ~8 hours)

", "votes": null, "replies": [] }, { "id": 616315, "author_name": "manojprabhaakr", "author_url": "", "post_date": "09/03/2019 02:10:45", "content": "

Yes. I had faced the same issue. The total commit for the kernel was around 40 mins. But on submission, the kernel ran for 9 hours and threw kernel error..

", "votes": null, "replies": [ { "id": 616403, "author_name": "valanm", "author_url": "", "post_date": "09/03/2019 05:35:18", "content": "

Play safe and make it <35 minutes

", "votes": null, "replies": [] }, { "id": 616521, "author_name": "tatianaskv", "author_url": "", "post_date": "09/03/2019 07:55:23", "content": "

I had the same situation yesterday. The commit took 45 minutes and and submission ended after 9 hours with an error. Very disappointing...

", "votes": null, "replies": [] } ] }, { "id": 616395, "author_name": "bibek777", "author_url": "", "post_date": "09/03/2019 05:16:23", "content": "

I believe Kaggle kernels have gone rogue...this is happening in other competitions as well :(

", "votes": null, "replies": [] }, { "id": 616398, "author_name": "kaniikura", "author_url": "", "post_date": "09/03/2019 05:29:23", "content": "

I've encountered the same situation.\nMy Kernel ends within 3 hours so far, but it still has not end after 5 hours.

", "votes": null, "replies": [] }, { "id": 616577, "author_name": "chanhu", "author_url": "", "post_date": "09/03/2019 08:56:58", "content": "

So... I lost two submissions.

\n\n

\"\"

", "votes": null, "replies": [] }, { "id": 616599, "author_name": "higepon", "author_url": "", "post_date": "09/03/2019 09:18:49", "content": "

This is happening to me as well. lost 3 submissions.

", "votes": null, "replies": [] }, { "id": 616611, "author_name": "kvlmll", "author_url": "", "post_date": "09/03/2019 09:34:27", "content": "

@sohier what's going on? :)

", "votes": null, "replies": [] }, { "id": 616778, "author_name": "glaskosk", "author_url": "", "post_date": "09/03/2019 13:10:14", "content": "

I lost a number of submissions.

", "votes": null, "replies": [] }, { "id": 616804, "author_name": "virajbagal", "author_url": "", "post_date": "09/03/2019 13:46:25", "content": "

same here.

", "votes": null, "replies": [] }, { "id": 616897, "author_name": "kvlmll", "author_url": "", "post_date": "09/03/2019 15:07:36", "content": "

well, another two submissions are wasted. Perfect 👍

", "votes": null, "replies": [] }, { "id": 616904, "author_name": "octpath0302", "author_url": "", "post_date": "09/03/2019 15:15:36", "content": "

I may lost 1 sub (still running). I also lost the power to throw the remaining 3 subs.

", "votes": null, "replies": [ { "id": 616906, "author_name": "philippsinger", "author_url": "", "post_date": "09/03/2019 15:19:12", "content": "

Why?

", "votes": null, "replies": [] }, { "id": 616912, "author_name": "kvlmll", "author_url": "", "post_date": "09/03/2019 15:27:12", "content": "

he gave up 🙌 :D

", "votes": null, "replies": [] }, { "id": 616922, "author_name": "octpath0302", "author_url": "", "post_date": "09/03/2019 15:36:50", "content": "

I have weakened in spirit. I will be fine after sleeping. (Maybe kerneler too)

", "votes": null, "replies": [] }, { "id": 616949, "author_name": "philippsinger", "author_url": "", "post_date": "09/03/2019 16:07:49", "content": "

You are not alone, that's why I hope it will end soon :)

", "votes": null, "replies": [] } ] }, { "id": 616925, "author_name": "omershect", "author_url": "", "post_date": "09/03/2019 15:40:02", "content": "

Update: It looks like kernels below 35 minutes (during commit) manage to pass submission. \nI have another kernel which took 42 minutes (during commit), and it is still running for 8 hours (submission). I hardly believe it can make it.

", "votes": null, "replies": [] }, { "id": 616933, "author_name": "overratedgman", "author_url": "", "post_date": "09/03/2019 15:51:58", "content": "

With TTA quite hard to get under 35 min, why has this suddenly become an issue?

", "votes": null, "replies": [] }, { "id": 616941, "author_name": "omershect", "author_url": "", "post_date": "09/03/2019 15:57:35", "content": "

I suspect like mentioned below, there is a large overload on Kaggle site, as there are two completions ending this week (APTOS and SIM-ACR). Probably a lot of teams committing and submitting.

", "votes": null, "replies": [] }, { "id": 616977, "author_name": "carlospk", "author_url": "", "post_date": "09/03/2019 16:28:37", "content": "

C´mon Kaggle! \nSo much time lost in this competition due to Kaggle problems..\nIt has been disappointing :(

", "votes": null, "replies": [] }, { "id": 617095, "author_name": "agentili", "author_url": "", "post_date": "09/03/2019 18:35:30", "content": "

I also lost 2 submission. I resubmitted one without making changes and the second time it worked.

", "votes": null, "replies": [] }, { "id": 617097, "author_name": "sheriytm", "author_url": "", "post_date": "09/03/2019 18:37:25", "content": "

@wcukierski , @sohier, this has become ridiculous at this point. My kernel completed running in only 37 minutes yet my submission timed out with an error after 9hrs? And now I have lost a submission. To add salt to injury, no one from Kaggle is telling us why is all this happening? This is more than disappointing with only 2 days left in this competition.

\n\n

How do we even select a final submission when a previously working kernel cannot be submitted successfully after a minor change?

\n\n

We need answers don't you think ... Kaggle team?

", "votes": null, "replies": [ { "id": 617120, "author_name": "philippsinger", "author_url": "", "post_date": "09/03/2019 18:59:04", "content": "

There is a whole thread from kaggle admin on this at the top.

", "votes": null, "replies": [] }, { "id": 617213, "author_name": "sheriytm", "author_url": "", "post_date": "09/03/2019 22:01:30", "content": "

> As several of you have noticed, some kernels are running more slowly than usual. Our engineers are looking into the issue now. I'll post updates once we have more information.

\n\n

@philippsinger , above is all they said in that thread. It did not answer anything about the submission errors. How are we supposed to make our last submissions and then make final selections when most submissions are wasted?

", "votes": null, "replies": [] } ] }, { "id": 619116, "author_name": "pyotam", "author_url": "", "post_date": "09/05/2019 20:54:30", "content": "

Recent update from the Kaggle team:\n> Update 3 (13:45 PM PST September 5th):

\n\n

> Our engineers located and alleviated an I/O bottleneck that was affecting this competition. New submissions should run at the previously normal speeds, but you may see unexpected timeout errors on currently running submissions as we can't move them off of the congested resource without canceling the run entirely.

", "votes": null, "replies": [] }, { "id": 619346, "author_name": "vishudhjain", "author_url": "", "post_date": "09/06/2019 05:26:04", "content": "

I am still facing the same issue. Kernel dies after just 2 hours. Very disappointing kaggle !! Lot of our time goes in participating and submitting the solutions. Please fix the issue asap.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "616097": "All submission fail today (run for 9 hours and then error)\nAnyone else has the same issue?", "616114": "Same for me. The same kernel was submitted with no problem yesterday (it took ~8 hours)", "616315": "Yes. I had faced the same issue. The total commit for the kernel was around 40 mins. But on submission, the kernel ran for 9 hours and threw kernel error..", "616395": "I believe Kaggle kernels have gone rogue...this is happening in other competitions as well :(", "616398": "I've encountered the same situation.\nMy Kernel ends within 3 hours so far, but it still has not end after 5 hours.", "616403": "Play safe and make it <35 minutes", "616521": "I had the same situation yesterday. The commit took 45 minutes and and submission ended after 9 hours with an error. Very disappointing...", "616577": "So... I lost two submissions. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2033774%2Fc4d7d3abd6e0c713f17d3044cc7bcc76%2F123.PNG?generation=1567500932622204&alt=media)", "616599": "This is happening to me as well. lost 3 submissions.", "616611": "sohier what's going on? :)", "616778": "I lost a number of submissions.", "616804": "same here.", "616897": "well, another two submissions are wasted. Perfect 👍", "616904": "I may lost 1 sub (still running). I also lost the power to throw the remaining 3 subs.", "616906": "Why?", "616912": "he gave up 🙌 :D", "616922": "I have weakened in spirit. I will be fine after sleeping. (Maybe kerneler too)", "616925": "Update: It looks like kernels below 35 minutes (during commit) manage to pass submission. \nI have another kernel which took 42 minutes (during commit), and it is still running for 8 hours (submission). I hardly believe it can make it.", "616933": "With TTA quite hard to get under 35 min, why has this suddenly become an issue?", "616941": "I suspect like mentioned below, there is a large overload on Kaggle site, as there are two completions ending this week (APTOS and SIM-ACR). Probably a lot of teams committing and submitting.", "616949": "You are not alone, that's why I hope it will end soon :)", "616977": "C´mon Kaggle! \nSo much time lost in this competition due to Kaggle problems..\nIt has been disappointing :(", "617095": "I also lost 2 submission. I resubmitted one without making changes and the second time it worked.", "617097": "wcukierski , @sohier, this has become ridiculous at this point. My kernel completed running in only 37 minutes yet my submission timed out with an error after 9hrs? And now I have lost a submission. To add salt to injury, no one from Kaggle is telling us why is all this happening? This is more than disappointing with only 2 days left in this competition.\n\nHow do we even select a final submission when a previously working kernel cannot be submitted successfully after a minor change?\n\nWe need answers don't you think ... Kaggle team?", "617120": "There is a whole thread from kaggle admin on this at the top.", "617213": "> As several of you have noticed, some kernels are running more slowly than usual. Our engineers are looking into the issue now. I'll post updates once we have more information.\n\n@philippsinger , above is all they said in [that thread](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/107323#latest-617204). It did not answer anything about the submission errors. How are we supposed to make our last submissions and then make final selections when most submissions are wasted?", "619116": "Recent [update](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/107323#latest-619109) from the Kaggle team:\n> Update 3 (13:45 PM PST September 5th):\n\n> *Our engineers located and alleviated an I/O bottleneck that was affecting this competition. New submissions should run at the previously normal speeds, but you may see unexpected timeout errors on currently running submissions as we can't move them off of the congested resource without canceling the run entirely.*", "619346": "I am still facing the same issue. Kernel dies after just 2 hours. Very disappointing kaggle !! Lot of our time goes in participating and submitting the solutions. Please fix the issue asap." }, "source": "meta" }