Hi,
\n\nMe and my partner are separately working on creating different models for grapheme, consonant and vowel. I created a final notebook to collate the outputs and make the submission. I am always getting this error:\nNotebook Exceeded Allowed Compute
", "messages": [ { "id": "731964", "postDate": "01/29/2020 09:57:03", "content": "Hi,
\n\nMe and my partner are separately working on creating different models for grapheme, consonant and vowel. I created a final notebook to collate the outputs and make the submission. I am always getting this error:\nNotebook Exceeded Allowed Compute
", "rawMarkdown": "Hi,\n\nMe and my partner are separately working on creating different models for grapheme, consonant and vowel. I created a final notebook to collate the outputs and make the submission. I am always getting this error:\nNotebook Exceeded Allowed Compute", "votes": null }, { "id": "731976", "postDate": "01/29/2020 10:24:25", "content": "That's strange, the final notebook should be pretty light, isn't it? Are you running other notebooks at the same time?
", "rawMarkdown": "That's strange, the final notebook should be pretty light, isn't it? Are you running other notebooks at the same time?", "votes": null }, { "id": "732734", "postDate": "01/30/2020 07:09:53", "content": "Now I am getting 'Submission Error' However my file is in the proper format
", "rawMarkdown": "Now I am getting 'Submission Error' However my file is in the proper format", "votes": null }, { "id": "733849", "postDate": "01/31/2020 15:23:04", "content": "I was doing the same thing and getting the same Notebook Exceeded Allowed Compute error message. I submitted a ticket to Kaggle.
", "rawMarkdown": "I was doing the same thing and getting the same Notebook Exceeded Allowed Compute error message. I submitted a ticket to Kaggle.", "votes": null }, { "id": "734643", "postDate": "02/01/2020 19:10:52", "content": "Hi @souviksbhowmik You can get that error for 2 main reasons. You exceed the GPU or CPU allowed time. That easy to check...if you get the error earlier then the limit..then thats not the error.
\n\nThe other main reasons is memory ( I run into that one..no clear error message but it consistently happened after 20 minutes or so) .. So I suspected memory to be an issue.
\n\nFor the test size a few parquet files needed to be loaded and processed. In my first round of code I loaded a complete test file. Processed it as a whole and then tried to do model.predict for the whole parquet file. For low resolution that can still be done...for higher resolution the kernel memory limit of 16GB is a limit.
\n\nI solved it now by loading the parquet files in a loop.. Within that loop I know process an image and then perform the predict on only that image.\nIt is fast and no more memory issues . give it a try.
", "rawMarkdown": "Hi @souviksbhowmik You can get that error for 2 main reasons. You exceed the GPU or CPU allowed time. That easy to check...if you get the error earlier then the limit..then thats not the error.\n\nThe other main reasons is memory ( I run into that one..no clear error message but it consistently happened after 20 minutes or so) .. So I suspected memory to be an issue.\n\nFor the test size a few parquet files needed to be loaded and processed. In my first round of code I loaded a complete test file. Processed it as a whole and then tried to do model.predict for the whole parquet file. For low resolution that can still be done...for higher resolution the kernel memory limit of 16GB is a limit.\n\nI solved it now by loading the parquet files in a loop.. Within that loop I know process an image and then perform the predict on only that image.\nIt is fast and no more memory issues . give it a try.", "votes": null }, { "id": "740313", "postDate": "02/09/2020 08:15:48", "content": "Hi Matt! Did you get any answer?
", "rawMarkdown": "Hi Matt! Did you get any answer?", "votes": null }, { "id": "740729", "postDate": "02/09/2020 17:47:21", "content": "Yeah, Kaggle said we're on our own and tickets are only for issues EVERYONE is experiencing. Since not everyone is getting this error - in short - figure it out yourself or with your peers lol.
", "rawMarkdown": "Yeah, Kaggle said we're on our own and tickets are only for issues EVERYONE is experiencing. Since not everyone is getting this error - in short - figure it out yourself or with your peers lol.", "votes": null }, { "id": "740732", "postDate": "02/09/2020 17:48:58", "content": "I'm doing all of this already and I'm still getting the error over and over again. What was the resolution of your photos?
", "rawMarkdown": "I'm doing all of this already and I'm still getting the error over and over again. What was the resolution of your photos?", "votes": null }, { "id": "744270", "postDate": "02/12/2020 17:32:34", "content": "I've been having huge issues with this and I couldn't even submit 1's. So far I found that when I go to reshape the data, that seems to be the breaking point for me. I created a public notebook related to my findings.\nhttps://www.kaggle.com/yeayates21/cant-submit-1s-whats-wrong-please-help
", "rawMarkdown": "I've been having huge issues with this and I couldn't even submit 1's. So far I found that when I go to reshape the data, that seems to be the breaking point for me. I created a public notebook related to my findings.\nhttps://www.kaggle.com/yeayates21/cant-submit-1s-whats-wrong-please-help", "votes": null }, { "id": "744300", "postDate": "02/12/2020 17:56:33", "content": "Hi @yeayates21 Sorry for my late response. I used different resolutions. The largest was 128 * 128. I resolved it by looping through the parquet files and then I resize and predict on 1 test image at a time.
\n\nNot the fastest approach...but i is using the least amount of memory. Let me know if I can be of further assistance.
", "rawMarkdown": "Hi @yeayates21 Sorry for my late response. I used different resolutions. The largest was 128 * 128. I resolved it by looping through the parquet files and then I resize and predict on 1 test image at a time.\n\nNot the fastest approach...but i is using the least amount of memory. Let me know if I can be of further assistance.", "votes": null }, { "id": "771817", "postDate": "03/14/2020 16:58:24", "content": "UPDATE: My issue was that I was trying to use 3 models instead of 1. Even loading 1 model at a time was causing issues. Once I switched to using just 1 model, everything worked fine :-/.
", "rawMarkdown": "UPDATE: My issue was that I was trying to use 3 models instead of 1. Even loading 1 model at a time was causing issues. Once I switched to using just 1 model, everything worked fine :-/.", "votes": null } ], "comments": [ { "id": 731976, "author_name": "brunhs", "author_url": "", "post_date": "01/29/2020 10:24:25", "content": "That's strange, the final notebook should be pretty light, isn't it? Are you running other notebooks at the same time?
", "votes": null, "replies": [] }, { "id": 732734, "author_name": "souviksbhowmik", "author_url": "", "post_date": "01/30/2020 07:09:53", "content": "Now I am getting 'Submission Error' However my file is in the proper format
", "votes": null, "replies": [ { "id": 733849, "author_name": "yeayates21", "author_url": "", "post_date": "01/31/2020 15:23:04", "content": "I was doing the same thing and getting the same Notebook Exceeded Allowed Compute error message. I submitted a ticket to Kaggle.
", "votes": null, "replies": [] }, { "id": 740313, "author_name": "raufyagfarov", "author_url": "", "post_date": "02/09/2020 08:15:48", "content": "Hi Matt! Did you get any answer?
", "votes": null, "replies": [] }, { "id": 740729, "author_name": "yeayates21", "author_url": "", "post_date": "02/09/2020 17:47:21", "content": "Yeah, Kaggle said we're on our own and tickets are only for issues EVERYONE is experiencing. Since not everyone is getting this error - in short - figure it out yourself or with your peers lol.
", "votes": null, "replies": [] }, { "id": 771817, "author_name": "yeayates21", "author_url": "", "post_date": "03/14/2020 16:58:24", "content": "UPDATE: My issue was that I was trying to use 3 models instead of 1. Even loading 1 model at a time was causing issues. Once I switched to using just 1 model, everything worked fine :-/.
", "votes": null, "replies": [] } ] }, { "id": 734643, "author_name": "rsmits", "author_url": "", "post_date": "02/01/2020 19:10:52", "content": "Hi @souviksbhowmik You can get that error for 2 main reasons. You exceed the GPU or CPU allowed time. That easy to check...if you get the error earlier then the limit..then thats not the error.
\n\nThe other main reasons is memory ( I run into that one..no clear error message but it consistently happened after 20 minutes or so) .. So I suspected memory to be an issue.
\n\nFor the test size a few parquet files needed to be loaded and processed. In my first round of code I loaded a complete test file. Processed it as a whole and then tried to do model.predict for the whole parquet file. For low resolution that can still be done...for higher resolution the kernel memory limit of 16GB is a limit.
\n\nI solved it now by loading the parquet files in a loop.. Within that loop I know process an image and then perform the predict on only that image.\nIt is fast and no more memory issues . give it a try.
", "votes": null, "replies": [ { "id": 740732, "author_name": "yeayates21", "author_url": "", "post_date": "02/09/2020 17:48:58", "content": "I'm doing all of this already and I'm still getting the error over and over again. What was the resolution of your photos?
", "votes": null, "replies": [] }, { "id": 744300, "author_name": "rsmits", "author_url": "", "post_date": "02/12/2020 17:56:33", "content": "Hi @yeayates21 Sorry for my late response. I used different resolutions. The largest was 128 * 128. I resolved it by looping through the parquet files and then I resize and predict on 1 test image at a time.
\n\nNot the fastest approach...but i is using the least amount of memory. Let me know if I can be of further assistance.
", "votes": null, "replies": [] } ] }, { "id": 744270, "author_name": "yeayates21", "author_url": "", "post_date": "02/12/2020 17:32:34", "content": "I've been having huge issues with this and I couldn't even submit 1's. So far I found that when I go to reshape the data, that seems to be the breaking point for me. I created a public notebook related to my findings.\nhttps://www.kaggle.com/yeayates21/cant-submit-1s-whats-wrong-please-help
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "731964": "Hi,\n\nMe and my partner are separately working on creating different models for grapheme, consonant and vowel. I created a final notebook to collate the outputs and make the submission. I am always getting this error:\nNotebook Exceeded Allowed Compute", "731976": "That's strange, the final notebook should be pretty light, isn't it? Are you running other notebooks at the same time?", "732734": "Now I am getting 'Submission Error' However my file is in the proper format", "733849": "I was doing the same thing and getting the same Notebook Exceeded Allowed Compute error message. I submitted a ticket to Kaggle.", "734643": "Hi @souviksbhowmik You can get that error for 2 main reasons. You exceed the GPU or CPU allowed time. That easy to check...if you get the error earlier then the limit..then thats not the error.\n\nThe other main reasons is memory ( I run into that one..no clear error message but it consistently happened after 20 minutes or so) .. So I suspected memory to be an issue.\n\nFor the test size a few parquet files needed to be loaded and processed. In my first round of code I loaded a complete test file. Processed it as a whole and then tried to do model.predict for the whole parquet file. For low resolution that can still be done...for higher resolution the kernel memory limit of 16GB is a limit.\n\nI solved it now by loading the parquet files in a loop.. Within that loop I know process an image and then perform the predict on only that image.\nIt is fast and no more memory issues . give it a try.", "740313": "Hi Matt! Did you get any answer?", "740729": "Yeah, Kaggle said we're on our own and tickets are only for issues EVERYONE is experiencing. Since not everyone is getting this error - in short - figure it out yourself or with your peers lol.", "740732": "I'm doing all of this already and I'm still getting the error over and over again. What was the resolution of your photos?", "744270": "I've been having huge issues with this and I couldn't even submit 1's. So far I found that when I go to reshape the data, that seems to be the breaking point for me. I created a public notebook related to my findings.\nhttps://www.kaggle.com/yeayates21/cant-submit-1s-whats-wrong-please-help", "744300": "Hi @yeayates21 Sorry for my late response. I used different resolutions. The largest was 128 * 128. I resolved it by looping through the parquet files and then I resize and predict on 1 test image at a time.\n\nNot the fastest approach...but i is using the least amount of memory. Let me know if I can be of further assistance.", "771817": "UPDATE: My issue was that I was trying to use 3 models instead of 1. Even loading 1 model at a time was causing issues. Once I switched to using just 1 model, everything worked fine :-/." }, "source": "meta" }