I tried submitting a kernel several times. Each time, Kaggle claims that the kernel was out of resources. I have tried to make my submissions less complex by lowering batch size, emptying large arrays, etc...
", "messages": [ { "id": 569917, "postDate": "2019-07-07T14:55:56.620Z", "content": "I tried submitting a kernel several times. Each time, Kaggle claims that the kernel was out of resources. I have tried to make my submissions less complex by lowering batch size, emptying large arrays, etc...
", "rawMarkdown": "I tried submitting a kernel several times. Each time, Kaggle claims that the kernel was out of resources. I have tried to make my submissions less complex by lowering batch size, emptying large arrays, etc...", "votes": 10 }, { "id": 585675, "postDate": "2019-07-27T20:46:52.783Z", "content": "I managed to get my program to work! Try to make your code more efficient. \nUsing:\nimport gc\ngc.collect()\nK.clear_session()\netc... can help free up memory space. The method that worked for me is to generate predictions as you convert the test dataset, instead of predicting after converting the entire test dataset:
\n\n```\nsubmission_df = pd.read_csv('../input/aptos2019-blindness-detection/sample_submission.csv')
\n\npredictions = []\nfor i, name in tqdm(enumerate(submission_df['id_code'])):\n path = os.path.join('../input/aptos2019-blindness-detection/test_images/', name+'.png')\n image = cv2.imread(path)\n image = cv2.resize(image, (224, 224), interpolation = cv2.INTER_LANCZOS4)\n image = image.astype('float32')\n image /= 255\n score_predict=model.predict(image.reshape(1, 224, 224, 3))\n label_predict=np.argmax(score_predict) #change this if you're using multilabels\n predictions.append(str(label_predict))
\n\nsubmission_df['diagnosis'] = predictions\nsubmission_df.to_csv('submission.csv', index=False)\n```
\n\nThis way, there is no need to create a huge array containing the entire test dataset.
", "rawMarkdown": "I managed to get my program to work! Try to make your code more efficient. \nUsing:\nimport gc\ngc.collect()\nK.clear_session()\netc... can help free up memory space. The method that worked for me is to generate predictions as you convert the test dataset, instead of predicting after converting the entire test dataset:\n\n```\nsubmission_df = pd.read_csv('../input/aptos2019-blindness-detection/sample_submission.csv') \n\npredictions = []\nfor i, name in tqdm(enumerate(submission_df['id_code'])):\n path = os.path.join('../input/aptos2019-blindness-detection/test_images/', name+'.png')\n image = cv2.imread(path)\n image = cv2.resize(image, (224, 224), interpolation = cv2.INTER_LANCZOS4)\n image = image.astype('float32')\n image /= 255\n score_predict=model.predict(image.reshape(1, 224, 224, 3))\n label_predict=np.argmax(score_predict) #change this if you're using multilabels\n predictions.append(str(label_predict))\n\nsubmission_df['diagnosis'] = predictions\nsubmission_df.to_csv('submission.csv', index=False)\n```\n\nThis way, there is no need to create a huge array containing the entire test dataset.", "votes": 4 }, { "id": 571644, "postDate": "2019-07-09T22:57:47.143Z", "content": "Yeah the same problem!
", "rawMarkdown": "Yeah the same problem!", "votes": 1 }, { "id": 571318, "postDate": "2019-07-09T13:44:29.587Z", "content": "I am having the same problem, since I've implemented TTA for my predictions. Previous kernels were submitted successfully, despite data augmentation for the training set. However, as soon as I wanted to do it with the test set (in spite of deleting the data augmenter, at the end of every loop and successful committing....) I wouldn't work anymore. \nIt seems that the Kernel, on which our code is tested, has less RAM available than on the ones provided to us by kaggle.
", "rawMarkdown": "I am having the same problem, since I've implemented TTA for my predictions. Previous kernels were submitted successfully, despite data augmentation for the training set. However, as soon as I wanted to do it with the test set (in spite of deleting the data augmenter, at the end of every loop and successful committing....) I wouldn't work anymore. \nIt seems that the Kernel, on which our code is tested, has less RAM available than on the ones provided to us by kaggle.", "votes": 1 }, { "id": 569963, "postDate": "2019-07-07T15:51:56.563Z", "content": "Are you running out of resources during training or during the predicting part? Also, what methods are you using to fit your model, if the issue is occurring during the training phase?
\n\nI too was having issues with resources (granted, I'm now looking at some cloud GPU providers to train outside of here), but using Keras's .flow_from_dataframe() helped me get the training done in the kernel.
", "rawMarkdown": "Are you running out of resources during training or during the predicting part? Also, what methods are you using to fit your model, if the issue is occurring during the training phase?\n\nI too was having issues with resources (granted, I'm now looking at some cloud GPU providers to train outside of here), but using Keras's .flow_from_dataframe() helped me get the training done in the kernel.", "votes": 1, "replies": [ { "id": 570111, "postDate": "2019-07-07T19:25:34.227Z", "content": "Committing and running normally seem to work, however, the error appears once I submit it. I can't seem to tell when it's running out of resources. Is there any way to tell when the submission is running out of resources?
", "rawMarkdown": "Committing and running normally seem to work, however, the error appears once I submit it. I can't seem to tell when it's running out of resources. Is there any way to tell when the submission is running out of resources?", "votes": 1 } ] }, { "id": 569943, "postDate": "2019-07-07T15:29:06.900Z", "content": "Me too....
", "rawMarkdown": "Me too....", "votes": 1 }, { "id": 570441, "postDate": "2019-07-08T09:47:24.463Z", "content": "Yes I'm facing the same issue!
", "rawMarkdown": "Yes I'm facing the same issue!\n", "votes": 2 }, { "id": 569920, "postDate": "2019-07-07T15:04:10.007Z", "content": "Silly as it sounds, did you try rebooting the session? Had the same problem, and eventually in my desperation i tried that :-) the legacy of Windows lives on...
", "rawMarkdown": "Silly as it sounds, did you try rebooting the session? Had the same problem, and eventually in my desperation i tried that :-) the legacy of Windows lives on...", "votes": 2, "replies": [ { "id": 570113, "postDate": "2019-07-07T19:30:29.813Z", "content": "There is no error when running the kernel normally or when comitting it. When I submit the kernel, along with the submission.csv, Kaggle claims that the kernel was out of resources. I made new kernels with the same code, and there is the same result. I even tried uploading a model that I trained using the exact dataset on my actual computer.
", "rawMarkdown": "There is no error when running the kernel normally or when comitting it. When I submit the kernel, along with the submission.csv, Kaggle claims that the kernel was out of resources. I made new kernels with the same code, and there is the same result. I even tried uploading a model that I trained using the exact dataset on my actual computer.", "votes": 1 } ] }, { "id": 583223, "postDate": "2019-07-24T07:17:34.337Z", "content": "I'm having the same issue, last 9 of my attempts to upload submission.csv to competition have errored out showing \"Kernel out of resources\". Someone from Kaggle team needs to provide a clarity here, its wasting our time and effort big time...
", "rawMarkdown": "I'm having the same issue, last 9 of my attempts to upload submission.csv to competition have errored out showing \"Kernel out of resources\". Someone from Kaggle team needs to provide a clarity here, its wasting our time and effort big time...", "votes": -1 }, { "id": 614448, "postDate": "2019-08-31T13:55:38.140Z", "content": "In my case, after crick the \"Submit to Competition\", then begin the calculation of the \"Public Score\", after few minutes, appeared the message of \"Our workers are spending a little extra time on your submission. Grab a coffee and refresh later to see your score.\". After one or two hours, status change from \"Kernel Running\" to \"Kernel out of resources\" and at the same time it appeared \"Error\". This symptom has already occurred more than 25 times continuously. So I don't have a score yet, and of course I'm not on the leader board.
", "rawMarkdown": "In my case, after crick the \"Submit to Competition\", then begin the calculation of the \"Public Score\", after few minutes, appeared the message of \"Our workers are spending a little extra time on your submission. Grab a coffee and refresh later to see your score.\". After one or two hours, status change from \"Kernel Running\" to \"Kernel out of resources\" and at the same time it appeared \"Error\". This symptom has already occurred more than 25 times continuously. So I don't have a score yet, and of course I'm not on the leader board. " }, { "id": 585654, "postDate": "2019-07-27T20:03:18.553Z", "content": "Same problem. How to resolve this issue?
", "rawMarkdown": "Same problem. How to resolve this issue?" }, { "id": 579478, "postDate": "2019-07-18T21:36:46.203Z", "content": "any new update on how to fix this problem guys? Thanks!
", "rawMarkdown": "any new update on how to fix this problem guys? Thanks!" }, { "id": 588075, "postDate": "2019-07-30T05:49:31.033Z", "rawMarkdown": "", "isDeleted": true }, { "id": 586343, "postDate": "2019-07-29T03:05:22.910Z", "rawMarkdown": "", "isDeleted": true }, { "id": 570670, "postDate": "2019-07-08T15:50:49.500Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 570980, "postDate": "2019-07-09T02:11:39.953Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 585675, "author_name": "David Liang", "author_url": "", "post_date": "2019-07-27T20:46:52.783000", "content": "I managed to get my program to work! Try to make your code more efficient. \nUsing:\nimport gc\ngc.collect()\nK.clear_session()\netc... can help free up memory space. The method that worked for me is to generate predictions as you convert the test dataset, instead of predicting after converting the entire test dataset:
\n\n```\nsubmission_df = pd.read_csv('../input/aptos2019-blindness-detection/sample_submission.csv')
\n\npredictions = []\nfor i, name in tqdm(enumerate(submission_df['id_code'])):\n path = os.path.join('../input/aptos2019-blindness-detection/test_images/', name+'.png')\n image = cv2.imread(path)\n image = cv2.resize(image, (224, 224), interpolation = cv2.INTER_LANCZOS4)\n image = image.astype('float32')\n image /= 255\n score_predict=model.predict(image.reshape(1, 224, 224, 3))\n label_predict=np.argmax(score_predict) #change this if you're using multilabels\n predictions.append(str(label_predict))
\n\nsubmission_df['diagnosis'] = predictions\nsubmission_df.to_csv('submission.csv', index=False)\n```
\n\nThis way, there is no need to create a huge array containing the entire test dataset.
", "votes": 4, "replies": [] }, { "id": 571644, "author_name": "Heqiao Ruan", "author_url": "", "post_date": "2019-07-09T22:57:47.143000", "content": "Yeah the same problem!
", "votes": 1, "replies": [] }, { "id": 571318, "author_name": "Claudio Fanconi", "author_url": "", "post_date": "2019-07-09T13:44:29.587000", "content": "I am having the same problem, since I've implemented TTA for my predictions. Previous kernels were submitted successfully, despite data augmentation for the training set. However, as soon as I wanted to do it with the test set (in spite of deleting the data augmenter, at the end of every loop and successful committing....) I wouldn't work anymore. \nIt seems that the Kernel, on which our code is tested, has less RAM available than on the ones provided to us by kaggle.
", "votes": 1, "replies": [] }, { "id": 569963, "author_name": "BenjaminE", "author_url": "", "post_date": "2019-07-07T15:51:56.563000", "content": "Are you running out of resources during training or during the predicting part? Also, what methods are you using to fit your model, if the issue is occurring during the training phase?
\n\nI too was having issues with resources (granted, I'm now looking at some cloud GPU providers to train outside of here), but using Keras's .flow_from_dataframe() helped me get the training done in the kernel.
", "votes": 1, "replies": [ { "id": 570111, "author_name": "David Liang", "author_url": "", "post_date": "2019-07-07T19:25:34.227000", "content": "Committing and running normally seem to work, however, the error appears once I submit it. I can't seem to tell when it's running out of resources. Is there any way to tell when the submission is running out of resources?
", "votes": 1, "replies": [] } ] }, { "id": 569943, "author_name": "makogarei", "author_url": "", "post_date": "2019-07-07T15:29:06.900000", "content": "Me too....
", "votes": 1, "replies": [] }, { "id": 570441, "author_name": "Revathi Vijay", "author_url": "", "post_date": "2019-07-08T09:47:24.463000", "content": "Yes I'm facing the same issue!
", "votes": 2, "replies": [] }, { "id": 569920, "author_name": "Konrad Banachewicz", "author_url": "", "post_date": "2019-07-07T15:04:10.007000", "content": "Silly as it sounds, did you try rebooting the session? Had the same problem, and eventually in my desperation i tried that :-) the legacy of Windows lives on...
", "votes": 2, "replies": [ { "id": 570113, "author_name": "David Liang", "author_url": "", "post_date": "2019-07-07T19:30:29.813000", "content": "There is no error when running the kernel normally or when comitting it. When I submit the kernel, along with the submission.csv, Kaggle claims that the kernel was out of resources. I made new kernels with the same code, and there is the same result. I even tried uploading a model that I trained using the exact dataset on my actual computer.
", "votes": 1, "replies": [] } ] }, { "id": 583223, "author_name": "Amir Ashraff", "author_url": "", "post_date": "2019-07-24T07:17:34.337000", "content": "I'm having the same issue, last 9 of my attempts to upload submission.csv to competition have errored out showing \"Kernel out of resources\". Someone from Kaggle team needs to provide a clarity here, its wasting our time and effort big time...
", "votes": -1, "replies": [] }, { "id": 614448, "author_name": "Kaoru Sasakawa", "author_url": "", "post_date": "2019-08-31T13:55:38.140000", "content": "In my case, after crick the \"Submit to Competition\", then begin the calculation of the \"Public Score\", after few minutes, appeared the message of \"Our workers are spending a little extra time on your submission. Grab a coffee and refresh later to see your score.\". After one or two hours, status change from \"Kernel Running\" to \"Kernel out of resources\" and at the same time it appeared \"Error\". This symptom has already occurred more than 25 times continuously. So I don't have a score yet, and of course I'm not on the leader board.
", "votes": 0, "replies": [] }, { "id": 585654, "author_name": "Rama@742", "author_url": "", "post_date": "2019-07-27T20:03:18.553000", "content": "Same problem. How to resolve this issue?
", "votes": 0, "replies": [] }, { "id": 579478, "author_name": "Vinh Ta", "author_url": "", "post_date": "2019-07-18T21:36:46.203000", "content": "any new update on how to fix this problem guys? Thanks!
", "votes": 0, "replies": [] }, { "id": 588075, "author_name": "", "author_url": "", "post_date": "2019-07-30T05:49:31.033000", "content": "", "votes": 0, "replies": [] }, { "id": 586343, "author_name": "", "author_url": "", "post_date": "2019-07-29T03:05:22.910000", "content": "", "votes": 0, "replies": [] }, { "id": 570670, "author_name": "", "author_url": "", "post_date": "2019-07-08T15:50:49.500000", "content": "", "votes": 0, "replies": [ { "id": 570980, "author_name": "", "author_url": "", "post_date": "2019-07-09T02:11:39.953000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "569917": "I tried submitting a kernel several times. Each time, Kaggle claims that the kernel was out of resources. I have tried to make my submissions less complex by lowering batch size, emptying large arrays, etc...", "585675": "I managed to get my program to work! Try to make your code more efficient. \nUsing:\nimport gc\ngc.collect()\nK.clear_session()\netc... can help free up memory space. The method that worked for me is to generate predictions as you convert the test dataset, instead of predicting after converting the entire test dataset:\n\n```\nsubmission_df = pd.read_csv('../input/aptos2019-blindness-detection/sample_submission.csv') \n\npredictions = []\nfor i, name in tqdm(enumerate(submission_df['id_code'])):\n path = os.path.join('../input/aptos2019-blindness-detection/test_images/', name+'.png')\n image = cv2.imread(path)\n image = cv2.resize(image, (224, 224), interpolation = cv2.INTER_LANCZOS4)\n image = image.astype('float32')\n image /= 255\n score_predict=model.predict(image.reshape(1, 224, 224, 3))\n label_predict=np.argmax(score_predict) #change this if you're using multilabels\n predictions.append(str(label_predict))\n\nsubmission_df['diagnosis'] = predictions\nsubmission_df.to_csv('submission.csv', index=False)\n```\n\nThis way, there is no need to create a huge array containing the entire test dataset.", "571644": "Yeah the same problem!", "571318": "I am having the same problem, since I've implemented TTA for my predictions. Previous kernels were submitted successfully, despite data augmentation for the training set. However, as soon as I wanted to do it with the test set (in spite of deleting the data augmenter, at the end of every loop and successful committing....) I wouldn't work anymore. \nIt seems that the Kernel, on which our code is tested, has less RAM available than on the ones provided to us by kaggle.", "569963": "Are you running out of resources during training or during the predicting part? Also, what methods are you using to fit your model, if the issue is occurring during the training phase?\n\nI too was having issues with resources (granted, I'm now looking at some cloud GPU providers to train outside of here), but using Keras's .flow_from_dataframe() helped me get the training done in the kernel.", "569943": "Me too....", "570441": "Yes I'm facing the same issue!\n", "569920": "Silly as it sounds, did you try rebooting the session? Had the same problem, and eventually in my desperation i tried that :-) the legacy of Windows lives on...", "583223": "I'm having the same issue, last 9 of my attempts to upload submission.csv to competition have errored out showing \"Kernel out of resources\". Someone from Kaggle team needs to provide a clarity here, its wasting our time and effort big time...", "614448": "In my case, after crick the \"Submit to Competition\", then begin the calculation of the \"Public Score\", after few minutes, appeared the message of \"Our workers are spending a little extra time on your submission. Grab a coffee and refresh later to see your score.\". After one or two hours, status change from \"Kernel Running\" to \"Kernel out of resources\" and at the same time it appeared \"Error\". This symptom has already occurred more than 25 times continuously. So I don't have a score yet, and of course I'm not on the leader board. ", "585654": "Same problem. How to resolve this issue?", "579478": "any new update on how to fix this problem guys? Thanks!", "588075": "", "586343": "", "570670": "" } }