I tried several times already but the my submission notebook isn't working somehow and they can't find my submission-file. I dont think that it has anything to do with the fact that the public test-file is only one instance bc I also ran my notebook with the training data and it worked. But now when I submit the notebook it runs for about 2h and then gives me the error mentioned above.
\nthis is how I submit my data:
\ndef convert_float(image,label):
\nreturn tf.cast(image,tf.float32),label
test_data = test_data.map(convert_float)
\ntest_images_data = test_data.map(lambda image,label: image)
test_pred_prob = model.predict(test_images_data)
test_pred = test_pred_prob.argmax(axis=1)
test_data_ids = [ids.numpy() for (image,ids) in test_data][0]
\ntest_data_ids = [i.decode() for i in test_data_ids]
submission_df = pd.DataFrame({\"image_id\":test_data_ids,\"label\":test_pred})
\nsubmission_df.to_csv(\"submission.csv\",index=False)
if anybody knows what's going and could tell me that would be much appreciated.Thanks🙌🏻
", "messages": [ { "id": "1122834", "postDate": "12/22/2020 18:26:38", "content": "I tried several times already but the my submission notebook isn't working somehow and they can't find my submission-file. I dont think that it has anything to do with the fact that the public test-file is only one instance bc I also ran my notebook with the training data and it worked. But now when I submit the notebook it runs for about 2h and then gives me the error mentioned above.
\nthis is how I submit my data:
\ndef convert_float(image,label):
\nreturn tf.cast(image,tf.float32),label
test_data = test_data.map(convert_float)
\ntest_images_data = test_data.map(lambda image,label: image)
test_pred_prob = model.predict(test_images_data)
test_pred = test_pred_prob.argmax(axis=1)
test_data_ids = [ids.numpy() for (image,ids) in test_data][0]
\ntest_data_ids = [i.decode() for i in test_data_ids]
submission_df = pd.DataFrame({\"image_id\":test_data_ids,\"label\":test_pred})
\nsubmission_df.to_csv(\"submission.csv\",index=False)
if anybody knows what's going and could tell me that would be much appreciated.Thanks🙌🏻
", "rawMarkdown": "I tried several times already but the my submission notebook isn't working somehow and they can't find my submission-file. I dont think that it has anything to do with the fact that the public test-file is only one instance bc I also ran my notebook with the training data and it worked. But now when I submit the notebook it runs for about 2h and then gives me the error mentioned above.\nthis is how I submit my data:\n`def convert_float(image,label):`\n` return tf.cast(image,tf.float32),label`\n\n`test_data = test_data.map(convert_float)`\n`test_images_data = test_data.map(lambda image,label: image)`\n\n`test_pred_prob = model.predict(test_images_data)`\n\n`test_pred = test_pred_prob.argmax(axis=1)`\n\n`test_data_ids = [ids.numpy() for (image,ids) in test_data][0]`\n`test_data_ids = [i.decode() for i in test_data_ids]`\n\n`submission_df = pd.DataFrame({\"image_id\":test_data_ids,\"label\":test_pred})`\n`submission_df.to_csv(\"submission.csv\",index=False)`\n\nif anybody knows what's going and could tell me that would be much appreciated.Thanks🙌🏻", "votes": null }, { "id": "1123069", "postDate": "12/22/2020 21:49:21", "content": "if the submission fails to run to completion, it could display that error. One reason could be running out of memory. There could be other reasons too.
", "rawMarkdown": "if the submission fails to run to completion, it could display that error. One reason could be running out of memory. There could be other reasons too.", "votes": null }, { "id": "1123090", "postDate": "12/22/2020 22:24:47", "content": "is there anything to prevent that from happening?
", "rawMarkdown": "is there anything to prevent that from happening?", "votes": null }, { "id": "1123235", "postDate": "12/23/2020 04:18:21", "content": "For one, you can use a smaller model or a smaller resolution for your images. For example, use VGG and see if your submission passes or use an image size of 224x224. If memory is an issue, then your submission will most likely pass but it may not score well. Then you can start going to bigger models, and or higher resolutions. Also, a big value for batch size could cause this. You may also want to play around with the batch size. start at say the batch size of 8 and go up…
", "rawMarkdown": "For one, you can use a smaller model or a smaller resolution for your images. For example, use VGG and see if your submission passes or use an image size of 224x224. If memory is an issue, then your submission will most likely pass but it may not score well. Then you can start going to bigger models, and or higher resolutions. Also, a big value for batch size could cause this. You may also want to play around with the batch size. start at say the batch size of 8 and go up...", "votes": null }, { "id": "1136791", "postDate": "01/03/2021 12:36:02", "content": "Do you know if it might help if I enable the gpu?
", "rawMarkdown": "Do you know if it might help if I enable the gpu?", "votes": null }, { "id": "1138463", "postDate": "01/04/2021 17:48:01", "content": "yes, with GPU your code runs much faster and you would not hit the 9-hour limit. Also, save and run your code and see if submission.csv is created. It should only have one entry in it.
\nimage_id label
\n2216849948.jpg 4
yeah it works fine on the single instance
", "rawMarkdown": "yeah it works fine on the single instance", "votes": null }, { "id": "1144699", "postDate": "01/08/2021 16:06:44", "content": "it works now. I changed the way it gets the image_ids
", "rawMarkdown": "it works now. I changed the way it gets the image_ids", "votes": null } ], "comments": [ { "id": 1123069, "author_name": "kamranmo", "author_url": "", "post_date": "12/22/2020 21:49:21", "content": "if the submission fails to run to completion, it could display that error. One reason could be running out of memory. There could be other reasons too.
", "votes": null, "replies": [ { "id": 1123090, "author_name": "docphilipp", "author_url": "", "post_date": "12/22/2020 22:24:47", "content": "is there anything to prevent that from happening?
", "votes": null, "replies": [] }, { "id": 1123235, "author_name": "kamranmo", "author_url": "", "post_date": "12/23/2020 04:18:21", "content": "For one, you can use a smaller model or a smaller resolution for your images. For example, use VGG and see if your submission passes or use an image size of 224x224. If memory is an issue, then your submission will most likely pass but it may not score well. Then you can start going to bigger models, and or higher resolutions. Also, a big value for batch size could cause this. You may also want to play around with the batch size. start at say the batch size of 8 and go up…
", "votes": null, "replies": [] }, { "id": 1136791, "author_name": "docphilipp", "author_url": "", "post_date": "01/03/2021 12:36:02", "content": "Do you know if it might help if I enable the gpu?
", "votes": null, "replies": [] }, { "id": 1138463, "author_name": "kamranmo", "author_url": "", "post_date": "01/04/2021 17:48:01", "content": "yes, with GPU your code runs much faster and you would not hit the 9-hour limit. Also, save and run your code and see if submission.csv is created. It should only have one entry in it.
\nimage_id label
\n2216849948.jpg 4
yeah it works fine on the single instance
", "votes": null, "replies": [] }, { "id": 1144699, "author_name": "docphilipp", "author_url": "", "post_date": "01/08/2021 16:06:44", "content": "it works now. I changed the way it gets the image_ids
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1122834": "I tried several times already but the my submission notebook isn't working somehow and they can't find my submission-file. I dont think that it has anything to do with the fact that the public test-file is only one instance bc I also ran my notebook with the training data and it worked. But now when I submit the notebook it runs for about 2h and then gives me the error mentioned above.\nthis is how I submit my data:\n`def convert_float(image,label):`\n` return tf.cast(image,tf.float32),label`\n\n`test_data = test_data.map(convert_float)`\n`test_images_data = test_data.map(lambda image,label: image)`\n\n`test_pred_prob = model.predict(test_images_data)`\n\n`test_pred = test_pred_prob.argmax(axis=1)`\n\n`test_data_ids = [ids.numpy() for (image,ids) in test_data][0]`\n`test_data_ids = [i.decode() for i in test_data_ids]`\n\n`submission_df = pd.DataFrame({\"image_id\":test_data_ids,\"label\":test_pred})`\n`submission_df.to_csv(\"submission.csv\",index=False)`\n\nif anybody knows what's going and could tell me that would be much appreciated.Thanks🙌🏻", "1123069": "if the submission fails to run to completion, it could display that error. One reason could be running out of memory. There could be other reasons too.", "1123090": "is there anything to prevent that from happening?", "1123235": "For one, you can use a smaller model or a smaller resolution for your images. For example, use VGG and see if your submission passes or use an image size of 224x224. If memory is an issue, then your submission will most likely pass but it may not score well. Then you can start going to bigger models, and or higher resolutions. Also, a big value for batch size could cause this. You may also want to play around with the batch size. start at say the batch size of 8 and go up...", "1136791": "Do you know if it might help if I enable the gpu?", "1138463": "yes, with GPU your code runs much faster and you would not hit the 9-hour limit. Also, save and run your code and see if submission.csv is created. It should only have one entry in it. \nimage_id label\n2216849948.jpg 4", "1144505": "yeah it works fine on the single instance", "1144699": "it works now. I changed the way it gets the image_ids" }, "source": "meta" }