Brief description of my problem:
\nSome more information:
\nThe part where i am making the predictions and creating the segmentation masks is based on the notebook linked above. I have only replaced the \"dummy masks\" of the sample submission with the masks generated by my model. The routine of generating the masks together with the prediction string works without any error on the public training data.
I'm using GPU T4x2 for my notebook. Only the first GPU is used for training, while the second GPU is used for prediction+submission. This means i have ~14GB of RAM for the test data set. Could it really be the case that i am still running out of memory while looping through the test data?
\nAnother question: Are all the test images 512x512, just like the train images?
\nI would appreciate any feedback and any guidance on how to fix this issue!
", "messages": [ { "id": "2328631", "postDate": "07/03/2023 18:14:15", "content": "Brief description of my problem:
\nSome more information:
\nThe part where i am making the predictions and creating the segmentation masks is based on the notebook linked above. I have only replaced the \"dummy masks\" of the sample submission with the masks generated by my model. The routine of generating the masks together with the prediction string works without any error on the public training data.
I'm using GPU T4x2 for my notebook. Only the first GPU is used for training, while the second GPU is used for prediction+submission. This means i have ~14GB of RAM for the test data set. Could it really be the case that i am still running out of memory while looping through the test data?
\nAnother question: Are all the test images 512x512, just like the train images?
\nI would appreciate any feedback and any guidance on how to fix this issue!
", "rawMarkdown": "Brief description of my problem: \n- My notebook runs without any errors on the public dataset \n- The \"submission part\" is based on the sample submission given in the following notebook : https://www.kaggle.com/code/michalaffek/sample-submission-dummy-seg-score-0-000 . This notebook works fine and i am able to submit it without error\n- However: Upon submission of my code (not the sample submission), i get the following error message: \"Notebook threw exception\" \n- Additionally, the following information is given: \"Your notebook hit an unhandled error while rerunning your code. Note that the hidden dataset can be larger/smaller/different than the public dataset See more debugging tips\" \n\nSome more information: \nThe part where i am making the predictions and creating the segmentation masks is based on the notebook linked above. I have only replaced the \"dummy masks\" of the sample submission with the masks generated by my model. The routine of generating the masks together with the prediction string works without any error on the public training data. \n\nI'm using GPU T4x2 for my notebook. Only the first GPU is used for training, while the second GPU is used for prediction+submission. This means i have ~14GB of RAM for the test data set. **Could it really be the case that i am still running out of memory while looping through the test data?** \n\nAnother question: **Are all the test images 512x512, just like the train images?**\n\nI would appreciate any feedback and any guidance on how to fix this issue!", "votes": null }, { "id": "2330150", "postDate": "07/04/2023 18:41:30", "content": "Hey! Did you check the data you're using? I guess you need to check out on the private data being assumed. You may also check out this discussion: https://www.kaggle.com/discussions/product-feedback/121112
", "rawMarkdown": "Hey! Did you check the data you're using? I guess you need to check out on the private data being assumed. You may also check out this discussion: https://www.kaggle.com/discussions/product-feedback/121112", "votes": null }, { "id": "2330181", "postDate": "07/04/2023 19:23:30", "content": "Thank you so much for linking that discussion!
\nCan you clarify what you mean by \"checking the data you're using\"? I have tested the program on the public training data and it works without issues there. However, i of course can't test it on the private test data.
It was from the discussion, it says that \"Notebook Threw Exception\" could be due to assumptions made on the private data.
", "rawMarkdown": "It was from the discussion, it says that \"Notebook Threw Exception\" could be due to assumptions made on the private data.", "votes": null }, { "id": "2330425", "postDate": "07/05/2023 02:01:53", "content": "The test data is similarly 512x512.
\n{train|test}/ Folders containing TIFF images of the tiles. Each tile is 512x512 in size.
", "rawMarkdown": "The test data is similarly 512x512.\n\n{train|test}/ Folders containing TIFF images of the tiles. Each tile is 512x512 in size.", "votes": null }, { "id": "2330778", "postDate": "07/05/2023 07:11:57", "content": "Thank you for the response. Then it must really be a memory issue, which i find surprising, given that i allocate ~14GB of memory purely for the prediction & submission. The training also runs fine without OOM errors.
", "rawMarkdown": "Thank you for the response. Then it must really be a memory issue, which i find surprising, given that i allocate ~14GB of memory purely for the prediction & submission. The training also runs fine without OOM errors.", "votes": null } ], "comments": [ { "id": 2330150, "author_name": "kaggleprollc", "author_url": "", "post_date": "07/04/2023 18:41:30", "content": "Hey! Did you check the data you're using? I guess you need to check out on the private data being assumed. You may also check out this discussion: https://www.kaggle.com/discussions/product-feedback/121112
", "votes": null, "replies": [ { "id": 2330181, "author_name": "felixkonrad", "author_url": "", "post_date": "07/04/2023 19:23:30", "content": "Thank you so much for linking that discussion!
\nCan you clarify what you mean by \"checking the data you're using\"? I have tested the program on the public training data and it works without issues there. However, i of course can't test it on the private test data.
It was from the discussion, it says that \"Notebook Threw Exception\" could be due to assumptions made on the private data.
", "votes": null, "replies": [] } ] } ] }, { "id": 2330425, "author_name": "mitsuyasuhoshino", "author_url": "", "post_date": "07/05/2023 02:01:53", "content": "The test data is similarly 512x512.
\n{train|test}/ Folders containing TIFF images of the tiles. Each tile is 512x512 in size.
", "votes": null, "replies": [ { "id": 2330778, "author_name": "felixkonrad", "author_url": "", "post_date": "07/05/2023 07:11:57", "content": "Thank you for the response. Then it must really be a memory issue, which i find surprising, given that i allocate ~14GB of memory purely for the prediction & submission. The training also runs fine without OOM errors.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2328631": "Brief description of my problem: \n- My notebook runs without any errors on the public dataset \n- The \"submission part\" is based on the sample submission given in the following notebook : https://www.kaggle.com/code/michalaffek/sample-submission-dummy-seg-score-0-000 . This notebook works fine and i am able to submit it without error\n- However: Upon submission of my code (not the sample submission), i get the following error message: \"Notebook threw exception\" \n- Additionally, the following information is given: \"Your notebook hit an unhandled error while rerunning your code. Note that the hidden dataset can be larger/smaller/different than the public dataset See more debugging tips\" \n\nSome more information: \nThe part where i am making the predictions and creating the segmentation masks is based on the notebook linked above. I have only replaced the \"dummy masks\" of the sample submission with the masks generated by my model. The routine of generating the masks together with the prediction string works without any error on the public training data. \n\nI'm using GPU T4x2 for my notebook. Only the first GPU is used for training, while the second GPU is used for prediction+submission. This means i have ~14GB of RAM for the test data set. **Could it really be the case that i am still running out of memory while looping through the test data?** \n\nAnother question: **Are all the test images 512x512, just like the train images?**\n\nI would appreciate any feedback and any guidance on how to fix this issue!", "2330150": "Hey! Did you check the data you're using? I guess you need to check out on the private data being assumed. You may also check out this discussion: https://www.kaggle.com/discussions/product-feedback/121112", "2330181": "Thank you so much for linking that discussion! \nCan you clarify what you mean by \"checking the data you're using\"? I have tested the program on the public training data and it works without issues there. However, i of course can't test it on the private test data.", "2330368": "It was from the discussion, it says that \"Notebook Threw Exception\" could be due to assumptions made on the private data.", "2330425": "The test data is similarly 512x512.\n\n{train|test}/ Folders containing TIFF images of the tiles. Each tile is 512x512 in size.", "2330778": "Thank you for the response. Then it must really be a memory issue, which i find surprising, given that i allocate ~14GB of memory purely for the prediction & submission. The training also runs fine without OOM errors." }, "source": "meta" }