After 2 days of painstaking debugging, I've created a checklist for newcomers who want to avoid the pain that I have to go through creating the first submission:
\nI hope these things may help you.
\nCheers!
", "messages": [ { "id": "1852272", "postDate": "07/12/2022 01:01:45", "content": "After 2 days of painstaking debugging, I've created a checklist for newcomers who want to avoid the pain that I have to go through creating the first submission:
\nI hope these things may help you.
\nCheers!
", "rawMarkdown": "After 2 days of painstaking debugging, I've created a checklist for newcomers who want to avoid the pain that I have to go through creating the first submission:\n1. Try inference on training set first to debug memory error. Use gc.collects() (not recommended) or check if you need to clean up garbage in the event memory overflow occurs.\n2. Make sure to **read the data description carefully**. For example: \"The HuBMAP images range in size from 4500x4500 down to 160x160 pixels\". My self-implemented sliding window inference did not account for this and it failed miserably.\n3. Submission file should be named \"submission.csv\" and saved with index=False\n4. Inference on the test set **should not exceed 9 hours** or 3 hours even for a single model. Again you can debug this using training set to alleviate inefficiency like running sequentially instead of parallel, etc...\n\nI hope these things may help you.\n\nCheers!", "votes": null }, { "id": "1855831", "postDate": "07/15/2022 00:58:26", "content": "Thanks for sharing your tips!
\nI wanted to add a couple of things that have caused issues for me in the past:
\nThanks again for sharing your tips!
", "rawMarkdown": "Thanks for sharing your tips!\n\nI wanted to add a couple of things that have caused issues for me in the past:\n\n5. Make sure your data types match the expected data types in the description. I've had issues in the past where my submission has been rejected because my data types didn't match (e.g. I had integers when it was expecting floats)\n6. Check the competition rules carefully! Sometimes there are specific formatting requirements for the submission that are not made clear in the description (e.g. specifying the column names in a certain order)\n7. If you're using external data, make sure you've read the terms and conditions carefully. Some competitions don't allow the use of external data, or they may have specific requirements about how that data can be used.\n\nThanks again for sharing your tips!", "votes": null }, { "id": "1858016", "postDate": "07/16/2022 15:28:37", "content": "@thedevastator thank you for creating all the datasets. I have used your 512x512 dataset and locally trained a model on my machine. My local CV is fine, and the validation predictions look fine too. However, when I use your inference code to make a submission it produces empty predictions. The public LB is also 0.0. Any idea what is wrong?
This does not happen with 256x256 input size. For 512x512, I set sz = 512, and reduce = 2.
s_th = 40 #saturation blancking threshold\np_th = 1000*(sz//512)**2 #threshold for the minimum number of pixels\nHello, I met the same error. I don't know why.
\nmy notebook here:https://www.kaggle.com/code/fortelean/hub-inference-single?scriptVersionId=101552027
Hi @fortelean , have you solve this problem?
", "rawMarkdown": "Hi @fortelean , have you solve this problem?", "votes": null }, { "id": "1938717", "postDate": "09/14/2022 10:15:12", "content": "Did you solve the problem?
", "rawMarkdown": "Did you solve the problem?", "votes": null } ], "comments": [ { "id": 1855831, "author_name": "thedevastator", "author_url": "", "post_date": "07/15/2022 00:58:26", "content": "Thanks for sharing your tips!
\nI wanted to add a couple of things that have caused issues for me in the past:
\nThanks again for sharing your tips!
", "votes": null, "replies": [ { "id": 1858016, "author_name": "aramisvesal", "author_url": "", "post_date": "07/16/2022 15:28:37", "content": "@thedevastator thank you for creating all the datasets. I have used your 512x512 dataset and locally trained a model on my machine. My local CV is fine, and the validation predictions look fine too. However, when I use your inference code to make a submission it produces empty predictions. The public LB is also 0.0. Any idea what is wrong?
This does not happen with 256x256 input size. For 512x512, I set sz = 512, and reduce = 2.
s_th = 40 #saturation blancking threshold\np_th = 1000*(sz//512)**2 #threshold for the minimum number of pixels\nDid you solve the problem?
", "votes": null, "replies": [] } ] } ] }, { "id": 1867471, "author_name": "fortelean", "author_url": "", "post_date": "07/23/2022 08:30:09", "content": "Hello, I met the same error. I don't know why.
\nmy notebook here:https://www.kaggle.com/code/fortelean/hub-inference-single?scriptVersionId=101552027
Hi @fortelean , have you solve this problem?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1852272": "After 2 days of painstaking debugging, I've created a checklist for newcomers who want to avoid the pain that I have to go through creating the first submission:\n1. Try inference on training set first to debug memory error. Use gc.collects() (not recommended) or check if you need to clean up garbage in the event memory overflow occurs.\n2. Make sure to **read the data description carefully**. For example: \"The HuBMAP images range in size from 4500x4500 down to 160x160 pixels\". My self-implemented sliding window inference did not account for this and it failed miserably.\n3. Submission file should be named \"submission.csv\" and saved with index=False\n4. Inference on the test set **should not exceed 9 hours** or 3 hours even for a single model. Again you can debug this using training set to alleviate inefficiency like running sequentially instead of parallel, etc...\n\nI hope these things may help you.\n\nCheers!", "1855831": "Thanks for sharing your tips!\n\nI wanted to add a couple of things that have caused issues for me in the past:\n\n5. Make sure your data types match the expected data types in the description. I've had issues in the past where my submission has been rejected because my data types didn't match (e.g. I had integers when it was expecting floats)\n6. Check the competition rules carefully! Sometimes there are specific formatting requirements for the submission that are not made clear in the description (e.g. specifying the column names in a certain order)\n7. If you're using external data, make sure you've read the terms and conditions carefully. Some competitions don't allow the use of external data, or they may have specific requirements about how that data can be used.\n\nThanks again for sharing your tips!", "1858016": "thedevastator thank you for creating all the datasets. I have used your `512x512` dataset and locally trained a model on my machine. My local CV is fine, and the validation predictions look fine too. However, when I use your inference code to make a submission it produces empty predictions. The public LB is also 0.0. Any idea what is wrong?\n\nThis does not happen with 256x256 input size. For 512x512, I set `sz = 512`, and `reduce = 2`. \n\n```\ns_th = 40 #saturation blancking threshold\np_th = 1000*(sz//512)**2 #threshold for the minimum number of pixels\n```", "1867471": "Hello, I met the same error. I don't know why.\nmy notebook here:[https://www.kaggle.com/code/fortelean/hub-inference-single?scriptVersionId=101552027](https://www.kaggle.com/code/fortelean/hub-inference-single?scriptVersionId=101552027)", "1888249": "Hi @fortelean , have you solve this problem?", "1938717": "Did you solve the problem?" }, "source": "meta" }