Dear Kagglers,
\nso I managed to train a CNN using segmentation models with backbone restnet34 (https://www.kaggle.com/alinaherderich/notebook4ca5c6eb1c - yes notebook title still raw). I used the code of @iafoss to tile the training images, but modified it to tile size 512x512 (https://www.kaggle.com/alinaherderich/iafoss-512x512).
\nNow to my question: I created a submission notebook, where I read my model from the training notebook, make predicitons on the test set and output the submission.csv. I repeatedly get a Submission Socring Error ALTHOUGH my code is nearly identical to submissions from my teammates (and other kagglers), whose submissions succeeded.
\nHere's the link to my notebook: https://www.kaggle.com/alinaherderich/submisson-alina-resnet34-v2
\nIf someone has any valuable tips, I would really appreciate it. It's very frustrating.
", "messages": [ { "id": "1176126", "postDate": "01/29/2021 14:15:20", "content": "Dear Kagglers,
\nso I managed to train a CNN using segmentation models with backbone restnet34 (https://www.kaggle.com/alinaherderich/notebook4ca5c6eb1c - yes notebook title still raw). I used the code of @iafoss to tile the training images, but modified it to tile size 512x512 (https://www.kaggle.com/alinaherderich/iafoss-512x512).
\nNow to my question: I created a submission notebook, where I read my model from the training notebook, make predicitons on the test set and output the submission.csv. I repeatedly get a Submission Socring Error ALTHOUGH my code is nearly identical to submissions from my teammates (and other kagglers), whose submissions succeeded.
\nHere's the link to my notebook: https://www.kaggle.com/alinaherderich/submisson-alina-resnet34-v2
\nIf someone has any valuable tips, I would really appreciate it. It's very frustrating.
", "rawMarkdown": "Dear Kagglers,\n\nso I managed to train a CNN using segmentation models with backbone restnet34 (https://www.kaggle.com/alinaherderich/notebook4ca5c6eb1c - yes notebook title still raw). I used the code of @iafoss to tile the training images, but modified it to tile size 512x512 (https://www.kaggle.com/alinaherderich/iafoss-512x512).\n\nNow to my question: I created a submission notebook, where I read my model from the training notebook, make predicitons on the test set and output the submission.csv. I repeatedly get a Submission Socring Error ALTHOUGH my code is nearly identical to submissions from my teammates (and other kagglers), whose submissions succeeded.\n\n- No, I do not hardcode any variables/filenames/etc.\n- No, my notebook does not stress RAM. (It reads one image at a time with rasterio, predicting one tile at a time, also using downsizing and upsizing on the tiles before/after prediciton.) It takes around 5-10 minutes to run.\n- No, my submission file does not have any empty rows/invalid data types/values. Column names are correct. I double checked everything. I read the masks back and plotted them and it seems very reasonable.\n\nHere's the link to my notebook: https://www.kaggle.com/alinaherderich/submisson-alina-resnet34-v2\n\nIf someone has any valuable tips, I would really appreciate it. It's very frustrating.", "votes": null }, { "id": "1176341", "postDate": "01/29/2021 15:31:28", "content": "WINDOW = 4×512,
\nDid you create 512×512 tiles from 2048×2048 tiff images and use them as training inputs?
The notebook I am using to preprocess my training data downsizes the original images/masks by a factor of 4, then tiles it into 512x512. Accordingly, when I am reading the test images, I would go tile by tile 1) read 2048x2048 tiles through indexing (see function make_grid) 2) downsize the current tile by a factor of 4 (results in 512x512) 3) predict 4) upsize the resulting mask by a factor of 4 5) \"overlay\" the predictions on a img.shape sized np.zeros array > repeat until predictions obtained for whole image > write to dictionary with id and rle.
", "rawMarkdown": "The notebook I am using to preprocess my training data downsizes the original images/masks by a factor of 4, then tiles it into 512x512. Accordingly, when I am reading the test images, I would go tile by tile 1) read 2048x2048 tiles through indexing (see function make_grid) 2) downsize the current tile by a factor of 4 (results in 512x512) 3) predict 4) upsize the resulting mask by a factor of 4 5) \"overlay\" the predictions on a img.shape sized np.zeros array > repeat until predictions obtained for whole image > write to dictionary with id and rle.", "votes": null }, { "id": "1177068", "postDate": "01/30/2021 02:53:31", "content": "Umm…no idea right now, but committing time within 5~10 mins seems to be too short according to your inference cord.
", "rawMarkdown": "Umm...no idea right now, but committing time within 5~10 mins seems to be too short according to your inference cord.", "votes": null }, { "id": "1177656", "postDate": "01/30/2021 12:42:53", "content": "Hm. Maybe 5 to 10 minutes was a quite optimistic estimate. (It's rather 10 to 15 ;D) I checked every step of the process and it really seems like it does what it should be doing. Thanks for checking, anyway! Really nice of you :)
", "rawMarkdown": "Hm. Maybe 5 to 10 minutes was a quite optimistic estimate. (It's rather 10 to 15 ;D) I checked every step of the process and it really seems like it does what it should be doing. Thanks for checking, anyway! Really nice of you :)", "votes": null }, { "id": "1181448", "postDate": "02/01/2021 22:24:22", "content": "I think the way you create slices in make_grid may potentially lead to the last slice being invalid when one of the sides is exactly divisible by the window size. If that happens you will get an exception which will manifest as a submission scoring error. This is also why you probably don't see it on every image as this requires either height or width to be an exact multiple of 2048.
", "rawMarkdown": "I think the way you create slices in make_grid may potentially lead to the last slice being invalid when one of the sides is exactly divisible by the window size. If that happens you will get an exception which will manifest as a submission scoring error. This is also why you probably don't see it on every image as this requires either height or width to be an exact multiple of 2048.", "votes": null }, { "id": "1182174", "postDate": "02/02/2021 10:39:39", "content": "Hi ;
\nDid you find the problem??
Hi ;
\nDid you find the problem??
I found a solution in the below link:
\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/198317#1085194
Sorry for the late reply. I did not have any time until today to try it out, but it actually solved the problem! Thank you very much :) Because of you, I was finally able to make a submission.
", "rawMarkdown": "Sorry for the late reply. I did not have any time until today to try it out, but it actually solved the problem! Thank you very much :) Because of you, I was finally able to make a submission.", "votes": null } ], "comments": [ { "id": 1176341, "author_name": "drtausamaru", "author_url": "", "post_date": "01/29/2021 15:31:28", "content": "WINDOW = 4×512,
\nDid you create 512×512 tiles from 2048×2048 tiff images and use them as training inputs?
The notebook I am using to preprocess my training data downsizes the original images/masks by a factor of 4, then tiles it into 512x512. Accordingly, when I am reading the test images, I would go tile by tile 1) read 2048x2048 tiles through indexing (see function make_grid) 2) downsize the current tile by a factor of 4 (results in 512x512) 3) predict 4) upsize the resulting mask by a factor of 4 5) \"overlay\" the predictions on a img.shape sized np.zeros array > repeat until predictions obtained for whole image > write to dictionary with id and rle.
", "votes": null, "replies": [] }, { "id": 1177068, "author_name": "drtausamaru", "author_url": "", "post_date": "01/30/2021 02:53:31", "content": "Umm…no idea right now, but committing time within 5~10 mins seems to be too short according to your inference cord.
", "votes": null, "replies": [ { "id": 1177656, "author_name": "alinaherderich", "author_url": "", "post_date": "01/30/2021 12:42:53", "content": "Hm. Maybe 5 to 10 minutes was a quite optimistic estimate. (It's rather 10 to 15 ;D) I checked every step of the process and it really seems like it does what it should be doing. Thanks for checking, anyway! Really nice of you :)
", "votes": null, "replies": [] } ] } ] }, { "id": 1181448, "author_name": "igor14497", "author_url": "", "post_date": "02/01/2021 22:24:22", "content": "I think the way you create slices in make_grid may potentially lead to the last slice being invalid when one of the sides is exactly divisible by the window size. If that happens you will get an exception which will manifest as a submission scoring error. This is also why you probably don't see it on every image as this requires either height or width to be an exact multiple of 2048.
", "votes": null, "replies": [ { "id": 1188756, "author_name": "alinaherderich", "author_url": "", "post_date": "02/06/2021 13:49:57", "content": "Sorry for the late reply. I did not have any time until today to try it out, but it actually solved the problem! Thank you very much :) Because of you, I was finally able to make a submission.
", "votes": null, "replies": [] } ] }, { "id": 1182174, "author_name": "rezahajalizadeh", "author_url": "", "post_date": "02/02/2021 10:39:39", "content": "Hi ;
\nDid you find the problem??
Hi ;
\nDid you find the problem??
I found a solution in the below link:
\nhttps://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/198317#1085194