My solution is in two stages.
\n\nStage 1: Predict rough outline of mask with 1/4 downscaled image
\n\nStage 2: Train/Predict mask only using images around edge areas.
\n\nThis reduces the amount of data to process a lot, and I can use more complex network in reasonable time.
\n\nDetails are:
\n\nExtract 256x256 tiles only along the edge of masks from stage 1
Used deeper UNET - 6 downsample/upsample. But for each depth, Conv\nlayers with small number of layers than original UNET to avoid\nexplosion of\n parameter numbers. Around 3million parameters are used.
During training, tiles are overlapped so that each edge pixel are\ncovered twice.
During inferencing, tiles are overlapped so that each edge\npixel are covered at least three times. So each pixel's prediction\nis voted at least by 3 predictions(see attached image) UNET tends to have bad\nprediction around edge area, but this method reduced this problem.
Used standard augmentation(scale+-10%, flip left-right, brightness, contrast, pixel shift, etc) and basic dice_coeff as loss.
I tried several other segmentation networks, but this one showed best result with large margin(0.0002) compared to other models. Single model LB was 0.9970 and with ensemble of several folds from same model, it got to 0.9971.
\n\nI think this pipeline works really well considering its simplicity, but I made a big mistake.\n - I didn't noticed that some test outputs from stage 1 were horribly wrong until today and didn't have chance to correct it. :(\nI should have found the issue early only if I had good visualization or simple sanity check logic.
\n\nThe best lesson from this competition is that good visualization/analyzing tool is really important.
\n\nOh, Heng CherKeng, thank you very much. I learned a lot from your posting!
", "messages": [ { "id": "224981", "postDate": "09/28/2017 02:33:30", "content": "My solution is in two stages.
\n\nStage 1: Predict rough outline of mask with 1/4 downscaled image
\n\nStage 2: Train/Predict mask only using images around edge areas.
\n\nThis reduces the amount of data to process a lot, and I can use more complex network in reasonable time.
\n\nDetails are:
\n\nExtract 256x256 tiles only along the edge of masks from stage 1
Used deeper UNET - 6 downsample/upsample. But for each depth, Conv\nlayers with small number of layers than original UNET to avoid\nexplosion of\n parameter numbers. Around 3million parameters are used.
During training, tiles are overlapped so that each edge pixel are\ncovered twice.
During inferencing, tiles are overlapped so that each edge\npixel are covered at least three times. So each pixel's prediction\nis voted at least by 3 predictions(see attached image) UNET tends to have bad\nprediction around edge area, but this method reduced this problem.
Used standard augmentation(scale+-10%, flip left-right, brightness, contrast, pixel shift, etc) and basic dice_coeff as loss.
I tried several other segmentation networks, but this one showed best result with large margin(0.0002) compared to other models. Single model LB was 0.9970 and with ensemble of several folds from same model, it got to 0.9971.
\n\nI think this pipeline works really well considering its simplicity, but I made a big mistake.\n - I didn't noticed that some test outputs from stage 1 were horribly wrong until today and didn't have chance to correct it. :(\nI should have found the issue early only if I had good visualization or simple sanity check logic.
\n\nThe best lesson from this competition is that good visualization/analyzing tool is really important.
\n\nOh, Heng CherKeng, thank you very much. I learned a lot from your posting!
", "rawMarkdown": "My solution is in two stages.\n\n**Stage 1: Predict rough outline of mask with 1/4 downscaled image**\n\n 1. Scale down all images/masks to 320x480\n 2. Train with default UNET.\n 3. Predict mask for test set\n\n**Stage 2: Train/Predict mask only using images around edge areas.**\n\nThis reduces the amount of data to process a lot, and I can use more complex network in reasonable time.\n\nDetails are:\n\n - Extract 256x256 tiles only along the edge of masks from stage 1 \n\n - Used deeper UNET - 6 downsample/upsample. But for each depth, Conv\n layers with small number of layers than original UNET to avoid\n explosion of\n parameter numbers. Around 3million parameters are used.\n\n - During training, tiles are overlapped so that each edge pixel are\n covered twice.\n\n - During inferencing, tiles are overlapped so that each edge\n pixel are covered at least three times. So each pixel's prediction\n is voted at least by 3 predictions(see attached image) UNET tends to have bad\n prediction around edge area, but this method reduced this problem.\n\n - Used standard augmentation(scale+-10%, flip left-right, brightness, contrast, pixel shift, etc) and basic dice_coeff as loss.\n - Used batch normalization, and to maximize the effectiveness of it, \n the batch consists of random tiles from multiple images. I guess this\n helped network to converge much faster.\n\nI tried several other segmentation networks, but this one showed best result with large margin(0.0002) compared to other models. Single model LB was 0.9970 and with ensemble of several folds from same model, it got to 0.9971. \n\nI think this pipeline works really well considering its simplicity, but I made a big mistake.\n - I didn't noticed that some test outputs from stage 1 were horribly wrong until today and didn't have chance to correct it. :(\nI should have found the issue early only if I had good visualization or simple sanity check logic.\n\n\nThe best lesson from this competition is that good visualization/analyzing tool is really important.\n\n\nOh, Heng CherKeng, thank you very much. I learned a lot from your posting!", "votes": null }, { "id": "224983", "postDate": "09/28/2017 02:37:41", "content": "Neat solution! Thanks!
", "rawMarkdown": "Neat solution! Thanks!", "votes": null }, { "id": "224991", "postDate": "09/28/2017 03:01:23", "content": "Amazing! This was our main idea, but I was worrying that uNet256 doesn't give a proper dice score on the edge. We even trained the edge detector using uNet, but stopped right there and decided to move another way. Nice to see you implemented this!
", "rawMarkdown": "Amazing! This was our main idea, but I was worrying that uNet256 doesn't give a proper dice score on the edge. We even trained the edge detector using uNet, but stopped right there and decided to move another way. Nice to see you implemented this!", "votes": null }, { "id": "225002", "postDate": "09/28/2017 03:22:16", "content": "This is cool !!
\nI have some questions~
\n1. Did you train with images/masks of 320x480 for your best result ?
\n2. Do you think train higher resolution of images (e.g 1024x1024) will increase the score?
\n3. Do you think extracting higher resolution tiles along the edge (e.g. 512x512) will help?
\nAnd the most important, Can you share your code, please, hahah?
Stage 1's output mask is used for generating stage2's test tile and fill areas that are not covered by tiles.\nI thought the quality of stage 1 output doesn't matter much since details are corrected anyway in second stage anyway. But in reality, there were a few images that have big errors.
I think higher resolution in stage 1 will increase score just little bit. With 320x480 image, details like antenna is lost, but higher resolution will help
I tried several other tile sizes. I saw heavy over-fitting with smaller tile size than 256. With 64 tile size, it works extremely well in local validation(even reached 0.9981 - well partially caused by the way that I split train/val), but terrible in public leader board. I tried tile size of 384 also, but it took too long to converge so I gave up. With 256x256 tile, it got okay model in a day.
For code, I guess I need to clean it up first (for myself also) and it can take some time, but I'll try to share it.
", "rawMarkdown": "1. Stage 1's output mask is used for generating stage2's test tile and fill areas that are not covered by tiles.\nI thought the quality of stage 1 output doesn't matter much since details are corrected anyway in second stage anyway. But in reality, there were a few images that have big errors.\n\n2. I think higher resolution in stage 1 will increase score just little bit. With 320x480 image, details like antenna is lost, but higher resolution will help\n\n3. I tried several other tile sizes. I saw heavy over-fitting with smaller tile size than 256. With 64 tile size, it works extremely well in local validation(even reached 0.9981 - well partially caused by the way that I split train/val), but terrible in public leader board. I tried tile size of 384 also, but it took too long to converge so I gave up. With 256x256 tile, it got okay model in a day. \n\nFor code, I guess I need to clean it up first (for myself also) and it can take some time, but I'll try to share it.", "votes": null }, { "id": "225020", "postDate": "09/28/2017 03:58:56", "content": "Thanks for your sharing!
\nI look forward to learn more from your code~
Hey! We thought of the same... and also implemented it a patches-based one.
", "rawMarkdown": "Hey! We thought of the same... and also implemented it a patches-based one.", "votes": null }, { "id": "225144", "postDate": "09/28/2017 11:17:21", "content": "Hi JandJ\nMay I ask how to you extract tile along the edge of the mask?
\n\nI image you have to slide the window over a image and capture ones that have roughly 50-50 share of true false pixels, for all the images.
\n\nhow long did this process take?
\n\nmany thanks!
", "rawMarkdown": "Hi JandJ\nMay I ask how to you extract tile along the edge of the mask?\n\nI image you have to slide the window over a image and capture ones that have roughly 50-50 share of true false pixels, for all the images.\n\nhow long did this process take?\n\nmany thanks!", "votes": null }, { "id": "225272", "postDate": "09/28/2017 16:18:08", "content": "Basically, generate image with edge, and eliminate pixels while getting tile location.
\n\nFor training set, it took 100 sec with 8 cores with overlap_ratio of 2.\nIn retrospect, I should have added some more code for checking sanity and correct big holes far inside the car edge, etc.
\n\nmask = mask >= 127 \nedge = mask - morphology.binary_erosion(mask)\nedge *= overlap_ratio\nhs = tile_size//2\nqs = tile_size//4\nwhile (True):\n nzs = np.transpose(np.nonzero(edge))\n if len(nzs) == 0:\n break\n # Pick random pixels among non-zero values.\n for i in range(len(nzs)//10):\n nz = nzs[random.randint(0, len(nzs)-1)]\n if edge[nz[0], nz[1]] <= 0:\n continue\n # Make it fit inside valid area\n nz[0] = np.clip(nz[0], hs, 1280 - hs)\n nz[1] = np.clip(nz[1], hs, 1918 - hs)\n\n tile_rcs.append((nz[0] - hs, nz[1] - hs)) # Record left-top coordinates\n\n edge[nz[0] - hs:nz[0] + hs, nz[1] - hs:nz[1] + hs] -= edge[nz[0] - hs:nz[0] + hs, nz[1] - hs:nz[1] + hs] > 0\n # Mark center area as 'done' part to avoid sampling again from similar location\n edge[nz[0] - qs:nz[0] + qs, nz[1] - qs:nz[1] + qs] = 0\n if nz[1] == hs or nz[1] == 1918 - hs: # Avoid oversampling on Left,right edge cases\n edge[nz[0] - qs // 2:nz[0] + qs // 2, nz[1] - hs:nz[1] + hs] = 0\nThanks @JandJ for sharing your solution and congratulations.
", "rawMarkdown": "Thanks @JandJ for sharing your solution and congratulations.", "votes": null }, { "id": "225433", "postDate": "09/29/2017 03:50:20", "content": "Thanks for sharing. that's a great idea.
", "rawMarkdown": "Thanks for sharing. that's a great idea.", "votes": null }, { "id": "225517", "postDate": "09/29/2017 10:48:02", "content": "Hi JandJ
\n\nThanks for sharing!
", "rawMarkdown": "Hi JandJ\n\nThanks for sharing!", "votes": null }, { "id": "225759", "postDate": "09/30/2017 00:19:45", "content": "Thanks for sharing!! It is very helpful.
", "rawMarkdown": "Thanks for sharing!! It is very helpful.", "votes": null }, { "id": "225773", "postDate": "09/30/2017 01:46:13", "content": "It is a very method with excellent tesults. You can consider a end to end solution with single network like faster rcnn. I think the features for stage one and two can be shared. You can write a new paper😀. I am think of focus loss for selecting the crops for stage two ...
\n\nThanks for sharing the work and it is a good job!
", "rawMarkdown": "It is a very method with excellent tesults. You can consider a end to end solution with single network like faster rcnn. I think the features for stage one and two can be shared. You can write a new paper😀. I am think of focus loss for selecting the crops for stage two ...\n\nThanks for sharing the work and it is a good job!", "votes": null } ], "comments": [ { "id": 224983, "author_name": "asanakoev", "author_url": "", "post_date": "09/28/2017 02:37:41", "content": "Neat solution! Thanks!
", "votes": null, "replies": [] }, { "id": 224991, "author_name": "truepk", "author_url": "", "post_date": "09/28/2017 03:01:23", "content": "Amazing! This was our main idea, but I was worrying that uNet256 doesn't give a proper dice score on the edge. We even trained the edge detector using uNet, but stopped right there and decided to move another way. Nice to see you implemented this!
", "votes": null, "replies": [] }, { "id": 225002, "author_name": "miaouhoho", "author_url": "", "post_date": "09/28/2017 03:22:16", "content": "This is cool !!
\nI have some questions~
\n1. Did you train with images/masks of 320x480 for your best result ?
\n2. Do you think train higher resolution of images (e.g 1024x1024) will increase the score?
\n3. Do you think extracting higher resolution tiles along the edge (e.g. 512x512) will help?
\nAnd the most important, Can you share your code, please, hahah?
Stage 1's output mask is used for generating stage2's test tile and fill areas that are not covered by tiles.\nI thought the quality of stage 1 output doesn't matter much since details are corrected anyway in second stage anyway. But in reality, there were a few images that have big errors.
I think higher resolution in stage 1 will increase score just little bit. With 320x480 image, details like antenna is lost, but higher resolution will help
I tried several other tile sizes. I saw heavy over-fitting with smaller tile size than 256. With 64 tile size, it works extremely well in local validation(even reached 0.9981 - well partially caused by the way that I split train/val), but terrible in public leader board. I tried tile size of 384 also, but it took too long to converge so I gave up. With 256x256 tile, it got okay model in a day.
For code, I guess I need to clean it up first (for myself also) and it can take some time, but I'll try to share it.
", "votes": null, "replies": [] }, { "id": 225020, "author_name": "miaouhoho", "author_url": "", "post_date": "09/28/2017 03:58:56", "content": "Thanks for your sharing!
\nI look forward to learn more from your code~
Hey! We thought of the same... and also implemented it a patches-based one.
", "votes": null, "replies": [] }, { "id": 225144, "author_name": "yl1202", "author_url": "", "post_date": "09/28/2017 11:17:21", "content": "Hi JandJ\nMay I ask how to you extract tile along the edge of the mask?
\n\nI image you have to slide the window over a image and capture ones that have roughly 50-50 share of true false pixels, for all the images.
\n\nhow long did this process take?
\n\nmany thanks!
", "votes": null, "replies": [ { "id": 225272, "author_name": "jandjenter", "author_url": "", "post_date": "09/28/2017 16:18:08", "content": "Basically, generate image with edge, and eliminate pixels while getting tile location.
\n\nFor training set, it took 100 sec with 8 cores with overlap_ratio of 2.\nIn retrospect, I should have added some more code for checking sanity and correct big holes far inside the car edge, etc.
\n\nmask = mask >= 127 \nedge = mask - morphology.binary_erosion(mask)\nedge *= overlap_ratio\nhs = tile_size//2\nqs = tile_size//4\nwhile (True):\n nzs = np.transpose(np.nonzero(edge))\n if len(nzs) == 0:\n break\n # Pick random pixels among non-zero values.\n for i in range(len(nzs)//10):\n nz = nzs[random.randint(0, len(nzs)-1)]\n if edge[nz[0], nz[1]] <= 0:\n continue\n # Make it fit inside valid area\n nz[0] = np.clip(nz[0], hs, 1280 - hs)\n nz[1] = np.clip(nz[1], hs, 1918 - hs)\n\n tile_rcs.append((nz[0] - hs, nz[1] - hs)) # Record left-top coordinates\n\n edge[nz[0] - hs:nz[0] + hs, nz[1] - hs:nz[1] + hs] -= edge[nz[0] - hs:nz[0] + hs, nz[1] - hs:nz[1] + hs] > 0\n # Mark center area as 'done' part to avoid sampling again from similar location\n edge[nz[0] - qs:nz[0] + qs, nz[1] - qs:nz[1] + qs] = 0\n if nz[1] == hs or nz[1] == 1918 - hs: # Avoid oversampling on Left,right edge cases\n edge[nz[0] - qs // 2:nz[0] + qs // 2, nz[1] - hs:nz[1] + hs] = 0\nHi JandJ
\n\nThanks for sharing!
", "votes": null, "replies": [] } ] }, { "id": 225354, "author_name": "sheriytm", "author_url": "", "post_date": "09/28/2017 20:22:32", "content": "Thanks @JandJ for sharing your solution and congratulations.
", "votes": null, "replies": [] }, { "id": 225433, "author_name": "jackkwok", "author_url": "", "post_date": "09/29/2017 03:50:20", "content": "Thanks for sharing. that's a great idea.
", "votes": null, "replies": [] }, { "id": 225759, "author_name": "utkarshsingh", "author_url": "", "post_date": "09/30/2017 00:19:45", "content": "Thanks for sharing!! It is very helpful.
", "votes": null, "replies": [ { "id": 225773, "author_name": "hengck23", "author_url": "", "post_date": "09/30/2017 01:46:13", "content": "It is a very method with excellent tesults. You can consider a end to end solution with single network like faster rcnn. I think the features for stage one and two can be shared. You can write a new paper😀. I am think of focus loss for selecting the crops for stage two ...
\n\nThanks for sharing the work and it is a good job!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "224981": "My solution is in two stages.\n\n**Stage 1: Predict rough outline of mask with 1/4 downscaled image**\n\n 1. Scale down all images/masks to 320x480\n 2. Train with default UNET.\n 3. Predict mask for test set\n\n**Stage 2: Train/Predict mask only using images around edge areas.**\n\nThis reduces the amount of data to process a lot, and I can use more complex network in reasonable time.\n\nDetails are:\n\n - Extract 256x256 tiles only along the edge of masks from stage 1 \n\n - Used deeper UNET - 6 downsample/upsample. But for each depth, Conv\n layers with small number of layers than original UNET to avoid\n explosion of\n parameter numbers. Around 3million parameters are used.\n\n - During training, tiles are overlapped so that each edge pixel are\n covered twice.\n\n - During inferencing, tiles are overlapped so that each edge\n pixel are covered at least three times. So each pixel's prediction\n is voted at least by 3 predictions(see attached image) UNET tends to have bad\n prediction around edge area, but this method reduced this problem.\n\n - Used standard augmentation(scale+-10%, flip left-right, brightness, contrast, pixel shift, etc) and basic dice_coeff as loss.\n - Used batch normalization, and to maximize the effectiveness of it, \n the batch consists of random tiles from multiple images. I guess this\n helped network to converge much faster.\n\nI tried several other segmentation networks, but this one showed best result with large margin(0.0002) compared to other models. Single model LB was 0.9970 and with ensemble of several folds from same model, it got to 0.9971. \n\nI think this pipeline works really well considering its simplicity, but I made a big mistake.\n - I didn't noticed that some test outputs from stage 1 were horribly wrong until today and didn't have chance to correct it. :(\nI should have found the issue early only if I had good visualization or simple sanity check logic.\n\n\nThe best lesson from this competition is that good visualization/analyzing tool is really important.\n\n\nOh, Heng CherKeng, thank you very much. I learned a lot from your posting!", "224983": "Neat solution! Thanks!", "224991": "Amazing! This was our main idea, but I was worrying that uNet256 doesn't give a proper dice score on the edge. We even trained the edge detector using uNet, but stopped right there and decided to move another way. Nice to see you implemented this!", "225002": "This is cool !! \nI have some questions~ \n1. Did you train with images/masks of 320x480 for your best result ? \n2. Do you think train higher resolution of images (e.g 1024x1024) will increase the score? \n3. Do you think extracting higher resolution tiles along the edge (e.g. 512x512) will help? \nAnd the most important, Can you share your code, please, hahah?", "225009": "1. Stage 1's output mask is used for generating stage2's test tile and fill areas that are not covered by tiles.\nI thought the quality of stage 1 output doesn't matter much since details are corrected anyway in second stage anyway. But in reality, there were a few images that have big errors.\n\n2. I think higher resolution in stage 1 will increase score just little bit. With 320x480 image, details like antenna is lost, but higher resolution will help\n\n3. I tried several other tile sizes. I saw heavy over-fitting with smaller tile size than 256. With 64 tile size, it works extremely well in local validation(even reached 0.9981 - well partially caused by the way that I split train/val), but terrible in public leader board. I tried tile size of 384 also, but it took too long to converge so I gave up. With 256x256 tile, it got okay model in a day. \n\nFor code, I guess I need to clean it up first (for myself also) and it can take some time, but I'll try to share it.", "225020": "Thanks for your sharing! \nI look forward to learn more from your code~", "225131": "Hey! We thought of the same... and also implemented it a patches-based one.", "225144": "Hi JandJ\nMay I ask how to you extract tile along the edge of the mask?\n\nI image you have to slide the window over a image and capture ones that have roughly 50-50 share of true false pixels, for all the images.\n\nhow long did this process take?\n\nmany thanks!", "225272": "Basically, generate image with edge, and eliminate pixels while getting tile location.\n\nFor training set, it took 100 sec with 8 cores with overlap_ratio of 2.\nIn retrospect, I should have added some more code for checking sanity and correct big holes far inside the car edge, etc. \n\n mask = mask >= 127 \n edge = mask - morphology.binary_erosion(mask)\n edge *= overlap_ratio\n hs = tile_size//2\n qs = tile_size//4\n while (True):\n nzs = np.transpose(np.nonzero(edge))\n if len(nzs) == 0:\n break\n # Pick random pixels among non-zero values.\n for i in range(len(nzs)//10):\n nz = nzs[random.randint(0, len(nzs)-1)]\n if edge[nz[0], nz[1]] <= 0:\n continue\n # Make it fit inside valid area\n nz[0] = np.clip(nz[0], hs, 1280 - hs)\n nz[1] = np.clip(nz[1], hs, 1918 - hs)\n \n tile_rcs.append((nz[0] - hs, nz[1] - hs)) # Record left-top coordinates\n \n edge[nz[0] - hs:nz[0] + hs, nz[1] - hs:nz[1] + hs] -= edge[nz[0] - hs:nz[0] + hs, nz[1] - hs:nz[1] + hs] > 0\n # Mark center area as 'done' part to avoid sampling again from similar location\n edge[nz[0] - qs:nz[0] + qs, nz[1] - qs:nz[1] + qs] = 0\n if nz[1] == hs or nz[1] == 1918 - hs: # Avoid oversampling on Left,right edge cases\n edge[nz[0] - qs // 2:nz[0] + qs // 2, nz[1] - hs:nz[1] + hs] = 0", "225354": "Thanks @JandJ for sharing your solution and congratulations.", "225433": "Thanks for sharing. that's a great idea.", "225517": "Hi JandJ\n\nThanks for sharing!", "225759": "Thanks for sharing!! It is very helpful.", "225773": "It is a very method with excellent tesults. You can consider a end to end solution with single network like faster rcnn. I think the features for stage one and two can be shared. You can write a new paper😀. I am think of focus loss for selecting the crops for stage two ...\n\nThanks for sharing the work and it is a good job!" }, "source": "meta" }