![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F421965%2F02e7ae5004764839d0e9aa49385ccf60%2FScreenshot%20from%202020-07-04%2007-19-19.png?generation=1593836398132082&alt=media\")
I have been playing around with ways of cropping the images to remove unnecessary skin or the black border created by the scope (gotta make those precious GPU cycles count!). I've tried 2 methods using 256x256 images to create bounding boxes for cropping:\n1. A purely skimage driven approach using things like histogram equalization, thresholding and then identifying blobs using scipy (bed boxes). This method struggled when there was a large scope border (as you can see below from a subset from the 2019 dataset. I think the 2020 dataset has fewer of this style of images)\n2. Manually checking 10,000 images from step 1 and deleting the boxes that visually don't look \"right\" (about 15%) and then training a CNN to predict a bounding box (blue boxes = benign, green boxes = malignant)
Both methods end up having lower CV scores than uncropped images (with method 2 performing slightly better than 1). I have a few hunches to why that might be:\n- The edges of the lesion that define the shape might be more important than what's in the middle, so a box that is too tight might cause issues\n- The texture of the surrounding skin might be important (see these awesome SHAP images). Although this could also be an avenue for overfitting\n- Cropping an irregular shaped lesion and then resizing to a square-shaped image will make it more round (normal benign moles are usually round). Although I did try to crop and preserve the aspect ratio and there was no significant difference\n- Some of the images have already been cropped by a professional, so maybe I don't know what a \"good\" bounding box looks like\n- ¯_(ツ)_/¯
\n\nPerhaps the images don't need cropping (apart from the ones with the black scope ring)? I'm interested to hear if people have seen any gains from cropping?
", "messages": [ { "id": 914589, "postDate": "2020-07-04T04:34:34.690Z", "content": "I have been playing around with ways of cropping the images to remove unnecessary skin or the black border created by the scope (gotta make those precious GPU cycles count!). I've tried 2 methods using 256x256 images to create bounding boxes for cropping:\n1. A purely skimage driven approach using things like histogram equalization, thresholding and then identifying blobs using scipy (bed boxes). This method struggled when there was a large scope border (as you can see below from a subset from the 2019 dataset. I think the 2020 dataset has fewer of this style of images)\n2. Manually checking 10,000 images from step 1 and deleting the boxes that visually don't look \"right\" (about 15%) and then training a CNN to predict a bounding box (blue boxes = benign, green boxes = malignant)
Both methods end up having lower CV scores than uncropped images (with method 2 performing slightly better than 1). I have a few hunches to why that might be:\n- The edges of the lesion that define the shape might be more important than what's in the middle, so a box that is too tight might cause issues\n- The texture of the surrounding skin might be important (see these awesome SHAP images). Although this could also be an avenue for overfitting\n- Cropping an irregular shaped lesion and then resizing to a square-shaped image will make it more round (normal benign moles are usually round). Although I did try to crop and preserve the aspect ratio and there was no significant difference\n- Some of the images have already been cropped by a professional, so maybe I don't know what a \"good\" bounding box looks like\n- ¯_(ツ)_/¯
\n\nPerhaps the images don't need cropping (apart from the ones with the black scope ring)? I'm interested to hear if people have seen any gains from cropping?
", "rawMarkdown": "I have been playing around with ways of cropping the images to remove unnecessary skin or the black border created by the scope (gotta make those precious GPU cycles count!). I've tried 2 methods using 256x256 images to create bounding boxes for cropping:\n1. A purely `skimage` driven approach using things like histogram equalization, thresholding and then identifying blobs using `scipy` (bed boxes). This method struggled when there was a large scope border (as you can see below from a subset from the 2019 dataset. I think the 2020 dataset has fewer of this style of images)\n2. Manually checking 10,000 images from step 1 and deleting the boxes that visually don't look \"right\" (about 15%) and then training a CNN to predict a bounding box (blue boxes = benign, green boxes = malignant)\n\n\n\nBoth methods end up having lower CV scores than uncropped images (with method 2 performing slightly better than 1). I have a few hunches to why that might be:\n- The edges of the lesion that define the shape might be more important than what's in the middle, so a box that is too tight might cause issues\n- The texture of the surrounding skin might be important (see these awesome [SHAP](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/163425) images). Although this could also be an avenue for overfitting\n- Cropping an irregular shaped lesion and then resizing to a square-shaped image will make it more round (normal benign moles are usually round). Although I did try to crop and preserve the aspect ratio and there was no significant difference\n- Some of the images have already been cropped by a professional, so maybe I don't know what a \"good\" bounding box looks like\n- ¯\\_(ツ)_/¯\n\nPerhaps the images don't need cropping (apart from the ones with the black scope ring)? I'm interested to hear if people have seen any gains from cropping?", "votes": 5 }, { "id": 914642, "postDate": "2020-07-04T05:43:44.183Z", "content": "I think RandomCropResize is beneficial. Like any augmentation, I would not go too overboard with it.
", "rawMarkdown": "I think RandomCropResize is beneficial. Like any augmentation, I would not go too overboard with it. ", "replies": [ { "id": 915020, "postDate": "2020-07-04T12:27:55.120Z", "content": "You are right but in this case, I'm talking about cropping as preprocessing before augmentation
", "rawMarkdown": "You are right but in this case, I'm talking about cropping as preprocessing before augmentation", "votes": 1 } ] }, { "id": 914687, "postDate": "2020-07-04T06:31:09.447Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 915019, "postDate": "2020-07-04T12:26:43.510Z", "content": "No, although if I had the time, perhaps it might work better
", "rawMarkdown": "No, although if I had the time, perhaps it might work better" } ] } ], "comments": [ { "id": 914642, "author_name": "Signal", "author_url": "", "post_date": "2020-07-04T05:43:44.183000", "content": "I think RandomCropResize is beneficial. Like any augmentation, I would not go too overboard with it.
", "votes": 0, "replies": [ { "id": 915020, "author_name": "datasaurus", "author_url": "", "post_date": "2020-07-04T12:27:55.120000", "content": "You are right but in this case, I'm talking about cropping as preprocessing before augmentation
", "votes": 1, "replies": [] } ] }, { "id": 914687, "author_name": "", "author_url": "", "post_date": "2020-07-04T06:31:09.447000", "content": "", "votes": 0, "replies": [ { "id": 915019, "author_name": "datasaurus", "author_url": "", "post_date": "2020-07-04T12:26:43.510000", "content": "No, although if I had the time, perhaps it might work better
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "914589": "I have been playing around with ways of cropping the images to remove unnecessary skin or the black border created by the scope (gotta make those precious GPU cycles count!). I've tried 2 methods using 256x256 images to create bounding boxes for cropping:\n1. A purely `skimage` driven approach using things like histogram equalization, thresholding and then identifying blobs using `scipy` (bed boxes). This method struggled when there was a large scope border (as you can see below from a subset from the 2019 dataset. I think the 2020 dataset has fewer of this style of images)\n2. Manually checking 10,000 images from step 1 and deleting the boxes that visually don't look \"right\" (about 15%) and then training a CNN to predict a bounding box (blue boxes = benign, green boxes = malignant)\n\n\n\nBoth methods end up having lower CV scores than uncropped images (with method 2 performing slightly better than 1). I have a few hunches to why that might be:\n- The edges of the lesion that define the shape might be more important than what's in the middle, so a box that is too tight might cause issues\n- The texture of the surrounding skin might be important (see these awesome [SHAP](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/163425) images). Although this could also be an avenue for overfitting\n- Cropping an irregular shaped lesion and then resizing to a square-shaped image will make it more round (normal benign moles are usually round). Although I did try to crop and preserve the aspect ratio and there was no significant difference\n- Some of the images have already been cropped by a professional, so maybe I don't know what a \"good\" bounding box looks like\n- ¯\\_(ツ)_/¯\n\nPerhaps the images don't need cropping (apart from the ones with the black scope ring)? I'm interested to hear if people have seen any gains from cropping?", "914642": "I think RandomCropResize is beneficial. Like any augmentation, I would not go too overboard with it. ", "914687": "" } }