![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1696976%2Fc53597a12fdbe39ab60252e5d7807c9b%2Fmalignant.jpg?generation=1592842981260269&alt=media\")
CutMix is pretty modern but already well known strategy.
\n\nA quote from the paper:
\n\n\n\n\npatches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches
\n
But the data we have is pretty specific. Image, labeled as 1 (Malignan) basically contain useful features only in the area of melanoma itself. All the surrounding area is just a skin, which would not provide that much of an information.
\n\nTo visualize what I mean lets take a look at the following two images. Top one has malignant (label 1) melanoma on it and the second one bening (label 0).
\n\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1696976%2F0a9e45b584e99a6f0c170658e98bf5af%2Fbening.jpg?generation=1592842980925828&alt=media\")
Now consider using CutMix only on the top-left corner of both images, thus cutting the are that contains only skin from the top image (label 1) and pasting it to the second image (label 0).
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1696976%2F71e4e361ea77e0403a07171ea00d3405%2Fcutmix_1.jpg?generation=1592843107105089&alt=media\")
So now second image contain ~10% of first image (roughly the size of the patch) and will be labeled as bening (0.9)/malignant(0.1). But the resulting image itself still contains only one bening melanoma so it is still 100% benign.
\n\nLet's take a look at another example. Now we cut melanoma part from the first image and paste it to the 'non-melanoma' part of the second image, thus, as a result, we have an image with two melanomas one of which is benign and another one is malignant. Pay attention to the size of melanoma. According to the computation it is still ~10% cut so labels will be the same: bening (0.9)/malignant(0.1) even though it is more like 50/50 now.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1696976%2F7c49f34b26b0c34cefe4afb62027e9d8%2Fcutmix_2.jpg?generation=1592843596906872&alt=media\")
Looks like CutMix method is hard to implement in this particular competition. I have tried several times but the results were only worse than simply using no augmentation at all.
", "messages": [ { "id": 897141, "postDate": "2020-06-22T16:36:56.980Z", "content": "CutMix is pretty modern but already well known strategy.
\n\nA quote from the paper:
\n\n\n\n\npatches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches
\n
But the data we have is pretty specific. Image, labeled as 1 (Malignan) basically contain useful features only in the area of melanoma itself. All the surrounding area is just a skin, which would not provide that much of an information.
\n\nTo visualize what I mean lets take a look at the following two images. Top one has malignant (label 1) melanoma on it and the second one bening (label 0).
\n\n
Now consider using CutMix only on the top-left corner of both images, thus cutting the are that contains only skin from the top image (label 1) and pasting it to the second image (label 0).
\n\n
So now second image contain ~10% of first image (roughly the size of the patch) and will be labeled as bening (0.9)/malignant(0.1). But the resulting image itself still contains only one bening melanoma so it is still 100% benign.
\n\nLet's take a look at another example. Now we cut melanoma part from the first image and paste it to the 'non-melanoma' part of the second image, thus, as a result, we have an image with two melanomas one of which is benign and another one is malignant. Pay attention to the size of melanoma. According to the computation it is still ~10% cut so labels will be the same: bening (0.9)/malignant(0.1) even though it is more like 50/50 now.\n
Looks like CutMix method is hard to implement in this particular competition. I have tried several times but the results were only worse than simply using no augmentation at all.
", "rawMarkdown": "[CutMix](https://arxiv.org/abs/1905.04899) is pretty modern but already well known strategy.\n\nA quote from the paper:\n\n> patches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches\n\nBut the data we have is pretty specific. Image, labeled as 1 (Malignan) basically contain useful features only in the area of melanoma itself. All the surrounding area is just a skin, which would not provide that much of an information.\n\nTo visualize what I mean lets take a look at the following two images. Top one has **malignant** (label 1) melanoma on it and the second one **bening** (label 0).\n\n### Malignant\n\n\n### Bening\n\n\nNow consider using CutMix only on the top-left corner of both images, thus cutting the are that contains only skin from the top image (label 1) and pasting it to the second image (label 0).\n\n### CutMix example 1\n\n\nSo now second image contain ~10% of first image (roughly the size of the patch) and will be labeled as bening (0.9)/malignant(0.1). But the resulting image itself still contains only one bening melanoma so it is still 100% benign.\n\n### CutMix example 2\nLet's take a look at another example. Now we cut melanoma part from the first image and paste it to the 'non-melanoma' part of the second image, thus, as a result, we have an image with two melanomas one of which is benign and another one is malignant. Pay attention to the size of melanoma. According to the computation it is still ~10% cut so labels will be the same: bening (0.9)/malignant(0.1) even though it is more like 50/50 now.\n\n\n\n### Conclusion\nLooks like CutMix method is hard to implement in this particular competition. I have tried several times but the results were only worse than simply using no augmentation at all.", "votes": 22 }, { "id": 913113, "postDate": "2020-07-03T01:46:23.317Z", "content": "If the goal is just to create more training data then you could mix two images that have the same target. So mix two benign image together or mix two malignant together.
\n\nIf you want to mix benign and malignant together, you can try CAM Cutmix explained here. The advantage of CAM Cutmix is that you are guarenteed to get a piece of image that contains malignant features when blending with a malignant image. And you can combine the targets with a heavier weight factor on malignant.
", "rawMarkdown": "If the goal is just to create more training data then you could mix two images that have the same target. So mix two benign image together or mix two malignant together.\n\nIf you want to mix benign and malignant together, you can try CAM Cutmix explained [here][1]. The advantage of CAM Cutmix is that you are guarenteed to get a piece of image that contains malignant features when blending with a malignant image. And you can combine the targets with a heavier weight factor on malignant.\n\n[1]: https://www.kaggle.com/c/bengaliai-cv19/discussion/136021", "votes": 14, "replies": [ { "id": 917724, "postDate": "2020-07-06T18:08:44.683Z", "content": "Chris, is there any snippet to help with the implementation of CAM Cutmix?
", "rawMarkdown": "Chris, is there any snippet to help with the implementation of CAM Cutmix?", "votes": 1 }, { "id": 917745, "postDate": "2020-07-06T18:19:44.827Z", "content": "See CAM Cutmix\nDetails are here.
", "rawMarkdown": "See [CAM Cutmix](https://www.kaggle.com/c/bengaliai-cv19/discussion/136025)\nDetails are [here](https://www.kaggle.com/c/bengaliai-cv19/discussion/136021#776800).", "votes": 3 }, { "id": 917766, "postDate": "2020-07-06T18:31:01.370Z", "content": "CAM Cutmix is a two step process. First you train one model to classify images. Then you use that one model to get the class activation maps. Save those maps to disk. (CAMs will be 32x smaller than original images, for example if you used 512x512x3 images, the CAMs are 16x16x1). Second you build another model and use the previous CAM as mask.
\n\nRegarding second model, in PyTorch TPU/GPU you can do whatever. In TensorFlow TPU, you'll need to work with TFRecords. It may get tricky but you will multiply the CAM by two images. For example. MASK = (CAM.resize((512,512))>0).astype('int'). Then new image = image1 * MASK + image2 * (1-MASK).
I posted a notebook here showing how to create CAM.
", "rawMarkdown": "CAM Cutmix is a two step process. First you train one model to classify images. Then you use that one model to get the class activation maps. Save those maps to disk. (CAMs will be 32x smaller than original images, for example if you used 512x512x3 images, the CAMs are 16x16x1). Second you build another model and use the previous CAM as mask.\n\nRegarding second model, in PyTorch TPU/GPU you can do whatever. In TensorFlow TPU, you'll need to work with TFRecords. It may get tricky but you will multiply the CAM by two images. For example. `MASK = (CAM.resize((512,512))>0).astype('int')`. Then `new image = image1 * MASK + image2 * (1-MASK)`.\n\nI posted a notebook [here][1] showing how to create CAM.\n\n[1]: https://www.kaggle.com/cdeotte/unsupervised-masks-cv-0-60", "votes": 5 } ] }, { "id": 898522, "postDate": "2020-06-23T15:10:30Z", "content": "Why not cut from taking the middle pixel and make a grid from it ? Any comments I want to try this
", "rawMarkdown": "Why not cut from taking the middle pixel and make a grid from it ? Any comments I want to try this", "votes": 3, "replies": [ { "id": 925313, "postDate": "2020-07-12T02:23:23.870Z", "content": "@yash612 have you tried it?? Is there any update on it?
", "rawMarkdown": "@yash612 have you tried it?? Is there any update on it?" } ] }, { "id": 897715, "postDate": "2020-06-23T03:48:19.523Z", "content": "Right. It really hasn't worked for me, so far.
", "rawMarkdown": "Right. It really hasn't worked for me, so far.", "votes": 1 }, { "id": 897287, "postDate": "2020-06-22T18:51:54.243Z", "content": "I have been having the same thought, but wouldn't that appear in all types of images where a non–relevant part of the background gets mixed in with another picture?
", "rawMarkdown": "I have been having the same thought, but wouldn't that appear in all types of images where a non–relevant part of the background gets mixed in with another picture?", "votes": 1 }, { "id": 898579, "postDate": "2020-06-23T15:44:05.410Z", "content": "And more advanced Attentive Cutmix
", "rawMarkdown": "And more advanced [Attentive Cutmix](https://arxiv.org/abs/2003.13048)", "votes": 2, "replies": [ { "id": 914129, "postDate": "2020-07-03T16:35:25.987Z", "content": "Cool paper. Thanks for posting. This looks similar to CAM Cutmix
", "rawMarkdown": "Cool paper. Thanks for posting. This looks similar to CAM Cutmix" } ] }, { "id": 897169, "postDate": "2020-06-22T17:02:26.840Z", "content": "What about GridMask?
", "rawMarkdown": "What about GridMask?", "votes": -2 }, { "id": 897277, "postDate": "2020-06-22T18:40:51.720Z", "content": "list of all mixup variant: https://www.kaggle.com/c/bengaliai-cv19/discussion/132492\nI will try progressive sprinkles
", "rawMarkdown": "list of all mixup variant: [https://www.kaggle.com/c/bengaliai-cv19/discussion/132492](https://www.kaggle.com/c/bengaliai-cv19/discussion/132492)\nI will try progressive sprinkles" }, { "id": 897152, "postDate": "2020-06-22T16:50:09.983Z", "content": "Maybe you could try to isolate a certain region and superimpose that instead?
", "rawMarkdown": "Maybe you could try to isolate a certain region and superimpose that instead? " }, { "id": 897151, "postDate": "2020-06-22T16:42:52.433Z", "content": "thanks for sharing... do you think use a bounding box would help?
", "rawMarkdown": "thanks for sharing... do you think use a bounding box would help?", "replies": [ { "id": 897977, "postDate": "2020-06-23T07:54:03.300Z", "rawMarkdown": "", "isDeleted": true }, { "id": 913009, "postDate": "2020-07-02T22:07:37.213Z", "content": "Like this?\n-> https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/163650\n-> https://www.kaggle.com/tschandl/ham10000-lesion-segmentations
", "rawMarkdown": "Like this?\n-> https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/163650\n-> https://www.kaggle.com/tschandl/ham10000-lesion-segmentations" } ] } ], "comments": [ { "id": 913113, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-07-03T01:46:23.317000", "content": "If the goal is just to create more training data then you could mix two images that have the same target. So mix two benign image together or mix two malignant together.
\n\nIf you want to mix benign and malignant together, you can try CAM Cutmix explained here. The advantage of CAM Cutmix is that you are guarenteed to get a piece of image that contains malignant features when blending with a malignant image. And you can combine the targets with a heavier weight factor on malignant.
", "votes": 14, "replies": [ { "id": 917724, "author_name": "Phi", "author_url": "", "post_date": "2020-07-06T18:08:44.683000", "content": "Chris, is there any snippet to help with the implementation of CAM Cutmix?
", "votes": 1, "replies": [] }, { "id": 917745, "author_name": "Sirish Somanchi", "author_url": "", "post_date": "2020-07-06T18:19:44.827000", "content": "See CAM Cutmix\nDetails are here.
", "votes": 3, "replies": [] }, { "id": 917766, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-07-06T18:31:01.370000", "content": "CAM Cutmix is a two step process. First you train one model to classify images. Then you use that one model to get the class activation maps. Save those maps to disk. (CAMs will be 32x smaller than original images, for example if you used 512x512x3 images, the CAMs are 16x16x1). Second you build another model and use the previous CAM as mask.
\n\nRegarding second model, in PyTorch TPU/GPU you can do whatever. In TensorFlow TPU, you'll need to work with TFRecords. It may get tricky but you will multiply the CAM by two images. For example. MASK = (CAM.resize((512,512))>0).astype('int'). Then new image = image1 * MASK + image2 * (1-MASK).
I posted a notebook here showing how to create CAM.
", "votes": 5, "replies": [] } ] }, { "id": 898522, "author_name": "yash chaudhary", "author_url": "", "post_date": "2020-06-23T15:10:30", "content": "Why not cut from taking the middle pixel and make a grid from it ? Any comments I want to try this
", "votes": 3, "replies": [ { "id": 925313, "author_name": "Redwan Sony", "author_url": "", "post_date": "2020-07-12T02:23:23.870000", "content": "@yash612 have you tried it?? Is there any update on it?
", "votes": 0, "replies": [] } ] }, { "id": 897715, "author_name": "KhanhVD", "author_url": "", "post_date": "2020-06-23T03:48:19.523000", "content": "Right. It really hasn't worked for me, so far.
", "votes": 1, "replies": [] }, { "id": 897287, "author_name": "Gilles Vandewiele", "author_url": "", "post_date": "2020-06-22T18:51:54.243000", "content": "I have been having the same thought, but wouldn't that appear in all types of images where a non–relevant part of the background gets mixed in with another picture?
", "votes": 1, "replies": [] }, { "id": 898579, "author_name": "shiba", "author_url": "", "post_date": "2020-06-23T15:44:05.410000", "content": "And more advanced Attentive Cutmix
", "votes": 2, "replies": [ { "id": 914129, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-07-03T16:35:25.987000", "content": "Cool paper. Thanks for posting. This looks similar to CAM Cutmix
", "votes": 0, "replies": [] } ] }, { "id": 897169, "author_name": "Awsaf", "author_url": "", "post_date": "2020-06-22T17:02:26.840000", "content": "What about GridMask?
", "votes": -2, "replies": [] }, { "id": 897277, "author_name": "AgentAuers", "author_url": "", "post_date": "2020-06-22T18:40:51.720000", "content": "list of all mixup variant: https://www.kaggle.com/c/bengaliai-cv19/discussion/132492\nI will try progressive sprinkles
", "votes": 0, "replies": [] }, { "id": 897152, "author_name": "Trigram", "author_url": "", "post_date": "2020-06-22T16:50:09.983000", "content": "Maybe you could try to isolate a certain region and superimpose that instead?
", "votes": 0, "replies": [] }, { "id": 897151, "author_name": "ZHU CHAO", "author_url": "", "post_date": "2020-06-22T16:42:52.433000", "content": "thanks for sharing... do you think use a bounding box would help?
", "votes": 0, "replies": [ { "id": 897977, "author_name": "", "author_url": "", "post_date": "2020-06-23T07:54:03.300000", "content": "", "votes": 0, "replies": [] }, { "id": 913009, "author_name": "chdlr", "author_url": "", "post_date": "2020-07-02T22:07:37.213000", "content": "Like this?\n-> https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/163650\n-> https://www.kaggle.com/tschandl/ham10000-lesion-segmentations
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "897141": "[CutMix](https://arxiv.org/abs/1905.04899) is pretty modern but already well known strategy.\n\nA quote from the paper:\n\n> patches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches\n\nBut the data we have is pretty specific. Image, labeled as 1 (Malignan) basically contain useful features only in the area of melanoma itself. All the surrounding area is just a skin, which would not provide that much of an information.\n\nTo visualize what I mean lets take a look at the following two images. Top one has **malignant** (label 1) melanoma on it and the second one **bening** (label 0).\n\n### Malignant\n\n\n### Bening\n\n\nNow consider using CutMix only on the top-left corner of both images, thus cutting the are that contains only skin from the top image (label 1) and pasting it to the second image (label 0).\n\n### CutMix example 1\n\n\nSo now second image contain ~10% of first image (roughly the size of the patch) and will be labeled as bening (0.9)/malignant(0.1). But the resulting image itself still contains only one bening melanoma so it is still 100% benign.\n\n### CutMix example 2\nLet's take a look at another example. Now we cut melanoma part from the first image and paste it to the 'non-melanoma' part of the second image, thus, as a result, we have an image with two melanomas one of which is benign and another one is malignant. Pay attention to the size of melanoma. According to the computation it is still ~10% cut so labels will be the same: bening (0.9)/malignant(0.1) even though it is more like 50/50 now.\n\n\n\n### Conclusion\nLooks like CutMix method is hard to implement in this particular competition. I have tried several times but the results were only worse than simply using no augmentation at all.", "913113": "If the goal is just to create more training data then you could mix two images that have the same target. So mix two benign image together or mix two malignant together.\n\nIf you want to mix benign and malignant together, you can try CAM Cutmix explained [here][1]. The advantage of CAM Cutmix is that you are guarenteed to get a piece of image that contains malignant features when blending with a malignant image. And you can combine the targets with a heavier weight factor on malignant.\n\n[1]: https://www.kaggle.com/c/bengaliai-cv19/discussion/136021", "898522": "Why not cut from taking the middle pixel and make a grid from it ? Any comments I want to try this", "897715": "Right. It really hasn't worked for me, so far.", "897287": "I have been having the same thought, but wouldn't that appear in all types of images where a non–relevant part of the background gets mixed in with another picture?", "898579": "And more advanced [Attentive Cutmix](https://arxiv.org/abs/2003.13048)", "897169": "What about GridMask?", "897277": "list of all mixup variant: [https://www.kaggle.com/c/bengaliai-cv19/discussion/132492](https://www.kaggle.com/c/bengaliai-cv19/discussion/132492)\nI will try progressive sprinkles", "897152": "Maybe you could try to isolate a certain region and superimpose that instead? ", "897151": "thanks for sharing... do you think use a bounding box would help?" } }