![\"\"](\"https://i.imgur.com/ndWCCBH.png\")
While looking through the dataset for corrupt images I noticed that there were images in the test and training set that were suspiciously similar.
\n\nHere are two examples of similar images:\n
It turns out that the reason for this is that there are two satellites (Terra and Aqua) capturing the images and they pass over the same areas a few hours after one another.
\n\nFor example:
\n\n\n\n\n\nUsing this information we can download thousands of pairs of images and train a Siamese Network (or something similar) to find similar images. I did this and found:
\n\n1,434 pairs with two images in train (ie. 2,868 images total)\n1,813 pairs with one in train, one in test\n634 pairs with two images in test (ie. 1,276 images total)
\n\nIn theory these images should have somewhat similar labels. Of course label noise makes it difficult to guarantee this, but I still think there must be some kind of signal we could use here. Ultimately I wasn't able to make use of this data but here's what I tried:
\n\nModifying the network to accept inputs of 7 channels.
\n\nModifying the network to accept inputs of 10 channels,
\n\nUsing the weights of the Siamese Network I trained as my pretrained encoder weights instead of standard Imagenet weights. My thinking was that this model would have learned about features that would allow it to distinguish between clouds.
I ran into a few challenges:\n - I don't think you can just add channels to the encoder input and expect your pretrained ImageNet weights to still work correctly. I believe this would change the distribution of activations we feed to the second layer?
\n\nWhat I wish I tried:\n - Investigating using image pairs to help with a classifier (I only worked on segmentation models)\n - Investigating which classes had the best labels between pairs of images
\n\nDid anyone manage to take advantage of this information? Does anyone have any insights into how we could have used this kind of information?
", "messages": [ { "id": "678139", "postDate": "11/21/2019 02:39:00", "content": "While looking through the dataset for corrupt images I noticed that there were images in the test and training set that were suspiciously similar.
\n\nHere are two examples of similar images:\n
It turns out that the reason for this is that there are two satellites (Terra and Aqua) capturing the images and they pass over the same areas a few hours after one another.
\n\nFor example:
\n\n\n\n\n\nUsing this information we can download thousands of pairs of images and train a Siamese Network (or something similar) to find similar images. I did this and found:
\n\n1,434 pairs with two images in train (ie. 2,868 images total)\n1,813 pairs with one in train, one in test\n634 pairs with two images in test (ie. 1,276 images total)
\n\nIn theory these images should have somewhat similar labels. Of course label noise makes it difficult to guarantee this, but I still think there must be some kind of signal we could use here. Ultimately I wasn't able to make use of this data but here's what I tried:
\n\nModifying the network to accept inputs of 7 channels.
\n\nModifying the network to accept inputs of 10 channels,
\n\nUsing the weights of the Siamese Network I trained as my pretrained encoder weights instead of standard Imagenet weights. My thinking was that this model would have learned about features that would allow it to distinguish between clouds.
I ran into a few challenges:\n - I don't think you can just add channels to the encoder input and expect your pretrained ImageNet weights to still work correctly. I believe this would change the distribution of activations we feed to the second layer?
\n\nWhat I wish I tried:\n - Investigating using image pairs to help with a classifier (I only worked on segmentation models)\n - Investigating which classes had the best labels between pairs of images
\n\nDid anyone manage to take advantage of this information? Does anyone have any insights into how we could have used this kind of information?
", "rawMarkdown": "While looking through the dataset for [corrupt images](https://www.kaggle.com/c/understanding_cloud_organization/discussion/110650) I noticed that there were images in the test and training set that were suspiciously similar.\n\nHere are two examples of similar images:\n\n\nIt turns out that the reason for this is that there are two satellites (Terra and Aqua) capturing the images and they pass over the same areas a few hours after one another. \n\nFor example:\n\nhttps://wvs.earthdata.nasa.gov/api/v1/snapshot?REQUEST=GetSnapshot&TIME=2019-09-24T00:00:00Z&BBOX=-26.523608349900595,-119.85108101391648,0.6927808151093444,-95.30684642147116&CRS=EPSG:4326&LAYERS=MODIS_Aqua_CorrectedReflectance_TrueColor,Coastlines&WRAP=day,x&FORMAT=image/jpeg&WIDTH=559&HEIGHT=619&ts=1569364996375\n\nhttps://wvs.earthdata.nasa.gov/api/v1/snapshot?REQUEST=GetSnapshot&TIME=2019-09-24T00:00:00Z&BBOX=-26.523608349900595,-119.85108101391648,0.6927808151093444,-95.30684642147116&CRS=EPSG:4326&LAYERS=MODIS_Terra_CorrectedReflectance_TrueColor,Coastlines&WRAP=day,x&FORMAT=image/jpeg&WIDTH=559&HEIGHT=619&ts=1569364996375\n\nUsing this information we can download thousands of pairs of images and train a Siamese Network (or something similar) to find similar images. I did this and found:\n\n1,434 pairs with two images in train (ie. 2,868 images total)\n1,813 pairs with one in train, one in test\n634 pairs with two images in test (ie. 1,276 images total)\n\n\n\nIn theory these images should have somewhat similar labels. Of course label noise makes it difficult to guarantee this, but I still think there must be some kind of signal we could use here. Ultimately I wasn't able to make use of this data but here's what I tried:\n\n- Modifying the network to accept inputs of 7 channels. \n - 3 for the RGB image, \n - 4 for the labels of the corresponding image pair.\n\n- Modifying the network to accept inputs of 10 channels, \n - 3 for the RGB image, \n - 3 for the pair RGB image\n - 4 for the labels of the corresponding image pair.\n\n- Using the weights of the Siamese Network I trained as my pretrained encoder weights instead of standard Imagenet weights. My thinking was that this model would have learned about features that would allow it to distinguish between clouds.\n\n\nI ran into a few challenges:\n - I don't think you can just add channels to the encoder input and expect your pretrained ImageNet weights to still work correctly. I believe this would change the distribution of activations we feed to the second layer?\n\n\nWhat I wish I tried:\n - Investigating using image pairs to help with a classifier (I only worked on segmentation models)\n - Investigating which classes had the best labels between pairs of images\n\n\nDid anyone manage to take advantage of this information? Does anyone have any insights into how we could have used this kind of information?", "votes": null }, { "id": "681443", "postDate": "11/26/2019 05:51:00", "content": "Nice finding!\nHow to find out similar images, can you share the code?
\n\nI think of Siamese Network too, but can not find out how to use it after training .
", "rawMarkdown": "Nice finding!\nHow to find out similar images, can you share the code?\n\nI think of Siamese Network too, but can not find out how to use it after training .", "votes": null }, { "id": "682955", "postDate": "11/28/2019 01:19:55", "content": "I used an approach from adambielski's repository.
\n\nI get the satellite images here: https://github.com/JoshVarty/KaggleClouds/blob/master/04_ImageSimilarity_GetData.ipynb
\n\nI train with OnlineContrastiveLoss here: \nhttps://github.com/JoshVarty/KaggleClouds/blob/master/04_ImageSimilarity_OnlineContrastiveLoss.ipynb
", "rawMarkdown": "I used an approach from [adambielski's repository](https://github.com/adambielski/siamese-triplet/blob/master/Experiments_MNIST.ipynb). \n\nI get the satellite images here: https://github.com/JoshVarty/KaggleClouds/blob/master/04_ImageSimilarity_GetData.ipynb\n\nI train with OnlineContrastiveLoss here: \nhttps://github.com/JoshVarty/KaggleClouds/blob/master/04_ImageSimilarity_OnlineContrastiveLoss.ipynb", "votes": null } ], "comments": [ { "id": 681443, "author_name": "chinafax", "author_url": "", "post_date": "11/26/2019 05:51:00", "content": "Nice finding!\nHow to find out similar images, can you share the code?
\n\nI think of Siamese Network too, but can not find out how to use it after training .
", "votes": null, "replies": [ { "id": 682955, "author_name": "joshvarty", "author_url": "", "post_date": "11/28/2019 01:19:55", "content": "I used an approach from adambielski's repository.
\n\nI get the satellite images here: https://github.com/JoshVarty/KaggleClouds/blob/master/04_ImageSimilarity_GetData.ipynb
\n\nI train with OnlineContrastiveLoss here: \nhttps://github.com/JoshVarty/KaggleClouds/blob/master/04_ImageSimilarity_OnlineContrastiveLoss.ipynb
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "678139": "While looking through the dataset for [corrupt images](https://www.kaggle.com/c/understanding_cloud_organization/discussion/110650) I noticed that there were images in the test and training set that were suspiciously similar.\n\nHere are two examples of similar images:\n\n\nIt turns out that the reason for this is that there are two satellites (Terra and Aqua) capturing the images and they pass over the same areas a few hours after one another. \n\nFor example:\n\nhttps://wvs.earthdata.nasa.gov/api/v1/snapshot?REQUEST=GetSnapshot&TIME=2019-09-24T00:00:00Z&BBOX=-26.523608349900595,-119.85108101391648,0.6927808151093444,-95.30684642147116&CRS=EPSG:4326&LAYERS=MODIS_Aqua_CorrectedReflectance_TrueColor,Coastlines&WRAP=day,x&FORMAT=image/jpeg&WIDTH=559&HEIGHT=619&ts=1569364996375\n\nhttps://wvs.earthdata.nasa.gov/api/v1/snapshot?REQUEST=GetSnapshot&TIME=2019-09-24T00:00:00Z&BBOX=-26.523608349900595,-119.85108101391648,0.6927808151093444,-95.30684642147116&CRS=EPSG:4326&LAYERS=MODIS_Terra_CorrectedReflectance_TrueColor,Coastlines&WRAP=day,x&FORMAT=image/jpeg&WIDTH=559&HEIGHT=619&ts=1569364996375\n\nUsing this information we can download thousands of pairs of images and train a Siamese Network (or something similar) to find similar images. I did this and found:\n\n1,434 pairs with two images in train (ie. 2,868 images total)\n1,813 pairs with one in train, one in test\n634 pairs with two images in test (ie. 1,276 images total)\n\n\n\nIn theory these images should have somewhat similar labels. Of course label noise makes it difficult to guarantee this, but I still think there must be some kind of signal we could use here. Ultimately I wasn't able to make use of this data but here's what I tried:\n\n- Modifying the network to accept inputs of 7 channels. \n - 3 for the RGB image, \n - 4 for the labels of the corresponding image pair.\n\n- Modifying the network to accept inputs of 10 channels, \n - 3 for the RGB image, \n - 3 for the pair RGB image\n - 4 for the labels of the corresponding image pair.\n\n- Using the weights of the Siamese Network I trained as my pretrained encoder weights instead of standard Imagenet weights. My thinking was that this model would have learned about features that would allow it to distinguish between clouds.\n\n\nI ran into a few challenges:\n - I don't think you can just add channels to the encoder input and expect your pretrained ImageNet weights to still work correctly. I believe this would change the distribution of activations we feed to the second layer?\n\n\nWhat I wish I tried:\n - Investigating using image pairs to help with a classifier (I only worked on segmentation models)\n - Investigating which classes had the best labels between pairs of images\n\n\nDid anyone manage to take advantage of this information? Does anyone have any insights into how we could have used this kind of information?", "681443": "Nice finding!\nHow to find out similar images, can you share the code?\n\nI think of Siamese Network too, but can not find out how to use it after training .", "682955": "I used an approach from [adambielski's repository](https://github.com/adambielski/siamese-triplet/blob/master/Experiments_MNIST.ipynb). \n\nI get the satellite images here: https://github.com/JoshVarty/KaggleClouds/blob/master/04_ImageSimilarity_GetData.ipynb\n\nI train with OnlineContrastiveLoss here: \nhttps://github.com/JoshVarty/KaggleClouds/blob/master/04_ImageSimilarity_OnlineContrastiveLoss.ipynb" }, "source": "meta" }