{
  "id": 156476,
  "title": "Target leaks hunt started ...",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/156476",
  "author_name": "",
  "post_date": "2020-06-06T10:22:10.311801200Z",
  "votes": 18,
  "comment_count": 30,
  "views": 0,
  "content": "<p>Main leaks for now : \n - JPEG image size \n - JPEG image mean color</p>\n\n<p>With a few other things and without any TF/PyTorch I managed to get to 0.8 LB.</p>\n\n<p>Code I used for the leaks : </p>\n\n<p>Image size : <br>\n```\nfrom tqdm import tqdm\nimport os\nimport pandas as pd </p>\n\n<p>train = pd.read_csv('train.csv')\ntrn_images = train['image_name'].values\ntrn_sizes = np.zeros(trn_images.shape[0])\nfor i, img_path in enumerate(tqdm(trn_images)):\n    trn_sizes[i] = os.path.getsize(os.path.join('jpeg/train/', f'{img_path}.jpg'))\n```</p>\n\n<p>Image average color : \n```\nfrom PIL import Image\nimport os</p>\n\n<p>trn_images = train['image_name'].values\ntrn_color = np.zeros(trn_images.shape[0])\nfor i, img_path in enumerate(tqdm(trn_images)):\n    trn_color[i] = np.array(Image.open(os.path.join('jpeg/train/', f'{img_path}.jpg'))).mean()\n```</p>\n\n<p>There is only a slight drift on these features ...</p>",
  "messages": [
    {
      "id": "875967",
      "postDate": "06/06/2020 10:22:10",
      "content": "<p>Main leaks for now : \n - JPEG image size \n - JPEG image mean color</p>\n\n<p>With a few other things and without any TF/PyTorch I managed to get to 0.8 LB.</p>\n\n<p>Code I used for the leaks : </p>\n\n<p>Image size : <br>\n```\nfrom tqdm import tqdm\nimport os\nimport pandas as pd </p>\n\n<p>train = pd.read_csv('train.csv')\ntrn_images = train['image_name'].values\ntrn_sizes = np.zeros(trn_images.shape[0])\nfor i, img_path in enumerate(tqdm(trn_images)):\n    trn_sizes[i] = os.path.getsize(os.path.join('jpeg/train/', f'{img_path}.jpg'))\n```</p>\n\n<p>Image average color : \n```\nfrom PIL import Image\nimport os</p>\n\n<p>trn_images = train['image_name'].values\ntrn_color = np.zeros(trn_images.shape[0])\nfor i, img_path in enumerate(tqdm(trn_images)):\n    trn_color[i] = np.array(Image.open(os.path.join('jpeg/train/', f'{img_path}.jpg'))).mean()\n```</p>\n\n<p>There is only a slight drift on these features ...</p>",
      "rawMarkdown": "Main leaks for now : \n - JPEG image size \n - JPEG image mean color\n\nWith a few other things and without any TF/PyTorch I managed to get to 0.8 LB.\n\nCode I used for the leaks : \n\nImage size :  \n```\nfrom tqdm import tqdm\nimport os\nimport pandas as pd \n\ntrain = pd.read_csv('train.csv')\ntrn_images = train['image_name'].values\ntrn_sizes = np.zeros(trn_images.shape[0])\nfor i, img_path in enumerate(tqdm(trn_images)):\n    trn_sizes[i] = os.path.getsize(os.path.join('jpeg/train/', f'{img_path}.jpg'))\n```\n\nImage average color : \n```\nfrom PIL import Image\nimport os\n\ntrn_images = train['image_name'].values\ntrn_color = np.zeros(trn_images.shape[0])\nfor i, img_path in enumerate(tqdm(trn_images)):\n    trn_color[i] = np.array(Image.open(os.path.join('jpeg/train/', f'{img_path}.jpg'))).mean()\n```\n\nThere is only a slight drift on these features ...",
      "votes": null
    },
    {
      "id": "875973",
      "postDate": "06/06/2020 10:28:14",
      "content": "<p>May I ask why would you call this a leak?</p>",
      "rawMarkdown": "May I ask why would you call this a leak?",
      "votes": null
    },
    {
      "id": "875975",
      "postDate": "06/06/2020 10:30:31",
      "content": "<p>It seems to me more like Images features extraction than leak.  But may be I'm wrong</p>",
      "rawMarkdown": "It seems to me more like Images features extraction than leak.  But may be I'm wrong",
      "votes": null
    },
    {
      "id": "875977",
      "postDate": "06/06/2020 10:32:55",
      "content": "<p>Image size ??? a feature ?  :) \nSome images in jpg are 3500 x 4500\nOthers are 1500 x 2000 \nThis should not give you any information about target</p>\n\n<p>Average color should not tell you anything either.</p>",
      "rawMarkdown": "Image size ??? a feature ?  :) \nSome images in jpg are 3500 x 4500\nOthers are 1500 x 2000 \nThis should not give you any information about target\n\nAverage color should not tell you anything either.",
      "votes": null
    },
    {
      "id": "875978",
      "postDate": "06/06/2020 10:33:19",
      "content": "<p>Great! <a href=\"/ogrellier\">@ogrellier</a>.\nI was exploring how the test data was generated. I was looking for something in \"image_name\" and \"patient_id\" for possible leaks. But no success yet! 😅</p>",
      "rawMarkdown": "Great! @ogrellier.\nI was exploring how the test data was generated. I was looking for something in \"image_name\" and \"patient_id\" for possible leaks. But no success yet! 😅",
      "votes": null
    },
    {
      "id": "875981",
      "postDate": "06/06/2020 10:37:27",
      "content": "<p>I could imagine image size to correlate quite well with the target. You might expect potentially malignant moles to be examined more carefully, with better equipment and thus higher resolution. </p>\n\n<p>The average color thing is also explainable as probably malignant images are darker on average.</p>\n\n<p>But of course I agree that the problem with this kind of information is not realistic, as you probably want to get an accurate diagnosis prior to any sophisticated tests.</p>",
      "rawMarkdown": "I could imagine image size to correlate quite well with the target. You might expect potentially malignant moles to be examined more carefully, with better equipment and thus higher resolution. \n\nThe average color thing is also explainable as probably malignant images are darker on average.\n\nBut of course I agree that the problem with this kind of information is not realistic, as you probably want to get an accurate diagnosis prior to any sophisticated tests.",
      "votes": null
    },
    {
      "id": "875986",
      "postDate": "06/06/2020 10:41:33",
      "content": "<p>And this is exactly a leak. </p>\n\n<p>Anything that shows you <code>after the facts</code> what people did because they <code>thought</code> a mole may be malignant is a leak. </p>\n\n<p>In real life you want to predict malignant moles before anyone thought anything, otherwise your model becomes useless when in production.</p>",
      "rawMarkdown": "And this is exactly a leak. \n\nAnything that shows you `after the facts` what people did because they `thought` a mole may be malignant is a leak. \n\nIn real life you want to predict malignant moles before anyone thought anything, otherwise your model becomes useless when in production.",
      "votes": null
    },
    {
      "id": "875987",
      "postDate": "06/06/2020 10:45:20",
      "content": "<p>Given same model can give different results depending on the resolution. Yes, I think images sizes may help to better detect malignant. </p>",
      "rawMarkdown": "Given same model can give different results depending on the resolution. Yes, I think images sizes may help to better detect malignant.",
      "votes": null
    },
    {
      "id": "875995",
      "postDate": "06/06/2020 10:53:57",
      "content": "<p>If the process is : \n 1. take standard pics\n 2. Have doubt on a mole\n 3. take high res pic to check if malignant</p>\n\n<p>Then high res pics are a leak by themselves. Doctors don't need a model to know that they took high res pics because they had a doubt :) </p>",
      "rawMarkdown": "If the process is : \n 1. take standard pics\n 2. Have doubt on a mole\n 3. take high res pic to check if malignant\n\nThen high res pics are a leak by themselves. Doctors don't need a model to know that they took high res pics because they had a doubt :)",
      "votes": null
    },
    {
      "id": "875999",
      "postDate": "06/06/2020 10:59:20",
      "content": "<p>Whether it's a leak depends on how exactly the host is planning to use the solutions. In case they want to apply them in the exact same setup (i.e. with the same test set distribution), I would say there is no leak.</p>",
      "rawMarkdown": "Whether it's a leak depends on how exactly the host is planning to use the solutions. In case they want to apply them in the exact same setup (i.e. with the same test set distribution), I would say there is no leak.",
      "votes": null
    },
    {
      "id": "876000",
      "postDate": "06/06/2020 11:00:40",
      "content": "<p>I got similar results.</p>\n\n<p>.69 from gender, age and site. Somewhat makes sense...?</p>\n\n<p>.74 by using the color by sampling imageSize / 13 750 pixels per image. Some sort of camera bias? Like <a href=\"https://www.kaggle.com/ddanevskyi\">Dmytro Danevskyi</a> pointed out, it could make sense, though...?</p>",
      "rawMarkdown": "I got similar results.\n\n.69 from gender, age and site. Somewhat makes sense...?\n\n.74 by using the color by sampling imageSize / 13 750 pixels per image. Some sort of camera bias? Like [Dmytro Danevskyi](https://www.kaggle.com/ddanevskyi) pointed out, it could make sense, though...?",
      "votes": null
    },
    {
      "id": "876088",
      "postDate": "06/06/2020 12:31:56",
      "content": "<p>I would not call it leak yet. I guess CNN model should do pretty well understanding the ratio of image dimensions, if you apply cv2.resize((512,512),.) operator during training.</p>",
      "rawMarkdown": "I would not call it leak yet. I guess CNN model should do pretty well understanding the ratio of image dimensions, if you apply cv2.resize((512,512),.) operator during training.",
      "votes": null
    },
    {
      "id": "876431",
      "postDate": "06/06/2020 17:20:49",
      "content": "<p>I tried to adjust Giba's simple baseline by including 2 most popular dimensions as feature categories:\n<a href=\"https://www.kaggle.com/raddar/simple-baseline-revamped?scriptVersionId=35600287\">https://www.kaggle.com/raddar/simple-baseline-revamped?scriptVersionId=35600287</a></p>\n\n<p>This scores ~0.75 on LB. however it does not help at all in my model ensemble</p>\n\n<p>Funny thing is, that if I would use all dimensions, the score would drop to 0.71! </p>\n\n<p>Overall, it's hard to call that a leak if you cannot use it :)</p>",
      "rawMarkdown": "I tried to adjust Giba's simple baseline by including 2 most popular dimensions as feature categories:\nhttps://www.kaggle.com/raddar/simple-baseline-revamped?scriptVersionId=35600287\n\nThis scores ~0.75 on LB. however it does not help at all in my model ensemble\n\nFunny thing is, that if I would use all dimensions, the score would drop to 0.71! \n\nOverall, it's hard to call that a leak if you cannot use it :)",
      "votes": null
    },
    {
      "id": "876437",
      "postDate": "06/06/2020 17:24:00",
      "content": "<p>Did I ever talked about dimensions ? \nI think I explicitly provided the features. Sorry you're unable to use them :) </p>",
      "rawMarkdown": "Did I ever talked about dimensions ? \nI think I explicitly provided the features. Sorry you're unable to use them :)",
      "votes": null
    },
    {
      "id": "876469",
      "postDate": "06/06/2020 17:50:33",
      "content": "<p>Taking a risk commenting at all...</p>\n\n<p>I'm not sure what the average colour is about but this quote indicates that colour, especially the ratio of green and red in an image, is diagnostic.</p>\n\n<blockquote>\n  <p>2012, Bekina et al. [7] analyzed lesions under a multispectral system with four different spectral bands, each one to obtain information from specific structures of the skin: 450 nm for superficial layers, 545 nm for blood distribution, 660 nm for melanin detection, and 940 nm for the evaluation of deeper skin layers. Then, a ratio was calculated between the intensities of green light (545 nm), where the hemoglobin absorption is high, and red light (660 nm), where it is low. It was proven that pathological tissues showed higher values of this index than the surrounding skin as a consequence of having higher blood content.  </p>\n</blockquote>\n\n<p><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5982599/\">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5982599/</a></p>",
      "rawMarkdown": "Taking a risk commenting at all...\n\nI'm not sure what the average colour is about but this quote indicates that colour, especially the ratio of green and red in an image, is diagnostic.\n\n&gt; 2012, Bekina et al. [7] analyzed lesions under a multispectral system with four different spectral bands, each one to obtain information from specific structures of the skin: 450 nm for superficial layers, 545 nm for blood distribution, 660 nm for melanin detection, and 940 nm for the evaluation of deeper skin layers. Then, a ratio was calculated between the intensities of green light (545 nm), where the hemoglobin absorption is high, and red light (660 nm), where it is low. It was proven that pathological tissues showed higher values of this index than the surrounding skin as a consequence of having higher blood content.  \n\n[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5982599/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5982599/)",
      "votes": null
    },
    {
      "id": "876478",
      "postDate": "06/06/2020 18:00:09",
      "content": "<p>Thanks ! this is very helpful. And maybe worth new features. \nI was surprised to see pictures with almost pink skin tones, that I was not expecting.\nInteresting to know that ratios between color channels is important.</p>\n\n<p>I guess the architecture most of the public kernels are using are capturing that but making our own features can give something :) </p>",
      "rawMarkdown": "Thanks ! this is very helpful. And maybe worth new features. \nI was surprised to see pictures with almost pink skin tones, that I was not expecting.\nInteresting to know that ratios between color channels is important.\n\nI guess the architecture most of the public kernels are using are capturing that but making our own features can give something :)",
      "votes": null
    },
    {
      "id": "876623",
      "postDate": "06/06/2020 20:58:11",
      "content": "<p><code>The average color thing is also explainable as probably malignant images are darker on average.</code></p>\n\n<p>I wouldn't be so sure. Note that people with darker skin are less prone to melanoma. More melanocytes helps to handle UV better.</p>",
      "rawMarkdown": "`The average color thing is also explainable as probably malignant images are darker on average.`\n\nI wouldn't be so sure. Note that people with darker skin are less prone to melanoma. More melanocytes helps to handle UV better.",
      "votes": null
    },
    {
      "id": "877220",
      "postDate": "06/07/2020 12:14:08",
      "content": "<p>I'm not quite sure of that ... at least it does not help me not getting all red after a few hours under the sun :) </p>",
      "rawMarkdown": "I'm not quite sure of that ... at least it does not help me not getting all red after a few hours under the sun :)",
      "votes": null
    },
    {
      "id": "877843",
      "postDate": "06/08/2020 03:44:32",
      "content": "<p>This  is really interesting. I can consider this feature also. Thanks a lot for information.</p>",
      "rawMarkdown": "This  is really interesting. I can consider this feature also. Thanks a lot for information.",
      "votes": null
    },
    {
      "id": "877846",
      "postDate": "06/08/2020 03:48:51",
      "content": "<p>0.8 LB for a good feature is not quite high. Since a simple grouped mean solution can also get 0.7 LB. \nMore interesting features may help us.</p>",
      "rawMarkdown": "0.8 LB for a good feature is not quite high. Since a simple grouped mean solution can also get 0.7 LB. \nMore interesting features may help us.",
      "votes": null
    },
    {
      "id": "953050",
      "postDate": "07/31/2020 13:30:54",
      "content": "<p>Why is average color a leak?  Color of lesion is the single mos t important predictor from what I read.</p>\n\n<p>Image size is another story...</p>",
      "rawMarkdown": "Why is average color a leak?  Color of lesion is the single mos t important predictor from what I read.\n\nImage size is another story...",
      "votes": null
    },
    {
      "id": "953697",
      "postDate": "08/01/2020 02:48:13",
      "content": "<p>The average color value is damaged by the images that contain the microscope black ring.  If you create a feature that evaluates black ring size and have an average color value feature,  the information from color improves.</p>",
      "rawMarkdown": "The average color value is damaged by the images that contain the microscope black ring.  If you create a feature that evaluates black ring size and have an average color value feature,  the information from color improves.",
      "votes": null
    },
    {
      "id": "953716",
      "postDate": "08/01/2020 03:24:00",
      "content": "<p>Image size has a strong correlation with target in train data but the same correlation does not exist in test set. The train set and test set have <strong>not</strong> been randomly chosen from the same distribution.</p>",
      "rawMarkdown": "Image size has a strong correlation with target in train data but the same correlation does not exist in test set. The train set and test set have **not** been randomly chosen from the same distribution.",
      "votes": null
    },
    {
      "id": "954675",
      "postDate": "08/02/2020 01:53:01",
      "content": "<p>True. Also if you center square crop before color analysis, you remove most microscope black rings and information from color improves.</p>",
      "rawMarkdown": "True. Also if you center square crop before color analysis, you remove most microscope black rings and information from color improves.",
      "votes": null
    },
    {
      "id": "954677",
      "postDate": "08/02/2020 01:56:17",
      "content": "<p>IMO, color of skin lesion isn't a leak, it is a data property. If it is color distortion from using different cameras, then it may be a leak.</p>\n\n<p>In train data, there are strong correlations between image size and target, but most of these correlations don't exist in test, so I'm not sure if I'd call it a leak or a trap.</p>",
      "rawMarkdown": "IMO, color of skin lesion isn't a leak, it is a data property. If it is color distortion from using different cameras, then it may be a leak.\n\nIn train data, there are strong correlations between image size and target, but most of these correlations don't exist in test, so I'm not sure if I'd call it a leak or a trap.",
      "votes": null
    },
    {
      "id": "954953",
      "postDate": "08/02/2020 08:04:00",
      "content": "<p>IMHO skin tone, hair and white balance should be sufficient to make average color of images random enough. But there may be a skin tone bias for melanoma...   </p>",
      "rawMarkdown": "IMHO skin tone, hair and white balance should be sufficient to make average color of images random enough. But there may be a skin tone bias for melanoma...",
      "votes": null
    },
    {
      "id": "954955",
      "postDate": "08/02/2020 08:05:40",
      "content": "<p>Yeah who am I to call for leaks ? LOL \nAnd in any case all this should be found/understood by DL models.</p>",
      "rawMarkdown": "Yeah who am I to call for leaks ? LOL \nAnd in any case all this should be found/understood by DL models.",
      "votes": null
    },
    {
      "id": "955096",
      "postDate": "08/02/2020 10:33:50",
      "content": "<blockquote>\n  <p>most of these correlations don't exist in test,</p>\n</blockquote>\n\n<p>Chris, how do you know that?</p>",
      "rawMarkdown": "&gt; most of these correlations don't exist in test,\n\nChris, how do you know that?",
      "votes": null
    },
    {
      "id": "955099",
      "postDate": "08/02/2020 10:35:05",
      "content": "<p>Olivier, there is the color of melanoma itself.  That's what I think your average color correlates with to some extent.</p>",
      "rawMarkdown": "Olivier, there is the color of melanoma itself.  That's what I think your average color correlates with to some extent.",
      "votes": null
    },
    {
      "id": "955103",
      "postDate": "08/02/2020 10:43:16",
      "content": "<p>If you would have a feature that describes the resolution, you could probe its public LB AUC. If the local AUC and LB AUC correspond you could probably deduce that there is some correlation (I did not test this myself though, but Chris did this for the Don't Overfit competition).</p>",
      "rawMarkdown": "If you would have a feature that describes the resolution, you could probe its public LB AUC. If the local AUC and LB AUC correspond you could probably deduce that there is some correlation (I did not test this myself though, but Chris did this for the Don't Overfit competition).",
      "votes": null
    },
    {
      "id": "955132",
      "postDate": "08/02/2020 11:23:57",
      "content": "<p><a href=\"/ogrellier\">@ogrellier</a> <a href=\"https://www.aimatmelanoma.org/melanoma-risk-factors/melanoma-in-people-of-color/\">https://www.aimatmelanoma.org/melanoma-risk-factors/melanoma-in-people-of-color/</a>\nThere is a skin tone bias for melanoma.</p>\n\n<p>Also the ABCD basic rule in dermatology tells you to look carefully about the colors of the lesion, independently of skin tone. So I'm really not surprised that you can get information only by looking at the colors.</p>",
      "rawMarkdown": "ogrellier https://www.aimatmelanoma.org/melanoma-risk-factors/melanoma-in-people-of-color/\nThere is a skin tone bias for melanoma.\n\nAlso the ABCD basic rule in dermatology tells you to look carefully about the colors of the lesion, independently of skin tone. So I'm really not surprised that you can get information only by looking at the colors.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 875973,
      "author_name": "ddanevskyi",
      "author_url": "",
      "post_date": "06/06/2020 10:28:14",
      "content": "<p>May I ask why would you call this a leak?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 875975,
      "author_name": "serigne",
      "author_url": "",
      "post_date": "06/06/2020 10:30:31",
      "content": "<p>It seems to me more like Images features extraction than leak.  But may be I'm wrong</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 875977,
      "author_name": "ogrellier",
      "author_url": "",
      "post_date": "06/06/2020 10:32:55",
      "content": "<p>Image size ??? a feature ?  :) \nSome images in jpg are 3500 x 4500\nOthers are 1500 x 2000 \nThis should not give you any information about target</p>\n\n<p>Average color should not tell you anything either.</p>",
      "votes": null,
      "replies": [
        {
          "id": 875987,
          "author_name": "serigne",
          "author_url": "",
          "post_date": "06/06/2020 10:45:20",
          "content": "<p>Given same model can give different results depending on the resolution. Yes, I think images sizes may help to better detect malignant. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 875995,
          "author_name": "ogrellier",
          "author_url": "",
          "post_date": "06/06/2020 10:53:57",
          "content": "<p>If the process is : \n 1. take standard pics\n 2. Have doubt on a mole\n 3. take high res pic to check if malignant</p>\n\n<p>Then high res pics are a leak by themselves. Doctors don't need a model to know that they took high res pics because they had a doubt :) </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 875978,
      "author_name": "meemr5",
      "author_url": "",
      "post_date": "06/06/2020 10:33:19",
      "content": "<p>Great! <a href=\"/ogrellier\">@ogrellier</a>.\nI was exploring how the test data was generated. I was looking for something in \"image_name\" and \"patient_id\" for possible leaks. But no success yet! 😅</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 875981,
      "author_name": "ddanevskyi",
      "author_url": "",
      "post_date": "06/06/2020 10:37:27",
      "content": "<p>I could imagine image size to correlate quite well with the target. You might expect potentially malignant moles to be examined more carefully, with better equipment and thus higher resolution. </p>\n\n<p>The average color thing is also explainable as probably malignant images are darker on average.</p>\n\n<p>But of course I agree that the problem with this kind of information is not realistic, as you probably want to get an accurate diagnosis prior to any sophisticated tests.</p>",
      "votes": null,
      "replies": [
        {
          "id": 875986,
          "author_name": "ogrellier",
          "author_url": "",
          "post_date": "06/06/2020 10:41:33",
          "content": "<p>And this is exactly a leak. </p>\n\n<p>Anything that shows you <code>after the facts</code> what people did because they <code>thought</code> a mole may be malignant is a leak. </p>\n\n<p>In real life you want to predict malignant moles before anyone thought anything, otherwise your model becomes useless when in production.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 875999,
          "author_name": "ddanevskyi",
          "author_url": "",
          "post_date": "06/06/2020 10:59:20",
          "content": "<p>Whether it's a leak depends on how exactly the host is planning to use the solutions. In case they want to apply them in the exact same setup (i.e. with the same test set distribution), I would say there is no leak.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 876623,
          "author_name": "valanm",
          "author_url": "",
          "post_date": "06/06/2020 20:58:11",
          "content": "<p><code>The average color thing is also explainable as probably malignant images are darker on average.</code></p>\n\n<p>I wouldn't be so sure. Note that people with darker skin are less prone to melanoma. More melanocytes helps to handle UV better.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 877220,
          "author_name": "ogrellier",
          "author_url": "",
          "post_date": "06/07/2020 12:14:08",
          "content": "<p>I'm not quite sure of that ... at least it does not help me not getting all red after a few hours under the sun :) </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 876000,
      "author_name": "glimmung",
      "author_url": "",
      "post_date": "06/06/2020 11:00:40",
      "content": "<p>I got similar results.</p>\n\n<p>.69 from gender, age and site. Somewhat makes sense...?</p>\n\n<p>.74 by using the color by sampling imageSize / 13 750 pixels per image. Some sort of camera bias? Like <a href=\"https://www.kaggle.com/ddanevskyi\">Dmytro Danevskyi</a> pointed out, it could make sense, though...?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 876088,
      "author_name": "raddar",
      "author_url": "",
      "post_date": "06/06/2020 12:31:56",
      "content": "<p>I would not call it leak yet. I guess CNN model should do pretty well understanding the ratio of image dimensions, if you apply cv2.resize((512,512),.) operator during training.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 876431,
      "author_name": "raddar",
      "author_url": "",
      "post_date": "06/06/2020 17:20:49",
      "content": "<p>I tried to adjust Giba's simple baseline by including 2 most popular dimensions as feature categories:\n<a href=\"https://www.kaggle.com/raddar/simple-baseline-revamped?scriptVersionId=35600287\">https://www.kaggle.com/raddar/simple-baseline-revamped?scriptVersionId=35600287</a></p>\n\n<p>This scores ~0.75 on LB. however it does not help at all in my model ensemble</p>\n\n<p>Funny thing is, that if I would use all dimensions, the score would drop to 0.71! </p>\n\n<p>Overall, it's hard to call that a leak if you cannot use it :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 876437,
          "author_name": "ogrellier",
          "author_url": "",
          "post_date": "06/06/2020 17:24:00",
          "content": "<p>Did I ever talked about dimensions ? \nI think I explicitly provided the features. Sorry you're unable to use them :) </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 876469,
      "author_name": "mutantspore",
      "author_url": "",
      "post_date": "06/06/2020 17:50:33",
      "content": "<p>Taking a risk commenting at all...</p>\n\n<p>I'm not sure what the average colour is about but this quote indicates that colour, especially the ratio of green and red in an image, is diagnostic.</p>\n\n<blockquote>\n  <p>2012, Bekina et al. [7] analyzed lesions under a multispectral system with four different spectral bands, each one to obtain information from specific structures of the skin: 450 nm for superficial layers, 545 nm for blood distribution, 660 nm for melanin detection, and 940 nm for the evaluation of deeper skin layers. Then, a ratio was calculated between the intensities of green light (545 nm), where the hemoglobin absorption is high, and red light (660 nm), where it is low. It was proven that pathological tissues showed higher values of this index than the surrounding skin as a consequence of having higher blood content.  </p>\n</blockquote>\n\n<p><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5982599/\">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5982599/</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 876478,
          "author_name": "ogrellier",
          "author_url": "",
          "post_date": "06/06/2020 18:00:09",
          "content": "<p>Thanks ! this is very helpful. And maybe worth new features. \nI was surprised to see pictures with almost pink skin tones, that I was not expecting.\nInteresting to know that ratios between color channels is important.</p>\n\n<p>I guess the architecture most of the public kernels are using are capturing that but making our own features can give something :) </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 877843,
          "author_name": "nlebang",
          "author_url": "",
          "post_date": "06/08/2020 03:44:32",
          "content": "<p>This  is really interesting. I can consider this feature also. Thanks a lot for information.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 877846,
      "author_name": "nlebang",
      "author_url": "",
      "post_date": "06/08/2020 03:48:51",
      "content": "<p>0.8 LB for a good feature is not quite high. Since a simple grouped mean solution can also get 0.7 LB. \nMore interesting features may help us.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 953050,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "07/31/2020 13:30:54",
      "content": "<p>Why is average color a leak?  Color of lesion is the single mos t important predictor from what I read.</p>\n\n<p>Image size is another story...</p>",
      "votes": null,
      "replies": [
        {
          "id": 954677,
          "author_name": "cdeotte",
          "author_url": "",
          "post_date": "08/02/2020 01:56:17",
          "content": "<p>IMO, color of skin lesion isn't a leak, it is a data property. If it is color distortion from using different cameras, then it may be a leak.</p>\n\n<p>In train data, there are strong correlations between image size and target, but most of these correlations don't exist in test, so I'm not sure if I'd call it a leak or a trap.</p>",
          "votes": null,
          "replies": [
            {
              "id": 954955,
              "author_name": "ogrellier",
              "author_url": "",
              "post_date": "08/02/2020 08:05:40",
              "content": "<p>Yeah who am I to call for leaks ? LOL \nAnd in any case all this should be found/understood by DL models.</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 954953,
          "author_name": "ogrellier",
          "author_url": "",
          "post_date": "08/02/2020 08:04:00",
          "content": "<p>IMHO skin tone, hair and white balance should be sufficient to make average color of images random enough. But there may be a skin tone bias for melanoma...   </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 955096,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "08/02/2020 10:33:50",
          "content": "<blockquote>\n  <p>most of these correlations don't exist in test,</p>\n</blockquote>\n\n<p>Chris, how do you know that?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 955099,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "08/02/2020 10:35:05",
          "content": "<p>Olivier, there is the color of melanoma itself.  That's what I think your average color correlates with to some extent.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 955103,
          "author_name": "group16",
          "author_url": "",
          "post_date": "08/02/2020 10:43:16",
          "content": "<p>If you would have a feature that describes the resolution, you could probe its public LB AUC. If the local AUC and LB AUC correspond you could probably deduce that there is some correlation (I did not test this myself though, but Chris did this for the Don't Overfit competition).</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 955132,
          "author_name": "optimo",
          "author_url": "",
          "post_date": "08/02/2020 11:23:57",
          "content": "<p><a href=\"/ogrellier\">@ogrellier</a> <a href=\"https://www.aimatmelanoma.org/melanoma-risk-factors/melanoma-in-people-of-color/\">https://www.aimatmelanoma.org/melanoma-risk-factors/melanoma-in-people-of-color/</a>\nThere is a skin tone bias for melanoma.</p>\n\n<p>Also the ABCD basic rule in dermatology tells you to look carefully about the colors of the lesion, independently of skin tone. So I'm really not surprised that you can get information only by looking at the colors.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 953697,
      "author_name": "pcjimmmy",
      "author_url": "",
      "post_date": "08/01/2020 02:48:13",
      "content": "<p>The average color value is damaged by the images that contain the microscope black ring.  If you create a feature that evaluates black ring size and have an average color value feature,  the information from color improves.</p>",
      "votes": null,
      "replies": [
        {
          "id": 954675,
          "author_name": "cdeotte",
          "author_url": "",
          "post_date": "08/02/2020 01:53:01",
          "content": "<p>True. Also if you center square crop before color analysis, you remove most microscope black rings and information from color improves.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 953716,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "08/01/2020 03:24:00",
      "content": "<p>Image size has a strong correlation with target in train data but the same correlation does not exist in test set. The train set and test set have <strong>not</strong> been randomly chosen from the same distribution.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "875967": "Main leaks for now : \n - JPEG image size \n - JPEG image mean color\n\nWith a few other things and without any TF/PyTorch I managed to get to 0.8 LB.\n\nCode I used for the leaks : \n\nImage size :  \n```\nfrom tqdm import tqdm\nimport os\nimport pandas as pd \n\ntrain = pd.read_csv('train.csv')\ntrn_images = train['image_name'].values\ntrn_sizes = np.zeros(trn_images.shape[0])\nfor i, img_path in enumerate(tqdm(trn_images)):\n    trn_sizes[i] = os.path.getsize(os.path.join('jpeg/train/', f'{img_path}.jpg'))\n```\n\nImage average color : \n```\nfrom PIL import Image\nimport os\n\ntrn_images = train['image_name'].values\ntrn_color = np.zeros(trn_images.shape[0])\nfor i, img_path in enumerate(tqdm(trn_images)):\n    trn_color[i] = np.array(Image.open(os.path.join('jpeg/train/', f'{img_path}.jpg'))).mean()\n```\n\nThere is only a slight drift on these features ...",
    "875973": "May I ask why would you call this a leak?",
    "875975": "It seems to me more like Images features extraction than leak.  But may be I'm wrong",
    "875977": "Image size ??? a feature ?  :) \nSome images in jpg are 3500 x 4500\nOthers are 1500 x 2000 \nThis should not give you any information about target\n\nAverage color should not tell you anything either.",
    "875978": "Great! @ogrellier.\nI was exploring how the test data was generated. I was looking for something in \"image_name\" and \"patient_id\" for possible leaks. But no success yet! 😅",
    "875981": "I could imagine image size to correlate quite well with the target. You might expect potentially malignant moles to be examined more carefully, with better equipment and thus higher resolution. \n\nThe average color thing is also explainable as probably malignant images are darker on average.\n\nBut of course I agree that the problem with this kind of information is not realistic, as you probably want to get an accurate diagnosis prior to any sophisticated tests.",
    "875986": "And this is exactly a leak. \n\nAnything that shows you `after the facts` what people did because they `thought` a mole may be malignant is a leak. \n\nIn real life you want to predict malignant moles before anyone thought anything, otherwise your model becomes useless when in production.",
    "875987": "Given same model can give different results depending on the resolution. Yes, I think images sizes may help to better detect malignant.",
    "875995": "If the process is : \n 1. take standard pics\n 2. Have doubt on a mole\n 3. take high res pic to check if malignant\n\nThen high res pics are a leak by themselves. Doctors don't need a model to know that they took high res pics because they had a doubt :)",
    "875999": "Whether it's a leak depends on how exactly the host is planning to use the solutions. In case they want to apply them in the exact same setup (i.e. with the same test set distribution), I would say there is no leak.",
    "876000": "I got similar results.\n\n.69 from gender, age and site. Somewhat makes sense...?\n\n.74 by using the color by sampling imageSize / 13 750 pixels per image. Some sort of camera bias? Like [Dmytro Danevskyi](https://www.kaggle.com/ddanevskyi) pointed out, it could make sense, though...?",
    "876088": "I would not call it leak yet. I guess CNN model should do pretty well understanding the ratio of image dimensions, if you apply cv2.resize((512,512),.) operator during training.",
    "876431": "I tried to adjust Giba's simple baseline by including 2 most popular dimensions as feature categories:\nhttps://www.kaggle.com/raddar/simple-baseline-revamped?scriptVersionId=35600287\n\nThis scores ~0.75 on LB. however it does not help at all in my model ensemble\n\nFunny thing is, that if I would use all dimensions, the score would drop to 0.71! \n\nOverall, it's hard to call that a leak if you cannot use it :)",
    "876437": "Did I ever talked about dimensions ? \nI think I explicitly provided the features. Sorry you're unable to use them :)",
    "876469": "Taking a risk commenting at all...\n\nI'm not sure what the average colour is about but this quote indicates that colour, especially the ratio of green and red in an image, is diagnostic.\n\n&gt; 2012, Bekina et al. [7] analyzed lesions under a multispectral system with four different spectral bands, each one to obtain information from specific structures of the skin: 450 nm for superficial layers, 545 nm for blood distribution, 660 nm for melanin detection, and 940 nm for the evaluation of deeper skin layers. Then, a ratio was calculated between the intensities of green light (545 nm), where the hemoglobin absorption is high, and red light (660 nm), where it is low. It was proven that pathological tissues showed higher values of this index than the surrounding skin as a consequence of having higher blood content.  \n\n[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5982599/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5982599/)",
    "876478": "Thanks ! this is very helpful. And maybe worth new features. \nI was surprised to see pictures with almost pink skin tones, that I was not expecting.\nInteresting to know that ratios between color channels is important.\n\nI guess the architecture most of the public kernels are using are capturing that but making our own features can give something :)",
    "876623": "`The average color thing is also explainable as probably malignant images are darker on average.`\n\nI wouldn't be so sure. Note that people with darker skin are less prone to melanoma. More melanocytes helps to handle UV better.",
    "877220": "I'm not quite sure of that ... at least it does not help me not getting all red after a few hours under the sun :)",
    "877843": "This  is really interesting. I can consider this feature also. Thanks a lot for information.",
    "877846": "0.8 LB for a good feature is not quite high. Since a simple grouped mean solution can also get 0.7 LB. \nMore interesting features may help us.",
    "953050": "Why is average color a leak?  Color of lesion is the single mos t important predictor from what I read.\n\nImage size is another story...",
    "953697": "The average color value is damaged by the images that contain the microscope black ring.  If you create a feature that evaluates black ring size and have an average color value feature,  the information from color improves.",
    "953716": "Image size has a strong correlation with target in train data but the same correlation does not exist in test set. The train set and test set have **not** been randomly chosen from the same distribution.",
    "954675": "True. Also if you center square crop before color analysis, you remove most microscope black rings and information from color improves.",
    "954677": "IMO, color of skin lesion isn't a leak, it is a data property. If it is color distortion from using different cameras, then it may be a leak.\n\nIn train data, there are strong correlations between image size and target, but most of these correlations don't exist in test, so I'm not sure if I'd call it a leak or a trap.",
    "954953": "IMHO skin tone, hair and white balance should be sufficient to make average color of images random enough. But there may be a skin tone bias for melanoma...",
    "954955": "Yeah who am I to call for leaks ? LOL \nAnd in any case all this should be found/understood by DL models.",
    "955096": "&gt; most of these correlations don't exist in test,\n\nChris, how do you know that?",
    "955099": "Olivier, there is the color of melanoma itself.  That's what I think your average color correlates with to some extent.",
    "955103": "If you would have a feature that describes the resolution, you could probe its public LB AUC. If the local AUC and LB AUC correspond you could probably deduce that there is some correlation (I did not test this myself though, but Chris did this for the Don't Overfit competition).",
    "955132": "ogrellier https://www.aimatmelanoma.org/melanoma-risk-factors/melanoma-in-people-of-color/\nThere is a skin tone bias for melanoma.\n\nAlso the ABCD basic rule in dermatology tells you to look carefully about the colors of the lesion, independently of skin tone. So I'm really not surprised that you can get information only by looking at the colors."
  },
  "source": "meta"
}