![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/446561/10946/a8d73536-bad8-11e8-b2b9-ac1f6b6435d0.png\")
There is much information scattering everywhere ... here is my consolidation. You are free to update more. thanks!
\n\n1) leaks (i.e. you know the label of some test images)
\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395
\n\nhttps://storage.googleapis.com/kaggle-forum-message-attachments/433234/10820/overlap.py
\n\n(i did not use this, so i cannot comment more)
\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/72534
\n\nTestEtraMatchingUnder_259_R14_G12_B10.csv
\n\nActually there are only 257 leak (there are 2 duplicates). improvement to public LB is could be 0.06 (depending if you have already predicted the leaked images correctly or not). leak126 and leak259 have overlap images and they sometimes do not agree on the labels. Personally, i think leak259 is more accurate?
\n\nupdate:
\n\nif your model is in the range of public LB 0.55, it may better not to use the leaks. (see below) Using leaks decrease my score by 0.001.
\n\n2) external data
\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/69984
\n\nHPAv18RBGY_wodpl.csv
\n\nHPAv18RBGY_wodpl has 74606 images. I do not know if HPAv18RBGY_wodpl has duplicates with kaggle images or the leak images.
", "messages": [ { "id": 445481, "postDate": "2018-12-26T14:55:16.337Z", "content": "There is much information scattering everywhere ... here is my consolidation. You are free to update more. thanks!
\n\n1) leaks (i.e. you know the label of some test images)
\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395
\n\nhttps://storage.googleapis.com/kaggle-forum-message-attachments/433234/10820/overlap.py
\n\n(i did not use this, so i cannot comment more)
\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/72534
\n\nTestEtraMatchingUnder_259_R14_G12_B10.csv
\n\nActually there are only 257 leak (there are 2 duplicates). improvement to public LB is could be 0.06 (depending if you have already predicted the leaked images correctly or not). leak126 and leak259 have overlap images and they sometimes do not agree on the labels. Personally, i think leak259 is more accurate?
\n\nupdate:
\n\nif your model is in the range of public LB 0.55, it may better not to use the leaks. (see below) Using leaks decrease my score by 0.001.
\n\n2) external data
\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/69984
\n\nHPAv18RBGY_wodpl.csv
\n\nHPAv18RBGY_wodpl has 74606 images. I do not know if HPAv18RBGY_wodpl has duplicates with kaggle images or the leak images.
", "rawMarkdown": "There is much information scattering everywhere ... here is my consolidation. You are free to update more. thanks!\n\n---\n\n1) leaks (i.e. you know the label of some test images)\n\n - leak126\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395\n\nhttps://storage.googleapis.com/kaggle-forum-message-attachments/433234/10820/overlap.py\n\n(i did not use this, so i cannot comment more)\n\n\n\n\n\n - leak259\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/72534\n\nTestEtraMatchingUnder_259_R14_G12_B10.csv \n\nActually there are only 257 leak (there are 2 duplicates). improvement to public LB is could be 0.06 (depending if you have already predicted the leaked images correctly or not). leak126 and leak259 have overlap images and they sometimes do not agree on the labels. Personally, i think leak259 is more accurate?\n\n\n*update*:\n\nif your model is in the range of public LB 0.55, it may better not to use the leaks. (see below) Using leaks decrease my score by 0.001.\n\n----\n\n2) external data\n\n - HPAv18RBGY_wodpl\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/69984\n\nHPAv18RBGY_wodpl.csv\n\nHPAv18RBGY_wodpl has 74606 images. I do not know if HPAv18RBGY_wodpl has duplicates with kaggle images or the leak images.\n", "votes": 16 }, { "id": 446102, "postDate": "2018-12-27T13:54:55.160Z", "content": "External dataset HPAv18RBGYwodpl actually has intersection with kaggle data. At least I saw a few images from kaggle dataset, which were just cropped from HPAv18RBGYwodpl. \nThough, I am not sure about how many images are intersecting.
", "rawMarkdown": "External dataset HPAv18RBGYwodpl actually has intersection with kaggle data. At least I saw a few images from kaggle dataset, which were just cropped from HPAv18RBGYwodpl. \nThough, I am not sure about how many images are intersecting.", "votes": 1 }, { "id": 447782, "postDate": "2018-12-30T15:05:54.943Z", "content": "is there any list of image ids of hpa external data that is not overlapping with the kaggle data? Thanks.
", "rawMarkdown": "is there any list of image ids of hpa external data that is not overlapping with the kaggle data? Thanks." }, { "id": 446561, "postDate": "2018-12-28T09:29:01.783Z", "content": "comparing hpa and kaggle data using leak259:
\n\n
![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/446561/10948/0a96bf2c-bad3-11e8-b2b8-ac1f6b6435d0.png\")
\n ![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/446561/10950/b54e8446-bacd-11e8-b2b8-ac1f6b6435d0.png\")
When I looked at this most of the hpa and test images in the leak259 are pretty similar in pixel intensity but there were a few odd differences (mainly in the green channel if I recall). Overall mean intensity stats per channel across both sets were very close.
", "rawMarkdown": "When I looked at this most of the hpa and test images in the leak259 are pretty similar in pixel intensity but there were a few odd differences (mainly in the green channel if I recall). Overall mean intensity stats per channel across both sets were very close." } ] }, { "id": 445779, "postDate": "2018-12-27T03:43:19.583Z", "content": "update:
\n\nThe leak259 is not completely correct?
\n\nsubmission with it sometimes lead to no change in LB score or worse score (although the predicted labels changes)
", "rawMarkdown": "update:\n\nThe leak259 is not completely correct?\n\nsubmission with it sometimes lead to no change in LB score or worse score (although the predicted labels changes)", "replies": [ { "id": 445800, "postDate": "2018-12-27T04:32:02.120Z", "content": "From philculliton (Kaggle Team) on the \"Let's talk about the HPA leak thread\":
\n\n\"On a related note, I should mention that hashing is not 100% accurate in this context, and that some of the \"leaked\" images are not, in fact, leaks (and further, some of the labels assigned them are incorrect). The leak appears to be so effective because some of the real, actual leaks include rare classes that have a high impact on LB score.
\n\nThe people who found the leak did excellent work, but it's very difficult to get completely right. I'd recommend not using the leak in any way.\"\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395#434012
\n\nAlso
\n\n\"I updated and reset the LB last night. Here's what changed:
\n\n1) The majority of the impacted samples are no longer included in scoring.
\n\n2) A small number of samples with very rare labels were not removed from scoring, but were shifted to the public leaderboard.\"\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395#433830
", "rawMarkdown": "From philculliton (Kaggle Team) on the \"Let's talk about the HPA leak thread\":\n\n\"On a related note, I should mention that hashing is not 100% accurate in this context, and that some of the \"leaked\" images are not, in fact, leaks (and further, some of the labels assigned them are incorrect). The leak appears to be so effective because some of the real, actual leaks include rare classes that have a high impact on LB score.\n\nThe people who found the leak did excellent work, but it's very difficult to get completely right. I'd recommend not using the leak in any way.\"\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395#434012\n\nAlso\n\n\"I updated and reset the LB last night. Here's what changed:\n\n1) The majority of the impacted samples are no longer included in scoring.\n\n2) A small number of samples with very rare labels were not removed from scoring, but were shifted to the public leaderboard.\"\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395#433830", "votes": 1 }, { "id": 447311, "postDate": "2018-12-29T15:58:06.083Z", "content": "@David Wagner
\n\nThanks!
", "rawMarkdown": "@David Wagner\n \n\nThanks!\n" } ] }, { "id": 445536, "postDate": "2018-12-26T17:08:22.543Z", "content": "Hi Heng, thanks for pointing this out.
\n\nCorrect if I am wrong. I have the impression that the leaks have been removed from the private set. Is the 0.06 improvement to public LB you mentioned tested before the leak removing or after?
", "rawMarkdown": "Hi Heng, thanks for pointing this out. \n\nCorrect if I am wrong. I have the impression that the leaks have been removed from the private set. Is the 0.06 improvement to public LB you mentioned tested before the leak removing or after? ", "replies": [ { "id": 445540, "postDate": "2018-12-26T17:15:35.470Z", "content": "0.06 improvement is for the public LB
", "rawMarkdown": " 0.06 improvement is for the public LB" }, { "id": 445649, "postDate": "2018-12-26T21:34:37.117Z", "content": "Just did an experiment on my shitty submission, simply replacing the predicted targets by the 259 leaks gives ~0.06 boost for the public LB.
\n\nI guess the boost brought by this will be weaker for:
\n\nAs suggested by the organizer, the leak should have no impact on private LB.
", "rawMarkdown": "Just did an experiment on my shitty submission, simply replacing the predicted targets by the 259 leaks gives ~0.06 boost for the public LB. \n\nI guess the boost brought by this will be weaker for: \n\n1. Better model.\n2. Using all (partial) external data. \n\nAs suggested by the organizer, the leak should have no impact on private LB. " }, { "id": 448750, "postDate": "2019-01-02T02:25:53.257Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 446102, "author_name": "Yurii Dzeryn", "author_url": "", "post_date": "2018-12-27T13:54:55.160000", "content": "External dataset HPAv18RBGYwodpl actually has intersection with kaggle data. At least I saw a few images from kaggle dataset, which were just cropped from HPAv18RBGYwodpl. \nThough, I am not sure about how many images are intersecting.
", "votes": 1, "replies": [] }, { "id": 447782, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-30T15:05:54.943000", "content": "is there any list of image ids of hpa external data that is not overlapping with the kaggle data? Thanks.
", "votes": 0, "replies": [] }, { "id": 446561, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-28T09:29:01.783000", "content": "comparing hpa and kaggle data using leak259:
\n\n
\n
When I looked at this most of the hpa and test images in the leak259 are pretty similar in pixel intensity but there were a few odd differences (mainly in the green channel if I recall). Overall mean intensity stats per channel across both sets were very close.
", "votes": 0, "replies": [] } ] }, { "id": 445779, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-27T03:43:19.583000", "content": "update:
\n\nThe leak259 is not completely correct?
\n\nsubmission with it sometimes lead to no change in LB score or worse score (although the predicted labels changes)
", "votes": 0, "replies": [ { "id": 445800, "author_name": "David Wagner", "author_url": "", "post_date": "2018-12-27T04:32:02.120000", "content": "From philculliton (Kaggle Team) on the \"Let's talk about the HPA leak thread\":
\n\n\"On a related note, I should mention that hashing is not 100% accurate in this context, and that some of the \"leaked\" images are not, in fact, leaks (and further, some of the labels assigned them are incorrect). The leak appears to be so effective because some of the real, actual leaks include rare classes that have a high impact on LB score.
\n\nThe people who found the leak did excellent work, but it's very difficult to get completely right. I'd recommend not using the leak in any way.\"\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395#434012
\n\nAlso
\n\n\"I updated and reset the LB last night. Here's what changed:
\n\n1) The majority of the impacted samples are no longer included in scoring.
\n\n2) A small number of samples with very rare labels were not removed from scoring, but were shifted to the public leaderboard.\"\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395#433830
", "votes": 1, "replies": [] }, { "id": 447311, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-29T15:58:06.083000", "content": "@David Wagner
\n\nThanks!
", "votes": 0, "replies": [] } ] }, { "id": 445536, "author_name": "Xuan Cao", "author_url": "", "post_date": "2018-12-26T17:08:22.543000", "content": "Hi Heng, thanks for pointing this out.
\n\nCorrect if I am wrong. I have the impression that the leaks have been removed from the private set. Is the 0.06 improvement to public LB you mentioned tested before the leak removing or after?
", "votes": 0, "replies": [ { "id": 445540, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-26T17:15:35.470000", "content": "0.06 improvement is for the public LB
", "votes": 0, "replies": [] }, { "id": 445649, "author_name": "Xuan Cao", "author_url": "", "post_date": "2018-12-26T21:34:37.117000", "content": "Just did an experiment on my shitty submission, simply replacing the predicted targets by the 259 leaks gives ~0.06 boost for the public LB.
\n\nI guess the boost brought by this will be weaker for:
\n\nAs suggested by the organizer, the leak should have no impact on private LB.
", "votes": 0, "replies": [] }, { "id": 448750, "author_name": "", "author_url": "", "post_date": "2019-01-02T02:25:53.257000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "445481": "There is much information scattering everywhere ... here is my consolidation. You are free to update more. thanks!\n\n---\n\n1) leaks (i.e. you know the label of some test images)\n\n - leak126\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/73395\n\nhttps://storage.googleapis.com/kaggle-forum-message-attachments/433234/10820/overlap.py\n\n(i did not use this, so i cannot comment more)\n\n\n\n\n\n - leak259\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/72534\n\nTestEtraMatchingUnder_259_R14_G12_B10.csv \n\nActually there are only 257 leak (there are 2 duplicates). improvement to public LB is could be 0.06 (depending if you have already predicted the leaked images correctly or not). leak126 and leak259 have overlap images and they sometimes do not agree on the labels. Personally, i think leak259 is more accurate?\n\n\n*update*:\n\nif your model is in the range of public LB 0.55, it may better not to use the leaks. (see below) Using leaks decrease my score by 0.001.\n\n----\n\n2) external data\n\n - HPAv18RBGY_wodpl\n\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/69984\n\nHPAv18RBGY_wodpl.csv\n\nHPAv18RBGY_wodpl has 74606 images. I do not know if HPAv18RBGY_wodpl has duplicates with kaggle images or the leak images.\n", "446102": "External dataset HPAv18RBGYwodpl actually has intersection with kaggle data. At least I saw a few images from kaggle dataset, which were just cropped from HPAv18RBGYwodpl. \nThough, I am not sure about how many images are intersecting.", "447782": "is there any list of image ids of hpa external data that is not overlapping with the kaggle data? Thanks.", "446561": "comparing hpa and kaggle data using leak259:\n\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n ![enter image description here][3]\n\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/446561/10946/a8d73536-bad8-11e8-b2b9-ac1f6b6435d0.png\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/446561/10948/0a96bf2c-bad3-11e8-b2b8-ac1f6b6435d0.png\n [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/446561/10950/b54e8446-bacd-11e8-b2b8-ac1f6b6435d0.png", "445779": "update:\n\nThe leak259 is not completely correct?\n\nsubmission with it sometimes lead to no change in LB score or worse score (although the predicted labels changes)", "445536": "Hi Heng, thanks for pointing this out. \n\nCorrect if I am wrong. I have the impression that the leaks have been removed from the private set. Is the 0.06 improvement to public LB you mentioned tested before the leak removing or after? " } }