{
  "id": 251808,
  "title": "Could this be a leak?",
  "url": "/competitions/siim-covid19-detection/discussion/251808",
  "author_name": "",
  "post_date": "2021-07-08T22:51:33.162163700Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I was messing with DICOM tags when I noticed that the distribution of some of the private tags isn't normal. </p>\n<p>It occurred to me that these could be part of the annotation process .. maybe not .. I dunno.</p>\n<p>I made a notebook to dump them out from the train set, and it seems there IS some correlation to the target classes for a few of these tags. </p>\n<p>I haven't looked through the test set, but I'm guessing all the sets were done the same way.</p>\n<p>Here's the notebook -&gt; <a href=\"https://www.kaggle.com/davidbroberts/private-tag-correlations\" target=\"_blank\">https://www.kaggle.com/davidbroberts/private-tag-correlations</a></p>\n<p>If you scroll down to the heatmap, you can see some of the tags seem important. Not the content, but the presence or absence of the tag itself.</p>",
  "messages": [
    {
      "id": "1381366",
      "postDate": "07/08/2021 22:51:33",
      "content": "<p>I was messing with DICOM tags when I noticed that the distribution of some of the private tags isn't normal. </p>\n<p>It occurred to me that these could be part of the annotation process .. maybe not .. I dunno.</p>\n<p>I made a notebook to dump them out from the train set, and it seems there IS some correlation to the target classes for a few of these tags. </p>\n<p>I haven't looked through the test set, but I'm guessing all the sets were done the same way.</p>\n<p>Here's the notebook -&gt; <a href=\"https://www.kaggle.com/davidbroberts/private-tag-correlations\" target=\"_blank\">https://www.kaggle.com/davidbroberts/private-tag-correlations</a></p>\n<p>If you scroll down to the heatmap, you can see some of the tags seem important. Not the content, but the presence or absence of the tag itself.</p>",
      "rawMarkdown": "I was messing with DICOM tags when I noticed that the distribution of some of the private tags isn't normal. \n\nIt occurred to me that these could be part of the annotation process .. maybe not .. I dunno.\n\nI made a notebook to dump them out from the train set, and it seems there IS some correlation to the target classes for a few of these tags. \n\nI haven't looked through the test set, but I'm guessing all the sets were done the same way.\n\nHere's the notebook -> https://www.kaggle.com/davidbroberts/private-tag-correlations\n\nIf you scroll down to the heatmap, you can see some of the tags seem important. Not the content, but the presence or absence of the tag itself.",
      "votes": null
    },
    {
      "id": "1381369",
      "postDate": "07/08/2021 22:59:36",
      "content": "<p>First of all, Great finding. The max <strong>corr</strong> is around <strong>0.22</strong> I'm not sure if it can be used to improve the score that much..</p>",
      "rawMarkdown": "First of all, Great finding. The max **corr** is around **0.22** I'm not sure if it can be used to improve the score that much..",
      "votes": null
    },
    {
      "id": "1383038",
      "postDate": "07/10/2021 12:52:01",
      "content": "<p>With the positive and negative correlations, it looks like there can be upwards of 0.5 corr for a couple of them. I haven't tried to see if it impacts the score or not.</p>",
      "rawMarkdown": "With the positive and negative correlations, it looks like there can be upwards of 0.5 corr for a couple of them. I haven't tried to see if it impacts the score or not.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1381369,
      "author_name": "awsaf49",
      "author_url": "",
      "post_date": "07/08/2021 22:59:36",
      "content": "<p>First of all, Great finding. The max <strong>corr</strong> is around <strong>0.22</strong> I'm not sure if it can be used to improve the score that much..</p>",
      "votes": null,
      "replies": [
        {
          "id": 1383038,
          "author_name": "davidbroberts",
          "author_url": "",
          "post_date": "07/10/2021 12:52:01",
          "content": "<p>With the positive and negative correlations, it looks like there can be upwards of 0.5 corr for a couple of them. I haven't tried to see if it impacts the score or not.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1381366": "I was messing with DICOM tags when I noticed that the distribution of some of the private tags isn't normal. \n\nIt occurred to me that these could be part of the annotation process .. maybe not .. I dunno.\n\nI made a notebook to dump them out from the train set, and it seems there IS some correlation to the target classes for a few of these tags. \n\nI haven't looked through the test set, but I'm guessing all the sets were done the same way.\n\nHere's the notebook -> https://www.kaggle.com/davidbroberts/private-tag-correlations\n\nIf you scroll down to the heatmap, you can see some of the tags seem important. Not the content, but the presence or absence of the tag itself.",
    "1381369": "First of all, Great finding. The max **corr** is around **0.22** I'm not sure if it can be used to improve the score that much..",
    "1383038": "With the positive and negative correlations, it looks like there can be upwards of 0.5 corr for a couple of them. I haven't tried to see if it impacts the score or not."
  },
  "source": "meta"
}