{
  "id": 279668,
  "title": "Did any of our models really learn?",
  "url": "/competitions/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/279668",
  "author_name": "",
  "post_date": "2021-10-18T21:33:49.871234300Z",
  "votes": 5,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Looking at the private leaderboard, it seems that people shot up or down up to 1000 places (lol) compared to the public leaderboard which seems to suggest very high volatility/lack of learning of the ML models.</p>",
  "messages": [
    {
      "id": "1549246",
      "postDate": "10/18/2021 21:33:49",
      "content": "<p>Looking at the private leaderboard, it seems that people shot up or down up to 1000 places (lol) compared to the public leaderboard which seems to suggest very high volatility/lack of learning of the ML models.</p>",
      "rawMarkdown": "Looking at the private leaderboard, it seems that people shot up or down up to 1000 places (lol) compared to the public leaderboard which seems to suggest very high volatility/lack of learning of the ML models.",
      "votes": null
    },
    {
      "id": "1549255",
      "postDate": "10/18/2021 21:40:10",
      "content": "<p>~0.6 AUROC? Somewhat better than tossing a coin… this means for me, that the information is in there, but maybe only  subgroups of the whole cohort, maybe only in specialised cases… It would be so great to know other biomarkers of the cases to be able to classify better!</p>",
      "rawMarkdown": "~0.6 AUROC? Somewhat better than tossing a coin... this means for me, that the information is in there, but maybe only  subgroups of the whole cohort, maybe only in specialised cases... It would be so great to know other biomarkers of the cases to be able to classify better!",
      "votes": null
    },
    {
      "id": "1549317",
      "postDate": "10/18/2021 22:59:03",
      "content": "<p>Training on 500 data samples doesn't help either 🙂</p>",
      "rawMarkdown": "Training on 500 data samples doesn't help either 🙂",
      "votes": null
    },
    {
      "id": "1549439",
      "postDate": "10/18/2021 23:58:47",
      "content": "<p>Using the <a href=\"https://www.kaggle.com/lars123/leak-in-metadata\" target=\"_blank\">metadata</a> alone got me to 0.55743. This is better than my CNN with 0.51.</p>\n<p>For this reason, I think we really learned a lot of noise. Examples: CT scans with ground truth 1 are smaller/bigger, CT scans with ground truth 1 are brighter/darker, 1 comes up more often/less often…</p>",
      "rawMarkdown": "Using the [metadata](https://www.kaggle.com/lars123/leak-in-metadata) alone got me to 0.55743. This is better than my CNN with 0.51.\n\nFor this reason, I think we really learned a lot of noise. Examples: CT scans with ground truth 1 are smaller/bigger, CT scans with ground truth 1 are brighter/darker, 1 comes up more often/less often...",
      "votes": null
    },
    {
      "id": "1549480",
      "postDate": "10/19/2021 01:21:08",
      "content": "<p>You would expect 0.500 auc under random probabilistic prediction, an auc of 0.6 in my opinion can be attributed to randomness alone </p>",
      "rawMarkdown": "You would expect 0.500 auc under random probabilistic prediction, an auc of 0.6 in my opinion can be attributed to randomness alone",
      "votes": null
    },
    {
      "id": "1549996",
      "postDate": "10/19/2021 10:55:17",
      "content": "<p>I am asking this question from the start ;)<br>\n<a href=\"https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/266173#1478867\" target=\"_blank\">https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/266173#1478867</a></p>",
      "rawMarkdown": "I am asking this question from the start ;)\nhttps://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/266173#1478867",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1549255,
      "author_name": "sandorkonya",
      "author_url": "",
      "post_date": "10/18/2021 21:40:10",
      "content": "<p>~0.6 AUROC? Somewhat better than tossing a coin… this means for me, that the information is in there, but maybe only  subgroups of the whole cohort, maybe only in specialised cases… It would be so great to know other biomarkers of the cases to be able to classify better!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1549317,
          "author_name": "mikecho",
          "author_url": "",
          "post_date": "10/18/2021 22:59:03",
          "content": "<p>Training on 500 data samples doesn't help either 🙂</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1549439,
      "author_name": "lars123",
      "author_url": "",
      "post_date": "10/18/2021 23:58:47",
      "content": "<p>Using the <a href=\"https://www.kaggle.com/lars123/leak-in-metadata\" target=\"_blank\">metadata</a> alone got me to 0.55743. This is better than my CNN with 0.51.</p>\n<p>For this reason, I think we really learned a lot of noise. Examples: CT scans with ground truth 1 are smaller/bigger, CT scans with ground truth 1 are brighter/darker, 1 comes up more often/less often…</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1549480,
      "author_name": "aryamansharma47",
      "author_url": "",
      "post_date": "10/19/2021 01:21:08",
      "content": "<p>You would expect 0.500 auc under random probabilistic prediction, an auc of 0.6 in my opinion can be attributed to randomness alone </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1549996,
      "author_name": "mrinath",
      "author_url": "",
      "post_date": "10/19/2021 10:55:17",
      "content": "<p>I am asking this question from the start ;)<br>\n<a href=\"https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/266173#1478867\" target=\"_blank\">https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/266173#1478867</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1549246": "Looking at the private leaderboard, it seems that people shot up or down up to 1000 places (lol) compared to the public leaderboard which seems to suggest very high volatility/lack of learning of the ML models.",
    "1549255": "~0.6 AUROC? Somewhat better than tossing a coin... this means for me, that the information is in there, but maybe only  subgroups of the whole cohort, maybe only in specialised cases... It would be so great to know other biomarkers of the cases to be able to classify better!",
    "1549317": "Training on 500 data samples doesn't help either 🙂",
    "1549439": "Using the [metadata](https://www.kaggle.com/lars123/leak-in-metadata) alone got me to 0.55743. This is better than my CNN with 0.51.\n\nFor this reason, I think we really learned a lot of noise. Examples: CT scans with ground truth 1 are smaller/bigger, CT scans with ground truth 1 are brighter/darker, 1 comes up more often/less often...",
    "1549480": "You would expect 0.500 auc under random probabilistic prediction, an auc of 0.6 in my opinion can be attributed to randomness alone",
    "1549996": "I am asking this question from the start ;)\nhttps://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/266173#1478867"
  },
  "source": "meta"
}