{
  "id": 232913,
  "title": "Score in NB vs Public LB",
  "url": "/competitions/birdclef-2021/discussion/232913",
  "author_name": "",
  "post_date": "2021-04-16T03:42:38.949787900Z",
  "votes": 2,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Hi<br>\nThere seems to be a difference of 0.01 in the scores showing in the notebooks vs what I see in my submission page<br>\nFor instance is my code/notebooks shows a submission score of 0.57,  my submission notebook (and public LB) shows a score of 0.58</p>\n<p>Are others facing the same issue… is it a bug? do we know why there is a difference?</p>",
  "messages": [
    {
      "id": "1275185",
      "postDate": "04/16/2021 03:42:38",
      "content": "<p>Hi<br>\nThere seems to be a difference of 0.01 in the scores showing in the notebooks vs what I see in my submission page<br>\nFor instance is my code/notebooks shows a submission score of 0.57,  my submission notebook (and public LB) shows a score of 0.58</p>\n<p>Are others facing the same issue… is it a bug? do we know why there is a difference?</p>",
      "rawMarkdown": "Hi\nThere seems to be a difference of 0.01 in the scores showing in the notebooks vs what I see in my submission page\nFor instance is my code/notebooks shows a submission score of 0.57,  my submission notebook (and public LB) shows a score of 0.58\n\nAre others facing the same issue... is it a bug? do we know why there is a difference?",
      "votes": null
    },
    {
      "id": "1275718",
      "postDate": "04/16/2021 15:54:50",
      "content": "<p>This looks like a bug, I'll get it filed. The score within the notebook doesn't look to be tracking the scores from new versions properly.</p>",
      "rawMarkdown": "This looks like a bug, I'll get it filed. The score within the notebook doesn't look to be tracking the scores from new versions properly.",
      "votes": null
    },
    {
      "id": "1275769",
      "postDate": "04/16/2021 17:08:50",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> !! Appreciate your response and thanks for looking into it</p>",
      "rawMarkdown": "Thanks @sohier !! Appreciate your response and thanks for looking into it",
      "votes": null
    },
    {
      "id": "1277111",
      "postDate": "04/18/2021 12:11:13",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> , once you fix this… may we also increase the decimal points of the public LB to 3 or 4 decimal points? All the subs seem so close and its a bit difficult to assess relative performance of submissions…</p>\n<p>Thanks!</p>",
      "rawMarkdown": "Hi @sohier , once you fix this... may we also increase the decimal points of the public LB to 3 or 4 decimal points? All the subs seem so close and its a bit difficult to assess relative performance of submissions...\n\nThanks!",
      "votes": null
    },
    {
      "id": "1277186",
      "postDate": "04/18/2021 13:54:38",
      "content": "<p>You can sort your submissions by public score.</p>\n<p>Limiting the display accuracy is meant to prevent LB probing.  I doubt it will be changed.  If you want to blame someone for this low digit display then blame me as I am the one who suggested limited display accuracy to Kaggle…</p>",
      "rawMarkdown": "You can sort your submissions by public score.\n\nLimiting the display accuracy is meant to prevent LB probing.  I doubt it will be changed.  If you want to blame someone for this low digit display then blame me as I am the one who suggested limited display accuracy to Kaggle...",
      "votes": null
    },
    {
      "id": "1277196",
      "postDate": "04/18/2021 14:07:16",
      "content": "<p>Thanks for clarifying <a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a> .. and while I agree it limits LB probing, I notice a wide range of decimal points across competitions,  and unsure of the reason for  the difference in this<br>\nsome comps have 3-4 decimal points and some 2…  <br>\nFor instance, Hubmap, HPA, Shoppee, Indoor all have 3 decimal points which Is why i asked…</p>",
      "rawMarkdown": "Thanks for clarifying @cpmpml .. and while I agree it limits LB probing, I notice a wide range of decimal points across competitions,  and unsure of the reason for  the difference in this\nsome comps have 3-4 decimal points and some 2...  \nFor instance, Hubmap, HPA, Shoppee, Indoor all have 3 decimal points which Is why i asked...",
      "votes": null
    },
    {
      "id": "1277309",
      "postDate": "04/18/2021 16:08:31",
      "content": "<p>It depends on the metric used and the number of samples in public test data.  You want that the display doe snot change if you change one sample prediction (or maybe a handful number of predictions to be safe).</p>",
      "rawMarkdown": "It depends on the metric used and the number of samples in public test data.  You want that the display doe snot change if you change one sample prediction (or maybe a handful number of predictions to be safe).",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1275718,
      "author_name": "sohier",
      "author_url": "",
      "post_date": "04/16/2021 15:54:50",
      "content": "<p>This looks like a bug, I'll get it filed. The score within the notebook doesn't look to be tracking the scores from new versions properly.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1275769,
          "author_name": "kmldas",
          "author_url": "",
          "post_date": "04/16/2021 17:08:50",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> !! Appreciate your response and thanks for looking into it</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1277111,
          "author_name": "kmldas",
          "author_url": "",
          "post_date": "04/18/2021 12:11:13",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> , once you fix this… may we also increase the decimal points of the public LB to 3 or 4 decimal points? All the subs seem so close and its a bit difficult to assess relative performance of submissions…</p>\n<p>Thanks!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1277186,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "04/18/2021 13:54:38",
          "content": "<p>You can sort your submissions by public score.</p>\n<p>Limiting the display accuracy is meant to prevent LB probing.  I doubt it will be changed.  If you want to blame someone for this low digit display then blame me as I am the one who suggested limited display accuracy to Kaggle…</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1277196,
          "author_name": "kmldas",
          "author_url": "",
          "post_date": "04/18/2021 14:07:16",
          "content": "<p>Thanks for clarifying <a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a> .. and while I agree it limits LB probing, I notice a wide range of decimal points across competitions,  and unsure of the reason for  the difference in this<br>\nsome comps have 3-4 decimal points and some 2…  <br>\nFor instance, Hubmap, HPA, Shoppee, Indoor all have 3 decimal points which Is why i asked…</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1277309,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "04/18/2021 16:08:31",
          "content": "<p>It depends on the metric used and the number of samples in public test data.  You want that the display doe snot change if you change one sample prediction (or maybe a handful number of predictions to be safe).</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1275185": "Hi\nThere seems to be a difference of 0.01 in the scores showing in the notebooks vs what I see in my submission page\nFor instance is my code/notebooks shows a submission score of 0.57,  my submission notebook (and public LB) shows a score of 0.58\n\nAre others facing the same issue... is it a bug? do we know why there is a difference?",
    "1275718": "This looks like a bug, I'll get it filed. The score within the notebook doesn't look to be tracking the scores from new versions properly.",
    "1275769": "Thanks @sohier !! Appreciate your response and thanks for looking into it",
    "1277111": "Hi @sohier , once you fix this... may we also increase the decimal points of the public LB to 3 or 4 decimal points? All the subs seem so close and its a bit difficult to assess relative performance of submissions...\n\nThanks!",
    "1277186": "You can sort your submissions by public score.\n\nLimiting the display accuracy is meant to prevent LB probing.  I doubt it will be changed.  If you want to blame someone for this low digit display then blame me as I am the one who suggested limited display accuracy to Kaggle...",
    "1277196": "Thanks for clarifying @cpmpml .. and while I agree it limits LB probing, I notice a wide range of decimal points across competitions,  and unsure of the reason for  the difference in this\nsome comps have 3-4 decimal points and some 2...  \nFor instance, Hubmap, HPA, Shoppee, Indoor all have 3 decimal points which Is why i asked...",
    "1277309": "It depends on the metric used and the number of samples in public test data.  You want that the display doe snot change if you change one sample prediction (or maybe a handful number of predictions to be safe)."
  },
  "source": "meta"
}