{
  "id": 220606,
  "title": "What contributed to the improvement of the Private LB score?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/220606",
  "author_name": "",
  "post_date": "2021-02-19T00:42:19.443040600Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I was in the bronze range in public LB, but out of the medal range in private LB.</p>\n<p>There were some submissions that were within medal range, but I couldn't pick them out!</p>\n<p>Please tell me what you do when learning and reasoning.</p>",
  "messages": [
    {
      "id": "1209584",
      "postDate": "02/19/2021 00:42:19",
      "content": "<p>I was in the bronze range in public LB, but out of the medal range in private LB.</p>\n<p>There were some submissions that were within medal range, but I couldn't pick them out!</p>\n<p>Please tell me what you do when learning and reasoning.</p>",
      "rawMarkdown": "I was in the bronze range in public LB, but out of the medal range in private LB.\n\nThere were some submissions that were within medal range, but I couldn't pick them out!\n\nPlease tell me what you do when learning and reasoning.",
      "votes": null
    },
    {
      "id": "1209597",
      "postDate": "02/19/2021 00:54:34",
      "content": "<p>Trusting local CV while comparing two models is a better indicator vs what they score on a subset of the LB which is public. The private LB (69% of LB) distribution  and image types could have (and in this case were) different from the public ones (31% of LB). The public part of the LB is supposed to be a good metric to evaluate a model but you have to be careful to not use that as the gold standard for generalized models.  Otherwise you are essentially picking models that are overfitting the public LB data.  </p>\n<p>Also going with your gut on what is better practice and a more robust model. For example: I did not pick my best performing public LB because I had not done enough augmentations and TTA vs another one that had a -0.002 score on public LB.   I selected the one that had TTA and a lesser public score and it landed up being better on private which is something I was hoping for. </p>",
      "rawMarkdown": "Trusting local CV while comparing two models is a better indicator vs what they score on a subset of the LB which is public. The private LB (69% of LB) distribution  and image types could have (and in this case were) different from the public ones (31% of LB). The public part of the LB is supposed to be a good metric to evaluate a model but you have to be careful to not use that as the gold standard for generalized models.  Otherwise you are essentially picking models that are overfitting the public LB data.  \n\nAlso going with your gut on what is better practice and a more robust model. For example: I did not pick my best performing public LB because I had not done enough augmentations and TTA vs another one that had a -0.002 score on public LB.   I selected the one that had TTA and a lesser public score and it landed up being better on private which is something I was hoping for.",
      "votes": null
    },
    {
      "id": "1209668",
      "postDate": "02/19/2021 02:18:09",
      "content": "<p>Thanks for the comment.<br>\nThis competition was at the mercy of LB!</p>",
      "rawMarkdown": "Thanks for the comment.\nThis competition was at the mercy of LB!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1209597,
      "author_name": "trushk",
      "author_url": "",
      "post_date": "02/19/2021 00:54:34",
      "content": "<p>Trusting local CV while comparing two models is a better indicator vs what they score on a subset of the LB which is public. The private LB (69% of LB) distribution  and image types could have (and in this case were) different from the public ones (31% of LB). The public part of the LB is supposed to be a good metric to evaluate a model but you have to be careful to not use that as the gold standard for generalized models.  Otherwise you are essentially picking models that are overfitting the public LB data.  </p>\n<p>Also going with your gut on what is better practice and a more robust model. For example: I did not pick my best performing public LB because I had not done enough augmentations and TTA vs another one that had a -0.002 score on public LB.   I selected the one that had TTA and a lesser public score and it landed up being better on private which is something I was hoping for. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1209668,
          "author_name": "nemuri1231",
          "author_url": "",
          "post_date": "02/19/2021 02:18:09",
          "content": "<p>Thanks for the comment.<br>\nThis competition was at the mercy of LB!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1209584": "I was in the bronze range in public LB, but out of the medal range in private LB.\n\nThere were some submissions that were within medal range, but I couldn't pick them out!\n\nPlease tell me what you do when learning and reasoning.",
    "1209597": "Trusting local CV while comparing two models is a better indicator vs what they score on a subset of the LB which is public. The private LB (69% of LB) distribution  and image types could have (and in this case were) different from the public ones (31% of LB). The public part of the LB is supposed to be a good metric to evaluate a model but you have to be careful to not use that as the gold standard for generalized models.  Otherwise you are essentially picking models that are overfitting the public LB data.  \n\nAlso going with your gut on what is better practice and a more robust model. For example: I did not pick my best performing public LB because I had not done enough augmentations and TTA vs another one that had a -0.002 score on public LB.   I selected the one that had TTA and a lesser public score and it landed up being better on private which is something I was hoping for.",
    "1209668": "Thanks for the comment.\nThis competition was at the mercy of LB!"
  },
  "source": "meta"
}