{
  "id": 101936,
  "title": "How to build a good local validation set",
  "url": "/competitions/aptos2019-blindness-detection/discussion/101936",
  "author_name": "",
  "post_date": "2019-07-29T22:04:40.245580400Z",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I am looking at building a good local validation set, that can be used reliably to measure the improvements. Right now, I am using a randomly split validation set. But the Kappa score of this set is not correlating well with that of the public LB.</p>\n\n<p>Can you pls share thoughts/tips on building a good local validation set.</p>",
  "messages": [
    {
      "id": "586919",
      "postDate": "07/29/2019 22:04:40",
      "content": "<p>I am looking at building a good local validation set, that can be used reliably to measure the improvements. Right now, I am using a randomly split validation set. But the Kappa score of this set is not correlating well with that of the public LB.</p>\n\n<p>Can you pls share thoughts/tips on building a good local validation set.</p>",
      "rawMarkdown": "I am looking at building a good local validation set, that can be used reliably to measure the improvements. Right now, I am using a randomly split validation set. But the Kappa score of this set is not correlating well with that of the public LB.\n\nCan you pls share thoughts/tips on building a good local validation set.",
      "votes": null
    },
    {
      "id": "588140",
      "postDate": "07/30/2019 07:55:21",
      "content": "<p>I meet the same problem,the gap of local and kernel  too big to eval my algorithm</p>",
      "rawMarkdown": "I meet the same problem,the gap of local and kernel  too big to eval my algorithm",
      "votes": null
    },
    {
      "id": "588236",
      "postDate": "07/30/2019 10:10:29",
      "content": "<p>I used old data to balance out validation and train set. For me balanced validation set is in good correlation with LB.</p>",
      "rawMarkdown": "I used old data to balance out validation and train set. For me balanced validation set is in good correlation with LB.",
      "votes": null
    },
    {
      "id": "588261",
      "postDate": "07/30/2019 10:47:48",
      "content": "<p>thanks <a href=\"/ashwan1\">@ashwan1</a> can you elaborate on how you used old data to do the balancing.</p>\n\n<p>Did you make sure that you have the same number of images in each class by adding images from the old data set to the new data set ?</p>",
      "rawMarkdown": "thanks @ashwan1 can you elaborate on how you used old data to do the balancing.\n\nDid you make sure that you have the same number of images in each class by adding images from the old data set to the new data set ?",
      "votes": null
    },
    {
      "id": "588291",
      "postDate": "07/30/2019 11:42:25",
      "content": "<blockquote>\n  <p>Did you make sure that you have the same number of images in each class by adding images from the old data set to the new data set ?</p>\n</blockquote>\n\n<p>yes, and then augmented images to increase number of images/category.</p>",
      "rawMarkdown": "&gt; Did you make sure that you have the same number of images in each class by adding images from the old data set to the new data set ?\n\nyes, and then augmented images to increase number of images/category.",
      "votes": null
    },
    {
      "id": "588297",
      "postDate": "07/30/2019 11:49:58",
      "content": "<p>thank you <a href=\"/ashwan1\">@ashwan1</a> </p>",
      "rawMarkdown": "thank you @ashwan1",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 588140,
      "author_name": "ggbrother",
      "author_url": "",
      "post_date": "07/30/2019 07:55:21",
      "content": "<p>I meet the same problem,the gap of local and kernel  too big to eval my algorithm</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 588236,
      "author_name": "ashwan1",
      "author_url": "",
      "post_date": "07/30/2019 10:10:29",
      "content": "<p>I used old data to balance out validation and train set. For me balanced validation set is in good correlation with LB.</p>",
      "votes": null,
      "replies": [
        {
          "id": 588261,
          "author_name": "ravivadapalli",
          "author_url": "",
          "post_date": "07/30/2019 10:47:48",
          "content": "<p>thanks <a href=\"/ashwan1\">@ashwan1</a> can you elaborate on how you used old data to do the balancing.</p>\n\n<p>Did you make sure that you have the same number of images in each class by adding images from the old data set to the new data set ?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 588291,
          "author_name": "ashwan1",
          "author_url": "",
          "post_date": "07/30/2019 11:42:25",
          "content": "<blockquote>\n  <p>Did you make sure that you have the same number of images in each class by adding images from the old data set to the new data set ?</p>\n</blockquote>\n\n<p>yes, and then augmented images to increase number of images/category.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 588297,
          "author_name": "ravivadapalli",
          "author_url": "",
          "post_date": "07/30/2019 11:49:58",
          "content": "<p>thank you <a href=\"/ashwan1\">@ashwan1</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "586919": "I am looking at building a good local validation set, that can be used reliably to measure the improvements. Right now, I am using a randomly split validation set. But the Kappa score of this set is not correlating well with that of the public LB.\n\nCan you pls share thoughts/tips on building a good local validation set.",
    "588140": "I meet the same problem,the gap of local and kernel  too big to eval my algorithm",
    "588236": "I used old data to balance out validation and train set. For me balanced validation set is in good correlation with LB.",
    "588261": "thanks @ashwan1 can you elaborate on how you used old data to do the balancing.\n\nDid you make sure that you have the same number of images in each class by adding images from the old data set to the new data set ?",
    "588291": "&gt; Did you make sure that you have the same number of images in each class by adding images from the old data set to the new data set ?\n\nyes, and then augmented images to increase number of images/category.",
    "588297": "thank you @ashwan1"
  },
  "source": "meta"
}