{
  "id": 118568,
  "title": "Why only 9% data for public lb?",
  "url": "/competitions/pku-autonomous-driving/discussion/118568",
  "author_name": "",
  "post_date": "2019-11-22T13:06:44.562376500Z",
  "votes": 8,
  "comment_count": 6,
  "views": 0,
  "content": "<p>with a lot of poor quality images in the test set(<a href=\"https://www.kaggle.com/mobassir/baidu-autonomous-driving-eda\">compared to train set</a>) what was the motivation behind using only 9% data for public lb? this competition brings few new concepts for a lot of kagglers to learn. 9% means 181 or 182 images  for public lb score which is really very small,this is not a 2 stage competition so i would like to know why such small amount of images for public lb? i think 20-25 percent at least would be better for understanding our model a bit about \"how they might will perform in private lb\".will the organizer increase the public lb test set a bit?\nwith 9% data only,no public lb score can tell if the model will generalize or not!\nand i can see huge shake-up coming with only 9% for public lb.\ncorrect me if i am wrong,thanks</p>",
  "messages": [
    {
      "id": "679217",
      "postDate": "11/22/2019 13:06:44",
      "content": "<p>with a lot of poor quality images in the test set(<a href=\"https://www.kaggle.com/mobassir/baidu-autonomous-driving-eda\">compared to train set</a>) what was the motivation behind using only 9% data for public lb? this competition brings few new concepts for a lot of kagglers to learn. 9% means 181 or 182 images  for public lb score which is really very small,this is not a 2 stage competition so i would like to know why such small amount of images for public lb? i think 20-25 percent at least would be better for understanding our model a bit about \"how they might will perform in private lb\".will the organizer increase the public lb test set a bit?\nwith 9% data only,no public lb score can tell if the model will generalize or not!\nand i can see huge shake-up coming with only 9% for public lb.\ncorrect me if i am wrong,thanks</p>",
      "rawMarkdown": "with a lot of poor quality images in the test set([compared to train set](https://www.kaggle.com/mobassir/baidu-autonomous-driving-eda)) what was the motivation behind using only 9% data for public lb? this competition brings few new concepts for a lot of kagglers to learn. 9% means 181 or 182 images  for public lb score which is really very small,this is not a 2 stage competition so i would like to know why such small amount of images for public lb? i think 20-25 percent at least would be better for understanding our model a bit about \"how they might will perform in private lb\".will the organizer increase the public lb test set a bit?\nwith 9% data only,no public lb score can tell if the model will generalize or not!\nand i can see huge shake-up coming with only 9% for public lb.\ncorrect me if i am wrong,thanks",
      "votes": null
    },
    {
      "id": "679223",
      "postDate": "11/22/2019 13:15:27",
      "content": "<p>looks like many people dont have the clarity about the metrics and some clarification pending with data as well :) .. So best of luck :D ..</p>",
      "rawMarkdown": "looks like many people dont have the clarity about the metrics and some clarification pending with data as well :) .. So best of luck :D ..",
      "votes": null
    },
    {
      "id": "679227",
      "postDate": "11/22/2019 13:21:06",
      "content": "<p>yeah, all those confusions need to be clarified ASAP</p>",
      "rawMarkdown": "yeah, all those confusions need to be clarified ASAP",
      "votes": null
    },
    {
      "id": "679354",
      "postDate": "11/22/2019 17:13:36",
      "content": "<p>In my view, a proper local validation is much more important than public LB. \nDon't let that number fool you. \nNevertheless, in this competition, I predict the public LB to be close to the private LB.</p>",
      "rawMarkdown": "In my view, a proper local validation is much more important than public LB. \nDon't let that number fool you. \nNevertheless, in this competition, I predict the public LB to be close to the private LB.",
      "votes": null
    },
    {
      "id": "679356",
      "postDate": "11/22/2019 17:21:14",
      "content": "<p>Very good!\nI will surely remind you your prediction when the competition is over,thanks</p>",
      "rawMarkdown": "Very good!\nI will surely remind you your prediction when the competition is over,thanks",
      "votes": null
    },
    {
      "id": "679387",
      "postDate": "11/22/2019 18:03:12",
      "content": "<p>hehe, deal ;)\nAs long as they didn't split somehow fancy, that is. \nLike only not flipped/altered images in public LB. \nThat's why local validation is so important!</p>",
      "rawMarkdown": "hehe, deal ;)\nAs long as they didn't split somehow fancy, that is. \nLike only not flipped/altered images in public LB. \nThat's why local validation is so important!",
      "votes": null
    },
    {
      "id": "679394",
      "postDate": "11/22/2019 18:17:57",
      "content": "<p>Yeah local validation is always the best way but still i thought little bit more images for testing would be nice because if not then whats the purpose of public lb? They could use private lb only? And for entry less than 1% would be enough? Right?\nI mean : if it is not less than 1% then it should be more than 20% always.\nAnyway forget about it and what do you think about lb probing? I think lb probing is important for this competition along with local cv,  any thought?</p>",
      "rawMarkdown": "Yeah local validation is always the best way but still i thought little bit more images for testing would be nice because if not then whats the purpose of public lb? They could use private lb only? And for entry less than 1% would be enough? Right?\nI mean : if it is not less than 1% then it should be more than 20% always.\nAnyway forget about it and what do you think about lb probing? I think lb probing is important for this competition along with local cv,  any thought?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 679223,
      "author_name": "phoenix9032",
      "author_url": "",
      "post_date": "11/22/2019 13:15:27",
      "content": "<p>looks like many people dont have the clarity about the metrics and some clarification pending with data as well :) .. So best of luck :D ..</p>",
      "votes": null,
      "replies": [
        {
          "id": 679227,
          "author_name": "mobassir",
          "author_url": "",
          "post_date": "11/22/2019 13:21:06",
          "content": "<p>yeah, all those confusions need to be clarified ASAP</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 679354,
      "author_name": "ilu000",
      "author_url": "",
      "post_date": "11/22/2019 17:13:36",
      "content": "<p>In my view, a proper local validation is much more important than public LB. \nDon't let that number fool you. \nNevertheless, in this competition, I predict the public LB to be close to the private LB.</p>",
      "votes": null,
      "replies": [
        {
          "id": 679356,
          "author_name": "mobassir",
          "author_url": "",
          "post_date": "11/22/2019 17:21:14",
          "content": "<p>Very good!\nI will surely remind you your prediction when the competition is over,thanks</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 679387,
          "author_name": "ilu000",
          "author_url": "",
          "post_date": "11/22/2019 18:03:12",
          "content": "<p>hehe, deal ;)\nAs long as they didn't split somehow fancy, that is. \nLike only not flipped/altered images in public LB. \nThat's why local validation is so important!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 679394,
          "author_name": "mobassir",
          "author_url": "",
          "post_date": "11/22/2019 18:17:57",
          "content": "<p>Yeah local validation is always the best way but still i thought little bit more images for testing would be nice because if not then whats the purpose of public lb? They could use private lb only? And for entry less than 1% would be enough? Right?\nI mean : if it is not less than 1% then it should be more than 20% always.\nAnyway forget about it and what do you think about lb probing? I think lb probing is important for this competition along with local cv,  any thought?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "679217": "with a lot of poor quality images in the test set([compared to train set](https://www.kaggle.com/mobassir/baidu-autonomous-driving-eda)) what was the motivation behind using only 9% data for public lb? this competition brings few new concepts for a lot of kagglers to learn. 9% means 181 or 182 images  for public lb score which is really very small,this is not a 2 stage competition so i would like to know why such small amount of images for public lb? i think 20-25 percent at least would be better for understanding our model a bit about \"how they might will perform in private lb\".will the organizer increase the public lb test set a bit?\nwith 9% data only,no public lb score can tell if the model will generalize or not!\nand i can see huge shake-up coming with only 9% for public lb.\ncorrect me if i am wrong,thanks",
    "679223": "looks like many people dont have the clarity about the metrics and some clarification pending with data as well :) .. So best of luck :D ..",
    "679227": "yeah, all those confusions need to be clarified ASAP",
    "679354": "In my view, a proper local validation is much more important than public LB. \nDon't let that number fool you. \nNevertheless, in this competition, I predict the public LB to be close to the private LB.",
    "679356": "Very good!\nI will surely remind you your prediction when the competition is over,thanks",
    "679387": "hehe, deal ;)\nAs long as they didn't split somehow fancy, that is. \nLike only not flipped/altered images in public LB. \nThat's why local validation is so important!",
    "679394": "Yeah local validation is always the best way but still i thought little bit more images for testing would be nice because if not then whats the purpose of public lb? They could use private lb only? And for entry less than 1% would be enough? Right?\nI mean : if it is not less than 1% then it should be more than 20% always.\nAnyway forget about it and what do you think about lb probing? I think lb probing is important for this competition along with local cv,  any thought?"
  },
  "source": "meta"
}