{
  "id": 158900,
  "title": "Are three rows in the open test data enough for good models?",
  "url": "/competitions/birdsong-recognition/discussion/158900",
  "author_name": "",
  "post_date": "2020-06-15T18:24:05.231339Z",
  "votes": 13,
  "comment_count": 8,
  "views": 0,
  "content": "<p>As per description:\n<code>\ntest.csv Only the first three rows are available for download; the full test.csv is in the hidden test set.\n</code>\nPublic leaderboard is 27%</p>\n\n<p>What do you think, is this enough for creating robust models?</p>",
  "messages": [
    {
      "id": "887555",
      "postDate": "06/15/2020 18:24:05",
      "content": "<p>As per description:\n<code>\ntest.csv Only the first three rows are available for download; the full test.csv is in the hidden test set.\n</code>\nPublic leaderboard is 27%</p>\n\n<p>What do you think, is this enough for creating robust models?</p>",
      "rawMarkdown": "As per description:\n```\ntest.csv Only the first three rows are available for download; the full test.csv is in the hidden test set.\n```\nPublic leaderboard is 27%\n\nWhat do you think, is this enough for creating robust models?",
      "votes": null
    },
    {
      "id": "887931",
      "postDate": "06/16/2020 02:30:31",
      "content": "<p>I think the format is similar to that of Bengali's competition (<a href=\"https://www.kaggle.com/c/bengaliai-cv19/\">https://www.kaggle.com/c/bengaliai-cv19/</a>), where we only had 12 images for <em>download</em>, but the public LB was calculated with larger hidden dataset when re-running, and so as private LB.</p>",
      "rawMarkdown": "I think the format is similar to that of Bengali's competition (https://www.kaggle.com/c/bengaliai-cv19/), where we only had 12 images for *download*, but the public LB was calculated with larger hidden dataset when re-running, and so as private LB.",
      "votes": null
    },
    {
      "id": "888915",
      "postDate": "06/16/2020 16:40:00",
      "content": "<p>Yes, that's what I thought too. The <code>test.csv</code> with 3 rows is only given so that we can design our inference pipelines accordingly.</p>",
      "rawMarkdown": "Yes, that's what I thought too. The `test.csv` with 3 rows is only given so that we can design our inference pipelines accordingly.",
      "votes": null
    },
    {
      "id": "889209",
      "postDate": "06/16/2020 20:38:27",
      "content": "<p>And only two decimal places on LB !   Very likely that there will be shake up. </p>",
      "rawMarkdown": "And only two decimal places on LB !   Very likely that there will be shake up.",
      "votes": null
    },
    {
      "id": "889242",
      "postDate": "06/16/2020 21:09:51",
      "content": "<p>Sorry if this is a silly question (I've not taken part in a completed competition yet), but why does only two decimal places mean there will be a shake up? Is it because a lot of people are going to have the same/similar public LB scores?</p>",
      "rawMarkdown": "Sorry if this is a silly question (I've not taken part in a completed competition yet), but why does only two decimal places mean there will be a shake up? Is it because a lot of people are going to have the same/similar public LB scores?",
      "votes": null
    },
    {
      "id": "889268",
      "postDate": "06/16/2020 21:37:55",
      "content": "<p>No, it’s not equivalent (ie. Shakeup and decimal places). I could be wrong.  It’s hard to gauge where you stand on the LB with 2 decimal places and 3 rows (27%). </p>\n\n<p>PS: only 2 submissions per day with a team size of 5, which also seems odd.  </p>",
      "rawMarkdown": "No, it’s not equivalent (ie. Shakeup and decimal places). I could be wrong.  It’s hard to gauge where you stand on the LB with 2 decimal places and 3 rows (27%). \n\nPS: only 2 submissions per day with a team size of 5, which also seems odd.",
      "votes": null
    },
    {
      "id": "889274",
      "postDate": "06/16/2020 21:51:19",
      "content": "<p>I believe there are 150 test files and then these need to be split up into different time windows (5 second intervals), so the whole test set is several thousand samples in size. I think the 3 rows are just an example so that we test our models work.</p>",
      "rawMarkdown": "I believe there are 150 test files and then these need to be split up into different time windows (5 second intervals), so the whole test set is several thousand samples in size. I think the 3 rows are just an example so that we test our models work.",
      "votes": null
    },
    {
      "id": "889979",
      "postDate": "06/17/2020 08:32:58",
      "content": "<p>Yes you are right. The rows are just an example, infact you can't even use them since the test_audio file is hidden. <code>test.csv</code> is given so that we can design our inference pipeline accordingly.</p>\n\n<p>Both the public and private test sets are hidden.</p>",
      "rawMarkdown": "Yes you are right. The rows are just an example, infact you can't even use them since the test_audio file is hidden. `test.csv` is given so that we can design our inference pipeline accordingly.\n\nBoth the public and private test sets are hidden.",
      "votes": null
    },
    {
      "id": "890368",
      "postDate": "06/17/2020 13:21:40",
      "content": "<p>Yes, Bengali AI is a good analogy here for the test set setup.</p>",
      "rawMarkdown": "Yes, Bengali AI is a good analogy here for the test set setup.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 887931,
      "author_name": "hidehisaarai1213",
      "author_url": "",
      "post_date": "06/16/2020 02:30:31",
      "content": "<p>I think the format is similar to that of Bengali's competition (<a href=\"https://www.kaggle.com/c/bengaliai-cv19/\">https://www.kaggle.com/c/bengaliai-cv19/</a>), where we only had 12 images for <em>download</em>, but the public LB was calculated with larger hidden dataset when re-running, and so as private LB.</p>",
      "votes": null,
      "replies": [
        {
          "id": 888915,
          "author_name": "dhruvrnaik",
          "author_url": "",
          "post_date": "06/16/2020 16:40:00",
          "content": "<p>Yes, that's what I thought too. The <code>test.csv</code> with 3 rows is only given so that we can design our inference pipelines accordingly.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 890368,
          "author_name": "sohier",
          "author_url": "",
          "post_date": "06/17/2020 13:21:40",
          "content": "<p>Yes, Bengali AI is a good analogy here for the test set setup.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 889209,
      "author_name": "rashmibanthia",
      "author_url": "",
      "post_date": "06/16/2020 20:38:27",
      "content": "<p>And only two decimal places on LB !   Very likely that there will be shake up. </p>",
      "votes": null,
      "replies": [
        {
          "id": 889242,
          "author_name": "cwthompson",
          "author_url": "",
          "post_date": "06/16/2020 21:09:51",
          "content": "<p>Sorry if this is a silly question (I've not taken part in a completed competition yet), but why does only two decimal places mean there will be a shake up? Is it because a lot of people are going to have the same/similar public LB scores?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 889268,
          "author_name": "rashmibanthia",
          "author_url": "",
          "post_date": "06/16/2020 21:37:55",
          "content": "<p>No, it’s not equivalent (ie. Shakeup and decimal places). I could be wrong.  It’s hard to gauge where you stand on the LB with 2 decimal places and 3 rows (27%). </p>\n\n<p>PS: only 2 submissions per day with a team size of 5, which also seems odd.  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 889274,
          "author_name": "cwthompson",
          "author_url": "",
          "post_date": "06/16/2020 21:51:19",
          "content": "<p>I believe there are 150 test files and then these need to be split up into different time windows (5 second intervals), so the whole test set is several thousand samples in size. I think the 3 rows are just an example so that we test our models work.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 889979,
          "author_name": "dhruvrnaik",
          "author_url": "",
          "post_date": "06/17/2020 08:32:58",
          "content": "<p>Yes you are right. The rows are just an example, infact you can't even use them since the test_audio file is hidden. <code>test.csv</code> is given so that we can design our inference pipeline accordingly.</p>\n\n<p>Both the public and private test sets are hidden.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "887555": "As per description:\n```\ntest.csv Only the first three rows are available for download; the full test.csv is in the hidden test set.\n```\nPublic leaderboard is 27%\n\nWhat do you think, is this enough for creating robust models?",
    "887931": "I think the format is similar to that of Bengali's competition (https://www.kaggle.com/c/bengaliai-cv19/), where we only had 12 images for *download*, but the public LB was calculated with larger hidden dataset when re-running, and so as private LB.",
    "888915": "Yes, that's what I thought too. The `test.csv` with 3 rows is only given so that we can design our inference pipelines accordingly.",
    "889209": "And only two decimal places on LB !   Very likely that there will be shake up.",
    "889242": "Sorry if this is a silly question (I've not taken part in a completed competition yet), but why does only two decimal places mean there will be a shake up? Is it because a lot of people are going to have the same/similar public LB scores?",
    "889268": "No, it’s not equivalent (ie. Shakeup and decimal places). I could be wrong.  It’s hard to gauge where you stand on the LB with 2 decimal places and 3 rows (27%). \n\nPS: only 2 submissions per day with a team size of 5, which also seems odd.",
    "889274": "I believe there are 150 test files and then these need to be split up into different time windows (5 second intervals), so the whole test set is several thousand samples in size. I think the 3 rows are just an example so that we test our models work.",
    "889979": "Yes you are right. The rows are just an example, infact you can't even use them since the test_audio file is hidden. `test.csv` is given so that we can design our inference pipeline accordingly.\n\nBoth the public and private test sets are hidden.",
    "890368": "Yes, Bengali AI is a good analogy here for the test set setup."
  },
  "source": "meta"
}