{
  "id": 175951,
  "title": " Why are the scores different even in the same csv file?",
  "url": "/competitions/landmark-recognition-2020/discussion/175951",
  "author_name": "",
  "post_date": "2020-08-20T01:12:54.598121Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I submit sample_submission.csv for testing.<br>\nThis has <code>137790 0.1</code> written on every line.<br>\nThe score of this sample_submission.csv was 0.000.<br>\nHowever, there are some other published Notebooks that give different scores even though the line is <code>137790 0.1</code>.</p>\n<p>At first glance, it does not look different from sample_submission, but what is the evaluation standard?</p>",
  "messages": [
    {
      "id": "978129",
      "postDate": "08/20/2020 01:12:54",
      "content": "<p>I submit sample_submission.csv for testing.<br>\nThis has <code>137790 0.1</code> written on every line.<br>\nThe score of this sample_submission.csv was 0.000.<br>\nHowever, there are some other published Notebooks that give different scores even though the line is <code>137790 0.1</code>.</p>\n<p>At first glance, it does not look different from sample_submission, but what is the evaluation standard?</p>",
      "rawMarkdown": "I submit sample_submission.csv for testing.\nThis has `137790 0.1` written on every line.\nThe score of this sample_submission.csv was 0.000.\nHowever, there are some other published Notebooks that give different scores even though the line is `137790 0.1`.\n\nAt first glance, it does not look different from sample_submission, but what is the evaluation standard?",
      "votes": null
    },
    {
      "id": "978177",
      "postDate": "08/20/2020 02:25:26",
      "content": "<p>If I understand that correctly, when you click submit, your notebook gets rerun with proper dataset as input, which is different from the one publicly available.</p>\n<p>The baseline notebook detects when it is run against the proper dataset and then it replaces dummy submission with the proper one, which uses neural nets to generate predictions.</p>\n<p>That means when you click submit on submission.csv generated by baseline notebook provided by organizers (which has the format that you describe), it won't be the one used for scoring your submission, the actual generated one will.</p>",
      "rawMarkdown": "If I understand that correctly, when you click submit, your notebook gets rerun with proper dataset as input, which is different from the one publicly available.\n\nThe baseline notebook detects when it is run against the proper dataset and then it replaces dummy submission with the proper one, which uses neural nets to generate predictions.\n\nThat means when you click submit on submission.csv generated by baseline notebook provided by organizers (which has the format that you describe), it won't be the one used for scoring your submission, the actual generated one will.",
      "votes": null
    },
    {
      "id": "978205",
      "postDate": "08/20/2020 03:01:47",
      "content": "<p>Thank you. I appreciate it</p>",
      "rawMarkdown": "Thank you. I appreciate it",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 978177,
      "author_name": "qiubit",
      "author_url": "",
      "post_date": "08/20/2020 02:25:26",
      "content": "<p>If I understand that correctly, when you click submit, your notebook gets rerun with proper dataset as input, which is different from the one publicly available.</p>\n<p>The baseline notebook detects when it is run against the proper dataset and then it replaces dummy submission with the proper one, which uses neural nets to generate predictions.</p>\n<p>That means when you click submit on submission.csv generated by baseline notebook provided by organizers (which has the format that you describe), it won't be the one used for scoring your submission, the actual generated one will.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 978205,
      "author_name": "koheist",
      "author_url": "",
      "post_date": "08/20/2020 03:01:47",
      "content": "<p>Thank you. I appreciate it</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "978129": "I submit sample_submission.csv for testing.\nThis has `137790 0.1` written on every line.\nThe score of this sample_submission.csv was 0.000.\nHowever, there are some other published Notebooks that give different scores even though the line is `137790 0.1`.\n\nAt first glance, it does not look different from sample_submission, but what is the evaluation standard?",
    "978177": "If I understand that correctly, when you click submit, your notebook gets rerun with proper dataset as input, which is different from the one publicly available.\n\nThe baseline notebook detects when it is run against the proper dataset and then it replaces dummy submission with the proper one, which uses neural nets to generate predictions.\n\nThat means when you click submit on submission.csv generated by baseline notebook provided by organizers (which has the format that you describe), it won't be the one used for scoring your submission, the actual generated one will.",
    "978205": "Thank you. I appreciate it"
  },
  "source": "meta"
}