{
  "id": 3459,
  "title": "Submission ordering?",
  "url": "/competitions/flight/discussion/3459",
  "author_name": "",
  "post_date": "2012-12-24T05:15:05.807Z",
  "votes": null,
  "comment_count": 2,
  "views": 2449,
  "content": "<p>Are the submission's entries supposed to be in a certain order (i.e. the one in test_flights_combined)?</p>\r\n<p>My leaderboard results aren't agreeing with what I'm getting on my own personal data sets.</p>\r\n<p>I created my own cutoff submission just like the example (except with the entries in a different order) and got a result of 94.96991 rather than the quoted 94.27294.</p>\r\n<p>&nbsp;</p>",
  "messages": [
    {
      "id": "18517",
      "postDate": "12/24/2012 05:15:05",
      "content": "<p>Are the submission's entries supposed to be in a certain order (i.e. the one in test_flights_combined)?</p>\r\n<p>My leaderboard results aren't agreeing with what I'm getting on my own personal data sets.</p>\r\n<p>I created my own cutoff submission just like the example (except with the entries in a different order) and got a result of 94.96991 rather than the quoted 94.27294.</p>\r\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "18520",
      "postDate": "12/24/2012 05:47:00",
      "content": "<p>Oops, sorry, just found 2 wrong entries in my cutoff solution. &nbsp;I can't rerun the fixed version, though, since I've hit my submission limit.</p>\r\n<p>Is anyone else submitting solutions that aren't in the same order as the test_flights_combined?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "18614",
      "postDate": "12/26/2012 06:52:36",
      "content": "<p>Responding in case anyone else has the same question:</p>\r\n<p>Submissions should be in the same order as test<em>flights</em>combined (and the benchmark submission files).</p>\r\n<p>If you submit a file with flights out of order and our backend correctly identifies the ID column, then it should be scored correctly. However, this has hit a lot of edge cases from various malformed submissions or mis-identified columns in the past, so\r\n you should follow the format in the sample submission files to be safe.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 18520,
      "author_name": "lonelyatthetop",
      "author_url": "",
      "post_date": "12/24/2012 05:47:00",
      "content": "<p>Oops, sorry, just found 2 wrong entries in my cutoff solution. &nbsp;I can't rerun the fixed version, though, since I've hit my submission limit.</p>\r\n<p>Is anyone else submitting solutions that aren't in the same order as the test_flights_combined?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 18614,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "12/26/2012 06:52:36",
      "content": "<p>Responding in case anyone else has the same question:</p>\r\n<p>Submissions should be in the same order as test<em>flights</em>combined (and the benchmark submission files).</p>\r\n<p>If you submit a file with flights out of order and our backend correctly identifies the ID column, then it should be scored correctly. However, this has hit a lot of edge cases from various malformed submissions or mis-identified columns in the past, so\r\n you should follow the format in the sample submission files to be safe.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "18517": "",
    "18520": "",
    "18614": ""
  },
  "source": "meta"
}