{
  "id": 3757,
  "title": "Model Submission Clarification",
  "url": "/competitions/flight/discussion/3757",
  "author_name": "",
  "post_date": "2013-02-04T03:26:36.637Z",
  "votes": null,
  "comment_count": 10,
  "views": 6137,
  "content": "<p>The timeline indicates the Public Leader board is frozen on Feb 8th. Is this Midnight EST?</p>\r\n<p>The submission instructions say:</p>\r\n<p style=\"padding-left:30px\"><em>During the time the public leaderboard is open, you may submit predictions on the public leaderboard set. In order to be eligible for prizes, you must make at least one submission for the public leaderboard set using the same\r\n model you will use for the final evaluation set.</em></p>\r\n<p style=\"padding-left:30px\">&nbsp;</p>\r\n<p>But the Timeline states that final models need to be submitted by Feb 14th. (Again what time is the cutoff?)</p>\r\n<p>My reading of the submission instructions says the final model must be submitted by Feb 8th but the timeline says we have to Feb 14th.</p>\r\n<p>&nbsp;</p>\r\n<p>Which is correct?</p>",
  "messages": [
    {
      "id": "19991",
      "postDate": "02/04/2013 03:26:36",
      "content": "<p>The timeline indicates the Public Leader board is frozen on Feb 8th. Is this Midnight EST?</p>\r\n<p>The submission instructions say:</p>\r\n<p style=\"padding-left:30px\"><em>During the time the public leaderboard is open, you may submit predictions on the public leaderboard set. In order to be eligible for prizes, you must make at least one submission for the public leaderboard set using the same\r\n model you will use for the final evaluation set.</em></p>\r\n<p style=\"padding-left:30px\">&nbsp;</p>\r\n<p>But the Timeline states that final models need to be submitted by Feb 14th. (Again what time is the cutoff?)</p>\r\n<p>My reading of the submission instructions says the final model must be submitted by Feb 8th but the timeline says we have to Feb 14th.</p>\r\n<p>&nbsp;</p>\r\n<p>Which is correct?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "19994",
      "postDate": "02/04/2013 03:59:23",
      "content": "<p>[quote=Russell Martin;19991]</p>\r\n<p>The timeline indicates the Public Leader board is frozen on Feb 8th. Is this Midnight EST?</p>\r\n<p>The submission instructions say:</p>\r\n<p style=\"padding-left:30px\"><em>During the time the public leaderboard is open, you may submit predictions on the public leaderboard set. In order to be eligible for prizes, you must make at least one submission for the public leaderboard set using the same\r\n model you will use for the final evaluation set.</em></p>\r\n<p style=\"padding-left:30px\">&nbsp;</p>\r\n<p>But the Timeline states that final models need to be submitted by Feb 14th. (Again what time is the cutoff?)</p>\r\n<p>My reading of the submission instructions says the final model must be submitted by Feb 8th but the timeline says we have to Feb 14th.</p>\r\n<p>&nbsp;</p>\r\n<p>Which is correct?</p>\r\n<p>[/quote]All deadlines are at 11:59 PM UTC of the corresponding date. Final models don't need to be submitted until February 14th, but you do need to make at least one submission to the public leaderboard by February 8th. I've removed the part of the submission\r\n instructions that stated &quot;<em>using the same model you will use for the final evaluation set</em>&quot;, which shouldn't have been there.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20013",
      "postDate": "02/04/2013 14:45:29",
      "content": "<p>How will we submit a final model between Feb 8 and 14 if the Public Leaderboard is closed?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20016",
      "postDate": "02/04/2013 17:25:36",
      "content": "<p>[quote=Vadnais;20013]</p>\r\n<p>How will we submit a final model between Feb 8 and 14 if the Public Leaderboard is closed?</p>\r\n<p>[/quote]</p>\r\nYou'll be able to attach your model or its hash to your best public leaderboard submission.",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20039",
      "postDate": "02/04/2013 23:21:06",
      "content": "<p>When attaching the hash, I'm presuming the hash needs to be created with the model and the supporting files zipped together. Is that correct?</p>\r\n<p>I would define model as the hypothesis, but there could be some adjustments in wrapper scripts that are outside the model. Thus, do we hash the model, plus any associated file together?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20040",
      "postDate": "02/04/2013 23:24:07",
      "content": "<p>[quote=levitation'r;20039]</p>\r\n<p>When attaching the hash, I'm presuming the hash needs to be created with the model and the supporting files zipped together. Is that correct?</p>\r\n<p>I would define model as the hypothesis, but there could be some adjustments in wrapper scripts that are outside the model. Thus, do we hash the model, plus any associated file together?</p>\r\n<p>[/quote]This is correct. Include any files and scripts needed to reproduce your results (outside the source data) in the hashed archive.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20148",
      "postDate": "02/07/2013 19:46:07",
      "content": "<p>The timeline says that between Feb 9 and 14, &quot;Entrants further develop models using all available data.&quot;&nbsp; But we need to attach the model to an entry made *before* Feb 9.&nbsp; So the model submitted does not need to be the one used to create the Public Leaderboard\r\n submission that it is attached to, correct?&nbsp; By definition if the submission is made before Feb 9, and the model is &quot;further developed&quot; after Feb 9, they won't match.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20149",
      "postDate": "02/07/2013 19:48:08",
      "content": "<p>[quote=BreakfastPirate;20148]</p>\r\n<p>So the model submitted does not need to be the one used to create the Public Leaderboard submission that it is attached to, correct?</p>\r\n<p>[/quote]That is correct.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20150",
      "postDate": "02/07/2013 19:50:07",
      "content": "<p>Ben, will we be able to change the models after 14th February?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20151",
      "postDate": "02/07/2013 19:51:01",
      "content": "<p>[quote=Paweł;20150]</p>\r\n<p>Ben, will we be able to change the models after 14th February?</p>\r\n<p>[/quote]No, by submitting them or a hash you lock them in.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "20156",
      "postDate": "02/07/2013 21:35:08",
      "content": "<p>@Ben can you give us some example what can be considered as a 'model' and what can't?</p>\r\n<p>Any automatized procedure/program/function &nbsp;(to score the flights on a test day &nbsp;with a given cutoff) which uses only the legal inputs (<strong>all training data</strong> [2012-11-11;2013-02-06],&nbsp;<strong>1 day final test data\r\n</strong>(before cutoff) ) is OK?</p>\r\n<p>I can imagine quite strange 'models' which are very far from the original concept: &nbsp;</p>\r\n<p><span>&quot;Advancements in <strong>real-time</strong> big data analysis are changing the course of flight as we know it&quot;</span></p>\r\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 19994,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "02/04/2013 03:59:23",
      "content": "<p>[quote=Russell Martin;19991]</p>\r\n<p>The timeline indicates the Public Leader board is frozen on Feb 8th. Is this Midnight EST?</p>\r\n<p>The submission instructions say:</p>\r\n<p style=\"padding-left:30px\"><em>During the time the public leaderboard is open, you may submit predictions on the public leaderboard set. In order to be eligible for prizes, you must make at least one submission for the public leaderboard set using the same\r\n model you will use for the final evaluation set.</em></p>\r\n<p style=\"padding-left:30px\">&nbsp;</p>\r\n<p>But the Timeline states that final models need to be submitted by Feb 14th. (Again what time is the cutoff?)</p>\r\n<p>My reading of the submission instructions says the final model must be submitted by Feb 8th but the timeline says we have to Feb 14th.</p>\r\n<p>&nbsp;</p>\r\n<p>Which is correct?</p>\r\n<p>[/quote]All deadlines are at 11:59 PM UTC of the corresponding date. Final models don't need to be submitted until February 14th, but you do need to make at least one submission to the public leaderboard by February 8th. I've removed the part of the submission\r\n instructions that stated &quot;<em>using the same model you will use for the final evaluation set</em>&quot;, which shouldn't have been there.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 20013,
      "author_name": "vadnais",
      "author_url": "",
      "post_date": "02/04/2013 14:45:29",
      "content": "<p>How will we submit a final model between Feb 8 and 14 if the Public Leaderboard is closed?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 20016,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "02/04/2013 17:25:36",
      "content": "<p>[quote=Vadnais;20013]</p>\r\n<p>How will we submit a final model between Feb 8 and 14 if the Public Leaderboard is closed?</p>\r\n<p>[/quote]</p>\r\nYou'll be able to attach your model or its hash to your best public leaderboard submission.",
      "votes": null,
      "replies": []
    },
    {
      "id": 20039,
      "author_name": "manishm",
      "author_url": "",
      "post_date": "02/04/2013 23:21:06",
      "content": "<p>When attaching the hash, I'm presuming the hash needs to be created with the model and the supporting files zipped together. Is that correct?</p>\r\n<p>I would define model as the hypothesis, but there could be some adjustments in wrapper scripts that are outside the model. Thus, do we hash the model, plus any associated file together?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 20040,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "02/04/2013 23:24:07",
      "content": "<p>[quote=levitation'r;20039]</p>\r\n<p>When attaching the hash, I'm presuming the hash needs to be created with the model and the supporting files zipped together. Is that correct?</p>\r\n<p>I would define model as the hypothesis, but there could be some adjustments in wrapper scripts that are outside the model. Thus, do we hash the model, plus any associated file together?</p>\r\n<p>[/quote]This is correct. Include any files and scripts needed to reproduce your results (outside the source data) in the hashed archive.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 20148,
      "author_name": "breakfastpirate",
      "author_url": "",
      "post_date": "02/07/2013 19:46:07",
      "content": "<p>The timeline says that between Feb 9 and 14, &quot;Entrants further develop models using all available data.&quot;&nbsp; But we need to attach the model to an entry made *before* Feb 9.&nbsp; So the model submitted does not need to be the one used to create the Public Leaderboard\r\n submission that it is attached to, correct?&nbsp; By definition if the submission is made before Feb 9, and the model is &quot;further developed&quot; after Feb 9, they won't match.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 20149,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "02/07/2013 19:48:08",
      "content": "<p>[quote=BreakfastPirate;20148]</p>\r\n<p>So the model submitted does not need to be the one used to create the Public Leaderboard submission that it is attached to, correct?</p>\r\n<p>[/quote]That is correct.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 20150,
      "author_name": "paweljankiewicz",
      "author_url": "",
      "post_date": "02/07/2013 19:50:07",
      "content": "<p>Ben, will we be able to change the models after 14th February?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 20151,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "02/07/2013 19:51:01",
      "content": "<p>[quote=Paweł;20150]</p>\r\n<p>Ben, will we be able to change the models after 14th February?</p>\r\n<p>[/quote]No, by submitting them or a hash you lock them in.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 20156,
      "author_name": "gaborfodor",
      "author_url": "",
      "post_date": "02/07/2013 21:35:08",
      "content": "<p>@Ben can you give us some example what can be considered as a 'model' and what can't?</p>\r\n<p>Any automatized procedure/program/function &nbsp;(to score the flights on a test day &nbsp;with a given cutoff) which uses only the legal inputs (<strong>all training data</strong> [2012-11-11;2013-02-06],&nbsp;<strong>1 day final test data\r\n</strong>(before cutoff) ) is OK?</p>\r\n<p>I can imagine quite strange 'models' which are very far from the original concept: &nbsp;</p>\r\n<p><span>&quot;Advancements in <strong>real-time</strong> big data analysis are changing the course of flight as we know it&quot;</span></p>\r\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "19991": "",
    "19994": "",
    "20013": "",
    "20016": "",
    "20039": "",
    "20040": "",
    "20148": "",
    "20149": "",
    "20150": "",
    "20151": "",
    "20156": ""
  },
  "source": "meta"
}