{
  "id": 2444,
  "title": "Timeline",
  "url": "/competitions/predict-closed-questions-on-stack-overflow/discussion/2444",
  "author_name": "",
  "post_date": "2012-08-26T10:52:32.087Z",
  "votes": null,
  "comment_count": 10,
  "views": 4612,
  "content": "<ul>\r\n<li><strong>Tuesday, August 21, 2012</strong>:&nbsp; Launch of Public Competition;&nbsp;Release of Training and Validation Data Sets\r\n</li><li><strong>Tuesday, October 9, 2012</strong>:&nbsp; Deadline to Submit Final Models. Public leaderboard frozen\r\n</li><li><strong>Wednesday, October 10 - Tuesday, October 23</strong>: Private leaderboard data collected\r\n</li><li><strong>Thursday, October 24</strong>: New training set and private leaderboard set released\r\n</li><li><strong>Thursday, November 1</strong>: Deadline to submit private leaderboard predictions\r\n</li></ul>\r\n<div>1) When you submit a final model (by the way, is that just a bunch of code?), is that merely to test that you haven't hand-made your public_leaderboard predictions?</div>\r\n<div>2) From October 9 to November 1, can a competitor refine a model while waiting for the new training set and private leaderboard, and thus submit a different model than at the October 9 deadline?</div>\r\n<div>3) Private leaderboard means private to everyone but the competitor admins, or private including other competitors but not the general public?</div>",
  "messages": [
    {
      "id": "13470",
      "postDate": "08/26/2012 10:52:32",
      "content": "<ul>\r\n<li><strong>Tuesday, August 21, 2012</strong>:&nbsp; Launch of Public Competition;&nbsp;Release of Training and Validation Data Sets\r\n</li><li><strong>Tuesday, October 9, 2012</strong>:&nbsp; Deadline to Submit Final Models. Public leaderboard frozen\r\n</li><li><strong>Wednesday, October 10 - Tuesday, October 23</strong>: Private leaderboard data collected\r\n</li><li><strong>Thursday, October 24</strong>: New training set and private leaderboard set released\r\n</li><li><strong>Thursday, November 1</strong>: Deadline to submit private leaderboard predictions\r\n</li></ul>\r\n<div>1) When you submit a final model (by the way, is that just a bunch of code?), is that merely to test that you haven't hand-made your public_leaderboard predictions?</div>\r\n<div>2) From October 9 to November 1, can a competitor refine a model while waiting for the new training set and private leaderboard, and thus submit a different model than at the October 9 deadline?</div>\r\n<div>3) Private leaderboard means private to everyone but the competitor admins, or private including other competitors but not the general public?</div>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "13495",
      "postDate": "08/27/2012 05:29:29",
      "content": "<p>[quote=Ashwin;13470]</p>\r\n<div>1) When you submit a final model (by the way, is that just a bunch of code?), is that merely to test that you haven't hand-made your public_leaderboard predictions?[/quote]</div>\r\n<div>Yes, your final model consists of everything necessary to reproduce your results (to train your model on the training data, and make predictions on the evaluation data).</div>\r\n<div>\r\n<p>[quote=Ashwin;13470]2) From October 9 to November 1, can a competitor refine a model while waiting for the new training set and private leaderboard, and thus submit a different model than at the October 9 deadline?[/quote]No. You are required to upload your\r\n code before this to prevent cheating by overfitting on the evaluation data.</p>\r\n</div>\r\n<div>[quote=Ashwin;13470]3) Private leaderboard means private to everyone but the competitor admins, or private including other competitors but not the general public?[/quote]This terminology is borrowed from competitions where there is no temporal split between\r\n the evaluation sets, and may be less appropriate here. The &quot;private leaderboard&quot; data is used for final evaluation, and the results on this will be publicly visible at the end of the competition.</div>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "13600",
      "postDate": "08/29/2012 10:23:13",
      "content": "<p>A follow up question:</p>\r\n<p>During 24 Oct to 1 Nov, will we be able to see the private leaderboard? </p>\r\n<p>If it is no. Can we submit several results, say 5 different predictions, and the best of the five will be chosen for ranking?</p>\r\n<p>Thanks. </p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "13724",
      "postDate": "08/31/2012 22:31:45",
      "content": "<p>[quote=Yin Zhu;13600]</p>\r\n<p>A follow up question:</p>\r\n<p>During 24 Oct to 1 Nov, will we be able to see the private leaderboard?</p>\r\n<p>If it is no. Can we submit several results, say 5 different predictions, and the best of the five will be chosen for ranking?</p>\r\n<p>Thanks.</p>\r\n<p>[/quote]The private leaderboard will not be visible until after 1 Nov, and you will only be able to make one final submission to it.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "13813",
      "postDate": "09/02/2012 21:10:32",
      "content": "<p>Just a logistics question. How will you enforce that users must use the model they submit on Oct 9 to evaluate the private leaderboard?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "13815",
      "postDate": "09/02/2012 22:24:03",
      "content": "<p>[quote=neggert;13813]</p>\r\n<p>Just a logistics question. How will you enforce that users must use the model they submit on Oct 9 to evaluate the private leaderboard?</p>\r\n<p>[/quote]This is why you are required to submit models. We will run the models of the preliminary prize winners to verify that they achieved the performance they claimed.</p>\r\n<p>In the event that any of the preliminary prize winners cheat by submitting results that were not generated from their Oct 9 models (and waste the time of everyone involved in doing so), the following actions will be taken:</p>\r\n<ul>\r\n<li>They will not receive any prize money or recognition for the results. </li><li>They will be removed from the leaderboard, and the change in final results will be publicly announced (along with their legal name) in the contest writeup\r\n</li><li>I will consider creating a blacklist of people ineligibile to win prizes on Kaggle and adding them to it\r\n</li></ul>\r\n<div>Hopefully this will be sufficient to disincentivize anyone from cheating and to catch anyone who attempts to do so. We're open to any other comments or suggestions you have.</div>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "13818",
      "postDate": "09/03/2012 00:24:01",
      "content": "<p>Sounds good. So we should write our software such that someone else could run it, at least with a little help. Good to keep in mind.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "16083",
      "postDate": "10/22/2012 23:09:31",
      "content": "<p>October 24 is Wednesday. Timeline says Thursday. The indicator at the top says there are 4.0 days to go right now, which would put us into Friday. When, exactly, can we expect the final training set, again?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "16084",
      "postDate": "10/22/2012 23:13:24",
      "content": "<p>The final training set will be available the 24th.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "16085",
      "postDate": "10/22/2012 23:21:42",
      "content": "<p>[quote=Andy Sloane;16083]</p>\r\n<p>October 24 is Wednesday. Timeline says Thursday. The indicator at the top says there are 4.0 days to go right now, which would put us into Friday. When, exactly, can we expect the final training set, again?</p>\r\n<p>[/quote]Sorry for the confusion, we have better support for more complex timelines coming soon. The current deadline refers to the final day to make submissions on the visualization contest, and it will be updated once the final dataset is released on October\r\n 24 to the end of the competition.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "179147",
      "postDate": "04/30/2017 11:46:46",
      "content": "<p>My name is Glenn Baker and i live in USA Florida and i am a happy man today, I told my self that any Loan lender that could change my Life and that of my family, i will refer any person that is looking for loan to Them. If you are in need of loan and you are 100% sure to pay back the loan please contact them and please tell them that Glenn Baker referred you to them. scottlarry918@gmail.com</p>\n\n<p>Glenn Baker</p>",
      "rawMarkdown": "My name is Glenn Baker and i live in USA Florida and i am a happy man today, I told my self that any Loan lender that could change my Life and that of my family, i will refer any person that is looking for loan to Them. If you are in need of loan and you are 100% sure to pay back the loan please contact them and please tell them that Glenn Baker referred you to them. scottlarry918@gmail.com\n\n\nGlenn Baker",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 13495,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "08/27/2012 05:29:29",
      "content": "<p>[quote=Ashwin;13470]</p>\r\n<div>1) When you submit a final model (by the way, is that just a bunch of code?), is that merely to test that you haven't hand-made your public_leaderboard predictions?[/quote]</div>\r\n<div>Yes, your final model consists of everything necessary to reproduce your results (to train your model on the training data, and make predictions on the evaluation data).</div>\r\n<div>\r\n<p>[quote=Ashwin;13470]2) From October 9 to November 1, can a competitor refine a model while waiting for the new training set and private leaderboard, and thus submit a different model than at the October 9 deadline?[/quote]No. You are required to upload your\r\n code before this to prevent cheating by overfitting on the evaluation data.</p>\r\n</div>\r\n<div>[quote=Ashwin;13470]3) Private leaderboard means private to everyone but the competitor admins, or private including other competitors but not the general public?[/quote]This terminology is borrowed from competitions where there is no temporal split between\r\n the evaluation sets, and may be less appropriate here. The &quot;private leaderboard&quot; data is used for final evaluation, and the results on this will be publicly visible at the end of the competition.</div>",
      "votes": null,
      "replies": []
    },
    {
      "id": 13600,
      "author_name": "blackswift",
      "author_url": "",
      "post_date": "08/29/2012 10:23:13",
      "content": "<p>A follow up question:</p>\r\n<p>During 24 Oct to 1 Nov, will we be able to see the private leaderboard? </p>\r\n<p>If it is no. Can we submit several results, say 5 different predictions, and the best of the five will be chosen for ranking?</p>\r\n<p>Thanks. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 13724,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "08/31/2012 22:31:45",
      "content": "<p>[quote=Yin Zhu;13600]</p>\r\n<p>A follow up question:</p>\r\n<p>During 24 Oct to 1 Nov, will we be able to see the private leaderboard?</p>\r\n<p>If it is no. Can we submit several results, say 5 different predictions, and the best of the five will be chosen for ranking?</p>\r\n<p>Thanks.</p>\r\n<p>[/quote]The private leaderboard will not be visible until after 1 Nov, and you will only be able to make one final submission to it.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 13813,
      "author_name": "neggert",
      "author_url": "",
      "post_date": "09/02/2012 21:10:32",
      "content": "<p>Just a logistics question. How will you enforce that users must use the model they submit on Oct 9 to evaluate the private leaderboard?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 13815,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "09/02/2012 22:24:03",
      "content": "<p>[quote=neggert;13813]</p>\r\n<p>Just a logistics question. How will you enforce that users must use the model they submit on Oct 9 to evaluate the private leaderboard?</p>\r\n<p>[/quote]This is why you are required to submit models. We will run the models of the preliminary prize winners to verify that they achieved the performance they claimed.</p>\r\n<p>In the event that any of the preliminary prize winners cheat by submitting results that were not generated from their Oct 9 models (and waste the time of everyone involved in doing so), the following actions will be taken:</p>\r\n<ul>\r\n<li>They will not receive any prize money or recognition for the results. </li><li>They will be removed from the leaderboard, and the change in final results will be publicly announced (along with their legal name) in the contest writeup\r\n</li><li>I will consider creating a blacklist of people ineligibile to win prizes on Kaggle and adding them to it\r\n</li></ul>\r\n<div>Hopefully this will be sufficient to disincentivize anyone from cheating and to catch anyone who attempts to do so. We're open to any other comments or suggestions you have.</div>",
      "votes": null,
      "replies": []
    },
    {
      "id": 13818,
      "author_name": "neggert",
      "author_url": "",
      "post_date": "09/03/2012 00:24:01",
      "content": "<p>Sounds good. So we should write our software such that someone else could run it, at least with a little help. Good to keep in mind.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 16083,
      "author_name": "andysloane",
      "author_url": "",
      "post_date": "10/22/2012 23:09:31",
      "content": "<p>October 24 is Wednesday. Timeline says Thursday. The indicator at the top says there are 4.0 days to go right now, which would put us into Friday. When, exactly, can we expect the final training set, again?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 16084,
      "author_name": "kevinmontrose",
      "author_url": "",
      "post_date": "10/22/2012 23:13:24",
      "content": "<p>The final training set will be available the 24th.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 16085,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "10/22/2012 23:21:42",
      "content": "<p>[quote=Andy Sloane;16083]</p>\r\n<p>October 24 is Wednesday. Timeline says Thursday. The indicator at the top says there are 4.0 days to go right now, which would put us into Friday. When, exactly, can we expect the final training set, again?</p>\r\n<p>[/quote]Sorry for the confusion, we have better support for more complex timelines coming soon. The current deadline refers to the final day to make submissions on the visualization contest, and it will be updated once the final dataset is released on October\r\n 24 to the end of the competition.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 179147,
      "author_name": "glennbake",
      "author_url": "",
      "post_date": "04/30/2017 11:46:46",
      "content": "<p>My name is Glenn Baker and i live in USA Florida and i am a happy man today, I told my self that any Loan lender that could change my Life and that of my family, i will refer any person that is looking for loan to Them. If you are in need of loan and you are 100% sure to pay back the loan please contact them and please tell them that Glenn Baker referred you to them. scottlarry918@gmail.com</p>\n\n<p>Glenn Baker</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "13470": "",
    "13495": "",
    "13600": "",
    "13724": "",
    "13813": "",
    "13815": "",
    "13818": "",
    "16083": "",
    "16084": "",
    "16085": "",
    "179147": "My name is Glenn Baker and i live in USA Florida and i am a happy man today, I told my self that any Loan lender that could change my Life and that of my family, i will refer any person that is looking for loan to Them. If you are in need of loan and you are 100% sure to pay back the loan please contact them and please tell them that Glenn Baker referred you to them. scottlarry918@gmail.com\n\n\nGlenn Baker"
  },
  "source": "meta"
}