{
  "id": 5397,
  "title": "Competition Scoring Metrics",
  "url": "/competitions/flight2-milestone/discussion/5397",
  "author_name": "BigData1024",
  "post_date": "2013-08-14T11:41:56.460000",
  "votes": 0,
  "comment_count": 0,
  "views": 1037,
  "content": "<p>Admins,</p>\n<p>Unlike other competitions I have seen, the evaluation criteria for this competition is very <strong>vague</strong> and this leaves a lot of room for <strong>misinterpretation</strong>.</p>\n<p>I understand it is hard to set up and manage a competition but a competition of this size deserved more prep time before launching it.</p>\n<p>With the number of errors and corrections I have witnessed, I have a strong feeling this one was rushed to launch when it was not ready.</p>\n<p>Instead of focusing on solving the problem, folks are very confused.</p>\n<p>This leads to participants second-guessing the quality of the dataset and/or instructions provided.</p>\n<p>After reading several pages over and over, the only thing I got was that <em><strong>the Agent that produces a set of routes with the lowest average cost</strong></em> will be at the top of the leader board.</p>\n<p>I don't think this is good enough. You have to be more specific.</p>\n<p>For example, <strong><em>the number of unique FlightId values expected during each submission is not specified</em></strong>.</p>\n<p>What if there are a total of X unique FlightId values and people turn in X - 200 unique values that produces a <em><strong>lower average cost</strong></em>. What happens then?</p>\n<p>For these submissions with lower average costs, how do you account for the missing flights whose routes were not recommended?</p>\n<p>Also, I don't see any columns named <strong>FlightId</strong> in the data set. Where are these FlightId values coming from?</p>\n<p>Are these the values in the <strong>asdiflightplanid</strong> column in asdiflightplan.csv, asdiairway.csv ?</p>\n<p>You are also stating that for each FlightId, <em><strong>provide the recommended route for the remainder of the flight</strong></em>.</p>\n<p>It seems all the data from FlightQuest2DataRelease1Train_rev1.7z already has routes in them already.</p>\n<p>So what exactly are you referring to when you say <em><strong>&quot;provide the recommended route for the remainder of the flight&quot;</strong></em>?</p>\n<p>I am asking these questions to be clear because I don't want to make any false assumptions.</p>\n<p>If you believe I have missed something please point me in the right direction so that I can find the information I need.</p>\n<p>Thanks.</p>",
  "messages": [
    {
      "id": 28680,
      "postDate": "2013-08-14T11:41:56.460Z",
      "content": "<p>Admins,</p>\n<p>Unlike other competitions I have seen, the evaluation criteria for this competition is very <strong>vague</strong> and this leaves a lot of room for <strong>misinterpretation</strong>.</p>\n<p>I understand it is hard to set up and manage a competition but a competition of this size deserved more prep time before launching it.</p>\n<p>With the number of errors and corrections I have witnessed, I have a strong feeling this one was rushed to launch when it was not ready.</p>\n<p>Instead of focusing on solving the problem, folks are very confused.</p>\n<p>This leads to participants second-guessing the quality of the dataset and/or instructions provided.</p>\n<p>After reading several pages over and over, the only thing I got was that <em><strong>the Agent that produces a set of routes with the lowest average cost</strong></em> will be at the top of the leader board.</p>\n<p>I don't think this is good enough. You have to be more specific.</p>\n<p>For example, <strong><em>the number of unique FlightId values expected during each submission is not specified</em></strong>.</p>\n<p>What if there are a total of X unique FlightId values and people turn in X - 200 unique values that produces a <em><strong>lower average cost</strong></em>. What happens then?</p>\n<p>For these submissions with lower average costs, how do you account for the missing flights whose routes were not recommended?</p>\n<p>Also, I don't see any columns named <strong>FlightId</strong> in the data set. Where are these FlightId values coming from?</p>\n<p>Are these the values in the <strong>asdiflightplanid</strong> column in asdiflightplan.csv, asdiairway.csv ?</p>\n<p>You are also stating that for each FlightId, <em><strong>provide the recommended route for the remainder of the flight</strong></em>.</p>\n<p>It seems all the data from FlightQuest2DataRelease1Train_rev1.7z already has routes in them already.</p>\n<p>So what exactly are you referring to when you say <em><strong>&quot;provide the recommended route for the remainder of the flight&quot;</strong></em>?</p>\n<p>I am asking these questions to be clear because I don't want to make any false assumptions.</p>\n<p>If you believe I have missed something please point me in the right direction so that I can find the information I need.</p>\n<p>Thanks.</p>"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "28680": ""
  }
}