{
  "id": 6709,
  "title": "Standard Time vs Daylight Saving Time",
  "url": "/competitions/flight2-main/discussion/6709",
  "author_name": "",
  "post_date": "2013-12-29T10:24:55.177Z",
  "votes": 1,
  "comment_count": 3,
  "views": 2078,
  "content": "<p>Hi everyone.</p>\n<p>Since we're dealing with times in UTC, after the release of the latest training set it occured to me that between the current leaderboard and the final one there will be, among others, an important difference: local times will switch from DST to standard time.</p>\n<p>This reasoning also made me doubt of the current process I'm using for dealing with times. According to public code here (https://github.com/benhamner/GEFlightQuest/blob/master/PythonModule/geflight/postgres_ingest/after_flighthistory_import.sql) this difference should be taken care of automatically when we're given datasets and flights to compute.</p>\n<p>I had a doubt, though, on how actualLandings, actualTakeoffs and groundConditions were created. It looks like these files are correct for September 10th (or sort of), but after a few tests with the current leaderboard, it looks to me like these files for days starting from September 11th on are computed using standard time instead of DST (which still applies in Semptember): am I doing something wrong?</p>\n<p>Thank you in advance.</p>",
  "messages": [
    {
      "id": "36796",
      "postDate": "12/29/2013 10:24:55",
      "content": "<p>Hi everyone.</p>\n<p>Since we're dealing with times in UTC, after the release of the latest training set it occured to me that between the current leaderboard and the final one there will be, among others, an important difference: local times will switch from DST to standard time.</p>\n<p>This reasoning also made me doubt of the current process I'm using for dealing with times. According to public code here (https://github.com/benhamner/GEFlightQuest/blob/master/PythonModule/geflight/postgres_ingest/after_flighthistory_import.sql) this difference should be taken care of automatically when we're given datasets and flights to compute.</p>\n<p>I had a doubt, though, on how actualLandings, actualTakeoffs and groundConditions were created. It looks like these files are correct for September 10th (or sort of), but after a few tests with the current leaderboard, it looks to me like these files for days starting from September 11th on are computed using standard time instead of DST (which still applies in Semptember): am I doing something wrong?</p>\n<p>Thank you in advance.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "37075",
      "postDate": "01/07/2014 11:19:36",
      "content": "<p>No news regarding this point?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "37105",
      "postDate": "01/08/2014 00:53:36",
      "content": "<p>The training set and testing set come from the same database and the <span style=\"line-height: 1.4\">Sept10th and Sept11+ flight files were generated from the same code. &nbsp;Could you be specific with what led you to this theory?&nbsp;</span></p>\n<p>Thanks</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "37116",
      "postDate": "01/08/2014 07:30:51",
      "content": "<p>Mine is indeed a theory because I have no hard facts to sustain it, but my process was this:<br>- produce a submission assuming no actualLandings and actualTakeoffs files<br>- produce a submission with a forecast of actualLandings and actualTakeoffs files<br>- produce a submission with the same forecast, but shifted 1 hour, to compensate for DST (which in this case it's assumed not be considered)</p>\n<p>The model for producing a forecast of actualLandings and actualTakeoffs was extensively tested against real actualLandings and actualTakeoffs from September 10th every time a submission was created in order to understand the differences and try to estimate the weight of such differences. Unfortunately, every time I submitted a solution produced with actualLandings and actualTakeoffs my results were worse compared to those with no forecast at all.</p>\n<p>After I tried the &quot;shifted&quot; forecast all of a sudden I didn't see my results get worse.</p>\n<p>My reasoning was, therefore, that the production of real actualLandings and actualTakeoffs was done using the same data we got as training (in the proper time period, of course), but either cutoff time or flight times were converted to standard time.</p>\n<p>Clearly the &quot;shifted&quot; forecast in my case are more precise, but I can't tell for sure if it's because of &quot;missing&quot; DST. Since the final leaderboard is in standard time, though, it would be nice to know if this behavior is going to reappear or if it's a non-existent problem.</p>\n<p>Thanks.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 37075,
      "author_name": "gabrieleseppi",
      "author_url": "",
      "post_date": "01/07/2014 11:19:36",
      "content": "<p>No news regarding this point?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 37105,
      "author_name": "nicksardo",
      "author_url": "",
      "post_date": "01/08/2014 00:53:36",
      "content": "<p>The training set and testing set come from the same database and the <span style=\"line-height: 1.4\">Sept10th and Sept11+ flight files were generated from the same code. &nbsp;Could you be specific with what led you to this theory?&nbsp;</span></p>\n<p>Thanks</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 37116,
      "author_name": "gabrieleseppi",
      "author_url": "",
      "post_date": "01/08/2014 07:30:51",
      "content": "<p>Mine is indeed a theory because I have no hard facts to sustain it, but my process was this:<br>- produce a submission assuming no actualLandings and actualTakeoffs files<br>- produce a submission with a forecast of actualLandings and actualTakeoffs files<br>- produce a submission with the same forecast, but shifted 1 hour, to compensate for DST (which in this case it's assumed not be considered)</p>\n<p>The model for producing a forecast of actualLandings and actualTakeoffs was extensively tested against real actualLandings and actualTakeoffs from September 10th every time a submission was created in order to understand the differences and try to estimate the weight of such differences. Unfortunately, every time I submitted a solution produced with actualLandings and actualTakeoffs my results were worse compared to those with no forecast at all.</p>\n<p>After I tried the &quot;shifted&quot; forecast all of a sudden I didn't see my results get worse.</p>\n<p>My reasoning was, therefore, that the production of real actualLandings and actualTakeoffs was done using the same data we got as training (in the proper time period, of course), but either cutoff time or flight times were converted to standard time.</p>\n<p>Clearly the &quot;shifted&quot; forecast in my case are more precise, but I can't tell for sure if it's because of &quot;missing&quot; DST. Since the final leaderboard is in standard time, though, it would be nice to know if this behavior is going to reappear or if it's a non-existent problem.</p>\n<p>Thanks.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "36796": "",
    "37075": "",
    "37105": "",
    "37116": ""
  },
  "source": "meta"
}