{
  "id": 5714,
  "title": "Relation between grib files and simulator weather files",
  "url": "/competitions/flight2-milestone/discussion/5714",
  "author_name": "",
  "post_date": "2013-09-10T18:12:59.403Z",
  "votes": null,
  "comment_count": 8,
  "views": 4027,
  "content": "<p>I was wondering, how are the weather files for the simulator created? My guess is they're built from the u and v wind components at hybrid levels from some grib files, but in grib files there are 50 levels, whereas in the simulator weather files there are only 15.</p>\n<p>Can we have some kind of tool that allows us to create such files? Or at least some insight in the creation process?</p>\n<p>Thank you.</p>",
  "messages": [
    {
      "id": "30666",
      "postDate": "09/10/2013 18:12:59",
      "content": "<p>I was wondering, how are the weather files for the simulator created? My guess is they're built from the u and v wind components at hybrid levels from some grib files, but in grib files there are 50 levels, whereas in the simulator weather files there are only 15.</p>\n<p>Can we have some kind of tool that allows us to create such files? Or at least some insight in the creation process?</p>\n<p>Thank you.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31001",
      "postDate": "09/16/2013 19:36:42",
      "content": "<p>As you stated, the weather files for the simulator were created from the u and v wind components at 15 of the hybrid wind levels that were relevant to the simulator (from 18,000 to 41,000 feet). This raw data came from the Grib2 files from the <a href=\"http://soostrc.comet.ucar.edu/data/grib/rap/\">NWS/Comet archive of RAP hybrid grids</a>.</p>\n<p>I started a Grib2 Decoder F# project, which you may find useful for extracting the wind information from the raw files:&nbsp;https://github.com/benhamner/GribDotNet/tree/master/GribDotNet</p>\n<p>The code to calculate the altitude from the hybrid wind level is here:&nbsp;https://github.com/benhamner/GribDotNet/blob/master/GribDotNet/VerticalCoordinateDecoder.fs</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31212",
      "postDate": "09/19/2013 18:23:01",
      "content": "<p>Might sound like a stupid question, but since I never worked woth meterological data...How do you read these weather files? I mean I managed to figure out that the first entry in square brackets is the altitudes. My guess is that the next entries in square brackets are either couples of u,v wind values or a set of u values followed by a set of v values.</p>\n<p>What the entries in curly brackets are, is not clear to me...</p>\n<p>&nbsp;</p>\n<p>Additionally, I assume that these are values with respect to some grid.</p>\n<p>&nbsp;</p>\n<p>On the noaa-site it is explained what RAP is and that it is a given grid, but I could not figure out how to translate that to longitude and latitude.</p>\n<p>&nbsp;</p>\n<p>Any help (also in form of a link) would be greatly appreciated!!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31425",
      "postDate": "09/23/2013 17:26:26",
      "content": "<p>Probably a little late to ask this with the milestone deadline 2 days away, but I could also use a bit of info on decoding the test weather files.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31457",
      "postDate": "09/24/2013 07:35:28",
      "content": "<p>Tanstaafl,</p>\n<p>what I got so far is the corresponding longitutdes and latitudes (you have easting and northing in the first 4 lines of the file as well as dx and dy so you can get the coordinates).</p>\n<p>What I am still not sure is the order of things following, I think though it is 451*2 entries of u,v (line x a column of u vals and line x+1 a column of v vals) at height h1 and then the 451*2 at height 2, etc.</p>\n<p>And since the easting and northing given is bottom left corner, I would assume that the first entry (el 1 line 1) starts there and (el 337 line 1) is top left; although it seems strange to me to define a matrix like this...</p>\n<p>&nbsp;</p>\n<p>Would be very grateful if some1 could clarify on this!!</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31458",
      "postDate": "09/24/2013 07:47:18",
      "content": "<p>As far as I've been able to understand (through trial-and-error and reading the simulator code) a wind file has the following structure (excluding the first 4 lines):</p>\n<p>- east and north component of wind vectors are interleaved<br>- each row contains 337 columns<br>- there are 451 rows (902 considering both east and north components)<br>- every 902 lines you have a &quot;layer&quot;, which is equivalent to a matrix of points at a specific altitude<br>- every 15 layers you have one hour of winds, so the eventual lines coming after the first hour define later hours compared to the initial cutoff time</p>\n<p>Furthermore, since the weather file is defined in a 451x337 matrix, you have to consider that this matrix is &quot;rotated&quot; 90&#176; compared to how the matrix should actually be projected on the map.</p>\n<p>The very first point should be the top left corner (northwest of Washington state).</p>\n<p>I hope this is clear enough, it's not a super-complicated structure once you've mastered it, but it's not easy to explain.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31464",
      "postDate": "09/24/2013 08:59:53",
      "content": "<p>Thanks very much!!! Almost 100% clear.</p>\n<p>I just wonder how you figured out the rotation and what you exactly mean by it: do you mean that</p>\n<p>a) element 1 at line 5 in the file is u-component to be located in the top left corner of the grid?</p>\n<p>b) something else?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31466",
      "postDate": "09/24/2013 10:51:29",
      "content": "<p>Well, 451x337 (at constant intervals of around 13km) is a &quot;vertical&quot; grid, so it must be rotated, otherwise it wouldn't cover all of the continental US (excluding Alaska).</p>\n<p>Regarding point a, yes, element 1 at line 5 is the u-component at the top left corner. Of course, element 1 at line 6 is the v-component at the top left corner as well.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31507",
      "postDate": "09/24/2013 16:45:08",
      "content": "<p>Thanks for the responses. I also dug into the fsharp code a bit last night and I think it will be easier to simply query the simulator function:</p>\n<p>&nbsp;</p>\n<p>let&nbsp;getWindVelocity&nbsp;(weather:WeatherState)&nbsp;(time:float&lt;Time&gt;)&nbsp;(position:Position)&nbsp;:&nbsp;(float&lt;DirectionX&gt;*float&lt;DirectionY&gt;)=</p>\n<p>&nbsp;</p>\n<p>with the loaded weather data inside their &quot;ProximityTree&quot; to get the wind vector at the specified time and location , this wind vector apparently affects the groundspeed which is what has an effect on the fuel cost. Not sure I can make use of this before the deadline though...</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 31001,
      "author_name": "benhamner",
      "author_url": "",
      "post_date": "09/16/2013 19:36:42",
      "content": "<p>As you stated, the weather files for the simulator were created from the u and v wind components at 15 of the hybrid wind levels that were relevant to the simulator (from 18,000 to 41,000 feet). This raw data came from the Grib2 files from the <a href=\"http://soostrc.comet.ucar.edu/data/grib/rap/\">NWS/Comet archive of RAP hybrid grids</a>.</p>\n<p>I started a Grib2 Decoder F# project, which you may find useful for extracting the wind information from the raw files:&nbsp;https://github.com/benhamner/GribDotNet/tree/master/GribDotNet</p>\n<p>The code to calculate the altitude from the hybrid wind level is here:&nbsp;https://github.com/benhamner/GribDotNet/blob/master/GribDotNet/VerticalCoordinateDecoder.fs</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31212,
      "author_name": "sandros",
      "author_url": "",
      "post_date": "09/19/2013 18:23:01",
      "content": "<p>Might sound like a stupid question, but since I never worked woth meterological data...How do you read these weather files? I mean I managed to figure out that the first entry in square brackets is the altitudes. My guess is that the next entries in square brackets are either couples of u,v wind values or a set of u values followed by a set of v values.</p>\n<p>What the entries in curly brackets are, is not clear to me...</p>\n<p>&nbsp;</p>\n<p>Additionally, I assume that these are values with respect to some grid.</p>\n<p>&nbsp;</p>\n<p>On the noaa-site it is explained what RAP is and that it is a given grid, but I could not figure out how to translate that to longitude and latitude.</p>\n<p>&nbsp;</p>\n<p>Any help (also in form of a link) would be greatly appreciated!!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31425,
      "author_name": "tanstaafl",
      "author_url": "",
      "post_date": "09/23/2013 17:26:26",
      "content": "<p>Probably a little late to ask this with the milestone deadline 2 days away, but I could also use a bit of info on decoding the test weather files.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31457,
      "author_name": "sandros",
      "author_url": "",
      "post_date": "09/24/2013 07:35:28",
      "content": "<p>Tanstaafl,</p>\n<p>what I got so far is the corresponding longitutdes and latitudes (you have easting and northing in the first 4 lines of the file as well as dx and dy so you can get the coordinates).</p>\n<p>What I am still not sure is the order of things following, I think though it is 451*2 entries of u,v (line x a column of u vals and line x+1 a column of v vals) at height h1 and then the 451*2 at height 2, etc.</p>\n<p>And since the easting and northing given is bottom left corner, I would assume that the first entry (el 1 line 1) starts there and (el 337 line 1) is top left; although it seems strange to me to define a matrix like this...</p>\n<p>&nbsp;</p>\n<p>Would be very grateful if some1 could clarify on this!!</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31458,
      "author_name": "gabrieleseppi",
      "author_url": "",
      "post_date": "09/24/2013 07:47:18",
      "content": "<p>As far as I've been able to understand (through trial-and-error and reading the simulator code) a wind file has the following structure (excluding the first 4 lines):</p>\n<p>- east and north component of wind vectors are interleaved<br>- each row contains 337 columns<br>- there are 451 rows (902 considering both east and north components)<br>- every 902 lines you have a &quot;layer&quot;, which is equivalent to a matrix of points at a specific altitude<br>- every 15 layers you have one hour of winds, so the eventual lines coming after the first hour define later hours compared to the initial cutoff time</p>\n<p>Furthermore, since the weather file is defined in a 451x337 matrix, you have to consider that this matrix is &quot;rotated&quot; 90&#176; compared to how the matrix should actually be projected on the map.</p>\n<p>The very first point should be the top left corner (northwest of Washington state).</p>\n<p>I hope this is clear enough, it's not a super-complicated structure once you've mastered it, but it's not easy to explain.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31464,
      "author_name": "sandros",
      "author_url": "",
      "post_date": "09/24/2013 08:59:53",
      "content": "<p>Thanks very much!!! Almost 100% clear.</p>\n<p>I just wonder how you figured out the rotation and what you exactly mean by it: do you mean that</p>\n<p>a) element 1 at line 5 in the file is u-component to be located in the top left corner of the grid?</p>\n<p>b) something else?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31466,
      "author_name": "gabrieleseppi",
      "author_url": "",
      "post_date": "09/24/2013 10:51:29",
      "content": "<p>Well, 451x337 (at constant intervals of around 13km) is a &quot;vertical&quot; grid, so it must be rotated, otherwise it wouldn't cover all of the continental US (excluding Alaska).</p>\n<p>Regarding point a, yes, element 1 at line 5 is the u-component at the top left corner. Of course, element 1 at line 6 is the v-component at the top left corner as well.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31507,
      "author_name": "tanstaafl",
      "author_url": "",
      "post_date": "09/24/2013 16:45:08",
      "content": "<p>Thanks for the responses. I also dug into the fsharp code a bit last night and I think it will be easier to simply query the simulator function:</p>\n<p>&nbsp;</p>\n<p>let&nbsp;getWindVelocity&nbsp;(weather:WeatherState)&nbsp;(time:float&lt;Time&gt;)&nbsp;(position:Position)&nbsp;:&nbsp;(float&lt;DirectionX&gt;*float&lt;DirectionY&gt;)=</p>\n<p>&nbsp;</p>\n<p>with the loaded weather data inside their &quot;ProximityTree&quot; to get the wind vector at the specified time and location , this wind vector apparently affects the groundspeed which is what has an effect on the fuel cost. Not sure I can make use of this before the deadline though...</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "30666": "",
    "31001": "",
    "31212": "",
    "31425": "",
    "31457": "",
    "31458": "",
    "31464": "",
    "31466": "",
    "31507": ""
  },
  "source": "meta"
}