{ "id": 5530, "title": "Simulator Release", "url": "/competitions/flight2-milestone/discussion/5530", "author_name": "", "post_date": "2013-08-24T22:47:04.407Z", "votes": 4, "comment_count": 29, "views": 15488, "content": "

We have released the code for the Flight Quest Simulator, which is downloadable from the Data Page as FlightQuestSimulator.zip. This zip file contains both the source code and compiled executables. The executables are cross-platform and run under mono.

\n

The Simulator is written entirely in F#, a functional language for the .NET platform that is based heavily on OCaml. We chose F# in implementing this project because we wanted both clean interoperability with the rest of our backend stack (which is written in C#) and a language well suited for numeric programming. We realize that the vast majority of you have never seen a line of F# before, so we hope this is a fun opportunity to explore a cool new language. We find F# code itself very clean, concise, and readable.

\n

We have not yet released samples of the input files necessary to run the Simulator. These include configuration settings, flight initialization files, wind data, and airport data. We also haven't released other files over the leaderboard period that you may find useful (flight and weather information prior to each day's cutoff time). Additionally, the code to generate these files from the raw data as well as the comprehensive unit tests for the Simulator aren't included in the current code release. These sample files, data, and code will be released in the coming days.

\n

The key components you'll find in the source code of the current version of the Simulator are as follows:

\n\n

We will be making updates to the Simulator over the course of the competition, which may include both bug fixes and feature improvements. The biggest update to the Simulator will likely come after the milestone. Potential changes include the addition of static no-fly zones and a more sophisticated landing model.

\n

Here's a sample of real-world constraints that the Simulator is not currently taking into account:

\n\n

This is by far the most complex competition we've ever hosted, which means there is a large surface area for potential bugs and issues in both the data and the Simulator. Please report any issues that you find on the competitions forums in a constructive manner. I've created the thread Simulator Issues and Bug Reports to use for any specific issues regarding the Simulator code. Remember, as George Box said, "All models are wrong, some models are useful." The current version of the Simulator is by no means a complete reflection of reality nor does it account for all the considerations that need to be made when determining a flight route.

\n

As a reminder, use the forums for any questions or comments that you have on the simulator, as opposed to contacting Kaggle's scientists directly. The purpose of this is twofold: we want all the discourse around this competition to be public and individual Kaggle scientists may be unavailable at times (e.g. I'll be travelling this week and have a completely booked schedule). We will be monitoring the forums and responding to questions as appropriate.

\n

We hope that you have fun with this contest, learn something, and develop great new technology.

", "messages": [ { "id": "29406", "postDate": "08/24/2013 22:47:04", "content": "

We have released the code for the Flight Quest Simulator, which is downloadable from the Data Page as FlightQuestSimulator.zip. This zip file contains both the source code and compiled executables. The executables are cross-platform and run under mono.

\n

The Simulator is written entirely in F#, a functional language for the .NET platform that is based heavily on OCaml. We chose F# in implementing this project because we wanted both clean interoperability with the rest of our backend stack (which is written in C#) and a language well suited for numeric programming. We realize that the vast majority of you have never seen a line of F# before, so we hope this is a fun opportunity to explore a cool new language. We find F# code itself very clean, concise, and readable.

\n

We have not yet released samples of the input files necessary to run the Simulator. These include configuration settings, flight initialization files, wind data, and airport data. We also haven't released other files over the leaderboard period that you may find useful (flight and weather information prior to each day's cutoff time). Additionally, the code to generate these files from the raw data as well as the comprehensive unit tests for the Simulator aren't included in the current code release. These sample files, data, and code will be released in the coming days.

\n

The key components you'll find in the source code of the current version of the Simulator are as follows:

\n\n

We will be making updates to the Simulator over the course of the competition, which may include both bug fixes and feature improvements. The biggest update to the Simulator will likely come after the milestone. Potential changes include the addition of static no-fly zones and a more sophisticated landing model.

\n

Here's a sample of real-world constraints that the Simulator is not currently taking into account:

\n\n

This is by far the most complex competition we've ever hosted, which means there is a large surface area for potential bugs and issues in both the data and the Simulator. Please report any issues that you find on the competitions forums in a constructive manner. I've created the thread Simulator Issues and Bug Reports to use for any specific issues regarding the Simulator code. Remember, as George Box said, "All models are wrong, some models are useful." The current version of the Simulator is by no means a complete reflection of reality nor does it account for all the considerations that need to be made when determining a flight route.

\n

As a reminder, use the forums for any questions or comments that you have on the simulator, as opposed to contacting Kaggle's scientists directly. The purpose of this is twofold: we want all the discourse around this competition to be public and individual Kaggle scientists may be unavailable at times (e.g. I'll be travelling this week and have a completely booked schedule). We will be monitoring the forums and responding to questions as appropriate.

\n

We hope that you have fun with this contest, learn something, and develop great new technology.

", "rawMarkdown": "", "votes": null }, { "id": "29562", "postDate": "08/27/2013 12:46:14", "content": "

I have just released the one day's worth of files to run the simulator. These are derived from the last day of training (July 4, 2013) and available as oneDaySimulatorFiles.zip.

\n

It also contains the corresponding test file (test_20130704_1540_sample.csv) and a oneDaySampleSubmission.csv file, which was generated with the Basic Python Agent.

\n

Additionally, we removed diverted flights from the simulator before we released the code, but this wasn't reflected in the test flights file on the data page. Accordingly, we've released testFlightsRev3.csv, which is now available for download & is a subset of the previous file.

", "rawMarkdown": "", "votes": null }, { "id": "29574", "postDate": "08/27/2013 14:59:05", "content": "

Simulator requires also:

\n

Taxi.csv

\n

scheduledLandings_{0}.csv

", "rawMarkdown": "", "votes": null }, { "id": "29726", "postDate": "08/29/2013 18:16:04", "content": "

[quote=Haru;29574]

\n

Simulator requires also:

\n

Taxi.csv

\n

scheduledLandings_{0}.csv

\n

[/quote]

\n

 

\n

Any news about this?

\n

 

\n

Someone has succeeded the simulator run?

\n

 

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "29731", "postDate": "08/29/2013 19:18:56", "content": "

I am patiently waiting for a response on this too.  Placing a zero byte taxi.csv file in the data folder results in an "Arithmetic operation resulted in an overflow" error.

", "rawMarkdown": "", "votes": null }, { "id": "29753", "postDate": "08/30/2013 00:30:48", "content": "

Why Airports.csv in oneDaySimulatorFiles is different from Airports.csv in the Data page? coordinates are in easting and northing format, instead of lon\\lat and has radius and height?

\n

Apologizes for the amount of questions thrown all at the same time...but there are so many bits not so clear!

", "rawMarkdown": "", "votes": null }, { "id": "29754", "postDate": "08/30/2013 00:33:08", "content": "

I've updated the simulator files to OneDaySimulatorFilesRev2.zip, which contains Taxi.csv.

", "rawMarkdown": "", "votes": null }, { "id": "29794", "postDate": "08/30/2013 19:36:44", "content": "

The second revision of the simulator is still not working with the second revision of the data. It throws System.FormatException. Did anybody succeed to run it?

", "rawMarkdown": "", "votes": null }, { "id": "29795", "postDate": "08/30/2013 19:41:40", "content": "

[quote=id;29794]

\n

The second revision of the simulator is still not working with the second revision of the data. It throws System.FormatException. Did anybody succeed to run it?

\n

[/quote]

\n

I can't run it. We are still waiting:

\n

scheduledLandings_{0}.csv

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "29796", "postDate": "08/30/2013 20:25:14", "content": "

For those with the system local not in english, switch it. It solved all the format problems for me.

", "rawMarkdown": "", "votes": null }, { "id": "29820", "postDate": "08/31/2013 01:26:42", "content": "

Notice: Revision 3 of the Simulator

\n

We updated the simulator today to fix a minor bug regarding the plane's weight when reaching the arrival zone.  

\n

 

\n

[quote=José A. Guerrero;29795]

\n

I can't run it. We are still waiting:

\n

scheduledLandings_{0}.csv

\n

[/quote]

\n

Hi José, 

\n

Please take a look at this.  You should now have all the files necessary to run the simulator.  Please let me know if you have any problems.

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "29887", "postDate": "09/01/2013 02:27:22", "content": "

[quote=Alessandro Mariani;29753]Why Airports.csv in
oneDaySimulatorFiles is different from Airports.csv in the Data page?
coordinates are in easting and northing format, instead of lon\\lat and
has radius and height?Apologizes for the amount of questions thrown all at the same time...but there are so many bits not so clear![/quote]

\n

 

\n

bump.

\n

Also:

\n

What's the reference ellipsoid and zone for the UTM data?

\n

[edit]

\n

Both airports.csv files seem to contain the same airports so I guess it doesn't matter for now.

\n

 

\n

[edit2]

\n

LambertConverter.fs seems to answer this.

", "rawMarkdown": "", "votes": null }, { "id": "29953", "postDate": "09/01/2013 19:28:02", "content": "

[quote=Ben Hamner;29406]

\n

The biggest update to the Simulator will likely come after the milestone. Potential changes include the addition of static no-fly zones and a more sophisticated landing model.

\n

Here's a sample of real-world constraints that the Simulator is not currently taking into account:

\n\n

[/quote]

\n

You say the simulator is not "currently" accounting for these constraints.  When will the final simulator be released? Can we plan to ignore any of these constraints for the final submission?

", "rawMarkdown": "", "votes": null }, { "id": "29954", "postDate": "09/01/2013 19:38:10", "content": "

Does the current simulator account for the relationship between the amount of fuel the plane needs to have at the start and the length of the flight plan? Will future versions account for this?

\n

(My understanding is that it is current practice is to limit the fuel loaded to the amount needed. If the simulator always starts with the same fuel load it seems the model favors longer flights than would be better in the real world).

", "rawMarkdown": "", "votes": null }, { "id": "30140", "postDate": "09/04/2013 10:17:46", "content": "

On the evaluation page it's said that 'The exact parameterization of the Cost function will be available in the Simulator documentation'. I haven't found any relevant documentation in simulator pack. So is it a deal of reverse engineering or there is some more or less formal mathematical description?

\n

Edit. Found an answer to the question at test sample file

", "rawMarkdown": "", "votes": null }, { "id": "30539", "postDate": "09/09/2013 16:14:24", "content": "

Hi,

\n

I want to know what's the current state of the competition. It's been a month since initial release of dataset and only about 2 weeks left before milestone.

\n

[quote=Ben Hamner;29406]

\n

We also haven't released other files over the leaderboard period that you may find useful (flight and weather information prior to each day's cutoff time). Additionally, the code to generate these files from the raw data as well as the comprehensive unit tests for the Simulator aren't included in the current code release. These sample files, data, and code will be released in the coming days.

\n

[/quote]

\n

I can run simulator for the training day, July 4th, but we cannot make use of other training days since most of the columns of files required to run simulator are not locally reproducible if I'm not wrong. And also all of ASDI files for test days are missing even before cutoff time. Is this intentional?

\n

I'd like to know especially if there are going to be any updates BEFORE MILESTONE and if so, when.

\n

Thanks.

", "rawMarkdown": "", "votes": null }, { "id": "30819", "postDate": "09/13/2013 00:06:16", "content": "

Partial weather data for test days has been uploaded.

\n

The weather files normally contain 8 snapshots of the wind at one hour intervals.  We have now uploaded files (TestWeatherData-Partial.zip) containing the first snapshot for each day, which is taken less than an hour prior to the cutoff time.

\n

 

\n

Flight Simulator Rev 4 has been released.

\n

We've modified the simulator so you can use these new partial weather files without having to populate the next seven wind snapshots. Therefore, the wind is effectively constant for that day with the first snapshot.  Should you predict wind for future hours, the simulator will take the closest snapshot point instead.

\n

We've added a new flight logging feature to the simulator. For each step in a simulated flight, a record of these status variables is generated and appended to a log:

\n\n

If you use the provided RunSimulation project or executable, these flight logs will be printed out to daily CSV files (see new parameters in Configuration.json and App.config).  If you have your own runner of the FlightQuest.dll, please look how we're processing the log output.  Because F# lists are singly-linked lists and items are appended to the head, the returned log is chronologically reversed. Furthermore, you need to calculate latitude/longitude given the LambertConverter and loaded projection data. 

\n

Since no changes were made to the models or to the cost function, our leaderboard does not require re-scoring. 

", "rawMarkdown": "", "votes": null }, { "id": "30841", "postDate": "09/13/2013 09:48:45", "content": "

Hi,

\n

I got some errors when running the Flight Simulator Version 4 with the new partial data.

\n

It complains about missing Airports/groundConditions_xxx.csv files for the new base dates.

\n

I guess this will also be the case for missing Airports/actualTakeoffs_xxx.csv and Airports/actualLandings_xxx.csv as I only have those files defined for 2013/07/04.

\n

Can you please provides the missing files.

\n

Thanks

\n

 

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "30844", "postDate": "09/13/2013 10:59:35", "content": "

[quote=Seby;30841]

\n

Hi,

\n

I got some errors when running the Flight Simulator Version 4 with the new partial data.

\n

It complains about missing Airports/groundConditions_xxx.csv files for the new base dates.

\n

I guess this will also be the case for missing Airports/actualTakeoffs_xxx.csv and Airports/actualLandings_xxx.csv as I only have those files defined for 2013/07/04.

\n

Can you please provides the missing files.

\n

Thanks

\n

 

\n

 

\n

[/quote]

\n

I agree! Can we have all the missing files that we need for running the test dates through the simulator. Obviously they exist, or the test (leaderboard) scoring wouldn't be working.

\n

If we aren't going to get these files, then please let us know.

\n

Ta!

", "rawMarkdown": "", "votes": null }, { "id": "30846", "postDate": "09/13/2013 11:18:55", "content": "

Hi

\n

When I run the new simulator code (v4), with the new OneDaySimulatorFiles-Rev3 files, I get many flights failing to complete:

\n

Simulating 20130704_1540...
553 of 1026 arrived - mean cost $57729.32, 473 messages

\n

 

\n

Previously, there were only 2 flights that failed (due to extreme proximity to destination).

\n

Can the admin please advise if this is intentional, or is a bug?

\n

(has the simulator somehow been tweaked to work better with the test data, but is now failing to 'fly' most of the 4th July flights?)

\n

Thanks!

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "30867", "postDate": "09/13/2013 17:03:01", "content": "

[quote=Dave;30844]

\n

[quote=Seby;30841]

\n

It complains about missing Airports/groundConditions_xxx.csv files for the new base dates.

\n

I guess this will also be the case for missing Airports/actualTakeoffs_xxx.csv and Airports/actualLandings_xxx.csv as I only have those files defined for 2013/07/04.

\n

[/quote]

\n

I agree! Can we have all the missing files that we need for running the test dates through the simulator. Obviously they exist, or the test (leaderboard) scoring wouldn't be working.

\n

[/quote]

\n

We are not providing these files since all this is future information.  You need to predict what the airport and weather conditions will be.

\n

 

\n

[quote=Dave;30844]

\n

Previously, there were only 2 flights that failed (due to extreme proximity to destination).

\n

Can the admin please advise if this is intentional, or is a bug?

\n

(has the simulator somehow been tweaked to work better with the test data, but is now failing to 'fly' most of the 4th July flights?)

\n

[/quote]

\n

Double check your using the same files for both simulation runs.   My testing has not revealed any differences in scores - I just confirmed with the oneDaySampleSubmission(July 4th) files.  

", "rawMarkdown": "", "votes": null }, { "id": "30870", "postDate": "09/13/2013 17:34:58", "content": "

I think the problem is in OneDaySimulatorFiles-Rev3.zip. I see the same problem as Dave (553 arrived) when I use the updated files, whether I use the Rev3 or Rev4 simulator, but not with the OneDaySimulatorFilesRev2.zip files.

", "rawMarkdown": "", "votes": null }, { "id": "30871", "postDate": "09/13/2013 17:39:48", "content": "

Ah, the sample submission file in OneDaySimulatorFiles-Rev3 had the altitudes reverted back to 18,000 ft.  They should be at 17,000 ft in order for Eastbound flights to hit the arrival zone.  Updating that file now.

\n

Thanks Dave & TecS

", "rawMarkdown": "", "votes": null }, { "id": "30895", "postDate": "09/14/2013 06:16:02", "content": "

[quote=ℕick ∈ Sardo;30867]

\n

You need to predict what the airport and weather conditions will be.[/quote]How can we predict information which already provided to us as Approved(!) Data Sets http://www.gequest.com/wiki/FlightQuest2 ?

", "rawMarkdown": "", "votes": null }, { "id": "30940", "postDate": "09/15/2013 14:26:09", "content": "

[quote=Haru;30895]

\n

[quote=ℕick ∈ Sardo;30867]

\n

You need to predict what the airport and weather conditions will be.[/quote]How can we predict information which already provided to us as Approved(!) Data Sets http://www.gequest.com/wiki/FlightQuest2 ?

\n

[/quote]

\n

I agree, the RAP data sets have been approved, so they should be usable. Can someone elaborate on the matter?

\n

Furthermore, what was the purpose of renaming some structures in the latest revision of the simulator? It's not easy (and surely it's time consuming) to go through the entire code every time there is an update... And updates seem to be rather frequent.

", "rawMarkdown": "", "votes": null }, { "id": "30947", "postDate": "09/15/2013 20:08:52", "content": "

[quote=Haru;30895]

\n

How can we predict information which already provided to us as Approved(!) Data Sets http://www.gequest.com/wiki/FlightQuest2 ?

\n

[/quote]

\n

 

\n

This is an important issue that calls for a response from the admin.

\n

If one were to obtain the weather data for the test dates, it may be possible to run the simulator on the test data. This would cause the contest to degenerate into a brute force search. Sure, the simulator still won't have access to ground conditions, congestion etc., but I would think that wind velocity by itself can go a long way in aiding with optimization.

\n

Also, I think weather data from the future would allow a contestant to fairly accurately estimate the leaderboard score (both public and private) of a solution without even making a submission. This would seem like a strong argument for banning usage of weather data beyond the cutoff time, or even disallowing outside data entirely.

\n

On the other hand, the contest as stated, sounds a bit contrived. It is possible that I misread the terms, but it seems like all the future waypoints are to be predicted based on data, at or before the cutoff time. While this constraint is reasonable for the first waypoint, it is not very real-life-like for the subsequent waypoints. After all, by the time the plane reaches its first waypoint, wouldn't the onboard agent have access to updated weather information? Why not use  it to come up with the next waypoint? Why rely on predictions when the agent has access to hard data?

\n

The answer may be that it is hard to pose this problem in the form of a contest if real-time weather information were allowed. Gaming would be an issue. It may be hard to detect illegal usage of weather information (such as using weather information at the next waypoint to optimize the current waypoint, which would not be possible in real life). Nevertheless, I would like to hear the admins' take on this.

\n

While I appreciate the amount of work that Kaggle has put into this contest, I think the rules must have been spelt out more clearly.

", "rawMarkdown": "", "votes": null }, { "id": "30948", "postDate": "09/15/2013 20:56:00", "content": "

[quote=Anil Thomas;30947]

\n

If one were to obtain the weather data for the test dates, it may be possible to run the simulator on the test data. [/quote]

\n

It's possible anyway. The impossible is to predict data which you can just take. In this case we are not blinded. And simulators behavior anyway far from reality for now. So it's not the biggest problem(I have a long list of much more important issues ;) ). In every competition some future information provided - at least leaderboard data. Also in real life routes can be updated at least every hour - in this competition we can't do it. So using this data closer to reality.

", "rawMarkdown": "", "votes": null }, { "id": "31010", "postDate": "09/16/2013 20:59:18", "content": "

Quoting the Basic Structure of FQ2 page,

\n

[quote]After the cutoff time, no additional information is available, which mimics real-life situations where the future is not known. Your models must be built using existing information and be robust to the uncertainty of the future.[/quote] You may use wind information or forecasts that were available prior to the cutoff time for each day, but not those that were created after it, in your solution.

", "rawMarkdown": "", "votes": null }, { "id": "31068", "postDate": "09/18/2013 03:58:11", "content": "

I made some tweaks (changed the default values for cruise elevation, speed and descend distance and some other minor things) to the benchmark code and made my first submission yesterday. I expected it to do a little better than the baseline. Instead it scored 12766 and ended up 12th! The current leader is just 58 points ahead. Unless someone comes up with an effective routing algorithm within the next week, there is some danger of the milestone prize going to the contestant with the luckiest choice of parameters.

", "rawMarkdown": "", "votes": null }, { "id": "31265", "postDate": "09/20/2013 17:07:26", "content": "

[quote=Anil Thomas;31068]

\n

there is some danger of the milestone prize going to the contestant with the luckiest choice of parameters.

\n

[/quote]

\n

I hit a wall going down that path. It doesn't look possible to get below 12760 by optimizing the default cruise altitude, speed and descend distance in the benchmark code. I was wrong when I said luck would be enough to win this.

\n

It turns out that optimizing these parameters on the OneDaySimulatorFiles training data produces more or less optimal settings for the test data as well. Further perturbations of the parameters did not improve the score much.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 29562, "author_name": "benhamner", "author_url": "", "post_date": "08/27/2013 12:46:14", "content": "

I have just released the one day's worth of files to run the simulator. These are derived from the last day of training (July 4, 2013) and available as oneDaySimulatorFiles.zip.

\n

It also contains the corresponding test file (test_20130704_1540_sample.csv) and a oneDaySampleSubmission.csv file, which was generated with the Basic Python Agent.

\n

Additionally, we removed diverted flights from the simulator before we released the code, but this wasn't reflected in the test flights file on the data page. Accordingly, we've released testFlightsRev3.csv, which is now available for download & is a subset of the previous file.

", "votes": null, "replies": [] }, { "id": 29574, "author_name": "dmytrolystopad", "author_url": "", "post_date": "08/27/2013 14:59:05", "content": "

Simulator requires also:

\n

Taxi.csv

\n

scheduledLandings_{0}.csv

", "votes": null, "replies": [] }, { "id": 29726, "author_name": "blindape", "author_url": "", "post_date": "08/29/2013 18:16:04", "content": "

[quote=Haru;29574]

\n

Simulator requires also:

\n

Taxi.csv

\n

scheduledLandings_{0}.csv

\n

[/quote]

\n

 

\n

Any news about this?

\n

 

\n

Someone has succeeded the simulator run?

\n

 

\n

 

", "votes": null, "replies": [] }, { "id": 29731, "author_name": "timhardin", "author_url": "", "post_date": "08/29/2013 19:18:56", "content": "

I am patiently waiting for a response on this too.  Placing a zero byte taxi.csv file in the data folder results in an "Arithmetic operation resulted in an overflow" error.

", "votes": null, "replies": [] }, { "id": 29753, "author_name": "alzmcr", "author_url": "", "post_date": "08/30/2013 00:30:48", "content": "

Why Airports.csv in oneDaySimulatorFiles is different from Airports.csv in the Data page? coordinates are in easting and northing format, instead of lon\\lat and has radius and height?

\n

Apologizes for the amount of questions thrown all at the same time...but there are so many bits not so clear!

", "votes": null, "replies": [] }, { "id": 29754, "author_name": "benhamner", "author_url": "", "post_date": "08/30/2013 00:33:08", "content": "

I've updated the simulator files to OneDaySimulatorFilesRev2.zip, which contains Taxi.csv.

", "votes": null, "replies": [] }, { "id": 29794, "author_name": "id118392", "author_url": "", "post_date": "08/30/2013 19:36:44", "content": "

The second revision of the simulator is still not working with the second revision of the data. It throws System.FormatException. Did anybody succeed to run it?

", "votes": null, "replies": [] }, { "id": 29795, "author_name": "blindape", "author_url": "", "post_date": "08/30/2013 19:41:40", "content": "

[quote=id;29794]

\n

The second revision of the simulator is still not working with the second revision of the data. It throws System.FormatException. Did anybody succeed to run it?

\n

[/quote]

\n

I can't run it. We are still waiting:

\n

scheduledLandings_{0}.csv

\n

 

", "votes": null, "replies": [] }, { "id": 29796, "author_name": "senormajete", "author_url": "", "post_date": "08/30/2013 20:25:14", "content": "

For those with the system local not in english, switch it. It solved all the format problems for me.

", "votes": null, "replies": [] }, { "id": 29820, "author_name": "nicksardo", "author_url": "", "post_date": "08/31/2013 01:26:42", "content": "

Notice: Revision 3 of the Simulator

\n

We updated the simulator today to fix a minor bug regarding the plane's weight when reaching the arrival zone.  

\n

 

\n

[quote=José A. Guerrero;29795]

\n

I can't run it. We are still waiting:

\n

scheduledLandings_{0}.csv

\n

[/quote]

\n

Hi José, 

\n

Please take a look at this.  You should now have all the files necessary to run the simulator.  Please let me know if you have any problems.

\n

 

", "votes": null, "replies": [] }, { "id": 29887, "author_name": "tanstaafl", "author_url": "", "post_date": "09/01/2013 02:27:22", "content": "

[quote=Alessandro Mariani;29753]Why Airports.csv in
oneDaySimulatorFiles is different from Airports.csv in the Data page?
coordinates are in easting and northing format, instead of lon\\lat and
has radius and height?Apologizes for the amount of questions thrown all at the same time...but there are so many bits not so clear![/quote]

\n

 

\n

bump.

\n

Also:

\n

What's the reference ellipsoid and zone for the UTM data?

\n

[edit]

\n

Both airports.csv files seem to contain the same airports so I guess it doesn't matter for now.

\n

 

\n

[edit2]

\n

LambertConverter.fs seems to answer this.

", "votes": null, "replies": [] }, { "id": 29953, "author_name": "davidwilson0", "author_url": "", "post_date": "09/01/2013 19:28:02", "content": "

[quote=Ben Hamner;29406]

\n

The biggest update to the Simulator will likely come after the milestone. Potential changes include the addition of static no-fly zones and a more sophisticated landing model.

\n

Here's a sample of real-world constraints that the Simulator is not currently taking into account:

\n\n

[/quote]

\n

You say the simulator is not "currently" accounting for these constraints.  When will the final simulator be released? Can we plan to ignore any of these constraints for the final submission?

", "votes": null, "replies": [] }, { "id": 29954, "author_name": "davidwilson0", "author_url": "", "post_date": "09/01/2013 19:38:10", "content": "

Does the current simulator account for the relationship between the amount of fuel the plane needs to have at the start and the length of the flight plan? Will future versions account for this?

\n

(My understanding is that it is current practice is to limit the fuel loaded to the amount needed. If the simulator always starts with the same fuel load it seems the model favors longer flights than would be better in the real world).

", "votes": null, "replies": [] }, { "id": 30140, "author_name": "", "author_url": "", "post_date": "09/04/2013 10:17:46", "content": "

On the evaluation page it's said that 'The exact parameterization of the Cost function will be available in the Simulator documentation'. I haven't found any relevant documentation in simulator pack. So is it a deal of reverse engineering or there is some more or less formal mathematical description?

\n

Edit. Found an answer to the question at test sample file

", "votes": null, "replies": [] }, { "id": 30539, "author_name": "naokazumizuta", "author_url": "", "post_date": "09/09/2013 16:14:24", "content": "

Hi,

\n

I want to know what's the current state of the competition. It's been a month since initial release of dataset and only about 2 weeks left before milestone.

\n

[quote=Ben Hamner;29406]

\n

We also haven't released other files over the leaderboard period that you may find useful (flight and weather information prior to each day's cutoff time). Additionally, the code to generate these files from the raw data as well as the comprehensive unit tests for the Simulator aren't included in the current code release. These sample files, data, and code will be released in the coming days.

\n

[/quote]

\n

I can run simulator for the training day, July 4th, but we cannot make use of other training days since most of the columns of files required to run simulator are not locally reproducible if I'm not wrong. And also all of ASDI files for test days are missing even before cutoff time. Is this intentional?

\n

I'd like to know especially if there are going to be any updates BEFORE MILESTONE and if so, when.

\n

Thanks.

", "votes": null, "replies": [] }, { "id": 30819, "author_name": "nicksardo", "author_url": "", "post_date": "09/13/2013 00:06:16", "content": "

Partial weather data for test days has been uploaded.

\n

The weather files normally contain 8 snapshots of the wind at one hour intervals.  We have now uploaded files (TestWeatherData-Partial.zip) containing the first snapshot for each day, which is taken less than an hour prior to the cutoff time.

\n

 

\n

Flight Simulator Rev 4 has been released.

\n

We've modified the simulator so you can use these new partial weather files without having to populate the next seven wind snapshots. Therefore, the wind is effectively constant for that day with the first snapshot.  Should you predict wind for future hours, the simulator will take the closest snapshot point instead.

\n

We've added a new flight logging feature to the simulator. For each step in a simulated flight, a record of these status variables is generated and appended to a log:

\n\n

If you use the provided RunSimulation project or executable, these flight logs will be printed out to daily CSV files (see new parameters in Configuration.json and App.config).  If you have your own runner of the FlightQuest.dll, please look how we're processing the log output.  Because F# lists are singly-linked lists and items are appended to the head, the returned log is chronologically reversed. Furthermore, you need to calculate latitude/longitude given the LambertConverter and loaded projection data. 

\n

Since no changes were made to the models or to the cost function, our leaderboard does not require re-scoring. 

", "votes": null, "replies": [] }, { "id": 30841, "author_name": "sebyml", "author_url": "", "post_date": "09/13/2013 09:48:45", "content": "

Hi,

\n

I got some errors when running the Flight Simulator Version 4 with the new partial data.

\n

It complains about missing Airports/groundConditions_xxx.csv files for the new base dates.

\n

I guess this will also be the case for missing Airports/actualTakeoffs_xxx.csv and Airports/actualLandings_xxx.csv as I only have those files defined for 2013/07/04.

\n

Can you please provides the missing files.

\n

Thanks

\n

 

\n

 

", "votes": null, "replies": [] }, { "id": 30844, "author_name": "dave40", "author_url": "", "post_date": "09/13/2013 10:59:35", "content": "

[quote=Seby;30841]

\n

Hi,

\n

I got some errors when running the Flight Simulator Version 4 with the new partial data.

\n

It complains about missing Airports/groundConditions_xxx.csv files for the new base dates.

\n

I guess this will also be the case for missing Airports/actualTakeoffs_xxx.csv and Airports/actualLandings_xxx.csv as I only have those files defined for 2013/07/04.

\n

Can you please provides the missing files.

\n

Thanks

\n

 

\n

 

\n

[/quote]

\n

I agree! Can we have all the missing files that we need for running the test dates through the simulator. Obviously they exist, or the test (leaderboard) scoring wouldn't be working.

\n

If we aren't going to get these files, then please let us know.

\n

Ta!

", "votes": null, "replies": [] }, { "id": 30846, "author_name": "dave40", "author_url": "", "post_date": "09/13/2013 11:18:55", "content": "

Hi

\n

When I run the new simulator code (v4), with the new OneDaySimulatorFiles-Rev3 files, I get many flights failing to complete:

\n

Simulating 20130704_1540...
553 of 1026 arrived - mean cost $57729.32, 473 messages

\n

 

\n

Previously, there were only 2 flights that failed (due to extreme proximity to destination).

\n

Can the admin please advise if this is intentional, or is a bug?

\n

(has the simulator somehow been tweaked to work better with the test data, but is now failing to 'fly' most of the 4th July flights?)

\n

Thanks!

\n

 

", "votes": null, "replies": [] }, { "id": 30867, "author_name": "nicksardo", "author_url": "", "post_date": "09/13/2013 17:03:01", "content": "

[quote=Dave;30844]

\n

[quote=Seby;30841]

\n

It complains about missing Airports/groundConditions_xxx.csv files for the new base dates.

\n

I guess this will also be the case for missing Airports/actualTakeoffs_xxx.csv and Airports/actualLandings_xxx.csv as I only have those files defined for 2013/07/04.

\n

[/quote]

\n

I agree! Can we have all the missing files that we need for running the test dates through the simulator. Obviously they exist, or the test (leaderboard) scoring wouldn't be working.

\n

[/quote]

\n

We are not providing these files since all this is future information.  You need to predict what the airport and weather conditions will be.

\n

 

\n

[quote=Dave;30844]

\n

Previously, there were only 2 flights that failed (due to extreme proximity to destination).

\n

Can the admin please advise if this is intentional, or is a bug?

\n

(has the simulator somehow been tweaked to work better with the test data, but is now failing to 'fly' most of the 4th July flights?)

\n

[/quote]

\n

Double check your using the same files for both simulation runs.   My testing has not revealed any differences in scores - I just confirmed with the oneDaySampleSubmission(July 4th) files.  

", "votes": null, "replies": [] }, { "id": 30870, "author_name": "hyperborean", "author_url": "", "post_date": "09/13/2013 17:34:58", "content": "

I think the problem is in OneDaySimulatorFiles-Rev3.zip. I see the same problem as Dave (553 arrived) when I use the updated files, whether I use the Rev3 or Rev4 simulator, but not with the OneDaySimulatorFilesRev2.zip files.

", "votes": null, "replies": [] }, { "id": 30871, "author_name": "nicksardo", "author_url": "", "post_date": "09/13/2013 17:39:48", "content": "

Ah, the sample submission file in OneDaySimulatorFiles-Rev3 had the altitudes reverted back to 18,000 ft.  They should be at 17,000 ft in order for Eastbound flights to hit the arrival zone.  Updating that file now.

\n

Thanks Dave & TecS

", "votes": null, "replies": [] }, { "id": 30895, "author_name": "dmytrolystopad", "author_url": "", "post_date": "09/14/2013 06:16:02", "content": "

[quote=ℕick ∈ Sardo;30867]

\n

You need to predict what the airport and weather conditions will be.[/quote]How can we predict information which already provided to us as Approved(!) Data Sets http://www.gequest.com/wiki/FlightQuest2 ?

", "votes": null, "replies": [] }, { "id": 30940, "author_name": "gabrieleseppi", "author_url": "", "post_date": "09/15/2013 14:26:09", "content": "

[quote=Haru;30895]

\n

[quote=ℕick ∈ Sardo;30867]

\n

You need to predict what the airport and weather conditions will be.[/quote]How can we predict information which already provided to us as Approved(!) Data Sets http://www.gequest.com/wiki/FlightQuest2 ?

\n

[/quote]

\n

I agree, the RAP data sets have been approved, so they should be usable. Can someone elaborate on the matter?

\n

Furthermore, what was the purpose of renaming some structures in the latest revision of the simulator? It's not easy (and surely it's time consuming) to go through the entire code every time there is an update... And updates seem to be rather frequent.

", "votes": null, "replies": [] }, { "id": 30947, "author_name": "anlthms", "author_url": "", "post_date": "09/15/2013 20:08:52", "content": "

[quote=Haru;30895]

\n

How can we predict information which already provided to us as Approved(!) Data Sets http://www.gequest.com/wiki/FlightQuest2 ?

\n

[/quote]

\n

 

\n

This is an important issue that calls for a response from the admin.

\n

If one were to obtain the weather data for the test dates, it may be possible to run the simulator on the test data. This would cause the contest to degenerate into a brute force search. Sure, the simulator still won't have access to ground conditions, congestion etc., but I would think that wind velocity by itself can go a long way in aiding with optimization.

\n

Also, I think weather data from the future would allow a contestant to fairly accurately estimate the leaderboard score (both public and private) of a solution without even making a submission. This would seem like a strong argument for banning usage of weather data beyond the cutoff time, or even disallowing outside data entirely.

\n

On the other hand, the contest as stated, sounds a bit contrived. It is possible that I misread the terms, but it seems like all the future waypoints are to be predicted based on data, at or before the cutoff time. While this constraint is reasonable for the first waypoint, it is not very real-life-like for the subsequent waypoints. After all, by the time the plane reaches its first waypoint, wouldn't the onboard agent have access to updated weather information? Why not use  it to come up with the next waypoint? Why rely on predictions when the agent has access to hard data?

\n

The answer may be that it is hard to pose this problem in the form of a contest if real-time weather information were allowed. Gaming would be an issue. It may be hard to detect illegal usage of weather information (such as using weather information at the next waypoint to optimize the current waypoint, which would not be possible in real life). Nevertheless, I would like to hear the admins' take on this.

\n

While I appreciate the amount of work that Kaggle has put into this contest, I think the rules must have been spelt out more clearly.

", "votes": null, "replies": [] }, { "id": 30948, "author_name": "dmytrolystopad", "author_url": "", "post_date": "09/15/2013 20:56:00", "content": "

[quote=Anil Thomas;30947]

\n

If one were to obtain the weather data for the test dates, it may be possible to run the simulator on the test data. [/quote]

\n

It's possible anyway. The impossible is to predict data which you can just take. In this case we are not blinded. And simulators behavior anyway far from reality for now. So it's not the biggest problem(I have a long list of much more important issues ;) ). In every competition some future information provided - at least leaderboard data. Also in real life routes can be updated at least every hour - in this competition we can't do it. So using this data closer to reality.

", "votes": null, "replies": [] }, { "id": 31010, "author_name": "benhamner", "author_url": "", "post_date": "09/16/2013 20:59:18", "content": "

Quoting the Basic Structure of FQ2 page,

\n

[quote]After the cutoff time, no additional information is available, which mimics real-life situations where the future is not known. Your models must be built using existing information and be robust to the uncertainty of the future.[/quote] You may use wind information or forecasts that were available prior to the cutoff time for each day, but not those that were created after it, in your solution.

", "votes": null, "replies": [] }, { "id": 31068, "author_name": "anlthms", "author_url": "", "post_date": "09/18/2013 03:58:11", "content": "

I made some tweaks (changed the default values for cruise elevation, speed and descend distance and some other minor things) to the benchmark code and made my first submission yesterday. I expected it to do a little better than the baseline. Instead it scored 12766 and ended up 12th! The current leader is just 58 points ahead. Unless someone comes up with an effective routing algorithm within the next week, there is some danger of the milestone prize going to the contestant with the luckiest choice of parameters.

", "votes": null, "replies": [] }, { "id": 31265, "author_name": "anlthms", "author_url": "", "post_date": "09/20/2013 17:07:26", "content": "

[quote=Anil Thomas;31068]

\n

there is some danger of the milestone prize going to the contestant with the luckiest choice of parameters.

\n

[/quote]

\n

I hit a wall going down that path. It doesn't look possible to get below 12760 by optimizing the default cruise altitude, speed and descend distance in the benchmark code. I was wrong when I said luck would be enough to win this.

\n

It turns out that optimizing these parameters on the OneDaySimulatorFiles training data produces more or less optimal settings for the test data as well. Further perturbations of the parameters did not improve the score much.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "29406": "", "29562": "", "29574": "", "29726": "", "29731": "", "29753": "", "29754": "", "29794": "", "29795": "", "29796": "", "29820": "", "29887": "", "29953": "", "29954": "", "30140": "", "30539": "", "30819": "", "30841": "", "30844": "", "30846": "", "30867": "", "30870": "", "30871": "", "30895": "", "30940": "", "30947": "", "30948": "", "31010": "", "31068": "", "31265": "" }, "source": "meta" }