I didn't participate in part one so am probably missing some key information that I can't find in the comp description. I am starting this thread to help get others also in this situation up to speed with exactly what it is we are supposed to be doing.
\nhere goes for starters,
\n1) When I look at the cost function for the route, as far as I can tell it would be best to go as fast as possible in a straight line. What are the added constraints that prevent this from being the best solution. I assume they are things like you can't fly through storms and can't land at the same time as another aircraft. I would expect penalties in the cost function for doing naughty things like this but I don't see any. Can you please give a little more background info or point me in the right direction?
\n\n
2) You refer to a 'Simulator' in various places in the description but I can't seem to locate any info on what this is and what it will be for. Can you please elaborate or point me in the right direction?
\n\n
Cheers ;-)
\n\n
\n
\n
\n
\n
\n
\n
\n
", "messages": [ { "id": "28321", "postDate": "08/06/2013 21:40:17", "content": "
I didn't participate in part one so am probably missing some key information that I can't find in the comp description. I am starting this thread to help get others also in this situation up to speed with exactly what it is we are supposed to be doing.
\nhere goes for starters,
\n1) When I look at the cost function for the route, as far as I can tell it would be best to go as fast as possible in a straight line. What are the added constraints that prevent this from being the best solution. I assume they are things like you can't fly through storms and can't land at the same time as another aircraft. I would expect penalties in the cost function for doing naughty things like this but I don't see any. Can you please give a little more background info or point me in the right direction?
\n\n
2) You refer to a 'Simulator' in various places in the description but I can't seem to locate any info on what this is and what it will be for. Can you please elaborate or point me in the right direction?
\n\n
Cheers ;-)
\n\n
\n
\n
\n
\n
\n
\n
\n
", "rawMarkdown": "", "votes": null }, { "id": "28325", "postDate": "08/06/2013 22:44:29", "content": "
Excellent point! I've added a Basic Structure page to address this. Please post again if you have other questions. Sorry for the oversight.
\n---
\nLink above has been corrected and is now working. The Basic Structure page is also available from the other competition pages.
", "rawMarkdown": "", "votes": null }, { "id": "28326", "postDate": "08/06/2013 22:50:54", "content": "[quote=Sali Mali;28321]
\n1) When I look at the cost function for the route, as far as I can tell it would be best to go as fast as possible in a straight line. What are the added constraints that prevent this from being the best solution. I assume they are things like you can't fly through storms and can't land at the same time as another aircraft. I would expect penalties in the cost function for doing naughty things like this but I don't see any. Can you please give a little more background info or point me in the right direction?
\n[/quote]
\nEach submitted route will be 'flown' through the Flight Simulator. Yes, you could devise a route that flies as fast as possible in a straight line, but that may be very expensive in terms of the fuel burned. Part of the challenge is optimizing fuel efficiency as well as time of flight, among other things.
", "rawMarkdown": "", "votes": null }, { "id": "28327", "postDate": "08/06/2013 22:52:53", "content": "[quote=joycenv;28325]
\nExcellent point! I've added a Basic Structure page to address this. Please post again if you have other questions. Sorry for the oversight.
\n[/quote]
\nHi joycenv, the link is not working.
\nI'd like to know Sali Mali's questions, too.Based on submission instruction, we will probably predict the following items:
\nLatitude, Longitude, Altitude, AirSpeed
\n\n
I imagine the following task for now. Fuel efficiency depends on the speed of aircraft, basically the slower the better, and also the altitude, basically the higher the better. But if it goes higher, it might be hit by jet stream and take more time to fly. Taking weather information into account, we will predict where it should fly and at what speed under the constraint that it arrives on time. But I don't know how GE confirms that proposed flight plan is feasible based on the above items.
", "rawMarkdown": "", "votes": null }, { "id": "28330", "postDate": "08/07/2013 00:32:59", "content": "Thanks for the further info.
\nSo the 'Flight Simulator' is the calculation engine that calculates the 'cost' of each flight, specified by the cost function - and it is just going to be a black box to us that tells us the cost of each flight; or are we supposed to reverse engineer the simulator to figure out the logic it uses to calculate the costs based on weather info etc, so we can develop an optimisation algorithm that can use this info in its decision process?
\nFor example - in a Genetic Algorithm the process would be..
\n1) generate a population of random solutions.
\n2) calculate the cost of each solution by passing the solutions through the simulator
\n3) combine the best solutions using crossover to come up with a new set of candidate solutions
\n4) goto 2 and continue until the population costs converge.
\n5) use the lowest cost solution in the population
\n\n
But I don't see how this could work in practice in this competition. My algorithm could essentially be..
\n1) test all possible solutions and use the best.
\nThat is just a brute force search.
\nSo can you please clarify how the simulator is to be used. Can it be a part of our algorithm? If so where are the penalties for how long it takes to calculate the optimal solution? What stops us from just testing the known universe of possible solutions?
\nIf we cannot use it as an integral part of the algorithm itself then how are we supposed to approach this problem without first reverse engineering it?
\n\n
Hope I'm not being too dumb here :-(
\n\n
\n
\n
\n
", "rawMarkdown": "", "votes": null }, { "id": "28332", "postDate": "08/07/2013 00:35:45", "content": "
[EDIT: I just saw I posted this at the same time as the above post...I basically have the same question]
\nSo it seems the Flight Simulator code is essentially our cost function -- albeit a complicated one.
\nSo that raises all sorts of questions, like the following:
\nI suppose I could wait a week for the release of the simulator to get the answers, but I'm curious now :)
\n", "rawMarkdown": "", "votes": null }, { "id": "28342", "postDate": "08/07/2013 05:51:57", "content": "
[quote=Sali Mali;28321]
\nWhen I look at the cost function for the route, as far as I can tell it would be best to go as fast as possible in a straight line. What are the added constraints that prevent this from being the best solution[/quote]
\nI guess it depends on how 'realistic' the flight simulator is. Since earth is not flat, you'd be flying in an arc, it would take extra fuel & time to gain altitude, but gives you lower air resistance, effects of the wind, rotation of the earth, avoiding nearby aircrafts, the Himalayas, etc.
\nIf the simulator does all this, it could be a contest in itself.
\n\n
", "rawMarkdown": "", "votes": null }, { "id": "28343", "postDate": "08/07/2013 06:54:32", "content": "
Not to mention the fact that ATC is not simulated (if it is, praise to those who wrote a wonderful piece of software): what if ATC put an aircraft on hold a 5000ft because too close to other incoming traffic (what about wake turbolence, is it included?)? It would certainly be more fuel- and time-efficient to fly slightly slower at optimal altitude, say FL340, and avoid the hold right before final.
\nI mean, there are so many variables - and restrictions, mainly due to FAA regulations - in the game that to me it looks more like an optimization problem of the specific simulator, rather than a general optimization problem.
", "rawMarkdown": "", "votes": null }, { "id": "28348", "postDate": "08/07/2013 08:37:04", "content": "Will be full set of simulation parameters of the simulator available for our agents? By other words, need we predict/estimate value of some variables during of optimization or all values will be known?
\nWill be the cost (computed by the simulator) deterministic function of the parameters?
\nIs there constraints on agents runtime, used hardware? If so, what they are?
", "rawMarkdown": "", "votes": null }, { "id": "28363", "postDate": "08/07/2013 13:42:07", "content": "These are all great questions. We'll be sure to include them in a Q&A section on the Basic Structure page (which can be found from the main comp pages) when we're ready to release the Simulator and leaderboard. Thanks for the input and please continue to post.
", "rawMarkdown": "", "votes": null }, { "id": "28365", "postDate": "08/07/2013 13:50:15", "content": "Some silly(?) questions:
\nOk so we are supposed to predict time-serie of following for each flight:
Latitude, Longitude, Altitude, AirSpeed
Few questions that come in my mind:
1) How evaluation formula/simulator penalizes if physics of airplane are not taken into account (smooth tangent of flying path vs ufo teleporting, etc)? The "ufo teleporting" is related to next question 2 (that is what are limits of changing the airspeed between two time points).
2) What is time frame between the time serie (lat,long,alt,airspeed) pairs? If it is fixed say s milliseconds then naturally airspeed between point t and point t+1 cannot change more than physical limits of flying airplane (and weather conditions).
3) If object is to land to say point E then how simulator penalizes solution if flight path does not end at point E. (thinking about "perfect solution" taking weather conditions, like headwind, etc 100% perfectly into account and landing to E versus "bad solution" which ignores weather conditions, and say there is heavy tailwind and lands far off point E - although the algorithm itself may think that "we" (simulator) lands at point E.
[quote=Christopher Hefele;28332]
\n[/quote]
\nIt seems like we'll get the source. I just saw the following quote on the Open Source and IP Terms thread, and the section I underlined below talks about what the Entrants are permitted to do with the source code. Let's hope it's heavily commented :)
[quote=Angus Christophersen;28303]
\nLicense to Code: All code provided to Entrants by GE or Kaggle on behalf of GE is provided to Entrants on a non-exclusive basis for the duration of the Competition and solely for Entrants to participate in the Competition, including rights to read and learn from the source code; compile and execute the source code; and alter the source code locally (including to interact with Entrant’s own code.
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "28378", "postDate": "08/07/2013 15:58:08", "content": "If we get access to the source code, it brings up questions about what, exactly, the goal is (as Sali Mali pointed out earlier).
\nI could spend my time reading the simulator source code and gaming its scoring, or I could spend it deriving a model to predict the most accurate paths.
\nGiven that we will be measured by the simulator, I would guess that understanding its scoring methodology will be the key determinant of performance. Will the "training" simulator differ from the "testing" simulator? And if so, will the differences be critical enough that the leaderboard isn't useful?
\nOther concerns about the simulator:
\n::putting on my former competitor hat and speaking as a neutral 3rd party to this comp::
\nLots of great questions and enthusiasm here, but think folks are getting a bit mired down in the details without even seeing the thing yet. You will get the simulator, you will spend some time with the data, maybe you'll take it out to dinner and share a nice bottle of wine. You'll open up to each other and find that deep and soulful connection you always wanted to feel. It's really easy to criticize the minutiae of any model (no relativistic correction for non-inertial reference frames and no geese simulations?!!! This competition is meaningless!) and it's really hard to write a flight simulator.
", "rawMarkdown": "", "votes": null }, { "id": "28398", "postDate": "08/07/2013 20:36:40", "content": "[quote=William Cukierski;28386]
\nLots of great questions and enthusiasm here, but think folks are getting a bit mired down in the details without even seeing the thing yet.
\n[/quote]
\nThanks Will, but detail is quite important - there have been numerous Kaggle competitions (dare I say the majority) where there has been too much haste in starting the comp only for the competitors to point out data issues and flaws.
\nwrt the simulator - I don't think we need to see it before we are told how we can use it.
\nMy solution is going to be to be an algorithm that tests every possible flight path with the simulator and chooses the best one. I don't see anything in the details we have been given to date that forbids this - so as Vik pointed out above, it is pointless even looking at the data or getting excited about this comp until we know if it is going to be the 'real' best algorithm that wins or just the person who is smart enough to 'game the game'.
\nAs Vik also pointed out - what is the point of 'other' data sets being made available?
\n\n
", "rawMarkdown": "", "votes": null }, { "id": "28409", "postDate": "08/08/2013 04:09:14", "content": "
I agree with Sali Mali that some Kaggle competitions have been very unstable in the description, the data and the rules for a time after they have been launched. I sympathize with the people setting this competition because the flight of a commercial aircraft is a very complex phenomenon. (If this competition wasn't so interesting I would put it away for a couple of weeks and hope things were more stable when I come back!)
\nI fancy this competition is too big for my humble desktop and my fossil languages so I am making the following observation in case it might be useful to someone else. The structure of the flight would seem to be amenable to 'Dynamic Programming'. Further, if the simulator is to be used to evaluate the cost function of answers submitted and if it accepts starting point and destination information and if it is made available for use by competitors then it could be used to evaluate the cost function at intermediate points and, using Dynamic Programming, give an answer that should be the best in the competition.
\nAm I wrong?
", "rawMarkdown": "", "votes": null }, { "id": "28414", "postDate": "08/08/2013 05:25:31", "content": "By any standard, this competition is a disaster. Even Admins don't know what is going on.
\n"but think folks are getting a bit mired down in the details without even seeing the thing yet"
\nReally! Is that your excuse for not being able to answer the questions! If you are not ready, why do you launch it!
\n", "rawMarkdown": "", "votes": null }, { "id": "28434", "postDate": "08/08/2013 14:33:00", "content": "
[quote=Sali Mali;28321]
I didn't participate in part one so am probably missing some key information that I can't find in the comp description. I am starting this thread to help get others also in this situation up to speed with exactly what it is we are supposed to be doing.
[/quote]
\nPhase 2 just launched, you're ahead of everyone and not behind.
\n[quote=Sali Mali;28398]
\nMy solution is going to be to be an algorithm that tests every possible flight path with the simulator and chooses the best one. I don't see anything in the details we have been given to date that forbids this
\n[/quote]
\nother than the sheer computational complexity of doing so?
\nIt has to run and be submitted by the deadline.
\n[quote=Sali Mali;28398]
As Vik also pointed out - what is the point of 'other' data sets being made available?
[/quote]
\nWeather is all part of one large connected system, so the simulator
weather data and the external sources are not independent and may have
some interesting connections.For most purposes, you would probably just want whatever weather data the simulator is using.
That said, you may be able to predict weather features that are "coming up" in the simulation dataset which have not "yet" (at any given simulation time) appeared, based on something like the following:
When X appears in the simulation's weather forecast, it tends to be associated with Y in the external dataset, which then tends to be associated with Z in the later simulation dataset. It's like exposing the middle layer of a neural network to where you can actually read the values, which makes designing algorithms a lot easier.
That being said, I probably won't use external weather data.
Per JC36's post, see you in a few weeks!
I was an active pilot years ago (weekend warrior - not commercial) , but I have seen first-hand what the 'big-guys' have to go through when arriving at an airport: ATC takes a firm hand and can shuffle these planes around based on fuel, weather, create stacked holding patterns, divert planes, handle emergencies, and let VIPS or military cut in line (just to name a few).
\nSurely we are not going to play ATC with this competition. (get into the pattern, turn into downwind leg, descend, turn to base, descend, get on the ILS or other system, and touch wheels-down... just beyond the numbers... on the correct runway).
\nI 'assume' that all we have to do is get the plane to a known 'goal post' and that is it.
\nIt would be nice to know the explicit 'goal post' that we are aiming for, for each flight.
\nI 'assume' that each goal post (or more accurately an imaginary keyhole) is a "Lat/Lon and altitude" somewhere in the sky near the destination airport?
\nDid I miss something in this pile of data? Thanks for in advance...
\n\n
EDIT: if the admins could post a low-scoring yet complete 'submission file', I think a lot of questions would be answered....?
\n", "rawMarkdown": "", "votes": null }, { "id": "28482", "postDate": "08/09/2013 13:57:13", "content": "
[quote=Vik Paruchuri;28378]
\n[/quote]
\nThe NOAA set contains weather information more relevant to the test set, although you don't have flight data for that. See https://www.gequest.com/c/flight2/forums/t/5358/warning-on-weather-training for details.
", "rawMarkdown": "", "votes": null }, { "id": "28727", "postDate": "08/15/2013 12:37:12", "content": "This can be seen as a search problem with a huge search space. Best approach would be to generate a large set of routes (sample routes) using some form of probabilistic distribution sampling of our suggestions (latitude, longitude, altitude, and airspeed) given state signals (use flight_history table to build state signals + additional data like weather). Then you would take the cost function to assign credit to different routes.
\nCan be done using Monte Carlo sampling of the suggestion search space. A good start to bound/seed the search space would be the actual flight tracks flown.
", "rawMarkdown": "", "votes": null }, { "id": "28729", "postDate": "08/15/2013 12:53:54", "content": "Let me moderate myself a bit. "Best approach" should be changed to "A possible approach". Additionally, constructing an effective state signal is not a trivial task.
", "rawMarkdown": "", "votes": null }, { "id": "28737", "postDate": "08/15/2013 14:37:29", "content": "Is this correct?
\nIf you were GE and asked some scientist to model this problem, how would you rank the solutions? Surely you would partition data into two sets: train(public) and test (private). Then, a scientist submits a solution of flight paths. And then what? GE checks that the paths are realistic, that there are no collisions between planes, that no planes go through a hurricane etc. and then calculates the cost function for each flight. Is this the point of the so called "simulator"? Maybe controversy can be spared if it is renamed to "path_validator" ?
", "rawMarkdown": "", "votes": null }, { "id": "28910", "postDate": "08/18/2013 20:36:26", "content": "
I'm sure we'll learn a lot more when we see the simulator code, but while we're waiting, here's my take on the contest.
It's an intelligent search/optimization problem: a routing problem. And it's well studied. Suppose I was hoping that I'd be the first one to ever think of using Ant Colony Optimization. A quick Google of "aircraft flight routing ant" would be sobering.
\nFor each flight, I must find (and submit before the contest ends) a low cost path to the destination airport. A path consists of a list of waypoints (maximum length TBD). A waypoint defines latitude, longitude, altitude, and airspeed. They are like bread-crumbs along the path.
\nTo win, I need to find paths with lower costs, on average, than any of the rest of you do.
\nThe simulator code is the cost function. I don't care (other than curiosity) how realistic it is. No Coriolis effect? Fine, today the Earth stands still. Doesn't use dew point? Fine, who wants to think about humidity in August anyway?
\nAs soon as we get the simulator, I'll be able to score my own solutions and won't have much need for submitting to the leaderboard.
\nThe organizers have given fair warning that they may, and probably will, modify the simulator while the contest is running. Whatever I come up with, I need to be prepared to make 11th hour adjustments. That alone imposes some speed constraints on my algorithm.
\nI see 2 basic approaches: on-line learning and off-line learning.
\nOn-line learning
\nAn example would be naive search. Start with some nominal path and define a reasonable range of perturbations to that path, adding waypoints as needed. Methodically construct and score as many paths as time and my computer resources permit. Submit the best path, for each flight, that I find before the deadline.
\nIt would be better, of course, to use a smarter search algorithm, e.g. Particle Swarm Optimizer. And it would be prudent to use some heuristics too, rather than treat this as a black box.
\nOff-line learning
\nAs an example, I could calculate a lot of features for each flight: distance to airport, angle to airport, time available, weather, flight-specific parameters to the cost function (the simulator), etc.. Then I could train an artificial neural net to output the next waypoint. A path would be formed by repeatedly moving to each waypoint and calling the neural net to create the next one. I could train the weights with a GA.
\n
Correction
So, if I’ve got this right…
\nWhen the simulator runs a flight path on my computer, it will use a truncated weather file that doesn’t have updates for the remainder of the flight, (past the cutoff time). Therefore, the path will be scored as if the weather didn’t change during the remainder of the flight. (This is not to say that it will be spatially uniform.) But when the path is scored on the leaderboard, it will use the full weather file, and the weather probably did change some during the course of the flight.
\nMy locally-calculated score will not be the same as the leaderboard score. So, it looks like I will need to use the leaderboard to measure how well I’m doing.
\nAlso, I’ll need to somehow factor in uncertainty in the weather, when optimizing with the simulator on my computer.
\nPlus, I gather there may be some other confounding factors that haven’t been revealed to us yet.
", "rawMarkdown": "", "votes": null }, { "id": "28937", "postDate": "08/19/2013 09:59:35", "content": "Thanks, glazed, for your thoughts.
\nSurely, the simulator (with or without the source code) can't be the cost function? If so, then your 'on-line learning' approach would surely yield near-perfect results.
\nIf we are at liberty to run the cost function as many times as we like, for an individual flight path, then I expect it to be easy to home in on a global minimum cost. (yes, there might be various local minima, but I seriously doubt these would be difficult to avoid). Then the problem just becomes an exercise in computing power. There would be no need to use weather data, or train any models. Just run the black box cost function with varying flight paths until you get the best answer...
\nAm I misunderstanding?
\nThanks
\nDave
", "rawMarkdown": "", "votes": null }, { "id": "28946", "postDate": "08/19/2013 14:31:35", "content": "[quote=Dave;28937]
\nThanks, glazed, for your thoughts.
\nSurely, the simulator (with or without the source code) can't be the cost function? If so, then your 'on-line learning' approach would surely yield near-perfect results.
\nIf we are at liberty to run the cost function as many times as we like, for an individual flight path, then I expect it to be easy to home in on a global minimum cost. (yes, there might be various local minima, but I seriously doubt these would be difficult to avoid). Then the problem just becomes an exercise in computing power. There would be no need to use weather data, or train any models. Just run the black box cost function with varying flight paths until you get the best answer...
\nAm I misunderstanding?
\nThanks
\nDave
\n[/quote]
\n
Kaggle recently ran a Traveling Santa contest. “All we had to do” was search through the space of all possible permutations of the list of cities and apply a simple cost function to find the best path. Some people have faster computers than others. And yes, size mattered, but there was a lot of technique involved too.
In this contest, the search space of possible paths is also very large. (Theoretically, it could be infinite.) As to how tricky the search will be, we’re just speculating about a piece of software we haven’t seen yet. But since we’re here: I boldly predict that the simulator code will produce a satisfyingly complex fitness landscape…
\n\n
by the end of the contest.
\n", "rawMarkdown": "", "votes": null }, { "id": "28979", "postDate": "08/19/2013 20:41:00", "content": "
[quote=glazed;28946]
\nwe’re just speculating about a piece of software we haven’t seen yet.
\n[/quote]
\nWe're not speculating about anything. All we want to know is..
\n'can the simulator be used as in integral part of the solution'
\nThe answer is either yes or no.
\n", "rawMarkdown": "", "votes": null }, { "id": "28980", "postDate": "08/19/2013 20:50:08", "content": "
Yes, the simulator can be used as an integral part of the solution. We wouldn't be releasing it otherwise.
", "rawMarkdown": "", "votes": null }, { "id": "28984", "postDate": "08/19/2013 21:03:50", "content": "[quote=Ben Hamner;28980]
\nYes, the simulator can be used as an integral part of the solution. We wouldn't be releasing it otherwise.
\n[/quote]
\n\n
Thanks Ben - its taken a while to get that answer ;-)
\nThis is not explicitly stated in the rules and getting the answer to that question was the whole point of this thread. May I suggest you update the rules to clarify.
\nThe next question is then what stops this becoming just a computing power problem? You are actually giving us the means to find the definitive solution given infinite time and resources. To me the cost function should not be based on who can find the best solution, but also how long it takes to find it. There is nothing smart about an algorithm that just does a brute force search.
\n", "rawMarkdown": "", "votes": null }, { "id": "29006", "postDate": "08/20/2013 03:43:40", "content": "
Thanks Ben.
\nCan you confirm that the simulator will provide the *same* cost evaluation function that the scoring system will provide? In other words, we'll have no great need to submit lots of online submission attempts because we will have access to the same functionality locally?
\nThanks!
", "rawMarkdown": "", "votes": null }, { "id": "29082", "postDate": "08/20/2013 17:58:20", "content": "[quote=Dave;29006]
\nCan you confirm that the simulator will provide the *same* cost evaluation function that the scoring system will provide? In other words, we'll have no great need to submit lots of online submission attempts because we will have access to the same functionality locally?
\n[/quote]
\nYes, the Simulator code release will also include the cost function so that you may test locally.
", "rawMarkdown": "", "votes": null }, { "id": "29084", "postDate": "08/20/2013 18:00:18", "content": "[quote=Dave;29006]
\nCan you confirm that the simulator will provide the *same* cost evaluation function that the scoring system will provide? In other words, we'll have no great need to submit lots of online submission attempts because we will have access to the same functionality locally?
\n[/quote]The simulator will include the same cost function. However, it won't include some input data that occurs after the cutoff time on each day that we use to run the simulator on our end (for example, future winds and airport landings).
", "rawMarkdown": "", "votes": null }, { "id": "29104", "postDate": "08/21/2013 00:01:43", "content": "Hi Ben,
\nI am trying to build an Agent that will help each pilot reduce his/her costs and stay on schedule ( I believe that is the core intent of the challenge ... correct me if I am wrong)
\nThe test file drops each flight 'out of the sky' and says you are here, at this altitude and ground speed, and here is your destination, and here are the costs incurred and penalties of not executing well ( and with no current weather data?)
\nIn the context of this challenge, the Agent must figure out where it came from (the Departure airport), what time it is (approximately at best) what probable flight number am I, what kind of probable plane am I, what was my probable flight plan, assume sunny skies, and other sundry probabilistic details.
\nShouldn't an on-board Agent know about these details as facts? Would you as a pilot, trust an on-board Agent that just woke up and then doesn't know the time, the plane, the flight number, and then must comb through thousands of flights of yore to figure this all out... ?
\nPILOT: The Agent rebooted and says:... hmmm .. 'go to warp factor 3...'
\nCo-PILOT: OK Sir, Turning off Agent. .... CLICK
\nI am pretty sure an Agent can figure out the potential Victor-airway, but Jet-airways above the Victor can pretty much self direct these days, with a nod to the VORS ,NAVAIDS, rules of the road, and with the help of ATC and flight plans.
\nYou gave the Agent gigabytes of past history, can you at least give the Agent (via the test data csv) a simple watch?
\nAs always, I could be very wrong about the contents of the massive data provided and the many many unstated intentions of this challenge
\nThanks in advance
", "rawMarkdown": "", "votes": null }, { "id": "29105", "postDate": "08/21/2013 00:11:09", "content": "Hi Karle,
\nThe information about the test flights (origin airport, aircraft type, etc) will be provided with the Simulator code release. The file testFlightsRev2.csv only contains information about the flights needed for calculating the cost function.
", "rawMarkdown": "", "votes": null }, { "id": "29115", "postDate": "08/21/2013 04:37:10", "content": "What is time interval between ordinals of a one flight? Or is this time interval varying and determined by path simulator based on delta coordinate and delta speed?
", "rawMarkdown": "", "votes": null }, { "id": "29160", "postDate": "08/21/2013 16:34:01", "content": "[quote=sfin;29115]
\nWhat is time interval between ordinals of a one flight? Or is this time interval varying and determined by path simulator based on delta coordinate and delta speed?
\n[/quote]
\nYou are correct, there is no fixed time interval between ordinals in the submitted flight route. Time between ordinals will depend on the route, airspeed and wind conditions.
", "rawMarkdown": "", "votes": null }, { "id": "29186", "postDate": "08/21/2013 19:47:47", "content": "[quote=joycenv;29160]
\n[quote=sfin;29115]
\nWhat is time interval between ordinals of a one flight? Or is this time interval varying and determined by path simulator based on delta coordinate and delta speed?
\n[/quote]
\nYou are correct, there is no fixed time interval between ordinals in the submitted flight route. Time between ordinals will depend on the route, airspeed and wind conditions.
\n[/quote]
\nSo how I am penalized (by cost function) if at interval 0 I am at point say p with velocity v and at interval 1 I am at point p+epsilon (consider epsilon vector almost 0,0 longitude,latitude change) and velocity v1 such that v1 >> v. So my coordinates say that I have moved "epsilon" (almost zero time) but velocity v1 says that I must have moved more than zero time. Thus basicalyl this is case of "breaking physics" (accelerating plane's speed from v to v1 in "almost zero time").
\n", "rawMarkdown": "", "votes": null }, { "id": "29544", "postDate": "08/27/2013 05:29:32", "content": "
Hi Mali
\n\n
I too think that the solution should be an algorithm that chooses the best path. But little confused about how should we define 'Best'. Is it that takes less time and thus consumes less fuel?
", "rawMarkdown": "", "votes": null }, { "id": "29545", "postDate": "08/27/2013 05:35:18", "content": "[quote=Sali Mali;28398]
\n[quote=William Cukierski;28386]
\nLots of great questions and enthusiasm here, but think folks are getting a bit mired down in the details without even seeing the thing yet.
\n[/quote]
\nThanks Will, but detail is quite important - there have been numerous Kaggle competitions (dare I say the majority) where there has been too much haste in starting the comp only for the competitors to point out data issues and flaws.
\nwrt the simulator - I don't think we need to see it before we are told how we can use it.
\nMy solution is going to be to be an algorithm that tests every possible flight path with the simulator and chooses the best one. I don't see anything in the details we have been given to date that forbids this - so as Vik pointed out above, it is pointless even looking at the data or getting excited about this comp until we know if it is going to be the 'real' best algorithm that wins or just the person who is smart enough to 'game the game'.
\nAs Vik also pointed out - what is the point of 'other' data sets being made available?
\n\n
\n
[/quote]
\n\n
Hi Mali
\n\n
I too think that the solution should be an algorithm that chooses the best path. But little confused about how should we define 'Best'. Is it that takes less time and thus consumes less fuel?
", "rawMarkdown": "", "votes": null }, { "id": "29639", "postDate": "08/28/2013 13:02:40", "content": "It's the lowest cost function from the simulator.
\nThe solution that is "an algorithm that tests every possible flight path with the simulator and chooses the best one" might run into computational complexity challenges.
\nRegarding the point of other data sets, see https://www.gequest.com/c/flight2/forums/t/5358/warning-on-weather-training as one resource. There is a definite danger of overfitting on the provided training set(s). I think there may also be another thread on this topic, but can't find it at the moment.
\n", "rawMarkdown": "", "votes": null }, { "id": "29646", "postDate": "08/28/2013 14:35:23", "content": "
[quote=WBTtheFROG;29639]
\nIt's the lowest cost function from the simulator.
\nThe solution that is "an algorithm that tests every possible flight path with the simulator and chooses the best one" might run into computational complexity challenges.
\nRegarding the point of other data sets, see https://www.gequest.com/c/flight2/forums/t/5358/warning-on-weather-training as one resource. There is a definite danger of overfitting on the provided training set(s). I think there may also be another thread on this topic, but can't find it at the moment.
\n\n
[/quote]
\nThanks for the reply.
The cost as described in the evaluation section is a function of the fuel consumed and the time taken.
Fuel consumption depends upon the aircraft speed and the distance covered.
Regarding the aircraft speed,
1) There has to an optimum speed recommended for aircraft by manufacturers at which the fuel consumption is minimum (we do not know).
2) Maximum permissible speed due to altitude and weather condition. Can we determine this using the historical flight and weather data? Or is there any ideal speed for all altitudes & weather.
Without knowing the maximum speed that an aircraft can go up to , is it not difficult to say which is the best path ?
", "rawMarkdown": "", "votes": null }, { "id": "30126", "postDate": "09/04/2013 03:21:44", "content": "
UNITS Please...
\nThe test data has a ground-speed column ( knots, mph?)
\nThe submission file wants airspeed in knots (which is variable due to altitude, temperature and pressure, % Mach is what most airline pilots use above FL30 ...) Or is this just our old friend ground-speed again, except that this time it is explicitly in knots
\nThe data dictionary is of no cohesive help on units
\nWould it be too much to ask for each column in the data files to have a name with a "-units" added?:
\nExamples:
\nit would make conversions (and wrong assumptions) obvious
", "rawMarkdown": "", "votes": null }, { "id": "30135", "postDate": "09/04/2013 07:05:48", "content": "Has anyone noticed there are 89 flights (rows) in 'testFlightsRev3.csv' with zero ground speed -- all above 21000 ft altitude.
\nAlso, the flight with FlightHistoryId = 303137309 has no history record in 'test1_flighthistory.csv'.
\nHave these two issues been discussed anywhere?
", "rawMarkdown": "", "votes": null }, { "id": "30152", "postDate": "09/04/2013 14:39:28", "content": "[quote=JC36;30135]
\nHas anyone noticed there are 89 flights (rows) in 'testFlightsRev3.csv' with zero ground speed -- all above 21000 ft altitude.
\n[/quote]
\n\n
Yes, noticed it just now. Looks like an error to me.
", "rawMarkdown": "", "votes": null }, { "id": "30172", "postDate": "09/04/2013 17:16:10", "content": "I've looked over the information on this web site, downloaded all the available data, (including the simulator), Got the simulator running on my system and ran a test of it. Here is my understanding of the task - please tell me if I've missed something.
\nSooooo, My understanding of how to realistically go about this is:
\nOf course, the above points are simply different ways of saying the same thing.
\nAgain, my real question is: Do I understand this correctly?
", "rawMarkdown": "", "votes": null }, { "id": "302110", "postDate": "03/23/2018 17:45:13", "content": "Hey, I've lived in Ukraine
", "rawMarkdown": "Hey, I've lived in Ukraine", "votes": null } ], "comments": [ { "id": 28325, "author_name": "joycenv", "author_url": "", "post_date": "08/06/2013 22:44:29", "content": "Excellent point! I've added a Basic Structure page to address this. Please post again if you have other questions. Sorry for the oversight.
\n---
\nLink above has been corrected and is now working. The Basic Structure page is also available from the other competition pages.
", "votes": null, "replies": [] }, { "id": 28326, "author_name": "joycenv", "author_url": "", "post_date": "08/06/2013 22:50:54", "content": "[quote=Sali Mali;28321]
\n1) When I look at the cost function for the route, as far as I can tell it would be best to go as fast as possible in a straight line. What are the added constraints that prevent this from being the best solution. I assume they are things like you can't fly through storms and can't land at the same time as another aircraft. I would expect penalties in the cost function for doing naughty things like this but I don't see any. Can you please give a little more background info or point me in the right direction?
\n[/quote]
\nEach submitted route will be 'flown' through the Flight Simulator. Yes, you could devise a route that flies as fast as possible in a straight line, but that may be very expensive in terms of the fuel burned. Part of the challenge is optimizing fuel efficiency as well as time of flight, among other things.
", "votes": null, "replies": [] }, { "id": 28327, "author_name": "naokazumizuta", "author_url": "", "post_date": "08/06/2013 22:52:53", "content": "[quote=joycenv;28325]
\nExcellent point! I've added a Basic Structure page to address this. Please post again if you have other questions. Sorry for the oversight.
\n[/quote]
\nHi joycenv, the link is not working.
\nI'd like to know Sali Mali's questions, too.Based on submission instruction, we will probably predict the following items:
\nLatitude, Longitude, Altitude, AirSpeed
\n\n
I imagine the following task for now. Fuel efficiency depends on the speed of aircraft, basically the slower the better, and also the altitude, basically the higher the better. But if it goes higher, it might be hit by jet stream and take more time to fly. Taking weather information into account, we will predict where it should fly and at what speed under the constraint that it arrives on time. But I don't know how GE confirms that proposed flight plan is feasible based on the above items.
", "votes": null, "replies": [] }, { "id": 28330, "author_name": "salimali", "author_url": "", "post_date": "08/07/2013 00:32:59", "content": "Thanks for the further info.
\nSo the 'Flight Simulator' is the calculation engine that calculates the 'cost' of each flight, specified by the cost function - and it is just going to be a black box to us that tells us the cost of each flight; or are we supposed to reverse engineer the simulator to figure out the logic it uses to calculate the costs based on weather info etc, so we can develop an optimisation algorithm that can use this info in its decision process?
\nFor example - in a Genetic Algorithm the process would be..
\n1) generate a population of random solutions.
\n2) calculate the cost of each solution by passing the solutions through the simulator
\n3) combine the best solutions using crossover to come up with a new set of candidate solutions
\n4) goto 2 and continue until the population costs converge.
\n5) use the lowest cost solution in the population
\n\n
But I don't see how this could work in practice in this competition. My algorithm could essentially be..
\n1) test all possible solutions and use the best.
\nThat is just a brute force search.
\nSo can you please clarify how the simulator is to be used. Can it be a part of our algorithm? If so where are the penalties for how long it takes to calculate the optimal solution? What stops us from just testing the known universe of possible solutions?
\nIf we cannot use it as an integral part of the algorithm itself then how are we supposed to approach this problem without first reverse engineering it?
\n\n
Hope I'm not being too dumb here :-(
\n\n
\n
\n
\n
", "votes": null, "replies": [] }, { "id": 28332, "author_name": "chefele", "author_url": "", "post_date": "08/07/2013 00:35:45", "content": "
[EDIT: I just saw I posted this at the same time as the above post...I basically have the same question]
\nSo it seems the Flight Simulator code is essentially our cost function -- albeit a complicated one.
\nSo that raises all sorts of questions, like the following:
\nI suppose I could wait a week for the release of the simulator to get the answers, but I'm curious now :)
\n", "votes": null, "replies": [] }, { "id": 28342, "author_name": "byang1", "author_url": "", "post_date": "08/07/2013 05:51:57", "content": "
[quote=Sali Mali;28321]
\nWhen I look at the cost function for the route, as far as I can tell it would be best to go as fast as possible in a straight line. What are the added constraints that prevent this from being the best solution[/quote]
\nI guess it depends on how 'realistic' the flight simulator is. Since earth is not flat, you'd be flying in an arc, it would take extra fuel & time to gain altitude, but gives you lower air resistance, effects of the wind, rotation of the earth, avoiding nearby aircrafts, the Himalayas, etc.
\nIf the simulator does all this, it could be a contest in itself.
\n\n
", "votes": null, "replies": [] }, { "id": 28343, "author_name": "gabrieleseppi", "author_url": "", "post_date": "08/07/2013 06:54:32", "content": "
Not to mention the fact that ATC is not simulated (if it is, praise to those who wrote a wonderful piece of software): what if ATC put an aircraft on hold a 5000ft because too close to other incoming traffic (what about wake turbolence, is it included?)? It would certainly be more fuel- and time-efficient to fly slightly slower at optimal altitude, say FL340, and avoid the hold right before final.
\nI mean, there are so many variables - and restrictions, mainly due to FAA regulations - in the game that to me it looks more like an optimization problem of the specific simulator, rather than a general optimization problem.
", "votes": null, "replies": [] }, { "id": 28348, "author_name": "alegro", "author_url": "", "post_date": "08/07/2013 08:37:04", "content": "Will be full set of simulation parameters of the simulator available for our agents? By other words, need we predict/estimate value of some variables during of optimization or all values will be known?
\nWill be the cost (computed by the simulator) deterministic function of the parameters?
\nIs there constraints on agents runtime, used hardware? If so, what they are?
", "votes": null, "replies": [ { "id": 302110, "author_name": "zakapar", "author_url": "", "post_date": "03/23/2018 17:45:13", "content": "Hey, I've lived in Ukraine
", "votes": null, "replies": [] } ] }, { "id": 28363, "author_name": "joycenv", "author_url": "", "post_date": "08/07/2013 13:42:07", "content": "These are all great questions. We'll be sure to include them in a Q&A section on the Basic Structure page (which can be found from the main comp pages) when we're ready to release the Simulator and leaderboard. Thanks for the input and please continue to post.
", "votes": null, "replies": [] }, { "id": 28365, "author_name": "sfin2015", "author_url": "", "post_date": "08/07/2013 13:50:15", "content": "Some silly(?) questions:
\nOk so we are supposed to predict time-serie of following for each flight:
Latitude, Longitude, Altitude, AirSpeed
Few questions that come in my mind:
1) How evaluation formula/simulator penalizes if physics of airplane are not taken into account (smooth tangent of flying path vs ufo teleporting, etc)? The "ufo teleporting" is related to next question 2 (that is what are limits of changing the airspeed between two time points).
2) What is time frame between the time serie (lat,long,alt,airspeed) pairs? If it is fixed say s milliseconds then naturally airspeed between point t and point t+1 cannot change more than physical limits of flying airplane (and weather conditions).
3) If object is to land to say point E then how simulator penalizes solution if flight path does not end at point E. (thinking about "perfect solution" taking weather conditions, like headwind, etc 100% perfectly into account and landing to E versus "bad solution" which ignores weather conditions, and say there is heavy tailwind and lands far off point E - although the algorithm itself may think that "we" (simulator) lands at point E.
[quote=Christopher Hefele;28332]
\n[/quote]
\nIt seems like we'll get the source. I just saw the following quote on the Open Source and IP Terms thread, and the section I underlined below talks about what the Entrants are permitted to do with the source code. Let's hope it's heavily commented :)
[quote=Angus Christophersen;28303]
\nLicense to Code: All code provided to Entrants by GE or Kaggle on behalf of GE is provided to Entrants on a non-exclusive basis for the duration of the Competition and solely for Entrants to participate in the Competition, including rights to read and learn from the source code; compile and execute the source code; and alter the source code locally (including to interact with Entrant’s own code.
\n[/quote]
", "votes": null, "replies": [] }, { "id": 28378, "author_name": "vikpar", "author_url": "", "post_date": "08/07/2013 15:58:08", "content": "If we get access to the source code, it brings up questions about what, exactly, the goal is (as Sali Mali pointed out earlier).
\nI could spend my time reading the simulator source code and gaming its scoring, or I could spend it deriving a model to predict the most accurate paths.
\nGiven that we will be measured by the simulator, I would guess that understanding its scoring methodology will be the key determinant of performance. Will the "training" simulator differ from the "testing" simulator? And if so, will the differences be critical enough that the leaderboard isn't useful?
\nOther concerns about the simulator:
\n::putting on my former competitor hat and speaking as a neutral 3rd party to this comp::
\nLots of great questions and enthusiasm here, but think folks are getting a bit mired down in the details without even seeing the thing yet. You will get the simulator, you will spend some time with the data, maybe you'll take it out to dinner and share a nice bottle of wine. You'll open up to each other and find that deep and soulful connection you always wanted to feel. It's really easy to criticize the minutiae of any model (no relativistic correction for non-inertial reference frames and no geese simulations?!!! This competition is meaningless!) and it's really hard to write a flight simulator.
", "votes": null, "replies": [] }, { "id": 28398, "author_name": "salimali", "author_url": "", "post_date": "08/07/2013 20:36:40", "content": "[quote=William Cukierski;28386]
\nLots of great questions and enthusiasm here, but think folks are getting a bit mired down in the details without even seeing the thing yet.
\n[/quote]
\nThanks Will, but detail is quite important - there have been numerous Kaggle competitions (dare I say the majority) where there has been too much haste in starting the comp only for the competitors to point out data issues and flaws.
\nwrt the simulator - I don't think we need to see it before we are told how we can use it.
\nMy solution is going to be to be an algorithm that tests every possible flight path with the simulator and chooses the best one. I don't see anything in the details we have been given to date that forbids this - so as Vik pointed out above, it is pointless even looking at the data or getting excited about this comp until we know if it is going to be the 'real' best algorithm that wins or just the person who is smart enough to 'game the game'.
\nAs Vik also pointed out - what is the point of 'other' data sets being made available?
\n\n
", "votes": null, "replies": [] }, { "id": 28409, "author_name": "jc3626439", "author_url": "", "post_date": "08/08/2013 04:09:14", "content": "
I agree with Sali Mali that some Kaggle competitions have been very unstable in the description, the data and the rules for a time after they have been launched. I sympathize with the people setting this competition because the flight of a commercial aircraft is a very complex phenomenon. (If this competition wasn't so interesting I would put it away for a couple of weeks and hope things were more stable when I come back!)
\nI fancy this competition is too big for my humble desktop and my fossil languages so I am making the following observation in case it might be useful to someone else. The structure of the flight would seem to be amenable to 'Dynamic Programming'. Further, if the simulator is to be used to evaluate the cost function of answers submitted and if it accepts starting point and destination information and if it is made available for use by competitors then it could be used to evaluate the cost function at intermediate points and, using Dynamic Programming, give an answer that should be the best in the competition.
\nAm I wrong?
", "votes": null, "replies": [] }, { "id": 28414, "author_name": "smallbang", "author_url": "", "post_date": "08/08/2013 05:25:31", "content": "By any standard, this competition is a disaster. Even Admins don't know what is going on.
\n"but think folks are getting a bit mired down in the details without even seeing the thing yet"
\nReally! Is that your excuse for not being able to answer the questions! If you are not ready, why do you launch it!
\n", "votes": null, "replies": [] }, { "id": 28434, "author_name": "wbtthefrog", "author_url": "", "post_date": "08/08/2013 14:33:00", "content": "
[quote=Sali Mali;28321]
I didn't participate in part one so am probably missing some key information that I can't find in the comp description. I am starting this thread to help get others also in this situation up to speed with exactly what it is we are supposed to be doing.
[/quote]
\nPhase 2 just launched, you're ahead of everyone and not behind.
\n[quote=Sali Mali;28398]
\nMy solution is going to be to be an algorithm that tests every possible flight path with the simulator and chooses the best one. I don't see anything in the details we have been given to date that forbids this
\n[/quote]
\nother than the sheer computational complexity of doing so?
\nIt has to run and be submitted by the deadline.
\n[quote=Sali Mali;28398]
As Vik also pointed out - what is the point of 'other' data sets being made available?
[/quote]
\nWeather is all part of one large connected system, so the simulator
weather data and the external sources are not independent and may have
some interesting connections.For most purposes, you would probably just want whatever weather data the simulator is using.
That said, you may be able to predict weather features that are "coming up" in the simulation dataset which have not "yet" (at any given simulation time) appeared, based on something like the following:
When X appears in the simulation's weather forecast, it tends to be associated with Y in the external dataset, which then tends to be associated with Z in the later simulation dataset. It's like exposing the middle layer of a neural network to where you can actually read the values, which makes designing algorithms a lot easier.
That being said, I probably won't use external weather data.
Per JC36's post, see you in a few weeks!
I was an active pilot years ago (weekend warrior - not commercial) , but I have seen first-hand what the 'big-guys' have to go through when arriving at an airport: ATC takes a firm hand and can shuffle these planes around based on fuel, weather, create stacked holding patterns, divert planes, handle emergencies, and let VIPS or military cut in line (just to name a few).
\nSurely we are not going to play ATC with this competition. (get into the pattern, turn into downwind leg, descend, turn to base, descend, get on the ILS or other system, and touch wheels-down... just beyond the numbers... on the correct runway).
\nI 'assume' that all we have to do is get the plane to a known 'goal post' and that is it.
\nIt would be nice to know the explicit 'goal post' that we are aiming for, for each flight.
\nI 'assume' that each goal post (or more accurately an imaginary keyhole) is a "Lat/Lon and altitude" somewhere in the sky near the destination airport?
\nDid I miss something in this pile of data? Thanks for in advance...
\n\n
EDIT: if the admins could post a low-scoring yet complete 'submission file', I think a lot of questions would be answered....?
\n", "votes": null, "replies": [] }, { "id": 28482, "author_name": "wbtthefrog", "author_url": "", "post_date": "08/09/2013 13:57:13", "content": "
[quote=Vik Paruchuri;28378]
\n[/quote]
\nThe NOAA set contains weather information more relevant to the test set, although you don't have flight data for that. See https://www.gequest.com/c/flight2/forums/t/5358/warning-on-weather-training for details.
", "votes": null, "replies": [] }, { "id": 28727, "author_name": "niclasko", "author_url": "", "post_date": "08/15/2013 12:37:12", "content": "This can be seen as a search problem with a huge search space. Best approach would be to generate a large set of routes (sample routes) using some form of probabilistic distribution sampling of our suggestions (latitude, longitude, altitude, and airspeed) given state signals (use flight_history table to build state signals + additional data like weather). Then you would take the cost function to assign credit to different routes.
\nCan be done using Monte Carlo sampling of the suggestion search space. A good start to bound/seed the search space would be the actual flight tracks flown.
", "votes": null, "replies": [] }, { "id": 28729, "author_name": "niclasko", "author_url": "", "post_date": "08/15/2013 12:53:54", "content": "Let me moderate myself a bit. "Best approach" should be changed to "A possible approach". Additionally, constructing an effective state signal is not a trivial task.
", "votes": null, "replies": [] }, { "id": 28737, "author_name": "harriken", "author_url": "", "post_date": "08/15/2013 14:37:29", "content": "Is this correct?
\nIf you were GE and asked some scientist to model this problem, how would you rank the solutions? Surely you would partition data into two sets: train(public) and test (private). Then, a scientist submits a solution of flight paths. And then what? GE checks that the paths are realistic, that there are no collisions between planes, that no planes go through a hurricane etc. and then calculates the cost function for each flight. Is this the point of the so called "simulator"? Maybe controversy can be spared if it is renamed to "path_validator" ?
", "votes": null, "replies": [] }, { "id": 28910, "author_name": "glazed", "author_url": "", "post_date": "08/18/2013 20:36:26", "content": "
I'm sure we'll learn a lot more when we see the simulator code, but while we're waiting, here's my take on the contest.
It's an intelligent search/optimization problem: a routing problem. And it's well studied. Suppose I was hoping that I'd be the first one to ever think of using Ant Colony Optimization. A quick Google of "aircraft flight routing ant" would be sobering.
\nFor each flight, I must find (and submit before the contest ends) a low cost path to the destination airport. A path consists of a list of waypoints (maximum length TBD). A waypoint defines latitude, longitude, altitude, and airspeed. They are like bread-crumbs along the path.
\nTo win, I need to find paths with lower costs, on average, than any of the rest of you do.
\nThe simulator code is the cost function. I don't care (other than curiosity) how realistic it is. No Coriolis effect? Fine, today the Earth stands still. Doesn't use dew point? Fine, who wants to think about humidity in August anyway?
\nAs soon as we get the simulator, I'll be able to score my own solutions and won't have much need for submitting to the leaderboard.
\nThe organizers have given fair warning that they may, and probably will, modify the simulator while the contest is running. Whatever I come up with, I need to be prepared to make 11th hour adjustments. That alone imposes some speed constraints on my algorithm.
\nI see 2 basic approaches: on-line learning and off-line learning.
\nOn-line learning
\nAn example would be naive search. Start with some nominal path and define a reasonable range of perturbations to that path, adding waypoints as needed. Methodically construct and score as many paths as time and my computer resources permit. Submit the best path, for each flight, that I find before the deadline.
\nIt would be better, of course, to use a smarter search algorithm, e.g. Particle Swarm Optimizer. And it would be prudent to use some heuristics too, rather than treat this as a black box.
\nOff-line learning
\nAs an example, I could calculate a lot of features for each flight: distance to airport, angle to airport, time available, weather, flight-specific parameters to the cost function (the simulator), etc.. Then I could train an artificial neural net to output the next waypoint. A path would be formed by repeatedly moving to each waypoint and calling the neural net to create the next one. I could train the weights with a GA.
\n
Correction
So, if I’ve got this right…
\nWhen the simulator runs a flight path on my computer, it will use a truncated weather file that doesn’t have updates for the remainder of the flight, (past the cutoff time). Therefore, the path will be scored as if the weather didn’t change during the remainder of the flight. (This is not to say that it will be spatially uniform.) But when the path is scored on the leaderboard, it will use the full weather file, and the weather probably did change some during the course of the flight.
\nMy locally-calculated score will not be the same as the leaderboard score. So, it looks like I will need to use the leaderboard to measure how well I’m doing.
\nAlso, I’ll need to somehow factor in uncertainty in the weather, when optimizing with the simulator on my computer.
\nPlus, I gather there may be some other confounding factors that haven’t been revealed to us yet.
", "votes": null, "replies": [] }, { "id": 28937, "author_name": "dave40", "author_url": "", "post_date": "08/19/2013 09:59:35", "content": "Thanks, glazed, for your thoughts.
\nSurely, the simulator (with or without the source code) can't be the cost function? If so, then your 'on-line learning' approach would surely yield near-perfect results.
\nIf we are at liberty to run the cost function as many times as we like, for an individual flight path, then I expect it to be easy to home in on a global minimum cost. (yes, there might be various local minima, but I seriously doubt these would be difficult to avoid). Then the problem just becomes an exercise in computing power. There would be no need to use weather data, or train any models. Just run the black box cost function with varying flight paths until you get the best answer...
\nAm I misunderstanding?
\nThanks
\nDave
", "votes": null, "replies": [] }, { "id": 28946, "author_name": "glazed", "author_url": "", "post_date": "08/19/2013 14:31:35", "content": "[quote=Dave;28937]
\nThanks, glazed, for your thoughts.
\nSurely, the simulator (with or without the source code) can't be the cost function? If so, then your 'on-line learning' approach would surely yield near-perfect results.
\nIf we are at liberty to run the cost function as many times as we like, for an individual flight path, then I expect it to be easy to home in on a global minimum cost. (yes, there might be various local minima, but I seriously doubt these would be difficult to avoid). Then the problem just becomes an exercise in computing power. There would be no need to use weather data, or train any models. Just run the black box cost function with varying flight paths until you get the best answer...
\nAm I misunderstanding?
\nThanks
\nDave
\n[/quote]
\n
Kaggle recently ran a Traveling Santa contest. “All we had to do” was search through the space of all possible permutations of the list of cities and apply a simple cost function to find the best path. Some people have faster computers than others. And yes, size mattered, but there was a lot of technique involved too.
In this contest, the search space of possible paths is also very large. (Theoretically, it could be infinite.) As to how tricky the search will be, we’re just speculating about a piece of software we haven’t seen yet. But since we’re here: I boldly predict that the simulator code will produce a satisfyingly complex fitness landscape…
\n\n
by the end of the contest.
\n", "votes": null, "replies": [] }, { "id": 28979, "author_name": "salimali", "author_url": "", "post_date": "08/19/2013 20:41:00", "content": "
[quote=glazed;28946]
\nwe’re just speculating about a piece of software we haven’t seen yet.
\n[/quote]
\nWe're not speculating about anything. All we want to know is..
\n'can the simulator be used as in integral part of the solution'
\nThe answer is either yes or no.
\n", "votes": null, "replies": [] }, { "id": 28980, "author_name": "benhamner", "author_url": "", "post_date": "08/19/2013 20:50:08", "content": "
Yes, the simulator can be used as an integral part of the solution. We wouldn't be releasing it otherwise.
", "votes": null, "replies": [] }, { "id": 28984, "author_name": "salimali", "author_url": "", "post_date": "08/19/2013 21:03:50", "content": "[quote=Ben Hamner;28980]
\nYes, the simulator can be used as an integral part of the solution. We wouldn't be releasing it otherwise.
\n[/quote]
\n\n
Thanks Ben - its taken a while to get that answer ;-)
\nThis is not explicitly stated in the rules and getting the answer to that question was the whole point of this thread. May I suggest you update the rules to clarify.
\nThe next question is then what stops this becoming just a computing power problem? You are actually giving us the means to find the definitive solution given infinite time and resources. To me the cost function should not be based on who can find the best solution, but also how long it takes to find it. There is nothing smart about an algorithm that just does a brute force search.
\n", "votes": null, "replies": [] }, { "id": 29006, "author_name": "dave40", "author_url": "", "post_date": "08/20/2013 03:43:40", "content": "
Thanks Ben.
\nCan you confirm that the simulator will provide the *same* cost evaluation function that the scoring system will provide? In other words, we'll have no great need to submit lots of online submission attempts because we will have access to the same functionality locally?
\nThanks!
", "votes": null, "replies": [] }, { "id": 29082, "author_name": "joycenv", "author_url": "", "post_date": "08/20/2013 17:58:20", "content": "[quote=Dave;29006]
\nCan you confirm that the simulator will provide the *same* cost evaluation function that the scoring system will provide? In other words, we'll have no great need to submit lots of online submission attempts because we will have access to the same functionality locally?
\n[/quote]
\nYes, the Simulator code release will also include the cost function so that you may test locally.
", "votes": null, "replies": [] }, { "id": 29084, "author_name": "benhamner", "author_url": "", "post_date": "08/20/2013 18:00:18", "content": "[quote=Dave;29006]
\nCan you confirm that the simulator will provide the *same* cost evaluation function that the scoring system will provide? In other words, we'll have no great need to submit lots of online submission attempts because we will have access to the same functionality locally?
\n[/quote]The simulator will include the same cost function. However, it won't include some input data that occurs after the cutoff time on each day that we use to run the simulator on our end (for example, future winds and airport landings).
", "votes": null, "replies": [] }, { "id": 29104, "author_name": "karle1", "author_url": "", "post_date": "08/21/2013 00:01:43", "content": "Hi Ben,
\nI am trying to build an Agent that will help each pilot reduce his/her costs and stay on schedule ( I believe that is the core intent of the challenge ... correct me if I am wrong)
\nThe test file drops each flight 'out of the sky' and says you are here, at this altitude and ground speed, and here is your destination, and here are the costs incurred and penalties of not executing well ( and with no current weather data?)
\nIn the context of this challenge, the Agent must figure out where it came from (the Departure airport), what time it is (approximately at best) what probable flight number am I, what kind of probable plane am I, what was my probable flight plan, assume sunny skies, and other sundry probabilistic details.
\nShouldn't an on-board Agent know about these details as facts? Would you as a pilot, trust an on-board Agent that just woke up and then doesn't know the time, the plane, the flight number, and then must comb through thousands of flights of yore to figure this all out... ?
\nPILOT: The Agent rebooted and says:... hmmm .. 'go to warp factor 3...'
\nCo-PILOT: OK Sir, Turning off Agent. .... CLICK
\nI am pretty sure an Agent can figure out the potential Victor-airway, but Jet-airways above the Victor can pretty much self direct these days, with a nod to the VORS ,NAVAIDS, rules of the road, and with the help of ATC and flight plans.
\nYou gave the Agent gigabytes of past history, can you at least give the Agent (via the test data csv) a simple watch?
\nAs always, I could be very wrong about the contents of the massive data provided and the many many unstated intentions of this challenge
\nThanks in advance
", "votes": null, "replies": [] }, { "id": 29105, "author_name": "joycenv", "author_url": "", "post_date": "08/21/2013 00:11:09", "content": "Hi Karle,
\nThe information about the test flights (origin airport, aircraft type, etc) will be provided with the Simulator code release. The file testFlightsRev2.csv only contains information about the flights needed for calculating the cost function.
", "votes": null, "replies": [] }, { "id": 29115, "author_name": "sfin2015", "author_url": "", "post_date": "08/21/2013 04:37:10", "content": "What is time interval between ordinals of a one flight? Or is this time interval varying and determined by path simulator based on delta coordinate and delta speed?
", "votes": null, "replies": [] }, { "id": 29160, "author_name": "joycenv", "author_url": "", "post_date": "08/21/2013 16:34:01", "content": "[quote=sfin;29115]
\nWhat is time interval between ordinals of a one flight? Or is this time interval varying and determined by path simulator based on delta coordinate and delta speed?
\n[/quote]
\nYou are correct, there is no fixed time interval between ordinals in the submitted flight route. Time between ordinals will depend on the route, airspeed and wind conditions.
", "votes": null, "replies": [] }, { "id": 29186, "author_name": "sfin2015", "author_url": "", "post_date": "08/21/2013 19:47:47", "content": "[quote=joycenv;29160]
\n[quote=sfin;29115]
\nWhat is time interval between ordinals of a one flight? Or is this time interval varying and determined by path simulator based on delta coordinate and delta speed?
\n[/quote]
\nYou are correct, there is no fixed time interval between ordinals in the submitted flight route. Time between ordinals will depend on the route, airspeed and wind conditions.
\n[/quote]
\nSo how I am penalized (by cost function) if at interval 0 I am at point say p with velocity v and at interval 1 I am at point p+epsilon (consider epsilon vector almost 0,0 longitude,latitude change) and velocity v1 such that v1 >> v. So my coordinates say that I have moved "epsilon" (almost zero time) but velocity v1 says that I must have moved more than zero time. Thus basicalyl this is case of "breaking physics" (accelerating plane's speed from v to v1 in "almost zero time").
\n", "votes": null, "replies": [] }, { "id": 29544, "author_name": "vikasbhargava", "author_url": "", "post_date": "08/27/2013 05:29:32", "content": "
Hi Mali
\n\n
I too think that the solution should be an algorithm that chooses the best path. But little confused about how should we define 'Best'. Is it that takes less time and thus consumes less fuel?
", "votes": null, "replies": [] }, { "id": 29545, "author_name": "vikasbhargava", "author_url": "", "post_date": "08/27/2013 05:35:18", "content": "[quote=Sali Mali;28398]
\n[quote=William Cukierski;28386]
\nLots of great questions and enthusiasm here, but think folks are getting a bit mired down in the details without even seeing the thing yet.
\n[/quote]
\nThanks Will, but detail is quite important - there have been numerous Kaggle competitions (dare I say the majority) where there has been too much haste in starting the comp only for the competitors to point out data issues and flaws.
\nwrt the simulator - I don't think we need to see it before we are told how we can use it.
\nMy solution is going to be to be an algorithm that tests every possible flight path with the simulator and chooses the best one. I don't see anything in the details we have been given to date that forbids this - so as Vik pointed out above, it is pointless even looking at the data or getting excited about this comp until we know if it is going to be the 'real' best algorithm that wins or just the person who is smart enough to 'game the game'.
\nAs Vik also pointed out - what is the point of 'other' data sets being made available?
\n\n
\n
[/quote]
\n\n
Hi Mali
\n\n
I too think that the solution should be an algorithm that chooses the best path. But little confused about how should we define 'Best'. Is it that takes less time and thus consumes less fuel?
", "votes": null, "replies": [] }, { "id": 29639, "author_name": "wbtthefrog", "author_url": "", "post_date": "08/28/2013 13:02:40", "content": "It's the lowest cost function from the simulator.
\nThe solution that is "an algorithm that tests every possible flight path with the simulator and chooses the best one" might run into computational complexity challenges.
\nRegarding the point of other data sets, see https://www.gequest.com/c/flight2/forums/t/5358/warning-on-weather-training as one resource. There is a definite danger of overfitting on the provided training set(s). I think there may also be another thread on this topic, but can't find it at the moment.
\n", "votes": null, "replies": [] }, { "id": 29646, "author_name": "vikasbhargava", "author_url": "", "post_date": "08/28/2013 14:35:23", "content": "
[quote=WBTtheFROG;29639]
\nIt's the lowest cost function from the simulator.
\nThe solution that is "an algorithm that tests every possible flight path with the simulator and chooses the best one" might run into computational complexity challenges.
\nRegarding the point of other data sets, see https://www.gequest.com/c/flight2/forums/t/5358/warning-on-weather-training as one resource. There is a definite danger of overfitting on the provided training set(s). I think there may also be another thread on this topic, but can't find it at the moment.
\n\n
[/quote]
\nThanks for the reply.
The cost as described in the evaluation section is a function of the fuel consumed and the time taken.
Fuel consumption depends upon the aircraft speed and the distance covered.
Regarding the aircraft speed,
1) There has to an optimum speed recommended for aircraft by manufacturers at which the fuel consumption is minimum (we do not know).
2) Maximum permissible speed due to altitude and weather condition. Can we determine this using the historical flight and weather data? Or is there any ideal speed for all altitudes & weather.
Without knowing the maximum speed that an aircraft can go up to , is it not difficult to say which is the best path ?
", "votes": null, "replies": [] }, { "id": 30126, "author_name": "karle1", "author_url": "", "post_date": "09/04/2013 03:21:44", "content": "
UNITS Please...
\nThe test data has a ground-speed column ( knots, mph?)
\nThe submission file wants airspeed in knots (which is variable due to altitude, temperature and pressure, % Mach is what most airline pilots use above FL30 ...) Or is this just our old friend ground-speed again, except that this time it is explicitly in knots
\nThe data dictionary is of no cohesive help on units
\nWould it be too much to ask for each column in the data files to have a name with a "-units" added?:
\nExamples:
\nit would make conversions (and wrong assumptions) obvious
", "votes": null, "replies": [] }, { "id": 30135, "author_name": "jc3626439", "author_url": "", "post_date": "09/04/2013 07:05:48", "content": "Has anyone noticed there are 89 flights (rows) in 'testFlightsRev3.csv' with zero ground speed -- all above 21000 ft altitude.
\nAlso, the flight with FlightHistoryId = 303137309 has no history record in 'test1_flighthistory.csv'.
\nHave these two issues been discussed anywhere?
", "votes": null, "replies": [] }, { "id": 30152, "author_name": "julesvanligtenberg", "author_url": "", "post_date": "09/04/2013 14:39:28", "content": "[quote=JC36;30135]
\nHas anyone noticed there are 89 flights (rows) in 'testFlightsRev3.csv' with zero ground speed -- all above 21000 ft altitude.
\n[/quote]
\n\n
Yes, noticed it just now. Looks like an error to me.
", "votes": null, "replies": [] }, { "id": 30172, "author_name": "robertramey", "author_url": "", "post_date": "09/04/2013 17:16:10", "content": "I've looked over the information on this web site, downloaded all the available data, (including the simulator), Got the simulator running on my system and ran a test of it. Here is my understanding of the task - please tell me if I've missed something.
\nSooooo, My understanding of how to realistically go about this is:
\nOf course, the above points are simply different ways of saying the same thing.
\nAgain, my real question is: Do I understand this correctly?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "28321": "", "28325": "", "28326": "", "28327": "", "28330": "", "28332": "", "28342": "", "28343": "", "28348": "", "28363": "", "28365": "", "28372": "", "28378": "", "28386": "", "28398": "", "28409": "", "28414": "", "28434": "", "28464": "", "28482": "", "28727": "", "28729": "", "28737": "", "28910": "", "28937": "", "28946": "", "28979": "", "28980": "", "28984": "", "29006": "", "29082": "", "29084": "", "29104": "", "29105": "", "29115": "", "29160": "", "29186": "", "29544": "", "29545": "", "29639": "", "29646": "", "30126": "", "30135": "", "30152": "", "30172": "", "302110": "Hey, I've lived in Ukraine" }, "source": "meta" }