powered by kaggle

Completed • $220,000 • 122 teams

Flight Quest 2: Flight Optimization, Main Phase

Thu 26 Sep 2013
– Sat 18 Jan 2014 (11 months ago)

Simulator Issues and Bug Reports

» Next
Topic

I completely agree with Gabriele Seppi, and in fact I was going to propose the same solution to the problem of undetected intersected restricted zones.

It can be an importante factor in the evaluated mean cost, and the proposed line-polygon intersection check in each step seems a simple solution.

I understand nobody likes simulator code changes, but it seems clearly better to consider the airplane a moving point (segment) during each step, that a tiny cylinder with quantum tunneling powers.

Hi admins,

I'm taking a look at the arrival model of the simulator, and there are a few things I found suspicious.

1. visibility terms in landing model and taxi model

visibility coefficients are given in the Landing.csv and Taxi.csv data files, but they are not used in the predictLanding and predictTaxi functions in Arrival.fs. Are they intentionally omitted?

2. utilScheduledRunwayArrival/untilScheduledGateArrival in landing model

Landing model data(Landing.csv) has the coefficent named time_at_cutoff_to_scheduled_gate_arrival_hours and time_at_cutoff_to_scheduled_runway_arrival_hours. From the namesI assume they are meant to be the coefficents for (scheduled gate/runway arrival time - cutoff time), but in predictLanding function, these coefficients are multiplied with (scheduled gate/runway arrival time - the starting time of descent for landing). Which is correct (the field name or the code)?

3. untilScheduledGateArrival in taxi model

Similarly, taxi model data(Taxi.csv) has the coefficient actual_runway_arrival_to_scheduled_gate_arrival_hours but it is multiplied by (scheduled gate arrival - the starting time of descent for landing). Again, which is correct one?

Aside from the arrival model, I have another issue with the simulator. In the simulator, the wind speed above 40880 feet or below 18115 feet are calculated as zero.This can be exploited for avoiding headwind, and I think the more realistic behavior for the weather modeling would be using wind vector from the nearest grid point.

I realise it's pretty late in the game for this, but is there any documentation on what the simulator is supposed to do, other than the actual code?  There have been a lot of good questions raised, but I can barely get through the F# well enough to figure out how it works, much less decide if the code is "correct" (obviously it's my own fault for not committing more time, but still).

My most recent question: if you ascend or descend over (or under, I guess) a restricted area, then whether or not you intersect it depends on the rate of ascent or descent, and I can SEE the math, obfuscated tho it is, in the F# code, but I have nothing tangible to compare it to, so I can't tell if there's a bug or not.  Does that make sense?  Should descent be a straight line?  Should it (a graph of altitude over time during descent) be steeper at the start and level off (convex), or should it start more slowly (concave or s-shaped)?  Concave would cause crashes, of course.  Should turns happen on a dime or cut the corner of the waypoint on the acute side?

For example, there's an

As a last example, was the eccentricity that allowed the winning milestone strategy not considered a bug in the ascent cost logic?  From the forum posts it sounds like the ascent costs weren't changed (only the 5-change penalty added).  With some sort of descriptive model to compare against it would be easier to tell and recommend improvements.

Anyway, let me know if I'm missing something... I have a hundred other questions, but it's too late to go into that level of detail, I think, and almost certainly too late to fix simulator bugs, but I finally had to break down and ask.  Thanks for reading!

Hi,

I've been trying to get RunSimulation.exe to compile. At first it couldn't find the files groundConditions_*, actualTakeoffs_* and actualLandings_*. If I'm interpreting correctly, those files aren't needed for the FQ2 Partial Simulator Data? Anyways, I removed the references to those files in RunSimulation.exe.config. But now I am getting an error:

System.UnauthorizedAccessException: Access to the path my_path_to_the_data is denied.

I am running the executable as an administrator and the folder has no special permissions required. Plus, the program has already read a bunch of data from the folder (Airports.csv for example).  Has anyone encountered this error? I am running on Windows 8.1.

Thanks!

I am concerned about the issue of a flight flying unperturbed through a restrictedZone.

After looking at the code, it seems to me that one can safely pass any restrictedZone that is small enough to be crossed in less than 0.6 minutes (one can take a safe margin to account for possible headwind) by putting a waypoint just before crossing it.

By doing it, he is assured that the next step at which the simulator will check the possible intersection would take place after 0.6 minute. So as long as one chooses a good enough margin, he would be safe to cross it (I have not tested this).

If it is indeed the case, any one who would not use this strategy would be penalized, since it is certain that cutting through corners of small enough restricted zones would save on fuel and time, with practically no risk of crash.

I think there should be an official ruling (and enforcement) on whether or not this is allowed. I agree with Gabriele and Jose that adding a check to disallow this would be best for the purpose of this competition.

One possible option that would not require any change in the simulator would be to postprocess the flight logs to check that this condition has not occured during the flight.

We've updated the simulator and have re-scored the leaderboard. 

Version 2.1
- Changed collision detection mechanism to prevent skipping over zones
- Added Visibility to Landing/Taxi GLMs
- Wind data is now relevant above 40.8k and below 18k feet

Hi!

Thank you for updating the simulator.

"Wind data is now relevant above 40.8k and below 18k feet". Will we get a new weather file snapshot in the cutoff time, to see the difference? 

Thank you in advance,
Christian

The leaderboard seems to like those changes. Is it still using the same flights as before? If so, I guess the improved scores are due to the aircraft now being a dimensionless dot which can get closer to restricted zones with impunity, so there are fewer collisions. Oh well, win some, lose some.

The new collision detection still has potential for "skipping". As stated at line 29 of Airspace.fs, using only destination altitude allows for clipping of upper and lower corners. Furthermore, flights which at cutoff are inside a zone (near the boundary), can easily escape it without being penalized.

Glad to see that segment-polygon intersection has been implemented, though :)

... or perhaps the improvement is because flights at 41,000 can also benefit from the wind

Thanks to the organizers for the new simulator.

Gabriele Seppi wrote:

The new collision detection still has potential for "skipping". As stated at line 29 of Airspace.fs, using only destination altitude allows for clipping of upper and lower corners. Furthermore, flights which at cutoff are inside a zone (near the boundary), can easily escape it without being penalized.

You're welcome to try making gains from clipping on the z-axis.  Furthermore, no flights start off within a zone. 

Christian Kiss-Tó wrote:
 

Will we get a new weather file snapshot in the cutoff time, to see the difference?

Sorry for being vague; there is no new wind data.  For altitudes outside the range of 18k-40.8k ft, the min or max altitude level is used.  

Arieli wrote:

... or perhaps the improvement is because flights at 41,000 can also benefit from the wind

Thanks to the organizers for the new simulator.

You're welcome :)

I just noticed on MSDN that System.Math.Round() can be unpredictable in rounding certain values (see http://msdn.microsoft.com/en-us/library/wyk4d9cy(v=vs.110).aspx for an example). Considering that flight level selection during instruction correction is an important step in the evaluation of a flight, wouldn't it better to replace System.Math.Round() with System.Math.Ceiling() (o System.Math.Floor()) for consistency? I'm talking about line 25 of InstructionCorrector.fs: why do westbound flight receive rounding with System.Math.Round() while eastbound flight with System.Math.Ceiling()?

ℕ wrote:

Gabriele Seppi wrote:

The new collision detection still has potential for "skipping". As stated at line 29 of Airspace.fs, using only destination altitude allows for clipping of upper and lower corners. Furthermore, flights which at cutoff are inside a zone (near the boundary), can easily escape it without being penalized.

You're welcome to try making gains from clipping on the z-axis.  Furthermore, no flights start off within a zone. 

I'm not saying we can make gains, but that, not knowing future winds, we could find ourselves in a situation where the aircraft hits a zone even if we didn't want to, simply because of a stronger tailwind or headwind.

Regarding flights starting off within a zone, I may be wrong, but I've come across a few which, at cutoff, are inside a turbulence zone (a couple examples: 309252255, 309256963). I haven't checked them all, of course, so I don't know it this is the case, but if a flight at cutoff is inside a zone, yet close enough to the lower or upper boundary, it can escape easily.

I don't think it's going to matter anyway, I was just trying to be open and notify of possible - yet remote - pitfalls :)

Gabriele Seppi wrote:

I'm not saying we can make gains, but that, not knowing future winds, we could find ourselves in a situation where the aircraft hits a zone even if we didn't want to, simply because of a stronger tailwind or headwind.

Sounds like real life.  :-)

Gabriele Seppi wrote:

Regarding flights starting off within a zone, I may be wrong, but I've come across a few which, at cutoff, are inside a turbulence zone (a couple examples: 309252255, 309256963).

Ah, I thought you were referring to restricted zones.  

Hi

Thanks to the organizers for the update.I just want to mention to everybody I had some initial problems with the updated simulator because violations of restricted (turbulent) zones with repeated vertices are not detected (because of the resulting (Nan, Nan) in calculating the unit normal vector of this zero-length polygon side).

The last version of "FQ2 Partial Simulator Data with 1hr weather" has the corrected "restrictedZones.csv" without vertices repetitions but the "Sept10-SimulatorFiles" still have the "Twenty Nine Palms" restricted zone with repeated vertices.

José wrote:

The last version of "FQ2 Partial Simulator Data with 1hr weather" has the corrected "restrictedZones.csv" without vertices repetitions but the "Sept10-SimulatorFiles" still have the "Twenty Nine Palms" restricted zone with repeated vertices.

Replaced the restrictedZones.csv in the Sept. 10 files.  

Hi,

sometimes the simulator doesn't reach the destination because of arrivalRadius (1mile in Configuration.json) simulator parameter.

Let me explain my self. Suppose the last waypoint is within 1 mile from the airport arrival frontier, i.e. waypoint between [64.1732, 65.1732]. stepToWaypoint function returns that destination is reached when you are at most at arrivalRadius from the last waypoint. Sometimes it happens that the last position is larger than 65.1732 (destination reached is set) but the aircraft won't land because it is out of airport radius.

Can we know the simulator parameters in the online simulator?

BR

Most expensive flight, ever!

With this itinerary:

308663997,1,34.9709,-103.1294,37000,406.5456
308663997,2,32.8996,-97.0574,17000,319.4624
308663997,3,32.897,-97.038,17000,319.3332

I get, 10^33 in fuel expenses ?!

...

308663997,33.3034728137757,-98.1717180290048,-160.875486538811,502.229931179563,1.500,319.46,316.36,17000.00,6689.14,False,False,125895.56,Flying
308663997,32.897,-97.038,-103.747366772221,476.470353064938,3208235363634990000000000000000000.000,0.00,0.00,607.00,6365645862639280000000000000000000000.00,False,False,114668.00,Crashed

It seems a bit odd that groundConditions for times outside those in the file are set to 0, 0, 0, T=45, rather than assigned to the nearest value, similar to the issue/change made for weather layers.

This means, if a cutoff is early in the hour, and the next report isn't until cutoff + value close to 1hr, all flights landing in < 1hr will ignore ground conditions and set to the constant values? Why not just use the closest report (as done for values within the timeaxis, and now similarly done for z outside wind layers)? Seems like an odd boundary condition to introduce, and fairly unrealistic.

Arieli wrote:

Most expensive flight, ever!

With this itinerary:

308663997,1,34.9709,-103.1294,37000,406.5456
308663997,2,32.8996,-97.0574,17000,319.4624
308663997,3,32.897,-97.038,17000,319.3332

Arieli, is this reproducible for you (or anyone else)?  My flight log came out normal with this itinerary.

Reply

Flag alert Flagging is a way of notifying administrators that this message contents inappropriate or abusive content. Are you sure this forum post qualifies?