We are not providing these files since all this is future information.  You need to predict what the airport and weather conditions will be.

Double check your using the same files for both simulation runs.   My testing has not revealed any differences in scores - I just confirmed with the oneDaySampleSubmission(July 4th) files.  

I think the problem is in OneDaySimulatorFiles-Rev3.zip. I see the same problem as Dave (553 arrived) when I use the updated files, whether I use the Rev3 or Rev4 simulator, but not with the OneDaySimulatorFilesRev2.zip files.

Ah, the sample submission file in OneDaySimulatorFiles-Rev3 had the altitudes reverted back to 18,000 ft.  They should be at 17,000 ft in order for Eastbound flights to hit the arrival zone.  Updating that file now.

Thanks Dave & TecS

I agree, the RAP data sets have been approved, so they should be usable. Can someone elaborate on the matter?

Furthermore, what was the purpose of renaming some structures in the latest revision of the simulator? It's not easy (and surely it's time consuming) to go through the entire code every time there is an update... And updates seem to be rather frequent.

This is an important issue that calls for a response from the admin.

If one were to obtain the weather data for the test dates, it may be possible to run the simulator on the test data. This would cause the contest to degenerate into a brute force search. Sure, the simulator still won't have access to ground conditions, congestion etc., but I would think that wind velocity by itself can go a long way in aiding with optimization.

Also, I think weather data from the future would allow a contestant to fairly accurately estimate the leaderboard score (both public and private) of a solution without even making a submission. This would seem like a strong argument for banning usage of weather data beyond the cutoff time, or even disallowing outside data entirely.

On the other hand, the contest as stated, sounds a bit contrived. It is possible that I misread the terms, but it seems like all the future waypoints are to be predicted based on data, at or before the cutoff time. While this constraint is reasonable for the first waypoint, it is not very real-life-like for the subsequent waypoints. After all, by the time the plane reaches its first waypoint, wouldn't the onboard agent have access to updated weather information? Why not use  it to come up with the next waypoint? Why rely on predictions when the agent has access to hard data?

The answer may be that it is hard to pose this problem in the form of a contest if real-time weather information were allowed. Gaming would be an issue. It may be hard to detect illegal usage of weather information (such as using weather information at the next waypoint to optimize the current waypoint, which would not be possible in real life). Nevertheless, I would like to hear the admins' take on this.

While I appreciate the amount of work that Kaggle has put into this contest, I think the rules must have been spelt out more clearly.

It's possible anyway. The impossible is to predict data which you can just take. In this case we are not blinded. And simulators behavior anyway far from reality for now. So it's not the biggest problem(I have a long list of much more important issues ;) ). In every competition some future information provided - at least leaderboard data. Also in real life routes can be updated at least every hour - in this competition we can't do it. So using this data closer to reality.

Quoting the Basic Structure of FQ2 page,

I made some tweaks (changed the default values for cruise elevation, speed and descend distance and some other minor things) to the benchmark code and made my first submission yesterday. I expected it to do a little better than the baseline. Instead it scored 12766 and ended up 12th! The current leader is just 58 points ahead. Unless someone comes up with an effective routing algorithm within the next week, there is some danger of the milestone prize going to the contestant with the luckiest choice of parameters.

I hit a wall going down that path. It doesn't look possible to get below 12760 by optimizing the default cruise altitude, speed and descend distance in the benchmark code. I was wrong when I said luck would be enough to win this.

It turns out that optimizing these parameters on the OneDaySimulatorFiles training data produces more or less optimal settings for the test data as well. Further perturbations of the parameters did not improve the score much.