Is there a data file corresponding to training2_flighthistory_USARRDEP.csv that was provided in the previous phase of the competition? I think it is FinalEval-training/trainingFinalEval_flighthistory.csv, but this file does not have history going back a few weeks.

Line counts are as follows:

22349 FinalEval-training/trainingFinalEval_flighthistory.csv

848932 training2_flighthistory_USARRDEP.csv

The organizers never said they would release another "full" multi-week training set. These trainingFinalEval* files contain "features" for January 18th, which fulfills the same purpose that September 10th did in the main phase of the competition: providing a single day of groundConditions, actualTakeoffs, and actualLandings temporally adjacent to the test set.

If your uploaded model contained code that "trained" on September 10th, you're allowed to use this same code (with no modifications except for the input filenames, of course) to "train" on January 18th in order to produce a model for predicting ground conditions, takeoffs, and landings on the test days.

Personally, I didn't find it useful to train on September 10th, so my model isn't using any of the "training" data for January 18th at all. But for those who did train on September 10th in that manner, hopefully I made the situation a bit clearer for you.

Anil,

Not sure what you're asking. For the Final Evaluation phase, the "training" period is just 1 day, Jan 18, 2014.

Previously, "_USARRDEP" was used to indicate that the flight data was updated to include flights ARRiving or DEParting from the US. That is "baked in" to this data set, so no special need to call it out.

So, features corresponding to FQ2DataRelease2_AggregateTrain will not be provided? I would like an official response on this. I would very much doubt this is the admins' stance on the issue. What was the point of providing multi-week training data if we were not allowed to train on it? If there was some communication that stated this data won't be available for the final stage, please point me to that. Maybe I missed something.

Going by the previous leaderboard, I am not in the running for a prize. Nevertheless, having come this far, I would like to at least make a submission. The way things stand, I just won't be able to.

Doug, it looks like you submitted your routes already. I have to say that is impressively quick.

They are provided in FinalEval-training.

"Training" is a misnomer here, there is no model training for this phase of the competition. There is only model running on the test set.

I use that nomenclature to mean "full days" of data, versus the test set where each test set day has a cutoff after which no data is available. The single training day of Jan 18 is provided to correspond to the 1-day simulator files we've provided. We're providing it now because we have also done so in the past.

To be more explicit, yes, please do make a submission. Take your model, which you have already built and trained from previous phases of this competition, and run it on the test set and create your submission file. Does this clear things up?

Anil, I'm not the official source but I'm basically 100% certain that in this final phase you are still allowed to use your existing code to read data from any of the previous 3 training sets (#1 from the milestone phase, or #2 and #3 that were provided with the main phase). I apologize if I made it sound like this isn't allowed. I was only trying to point out that they aren't providing any *additional* training sets.

Keep in mind that training set #3 ran from 11/14/13 to 12/11/13. Since this period contains the early part of winter, it is a decent proxy for late January. (The wind/METAR climatology and flight schedules wouldn't be a whole lot different than late January.) I think it is entirely reasonable that this was chosen to be the last training set. Do you disagree?

Yes, somewhat. I seem to have misunderstood the purpose of the flight history file. I assumed that we were allowed to construct models that predict takeoff and landing schedules, given schedules from the previous weeks. Running these models, of course requires multi-week history prior to the test interval, which aren't given.

I will make a submission, but it likely wouldn't be competitive as my code was based on incorrect assumptions. Bummer...

Using set #3 seems to be my best option. Thanks for the suggestion.