Hi all,

Thanks everyone for your participation in this quest so far, along with your patience with us on the data issues.

I'm tremendously excited to announce the release of the Public Leaderboard Set, which contains 14 days of test data from November 26, 2012 through December 9, 2012. Each day is truncated at a randomly selected cutoff time. This data, along with sample benchmark submission files, can be downloaded from the data page.

This contest is far more complex than the typical Kaggle contest and currently it is relatively challenging to get started. We will be providing additional code and guides soon to help you with this, as well as clarifying and simplyifing some of the data descriptions.

Please let us know if you have any questions, and good luck!

Ben

Comparing the benchmark scores on the public leaderboard to scores of my reimplementations on the training set, I get ~95 for the cutoff benchmark, but only 10-12 for the estimated and sceduled arrival benchmarks. It would be really nice to have the sources of the benchmarks.

Two unrelated things:

The capitalization/case of some folder names (metar vs Metar, for instance) has changed. Of course, this is hardly fatal just inconvenient.

Why does test_flights.csv contain anything else then flight_history_ids? Aren't those columns just duplicated from flighthistory.csv?

Thanks for pointing this out. ]There was a bug in the estimated arrival benchmark and I've updated it accordingly. (Code is on github, and I'll publicize here once I get a chance to refactor & document it).

This will be consistent moving forward ("Metar", etc.).

These are duplicated from flighthistory.csv. Added so a newcomer to the site could download a combined test_flights.csv file and run a quick script to make a basic submission without downloading a large zip file (haven't put this up yet).

Thanks. One more thing: are non-integer minutes in the prediction ok? The rounding (that was apparently done with floor instead of the more correct round in the cutoff benchmark) could be an unncessary (?) source of noise.

Real-valued predictions are fine

Hi Ben,

On the leaderboard I see only the entry above the top benchmark as being “in the money”, shouldn't this be the first 5 entries made by individuals/teams?

Or is beating a benchmark in anyway conditional for winning a prize?

Thanks,

Jules

Thanks again.

Are all flights to be predicted included in the calculation of the public score on the leaderboard or is there a private component at this early stage too? I'm asking so that the noise level can be estimated.

No private component at this stage - that will only come with the final evaluation set.

Thanks for pointing this out, this is fixed now.

Ben,

I was wondering what the status was for publishing the code needed to generate the benchmark.

rg,

K

Hi,
i've noticed in some FlightHistory.csv of the Leaderboard set that the field scheduled_aircraft_type is not filled for several rows. Is there any reason for this?

I've also found out that always in the field airline_code sometimes there's the '1I' value, that's not a valid IATA code and there wasn't in the files of training set data.

Thank you,
Pierluigi

Hi,

It is still a pain for a newcomer to get an idea on how to start. It takes roughly 4 hours of forum scanning and python parsing (once found on the forum) to figure out that days.csv, testflights.csv, and test-flightscombined.csv are crucial and what they contain.

Further, the only clear explanation of what is the allowed training data and what is the test set is well hidden in one of the many 'clarification needed' threads (I have lost it already, I will have to dig again)

Is it possible to put the description of the PublicLeaderboard files on the data-wiki?
* what is test-flights.csv + columns (or where they are copied from, where MISSING/HIDDEN values are added)
* what is testflightscombined.csv + columns
* what is days.csv
* how PublicLeaderboard set is different from the InitialTraining set (values missing after cutoff)
* that one should not use PublicLeaderboard set for training

I can do it myslef if you open the front page for editing. Or please move 'DirectoryStructure' to a new, editable page.

Thanks!

Ben:
Can you confirm that the timezone of flighthistory.csv in both PublicLeaderboard and also in InitialRelease (training) folder is in UTC?
And that the rest are not in UTC

Thanks
kiran

Are there links to the wiki and github repository somewhere on the information page? If not, could you please provide them?

Hi,

The benchmark files have only three columns, each row corresponding to a distinct flight ID. The for which this prediction has been made is not mentioned.

In the submission file for a prticular flight which day should the prediction of actual flight gate and arrival time correspond to ?

Should there be 15 files for the 15 days of the test set ?

Sorry wrong post

Hi  Ben,

What is the format of model submission. 

Algorithm pseudocode or detailed algorithm ?

regards,

I. Jaganjac

A hash of a zip archive containing everything necessary to make flight arrival time predictions on a new day's data.