And now, what's to stop some competitor from creating some throw-away account, taking rap forecasts, adding some level of noise, and saying "here"? (While using the actual forecasts for their submission)

Goes back to my previous post I guess -- without having the files yourself/knowing which forecasts are approved, how were/are you going to verify?

This isn't relevant for the current leaderboard data, since no verification is done on that leaderboard and no prize money is awarded based on that leaderboard (note all future prize money is awarded based on the final evaluation leaderboard, the data for which doesn't exist yet).

Sure, for which the data isn't released yet, but one would hope the same policies/procedures are in place for the data acquisition, handling, and generation.

And I would say it still is relevant for current leaderboard -- someone providing false forecast data for this set could mislead competitors in the development process (ie about the value of weather information, leading people to believe "oh these forecasts are crap, i'm not going to bother with them in my model"). Even if it isn't used for any prizes awarded...

I have uploaded a zip file with cutoff hour and subsequent forecast RAPs for September
11 through 24. Sorry it took me so long to hack up an encoder and fake the data with
noise. :P I also named the file "sep11-18rap.zip" to really confuse you all.

Good news: RAP forecasts are now once again available all the way back to June 20! http://soostrc.comet.ucar.edu/data/grib/rap/

I'd urge the Kaggle admins to immediately download all September 11-24 forecasts in case they disappear again.

What the heck? They're offline once again...

(actually they're online, but the links are hidden and I get a 403 forbidden error page if I try to access them through a reconstructed link)

I got the files 10 minutes ago without any problem. So maybe some local problem with your computer/internet connection?

I tried both from behind a proxy and from a smartphone, but I'm back to the situation of a few days ago: the first day on the list i 2013-09-29.

Seems they are really gone ... Now I cannot see them anymore in the browser.

@Gabriele: I have at least all the GRIB files for September 11-24 forecasts at cut-off time. If you want them (7.4 GB) write me a short mail (I cannot contact other users directly because I have any kaggle points :)).

I wonder why Kaggle still hasn't posted a torrent of the files I uploaded last week.

Anyway, to reiterate: this on/off appearance of what seems to be at least two servers with different sets of RAP files is what I have been seeing for months. This time it just took a little longer than usual for the larger set to reappear.

You're probably right: today they seem to be available again.

Thanks to TecS for supplying a copy of the RAP files. Does anyone have the forecast RAP files for Sep 10th? I noticed that Kaggle has "Weather GRIB2 Files (Sept. 10th).zip" available for download, but this archive does not contain forecasts.

To be clear, I am looking for the following files:

13091018.rap.t18z.awp130bgrbf01.grib2

13091018.rap.t18z.awp130bgrbf02.grib2

13091018.rap.t18z.awp130bgrbf03.grib2

13091018.rap.t18z.awp130bgrbf04.grib2

13091018.rap.t18z.awp130bgrbf05.grib2

13091018.rap.t18z.awp130bgrbf06.grib2

13091018.rap.t18z.awp130bgrbf07.grib2

I understand that these were previously available for download at http://soostrc.comet.ucar.edu/data/grib/rap/.

By the way, the file "Weather Forecast GRIB2 Files (Sept 11-24) - Provided by TecS.zip" that Kaggle has made available for download, appears to be corrupt. I had to use the fix option in the zip utility to salvage the data. Haven't checked if the data is actually usable. I tried to download from two different machines and got the same result. The downloaded file is 8227061008 bytes long and has an md5 sum of e92d80340e68452675badec15564eaa6.

Admins, please check... It is no fun to download corrupt 8GB files.

I still have the zip archive I uploaded, just checked, and size & md5sum are as you reported. So nothing seems to have been corrupted in transfer.

Windows Explorer (8.1, 64 bit) has no problem opening and navigating it. It was created using "Free Zip Opener" ( http://www.radzipper.com/ ). Maybe if you try using that it will work for you? Only thing I can think of is that older and/or 32 bit zip utilities might choke on archives larger than 2 GB.

I have now also uploaded a zip file with 0th hour weather and 7 subsequent hourly forecasts for September 10 to

http://bayfiles.net/file/12AyX/Ujak6Z/sep10rap.zip

md5sum for this one is

bed765eb462424e99cb6ce7abf6203e4

size is

248,876,189 bytes

(so not large enough to be a problem even with old unzippers).

Much appreciated! I am pulling down the file as I type.

I had trouble with the other zip file on both Mac OS and Ubuntu. The error message was:

"start of central directory not found;  zipfile corrupt."

Didn't try Windows.

UPDATE: It was the unzip utility on Mac OS and Ubuntu that had problems with the zip file. I just tried 7za instead and it worked fine.

Seems consistent with http://superuser.com/questions/114011/extract-large-zip-file-50-gb-on-mac-os-x/249689#249689