I've been downloading global wind (u-,v-components) data and I can only request 2 days worth of hourly data and it takes many hours to download. I have tried requesting more days but I get a message saying that the request is too large. I would like to download the wind and precipitation data for all of the available years globally. Is it possible to put in a request to get the information more efficiently? Is there a way to purchase summary datasets?
Thanks!
That's strange - you should be able to download more. We download a month's worth of ERA5-Land data at a time routinely.
One option might be to download monthly averages for ERA5-Land (https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-land-monthly-means?tab=overview).
We need to identify maximums, so getting monthly averages defeats the purpose. Do you download a month's worth of hourly data or the averages? How long does it take you?
I don't understand why it's so slow for me, I've been working like this and trying to figure out a better method for 2 weeks..
Is it the processing or the download that's slow?
We download hourly data (we run a commercial service for faster access to ERA5 data, usually multi-year time-series for point locations). For a single variable for a month in NetCDF format, it takes up to about 20 minutes for CDS to process and anywhere between 10 to 20 minutes to download the file that's about 10GB.
If you were downloading this two weeks ago, it may have been extremely slow because CDS was having technical issues. I think it's mostly solved now and I haven't had any trouble in the last week or so.
If you can share your download script, happy to take a look.
I was able to download a month's worth of data in 50 minutes but my colleague kept getting 9 hour estimates and then the site crashed on him several times.
Would your commercial service be able to provide global data?
Certainly - please feel free to reach out via my email (joseph.yang@oikolab.com) and we can discuss further.