Python package to (hopefully) retrieve data without cdsapi 403 Client Error

Hello,

I was having troubles downloading model level data from different locations. I needed data for two years and it had suddenly become impossible because CDS changed the retrieval limit to something quite low (max 15 days of data at the time, in my case).

This was quite tedious and took some time to solve (see Forum discussion here), so once I fixed it I thought it could be useful to others as well and created a small package to do the fetching/download and filtering without getting the “403 Client Error” associated with a too big request.

In the package you define an area and possible gridpoints of interest (such as locations you want to get the data for) and the package organises the downloads accordingly, actively checking to avoid things getting downloaded twice and so on. It then filters the data to only maintain the locations you are interested in (this is far more efficient than submitting several requests for different locations).

I tried to respect all the indications and best practices I was told here on the forum and via Jira tickets with ECMWF.

You can find it here: GitHub - loretet/ERA5_bulk_retrieval: Scripts to bulk retrieve ERA5 data, optimised for the MARS archive after recent decrease in request size limits. Handle submission, fetching, and gridpoint extraction autonomously and can be automated. Designed with best practices from CDS User Forum and ECMWF support in mind. · GitHub

Happy to get some inputs or contributions. I don’t think I will actively maintain it for long - my project is about to end.

Cheers

Lorenzo