Hi all,
Over the past few days, I’ve been noticing frequent 503 errors when requesting AIFS data from the s3://ecmwf-forecasts S3 bucket.
aws s3 cp --no-sign-request s3://ecmwf-forecasts/20251101/00z/aifs-ens/0p25/enfo/README.txt ~/Downloads/README.txt
gives back
download failed: s3://ecmwf-forecasts/20251101/00z/aifs-ens/0p25/enfo/README.txt to Downloads/README.txt An error occurred (SlowDown) when calling the GetObject operation (reached max retries: 2): Please reduce your request rate.
More complicated data requests are even less likely to succeed. Is there an overwhelming user request volume on that bucket?
Thanks,
Evan
Hi Evan,
This data storage is in charge of AWS, so we at ECMWF have no visibility if there is a bigger than usual demand on the service.
If it is of any help, when the download fails from AWS, you can try downloading directly from ECMWF, using opendata api Python client: GitHub - ecmwf/ecmwf-opendata: A package to download ECMWF open data
I hope this helps!
Milana
@Evan_Schechter , I was having similar issues that I first noticed on 30 Oct. Some discussion is here ECMWF AWS issues? (Oct 30th) · blaylockbk/Herbie · Discussion #487 · GitHub .
As an alternative to AWS, Google also appears to keep a copy of IFS/AIFS Open Data. See [1]. It’s now available as a “source” in Herbie (GitHub - blaylockbk/Herbie: Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.).
[1] I’m a new user here, so I can only include 2 links in a post. If you follow the first link I posted above, my latest comment has a link to PR #489, which has details on Google’s copy of the data.
I started having the same issues again starting a few weeks ago (approximately April 3 2026) and it’s still ongoing, although it seems to be intermittent.
I’ve also been experiencing errors with IFS GRIB downloads, especially the ensemble ones. I suspect this is due to capacity, as the problems seem to go away between updates.
Not sure whether this is the right place to make suggestions, but it would be very helpful if regional subsetting (specifying a lat–lon range) were available. As far as I’m aware, it’s not currently possible. If it were, I suspect the amount of data downloaded would be significantly reduced.
Hi all,
Thanks for raising this. We’re not aware of any recent changes to the AWS S3 bucket configuration, but it’s possible that periods of high demand are causing some throttling on AWS’s end. If you’re hitting 503s during busier windows, it’s worth trying our Azure or Google Cloud mirrors, or our own ECMWF endpoint, as alternatives — the data is published identically across all four locations.
On subsetting: this isn’t something we currently offer as part of the open data service, though it’s something we may revisit in future. The current open data service is designed to ensure our operational delivery is not impacted.
In the meantime, two options worth knowing about:
• Earthkit (https://earthkit.readthedocs.io) lets you subset GRIB files client-side after download, which works well if bandwidth rather than storage is your constraint.
• Server-side subsetting is available via our service charge scheme ( Service Charges: From 01 July 2024 - Data and Charts - ECMWF Confluence Wiki ) for users who need it as a managed service.
Best wishes,
Emma
Hi everyone,
Another alternative is to use the Icechunk versions we have on the Earthmover marketplace. We have IFS (on Cloudflare via Brightband) and AIFS (on AWS via Dynamical).
Generally your client library needs to handle the various errors that the server can throw, which Icechunk does for you.
We haven’t heard any complaints about availability for these datasets, but let us know your experiences with them.