ERA5 Download Restrictions: Previously Possible Annual Requests Now Limited to Daily or Few-Day Requests, with Extremely Long Queues

ERA5 Download Restrictions: Previously Possible Annual Requests Now Limited to Daily or Few-Day Requests, with Extremely Long Queues

Hello everyone,

I would like to raise several serious concerns regarding the current ERA5 data retrieval system through the Copernicus Climate Data Store (CDS).

My research involves long-term wind and wave resource assessment, including historical ERA5 Complete two-dimensional ocean-wave spectra (parameter 140251 / 2dfd), potentially covering 1979–2025.

I have encountered significant changes in request size limits, MARS processing errors, and extremely long queueing times.

These problems are making the retrieval of long-term historical ERA5 datasets increasingly impractical.

1. Previously, I could request an entire year of data at once. Now, even one year requires hundreds of requests.

This is the most important issue I would like to highlight.

Previously, I was able to request a full year of ERA5 Complete data for a particular geographical region in a single request.

However, when requesting data for the same region now, the situation has changed dramatically.

With the current request cost limits:

  • Previously: One request could cover an entire year.

  • Now: With NetCDF output, I can only successfully request approximately one day of the complete two-dimensional wave spectra using my required configuration.

  • Even after switching to GRIB format to reduce conversion costs, requests are still limited to relatively short periods of only a few days, depending on the regional selection and processing requirements.

Requests exceeding the current limit immediately return:

HTTP 403 Forbidden

cost limits exceeded

Your request is too large, please reduce your selection.

This represents a substantial reduction in the amount of data that can be retrieved in a single request compared with my previous experience.

What previously required a single annual request may now require hundreds of individual requests.

I would like to understand why such a significant change has occurred.

Have the request cost limits been officially reduced? If so, when were these changes introduced, and what are the current limits?

2. ERA5 Complete regional GRIB requests encounter MARS errors

My current research configuration is:

  • Dataset: reanalysis-era5-complete

  • Parameter: 140251 (2D ocean-wave spectra)

  • Area: 52/98/0/142 (North/West/South/East)

  • Grid: 0.5/0.5 when requesting regular-grid output

  • Time: 00/03/06/09/12/15/18/21 UTC

  • Frequencies: 1/to/30

  • Directions: 1/to/24

  • Stream: wave

  • Type: an

I understand that GRIB is preferable to NetCDF for reducing server-side conversion costs.

However, when requesting regional GRIB data using area without specifying grid, the request is initially accepted but subsequently fails during MARS processing:

ReducedLL::croppedRepresentation() not supported

MIR: Not implemented

Interpolation failed (-2)

Adding grid=0.5/0.5 avoids this particular native-grid cropping issue, but increases the request processing cost and may cause larger requests to exceed the current limit.

Requesting global native-grid GRIB data without area and grid is another possibility, but downloading unnecessary global data is inefficient when only a specific research region is required.

Is there a supported way to retrieve regional data directly from the native reduced latitude-longitude grid without triggering this error or incurring additional regridding costs?

3. Subsequent requests frequently remain in the queue for several hours

Another major problem is the queueing behavior.

I have repeatedly observed a similar pattern across:

  • ERA5 Complete

  • ERA5-Land post-processed daily statistics

  • ERA5 post-processed daily statistics

After a relatively long period without submitting requests, the first new request often enters the Running state almost immediately.

However, subsequent requests frequently remain in the Accepted state for 1–2 hours, sometimes even 5–6 hours, before processing begins.

I have not measured the exact inactivity period, and I do not know whether there is any fixed waiting interval.

During one test, the ERA5 Complete queue showed:

Running: 100 — Queued: 2,711

With the message:

Number of available slots for this dataset has been reached. Please try again later.

Is there a per-user scheduling or prioritization mechanism that could explain this repeated pattern, or is it entirely caused by overall system congestion?

4. Downloading decades of historical data has become practically unrealistic

Consider the impact of these changes on long-term research.

If regional GRIB requests are limited to approximately two days each:

Historical period Required requests
1 year Approximately 183
10 years Approximately 1,827
1979–2025 (47 years) Approximately 8,584

If subsequent requests each require 2–6 hours of queueing, retrieving a single year could involve approximately 15–46 days of cumulative queueing time alone.

This excludes the time required for MARS processing, data preparation, and actual downloading.

For 47 years, the cumulative waiting time could become enormous under the same assumptions.

These calculations illustrate the potential impact of the observed delays; they are not predictions that every future request will experience the same waiting time.

The fundamental problem is that the system previously allowed much larger requests, but now forces users to divide the same historical data into hundreds or thousands of smaller requests, each potentially requiring several hours of waiting.

Simply recommending smaller requests does not solve this problem. It can make the overall download process considerably less efficient.

5. Questions for the ECMWF Data Stores team

I would greatly appreciate clarification on the following questions:

  1. Why was it previously possible to request an entire year of ERA5 Complete data for a given region, while similar requests are now limited to one day or just a few days?

  2. Have the request cost limits recently been changed? If so, could ECMWF publish the current limits, cost calculation methods, and effective dates of these changes?

  3. Is there a supported way to extract regional native-grid GRIB wave spectra without the ReducedLL::croppedRepresentation() error?

  4. Why does the first request after a period of inactivity often start quickly, while subsequent requests may spend several hours in the queue?

  5. Is there an officially supported bulk retrieval service, alternative archive access method, or other efficient solution for obtaining decades of historical ERA5 data?

  6. Are there plans to improve the current request limits and queueing mechanism for large-scale scientific research?

Conclusion

I sincerely appreciate the valuable ERA5 datasets provided by ECMWF and understand the need to manage shared computing resources fairly.

However, the combination of substantially reduced request sizes, MARS regional extraction errors, and extremely long queueing times has created a serious obstacle to long-term scientific research.

The most frustrating aspect is that downloading an entire year of data in a single request was previously possible for my regional selection, whereas now the same research task may require hundreds of individual requests.

For researchers working with several decades of historical data, this is a significant practical problem.

I hope the ECMWF Data Stores team can provide a clear technical explanation of these changes and, more importantly, a realistic solution for efficiently retrieving large historical datasets.

I would also be interested to know whether other researchers have experienced similar changes in ERA5 request limits and queueing times.

Thank you for your attention and any suggestions.

2 Likes