Hello, I’m really surprised by the latest poor downloading performance of both the API and the online page. I just don’t understand why is the whole thing being slower and overall in a worse shape to what it was on previous years. I really would like to understand were are the current roadwork for the tools. Because the whole update process for me has been a failure (at least for the end user). Hope to see a little improvement in upcoming months. Just to give you a comparison, right now to download a full year of just one variable it takes around an hour of processing and that if you’re just lucky that the number of request running are not over the limit (extremely low). To download some 20 years of data or so it’s taking almost 3 to 4 days and sometimes with internal errors between request.
I have been trying to download the same 12 completed datasets for four days now, using the API and the web page. Today is the first day where download speeds have been faster than 100 kB/s. It has maxed at 90 kB/s the previous days, which led to a server timeout and download failure.
Fingers crossed I can actually downloaded the processed datasets before they get deleted! 
My requests are taking a loooooong time to run. Hours. Once they do run the download speeds have been ‘ok’. 1-2MB/s
I’ve had the same experience. Just waiting and hoping the download process works. My job was stuck for about 2 hours. I am waiting.
I am also having the same experience. I started trying to download the same data last Tuesday, and my jobs have kept failing/being timed out since. Each file I am trying to download is about 10 MB, it can take up to more than one hour for each download and I stay queued for several hours between two downloads.
This is a huge problem for people needing to use ERA5 and that have time constraints on their projects.
[Update]
It took days to download 9 of the 12 datasets at ~200kB/s, and 3 of them ended up expiring, so I have to start the process all over for those. I have a support request in to try to figure out what is going on. I’ll keep everyone posted on what I find out. The processing times are not a concern for me, it’s just the downloads once the datasets are ready.
I’m also experiencing very long queuing time before data downloads even begin, often in queues of more than 10,000, with only 300 requests running at a given time.
A request for 20 days of hourly ERA5 data for 6 variables, over only the New Zealand region, took over 17 hours to download (most of that time was spent sat in a queue).
Following up on this again. It appears they had some server issues that required them to reboot the systems. This was the response from ECMWF staff…
“During the weekend and start of the week, there has been some issues on different layers of the infrastructure affecting its normal functioning. Some components got stuck and had to be updated / rebooted .
Just for future information, you can track if there are issues with the infrastructure or checking the banners on CDS Portal or https://status.ecmwf.int/ (Data Stores).
Currently the number of data access slots have been decreased to decongestion network traffic. Situation should be better now.”
I am now able to download at speeds between 5-15MB/s, so I consider my issue resolved.
Hi all,
First of all really appreciate the work ECMWF does to make this data publicly availbale, I am sure its not an easy task. It is however getting to the point where queing times, both through the API or the web UI are forcing us to seriously consider other data providers. In this specific post, and many others on the forums there are variations on this question, on why its taking so long to get data from the cds.
So far the answers have been “this is temporary,” but, given my own experience and that of many other users, it hardly seems temporary. The que and the wait times are very long and have been very long for the last couple of months.
Is there a plan from cds on increasing speed in the long term? Or are the wait times over the last couple of months what can be expected in the near future? we are specifically wondering about ERA5 data.
I have been running a query to download 1 month of data over 4 ERA5 grid cells. It has been stuck in the que for 5 hours…
Thank you
I hit a new record, 17 hours for a month of 1 variable to be activated. It is unbearable. During some time it seemed that at night it was working better, Europe time, but now it’s the opposite, at night the queue stops, and if you are lucky next morning it restarts.
I can understand and accept that the migration is still on going, but at the very least, i would like to see some explanation of what is going on and when is expected to be solved. Not even an apology, just some real expectations.