Error "Conection broken" downloading ERA5 hourly data on single levels

Software and Tools API
,
1

Hi everyone.

I been downloading data in block of months to ERA5 hourly data on single levels with the API, but sometimes the connection is broken and in the terminal shows the next error message, but is not my internet connection:

requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(35463808 bytes read, 9526290 more expected)', Incomp
leteRead(35463808 bytes read, 9526290 more expected))

I can download the information retry the solicitude but is annoying because i have to download in sets of months for many years and every 2 or 3 month files the connections broken.

Anyone has the same problem?

Thanks.

2

Hi Eduardo,
I’m encountering a similar issue. Did you manage to resolve it?
Best, Fei

3

Hello,
I also would like to know if any of you has find an answer to this problem.
I’m having the same issue.

Thanks

4

Hi, Tamires and Fei.

I did not find a solution. I just put a sleep line of 15 second between each request and sometimes it works, and other times it doesn’t. I think it depends of infrestructure of the system.

This is my python code to do request monthly. You could use it, just change the variables and area that you are working with.

################################################################

import cdsapi
import os
import time

################################################################
# Begin user input

year_ini: int = 1966
month_ini: int = 3

year_final: int = 1970
month_final: int = 1

lon_min: int = -99
lon_max: int = -73
lat_min: int = 12
lat_max: int = 35

################################################################
# End user input

print("--- Comienza proceso para descarga de datos de ERA5 ---")

for year in range(year_ini, year_final + 1):

    if year == year_ini and year_ini != year_final:
        month_i: int = month_ini
        month_f: int = 12

    if year != year_ini and year != year_final:
        month_i: int = 1
        month_f: int = 12

    if year == year_final and year_ini != year_final:
        month_i: int = 1
        month_f: int = month_final

    if year == year_final and year_ini == year_final:
        month_i: int = month_ini
        month_f: int = month_final

    year = str(year)
    year_i = year

    for month in range(month_i, month_f + 1):

        month = str(month)
        file_name: str = f"{year}_{month}.nc"

        print("-> Inicia descarga de datos de viento")
        print("\t Año: " + year)
        print("\t Mes: " + month)
        print(" ")

        dataset = "reanalysis-era5-single-levels"
        request = {
            "product_type": ["reanalysis"],
            "variable": [
                "10m_u_component_of_wind",
                "10m_v_component_of_wind",
                "surface_pressure"
            ],
            "year": [year],
            "month": [month],
            "day": [
                "01", "02", "03",
                "04", "05", "06",
                "07", "08", "09",
                "10", "11", "12",
                "13", "14", "15",
                "16", "17", "18",
                "19", "20", "21",
                "22", "23", "24",
                "25", "26", "27",
                "28", "29", "30",
                "31"
            ],
            "time": [
                "00:00", "01:00", "02:00",
                "03:00", "04:00", "05:00",
                "06:00", "07:00", "08:00",
                "09:00", "10:00", "11:00",
                "12:00", "13:00", "14:00",
                "15:00", "16:00", "17:00",
                "18:00", "19:00", "20:00",
                "21:00", "22:00", "23:00"
            ],
            "data_format": "netcdf",
            "download_format": "unarchived",
            "area": [lat_max, lon_min, lat_min, lon_max]
        }

        client = cdsapi.Client()
        client.retrieve(dataset, request,file_name)
	
        time.sleep(15)

        print(" ")
        print("-> Descarga finalizada")
        print("-> Archivo descargado: " + file_name)
        print(" ")
1 Like
5

Hello,
I am having the same issue, on the AgERA5 dataset (“Agrometeorological indicators from 1979 to present derived from reanalysis”). Did not find a workaround so far !

6

Thank you, Eduardo
I tried your suggestion and it worked!