More than 30 minutes to download less than 50Kb

Feedback and requirements
,
1

I’m downloading some data from ERA5 and it’s taking me so much time, I’d like to know if there’s someone who can explain to me why all this time to download a 45.5Kb data, or if I’m doing anything wrong or that may cause this delay to download my data.

# Dicionário com as coordenadas para cada sigla de capital
capitais = {
    # 'ac': [-9.97, -67.81, -9.98, -67.80],    # Rio Branco
    # 'al': [-9.65, -35.73, -9.66, -35.72],    # Maceió
    # 'ap': [0.03, -51.05, 0.02, -51.04],      # Macapá
    'am': [-3.11, -60.02, -3.12, -60.01],    # Manaus
    'ba': [-12.97, -38.50, -12.98, -38.49],  # Salvador
    'ce': [-3.72, -38.54, -3.73, -38.53],    # Fortaleza
    'df': [-15.78, -47.93, -15.79, -47.92],  # Brasília
    'es': [-20.32, -40.34, -20.33, -40.33],  # Vitória
    'go': [-16.68, -49.26, -16.69, -49.25],  # Goiânia
    'ma': [-2.53, -44.30, -2.54, -44.29],    # São Luís
    'mt': [-15.60, -56.10, -15.61, -56.09],  # Cuiabá
    'ms': [-20.47, -54.61, -20.48, -54.60],  # Campo Grande
    'mg': [-19.92, -43.94, -19.93, -43.93],  # Belo Horizonte
    'pa': [-1.46, -48.49, -1.47, -48.48],    # Belém
    'pb': [-7.12, -34.88, -7.13, -34.87],    # João Pessoa
    'pr': [-25.43, -49.27, -25.44, -49.26],  # Curitiba
    'pe': [-8.05, -34.88, -8.06, -34.87],    # Recife
    'pi': [-5.09, -42.80, -5.10, -42.79],    # Teresina
    'rj': [-22.91, -43.17, -22.92, -43.16],  # Rio de Janeiro
    'rn': [-5.79, -35.21, -5.80, -35.20],    # Natal
    'rs': [-30.03, -51.23, -30.04, -51.22],  # Porto Alegre
    'ro': [-8.76, -63.90, -8.77, -63.89],    # Porto Velho
    'rr': [2.82, -60.67, 2.81, -60.66],      # Boa Vista
    'sc': [-27.60, -48.55, -27.61, -48.54],  # Florianópolis
    'sp': [-23.55, -46.63, -23.56, -46.62],  # São Paulo
    'se': [-10.91, -37.07, -10.92, -37.06],  # Aracaju
    'to': [-10.18, -48.33, -10.19, -48.32]   # Palmas
}

# Cria pares de anos
anos = ['2000', '2001', '2002', '2003', '2004', '2005', '2006', '2007', '2008', '2009',
        '2010', '2011', '2012', '2013', '2014', '2015', '2016', '2017', '2018', '2019',
        '2020', '2021', '2022', '2023']

pares_anos = [anos[i:i + 2] for i in range(0, len(anos), 2)]

# Instancia o cliente da API
c = cdsapi.Client()

# Itera sobre as siglas e suas respectivas coordenadas
for sigla, area in capitais.items():
    # Itera sobre os pares de anos para baixar os dados
    for i, par in enumerate(pares_anos, 1):
        c.retrieve(
            'reanalysis-era5-single-levels',
            {
                'product_type': 'reanalysis',
                'format': 'netcdf',
                'variable': [
                    '2m_temperature',
                    'maximum_2m_temperature_since_previous_post_processing',
                    'minimum_2m_temperature_since_previous_post_processing',
                ],
                'year': par,
                'month': [
                    '01', '02', '03', '04', '05', '06',
                    '07', '08', '09', '10', '11', '12',
                ],
                'day': [
                    '01', '02', '03', '04', '05', '06',
                    '07', '08', '09', '10', '11', '12',
                    '13', '14', '15', '16', '17', '18',
                    '19', '20', '21', '22', '23', '24',
                    '25', '26', '27', '28', '29', '30',
                    '31',
                ],
                'time': ['07:00', '15:00',
                    '16:00', '17:00', '18:00', '19:00',
                ],
                'area': area,
            },
            f'/content/gdrive/MyDrive/Trabalhando dado/inputs/meteo/nc_pontuais/{sigla}_{i}.nc'
        )

imagem_2024-09-02_215258999
imagem_2024-09-02_2152589991830×37 4.44 KB

2

I wrote to the support recently and it seems that there’s a problem.
I have much difficulty too since a few weeks …

From their mail :
“There is an ongoing issue with the queueing on CDS-Beta,”

1 Like
3

You’re looping over areas, that will be pretty inefficient I think. In general, you want to loop over months (same tape), and make one request for as much data as possible, so in your case potentially requesting the whole Brazil area.

4

This was my first attempt, but when I take the whole Brazil area I can’t select all the
I need because the data becomes very large, unfortunately I wouldn’ have enough storage