Certificate verify failed while downloading ERA-5

Software and Tools API
, ,
1

Hello,

I got the following error while downloading ERA-5 on single level using the CDSAPI, 


DEBUG Starting new HTTPS connection (1): download-0004.copernicus-climate.eu:443

2021-11-23 04:06:03,467 WARNING Recovering from connection error [HTTPSConnectionPool(host='download-0004.copernicus-climate.eu', port=443): Max retries exceeded with url: /cache-compute-0004/cache/data5/adaptor.mars.internal-1637617040.5846336-21917-14-b10f56b0-079b-44f4-80cd-50f0b4307922.nc (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))], attemps 174 of 500

2021-11-23 04:06:03,468 WARNING Retrying in 120 seconds


Here is my python snippet

c = cdsapi.Client(timeout=600,quiet=False,debug=True)

c.retrieve('reanalysis-era5-single-levels', 

         {

        'product_type':'reanalysis',

        'variable':parm_name,

        'year':year,

        'month':month,


        'day':['01','02','03','04','05',

               '06','07','08','09','10',

               '11','12','13','14','15',

               '16','17','18','19','20',

               '21','22','23','24','25',

               '26','27','28','29','30','31'],


        'time':[

            '00:00','01:00','02:00',

            '03:00','04:00','05:00',

            '06:00','07:00','08:00',

            '09:00','10:00','11:00',

            '12:00','13:00','14:00',

            '15:00','16:00','17:00',

            '18:00','19:00','20:00',

            '21:00','22:00','23:00'],

        'format':'netcdf',

        'grid': [1,1], # latitude and longitude grid: east-west and north-south.

        'area': [90,-180,-90,180] # north, west, south, and east

        },

        diri_input+"%s_%s_%s.nc"%(parm_name,year,month))


Thanks 

2

Similar Error - host 0004 probably the issue

Hi guys, a couple of people have this or similar errors:

2021-11-23 10:55:33,907 WARNING Recovering from connection error [HTTPSConnectionPool(host='download-0004.copernicus-climate.eu', port=443): Max retries exceeded with url: /cache-compute-0004/cache/data8/adaptor.mars.internal-1637660534.464421-26094-10-a43a3f36-0a7b-4eed-bda2-7a54002af6ae.grib (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1125)')))], attemps 4 of 500

We tried to track down the error by changing the time, area, etc. For the same request, the error only appears when (host='download-0004.copernicus-climate.eu', port=443). It works when "download-0000.copernicus-climate.eu"

3

Thanks

just wondering how do we choose specific host  “download-0000.Copernicus-climate.edu” when using cdsapi

thanks

4

Im facing the same issue in dataset reanalysis-era5-single-levels-preliminary-back-extension

Checking pr 1978-01
Requesting total_precipitation 1978-01
2021-11-23 07:06:54,534 INFO Welcome to the CDS
2021-11-23 07:06:54,534 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels-preliminary-back-extension
2021-11-23 07:06:54,739 INFO Request is queued
2021-11-23 07:13:16,343 INFO Request is running
2021-11-23 07:15:17,197 INFO Request is completed
2021-11-23 07:15:17,198 INFO Downloading https://download-0004.copernicus-climate.eu/cache-compute-0004/cache/data3/adaptor.mars.internal-1637662433.8384476-20200-2-3222549f-0bbd-4856-acef-5c8103306a6b.nc to /mnt/cirrus/cr2met_prodution/data_folder_cr2met/ERA5/1h_temporal/pr/ERA5_pr_1h_1978_01_Chile_025deg.nc (13.9M)
2021-11-23 07:15:19,336 WARNING Recovering from connection error [HTTPSConnectionPool(host='download-0004.copernicus-climate.eu', port=443): Max retries exceeded with url: /cache-compute-0004/cache/data3/adaptor.mars.internal-1637662433.8384476-20200-2-3222549f-0bbd-4856-acef-5c8103306a6b.nc (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1108)')))], attemps 0 of 500
2021-11-23 07:15:19,338 WARNING Retrying in 120 seconds
2021-11-23 07:17:19,439 INFO Retrying now...
2021-11-23 07:17:19,839 WARNING Recovering from connection error [HTTPSConnectionPool(host='download-0004.copernicus-climate.eu', port=443): Max retries exceeded with url: /cache-compute-0004/cache/data3/adaptor.mars.internal-1637662433.8384476-20200-2-3222549f-0bbd-4856-acef-5c8103306a6b.nc (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1108)')))], attemps 1 of 500
2021-11-23 07:17:19,841 WARNING Retrying in 120 seconds


and it keep retrying and failing

5

Hi, I have a similar issue

any sugestion?


6

Same issue : downloading the resulting ncfile using cdsapi repeatedly crashes.

2021-11-23 09:49:09,140 WARNING Recovering from connection error [HTTPSConnectionPool(host='download-0010.copernicus-climate.eu', port=443): Max retries exceeded with url: /cache-compute-0010/cache/data7/adaptor.mars.internal-1637624939.6084557-25851-2-90faefb3-171f-4826-8b83-01e0968aef02.nc (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1108)')))], attemps 0 of 500

however the file exists (with internal name adaptor.mars.* ) is available for download in the myrequest page.


7

The automatic download works for:
Servers 0000, 0003

Error exists for (so far tested):
0002, 0004, 0008, 0010


8

How to change server in python script?

9

I see things have now restarted, someone must have acted appropriately. I hope this place is also the right place to thank support people !

10

The API and downloading processes have resumed working. Thanks a lot for the ECMWF and Copernicus staff for their support. 


11

I am still having this error and can't fix it! 

tried many things, but never worked!

please help


import cdsapi

def main():
c = cdsapi.Client(timeout=600,quiet=False,debug=True)

c.retrieve("reanalysis-era5-pressure-levels",
{
"variable": "temperature",
"pressure_level": "1000",
"product_type": "reanalysis",
"year": "2008",
"month": "01",
"day": "01",
"time": "12:00",
"format": "grib"
}, "download.grib")

print("downloaded")

main()



2021-11-28 16:17:54,382 DEBUG CDSAPI {'url': 'https://cds.climate.copernicus.eu/api/v2', 'key': '113136:fd0592f5-2163-4eaa-9876-cf9da251250d', 'quiet': False, 'verify': True, 'timeout': 600, 'progress': True, 'sleep_max': 120, 'retry_max': 500, 'full_stack': False, 'delete': True, 'metadata': None, 'forget': False}
2021-11-28 16:17:54,382 DEBUG Starting new HTTPS connection (1): cds.climate.copernicus.eu:443
2021-11-28 16:17:56,058 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels
2021-11-28 16:17:56,074 DEBUG POST https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels {"variable": "temperature", "pressure_level": "1000", "product_type": "reanalysis", "year": "2008", "month": "01", "day": "01", "time": "12:00", "format": "grib"}
2021-11-28 16:17:56,074 DEBUG Starting new HTTPS connection (2): cds.climate.copernicus.eu:443
2021-11-28 16:17:56,993 WARNING Recovering from connection error [HTTPSConnectionPool(host='cds.climate.copernicus.eu', port=443): Max retries exceeded with url: /api/v2/resources/reanalysis-era5-pressure-levels (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))], attemps 0 of 500