SSL errors when retrieving ERA5 data using CDS API

Software and Tools API
,
1

Hello,

I’m having some trouble downloading ERA5 data using the CDS API. At the beginning of the downloading process, I get the following warning:

UserWarning: HTTPSConnectionPool(host=‘cds.climate.copernicus.eu’, port=443): Max retries exceeded with url: /api/catalogue/v1/messages (Caused by SSLError(SSLEOFError(8, ‘[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1028)’)))

But then the download process proceeds normally, going through my script to download a number of files successfully. Eventually, however, the script crashes with the following error message:

requests.exceptions.SSLError: HTTPSConnectionPool(host=‘cds.climate.copernicus.eu’, port=443): Max retries exceeded with url: /api/retrieve/v1/jobs/a77e87de-0319-4b10-91aa-db09f857659e?log=True&request=True (Caused by SSLError(SSLEOFError(8, ‘[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1028)’)))

Do you have any suggestions for how to prevent this from happening? It’s strange because I am sometimes able to download successfully; the error seems to happen at random. I’m using the latest version of cdsapi, and increasing the ‘timeout’ parameter does not help.

2

I had this frustrating issue for the past few days, I could hardly download anything at all after an upgrade of the HPC system. What (seems to have) fixed it was to add have this conda environment, conda create -n cds-env python=3.11 "openssl<3.3" requests=2.31 urllib3=1.26 cdsapi – I think the main issue was the openssl which was newer on the system and (somehow) seems to have caused issues; therefore maybe the ‘requests’ and ‘urllib3’ bits aren’t even important or relevant. But maybe this is helpful for others.