Hi all, I just migrated from old CDSAPI to the latest cdsapi-0.7.5 version, and the codes were tested good in downloading 2-d data from “reanalysis-era5-single-levels”, and the 2-d data is about 30M. However, when I tried to get 3-d “reanalysis-era5-pressure-levels” data, which is 529M, the downloading process cannot be finished, and the downloading speed was getting more and more slowly (at beginning it was quite fast). It seems that the 3-d data has been produced successfully, because I can successfully get it manually by clicking the “download” button at " Your requests" webpage. The version of Python is v3.13. I got error massages as below. Thanks a lot.
Gary
Jan 12, 2025
…
2025-01-12 10:44:51,941 INFO status has been updated to accepted
2025-01-12 10:44:57,659 INFO status has been updated to running
2025-01-12 10:49:13,477 INFO status has been updated to successful
f42e4956a66e667ca308ec48d262c832.grib: 0%| | 0.00/529M [00:00<?, ?B/s]
f42e4956a66e667ca308ec48d262c832.grib: 0%| | 1.00M/529M [00:02<19:48, 465kB/s]
f42e4956a66e667ca308ec48d262c832.grib: 1%| | 4.00M/529M [00:03<06:26, 1.42MB/s]
…
f42e4956a66e667ca308ec48d262c832.grib: 3%|▎ | 14.0M/529M [00:15<13:16, 678kB/s]
f42e4956a66e667ca308ec48d262c832.grib: 3%|▎ | 15.0M/529M [00:17<14:48, 606kB/s]
…
f42e4956a66e667ca308ec48d262c832.grib: 4%|▎ | 19.0M/529M [00:29<23:08, 385kB/s]
f42e4956a66e667ca308ec48d262c832.grib: 4%|▍ | 20.0M/529M [00:33<25:11, 353kB/s]
f42e4956a66e667ca308ec48d262c832.grib: 4%|▍ | 23.0M/529M [00:46<33:32, 263kB/s]
f42e4956a66e667ca308ec48d262c832.grib: 5%|▍ | 24.0M/529M [00:51<35:17, 250kB/s]
…
f42e4956a66e667ca308ec48d262c832.grib: 8%|▊ | 41.0M/529M [05:41<4:11:10, 33.9kB/s]
f42e4956a66e667ca308ec48d262c832.grib: 8%|▊ | 42.0M/529M [06:47<4:35:30, 30.9kB/s]
f42e4956a66e667ca308ec48d262c832.grib: 8%|▊ | 43.0M/529M [07:38<5:01:31, 28.1kB/s]
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\urllib3\response.py”, line 754, in error_catcher
yield
File "C:\Users\gary\AppData\Local\Programs\Python\Python313\Lib\site-packages\urllib3\response.py", line 900, in _raw_read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
urllib3.exceptions.IncompleteRead: IncompleteRead(45288768 bytes read, 508952256 more expected)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\requests\models.py”, line 820, in generate
yield from self.raw.stream(chunk_size, decode_content=True)
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\urllib3\response.py”, line 1066, in stream
data = self.read(amt=amt, decode_content=decode_content)
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\urllib3\response.py”, line 983, in read
data = self.raw_read(amt)
File "C:\Users\gary\AppData\Local\Programs\Python\Python313\Lib\site-packages\urllib3\response.py", line 878, in raw_read
with self.error_catcher():
File "C:\Users\gary\AppData\Local\Programs\Python\Python313\Lib\contextlib.py", line 162, in exit
self.gen.throw(value)
File "C:\Users\gary\AppData\Local\Programs\Python\Python313\Lib\site-packages\urllib3\response.py", line 778, in _error_catcher
raise ProtocolError(arg, e) from e
urllib3.exceptions.ProtocolError: (‘Connection broken: IncompleteRead(45288768 bytes read, 508952256 more expected)’, IncompleteRead(45288768 bytes read, 508952256 more expected))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “E:\DATA_of_lxg\DATA_ERA5_newCDS_API\newCDS_3D_everyday_forCYCLE_v1_202412.py”, line 83, in
client.retrieve(dataset, request, target)
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\datapi\legacy_api_client.py”, line 179, in retrieve
return submitted if target is None else submitted.download(target)
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\datapi\processing.py”, line 659, in download
robust_download(url, target)
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\multiurl\http.py”, line 479, in wrapped
r = call(main_url, *args, **kwargs)
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\datapi\processing.py”, line 625, in download
multiurl.download(
File "C:\Users\gary\AppData\Local\Programs\Python\Python313\Lib\site-packages\multiurl\downloader.py", line 111, in download
return Downloader(url, **kwargs).download(target)
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\multiurl\base.py”, line 129, in download
total = self.transfer(f, pbar)
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\multiurl\http.py”, line 133, in transfer
for chunk in stream(chunk_size=self.chunk_size):
File “C:\Users\gary_\AppData\Local\Programs\Python\Python313\Lib\site-packages\requests\models.py”, line 822, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: (‘Connection broken: IncompleteRead(45288768 bytes read, 508952256 more expected)’, IncompleteRead(45288768 bytes read, 508952256 more expected))