I'm trying to download daily averaged data using the toolbox. The code I'm using is at the bottom of the post. As a side note, I can use the api to get hourly data, it's when I use the toolbox that things break.
Thanks for any help!
--Brent
After I start the code it tells me that things are queued and then it fails with:
2023-04-17 18:31:07,036 INFO Welcome to the CDS
2023-04-17 18:31:07,037 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/tasks/services/tool/toolbox/orchestrator/workflow/clientid-805d8a96edb44aa7b209f7dfd30eda68
2023-04-17 18:31:07,215 INFO Request is queued
2023-04-17 18:31:58,379 INFO Request is failed
2023-04-17 18:31:58,379 ERROR Message: an internal error occurred processing your request
2023-04-17 18:31:58,379 ERROR Reason: Cmd('git') failed due to: exit code(128)
cmdline: git clone git@gitrepo:c3s/era5.git /home/cds/compute_workflows/c3s/era5/master
stdout: 'Cloning into '/home/cds/compute_workflows/c3s/era5/master'...'
stderr: 'Warning: Permanently added the RSA host key for IP address '192.168.0.248' to the list of known hosts.
GitLab: The project you were looking for could not be found.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.'
2023-04-17 18:31:58,379 ERROR Traceback (most recent call last):
2023-04-17 18:31:58,379 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/cdshandlers/services/handler.py", line 59, in handle_request
2023-04-17 18:31:58,379 ERROR result = cached(context.method, proc, context, context.args, context.kwargs)
2023-04-17 18:31:58,380 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/caching.py", line 108, in cached
2023-04-17 18:31:58,380 ERROR result = proc(context, *context.args, **context.kwargs)
2023-04-17 18:31:58,380 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 124, in __call__
2023-04-17 18:31:58,380 ERROR return p(*args, **kwargs)
2023-04-17 18:31:58,380 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 60, in __call__
2023-04-17 18:31:58,380 ERROR return self.proc(context, *args, **kwargs)
2023-04-17 18:31:58,380 ERROR File "/home/cds/cdsservices/services/workflow.py", line 26, in execute
2023-04-17 18:31:58,380 ERROR cacheurl=cacheurl)
2023-04-17 18:31:58,380 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/workflow.py", line 56, in submit
2023-04-17 18:31:58,380 ERROR gitcache.ensure_repo_version(local_repos_path, remote_repo_url, params['version'])
2023-04-17 18:31:58,380 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/gitcache.py", line 38, in ensure_repo_version
2023-04-17 18:31:58,380 ERROR fetch_repo_version(remote_repo_url, local_repo_path, version)
2023-04-17 18:31:58,380 ERROR File "/opt/cdstoolbox/cdscompute/cdscompute/gitcache.py", line 20, in fetch_repo_version
2023-04-17 18:31:58,380 ERROR git.Git().clone(remote_repo_url, local_repo_path)
2023-04-17 18:31:58,380 ERROR File "/usr/local/lib/python3.6/site-packages/git/cmd.py", line 548, in <lambda>
2023-04-17 18:31:58,380 ERROR return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
2023-04-17 18:31:58,380 ERROR File "/usr/local/lib/python3.6/site-packages/git/cmd.py", line 1014, in _call_process
2023-04-17 18:31:58,380 ERROR return self.execute(call, **exec_kwargs)
2023-04-17 18:31:58,381 ERROR File "/usr/local/lib/python3.6/site-packages/git/cmd.py", line 825, in execute
2023-04-17 18:31:58,381 ERROR raise GitCommandError(command, status, stderr_value, stdout_value)
2023-04-17 18:31:58,381 ERROR git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
2023-04-17 18:31:58,381 ERROR cmdline: git clone git@gitrepo:c3s/era5.git /home/cds/compute_workflows/c3s/era5/master
2023-04-17 18:31:58,381 ERROR stdout: 'Cloning into '/home/cds/compute_workflows/c3s/era5/master'...'
2023-04-17 18:31:58,381 ERROR stderr: 'Warning: Permanently added the RSA host key for IP address '192.168.0.248' to the list of known hosts.
2023-04-17 18:31:58,381 ERROR GitLab: The project you were looking for could not be found.
2023-04-17 18:31:58,381 ERROR fatal: Could not read from remote repository.
Traceback (most recent call last):
File "/Users/data/era5/get_daily_data.py", line 5, in <module>
result = c.service("tool.toolbox.orchestrator.workflow",
File "/usr/local/lib/python3.9/site-packages/cdsapi/api.py", line 382, in service
result = self._api(
File "/usr/local/lib/python3.9/site-packages/cdsapi/api.py", line 519, in _api
raise Exception(
Exception: an internal error occurred processing your request. Cmd('git') failed due to: exit code(128)
cmdline: git clone git@gitrepo:c3s/era5.git /home/cds/compute_workflows/c3s/era5/master
stdout: 'Cloning into '/home/cds/compute_workflows/c3s/era5/master'...'
stderr: 'Warning: Permanently added the RSA host key for IP address '192.168.0.248' to the list of known hosts.
GitLab: The project you were looking for could not be found.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.'.
____________________________ My code below _________________
Using the following code:
import cdsapi
c = cdsapi.Client()
MONTHS = ["01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12"]
for month in MONTHS:
result = c.service("tool.toolbox.orchestrator.workflow",
params= {"realm": "c3s", "project": "era5", "version": "master",
"kwargs": {"dataset": "reanalysis-era5-single-levels", "product_type": "reanalysis","variable": "2m_temperature","statistic": "daily_mean","year": "2020","month": month,"time_zone": "UTC+00:0","frequency": "1-hourly","grid": "2.5/2.5","area": {"lat": [-90, 90], "lon": [-180, 180]}
},
"workflow_name": "application"
})
c.download(result)