What is the time period after a data request that is successful expires?
Hi
Don’t know if this is the right place …
I’ve done the transition and can use the scripts we used before - but - the format of the downloaded NetCDF file is quite different from the old one.
I can see that the version of the grib to NetCDF program has been changed: fromgrib_to_netcdf-2.28.1 to cfgrib-0.9.14.0/ecCodes-2.36.0 .
This has a least the following impacts:
- The time units changed from “hours since 1900-01-01 00:00:00.0” to “seconds since 1970-01-01”
- The coordinate attribute of variables changed from not being included to string u10:coordinates = “number valid_time latitude longitude expver”
The first point might be solved by libraries - but xarray chokes on the latter as the variables are defined by 3 dimensions e.g. float u10(valid_time, latitude, longitude) - similar to the old version (but with different naming).
Should the coordinates attribute and the number of dimensions not agree?
Reading the new version with xarray results in:
…/python3.12/site-packages/xarray/core/concat.py
xarray.core.merge.MergeError: conflicting values for variable ‘expver’ on objects to be combined. You can skip this check by specifying compat=‘override’.
if I try specifying compat=‘override’ I get:
numpy._core._exceptions._ArrayMemoryError: Unable to allocate 27.5 GiB for an array with shape (17544, 3, 2, 35066) and data type float64
where the shape somehow does not at all match the field I want to read.
Am I missing something? Using a wrong version or?
Best regards,
Karsten
2 Likes
Hi @Anabelle , it certainly helps! Do you know if the feature will be available before the discontinuation of the current API and toolbox? otherwise I’ll have to find a temporary solution…
Best,
Joao
They deployed a new .netcdf builder and it seems to have a bunch of problems (or differences). We’ve switched to the .grib, seems more stable.
Hi @Anabelle ,
I don’t find any api key or url in my beta login. How do i create the .cdsapirc file? The installation documentation addresses the older version. I am confused.
Hello @Anabelle, I am doing the same observation, is that the expected behaviour ? Is this the new standard for the data format ? Is it going to change any further ? Is it documented somewhere ?
1 Like
I noticed the same. Also @Anabelle I’ve noticed that the same request with the new api is rounding the metric result to 4 decimal places whereas with the old api I get the full amount of decimal places. How can I switch this off/change this?
The informative page appears to be hosted on confluence, which at least at this moment, trying to load from California in the USA, does not work. Confluence just dies and fails to serve anything but a blank page. Not sure why, maybe it’s temporary. I might suggest also making a static export of that page available as a regular web page?
Hi Anusha - I just got this to work, so see if this helps:
- Create account on ECMWF site.
- Create new beta account on CDS site.
- On the CDS site, in the upper right, under your username, hover and in the dropdown it should say “Profile”.
- Under profile, scroll down and look for “Personal Access Token”. Copy it.
- Locally (this is on Mac), create/edit the ~/.cdsapirc file. You can do this in any text editor.
- Add this to the file:
url: https://cds-beta.climate.copernicus.eu/api
key: <personal_access_token_pasted_value>
- Save it
- Try and hit the api with that. If you are pulling ERA data, you may see an error indicating that you have to accept the license - in which case you have to go back to the UI, find the ERA data page and click the “Accept” button for the terms and conditions.
1 Like
Migration date to CDS-Beta has been extended to 26 September 2024 but this is final! 
We posted a separate announcement earlier this morning: Decommissioning of CDS and ADS on September 26, 2024 - Final
Thank you so much Victor… this is another case where I can’t believe anybody has actually ever gotten something to work without added input like yours. As an IT novice I find that people who write help files make a lot of assumptions about the blanks that users will be able to fill in on their own.
Hi @Anabelle , I hope you are doing great!
Do you have any news on the daily means feature?
1 Like
Dear Victor,
thank you for your guidelines. I’m looking for the “Accept” buttom, but I’m not finding the correct webpage. Can you attach here the link?
Best Regards
Salvatore
Also I’ve noticed this. First the maximum number of requests - 250 means that requests are queuing for a long time and now we’re limited to 2 per user requests which also means that when I need to get multiple metrics at once, I’ll be queuing for a long time.
How can this be more restrictive than the old api? Will these restrictions be removed when the CDS is moved from beta?
Does anyone know where to sign the license?
CDS-beta API keeps returning this error message:
required licences not accepted; please accept the following licences to proceed: [{'id': 'licence-to-use-copernicus-products', 'revision': 12}]
I thought it was referring to this document below,
https://apps.ecmwf.int/datasets/licences/copernicus/
I have accepted the term but it does not solve the issue.
Can anyone help with this? Thanks in advance.
Thanks a lot Victor. That’s the most helpful answer and instruction I found on this webpage. Normally my code was running perfectly fine with CDS. However now having the CDS-Beta, if I run the same code I get python error messages like attrs module is missing. I tried to install and re-install attrs modules, but that did not help me to make the beta version work. switching the url inside .cdsapirc file to the old CDS, make everything works again as normal. Do you have any suggestions how I can solve this problem?
I posted Tutorial on Using the CDS-beta with PyCharm on Windows. In this post I published the revised version of the official Observing floods with GloFAS tutorial (link).
Compare the the part of the revised code followed by download_file = f"{DATADIR}/glofas-2012_2024.grib" related to data download from the new CDS-beta with original and you can see what you should change in your own scripts.
Hi @Anabelle, I am also using the toolbox to download the ERA5 daily means, look forward to hearing from you about the new feature!!!