Unexpected behavior CDS API - 6 variables (excluding dimension variables)

Datasets and Usage C3S - Datasets and Usage
, ,
1

Hello,


I'm experiencing a strange (in my opinion) behavior while download data in netCDF format using the CDS API.


Here is how I'm trying to request the data from Python:


import cdsapi

c = cdsapi.Client()

c.retrieve(
'reanalysis-era5-single-levels',
{
'product_type': 'reanalysis',
'variable': [
'100m_u_component_of_wind', '100m_v_component_of_wind', '2m_temperature',
],
'year': '2019',
'month': [
'06', '07', '08',
'09', '10', '11',
'12',
],
'day': [
'01', '02', '03',
'04', '05', '06',
'07', '08', '09',
'10', '11', '12',
'13', '14', '15',
'16', '17', '18',
'19', '20', '21',
'22', '23', '24',
'25', '26', '27',
'28', '29', '30',
'31',
],
'time': [
'00:00', '01:00', '02:00',
'03:00', '04:00', '05:00',
'06:00', '07:00', '08:00',
'09:00', '10:00', '11:00',
'12:00', '13:00', '14:00',
'15:00', '16:00', '17:00',
'18:00', '19:00', '20:00',
'21:00', '22:00', '23:00',
],
'format': 'netcdf',
'grid': '0.25/0.25',
'area': '0/-48/-18/-34'
},
'some_name.nc')

I can download the netcdf without any problems, but as soon as I try to read the file, that's what I get:


6 variables (excluding dimension variables):


short u100_0001[longitude,latitude,time]
scale_factor: 0.000388201867135403
add_offset: -3.75457337339451
_FillValue: -32767
missing_value: -32767
units: m s**-1
long_name: 100 metre U wind component


short v100_0001[longitude,latitude,time]
scale_factor: 0.000463298542471565
add_offset: 1.89131650746705
_FillValue: -32767
missing_value: -32767
units: m s**-1
long_name: 100 metre V wind component


short t2m_0001[longitude,latitude,time]
scale_factor: 0.000533161876697618
add_offset: 297.217384786249
_FillValue: -32767
missing_value: -32767
units: K
long_name: 2 metre temperature


short u100_0005[longitude,latitude,time]
scale_factor: 0.000424499730589574
add_offset: -1.08838516246295
_FillValue: -32767
missing_value: -32767
units: m s**-1
long_name: 100 metre U wind component


short v100_0005[longitude,latitude,time]
scale_factor: 0.000422448398716463
add_offset: 0.298792376874861
_FillValue: -32767
missing_value: -32767
units: m s**-1
long_name: 100 metre V wind component


short t2m_0005[longitude,latitude,time]
scale_factor: 0.000411618716905986
add_offset: 300.820839112517
_FillValue: -32767
missing_value: -32767
units: K
long_name: 2 metre temperature


The way I did my script to treat this data, I always looked for the short name as v100, u100 and t2m, and it worked until today, butnow, I cannot use my script anymore.

So, in summary, my questions are:


1) Why is this happening now? 

2) If this is an expected behavior, what does it mean? What's the difference between v100_0001 and v100_0005?

3) If this is not an expected behavior, is there any work round it?


Thanks in advance!



2

Dear Matheus,

Thank you for reporting this, which we are aware of. It is caused by the recent release of ERA5T which fills the gap for the past 2-3 months of gap of ERA5. Please have a look at Release of ERA5T.

We were trying to make the release of ERA5T transparent to users. That's why when you request ERA5 data you can simply set day to five days ago. If you download ERA5 and ERA5T data in the GRIB format, you will be able to use the 'grib_ls' command from ECMWF's ecCodes to check 'expver', which will show '0001' for ERA5 and '0005' for ERA5T. Unfortunately, when you download the data in the NetCDF format, the converter being used will handle the data in the way you observed. We are in discussion with our colleagues in development and hopefully a solution will be found in the near future.

Kind regards,

Xiaobo

3

Dear Xiaobo,


Thanks for your answer, now I have understood the issue. I still have one question though:


If I need to get the data in NetCDF format, can I simply look for v100_0001 and get the same data structure as before?

And if I look for v100_0005, I will get the ERA5T instead?


As you only talked about .grid format, I still have this doubt.

If this is the problem, a simple change in my script could handle it easily. 


Thanks again!

4

Dear Matheus,

Unfortunately this is not always the case. You could get variables names as v100_0001 and v100_0005. You cold also have names like p0001 and p0005. So at the moment I don't think there is a simple way to handle this.

I want to confirm that I have escalated this issue to our development team although I cannot tell when this can be fixed.

Kind regards,

Xiaobo

5

Dear Xiaobo,


Understood. I will probably change my scripts to get and treat .grib files then.

Thanks for your answer! 


Best regards,

6

You are most welcome, Matheus.