Hi,
After getting errors while concatenating geographical subsets from CARRA-East domain, I noticed the grid has changed for gribfiles in 2023 and after, compared to 2022 and before.
This seems to coincide with the introduction of the timely update every month?
Does anyone have more information on this? Why was it changed? How exactly was it changed? Potential impact on orography and dataset?
Regards,
Jules
Hi Jules,
Thanks for your comment, it seems very strange. I am sure that there was no any intentional change, when the timely update started on our new supercomputer. Would you give a bit of more details, please? What is exactly changed? Is that the metadata or the actual geographic values?
I am in contact with the CARRA team to check this, but it would be nice to have some more information.
Thank you, best regards
Andras
Hi Andras,
It seems to me the actual geographic values of latitude and longitude changed slightly. Not much, but enough to select one more row of data when subsetting (geographically) with the exact same filter which when triggers errors when trying to concatenate the smaller area files together.
I am including screenshots showing the difference between the latitudes/longitude from 2019 and 2023 files. The topleft corner of my lat/lon tables are the same and as I go towards bottom right values start to differ slightly (see screenshots), which makes me think there was a small rotation of the grid from one corner of the CARRA-East domain.
Was that done to align to the grid of Arome-Arctic operational model grid?
I used python with xarray and numpy in spyder IDE. This is the code I used to reach this conclusion:
import xarray as xr
import numpy as np
file_SW_grib = [‘DATA/CARRA/SW_surf/CARRA-East_SW_surfForecast123_2024til09.grib’,
‘DATA/CARRA/SW_surf/CARRA-East_SW_surfForecast123_2019.grib’,
‘DATA/CARRA/SW_surf/CARRA-East_SW_surfForecast123_2023.grib’]
grib_lat = {};grib_lon = {}
for fil in file_SW_grib: # fil = file_SW_grib[0]
yearcur = fil.split(“_”)[-1].split(“.grib”)[0]
ds_tmp = xr.open_dataset(fil, chunks={“time”: 5})
grib_lat[yearcur] = ds_tmp.latitude.compute().load().values
grib_lon[yearcur] = ds_tmp.longitude.compute().load().values
Hi Jules,
Many thanks for the additional details. I am forwarding this to the CARRA team to cross-check and for feedbacks.
I confirm that there was no any intention to change the grid and it could have happened only by accident when the CARRA system was ported to our new supercomputer. It seems there is similar issue for the CARRA-West domain too.
In principle, the migration to a new computer platform should not cause such changes in the grid (the meteorological fields were validated in the new platform with respect to the previous one).
I cannot give you now a definite answer apart of the fact that I am speaking to the CARRA team to check this and provide explanation.
I hope to come back to you early next week with some explanations.
Thanks again, best regards
Andras
Hi Andras,
Were you able to get more information on this?
Regards,
Jules
Hi Jules,
I am sorry not giving any more news. Unfortunately, I don’t have any substantial new information on top of the fact that the CARRA grid in the new ECMWF computer is slightly different than for the old one. The change happens for the data from July 2021 onwards.
I asked the CARRA team again and I hope we will be able to understand the details of the problem, though I don’t think that we would create new GRIB files from July 2021 onwards. Certainly, we would document the issue though.
Please let me know how this grid change influences your work with the CARRA dataset and if you can find a solution to use the pre-2021July and post-2021July data together.
Sorry again for the delay!
Thank you, best regards
Andras
I only had noticed difference from 2023 onward (I have downloaded yearly files with radiation fluxes on the surface for East domain).
What I have done is interpolate the “new” files on the “old” grid with cdo.
I guess in changing the grid there is an impact on orography which can in turn impact the fields values but I have not looked into this. Could that be documented as well in this issue?
Regards
Jules
Thanks Jules, I agree the interpolation looks a good idea, but we need to understand what that would mean, if the orography is also influenced. We will try to document the issue as much as we can and we will consult with you once we know the problem more precisely.
Hi Jules,
Just to let you know that we had a discussion with the CARRA team and indeed we need to understand the problem better and document it afterwards. This study would include to look at orography, land-sea mask and similar fixed fields too.
There is another related issue reported in this Forum: Carra snow depth variable changes level type at midnight june 30th 2022
Contrary, what I said above the change happens for data July 2022 (not 2021) onwards.
Thanks, best regards
Andras
Hi Jules,
I am sorry for the long silence. Meanwhile, I got some information from the CARRA team regarding the issue what you have indicated.
We think that the problem is properly understood now and the main conclusion is that the change of the grid is a metadata issue and not a problem related to the model.
This means that after 1st of July 2022 the CARRA data files contain wrong grid information and the grid before 1st of July 2022 is the right one. Therefore, if you wish to use the data through this date then you need to take into account the grid information prior to 1st if July.
We are going to describe this problem in the known issues part of the documentation in the following way:
Small changes in the dataset after 1st July 2022 - after the move of the CARRA production from the HPC in Reading to that in Bologna (1 July 2022 in the produced data set), there are two unintended changes to the dataset:
I hope, it helps, sorry for the problem and thanks for pointing to it.
Best regards
Andras
