# Processing ICMUA output files **URL:** \ **Category:** OpenIFS\ **Tags:** openifs-configuration-and-namelists, openifs-43r3, openifs-model-output\ **Created:** [16 July 2019 08:34 UTC](https://forum.ecmwf.int/t/processing-icmua-output-files/12132 "2019-07-16T08:34:58Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Jan\_Streffing](https://forum.ecmwf.int/letter_avatar_proxy/v4/letter/j/d26b3c/32.png) [@Jan\_Streffing](https://forum.ecmwf.int/u/Jan_Streffing)\ **Post date:** [16 July 2019 08:34 UTC](https://forum.ecmwf.int/t/processing-icmua-output-files/12132/1 "2019-07-16T08:34:58Z")
Good morning, I'm running cy43r3 and I would like to process the ICMUA output files. The new files contain a mix of grib1 and grib2 fields that is not easly processable by the cdo provided our computing center: ``` cdo -R -t ecmwf -f nc copy -remapbil,r360x180 ICMUA4317+000000 ICMUA4317+000000.nc cdo(2) remapbil: Process started Segmentation fault (core dumped) grib\_ls ICMUA4317+000000 ICMUA4317+000000 edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType 1 ecmf isobaricInhPa 1000 20160925 0 fc q grid\_simple reduced\_gg 1 ecmf isobaricInhPa 925 20160925 0 fc q grid\_simple reduced\_gg 1 ecmf isobaricInhPa 850 20160925 0 fc q grid\_simple reduced\_gg 1 ecmf isobaricInhPa 700 20160925 0 fc q grid\_simple reduced\_gg . . . 2 ecmf hybrid 57 20160925 0 fc vtpha grid\_simple reduced\_gg 2 ecmf hybrid 58 20160925 0 fc vtpha grid\_simple reduced\_gg 2 ecmf hybrid 59 20160925 0 fc vtpha grid\_simple reduced\_gg 2 ecmf hybrid 60 20160925 0 fc vtpha grid\_simple reduced\_gg 360 of 360 grib messages in ICMUA4317+000000\_grib2 360 of 360 total grib messages in 1 files ``` I tried to split the file into grib1 and grib2 fields. The grib1 file looks fine and can be processed by cdo, but the grib2 file has the gridSize changed (maybe?) which seems to prevent further post processing: ``` grib_copy ICMUA4317+000000 -w edition=1 ICMUA4317+000000_grib1 grib_copy ICMUA4317+000000 -w edition=2 ICMUA4317+000000_grib2 grib\_ls ICMUA4317+000000\_grib2 ICMUA4317+000000\_grib2 edition centre date dataType gridType stepRange typeOfLevel level shortName packingType 2 ecmf 20160925 fc reduced\_gg 0 hybrid 1 q grid\_simple 2 ecmf 20160925 fc reduced\_gg 0 hybrid 2 q grid\_simple 2 ecmf 20160925 fc reduced\_gg 0 hybrid 3 q grid\_simple 2 ecmf 20160925 fc reduced\_gg 0 hybrid 4 q grid\_simple . . . cdo -R -t ecmwf -f nc copy -remapbil,r360x180 ICMUA4317+000000\_grib2 ICMUA4317+000000.nc cdo(2) remapbil: Process started Error (gribapiGetGridRegular): numberOfPoints (35718) and gridSize (343597383520) differ! ``` I tried to convert the mixed files with grib\_to\_netcdf but here I get an error related to the gridType: ``` grib_to_netcdf -T -o ICMUA4317+000000.nc ICMUA4317+000000 grib_to_netcdf: Version 2.10.0 grib_to_netcdf: Processing input file 'ICMUA4317+000000'. grib_to_netcdf: Found 489 GRIB fields in 1 file. grib_to_netcdf: Ignoring key(s): method, type, stream, refdate, hdate grib_to_netcdf: Creating netCDF file 'ICMUA4317+000000.nc' grib_to_netcdf: NetCDF library version: 4.3.3.1 of Dec 10 2015 16:44:18 $ grib_to_netcdf: Creating large (64 bit) file format. ECCODES ERROR : First GRIB is not on a regular lat/lon grid or on a regular Gaussian grid. Exiting. ``` How to I get my hands on the sweet data inside this file? ![(wink)](https://confluence.ecmwf.int/s/-ol1tgb/9012/1962nzv/_/images/icons/emoticons/wink.svg) Cheers, Jan ---
**Author:** ![Jan\_Streffing](https://forum.ecmwf.int/letter_avatar_proxy/v4/letter/j/d26b3c/32.png) [@Jan\_Streffing](https://forum.ecmwf.int/u/Jan_Streffing)\ **Post date:** [24 July 2019 10:45 UTC](https://forum.ecmwf.int/t/processing-icmua-output-files/12132/2 "2019-07-24T10:45:57Z")
Problem solved. For posterity: In the namelist that came with the input files, 3D output on both pressure and surface levels was turned on. Pressure level output is written in grib1 and surface level output written in grib2. Resulting in the unprocessable files I described. I just turned off the surface level output. I also deactivated the K-Index(260121) and Total totals index (260123) that were written as grib2 into the 2D output file. Cheers, Jan ---
**Author:** ![Glenn\_Carver](https://forum.ecmwf.int/letter_avatar_proxy/v4/letter/g/e495f1/32.png) [@Glenn\_Carver](https://forum.ecmwf.int/u/Glenn_Carver)\ **Post date:** [25 July 2019 11:24 UTC](https://forum.ecmwf.int/t/processing-icmua-output-files/12132/3 "2019-07-25T11:24:38Z")
Hi Jan, Sorry, I missed this page earlier. You are a bit ahead of the curve, but my understanding was that the ICM\*UA+\* output file for 43r3 was just a rename of the 40r1 ICM\*GG+\* file, so the contents should be the same and can be treated the same way, though some of the grib keys might have changed (for the new cubic grid for example). I think it's the other way around, the surface fields are GRIB1 and the pressure level output is GRIB2. Only GRIB2 can encode greater than 127 levels so it must be that way around. I know that cdo can choke with there are multiple level types involved. The surface fields include the sub-surface levels and mixing this with the pressure level fields in one file seems to confuse cdo.  The recommendation is to split the level types first (see [How to convert GRIB to netCDF](https://confluence.ecmwf.int/display/OIFS/How+to+convert+GRIB+to+netCDF) for much more detail. You can split the vertical axes in two ways. Either using cdo: ``` cdo splitzaxis ICMSHg4a4+000000.grb ICMSHg4a4+000000_split ls ICMSH*split* ICMSHg4a4+000000_split01.grb ICMSHg4a4+000000_split02.grb ICMSHg4a4+000000_split03.grb ICMSHg4a4+000000_split04.grb ``` or by using the grib\_copy command that comes with ecCodes (which personally I prefer): ``` grib_copy ICMSHg4a4+000000.grb ICMSHg4a4+000000_[typeOfLevel].grb ``` where 'typeOfLevel' is a grib key. Once the level types are split cdo should work ok? Was there another problem with the K-index and Total totals index? Cheers,  Glenn ---
**Author:** ![Jan\_Streffing](https://forum.ecmwf.int/letter_avatar_proxy/v4/letter/j/d26b3c/32.png) [@Jan\_Streffing](https://forum.ecmwf.int/u/Jan_Streffing)\ **Post date:** [25 July 2019 12:23 UTC](https://forum.ecmwf.int/t/processing-icmua-output-files/12132/4 "2019-07-25T12:23:19Z")
Hello Glenn, that is an interesting point. I removed the lines ``` NFP3DFS=10, MFP3DFS(:)=130,135,138,155,133,162108,162110,162111,162112,162113, ``` and kept ``` NFP3DFP=13, MFP3DFP(:)=129,130,135,138,155,157,3,133,246,247,248,75,76, ``` This should leave me with pressure level output if I interpret the names correctly. `grib_ls` tells me that the ICMSH output contain fields with `typeOfLevel=isobaricInhPa` and` edition=1`. I interpreted that as grib1 pressure level data and the contents do look like pressure data. Something wrong here? K-index and Total totals index were also of `edition=2` and resulted in inability to post-process the ICMGG file until I removed them from the output. Ultimately I don't have much use for surface level data, so I am happy not writing it out. For me it would only make the model slower and use up disc space anyway. Cheers, Jan ---
**Author:** ![Joakim\_Kjellsson](https://forum.ecmwf.int/letter_avatar_proxy/v4/letter/j/e19b73/32.png) [@Joakim\_Kjellsson](https://forum.ecmwf.int/u/Joakim_Kjellsson)\ **Post date:** [30 July 2019 08:17 UTC](https://forum.ecmwf.int/t/processing-icmua-output-files/12132/5 "2019-07-30T08:17:26Z")
Hi Jan  I never had this problem with 43r3 using our post processing in esm-tools. I do what Glenn suggests: split into one file per vertical level and also GG and SH. So I get output files like ending with: SH\_isobaricInhPa (U,V,T,Z) GG\_isobaricInhPa (Q,O3) SH\_hybrid GG\_hybrid GG\_surface but with 43r3 I also get "UA\_isobaricInhPa" and "UA\_hybrid" rather than "GG\_isobaricInhPa" and "GG\_hybrid".  I do the following: cat ICMUA\* \> ICMUACAT grib\_copy -w typeOfLevel=isobaricInhPa ICMUACAT ICMUACAT\_isobaricInhPa cdo --eccodes -f nc4c -z zip\_4 -t ecmwf -setgridtype,regular ICMUACAT\_isobaricInhPa h6mv\_19820101\_19820105\_UA\_isobaricInhPa.nc  If CDO has been compiled with ecCodes, the above will convert the UA output to a netCDF file on a regular grid with moderate compression.  Cheers Joakim PS: The error you get is however exactly what I get when I try to post process the output from WAM at T511. WAM (for some reason) writes data to different grid types (regular,reduced etc) depending on the resolution. No idea why...