Glue restart files to change number of processors for restart

AIFS, IFS and OpenIFS OpenIFS
, ,
21

Hi Joakim,

Yes, looked at the code. And it's as I thought. If you add KIOMASTER=1 to the call to SETUP_IOSTREAM, that routine will essentially just limits I/O to only the MPI task specified by KIOMASTER.  There is no code that will do any kind of MPI send to KIOMASTER unfortunately. That would need to be added, but I don't think it should go into the iostream_mix.F90 module.  It's specific to the restarts so should reside with wrresf.F90 (or wrappers around them).

There was alot of discussion of this today at the EC-Earth meeting.   Unfortunately the IO code in IFS is rather impenetratable. I am coming round to the idea of writing an external, separate code (but uses relevant code from OpenIFS) to read in the separate restarts and combine them after the run.

Cheers,  Glenn


22

Glad to hear the discussion is still going. I hope you or someone else finds a way to do this.

Best wishes
Joakim

23

according to Glenn, writing restarts is in ifs/utility/wrresf.F90 and reading in ifs/control/reresf.F90 .

It would be great to develop an option to write only one restart (either in ifs or as a standalone utility), and then to be able to read this single restart file in ifs.

24

Hi Joakim,

Could you tell me how to "glue" all the NEMO restart files into one global file, and then restart from the global file? 

I'd like to start a new experiment from existing NEMO restart files, but as you said, this existing restart files were produced with different number of produces.

Best regards.

Zhenqian

25

There is a tool that comes with NEMO
https://salishsea-meopar-tools.readthedocs.io/en/latest/nemo-tools/rebuild_nemo.html

I usually use nocs_combine. It's fine for low-res grids, e.g. global 1° etc.

https://salishsea-meopar-tools.readthedocs.io/en/latest/legacy_docs/nocscombine/

Cheers
Joakim

26

Thanks for your answer.

In fact, I encountered the above problem when using EC-Earth3.

After combine each restart_*.nc,  i had set the restart directory to the merged file directory, but it doesn't work.

Could you tell me the detail about after combined?


Best regards.

Zhenqian

27

Thanks to Etienne's script that I got some time ago, I've now put together a script to take OpenIFS GRIB output and create new initial conditions (attached).

oifs_grib_output_to_restart.sh (2.9 KB)

My strategy to create restart files from a long piControl run is to continue the same run for 1 more day. Then take the output (ICMGGECE3+400001, ICMUAECE3+400001, ICMSHECE3+400001) and create new initial condtions (ICMGGECE3INIT, ICMGGECE3INIUA, ICMSHECE3INIT).

This requires a bit of manual work etc, but its not very often that one wishes to restart a model run using a different number of tasks/threads.

The following is required in the fort.4 namelist:

&NAMFPC
    CFPFMT = 'MODEL'
    NFP3DFS = 10
    MFP3DFS = 133, 75, 76, 246, 247, 248, 138, 155, 130, 203
    NRFP3S = -99
    NFP2DF = 2
    MFP2DF = 152, 129
    MFP3DFP = 75, 76, 246, 247, 248, 138, 155, 130, 133, 203
    MFPPHY =  15,  16,  17,  18,  26,  27,  28,  29,  30,  31,
              32,  33,  34,  35,  36,  37,  38,  39,  40,  41,
              42,  43,  66,  67,  74  139, 141, 148, 160, 161,
             162, 163, 170, 172, 173, 174, 183, 198, 234, 235,
             236, 238, 228008, 228009, 228010, 228011, 228012, 228013, 228014,
    228007
    NFP3DFP = 10
    NFPPHY = 50
    RFP3P = 85000.0, 50000.0, 5000.0
    NFP3DFT = 0
    NFP3DFV = 0
    NFPCLI = 0
    LFPQ = .false.
    LTRACEFP = .false.
    RFPCORR = 60000.0
/

Appears to work with OpenIFS 43r3v2. Should also work for 43r3v1, but I haven't checked.

Note: My script assumes L91 vertical grid, so it's hard coded in there, but shouldn't be too hard to change to L62 or L137.

Cheers and thanks all for the advice!
Joakim