Is any team working on using aifs-ens-1.0 at ecmwf/aifs-ens-1.0 at main for the quest?. The leader board would be quite similar with multiple team pushing similar forecasts:). However using solution such as the surrogate modeling https://www.youtube.com/watch?v=k-8ljdnskyM which could be avoid resource intensive training but optimizing top of the already available trained model.
Good morning @team-fahamu ,
A team from ECMWF will be submitting three test configurations of the AIFS-ens. Several teams may use AIFS infrastructure but I suspect they’ll be differences in initialisation data as well as alterations to consider sub-seasonal timescales.
Your suggestion of surrogate modelling may be taken forward by other teams.
Thanks,
Josh
1 Like
Thanks for the info on the use of AIFS-ens in AI weather quest.
As a next step in the surrogate method of deep learning on top of AIFS and receiving peer review methods on the AIFS-ENS-v1.0, the workflow related to the forecast submission is available at this GitHub - icpac-igad/ea-aifs: ECMWF AIFS ensemble model run routines.
Such as the method for the conversion of AIFS-ens output into forecast submission format ea-aifs/aifs_792hr_forecast_grib_check_vars.py at main · icpac-igad/ea-aifs · GitHub and the upload routine ea-aifs/ensemble_quintile_analysis.py at main · icpac-igad/ea-aifs · GitHub. The code is written for the forecast submission in 20250814, but unfortunately, couldn’t, will be checking for the next 20250821 forecast.
Also there are cost optimisations using a coiled GPU notebook, where the initial test with 10 ensemble members costs 10$. There are methods that could be explored to reduce this cost further, as noted in ea-aifs/docs/COILED_GPU_INFERENCE_GUIDE.md at main · icpac-igad/ea-aifs · GitHub
Hi Fahamu,
Sorry you were unable to submit a forecast last week.
Can you please provide a screenshot of the error?
Additionally, are you using the following teamname and modelname, including correct captialisation?
Fahamu, FahamuAIFSv1
Kind regards,
Josh
There was correct error message in forecast submission, the UTC time for the forecast submission exceeded (by three hours) and that is why couldn’t submit, sorry missed to mention this in the post. Thank you. Yes, the team names are correctly used. Following is the error message received.
```
Fahamu is registered to the AI Weather Quest. You may submit your forecast.
FahamuAIFSv1 is registered to the AI Weather Quest. You may submit your forecast.
Unique_ID expver_ID Teamname Modelname
0 Qronon_01 Qronon_01 Qronon QRCML
1 IFUAet_01 IFUAIHydromet_01 IFUAIHydromet ProS2St
2 UWAtIA_01 UWAtmosNVIDIA_01 UWAtmosNVIDIA DLESyMS2Sv1
3 CLINz_01 CLINT_01 CLINT CLINTDD
4 Nord2S_01 NordicS2S_01 NordicS2S NordicS2S2
5 Igni42_01 IgnisNeuralis42_01 IgnisNeuralis42 GCast42
Fahamu is registered to the AI Weather Quest. You may submit your forecast.
FahamuAIFSv1 is registered to the AI Weather Quest. You may submit your forecast.
…..
File /opt/coiled/env/lib/python3.12/site-packages/AI_WQ_package/check_fc_submission.py:98, in check_forecast_data_window(fc_start_date)
96 print (‘forecast submitted within competition time window’)
97 else:
—> 98 raise ValueError(f"You are not allowed to submit a forecast for the following forecast start date, {fc_start_date}, at this point in time. Allowed time window for this forecast start date is {fc_start_date} to {end_of_next_sun_str}")
ValueError: You are not allowed to submit a forecast for the following forecast start date, 20250814, at this point in time. Allowed time window for this forecast start date is 20250814 to 20250817
```
Oh. Now I understand your issue. You tried to submit a forecast at 0300 UTC 20250818. This is outside the forecast submission window and not allowed. Sorry.
I recommend that you resubmit this week’s forecasts between 21st and 24th August. Please can I remind you of the following rule:
Participants may use any data with a timestamp strictly prior or equal to Thursday 00 UTC to initialise their models, regardless of when the data becomes available. The four-day submission window allows participants to process initial data with relatively long latency (e.g. dynamical sub-seasonal forecast data which has a two-day delay) and provides additional time for those with limited computational resource to submit their forecasts.
Please do not initialise your model on the Monday (day 5) of the forecast workflow. Please only use data with a timestamp prior or equal to Thursday 00 UTC. Further details can be found at: Submitting Forecasts • AI Weather Quest
Kind regards,
Josh
Thank you, this is well noted. Reiterating the steps carried out for 20250814 and plans for 20250821
For 20250814 forecast:
-
Downloaded and made the pickle file for the IFS initial data for 20250814T00 using the ECMWF open data py library, ran the GPU inference on set of members for 792 hours.
-
Using the climatology data downloaded from AI_WQ_package for the dates 20250901(19th day forecast) and 20250908(26th day forecast), compared with the AIFS inference output ensemble (4 nos)members to calculate weekly quintile probabilities and attempted to submit the forecast, but the submission window is exceeded as the attempt made on 20250818T0300 UTC.
Plan for 20250821
- Download the initial data from IFS on 20250821T00. and follow the same step but different climatology day datasets but submit the forecast well within the submission window.
Perfect. I look forward to seeing your submission next week.
Josh
1 Like
Great to see your submissions @team-fahamu ! AI Weather Quest | ECMWF
1 Like
Thank you for the support, in the leaderboard, the FahamuAIFSv1 Leaderboards • AI Weather Quest model output only listed for the weekley RPSS. In period Period-aggregated RPSSs, is missing, any changes needed from our side?
Hello Fahamu team!
This is because your team did not submit forecasts during the first competition week (see participation table on your team page).
To have a model featured in a period-aggregated leaderboards, teams need to submit consistently with that model during every week of the given period. During the current period, your model will therefore only appear in the weekly leaderboards for the weeks when submissions were made.
From the next period (submissions starting on 13 November 2025, with results published from Friday 19 December 2025), consistent submissions will allow your model to be featured in the period-aggregated leaderboards as well.
Best regards,
Olga
Thank you @Olga_Loegel , much appreciated, looking forward to the mark at Period aggregated RPSSs.