S2S dynamical operational data now available from ECMWF Data Store (ECDS)

Workshops, competitions and events AI Weather Quest
1

Dear all,

Operational S2S forecast data currently accessed via ECMWF WEB-API is transitioning to a new data service, ECMWF Data Store (ECDS). This new service officially opened on Tuesday.

I recommend all users who use operational dynamical forecast data to produce their ML-based predictions take time relatively soon to configure their software pipelines to work with ECDS. Access to S2S data via ECMWF WEB-API is planned to close in mid-May.

For the Quest, we will allow late forecast submissions via email if this infrastructure change causes any issues. As previous competitive seasons, we will only allow late submission for a single week.

ECMWF colleagues have created a user guide for migrating scripts from a WEB-API service to ECDS. In parallel, I will ensure that the AI Weather Quest continues to compare dynamical and ML‑based forecasts using the new service.

Happy to communicate any feedback regarding the new service to colleagues.

Thanks all.

Josh

2

Dear Josh,

Thank you for the update regarding the transition to ECMWF Data Store (ECDS).

I would like to share some feedback from my experience so far: in the previous WEB-API environment, downloads could be performed almost immediately without waiting. In contrast, in the new ECDS environment, the queue times for downloading operational S2S forecast data appear to be significantly longer, often far exceeding the actual processing time.

As a suggestion, it might be helpful if smaller files (e.g., under 100 MB) could be prioritized for faster downloads. This could improve workflow efficiency while larger datasets are still queued.

I hope this feedback is useful for your discussions with ECMWF colleagues. Please let me know if additional details or examples would be helpful.

Best regards,
Lu Peng