Hi Josh,
Looking at our logs, the slowness seems to come from the package still calling the FTP site, even though FTP is now closed. From our servers, each FTP attempt hangs about 16 minutes before timing out and falling back to ECBox. As far as we can tell, the submission does this twice: `AI_WQ_create_empty_dataarray` first tries to read the team list (`AI_WQ_unique_IDs.csv`) from FTP (a bare `except:` then falls back to ECBox, silently), and `AI_WQ_forecast_submission` calls it a second time. This would explain why the team list takes so long to appear. Our workflow adds a third wait when downloading the climatology with the package. Over 12 of our submissions with 3.28, these timeouts accounted for about 48 of the 49 minutes of each submission, while the ECBox steps took seconds. The same FTP calls seem to still be in 3.31, and the one in `AI_WQ_create_empty_dataarray` was mentioned in Connect Timeout .
Skipping the FTP calls for dates after `ftp_closure_date`, as the upload already does, might remove most of the wait.
Best regards