Huge performance drop when reading new ERA5 netcdf4 format

Thank you @Kenneth_Bowman for sharing your approach to this problem.

We would like to avoid re-chunking if possible since it adds an ad-hoc step to our process. Plus, I really think there is something wrong with the chunk sizes chosen in these files. All of them save for longitude don’t even divide their respective dimension size.

At least, the API should allow users to request specific chunk values. In that respect, I think this post then is a duplicate for your own: Specifying the chunk size for netCDF files