I'm reading some NaN values for ERA5-Land Solar Surface Radiation Downwards on 2018/08/27
at location (32.3, -86.9) from 0 - 6 hours. I think I downloaded this data previously and
and it did not contain NaN values, so I was wondering if these are intended to be blank?
from GRIB file
--------------
In [8]: xarray.open_dataset("/tmp/solar_201808_32.3_-86.9.grib", engine="cfgrib").ssrd.sel(
latitude=32.3, longitude=-86.9, time="2018-08-27")
Out[8]:
<xarray.DataArray 'ssrd' (step: 24)>
array([9.998750e+03, nan, nan, nan, nan,
1.000000e+04, 1.000000e+04, 1.000000e+04, 1.000000e+04, 1.000000e+04,
1.000000e+04, 8.096800e+04, 6.839000e+05, 2.008996e+06, 4.061476e+06,
6.352708e+06, 9.096770e+06, 1.135856e+07, 1.347785e+07, 1.585578e+07,
1.767849e+07, 1.907706e+07, 2.003201e+07, 2.035096e+07], dtype=float32)
from netCDF file
----------------
In [20]: xarray.open_dataset("/tmp/solar_201808_32.3_-86.9.nc").ssrd.sel(
latitude=32.3, longitude=-86.9, time=pandas.date_range(datetime.date(2018, 8, 27),
datetime.date(2018, 8, 28), freq="H"))
Out[20]:
<xarray.DataArray 'ssrd' (time: 25)>
array([2.199908e+07, 1.014600e+04, nan, nan, nan,
nan, 1.014600e+04, 1.014600e+04, 1.014600e+04, 1.014600e+04,
1.014600e+04, 1.014600e+04, 8.107800e+04, 6.838120e+05, 2.009003e+06,
4.061529e+06, 6.352746e+06, 9.096953e+06, 1.135852e+07, 1.347785e+07,
1.585576e+07, 1.767860e+07, 1.907698e+07, 2.003212e+07, 2.035113e+07],
dtype=float32)
I am also seeing the NaN values in the first 6 hours of ERA5-Land data. Every single forecast, the first 6 hours have a handful of extra NaN values for SSRD. Has any reason for this been given?
hi Cathryn,
can you share the request you used to retrieve the data, please?
Thanks,
Michela
c.retrieve(
'reanalysis-era5-land',
{
'variable': [
'10m_u_component_of_wind', '10m_v_component_of_wind', '2m_dewpoint_temperature',
'2m_temperature', 'snow_depth', 'snowfall',
'snowmelt', 'surface_net_solar_radiation', 'surface_pressure',
'surface_solar_radiation_downwards', 'total_precipitation',
],
'year': 2010,
'month': 01,
'day': 02,
'time': [
'00:00', '01:00', '02:00',
'03:00', '04:00', '05:00',
'06:00', '07:00', '08:00',
'09:00', '10:00', '11:00',
'12:00', '13:00', '14:00',
'15:00', '16:00', '17:00',
'18:00', '19:00', '20:00',
'21:00', '22:00', '23:00'
],
'format': 'grib',
},
outputFileFinal)
Hi Cathryn, I ran your request for 3 variables (ssrd,tp,snsr) and the grib file contains
% cdo info test3vars.grib
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter ID
1 : 2010-01-02 00:00:00 0 6483600 4270737 : 0.0000 6.7921e+06 3.3273e+07 : 176.128
2 : 2010-01-02 00:00:00 0 6483600 4270737 : 0.0000 1.7855e+07 4.1079e+07 : 169.128
3 : 2010-01-02 00:00:00 0 6483600 4270737 : 0.0000 0.0016304 0.31887 : 228.128
4 : 2010-01-02 01:00:00 0 6483600 4270739 : 0.0000 1.5303e+05 3.7536e+06 : 176.128
5 : 2010-01-02 01:00:00 0 6483600 4270737 : 0.0000 5.6068e+05 3.9311e+06 : 169.128
6 : 2010-01-02 01:00:00 0 6483600 4270737 : 0.0000 5.8033e-05 0.020378 : 228.128
7 : 2010-01-02 02:00:00 0 6483600 4270740 : 0.0000 3.4324e+05 7.4105e+06 : 176.128
8 : 2010-01-02 02:00:00 0 6483600 4270737 : 0.0000 1.1801e+06 7.7778e+06 : 169.128
9 : 2010-01-02 02:00:00 0 6483600 4270737 : 0.0000 0.00011777 0.036021 : 228.128
10 : 2010-01-02 03:00:00 0 6483600 4270742 : 0.0000 5.7390e+05 1.0709e+07 : 176.128
11 : 2010-01-02 03:00:00 0 6483600 4270737 : 0.0000 1.8642e+06 1.1387e+07 : 169.128
12 : 2010-01-02 03:00:00 0 6483600 4270737 : 0.0000 0.00017909 0.046740 : 228.128
13 : 2010-01-02 04:00:00 0 6483600 4270743 : 0.0000 8.4012e+05 1.4075e+07 : 176.128
14 : 2010-01-02 04:00:00 0 6483600 4270737 : 0.0000 2.6066e+06 1.5278e+07 : 169.128
15 : 2010-01-02 04:00:00 0 6483600 4270737 : 0.0000 0.00024377 0.054088 : 228.128
16 : 2010-01-02 05:00:00 0 6483600 4270737 : 0.0000 1.1429e+06 1.7551e+07 : 176.128
17 : 2010-01-02 05:00:00 0 6483600 4270737 : 0.0000 3.4072e+06 1.8949e+07 : 169.128
18 : 2010-01-02 05:00:00 0 6483600 4270737 : 0.0000 0.00030913 0.069972 : 228.128
19 : 2010-01-02 06:00:00 0 6483600 4270737 : 0.0000 1.4840e+06 2.0565e+07 : 176.128
20 : 2010-01-02 06:00:00 0 6483600 4270737 : 0.0000 4.2655e+06 2.2111e+07 : 169.128
21 : 2010-01-02 06:00:00 0 6483600 4270737 : 0.0000 0.00037486 0.087644 : 228.128
22 : 2010-01-02 07:00:00 0 6483600 4270737 : 0.0000 1.8578e+06 2.3249e+07 : 176.128
23 : 2010-01-02 07:00:00 0 6483600 4270737 : 0.0000 5.1730e+06 2.5103e+07 : 169.128
24 : 2010-01-02 07:00:00 0 6483600 4270737 : 0.0000 0.00044155 0.098348 : 228.128
Could the software you are using to view the data be adding extra (empty) steps at the start of the file? can you please share an example of the results you are observing?
Thanks,
Kevin
Sorry I responded to this last week but I guess it didn't go through. Your output is showing the exact issue. I've bolded and underlined the issue with the differeing number of missing values at different time steps. Is your land mask not consistent for parameter 176.128? We need to subtract off the previous hour from the current hour in this data in order to get an hourly value, it causes NaN values when the number of NaNs differs between timesteps.
-1 : Date Time Level Gridsize Miss : Minimum Mean Maximum : Parameter ID 1 : 2010-01-02 00:00:00 0 6483600 4270737 : 0.0000 6.7921e+06 3.3273e+07 : 176.128 2 : 2010-01-02 00:00:00 0 6483600 4270737 : 0.0000 1.7855e+07 4.1079e+07 : 169.128 3 : 2010-01-02 00:00:00 0 6483600 4270737 : 0.0000 0.0016304 0.31887 : 228.128 4 : 2010-01-02 01:00:00 0 6483600 4270739 : 0.0000 1.5303e+05 3.7536e+06 : 176.128 5 : 2010-01-02 01:00:00 0 6483600 4270737 : 0.0000 5.6068e+05 3.9311e+06 : 169.128 6 : 2010-01-02 01:00:00 0 6483600 4270737 : 0.0000 5.8033e-05 0.020378 : 228.128 7 : 2010-01-02 02:00:00 0 6483600 4270740 : 0.0000 3.4324e+05 7.4105e+06 : 176.128 8 : 2010-01-02 02:00:00 0 6483600 4270737 : 0.0000 1.1801e+06 7.7778e+06 : 169.128 9 : 2010-01-02 02:00:00 0 6483600 4270737 : 0.0000 0.00011777 0.036021 : 228.128 10 : 2010-01-02 03:00:00 0 6483600 4270742 : 0.0000 5.7390e+05 1.0709e+07 : 176.128 11 : 2010-01-02 03:00:00 0 6483600 4270737 : 0.0000 1.8642e+06 1.1387e+07 : 169.128 12 : 2010-01-02 03:00:00 0 6483600 4270737 : 0.0000 0.00017909 0.046740 : 228.128 13 : 2010-01-02 04:00:00 0 6483600 4270743 : 0.0000 8.4012e+05 1.4075e+07 : 176.128 14 : 2010-01-02 04:00:00 0 6483600 4270737 : 0.0000 2.6066e+06 1.5278e+07 : 169.128 15 : 2010-01-02 04:00:00 0 6483600 4270737 : 0.0000 0.00024377 0.054088 : 228.128 16 : 2010-01-02 05:00:00 0 6483600 4270737 : 0.0000 1.1429e+06 1.7551e+07 : 176.128 17 : 2010-01-02 05:00:00 0 6483600 4270737 : 0.0000 3.4072e+06 1.8949e+07 : 169.128 18 : 2010-01-02 05:00:00 0 6483600 4270737 : 0.0000 0.00030913 0.069972 : 228.128 19 : 2010-01-02 06:00:00 0 6483600 4270737 : 0.0000 1.4840e+06 2.0565e+07 : 176.128 20 : 2010-01-02 06:00:00 0 6483600 4270737 : 0.0000 4.2655e+06 2.2111e+07 : 169.128 21 : 2010-01-02 06:00:00 0 6483600 4270737 : 0.0000 0.00037486 0.087644 : 228.128 22 : 2010-01-02 07:00:00 0 6483600 4270737 : 0.0000 1.8578e+06 2.3249e+07 : 176.128 23 : 2010-01-02 07:00:00 0 6483600 4270737 : 0.0000 5.1730e+06 2.5103e+07 : 169.128 24 : 2010-01-02 07:00:00 0 6483600 4270737 : 0.0000 0.00044155 0.098348 : 228.128
Hi Cathryn, Michela, Kevin,
I came across this thread while investigating a related-looking issue and wanted to share it here rather than open a separate topic, since it may be the same underlying cause surfacing in a different product.
What we’re seeing: In the daily-aggregated ERA5-Land surface_solar_radiation_downwards_sum (via ECMWF/ERA5_LAND/DAILY_AGGR), specific coastal/mixed land-sea grid cells in Greece report an implausible exact zero value for a period of roughly 2–3 weeks around both equinoxes, every year we’ve checked (2019–2022, four independent years). Outside these windows, the same pixels are masked/no-data essentially year-round — so the anomaly appears to be the only time these specific cells report any value at all, and when they do, it’s a physically implausible flat zero rather than a genuinely low value.
The pattern is remarkably consistent across years — every spring window we found begins on March 4, and every autumn window begins Sep 22–23 — which suggests something systematic tied to the land-sea mask handling for this variable specifically, rather than real meteorological variability. Other co-located variables (temperature, etc.) are unaffected on the same dates/pixels.
Confirmed windows (checked exhaustively, day-by-day, across all of Greece):
| Year | Window (inclusive) |
|---|---|
| 2019 | Sep 23 – Oct 10 |
| 2020 | Mar 4 – Mar 20 |
| 2020 | Sep 22 – Oct 9 |
| 2021 | Mar 4 – Mar 21 |
| 2021 | Sep 23 – Oct 9 |
| 2022 | Mar 4 – Mar 21 |
| 2022 | Sep 23 – Oct 9 |
Example affected pixel: 20.9787°E, 38.8730°N (coastal Greece, Ionian side, mixed land/water grid cell). Outside the windows above, this pixel is masked/no-data essentially year-round; during the windows it becomes “valid” with SSRD = 0 exactly, then reverts to fully masked immediately afterward.
This looks structurally similar to what’s described above: Cathryn’s data shows the land mask for 176.128 (SSRD) shifting between timesteps while other parameters stay fixed — i.e. an inconsistent/unstable mask specifically for the radiation parameter. Ours is the daily-aggregated ERA5-Land product rather than raw hourly ERA5, and the failure mode is a valid-but-wrong zero rather than a NaN from differencing, but the common thread (radiation-specific mask instability at land-sea boundary cells) seems like it could be the same root mechanism.
Questions:
- Is this land-mask instability for SSRD/176.128 a known, tracked issue at this point, and does it affect ERA5-Land the same way it affects ERA5?
- Would this also explain a mask that flips from “missing” to “valid but zero” specifically around equinox transitions, rather than producing NaNs?
Happy to share the Earth Engine scripts we used to confirm this pattern if that’s useful for tracking it down further.
Thanks, Aris