Increasing the size of the dataset downloaded at once: fire fuel characteristics

Datasets and Usage
1

Hi, everyone.

Downloading this dataset: fire fuel characteristics

There are very strict size restrictions. For example, you can select one variable and one year at a time. The size of the resulting file is 347.7 KB (which is very small). Even if you take a small coverage area, then all years and all variables will be approximately 20 MB in my case.
And another problem, often the download gets the status: Failed and you have to download it again.

Apparently a similar dataset was hosted in: CEMS Early Warning Data Store, but was later removed.

Apparently, there is a forum thread on this topic, in which the biomass I need is discussed

To start my research, data from: ECMWF Cross Data Store would be enough.

What will the community say?

2

Hello @Kamil_Gazizullin,

Yes the dataset was originally available in the CEMS Early Warning Data Store but it was relocated to the Cross Data Store as it was a better fit for the product.

The cost issue is one that we understand can be frustrating.
The limit of 20 allows more than 1 year of data to be downloaded for 1 variable, or around 90GB-120GB of data if retrieving globally. Unfortunately we cannot alter this as the cost internally still involves the processing of that 90-120GB of files to produce the cropped area.

The CDS team are working to resolve the spurious failures that we get in the retrievals, which is also another reason why we keep the cost low.

Kind regards,
Chris

3

@Christopher_Barnard thanks for the answer.

I’ll clarify one point: one variable and one year - that’s what I was able to download at once and the size of all the files (347.7 KB). And 20 MB is what I said would be the size if I downloaded everything. In principle, I can of course download one variable for each year, then put all the files in one folder and process them locally with a script, if there are no failures during downloading.

4

@Christopher_Barnard
data: Fuel group is presented for the 1st and 2nd of each month, for each coordinate at a frequency of one hour. Checked 2003 and 2021.
I tried to convert the data to a dataframe and output unique dates:

import xarray as xr
import pandas as pd

ds = xr.open_dataset('/FUEL_MAP_2021_01.area-subset.37.95.-119.31.36.59.-122.04.nc')
df = ds.to_dataframe().reset_index()

print(pd.unique(df['time'].dt.date))

Output:
[datetime.date(2021, 1, 1) datetime.date(2021, 1, 2)]

Dead fuel moisture content group (DFMC) and Live fuel moisture content group (LFMC) have data for all days and the frequency for each coordinate is one day.

Can you correct the data: Fuel group so that it matches the data: DFMC, LFMC?