Download CDS data directly to AWS S3 cloud file using fsspec on python


Hello, I need to download a netcdf file from the cds servers directly to an AWS S3 storage location.

To do that I would usually have the following code:
import fsspec
import cdsapi

ct = cdsapi.Client(quiet=True)
file_to_download_path="simplecache::s3://"+aws_bucket_name+'/'+global_climate_monthly_aws_s3_prefix_nc + '' % (lat, year, m)

with, mode='wb') as f:
                  ----do something that writes to file f

when I try to use the following command:

ct.retrieve(dataset, kwargs, f) it doesn't work because ct.retrieve expects a file name, not a file Instance. I need to open my file with otherwise it doesn't work to write directly to the AWS S3 cloud.

Any idea how I can have the ct.retrieve command write directly to an AWS S3 file.

I am desperately trying to avoid having to write the file locally and then upload it.

Thank you