Test grib

[VictorPozsgay]

Caution

Negative potential consequences of an action.
DO NOT MERGE AS IT IS.

Important

Crucial information necessary for the branch to be merged.
Package compatibility needs to be ensured. Currently, installing 'cfgrib' introduces some package clashes.

Recent changes to the ERA5 download API mean that it is no longer possible to download bulk netCDF files, but GRIB files instead. As it is, globsim is not able to handle such requests. This branch requests and retrieves GRIB data, and converts all files into globsim-compliant netCDF files, ready for interpolation/scaling. Note that the conversion uses xarray with the engine='cfgrib', which needs to be installed. Currently, this clashes with some other packages on the master branch and the interpolation crashes with the ERROR: 'ffi_prep_closure(): bad user_data (it seems that the version of the libffi library seen at runtime is different from the 'ffi.h' file seen at compile-time)'.

New Features

• GRIB download supported for ERA5
• seamless flow with conversion to netCDF (up to a branch change, which will hopefully be fixed soon)

Limitations

• Package clash with the master branch
• After successfully running the download, interpolate crashes with ERROR code:

ffi_prep_closure(): bad user_data (it seems that the version of the libffi library seen at runtime is different from the 'ffi.h' file seen at compile-time)

How to set up the environment(s)

Changes to libraries and supported python version means the globsim environment will have to be updated.

1. Create a new conda environment globsim4 as explained in Globsim 4.0 #149
2. Create a second conda environment, e.g. globsim4grib, with the exact same method.
3. Activate globsim4grib and install cfgrib:

conda install -c conda-forge cfgrib

4. Until further improvement, this creates a package clash with the master branch, we will then use the branch test_grib for the download command, and the branch master for interpolate and scale

How to use the globsim workflow

1. Activate globsim4grib
2. Go to the folder where globsim is installed
3. Change branch to test_grib

git checkout test_grib

4. Download as usual.
5. Once the download is finished, deactivate the conda environment and switch back to the regular one, globsim4.
6. Go back to the folder where globsim is installed
7. Change branch back to master

git checkout master

8. Interpolate and scale as usual.