How can I specify the Unicode normalization when writing to NetCDF?

[tomchor]

What is your issue?

Hi! Is there a way to specify Unicode normalization (e.g., NFD, NFC) when using to_netcdf()? I have variable names with Unicode characters and want to ensure consistent normalization.

Here's a simple example of the issue:

import xarray as xr
# Create dataset with Unicode variable name and save it
original_name = "ā"
ds = xr.Dataset({original_name: ([], 1)}).to_netcdf("test.nc")
ds_loaded = xr.open_dataset("test.nc")
loaded_name = list(ds_loaded.variables.keys())[0]
original_name == loaded_name # Returns false

Doing some digging, it seems that when writing to NetCDF, everything gets normalized with the NFC normalization. However, I'd like to normalize them with the NFD option, since it matches the characters I can compose on IPython, vim, vscode, etc.

Thanks!

[kmuehlbauer]

@tomchor Can you please add details about your system? Your example returns True for me on my system (linux, Python 3.12) for Jupyter Notebook kernekl, IPython and Python CLI.

[tomchor]

@tomchor Can you please add details about your system? Your example returns True for me on my system (linux, Python 3.12) for Jupyter Notebook kernekl, IPython and Python CLI.

Hmm, it is now returning true to me. That's crazy, I spent so long dealing with this the other day, and this MWE was for sure returning False. My best guess is that it depends on the login server I'm on when access the server.

In any case, regardless of the MWE, can one specify the normalization when saving to disk?