create netCDF file with xarray, define variable data types

Question

I created a netCDF-File with xarray for inputting topography height info, surface type info etc. into a numerical weather prediction model. I managed to create a file, however, the model requires the different variables to be different data types.

My dataset bolund_static looks as follows:

<xarray.Dataset>
Dimensions:          (x: 800, y: 200)

Coordinates:
  * y                (y) float64 1.0 3.0 5.0 7.0 9.0 ... 393.0 395.0 397.0 399.0
  * x                (x) float64 1.0 3.0 5.0 ... 1.595e+03 1.597e+03 1.599e+03
Data variables:
    zt               (y, x) float64 1.0 1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0 1.0 1.0
    vegetation_type  (y, x) float64 -127.0 -127.0 -127.0 ... -127.0 -127.0
    water_type       (y, x) float64 3.0 3.0 3.0 3.0 3.0 ... 3.0 3.0 3.0 3.0 3.0
    soil_type        (y, x) float64 -127.0 -127.0 -127.0 ... -127.0 -127.0
    pavement_type    (y, x) float64 -127.0 -127.0 -127.0 ... -127.0 -127.0
Attributes:
    version:         1
    origin_z:        0.0
    origin_y:        694682.098
    origin_x:        6177441.825
    origin_lat:      12.098271
    origin_lon:      55.70364
    rotation_angle:  0.0
    palm_version:    6.0
    origin_time:     2019-04-01 12:00:00 +01

Saving this array with bolund_static.to_netcdf(), it saves all variables as double datatypes. This info I got by making an ncdump of the created netcdf-file.

netcdf bolund_static {
dimensions:
    y = 200 ;
    x = 800 ;
variables:
    double y(y) ;
        y:_FillValue = NaN ;
    double x(x) ;
        x:_FillValue = NaN ;
    double zt(y, x) ;
        zt:_FillValue = NaN ;
    double vegetation_type(y, x) ;
        vegetation_type:_FillValue = NaN ;
    double water_type(y, x) ;
        water_type:_FillValue = NaN ;
    double soil_type(y, x) ;
        soil_type:_FillValue = NaN ;
    double pavement_type(y, x) ;
        pavement_type:_FillValue = NaN ;

// global attributes:
        :version = 1 ;
        :origin_z = 0. ;
        :origin_y = 694682.098 ;
        :origin_x = 6177441.825 ;
        :origin_lat = 12.098271 ;
        :origin_lon = 55.70364 ;
        :rotation_angle = 0. ;
        :palm_version = 6. ;
        :origin_time = "2019-04-01 12:00:00 +01" ;
data: <...>

I would need vegetation_type, water_type, soil_type and pavement_type to be of type NC_BYTE instead of NC_DOUBLE after exporting, and x,y,zt as NC_FLOAT. How would I go about changing these datatypes? Is that possible from within the xarray/Python environment?

Light_B · Accepted Answer · 2019-03-12 09:50:21Z

5

While saving your dataset you can make use of encoding parameter which takes input as a dictionary.

bolund_static.to_netcdf(filename.nc, encoding={'var1':{'dtype':'int8'}, 'var2':{'dtype':'int8'}})
# replace var1, var2 with your desired variable names
# for NC_float save dtype as int32, for NC_Byte save as int8

You can also modify other attributes of the variable such as _FillValue, etc. Another important note is that xarray automatically saves the time unit from the nearest possible starting time by default. If you want to change that then you can do that in the same way as well

bolund_static.to_netcdf(filename.nc, encoding={'var1':{'dtype':'int8'},\
'var2':{'dtype':'int8'}, 'origin_time':{'units':'seconds/hours since a 2000-01-01 00:00:00'}})

answered Mar 12, 2019 at 9:50

Light_B

1,8003 gold badges17 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

sfluck Over a year ago

Thank you for your help, it worked up to one detail. When using int32 as datatype, it will show 'int' in the ncdump. Choosing float32 will return a float datatype in the ncdump. Is that possible?

Light_B Over a year ago

That's correct. You can save float32 or float32 or as float64 according to your need. More info on datatypes in Python. If it answered your question then you can mark it as checked.

sfluck Over a year ago

speaking of datatypes, is it possible to forge strings into a NC_CHAR datatype? I need to turn string attributes into NC_CHAR and have not found any solution to that yet.

Light_B Over a year ago

NC_CHAR is 8 bits. My guess is if your string is more than 8 bits then it will get chopped off. NC_STRING is the data type you're looking for if you want to save strings. More on NetCDF datatypes here

Vinod Kumar Over a year ago

As far as I know, "units" can not be set via encoding.

Collectives™ on Stack Overflow

create netCDF file with xarray, define variable data types

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related