Filter data array on multiple conditions

Question

From xarray's tutorial data, I want to extract a DataArray that contains number of warm days in month, defined as between 22 and 30 Celcius:

airtemps = xr.tutorial.load_dataset('air_temperature')
airtemps = airtemps.sel(time=slice('2013-01-01', '2013-12-31'))
airtemps['air'] = airtemps.air - 273.15

air_day = airtemps.resample('1D', 'time', how='mean')
# Define multiple conditions - How can this be done programatically?

I am now looking for a way to create this map below programmatically, also being able to add more conditions

meets_condition = (air_day.air > 22) & (air_day.air < 30)
warm_days = meets_condition.resample('M', dim='time', how='sum')

Conditions could be defined like so:

not_cold = ('air',  operator.ge, 22)
not_hot  = ('air',  operator.le, 30)

I can do this simply with one condition:

variable, operator_fn, value = not_cold
meets_condition = operator_fn(air_day[variable], value)
warm_but_possibly_hot_days = meets_condition.resample('M', dim='time', how='sum')

But I'm struggeling adding multiple, dynamically. I can do this

comfy_warm = [not_cold, not_hot]

all_maps = [fn(air_day[var], val) for var, fn, val in comfy_warm]
(all_maps[0] & all_maps[1]).resample('M', dim='time', how='sum')

But I'm looking to do sth. like this

np.all(all_maps).resample('M', dim='time', how='sum')

Here's a gist for convenience. As always, I am thankful in advance

TomTom101 · Accepted Answer · 2016-11-18 17:16:56Z

1

We cannot feed np.logical_and.reduce() with DataArrays, but it works nicely with pure arrays, so that's I then feed it with

all_maps = [fn(air_day[var], val).values for var, fn, val in comfy_warm]

I then add this map as a new data variable and resample it

air_day['nice_days'] = (['time', 'lat', 'lon'], np.logical_and.reduce(all_maps))    
air_day.nice_days.resample('M', dim='time', how='sum')

answered Nov 18, 2016 at 17:16

TomTom101

6,9413 gold badges27 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

John Zwinck · Accepted Answer · 2016-11-19 12:12:20Z

1

I think you want this:

xr.ufuncs.logical_and.reduce(all_maps)

It applies the and operation element-wise across all the boolean arrays of the same length, and produces one result of that same length. Of course you can use logical_or or any other suitable ufunc.

edited Nov 19, 2016 at 12:12

answered Nov 18, 2016 at 12:59

John Zwinck

252k44 gold badges346 silver badges459 bronze badges

4 Comments

TomTom101 Over a year ago

I already tried this, but it a) takes forever (once 2m, now 7m(!)) compared to the 18 ms when doing it the manual way and b) it returns an np.array, I'd prefer a DataArray

John Zwinck Over a year ago

@TomTom101: Oh, I see--I had written np.logical_and but I see now that xarray has its own reimplementations of some of the ufuncs. I guess they're not compatible with NumPy, so I'll update my answer now to use the xarray version. Please try again and I hope you'll see good performance.

TomTom101 Over a year ago

Still complains: AttributeError: '_UFuncDispatcher' object has no attribute 'reduce'

John Zwinck Over a year ago

@TomTom101: Oh well, I give up. I just use Pandas and NumPy like everybody else.

Collectives™ on Stack Overflow

Filter data array on multiple conditions

2 Answers 2

Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related