Retain numpy mask when inserting arrays

Question

I am attempting to use numpy masked arrays to filter "no data" from raster images. The array calculations I am doing subtract the values of adjacent pixels (see Applying functions to multidimensional numpy arrays without loops), but I need to ignore no data values. For example, with the array:

array = np.array([[78, 72, 69, 71, 58, 49],
                 [74, 67, 56, 49, 46, 50],
                 [69, 53, 44, 37, -9999, -9999],
                 [64, 58, 55, 22, 33, 24],
                 [68, 61, 47, 21, 16, 19],
                 [74, 53, 34, 12, 11, 12]])

Lets say -9999 is a no data value. So I mask the array as follows:

array = np.ma.masked_where(array == -9999, array, False)

This produces an expected result. I then need to calculate the difference each element against it's surrounding 8 neighbors. So for the "lower right" neighbor, this would be (ignoring edge pixels here):

result = array[1: 5, 1: 5] - array[2: 6, 2: 6]

This also produces the result I intend:

[[23, 19, --, --],
 [-2, 22, 4, --],
 [11, 34, 6, 14],
 [27, 35, 10, 4]]

However, when I insert the result array as a dimension of an empty array, the mask is lost from -9999, and worse still, the elements that "inherited" the mask ([1, 3] & [1, 4]) are inserted with their original values.

gradient = np.empty((8, 4, 4), dtype = np.int)
gradient[0] = result

I can mask out the -9999 values again, but as I will be applying the argmax function across axis 0 of the gradient array, the retention of the original values causes issues. I am looking for a way to either insert the result array as it appears above, or an alternative method to the masking to exclude neighbor elements matching the null value and/or replace the value in the result.

A masked array has a data array and a mask array. gradient is not masked, so only gets the data part. Masking is fragile, and is only maintained by mask aware operations. — hpaulj
– hpaulj, Commented Dec 22, 2015 at 14:02
Explanation makes sense. I'll avoid it for what I'm working on here. — ssast
– ssast, Commented Dec 22, 2015 at 14:04

Randrian · Accepted Answer · 2015-12-22 12:40:27Z

2

You should use nan (nan = not a number) values instead of -9999:

array[array == -9999] = np.nan

then do your calculation. Every value which is derived from this value should also be nan.

answered Dec 22, 2015 at 12:40

Randrian

1,08512 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ssast Over a year ago

So I should set the dtype of all arrays to np.float? This is fine, as my input arrays are often floating point anyway. With np.nan in place, If I try gradient[0].argmin(axis=0), the result is [1 0 0 0], which I think means nan is being considered. Is that correct, and is there a way to exclude nan from such calculations?

ssast Over a year ago

This question does this with nanargmin: stackoverflow.com/questions/2821072/… So your solution to use nan instead of a mask works for me.

Collectives™ on Stack Overflow

Retain numpy mask when inserting arrays

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related