Pythonic way to access certain parts of an array

Question

I have a 2D numpy array containing x (data[:,0]) and y(data[:,1]) information for a plot.

I’d like to fit a curve to the data, but only using certain parts of the data to determine the fitting parameters (e.g. using data in the range x = x1 -> x2, and x3 -> x4). My plan to do this is to create a new numpy array only containing the data I intend to pass to a SciPy CurveFitting routine.

index_range1 = np.where((data[:,0] > x1) and (data[:,0] < x2)
index_range2 = np.where((data[:,0] > x3) and (data[:,0] < x4)

and then I'd use these index ranges to pull the data of interest into a new array which I could pass to CurveFit.

Firstly, given that Python can handle complex arrays, this seems a very un-pythonic approach. Secondly, when running my script, I get an error saying that I need to use .any() or .all() in my expression for index_range 1 and 2.

I wonder, therefore, if anyone has any suggestions for an improved, more pythonic approach to solving this problem.

Thanks!

askewchan · Accepted Answer · 2013-05-07 14:23:04Z

2

To get a boolean array out of two others, use & to compare element-wise:

index_range1 = np.where((data[:,0] > x1) & (data[:,0] < x2))
index_range2 = np.where((data[:,0] > x3) & (data[:,0] < x4))

Using boolean arrays to index an array is probably more 'pythonic'. You don't need to find (using where) and save the indices, you can access the data directly from the array:

range1 = data[(data[:,0] > x1) & (data[:,0] < x2)]
range2 = data[(data[:,0] > x3) & (data[:,0] < x4)]

You could shorten this/make it more readable with:

x, y = data.T
range1 = data[(x > x1) & (x < x2)]
range2 = data[(x > x3) & (x < x4)]

Note: x and y are views, so modifying x, y modifies data, and there's no copying so it shouldn't slow your code down. But the ranges are copies, since fancy indexing makes a copy, so modifying them won't affect x, y, or data.

edited May 7, 2013 at 14:23

answered May 7, 2013 at 13:48

askewchan

46.7k18 gold badges125 silver badges135 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

IanRoberts Over a year ago

Thanks, I've combined this and the below answer to solve my problem in one line: my_data = data[np.logical_or(((data[:,0] > x1) & (data[:,0] < x2)),((data[:,0] > x3) & (data[:,0] < x4)))]

askewchan Over a year ago

@Ian You can use a pipe, a | b instead of logical_or(a,b).

askewchan Over a year ago

As in, my_data = data[((data[:,0] > x1) & (data[:,0] < x2)) | ((data[:,0] > x3) & (data[:,0] < x4))] Use lots of parenthesis!

IanRoberts Over a year ago

Thanks, that's even better!

mgilson Over a year ago

FWIW, I think I prefer the "logical_XXX" functions to the & and | operators. The latter actually do bitwise operations. It turns out that they give the same results for arrays of booleans since True is a subclass of 1, but I find that to be a little less clear.

|

mgilson · Accepted Answer · 2013-05-07 13:49:19Z

2

In this case, you want to use numpy.logical_and:

>>> import numpy as np
>>> data = np.arange(100)
>>> data[np.logical_and(50 <= data,data <= 70)]
array([50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66,
       67, 68, 69, 70])

The reason is that with regular and, python looks to the left array and says "Is this true-like?". However, arrays don't have a concept of "true-like" (They refuse to guess what a "True" array would look like), so they raise the exception that you see (suggesting that maybe you meant to use the all or any methods).

answered May 7, 2013 at 13:49

mgilson

312k70 gold badges656 silver badges722 bronze badges

1 Comment

IanRoberts Over a year ago

Thanks, I wasn't aware of the logical and function. That helps greatly.

Collectives™ on Stack Overflow

Pythonic way to access certain parts of an array

2 Answers 2

6 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related