minimize runtime for numpy array manipulation

Question

I have an 2 dimensional array with np.shape(input)=(a,b) and that looks like

input=array[array_1[0,0,0,1,0,1,2,0,3,3,2,...,entry_b],...array_a[1,0,0,1,2,2,0,3,1,3,3,...,entry_b]]

Now I want to create an array np.shape(output)=(a,b,b) in which every entry that had the same value in the input get the value 1 and 0 otherwise

for example:

input=[[1,0,0,0,1,2]]

output=[array([[1., 0., 0., 0., 1., 0.],
               [0., 1., 1., 1., 0., 0.],
               [0., 1., 1., 1., 0., 0.],
               [0., 1., 1., 1., 0., 0.],
               [1., 0., 0., 0., 1., 0.],
               [0., 0., 0., 0., 0., 1.]])]

My code so far is looking like:

def get_matrix(svdata,padding_size):
List=[]
for k in svdata:
    matrix=np.zeros((padding_size,padding_size))
    for l in range(padding_size):
        for m in range(padding_size):
            if k[l]==k[m]:
                matrix[l][m]=1
    List.append(matrix)
return List

But it takes 2:30 min for an input array of shape (2000,256). How can I become more effiecient by using built in numpy solutions?

hpaulj · Accepted Answer · 2020-12-01 02:46:14Z

2

res = input[:,:,None]==input[:,None,:]

Should give boolean (a,b,b) array

res = res.astype(int)

to get a 0/1 array

answered Dec 1, 2020 at 2:46

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

SergeantIdiot Over a year ago

This seems not to work for my case. I looked up and the shape of my input is (2000,256,1) could this be the problem? where as a=2000 and b=256

hpaulj Over a year ago

Yes, my answer is based on the original claim that it is (a,b) shape. WIth the None, the 2 shapes become (a,b,1) and (a,1,b) which broadcast together to (a,b,b). So what do you want to do with the trailing size one? You could leave it in one, and swap axes in the other.

SergeantIdiot Over a year ago

Hey, I used np.squeeze() to get rid of the unwanted dimension and it works now like intended. Thank you for your help!

Frank Yellin · Accepted Answer · 2020-12-01 02:53:13Z

1

You're trying to create the array y where y[i,j,k] is 1 if input[i,j] == input[i, k]. At least that's what I think you're trying to do.

So y = input[:,:,None] == input[:,None,:] will give you a boolean array. You can then convert that to np.dtype('float64') using astype(...) if you want.

answered Dec 1, 2020 at 2:53

Frank Yellin

11.6k1 gold badge15 silver badges27 bronze badges

Collectives™ on Stack Overflow

minimize runtime for numpy array manipulation

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related