Sorting numpy array with lexsort. Alternative to pandas sort_values

Question

I would like the order of b to equal the result of a below.

The result of b is not as I expected. I thought it would be sorting by the columns in ascending order but I think I misunderstood how lexsort works.

My goal is to be able to sort an array the way the df below is sorted. I'm using lexsort because I think it would be the best thing to use for an array that also contained categorical values.

import numpy as np
import pandas as pd

x = np.array([[1,2],[1,3],[1,4],[2,1],[2,2],[2,3],[2,4],[0,1],[1,0],[0,2]])

a=pd.DataFrame(x).sort_values(by=[0,1], ascending=[0,1])

b=x[np.lexsort((x[:,1],x[:,0][::-1]))]

print(a)
print(b)

sammywemmy · Accepted Answer · 2021-10-19 11:55:22Z

2

From the docs, it should be last, first to get the sort order:

sorter = np.lexsort((x[:, 1], x[:, 0]))

x[sorter][::-1] # sorting in descending order
Out[899]: 
array([[2, 4],
       [2, 3],
       [2, 2],
       [2, 1],
       [1, 4],
       [1, 3],
       [1, 2],
       [1, 0],
       [0, 2],
       [0, 1]])

To simulate descending on one end, with ascending on the other, you could combine np.unique, with np.split and np.concatenate:

temp = x[sorter]
_, s = np.unique(temp[:, 0], return_counts=True)
np.concatenate(np.split(temp, s.cumsum())[::-1])
Out[972]: 
array([[2, 1],
       [2, 2],
       [2, 3],
       [2, 4],
       [1, 0],
       [1, 2],
       [1, 3],
       [1, 4],
       [0, 1],
       [0, 2]])

edited Oct 19, 2021 at 11:55

answered Oct 19, 2021 at 10:40

sammywemmy

28.9k4 gold badges21 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Kdog Over a year ago

My real data will have many columns and a mix of ascending 0/1 settings. Your solution works on the example but what if the order had to be ascending=[0,1]?

sammywemmy Over a year ago

Any particular reason y u r not using pandas for this?

Kdog Over a year ago

I have to do this operation millions of times and trying to cut down on the time. The pandas code takes 853 microseconds and numpy takes 2.78. I thought there has to be a simple way to not rely on pandas but it is more tricky than it looks.

sammywemmy Over a year ago

you could do an np.spit with the return counts, but then you lose some speed; added it to the example. you could probably write a function and adjust it sort of

Collectives™ on Stack Overflow

Sorting numpy array with lexsort. Alternative to pandas sort_values

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related