Merge two numpy arrays and delete duplicates?

Question

I have one numpy array =

[1,6,7,9,3,5]

and a second numpy array =

[3,5,8,9,2]

I would like to merge these two arrays together:

[1,6,7,9,3,5,3,5,8,9,2]

and then remove the duplicates in the numpy array to get :

[1,6,7,9,3,5,8,2]

I would like to keep as much of array one as possible and take out elements of array two, that don't appear in array one, and append these.

I am not sure if it makes more sense to:

merge both arrays and remove duplicates. or
loop through elements of array 2, and if they don't appear in array 1 then concatenate to array 1.

I have tried using various loops but these appear to work mostly for lists, I have also tried using set() but this orders the numpy array, I would like to keep the random order form.

Yes, the order of the values in array 1 is important in this case — Will.S89
– Will.S89, Commented Apr 20, 2018 at 22:18

user3483203 · Accepted Answer · 2018-04-20 22:24:56Z

11

To join the two arrays, you can simply use np.concatenate

Removing duplicates while preserving order is a bit tricky, because normally np.unique also sorts, but you can use return_index then sort to get around this:

In [61]: x
Out[61]: array([1, 6, 7, 9, 3, 5])

In [62]: y
Out[62]: array([3, 5, 8, 9, 2])

In [63]: z = np.concatenate((x, y))

In [64]: z
Out[64]: array([1, 6, 7, 9, 3, 5, 3, 5, 8, 9, 2])

In [65]: _, i = np.unique(z, return_index=True)

In [66]: z[np.sort(i)]
Out[66]: array([1, 6, 7, 9, 3, 5, 8, 2])

edited Apr 20, 2018 at 22:24

answered Apr 20, 2018 at 22:22

user3483203

51.3k10 gold badges72 silver badges104 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Will.S89 Over a year ago

This is perfect, thank you ! can I ask you what ""_, """after in[65] does, please?

user3483203 Over a year ago

When you use return_index, you get a sorted list with no duplicates, as well as the preserved indices. Since I don't care about the sorted list, I just use _ as a throwaway variable.

user3483203 Over a year ago

Basically, np.unique() is returning two results and I only care about the second.

Will.S89 Over a year ago

Could I ask you one last question, please? I have my arrays with an axis=1. I have specified axis=1 in the concat function, the unique and the sort function. It appears to work as far as the sort function I am getting an error: axis 1 is out of bounds for array of dimension 1, are you familiar with this?

user3483203 Over a year ago

I'm surprised the concat function works when you do that. It throws that error for me.

|

Gerges · Accepted Answer · 2018-04-20 22:38:27Z

4

Here is slightly different way to do it also:

z = np.concatenate([x, y[~np.isin(y,x)])

answered Apr 20, 2018 at 22:38

Gerges

6,6492 gold badges28 silver badges49 bronze badges

2 Comments

Will.S89 Over a year ago

Thank you, does this only concat the elements of y that aren't already in x?

Ajay Maity Jan 2 at 9:51

Very creative! :)

Collectives™ on Stack Overflow

Merge two numpy arrays and delete duplicates?

2 Answers 2

6 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related