Efficient implementation to check: for each element in array check if it exists in another array

Question

I have two large 1d numpy arrays in the range of 400K elements. I need to check for each element in array A if it exists in array B. I used in1d but it seems to be too slow and takes a lot of time. I would like to know if there is any way to speed this up?

A = np.array([1,2,3,4,5,6,7]) 
B = np.array([3,4,7])
result = np.in1d(A, B, invert=True)
result
>> array([ True,  True, False, False,  True,  True, False]

Please post your code, and a minimum running example.

Dinari
– Dinari

2018-12-17 06:05:33 +00:00
Commented Dec 17, 2018 at 6:05 — Dinari
– Dinari, Commented Dec 17, 2018 at 6:05
Perhaps, a sample I/O?

DirtyBit
– DirtyBit

2018-12-17 06:13:16 +00:00
Commented Dec 17, 2018 at 6:13 — DirtyBit
– DirtyBit, Commented Dec 17, 2018 at 6:13
@Dinari I have updated it

frank
– frank

2018-12-17 06:19:54 +00:00
Commented Dec 17, 2018 at 6:19 — frank
– frank, Commented Dec 17, 2018 at 6:19
@user5173426 I have updated it

frank
– frank

2018-12-17 06:20:07 +00:00
Commented Dec 17, 2018 at 6:20 — frank
– frank, Commented Dec 17, 2018 at 6:20

Roy Shahaf · Accepted Answer · 2018-12-17 06:20:41Z

3

Try transforming B into a structure better fitted for search (hash set or sorted set)

answered Dec 17, 2018 at 6:20

Roy Shahaf

4945 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

koPytok · Accepted Answer · 2018-12-17 06:33:01Z

1

I prefer pandas for that task:

import pandas as pd

A, B = pd.DataFrame(A), pd.DataFrame(B)
A.merge(B, on=0, how="left", indicator=True)

>>> 0   _merge
0   1   left_only
1   2   left_only
2   3   both
3   4   both
4   5   left_only
5   6   left_only
6   7   both

answered Dec 17, 2018 at 6:33

koPytok

3,7631 gold badge16 silver badges29 bronze badges

1 Comment

frank Over a year ago

Unique solution. Thanks a lot.

Collectives™ on Stack Overflow

Efficient implementation to check: for each element in array check if it exists in another array

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related