Using an array to filter dataframe in Pandas

Question

I would like to use the output of this ↓↓↓ dataframe column. Using the values of it, I want to filter another dataframe.

 X.ColumnA.unique()
array(['0', '222.33.222.106', '12.12.32.44', '122.122.1.1',
       '122.222.180.150', '142.222.180.142', '222.99.222.78',
       '33.33.221.240', '151.99.222.76', '222.251.222.1',
       '222.250.184.46', '22.33.44.55', ........ ]

I was using some of its values to filter and create another dataframe but I understood that I need all the values above↑↑. How can I pass the array to filter the dataframe?

Y = ((df['ColumnnA'] == "22.33.44.55")
| (df['ColumnnA'] == "12.12.32.44") 
| (df['ColumnnA'] == "45.142.22.22") 
| (df['ColumnnA'] == "55.197.55.8") 
| (df['ColumnnA'] == "44.44.211.254") 
| (df['ColumnnA'] == "33.44.234.83") 
| (df['ColumnnA'] == "33.33.221.240") 
| (df['ColumnnA'] == "33.33.33.1"))
restdataframe = df[~Y]
Y=df[Y]

Here some of the values

df.head(5).to_dict()
{'Column0': {0: 0.00192, 1: 0.0, 2: 0.834324, 3: 8.588816, 4: 2.908711},
 'Column1': {0: '0',
  1: '192.168.1.1',
  2: '22.22.2.15',
  3: '10.22.2.15',
  4: '10.22.22.15'},
 'ColumnA': {0: '0',
  1: '10.0.2.22',
  2: '20.55.22.22',
  3: '22.44.1.1',
  4: '44.33.1.1'},
 'Column2': {0: 'yyy', 1: 'xxx', 2: 'zzz', 3: 'xxx', 4: 'yyy'},
 'Column3': {0: '88', 1: '88', 2: '777', 3: '666', 4: '555'},
 'Column4': {0: '0', 1: '111', 2: '222', 3: '333', 4: '444'},
 'Column5': {0: 0, 1: 1, 2: 17, 3: 8, 4: 4},
 'Column6': {0: 0, 1: 1, 2: 7, 3: 4, 4: 2},
 'Column7': {0: 0, 1: 0, 2: 10, 3: 4, 4: 2},
 'Column8': {0: 0, 1: 110, 2: 5798, 3: 504, 4: 408},
 'Column9': {0: 0, 1: 110, 2: 775, 3: 264, 4: 188},
 'Column10': {0: 0, 1: 0, 2: 5023, 3: 240, 4: 220},
 'Column11': {0: 0, 1: 0, 2: 0, 3: 3, 4: 0},
 'Column12': {0: 'DDD', 1: 'EEE', 2: 'AAA', 3: 'BBB', 4: 'CCC'}}

Himanshu Poddar · Accepted Answer · 2022-08-16 09:36:49Z

1

You can use pd.isin and pass the array to filter the dataframe.

df[df['ColumnA'].isin(X.ColumnA.unique())]

edited Aug 16, 2022 at 9:36

answered Aug 16, 2022 at 9:17

Himanshu Poddar

7,90116 gold badges60 silver badges120 bronze badges

Sign up to request clarification or add additional context in comments.

14 Comments

linuxpanther Over a year ago

I used your solution and created b=df[df.isin({'columnname': X.ColumnA.unique()})] But when I run "b.head()", all other columns are showing "NaN", only ColumnA has some values

Himanshu Poddar Over a year ago

Can you share your dataframe as dict please. like df.to_dict()

Himanshu Poddar Over a year ago

@linuxpanther please try my updated answer and let me know what you get

Himanshu Poddar Over a year ago

Yes you can use np.intersect1d(t1, t2) This will give you all common value between both the array

Himanshu Poddar Over a year ago

What I meant was you cannot use negation operation on a non mask dataframe, mask dataframe means the one containing True and False values only. To negate it please try this df[~df['ColumnA'].isin(X.ColumnA.unique())]

|

Collectives™ on Stack Overflow

Using an array to filter dataframe in Pandas

1 Answer 1

14 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

14 Comments

Your Answer

Sign up or log in

Post as a guest

Related