Selecting rows from one DataFrame depending on values from another

Question

Take two data frames

print(df1)

   A  B
0  a  1
1  a  3
2  a  5
3  b  7
4  b  9
5  c  11
6  c  13
7  c  15

print(df2)

   C     D
a  apple 1
b  pear  1
c  apple 1

So the values in column df1['A'] are the indexes of df2.

I want to select the rows in df1 where the values in column A are 'apple' in df2['C']. Resulting in:

adir abargil · Accepted Answer · 2020-12-17 18:01:35Z

2

Made many edits due to comments and question edits, Basically you first extract the indexes of df2 by filtering the dataframe by values in C, then filter the df2 by indexes with isin

indexes = df2[df2['C']=='apple'].index
df1[df1['A'].isin(indexes)]
>>>
   A  B
0  a  1
1  a  3
2  a  5
5  c  11
6  c  13
7  c  15

UPDATE

If you want to minimize memory allocation try to prevent saving information, (note. That i am not sure ot will solve your menory allocation issue because i didnt have full details of the situation and maybe even not suited enough to provide a solution):

df1[df1['A'].isin( df2[df2['C']=='apple'].index)]

edited Dec 17, 2020 at 18:01

answered Dec 17, 2020 at 17:47

adir abargil

5,7453 gold badges23 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

AndrewK Over a year ago

Thanks. This works for my smaller data frames but but my df1 has 600 rows and my df2 has 700,000 rows and I get a memory error.

AndrewK Over a year ago

Thanks for your help but still memory error. I'll try structure the data frames differently.

adir abargil Over a year ago

Good luck! Was nice to help

Collectives™ on Stack Overflow

Selecting rows from one DataFrame depending on values from another

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related