select DataFrame Rows Based on multiple conditions on columns when column name are in a list

Question

I need to filter rows on certain conditions on some columns. Those columns are present in a list. Condition will be same for all columns or can be different. For my work, condition is same.

Not working

labels = ['one', 'two', 'three']

df = df [df [x] == 1 for x in labels]

Below code works:

df_list = []

for x in labels:

  df_list.append(df[(df [x] == 1)])

df5 = pd.concat(df_list).drop_duplicates()

Creating different dataframes and concating them by avoiding duplicates works.

Expected: It should filter out those rows where value of those column is 1.

Actual: ValueError: too many values to unpack (expected 1)

I understand the reason for the error. Is there any way I can construct the condition by modifying the not working code ?

possible duplicate of this thread: stackoverflow.com/questions/42711186/… — Saahil
– Saahil, Commented Aug 1, 2019 at 13:22

Scott Boston · Accepted Answer · 2019-08-01 13:25:05Z

2

I think you are able to re-write this using the following.

labels = ['one','two','three']

df5 = df[(df[labels] == 1).any(1)]

Let's test with this MCVE:

#Create test data
df = pd.DataFrame(np.random.randint(1,5,(10,5)), columns=[*'ABCDE'])
labels = ['A','B','E']

#Your code
df_list = []
for x in labels:

  df_list.append(df[(df [x] == 1)])

df5 = pd.concat(df_list).drop_duplicates()


#Suggested modification
df6 = df[(df[labels] == 1).any(1)]

Are they equal?

df5.eq(df6)

Output:

      A     B     C     D     E
1  True  True  True  True  True
4  True  True  True  True  True
6  True  True  True  True  True
7  True  True  True  True  True
8  True  True  True  True  True

edited Aug 1, 2019 at 13:25

answered Aug 1, 2019 at 13:18

Scott Boston

154k15 gold badges160 silver badges207 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Lucas Damian · Accepted Answer · 2019-08-01 13:12:19Z

0

Do you need this ?

new_df = df[(df['one'] == 1) & (df['two']== 1) & (df['three'] == 1)].drop_duplicates()

answered Aug 1, 2019 at 13:12

Lucas Damian

1781 gold badge2 silver badges12 bronze badges

Collectives™ on Stack Overflow

select DataFrame Rows Based on multiple conditions on columns when column name are in a list

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related