Using Pandas to determine if values from one CSV file are missing in another CSV file

Question

I am in the process of creating a python script that determines if there are values from a column in file_1 not present in a column from file_2. If there are values from from a column in file_1 not present in a column from file_2, I would like for these to be printed in output.

Currently, I am loading the values into a dataframe using Pandas and then looping through the values using a 'while' loop however, this doesn't seem to be looping through the values as intended. Any idea?

import pandas as pd

dataframe_server_permissions = pd.read_csv("server_permissions.csv")
dataframe_database_permissions = pd.read_csv("database_permissions.csv") 

number_of_values = len(dataframe_server_permissions)

i = 0  

while i < number_of_values:

if dataframe_server_permissions['Permissions'][i] in dataframe_database_permissions['Rights']:      
        print("The permission on the server #" + dataframe_server_permissions['Permissions'][i] + "# exists in the database")
    elif dataframe_server_permissions['Permissions'][i] not in dataframe_database_permissions['Rights']: 
        print("The permission on the server #" + dataframe_server_permissions['Permissions'][1] + "# DOES NOT exist in the database #")
i+=1

Sreekiran A R · Accepted Answer · 2018-12-05 12:48:11Z

1

You can make use of .isin() function without looping

compare=dataframe_database_permissions['Rights'].tolist()

dataframe_server_permissions['flag'] = 
dataframe_server_permissions['Permissions'].isin(compare).astype(int)

#now you can check for values which are not in rights by filtering using flag
print (dataframe_server_permissions[dataframe_server_permissions['flag']==0])

edited Dec 5, 2018 at 12:48

answered Dec 5, 2018 at 11:06

Sreekiran A R

3,4253 gold badges25 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

specbug · Accepted Answer · 2018-12-05 11:12:52Z

0

You can use the isin method which may produce NaN rows, that can be dropped:

dataframe_server_permissions[~dataframe_database_permissions.isin(dataframe_server_permissions)].dropna(how='all')

answered Dec 5, 2018 at 11:12

specbug

5626 silver badges17 bronze badges

Collectives™ on Stack Overflow

Using Pandas to determine if values from one CSV file are missing in another CSV file

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related