python pandas add multiple columns not existing in another list

Question

I have a dataframe whose column names are a subset of the index values of another dataframe.

I would like to add to the initial dataframe empty columns that exists in the second index but do not exist as column names in the first. I have solved this using the following loop but wonder is there a smarter way to do this using, for example, assign or append(axis=1)?

My example below with the output it produces (the correct desired result)

Thanks

import pandas as pd
df_matrix = pd.DataFrame([(1, 2),
                   (3, 4)],
                  index=[['a','b']],
                  columns=('value_1','value_3'))

df_matrix

	value_1	value_3
a	1	2
b	3	4

df_column = pd.DataFrame({'value':[11, 22, 33, 44]}, index=['value_1','value_2','value_3','value_4'])

df_column

	value
value_1	11
value_2	22
value_3	33
value_4	44

for index, row in df_column.iterrows():
    if index not in df_matrix.columns:
        df_matrix[index]=0

updated df_matrix

	value_1	value_3	value_2	value_4
a	1	2	0	0
b	3	4	0	0

Are you sure those are the column names you want in the final output? — Chris
– Chris, Commented Jan 11, 2021 at 17:41

Chris · Accepted Answer · 2021-01-11 17:58:27Z

3

This would essentially do what you want without the loop.

df_matrix[[x for x in df_column.index if x not in df_matrix.columns]] = 0

 value_1  value_3   value_2 value_4
a   1          2        0      0
b   3          4        0      0

answered Jan 11, 2021 at 17:58

Chris

16.3k3 gold badges26 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

FullMetalScientist Over a year ago

Doing this with larger datasets, I got: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling frame.insert many times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use newframe = frame.copy() self[col] = value

Collectives™ on Stack Overflow

python pandas add multiple columns not existing in another list

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related