Pandas - adding new string column results in NaN

Question

I've had some trouble assigning a new column in a Pandas dataframe - I have got it working, but want to understand why it happens;

When I first tried to assign the ID to a string, the result was NaN..

df = pandas.json_normalize(data)
all_per = pandas.DataFrame()
for person in peopleList:
    all_per['id'] = person
    all_per['name'] = df['results.(id:'+person+').localizedFirstName'] + ' ' + \
                      df['results.(id:'+person+').localizedLastName']

Results:

    id          name
0  NaN    Adam Smith

However if I move the ID assignment down a bit, it works..

df = pandas.json_normalize(data)
all_per = pandas.DataFrame()
for person in peopleList:
    all_per['name'] = df['results.(id:'+person+').localizedFirstName'] + ' ' + \
                      df['results.(id:'+person+').localizedLastName']
    all_per['id'] = person

Results:

           name          id
0    Adam Smith    FQR4bL_80K

This took up a lot of my time, and I have no idea why it happened? Any ideas?

Corralien · Accepted Answer · 2021-10-28 08:28:05Z

1

You can't add a scalar value. You have to enclose person into a list:

df = pandas.json_normalize(data)
all_per = pandas.DataFrame()
for person in peopleList:
    all_per['id'] = [person]  # <- HERE
    all_per['name'] = df['results.(id:'+person+').localizedFirstName'] + ' ' + \
                      df['results.(id:'+person+').localizedLastName']

Output:

>>> all_per

           id        name
0  FQR4bL_80K  Adam Smith

answered Oct 28, 2021 at 8:28

Corralien

121k8 gold badges44 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Bob2366172 Over a year ago

Oh thank you - right, yes I'm adding a scalar value which doesn't make sense as I can have multiple people in the list.. I'll use all_per.append . Much appreciated

Corralien Over a year ago

This is not the best strategy. You should collect your data into python data structure (list or dict) then merge them with pd.concat at the end of your loop.

Collectives™ on Stack Overflow

Pandas - adding new string column results in NaN

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related