Pandas Multiple String Columns to a List of Integers

Question

I have a dataframe df like this where both columns are object.

    +-----+--------------------+--------------------+
    |  id |         col1       |         col2       |
    +-----+--------------------+--------------------+
    |   1 |  0,1,4,0,1         |  1,2,4,0,0         |
    +-----+--------------------+--------------------+

I convert them into a list like this

test = df["col1"]+','+df["col2"]
test.tolist()

Which produces the following results as a SINGLE STING element in a list

['0,1,4,0,1,1,2,4,0,0']

However, I want them as a list of integers like this

[0,1,4,0,1,1,2,4,0,0]

Any suggestions? Just FYI, the columns are really huge in my original dataset so performance might be an issue too.

Do you have control over how those columns were created in the first place? If performance might be an issue that's the place to spend the effort. — ALollz
– ALollz, Commented Apr 16, 2020 at 20:29

Quang Hoang · Accepted Answer · 2020-04-16 20:08:55Z

5

I think you want:

(df['col1'] + ',' + df['col2']).apply(lambda row: [int(s) for s in row.split(',')])

Output:

0    [0, 1, 4, 0, 1, 1, 2, 4, 0, 0]
dtype: object

answered Apr 16, 2020 at 20:08

Quang Hoang

151k11 gold badges64 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Abu Shoeb Over a year ago

How can I have it as a pure list instead of an object as the output goes to other functions to be processed?

Quang Hoang Over a year ago

does your dataframe have just one row?

Abu Shoeb Over a year ago

Good question. I do have multiple rows but as a preprocess, I already make sure that there is one row comes in at a time. It would be better if the solution can handle multiple rows :-)

Quang Hoang Over a year ago

This solution gives you a series of lists, each for a row. So if you want to pass each list to your function, you can do a loop for r in series: your_func(r), where series is the output above.

Umar.H · Accepted Answer · 2020-04-16 20:10:37Z

3

another method using str.split and explode

arr = df.set_index('id').stack().str.split(',').explode().astype(int).values

print(arr)
array([0, 1, 4, 0, 1, 1, 2, 4, 0, 0])

answered Apr 16, 2020 at 20:10

Umar.H

23.1k7 gold badges50 silver badges94 bronze badges

4 Comments

Abu Shoeb Over a year ago

Encountered with this error AttributeError: 'Series' object has no attribute 'explode'

Umar.H Over a year ago

@AbuShoeb what version of pandas are you on ? it was added in 0.25

Abu Shoeb Over a year ago

I see. Mine is 0.23.4

Umar.H Over a year ago

@AbuShoeb if you can upgrade then do that otherwise you can use stack again after str.split with arg expand=True

MohitC · Accepted Answer · 2020-04-16 23:19:32Z

0

You can do it with map as

 test = str(df["col1"]+','+df["col2"])
 list(map(int, test.split(',')))

edited Apr 16, 2020 at 23:19

answered Apr 16, 2020 at 20:07

MohitC

4,8112 gold badges38 silver badges59 bronze badges

3 Comments

Abu Shoeb Over a year ago

As I mentioned in another answer, would it possible to have the final result as a list instead of map object?

MohitC Over a year ago

Typecast the entire thing to list. Updated

Abu Shoeb Over a year ago

I think I tried it before and tried again now and ended up with ValueError: invalid literal for int() with base 10: '0 1'

Collectives™ on Stack Overflow

Pandas Multiple String Columns to a List of Integers

3 Answers 3

4 Comments

4 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

4 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related