Python converting a dataframe of string type list to list

Question

I have a unique situation. I get a data from the field in row format. Each row contains a timestamp, a list of values. It is in string format. I am trying to convert it to a normal list.

My code:

df=
     A
0    '1.2,1.3'
1    '2.2,2.3'   
2    '3.2,3.3,'
3    '4.2,4.3'
import ast
df['A'] = df['A'].applymap(ast.literal_eval).applymap(list)

Present output:

ValueError: malformed node or string: 
[3.2,3.3]

Expected output:

df=
     A
0    [1.2,1.3]
1    [2.2,2.3]   
2    [3.2,3.3]
3    [4.2,4.3]

Andrej Kesely · Accepted Answer · 2021-08-11 14:10:18Z

2

Try:

import ast

df["A"] = df["A"].apply(lambda x: ast.literal_eval("[" + x + "]"))
print(df)

Prints:

            A
0  [1.2, 1.3]
1  [2.2, 2.3]
2  [3.2, 3.3]
3  [4.2, 4.3]

If there are ' in the strings:

df["A"] = (
    df["A"].str.strip("'").apply(lambda x: ast.literal_eval("[" + x + "]"))
)

answered Aug 11, 2021 at 14:10

Andrej Kesely

196k15 gold badges60 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Mainland Over a year ago

The first part gave error: ValueError: malformed node or string: timestamp 2019-02-05 10:52:33 , second part gave error: AttributeError: 'DataFrame' object has no attribute 'str'

Andrej Kesely Over a year ago

@Mainland Can you please update your question with real data? It seems, that the strings don't contain numbers in format xx.yy.

Mainland Over a year ago

The data is so big. I presented here an example. How do I share the big data?

Andrej Kesely Over a year ago

@Mainland It doesn't have to by big. Just the sample that produces the error.

Andrej Kesely Over a year ago

@Mainland You can try to use lambda x: ast.literal_eval("[" + x.strip(",") + "]")

|

Nathan Furnal · Accepted Answer · 2021-08-11 14:16:59Z

0

Try :

df["A"].str.replace("'", "").str.rstrip(",").str.split(",").apply(lambda x : list(map(lambda y : float(y), x)))

This cleans up the string for trailing commas and casts the values to floats.

answered Aug 11, 2021 at 14:16

Nathan Furnal

2,4503 gold badges14 silver badges28 bronze badges

Comments

Mabel Villalba · Accepted Answer · 2021-08-11 14:40:27Z

0

One option can be to use pandas.str.extractall combined with pd.to_numeric:

df.apply(
     lambda s: s.str.extractall(r'(\d+(?:\.\d+))').values.ravel(),axis=1)
  .apply(pd.Series)
  .apply(pd.to_numeric)

     0    1
0  1.2  1.3
1  2.2  2.3
2  3.2  3.3
3  4.2  4.3

answered Aug 11, 2021 at 14:40

Mabel Villalba

2,59811 silver badges20 bronze badges

Collectives™ on Stack Overflow

Python converting a dataframe of string type list to list

3 Answers 3

7 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related