how to convert pandas dataframe convert string array to array

Question

I am going to read a csv file and send to json stream format.
One of the coulmn is in array format but in pandas dataframe it will return string format.
How cany I convert "attributes_changed" column from string array to normal array format?

Sample Code:

import json
import pandas as pd

sample = {"event_id": "dd9726c4-9c22-47b3-9a1b-fd6ecf494076", "event_type": "UPDATED", "attributes_changed": "[\'metadataStatus\', \'elements\']"}
df = pd.DataFrame(data=sample, index=[0])

df_with_seq_id = [(json.dumps(rec), idx) for (idx, rec) in df.to_dict(orient='index').items()]

Current Result

[('{"event_id": "dd9726c4-9c22-47b3-9a1b-fd6ecf494076", "event_type": "UPDATED", "attributes_changed": "['metadataStatus', 'elements']"}', 0)]

Expected Result (Removed double quotes)

[('{"event_id": "dd9726c4-9c22-47b3-9a1b-fd6ecf494076", "event_type": "UPDATED", "attributes_changed": ['metadataStatus', 'elements']}', 0)]

Antoine Dubuis · Accepted Answer · 2021-12-03 12:19:09Z

2

You can simply apply eval on column attributes_changed to interpret the string values as an array:

import json
import pandas as pd

sample = {
    "event_id": "dd9726c4-9c22-47b3-9a1b-fd6ecf494076",
    "event_type": "UPDATED",
    "attributes_changed": "['metadataStatus', 'elements']",
}
df = pd.DataFrame(data=sample, index=[0])
df["attributes_changed"] = df["attributes_changed"].apply(eval)
df_with_seq_id = [
    (json.dumps(rec), idx) for (idx, rec) in df.to_dict(orient="index").items()
]

This produce the following array:

[('{"event_id": "dd9726c4-9c22-47b3-9a1b-fd6ecf494076", "event_type": "UPDATED", "attributes_changed": ["metadataStatus", "elements"]}',
  0)]

answered Dec 3, 2021 at 12:19

Antoine Dubuis

5,3641 gold badge19 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sivaguru Over a year ago

This is streaming data, so I am not sure about column name. Is there any way to do this in generic way without using column name?

Collectives™ on Stack Overflow

how to convert pandas dataframe convert string array to array

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related