Convert DataFrame to Array of Objects

Question

Is there a way to convert a Pandas DataFrame into an array of objects (something that is more 'comfortable' to work with in JavaScript)?

I am using Facebook Prophet to run a timeseries forecast and return the data back to the client to do something with it.

I essentially want to take a DataFrame like this:

But, return something like this:

[
    {
        'ds': <value>,
        'trend': <value>,
        'yhat_lower': <value>,
        'yhat_upper': <value>
        ...
    },
    {
        'ds': <value>,
        'trend': <value>,
        'yhat_lower': <value>,
        'yhat_upper': <value>
        ...
    },
    ...
]

I have tried DataFrame.to_json() which is kind of close to what I need, but it presents other issues. I also tried DataFrame.to_dict() and that isn't really what I want either. Same story fo DataFrame.to_records()

Do I really need to loop through the DataFrame manually to build up the list how I want it or is there some parameter/method I'm missing on getting a DataFrame to format as an array of objects with column names as the object key's?

UPDATE

.to_dict() is close to what I want, but there's still a nested object. Is there a way to get rid of that?

{'additive_terms': {0: 1821.6658106578184},
 'additive_terms_lower': {0: 1821.6658106578184},
 'additive_terms_upper': {0: 1821.6658106578184},
 'daily': {0: -904.5939055630084},
 'daily_lower': {0: -904.5939055630084},
 'daily_upper': {0: -904.5939055630084},
 'ds': {0: Timestamp('2016-01-01 00:00:00')},
 'multiplicative_terms': {0: 0.0},
 'multiplicative_terms_lower': {0: 0.0},
 'multiplicative_terms_upper': {0: 0.0},
 'trend': {0: 3959.7525337335633},
 'trend_lower': {0: 3959.7525337335633},
 'trend_upper': {0: 3959.7525337335633},
 'weekly': {0: 1382.1213748832024},
 'weekly_lower': {0: 1382.1213748832024},
 'weekly_upper': {0: 1382.1213748832024},
 'yearly': {0: 1344.1383413376243},
 'yearly_lower': {0: 1344.1383413376243},
 'yearly_upper': {0: 1344.1383413376243},
 'yhat': {0: 5781.418344391382},
 'yhat_lower': {0: -4262.772973874018},
 'yhat_upper': {0: 15333.709906373766}}

UPDATE 2

It looks like @busybear's answer is what I want, however, I want it as an array of objects instead of a large object using the index as the key to the individual record:

{0: {'additive_terms': 1821.6658106578184,
  'additive_terms_lower': 1821.6658106578184,
  'additive_terms_upper': 1821.6658106578184,
  'daily': -904.5939055630084,
  'daily_lower': -904.5939055630084,
  'daily_upper': -904.5939055630084,
  'ds': Timestamp('2016-01-01 00:00:00'),
  'multiplicative_terms': 0.0,
  'multiplicative_terms_lower': 0.0,
  'multiplicative_terms_upper': 0.0,
  'trend': 3959.7525337335633,
  'trend_lower': 3959.7525337335633,
  'trend_upper': 3959.7525337335633,
  'weekly': 1382.1213748832024,
  'weekly_lower': 1382.1213748832024,
  'weekly_upper': 1382.1213748832024,
  'yearly': 1344.1383413376243,
  'yearly_lower': 1344.1383413376243,
  'yearly_upper': 1344.1383413376243,
  'yhat': 5781.418344391382,
  'yhat_lower': -4262.772973874018,
  'yhat_upper': 15333.709906373766},
 1: {'additive_terms': 1609.1847938356425,
  'additive_terms_lower': 1609.1847938356425,
  'additive_terms_upper': 1609.1847938356425,
  'daily': -904.5939055630084,
  'daily_lower': -904.5939055630084,
  'daily_upper': -904.5939055630084,
  'ds': Timestamp('2016-01-02 00:00:00'),
  'multiplicative_terms': 0.0,
  'multiplicative_terms_lower': 0.0,
  'multiplicative_terms_upper': 0.0,
  'trend': 3954.608221609561,
  'trend_lower': 3954.608221609561,
  'trend_upper': 3954.608221609561,
  'weekly': 1056.9172554279028,
  'weekly_lower': 1056.9172554279028,
  'weekly_upper': 1056.9172554279028,
  'yearly': 1456.8614439707483,
  'yearly_lower': 1456.8614439707483,
  'yearly_upper': 1456.8614439707483,
  'yhat': 5563.793015445203,
  'yhat_lower': -4892.457856774376,
  'yhat_upper': 15305.24188601227}}

It's close. It comes back with a nested object which is what I don't want. — mwilson
– mwilson, Commented Feb 5, 2019 at 17:13

anky · Accepted Answer · 2019-02-05 17:18:04Z

15

Try This:

df.to_dict('records')

It gives you a list of dictionary without the index.

answered Feb 5, 2019 at 17:18

anky

75.3k11 gold badges46 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

mwilson Over a year ago

This solution is shorter than busybears and gives the same result. I think this makes a little more sense.

busybear · Accepted Answer · 2019-02-05 17:17:54Z

3

to_dict sounds like what you want actually. How is it not working for you? You could transpose your dataframe and take just the values of the dictionary. that will match what you expect:

data = df.T.to_dict()
list(data.values())

edited Feb 5, 2019 at 17:17

answered Feb 5, 2019 at 17:14

busybear

10.7k1 gold badge29 silver badges44 bronze badges

3 Comments

mwilson Over a year ago

Even closer! Is there a way to kill the index and make an array of objects rather than one large object with the index as the key?

anky Over a year ago

@mwilson can you print a sample please, would be helpful

busybear Over a year ago

Is that not accomplished with values? That will take just the values of the dictionary and not the keys. You may want to convert it to a list since it's a dict_value object.

Rui Nian · Accepted Answer · 2023-02-08 22:55:58Z

0

To do so:

df.to_json(orient="records")

That should get you to exactly what you need. Other orientations are split, index, and table.

For more information, see: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_json.html

answered Feb 8, 2023 at 22:55

Rui Nian

3,0355 gold badges24 silver badges39 bronze badges

Collectives™ on Stack Overflow

Convert DataFrame to Array of Objects

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related