Python convert Dict with multiple keys to Dataframe

Question

I have a Dict created from web data. I start with an empty Dict, like d = {}. I then loop through each web page and store each page of data in d[i] where i starts out as 0, i = 0. I then have a Dict with multiple keys/indexes (I'm not sure what the proper terminology is). I can enter d[0] and see the first page, d[1] to see the second page and so on. I am trying to union each of these keys/indexes so that I can convert this Dict to a Dataframe. The Dict looks like this:

{0:    Age (Days) Number of Items in Queue Number of Items in WIP  \
 3           4                        0                      2
 4           5                        0                      1
 5           6                        0                      2
 6           7                        0                      4
 7           9                        0                      1
 8          12                        0                      2
 9          14                        0                      4
 10         15                        0                      3
 11         17                        0                      1
 12         19                        0                      2
1:    Age (Days) Number of Items in Queue Number of Items in WIP  \
 3           0                        0                      1
 4           1                        0                      8
 5           2                        0                      3
 6           3                        0                      6
 7           4                        0                      1
 8           5                        0                      7
 9           9                        0                      9
 10         10                        0                      4
 11         11                        0                      3
 12         12                        0                      8
 13         13                        0                      2
 14         14                        0                      1
 15         15                        0                      5
 16         16                        0                      6
 17         17                        0                      1
 18         18                        0                      5
 19         19                        0                      7
 20         20                        0                      2}

I can't simply use pd.DataFrame(d). I tried to use pd.DataFrame.from_dict(list(d.values())) but that didn't give the output I was expecting. I get the same output with pd.DataFrame.from_dict(d, orient='index'). The unwanted output is just listing each key/index and looks like this:

In [102]: pd.DataFrame.from_dict(d, orient='index')
Out[102]:
                                                   0
0     Age (Days) Number of Items in Queue Number ...
1     Age (Days) Number of Items in Queue Number ...
2     Age (Days) Number of Items in Queue Number ...

I'm expecting the rows from d[1] to be added to d[0] and keep the same column names.

Try this: df = pd.concat((d[k] for k in d.keys()), ignore_index=True) — MaxU - stand with Ukraine
– MaxU - stand with Ukraine, Commented Jan 3, 2017 at 21:42
That was quick and perfect! Thank you! Want to post this as an answer and I'll mark it? — d84_n1nj4
– d84_n1nj4, Commented Jan 3, 2017 at 21:45

MaxU - stand with Ukraine · Accepted Answer · 2017-01-03 21:44:52Z

1

IIUC you can concatenate them this way:

df = pd.concat(d.values(), ignore_index=True)

answered Jan 3, 2017 at 21:44

MaxU - stand with Ukraine

212k37 gold badges402 silver badges436 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python convert Dict with multiple keys to Dataframe

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related