Python: Trying to create nested dictionary from CSV File, but getting KeyError

Question

I am currently am trying to create a nested dictionary from a csv file.

The CSV File represents how many people there are for each demographic region. In the nested dictionary each key is a region and the value is another dictionary. The inner dictionary uses the demographic as key and the number of people for its value.

Region,American,Asian,Black
midwest,2500,2300,2150
north,1200,2300,2300
south,1211,211,2100

Currently have:

def load_csv(filename):
data={}
    with open(filename) as csvfile:
        fh = csv.DictReader(csvfile)
        for row in fh:
            key = row.pop('Region')
            data[key] = row
        return data

Expected Output (must convert the numbers from strings to integers):

{'west':{'American': 2500, 'ASIAN': 2300, ...}, 'north':{'American': 1200, ..}...}

I'm getting stuck when running my code as it is giving me "KeyError: 'Region'"

This would be so much shorter with pandas .read_csv and .to_dict : just def load_csv(filename): return pd.read_csv(filename, index_col=0).to_dict('index') [This just takes the first column as index, no matter what it is; you could also use index_col='Region' and it would raise error if Region column was missing] — Driftr95
– Driftr95, Commented Feb 8, 2023 at 22:32

Corralien · Accepted Answer · 2023-02-08 23:20:21Z

1

Use a comprehension to convert string values to integers:

import csv

def load_csv(filename):
    data = {}
    with open(filename) as csvfile:
        # Your file has 3 invisible characters at the beginning, skip them
        csvfile.seek(3)
        fh = csv.DictReader(csvfile)
        for row in fh:
            key = row.pop('Region')
            data[key] = {k: int(v) for k, v in row.items()}  # <- HERE
        return data

data = load_csv('data.csv')

Output:

>>> data
{'midwest': {'American': 2500, 'Asian': 2300, 'Black': 2150},
 'north': {'American': 1200, 'Asian': 2300, 'Black': 2300},
 'south': {'American': 1211, 'Asian': 211, 'Black': 2100}}

Bonus: The same operation with Pandas:

import pandas as pd

data = pd.read_csv('data.csv', index_col='Region').T.to_dict()
print(data)

# Output
{'midwest': {'American': 2500, 'Asian': 2300, 'Black': 2150},
 'north': {'American': 1200, 'Asian': 2300, 'Black': 2300},
 'south': {'American': 1211, 'Asian': 211, 'Black': 2100}}

edited Feb 8, 2023 at 23:20

answered Feb 8, 2023 at 22:19

Corralien

121k8 gold badges43 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

15 Comments

Ak2012 Over a year ago

Hi, using first method still getting the KeyError do you know why this could be? It is spelled Region in the file

Corralien Over a year ago

As I haven't this problem on your sample, I guess there is something wrong with your file. Can you share it?

Ak2012 Over a year ago

Yes, how do I share it?

Ak2012 Over a year ago

I have attached a picture of the csv file in VSCode

Corralien Over a year ago

You can use wetransfer or google drive?

|

Driftr95 · Accepted Answer · 2023-02-12 03:23:53Z

0

This solution requires no imports at all, but will only work if there are no escaped separators [i.e., none of the values contain a , or newline]:

def load_csv(filename, sep=','):
    with open(filename, 'r') as csvfile:
        csvlines = csvfile.read().strip().splitlines()
    csvRows = [[v.strip() for v in l.split(sep)] for l in csvlines]
    if not csvRows: return {}
    keys = csvRows[0][1:]
    return {r[0]: dict(zip(keys, r[1:])) for r in csvRows[1:] if r}

At the very least, it works for the csv in your snippet, but if your csv contains , or newlines at any unexpected positions, this function will no longer be reliable - it would definitely be better to use a module built for reading and parsing csv.

answered Feb 12, 2023 at 3:23

Driftr95

4,7792 gold badges13 silver badges27 bronze badges

Collectives™ on Stack Overflow

Python: Trying to create nested dictionary from CSV File, but getting KeyError

2 Answers 2

15 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

15 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related