Python Appending dictionary in Loop

Question

I have several netCDF data files with the same variables. I want to read in multiple data files and store all of the data in a giant dictionary. The problem with my code is that I can append the new dictionary, however, I can no longer separate the elements within the dictionary. For example:

def parseFile(file_name):
    ret_dict = {}
    with Dataset(file_name,'r') as fid:
        ret_dict['time'] = list(fid.variables['time'][:])

If I call this function for one data file, I might have:

>>> ret_dict['time'] 
[1,2,3,4]

for the second file, I might expect:

>>> ret_dict['time'] 
[5,6,7,8]

Within a loop I would like to create another dictionary with all of the values:

>>> new_dict['time']
[1,2,3,4,5,6,7,8]

Here is what I have tried:

new_dict = {}
for f in file_name:
    ret_dict = parseFile(f)
    new_dict.setdefault('time',[]).append(ret_dict['time'])

This yields:

>>> new_dict['time']
[[1,2,3,4],[5,6,7,8]]

and

>>> new_dict['time'][0] 
[1,2,3,4]

I want:

>>> new_dict['time'][0]
[1]

I thought about appending new_dict in a loop, however, the data files are large, and that takes a while. Is there a faster way to append dictionaries while still allowing the individual values to be accessed?

illright · Accepted Answer · 2016-06-13 15:53:54Z

1

Your problem is that you're appending lists to a list, creating a nested list, which is not what you want.

>>> ls = []
>>> ls.append([1, 2, 3])
>>> ls
[[1, 2, 3]]
>>> ls[0]
[1, 2, 3]

There is, however, another method to add items to a list. It's called .extend

>>> ls = []
>>> ls.extend([1, 2, 3])
>>> ls
[1, 2, 3]
>>> ls[0]
1

Use this when you're adding the list like so:

new_dict = {}
for f in file_name:
    ret_dict = parseFile(f)
    new_dict.set_default('time',[]).extend(ret_dict['time'])

This will produce the expected list which you can easily index

By the way, your code will not work because there's no such method as set_default defined for dicts. You probably meant setdefault, which does what you want.

answered Jun 13, 2016 at 15:53

illright

4,0732 gold badges31 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python Appending dictionary in Loop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related