python join list of list of lists using key

Question

I have this list structure:

lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]]]

'lst' can contain an arbitrary number of sublists (len(lst) can be bigger than 2)

As an output I want:

output = [['a',100,50],['b',200,250],['c',0,75],['d',325,0]]

Here is another example:

lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]], [['a', 22], ['b', 10]]]

output = [['a', 100, 50, 22],['b', 200, 250, 10], ['c', 0, 75, 0], ['d', 325, 0, 0]]

How would you do that?

Will lst always have that structure? IOW, will it always be a list of (lists of (two item lists))? — PM 2Ring
– PM 2Ring, Commented Oct 4, 2017 at 13:32
@RomanPerekhrest It would appear because the first sub list doesn't contain a c "key", so the OP want it to default to 0 — Wondercricket
– Wondercricket, Commented Oct 4, 2017 at 13:33

PM 2Ring · Accepted Answer · 2017-10-04 14:15:48Z

3

This task would be a little simpler if we had a list of all the letter keys used in lst, but it's easy enough to extract them.

My strategy is to convert the sublists into dictionaries. That makes it easy & efficient to grab the value associated with each key. And the dict.get method allows us to supply a default value for missing keys.

lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]]]

# Convert outer sublists to dictionaries
dicts = [*map(dict, lst)]

# Get all the keys
keys = set()
for d in dicts:
    keys.update(d.keys())

# Get data for each key from each dict, using 0 if a key is missing
final = [[k] + [d.get(k, 0) for d in dicts] for k in sorted(keys)]
print(final)

output

[['a', 100, 50], ['b', 200, 250], ['c', 0, 75], ['d', 325, 0]]

If we use

lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]], [['a', 22], ['b', 10]]]

then the output is

[['a', 100, 50, 22], ['b', 200, 250, 10], ['c', 0, 75, 0], ['d', 325, 0, 0]]

If you want to run this on Python 2 you need to make a minor change to the code that converts the outer sublists to dictionaries. Change it to

dicts = list(map(dict, lst))

That will work correctly on both Python 2 & 3. And if you only need to run it on Python 2, you could simply do

dicts = map(dict, lst)

since map in Python 2 return a list, not an iterator.

edited Oct 4, 2017 at 14:15

answered Oct 4, 2017 at 14:06

PM 2Ring

55.6k6 gold badges96 silver badges202 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Wondercricket Over a year ago

Great answer, but it might be good to mention that this is a Python 3.5< solution. The OP hasn't made it clear which version they are using

PM 2Ring Over a year ago

@Wondercricket The SO Python community policy is to assume Python 3, unless Python 2 is explicitly stated. ;) But I'll add some relevant info to my answer.

gimba Over a year ago

good stuff, but, yeah... I am actually on Python 2.x and switching would be pretty painful bc some of the other stuff I use is dependent on it

Ajax1234 · Accepted Answer · 2017-10-04 13:43:41Z

2

You can use a defaultdict:

from collections import defaultdict
import itertools
d = defaultdict(list)
lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]]]
for a, b in itertools.chain.from_iterable(lst):
   d[a].append(b)

new_lst = sorted([list(itertools.chain.from_iterable([[a], [0 for i in range(len(max(d.items(), key=lambda x:len(x[-1])))-len(b))]+b])) for a, b in d.items()])

Output:

[['a', 100, 50], ['b', 200, 250], ['c', 0, 75], ['d', 0, 325]]

edited Oct 4, 2017 at 13:43

answered Oct 4, 2017 at 13:26

Ajax1234

71.7k9 gold badges67 silver badges110 bronze badges

1 Comment

gimba Over a year ago

thx @Ajax1234 your solution works well with a list of two sublists but with more than two sublists like [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]], [['a', 22], ['b': 10]]] I get the output: [['a', 100, 50, 22], ['b', 200, 250, 10], ['c', 0, 75], ['d', 0, 325]] It should be [['a', 100, 50, 22],['b', 200, 250, 10], ['c', 0, 75, 0], ['d', 325, 0, 0]]

RomanPerekhrest · Accepted Answer · 2017-10-04 14:24:46Z

2

With itertools.chain.from_iterable(), itertools.groupby() functions and built-in next() function:

import itertools

lst = [ [['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]], [['a', 22], ['b', 10]] ]
lst_len = len(lst)
sub_keys = [{k[0] for k in _} for _ in lst]
result = [[k] + [next(g)[1] if k in sub_keys[i] else 0 for i in range(lst_len)]
          for k,g in itertools.groupby(sorted(itertools.chain.from_iterable(lst), key=lambda x:x[0]), key=lambda x: x[0])]

print(result)

The output:

[['a', 100, 50, 22], ['b', 200, 250, 10], ['c', 0, 75, 0], ['d', 325, 0, 0]]

edited Oct 4, 2017 at 14:24

answered Oct 4, 2017 at 14:16

RomanPerekhrest

93.1k4 gold badges75 silver badges112 bronze badges

1 Comment

RomanPerekhrest Over a year ago

@PM2Ring, added set for bigger lists

Bill Oldroyd · Accepted Answer · 2017-10-04 14:35:18Z

0

This is my "long-hand" method, I just had to work out what was going on :

lst = [[['a', 100],['b', 200],['d', 325]],
      [['a', 50],['b', 250],['c', 75]],
      [['a', 22], ['b', 10]],
      [['c', 110],['f', 200],['g', 425]],
      [['a', 50],['f', 250],['h', 75]],
      [['a', 32], ['b', 10]], ]
nlist = []
store={}
for n,j in enumerate(lst):
    for i in j  :
        if i[0] in store :
            store[i[0]].append(i[1])
        else :
            store[i[0]] = nlist + [i[1]]
    nlist += [0]
    for k,v in store.items() :
        if len(v) < n+1 :
            store[k] = v + [0]
print(store)
result=[]
for k,v in store.items():
    result += [[k] + v]
print(sorted(result))

answered Oct 4, 2017 at 14:35

Bill Oldroyd

3792 silver badges5 bronze badges

Collectives™ on Stack Overflow

python join list of list of lists using key

4 Answers 4

3 Comments

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related