Sorting and Grouping Nested Lists in Python

Question

I have the following data structure (a list of lists)

[
 ['4', '21', '1', '14', '2008-10-24 15:42:58'], 
 ['3', '22', '4', '2somename', '2008-10-24 15:22:03'], 
 ['5', '21', '3', '19', '2008-10-24 15:45:45'], 
 ['6', '21', '1', '1somename', '2008-10-24 15:45:49'], 
 ['7', '22', '3', '2somename', '2008-10-24 15:45:51']
]

I would like to be able to

Use a function to reorder the list so that I can group by each item in the list. For example I'd like to be able to group by the second column (so that all the 21's are together)
Use a function to only display certain values from each inner list. For example I'd like to reduce this list to only contain the 4^th field value of '2somename'

so the list would look like this

[
     ['3', '22', '4', '2somename', '2008-10-24 15:22:03'], 
     ['7', '22', '3', '2somename', '2008-10-24 15:45:51']
]

minor point, but you probably should use tuples instead of the inner lists — user3850
– user3850, Commented Jan 4, 2009 at 0:26

Sarai · Accepted Answer · 2018-04-02 20:59:46Z

83

For the first question, the first thing you should do is sort the list by the second field using itemgetter from the operator module:

x = [
 ['4', '21', '1', '14', '2008-10-24 15:42:58'], 
 ['3', '22', '4', '2somename', '2008-10-24 15:22:03'], 
 ['5', '21', '3', '19', '2008-10-24 15:45:45'], 
 ['6', '21', '1', '1somename', '2008-10-24 15:45:49'], 
 ['7', '22', '3', '2somename', '2008-10-24 15:45:51']
]

from operator import itemgetter

x.sort(key=itemgetter(1))

Then you can use itertools' groupby function:

from itertools import groupby
y = groupby(x, itemgetter(1))

Now y is an iterator containing tuples of (element, item iterator). It's more confusing to explain these tuples than it is to show code:

for elt, items in groupby(x, itemgetter(1)):
    print(elt, items)
    for i in items:
        print(i)

Which prints:

21 <itertools._grouper object at 0x511a0>
['4', '21', '1', '14', '2008-10-24 15:42:58']
['5', '21', '3', '19', '2008-10-24 15:45:45']
['6', '21', '1', '1somename', '2008-10-24 15:45:49']
22 <itertools._grouper object at 0x51170>
['3', '22', '4', '2somename', '2008-10-24 15:22:03']
['7', '22', '3', '2somename', '2008-10-24 15:45:51']

For the second part, you should use list comprehensions as mentioned already here:

from pprint import pprint as pp
pp([y for y in x if y[3] == '2somename'])

Which prints:

[['3', '22', '4', '2somename', '2008-10-24 15:22:03'],
 ['7', '22', '3', '2somename', '2008-10-24 15:45:51']]

edited Apr 2, 2018 at 20:59

Sarai

881 silver badge5 bronze badges

answered Jan 3, 2009 at 17:29

llimllib

3,7321 gold badge31 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

jfs Over a year ago

I've added the list comprehension example.

llimllib Over a year ago

This answer was written a long time ago and you should nowadays use a generator expression instead of a list comprehension: pp(y for y in x if y[3] == '2somename')

jfs Over a year ago

no. It is wrong. genexpr is not appropriate here. Try to run the code.

Max Over a year ago

isn't operator.itemgetter(1) the same as lambda x: x[1] and why would one prefer to import a library instead of directly using such a simple function?

llimllib Over a year ago

Functionally the same, yup. I imported it from the stdlib here because I used it twice and I felt that it improved clarity rather than repeating two lambdas, but use whichever you prefer.

morteza kavakebi · Accepted Answer · 2018-08-16 16:56:48Z

28

If you assigned it to var "a"...

python 2.x:

#1:

a.sort(lambda x,y: cmp(x[1], y[1]))

#2:

filter(lambda x: x[3]=="2somename", a)

python 3:

#1:

a.sort(key=lambda x: x[1])

edited Aug 16, 2018 at 16:56

morteza kavakebi

1,6803 gold badges19 silver badges40 bronze badges

answered Jan 3, 2009 at 17:26

Jimmy2Times

6835 silver badges11 bronze badges

3 Comments

Hamman Samuel Over a year ago

Simpler and cleaner approach than itemgetter

alfredocambera Over a year ago

lambda for the win. I really liked this solution

Aster Over a year ago

See performance comparison here: stackoverflow.com/questions/17243620/…

Federico A. Ramponi · Accepted Answer · 2009-01-03 17:31:13Z

3

If I understand your question correctly, the following code should do the job:

l = [
 ['4', '21', '1', '14', '2008-10-24 15:42:58'], 
 ['3', '22', '4', '2somename', '2008-10-24 15:22:03'], 
 ['5', '21', '3', '19', '2008-10-24 15:45:45'], 
 ['6', '21', '1', '1somename', '2008-10-24 15:45:49'], 
 ['7', '22', '3', '2somename', '2008-10-24 15:45:51']
]

def compareField(field):
   def c(l1,l2):
      return cmp(l1[field], l2[field])
   return c

# Use compareField(1) as the ordering criterion, i.e. sort only with
# respect to the 2nd field
l.sort(compareField(1))
for row in l: print row

print
# Select only those sublists for which 4th field=='2somename'
l2somename = [row for row in l if row[3]=='2somename']
for row in l2somename: print row

Output:

['4', '21', '1', '14', '2008-10-24 15:42:58']
['5', '21', '3', '19', '2008-10-24 15:45:45']
['6', '21', '1', '1somename', '2008-10-24 15:45:49']
['3', '22', '4', '2somename', '2008-10-24 15:22:03']
['7', '22', '3', '2somename', '2008-10-24 15:45:51']

['3', '22', '4', '2somename', '2008-10-24 15:22:03']
['7', '22', '3', '2somename', '2008-10-24 15:45:51']

edited Jan 3, 2009 at 17:31

answered Jan 3, 2009 at 17:17

Federico A. Ramponi

47.2k31 gold badges113 silver badges134 bronze badges

2 Comments

Aaron Maenpaa Over a year ago

The 'cmp' argument to sort is being removed in 2.6/3.0, thus, it is preferable to use the 'key' parameter which extracts a sort key, but otherwise, +1.

Federico A. Ramponi Over a year ago

removed 'cmp=', should be the first argument anyway. By the way, I'm using python 2.6.1 and all works fine...

Aaron Maenpaa · Accepted Answer · 2009-01-03 17:27:29Z

2

Use a function to reorder the list so that I can group by each item in the list. For example I'd like to be able to group by the second column (so that all the 21's are together)

Lists have a built in sort method and you can provide a function that extracts the sort key.

>>> import pprint
>>> l.sort(key = lambda ll: ll[1])
>>> pprint.pprint(l)
[['4', '21', '1', '14', '2008-10-24 15:42:58'],
 ['5', '21', '3', '19', '2008-10-24 15:45:45'],
 ['6', '21', '1', '1somename', '2008-10-24 15:45:49'],
 ['3', '22', '4', '2somename', '2008-10-24 15:22:03'],
 ['7', '22', '3', '2somename', '2008-10-24 15:45:51']]

Use a function to only display certain values from each inner list. For example i'd like to reduce this list to only contain the 4th field value of '2somename'

This looks like a job for list comprehensions

>>> [ll[3] for ll in l]
['14', '2somename', '19', '1somename', '2somename']

answered Jan 3, 2009 at 17:27

Aaron Maenpaa

124k11 gold badges99 silver badges108 bronze badges

1 Comment

jfs Over a year ago

Replace [ll[3] for ll in l] by [ll for ll in l if ll[3] == '2somename'] and fix the output.

Kenan Banks · Accepted Answer · 2009-01-03 17:46:56Z

2

If you'll be doing a lot of sorting and filtering, you may like some helper functions.

m = [
 ['4', '21', '1', '14', '2008-10-24 15:42:58'], 
 ['3', '22', '4', '2somename', '2008-10-24 15:22:03'], 
 ['5', '21', '3', '19', '2008-10-24 15:45:45'], 
 ['6', '21', '1', '1somename', '2008-10-24 15:45:49'], 
 ['7', '22', '3', '2somename', '2008-10-24 15:45:51']
]

# Sort and filter helpers.
sort_on   = lambda pos:     lambda x: x[pos]
filter_on = lambda pos,val: lambda l: l[pos] == val

# Sort by second column
m = sorted(m, key=sort_on(1))

# Filter on 4th column, where value = '2somename'
m = filter(filter_on(3,'2somename'),m)

edited Jan 3, 2009 at 17:46

answered Jan 3, 2009 at 17:37

Kenan Banks

213k36 gold badges160 silver badges176 bronze badges

6 Comments

jfs Over a year ago

sort_on == operator.itemgetter

S.Lott Over a year ago

Please use DEF's instead of lambdas.

Kenan Banks Over a year ago

@ s.lott - why defs over lambdas here?

nosklo Over a year ago

@Triptych: because lambdas with a name are just like defs but more confusing for absolutely no benefit.

Kenan Banks Over a year ago

Meh. In this case, I think lambda's are more readable. And "more confusing" is certainly subjective!

|

Adrian Bool · Accepted Answer · 2009-01-03 18:15:03Z

2

For part (2), with x being your array, I think you want,

[y for y in x if y[3] == '2somename']

Which will return a list of just your data lists that have a fourth value being '2somename'... Although it seems Kamil is giving you the best advice with going for SQL...

answered Jan 3, 2009 at 18:15

Adrian Bool

Comments

Community · Accepted Answer · 2017-05-23 12:03:08Z

1

It looks a lot like you're trying to use a list as a database.

Nowadays Python includes sqlite bindings in the core distribution. If you don't need persistence, it's really easy to create an in-memory sqlite database (see How do I create a sqllite3 in-memory database?).

Then you can use SQL statements to do all this sorting and filtering without having to reinvent the wheel.

edited May 23, 2017 at 12:03

CommunityBot

11 silver badge

answered Jan 3, 2009 at 17:56

Kamil Kisiel

20.6k13 gold badges50 silver badges57 bronze badges

1 Comment

m3clov3n Over a year ago

Kamil, you are correct. However I am learning Python and wanted to do things using lists so that I can learn some more about them. I will check this out though thanks

S.Lott · Accepted Answer · 2009-01-03 19:39:45Z

You're simply creating indexes on your structure, right?

>>> from collections import defaultdict
>>> def indexOn( things, pos ):
...     inx= defaultdict(list)
...     for t in things:
...             inx[t[pos]].append(t)
...     return inx
... 
>>> a=[
...  ['4', '21', '1', '14', '2008-10-24 15:42:58'], 
...  ['3', '22', '4', '2somename', '2008-10-24 15:22:03'], 
...  ['5', '21', '3', '19', '2008-10-24 15:45:45'], 
...  ['6', '21', '1', '1somename', '2008-10-24 15:45:49'], 
...  ['7', '22', '3', '2somename', '2008-10-24 15:45:51']
... ]

Here's your first request, grouped by position 1.

>>> import pprint
>>> pprint.pprint( dict(indexOn(a,1)) )
{'21': [['4', '21', '1', '14', '2008-10-24 15:42:58'],
        ['5', '21', '3', '19', '2008-10-24 15:45:45'],
        ['6', '21', '1', '1somename', '2008-10-24 15:45:49']],
 '22': [['3', '22', '4', '2somename', '2008-10-24 15:22:03'],
        ['7', '22', '3', '2somename', '2008-10-24 15:45:51']]}

Here's your second request, grouped by position 3.

>>> dict(indexOn(a,3))
{'19': [['5', '21', '3', '19', '2008-10-24 15:45:45']], '14': [['4', '21', '1', '14', '2008-10-24 15:42:58']], '2somename': [['3', '22', '4', '2somename', '2008-10-24 15:22:03'], ['7', '22', '3', '2somename', '2008-10-24 15:45:51']], '1somename': [['6', '21', '1', '1somename', '2008-10-24 15:45:49']]}
>>> pprint.pprint(_)
{'14': [['4', '21', '1', '14', '2008-10-24 15:42:58']],
 '19': [['5', '21', '3', '19', '2008-10-24 15:45:45']],
 '1somename': [['6', '21', '1', '1somename', '2008-10-24 15:45:49']],
 '2somename': [['3', '22', '4', '2somename', '2008-10-24 15:22:03'],
               ['7', '22', '3', '2somename', '2008-10-24 15:45:51']]}

Adarsh Shukla · Accepted Answer · 2021-09-27 14:30:47Z

You can use for loop to sort and group the elements in the nested list. The code will be:

l = [['3', '21', '1', '14', '2008-10-24 15:42:58'], 
['4', '22', '4','2somename','2008-10-24 15:22:03'], 
['5', '21', '3', '19', '2008-10-24 15:45:45'], 
['6', '21', '1', '1somename', '2008-10-24 15:45:49'], 
['7', '35', '3','2somename', '2008-10-24 15:45:51']]
col = int(input("Enter the column to search(1-5):"))
val = str(input("Enter the element to group by:"))
val1=[]
print('Searching...')
for x in l:
    cmp=x[col-1]
    if cmp==val:
        val1=x
        print(val1)
emp=[]
if val1 == emp:
    print('No search result. Please Try Again!!')

The output would look like this:

Enter the column to search(1-5):4
Enter the element to group by:2somename
Searching...
['4', '22', '4', '2somename', '2008-10-24 15:22:03']
['7', '35', '3', '2somename', '2008-10-24 15:45:51']

Collectives™ on Stack Overflow

Sorting and Grouping Nested Lists in Python

9 Answers 9

5 Comments

3 Comments

2 Comments

1 Comment

6 Comments

Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

9 Answers 9

5 Comments

3 Comments

2 Comments

1 Comment

6 Comments

Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related