What is the most pythonic way of making a list unique using custom equality operator?
For instance you have a list of dicts L, and you want a new list M such that for all dicts d, e in M and one specific x
d[x] != e[x]
How can this be done?
In your case (and all cases where equivalence boils down to the equivalence of some kind of key), you can simply construct a dictionary, where the keys are the values you want to compare:
L = [{'key': 'foo', 'v': 42}, {'key': 'bar', 'v': 43}, {'key': 'foo', 'v': 44}]
x = 'key'
M = {d[x]:d for d in L}.values()
# In old Python versions: dict((d[x],d for d in L)).values()
Note that the result is not deterministic, both
[{'key': 'foo', 'v': 44}, {'key': 'bar', 'v': 43}]
and
[{'key': 'foo', 'v': 42}, {'key': 'bar', 'v': 43}]
are valid results.
In the general case, simply check all accepted values:
def unique(iterable, is_eq):
tmp = []
for el in iterable:
if not any(is_eq(inTmp, el) for inTmp in tmp):
tmp.append(is_eq)
return tmp
Note that this means that your comparison function will be called O(n²) times instead of n times.
Based on FUD's comment to phihag. Note that key function must return a hashable value.
def unique(iterable, key=lambda x : x):
seen = set()
res = []
for item in iterable:
k = key(item)
if k not in seen:
res.append(item)
seen.add(k)
return res
from operator import itemgetter
L = [{'key': 'foo', 'v': 42}, {'key': 'bar', 'v': 43}, {'key': 'foo', 'v': 44}]
print unique(L, key=itemgetter('key'))
#[{'key': 'foo', 'v': 42}, {'key': 'bar', 'v': 43}]
I'm not sure this sort of thing admits a one-liner, but it seems to me that the set class is the key to what you want.
M = []
uniques = set(d[x] for d in L)
for d in L:
if d[x] in uniques:
uniques.remove(d[x])
M.append(d)
Note: phihag's answer seems more Pythonic, but this might be a bit more self-documenting
Using dictionary comprehension:
def unique(itrable,key):
return {key(x):x for x in itrable}.values()
>>> unique('abcdbbcdab', lambda x: x)
['a', 'c', 'b', 'd']
>>> unique([10, -20, 20, 30], lambda x: abs(x))
[10, 20, 30]
