How do I remove duplicate arrays in a list in Python

Very simple way to remove duplicates (if you're okay with converting to tuples/other hashable item) is to use a set as an intermediate element.

lst = ([4,1,2],[1,2,3],[4,1,2])
# convert to tuples
tupled_lst = set(map(tuple, lst))
lst = map(list, tupled_lst)

If you have to preserve order or don't want to convert to tuple, you can use a set to check if you've seen the item before and then iterate through, i.e.,

seen = set()
def unique_generator(lst)
    for item in lst:
       tupled = tuple(item)
       if tupled not in seen:
           seen.add(tupled)
           yield item
lst = list(unique_generator(lst))

This isn't great python, but you can write this as a crazy list comprehension too :)

seen = set()
lst = [item for item in lst if not(tuple(item) in seen or seen.add(tuple(item)))]

l = ([4,1,2],[1,2,3],[4,1,2])
uniq = []
for i in l:
    if not i in uniq:
        uniq.append(i)
print('l=%s' % str(l))
print('uniq=%s' % str(uniq))

which produces:

l=([4, 1, 2], [1, 2, 3], [4, 1, 2])
uniq=[[4, 1, 2], [1, 2, 3]]

Use sets to keep track of seen items, but as sets can only contain hashable items so you may have to convert the items of your tuple to some hashable value first( tuple in this case) .

Sets provide O(1) lookup, so overall complexity is going to be O(N)

This generator function will preserve the order:

def solve(lis):
    seen = set()
    for x in lis:
        if tuple(x) not in seen:
            yield x
            seen.add(tuple(x))

>>> tuple( solve(([4,1,2],[1,2,3],[4,1,2])) )
([4, 1, 2], [1, 2, 3])

If the order doesn't matter then you can simply use set() here:

>>> lis = ([4,1,2],[1,2,3],[4,1,2]) # this contains mutable/unhashable items 
>>> set( tuple(x) for x in lis)    # apply tuple() to each item, to make them hashable
set([(4, 1, 2), (1, 2, 3)])  # sets don't preserve order

>>> lis = [1, 2, 2, 4, 1]    #list with immutable/hashable items
>>> set(lis)
set([1, 2, 4])