Python filtering out a list using elements from another list

Question

I am trying to filter out a list using another list. However, the elements of the list I am using to filter the other list is not identical strings. Please see my example as it will make more sense:

mylist = ['14001IB_L1P0', '14001OB_L1P1', '14002IB_L3P0', '14003OB_L1P1', '14001OB_L2P0']
remove_list = ['14001', '14002']

I want to remove the values from mylist that start with the values from remove_list. I have tried doing this:

filtered_mylist = mylist[:]

for x in remove_list:
    for i in filtered_mylist:
        if x in i:
            print('remove ' +i)
            filtered_mylist.remove(i)
        else:
            print('keep '+i)

However, this is the result:

remove 14001IB_L1P0
keep 14002IB_L3P0
keep 14003OB_L1P1
remove 14001OB_L2P0
keep 14001OB_L1P1
remove 14002IB_L3P0

and this is what filtered_mylist consists of:

['14001OB_L1P1', '14003OB_L1P1']

However, it should consist of only 1 element: ['14003OB_L1P1']

It seems to me that for some reason, the loop has skipped over '14001OB_L1P1', the second element in the first loop. Why has this happened?

Don't remove the items from the list on which your are currently iterating, this will cause some items to be skipped. — Shubham Sharma
– Shubham Sharma, Commented May 19, 2020 at 7:31

Abhishek J · Accepted Answer · 2020-05-19 08:03:24Z

1

Here's a one liner

mylist = list(filter(lambda x: all([x.find(y) != 0 for y in remove_list]), mylist))

#Output
['14003OB_L1P1']

The all([x.find(y) != 0 for y in remove_list]) will return True if and only if x does not start with a single value from remove_list.

all() means all have to be True. x.find(y) != 0 means x does not begin with y.

The rest is just executing the filter.

edited May 19, 2020 at 8:03

answered May 19, 2020 at 7:41

Abhishek J

2,6143 gold badges24 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

meronpan Over a year ago

Thanks! This looks to have worked. And I like that it's contained within 1 line. It's too complicated for me to understand though. Would you mind explaining how this works?

Ala Tarighati · Accepted Answer · 2020-05-19 07:35:32Z

1

Would this help?

remove_final = []
keep_final = []
for element in mylist:
    if any([element.startswith(x) for x in remove_list]):
        print(f'remove {element}')
        remove_final.append(element)
    else:
        print(f'keep {element}')
        keep_final.append(element)

Output:

remove 14001IB_L1P0
remove 14001OB_L1P1
remove 14002IB_L3P0
keep 14003OB_L1P1
remove 14001OB_L2P0

And final lists:

keep_final
['14003OB_L1P1']

remove_final
['14001IB_L1P0', '14001OB_L1P1', '14002IB_L3P0', '14001OB_L2P0']

answered May 19, 2020 at 7:35

Ala Tarighati

3,8475 gold badges20 silver badges39 bronze badges

Comments

Albert Nguyen · Accepted Answer · 2020-05-19 07:36:45Z

1

Hope this code help you.

mylist = ['14001IB_L1P0', '14001OB_L1P1', '14002IB_L3P0', '14003OB_L1P1', '14001OB_L2P0']
remove_list = ['14001', '14002']

filtered_mylist = mylist[:]

for x in remove_list:

    i = 0
    while i < len(filtered_mylist):
        if x in filtered_mylist[i]:
            print('remove ' + filtered_mylist[i])
            filtered_mylist.remove(filtered_mylist[i])
        else:
            print('keep '+ filtered_mylist[i])
            i+=1

answered May 19, 2020 at 7:36

Albert Nguyen

3773 silver badges11 bronze badges

Comments

Fu Hanxi · Accepted Answer · 2020-05-19 07:37:47Z

1

Here's another method - append method.

Try to use "filter function + append" to do this instead of remove. That's much safer.

mylist = ['14001IB_L1P0', '14001OB_L1P1', '14002IB_L3P0', '14003OB_L1P1', '14001OB_L2P0']
remove_list = ['14001', '14002']


def is_valid(item):
    for pattern in remove_list:
        if item.startswith(pattern):
            return False
    return True


res = []
for item in mylist:
    if is_valid(item):
        res.append(item)
print(res)

answered May 19, 2020 at 7:37

Fu Hanxi

3062 silver badges15 bronze badges

Collectives™ on Stack Overflow

Python filtering out a list using elements from another list

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related