Remove duplicate data from an array in usin python

Question

I have an array with data and I notice that I have each data twice. Is there any method to remove the duplicate data to simplify the array content? Below is the code that I made in python:

import requests
import re
import bs4

r = requests.get("http://as.com/tag/moto_gp/a/")

r.raise_for_status()

html = r.text


matches = re.findall(r"http://motor\.as\.com/motor/\d+/\d+/\d+/motociclismo/\d+_\d+.html", html)

print (matches)

Rahul K P · Accepted Answer · 2016-05-22 10:45:24Z

7

I hope your matches is a list.Then you can use simple method.

In [1]: a = [1,1,2,2,3,3,4,4,5]
In [2]: list(set(a))
Out[2]: [1, 2, 3, 4, 5]

For your code only one modification.

matches = list(set(re.findall(r"http://motor\.as\.com/motor/\d+/\d+/\d+/motociclismo/\d+_\d+.html", html)))

edited May 22, 2016 at 10:45

answered May 22, 2016 at 10:37

Rahul K P

16.2k4 gold badges40 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Rahul K P Over a year ago

@SergeiLebedev You are right. It will work with all iterable.

Collectives™ on Stack Overflow

Remove duplicate data from an array in usin python

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related