Invalid URL parser in python

Question

[Yes, the title is not a typo!]

In python I need something that parses a URL. I cannot believe something standard does not already exist. As the URL is set in a config I'd like to make sure it is not garbage.

There's urlparse.urlparse, but that only parses 'valid URLs' (with some invalid URLs sometimes raising an undocumented ValueError)

e.g.

>>> import urlparse
>>> urlparse.urlparse('http://aa :: aa ! aa:11.com:aa').netloc
'aa :: aa ! aa:11.com:aa'

shows how urlparse parses what I would consider an invalid URL.

Can't you just use urlparse and catch the errors?

codebox
– codebox

2012-08-15 10:31:22 +00:00
Commented Aug 15, 2012 at 10:31 — codebox
– codebox, Commented Aug 15, 2012 at 10:31
Whats about a regex? I guees youll find thousands for urls.

Sebastian Hoffmann
– Sebastian Hoffmann

2012-08-15 10:32:27 +00:00
Commented Aug 15, 2012 at 10:32 — Sebastian Hoffmann
– Sebastian Hoffmann, Commented Aug 15, 2012 at 10:32

Rostyslav Dzinko · Accepted Answer · 2012-08-15 10:32:58Z

2

URL parsing and URL validation are actually different tasks.

urlparse.urlparse makes parsing, validation is usually made using regular expressions machine (built-in re module in Python).

Here's and example of URL validation from Django framework:

regex = re.compile(
    r'^(?:http|ftp)s?://' # http:// or https://
    r'(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}\.?|[A-Z0-9-]{2,}\.?)|' #domain...
    r'localhost|' #localhost...
    r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})' # ...or ip
    r'(?::\d+)?' # optional port
    r'(?:/?|[/?]\S+)$', re.IGNORECASE)

answered Aug 15, 2012 at 10:32

Rostyslav Dzinko

41k5 gold badges55 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

typingduck Over a year ago

I had assumed there was something standard, preferrable to just cut-n-pasting code. But, django.core.validators.URLValidator is exactly what I need, thanks :)

Frederic Bazin Over a year ago

It is almost impossible to write an extensive url validator with regex. so many cases suc as tel: or mailto: are missing. Also IPv6 and many more...

Collectives™ on Stack Overflow

Invalid URL parser in python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related