Regular expression for removing all URLs in a string in Python

Question

I want to delete all the URLs in the sentence.

Here is my code:

import ijson
f = open("/content/drive/My Drive/PTT 爬蟲/content/MakeUp/PTT_MakeUp_content_0_1000.json")
objects = ijson.items(f, 'item')

for obj in list(objects):
    article = obj['content']
    ret = re.findall("http[s*]:[a-zA-Z0-9_.+-/#~]+ ", article) # Question here
    for r in ret:
        article = article.replace(r, "")
    print(article)

But a URL with "http" is still left in the sentence.

article_example = "眼影盤長這樣 http://i.imgur.com/uxvRo3h.jpg 說真的 很不好拍"

How can I fix it?

The regex should be "https?:[a-zA-Z0-9_.+-/#~]+ " if you want to match both http and https prefixes. — Will Richardson
– Will Richardson, Commented Aug 15, 2019 at 6:27
A canonical question is How can I split a URL string up into separate parts in Python? (2009). — Peter Mortensen
– Peter Mortensen, Commented Nov 28, 2022 at 2:35

Peter Mortensen · Accepted Answer · 2022-11-29 10:36:58Z

5

One simple fix would be to just replace the pattern https?://\S+ with an empty string:

article_example = "眼影盤長這樣 http://i.imgur.com/uxvRo3h.jpg 說真的 很不好拍"
output = re.sub(r'https?://\S+', '', article_example)
print(output)

This prints:

眼影盤長這樣  說真的 很不好拍

My pattern assumes that whatever non whitespace characters which follow http:// or https:// are part of the URL.

edited Nov 29, 2022 at 10:36

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Aug 15, 2019 at 6:28

Tim Biegeleisen

526k32 gold badges323 silver badges399 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Peter Mortensen · Accepted Answer · 2022-11-29 10:34:15Z

2

The URL starts with http and in your pattern you match [s*] which will match either a s or * in the character class.

I think you are looking for

https?:[a-zA-Z0-9_.+-/#~]+

Regex demo | Python demo

import re
regex = r"https?:[a-zA-Z0-9_.+-/#~]+ "
article = "眼影盤長這樣 http://i.imgur.com/uxvRo3h.jpg 說真的 很不好拍"
result = re.sub(regex, "", article)
print(result)

Result

眼影盤長這樣 說真的 很不好拍

A shortened expression, which is a bit broader match, could also be matching a non whitespace \S+ char one or more times, followed by a space zero or more times to match the trailing space as in your original pattern.

\bhttps?:\S+ *

Regex demo

edited Nov 29, 2022 at 10:34

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Aug 15, 2019 at 6:27

The fourth bird

165k16 gold badges61 silver badges75 bronze badges

Comments

gilch · Accepted Answer · 2019-08-15 06:27:52Z

1

Change the [s*] to s?. The former is a set of two characters. The latter is an optional character. There are websites like regex101.com that let you experiment with regular expressions in the Python dialect. It will explain the interpretation of each part of the regex.

answered Aug 15, 2019 at 6:27

gilch

12k3 gold badges25 silver badges28 bronze badges

Collectives™ on Stack Overflow

Regular expression for removing all URLs in a string in Python

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related