Parse/split URLs in a pandas dataframe using urllib

Question

I'm trying to split URLs and put the fragments in a dataframe. I found this thread pythonic way to parse/split URLs in a pandas dataframe and try to apply it, but for some reason it gives me an error.

I am under Python 3.x so I used the following:

import pandas
import urllib

urls = ['https://www.google.com/something','https://mail.google.com/anohtersomething', 'https://www.amazon.com/yetanotherthing']
df['protocol'],df['domain'],df['path'],df['query'],df['fragment'] = zip(*df['url'].map(urllib.parse.urlsplit))

I get an error saying KeyError: 'urls', not sure what it means.

If someone could help would be great. Thanks.

Scratch'N'Purr · Accepted Answer · 2018-02-22 13:33:30Z

1

The example you used assumes that the links are in a dataframe. Here's the correct solution:

import urllib
import pandas as pd

df = pd.DataFrame()
urls = ['https://www.google.com/something','https://mail.google.com/anohtersomething', 'https://www.amazon.com/yetanotherthing']
df['protocol'],df['domain'],df['path'],df['query'],df['fragment'] = zip(*[urllib.parse.urlsplit(x) for x in urls])

Result

  protocol           domain               path query fragment
0    https   www.google.com         /something
1    https  mail.google.com  /anohtersomething
2    https   www.amazon.com   /yetanotherthing

answered Feb 22, 2018 at 13:33

Scratch'N'Purr

10.5k2 gold badges39 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Parse/split URLs in a pandas dataframe using urllib

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related