10

I know about urllib and urlparse, but I want to make sure I wouldn't be reinventing the wheel.

My problem is that I am going to be fetching a bunch of urls from the same domain via the urllib library. I basically want to be able to generate urls to use (as strings) with different paths and query params. I was hoping that something might have a syntax like:

url_builder = UrlBuilder("some.domain.com")
# should give me "http://some.domain.com/blah?foo=bar
url_i_need_to_hit = url_builder.withPath("blah").withParams("foo=bar")  # maybe a ".build()" after this

Basically I want to be able to store defaults that get passed to urlparse.urlunsplit instead of constantly clouding up the code by passing in the whole tuple every time.

Does something like this exist? Do people agree it's worth throwing together?

Improve this question

4 Answers 4

Reset to default
4

Are you proposing an extension to http://docs.python.org/library/urlparse.html#urlparse.urlunparse that would substitute into the 6-item tuple?

Are you talking about something like this?

def myUnparse( someTuple, scheme=None, netloc=None, path=None, etc. ):
    parts = list( someTuple )
    if scheme is not None: parts[0] = scheme
    if netloc is not None: parts[1]= netloc
    if path is not None: parts[2]= path
    etc.
    return urlunparse( parts )

Is that what you're proposing?

This?

class URLBuilder( object ):
    def __init__( self, base ):
        self.parts = list( urlparse(base) )
    def __call__( self, scheme=None, netloc=None, path=None, etc. ):
        if scheme is not None: self.parts[0] = scheme
        if netloc is not None: self.parts[1]= netloc
        if path is not None: self.parts[2]= path
        etc.
        return urlunparse( self.parts )

bldr= URLBuilder( someURL )
print bldr( scheme="ftp" )

Something like that?

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

mehhh... maybe. But I was thinking of something more along the lines of a class that stored the default the values. I don't want to keep passing in "someTuple". I just want to pass in some default values to some class, then reuse an instance to generate all different urls. Something similar to the Builder pattern that is ubiquitous in Java.
@Tom: It doesn't seem complex enough to wring your hands over. Indeed, if you put more more than 25 lines of code into it, you've probably done too much.
4

You might want consider having a look at furl because it might be an answer to your needs.

Improve this answer

1 Comment

Original answer was a question more for a comment than an answer.
3

Still not quite sure what you're looking for... But I'll give it a shot. If you're just looking to make a class that will keep your default values and such, it's simple enough to make your own class and use Python magic like str. Here's a scratched-out example (suboptimal):

class UrlBuilder:
    def __init__(self,domain,path="blah",params="foo=bar"):
        self.domain = domain
        self.path = path
        self.params = params

    def withPath(self,path):
        self.path = path
        return self

    def withParams(self,params):
        self.params = params
        return self

    def __str__(self):
        return 'http://' + self.domain + '/' + self.path + '?' + self.params
        # or return urlparse.urlunparse( ( "http", self.domain, self.path, self.params, "", "" )

    def build(self):
        return self.__str__()

if __name__ == '__main__':
    u = UrlBuilder('www.example.com')
    print u.withPath('bobloblaw')
    print u.withParams('lawyer=yes')
    print u.withPath('elvis').withParams('theking=true')

If you're looking for more of the Builder Design Pattern, the Wikipedia article has a reasonable Python example (as well as Java).

Improve this answer

4 Comments

I prefer to use something like urlparse.urlunparse rather than hard-code punctuation.
Accepting this answer pretty late... In the end I really didn't need something like this... but this is what I was looking for. It's obvious there was nothing built in to Python like this, but that was all I was wondering. (I knew I could easily code it myself). My only comment is like S.Lott's... I would have used urlparse in the implementation of the class... thanks ewall.
In order to avoid hard-coding the URL escaping of the parameters, which is what I came here for, you can use urllib.urlencode, for example: params = urllib.urlencode({'user': self.request.user.pk, 'token': m.hexdigest()})
Note that the example above will not handle scheme (https/http), the #hash that may be at the end, and will include a trailing slash when there is no path, and a trailing ? when there are no params. It does seem like python would have something to cleanly compose this without everyone writing their own.
1

I think you want http://pythonhosted.org/uritools/.

Example from the docs:

parts = urisplit('foo://[email protected]:8042/over/there?name=ferret#nose')

orig_uri = uriunsplit(parts)

The split value is a named tuple, not a regular list. It is accessible by name or index:

assert(parts[0] == parts.schema)
assert(parts[1] == parts.authority)
assert(parts[2] == parts.path)
assert(parts[3] == parts.query)
assert(parts[4] == parts.fragment)

Make a copy to make changes:

new_parts = [part for part in parts]
new_parts[2] = "/some/other/path"
new_uri = uriunsplit(new_parts)
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.