How to fix complicated HTML encoding for URL in python script?

Question

I have a nightmare situation on my hands (or maybe it is easy, I don't know)...So I have a small function that runs in a rather large python script...I have everything worked out in the larger script, and at the end the script will call our web map services and show the parcels in question...We have 20K parcels, and ONLY 10 of them have a '%' in the Deedholder name. So this works over 99% of the time, but there is always that 1% (or way less in this case)

The problem is, in the rare situation where there is a percent sign in the deedholder name, when I supply a url it cannot find the query. So I have tested a ton of names, and it only will not work when there is a percent sign in the name.

So the prefix will look like this:

'https://cedar.integritygis.com/default.aspx?ql=Parcel&qf=REALDATA_DEEDHOLDER&qv='

and the name is added to the end, which looks like this:

'COOPER MICHAEL A & DEBRA K'

My code can easily replace the spaces with '%20' and the & with '%26'...etc. But what do I do when THIS is the deedholder name:

'SIEBELS LAWRENCE J (75%) & LOUISE F TRUST (25%)'

I cannot successfully get this query to work. Here is my test code with just the function in question:

import webbrowser, time

def FixURL(string):

##    string = string.replace('%','~')
    print string
    fix_dict = {' ':'%20','!':'%21','"':'%22','#':'%23','$':'%24',
                '&':'%26',"'":'%27','(':'%28',')':'%29',
                '*':'%2A','+':'%2b','.':'%2E','/':'%2F',':':'%3A',
                ';':'%3B','?':'%3F','@':'%40','{':'%7B','{':'%7D'}

    for k,v in fix_dict.iteritems():
        if k in string:
            string = string.replace(k,v)
##    return string.replace('~','%25')
    return string

if __name__ == '__main__':

    # testing
    easy = FixURL('COOPER MICHAEL A & DEBRA K')
    prefix = 'https://cedar.integritygis.com/default.aspx?ql=Parcel&qf=REALDATA_DEEDHOLDER&qv='
    url = '{}{}'.format(prefix,easy)
    print easy
    webbrowser.open(url)
    time.sleep(15)  # give it time to work

    hard = FixURL('SIEBELS LAWRENCE J (75%) & LOUISE F TRUST (25%)')
    print hard
    url = '{}{}'.format(prefix,hard)
    webbrowser.open(url)

I cannot figure out how to 'trick' it...You can see my unsucessful attempts are commented out. Does anyone have a fix? One thing I am thinking of doing is removing the space from the dictionary and using '%20'.join(string.split()) and testing each item in the list for replacement values for the url...Does have any ideas? It seems I have been squeezed by Python yet again. Thanks.

EDIT:

I have since scratched the entire function and am just urllib.quote(). this as a test:

import webbrowser, urllib, time

prefix = 'https://cedar.integritygis.com/default.aspx?ql=Parcel&qf=REALDATA_DEEDHOLDER&qv='
easy = urllib.quote('COOPER MICHAEL A & DEBRA K')
url = '{}{}'.format(prefix,easy)
print easy
webbrowser.open(url)
time.sleep(15)  # give it time to work

hard = urllib.quote('SIEBELS LAWRENCE J (75%) & LOUISE F TRUST (25%)')
print hard
url = '{}{}'.format(prefix,hard)
webbrowser.open(url)

This is suppposed to zoom to the parcels owned by the name supplied...The first one works, the second does not because of the % in the parenthesis (I think). I get the 'ol query returned no results error.

arghbleargh · Accepted Answer · 2013-08-14 18:39:47Z

2

You can use python's standard urllib to do this.

http://docs.python.org/2/library/urllib.html#utility-functions

Look at the utility functions. urllib.quote will probably do the job.

answered Aug 14, 2013 at 18:39

arghbleargh

3,17824 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

cmackey Over a year ago

Wow, I do not know why I did not look to use that in the first place...I guess I can scratch the entire function...However, it did not work for the problem one:

cmackey Over a year ago

This did not work: hard = urllib.quote('SIEBELS LAWRENCE J (75%) & LOUISE F TRUST (25%)')

cmackey Over a year ago

Thanks again...but no luck. It is basically giving me the same result my function did...I have now replaced the FixURL() with urllib.quote() and still no dice. Still having problems encoding that '%' sign.

arghbleargh Over a year ago

Do you know for sure that you have the right deedholder name? You could also try escaping the % first (i.e. replace '75%' with '75\%' in the original string).

arghbleargh Over a year ago

If you can query from the website manually, you should use your browsers network inspector to check exactly what URL the successful request used.

|

Collectives™ on Stack Overflow

How to fix complicated HTML encoding for URL in python script?

1 Answer 1

10 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Related