python dictionary key error cannot resolve

Question

I am performing status check across a huge list of links and my snippet is as follows:

link = 'http://xyz'
proxyDict = { "http" : "ip:80", "https" : "https://ip:443"}
r = requests.get(link, allow_redirects=False, verify=False)
http_status = r.status_code
print (r.headers)

# check the status and react accordingly

if http_status == 200 and r.headers['content-length'] == "0":
   print ('Link Alive - NO content'+';'+str(http_status)+';'+link, file = log)
elif http_status == 200 and "text/html" in r.headers['content-type']:
   print ('External- direct HTML link'+';'+str(http_status)+';'+link, file = log)  
elif http_status == 200 and "application" in r.headers['content-type']:
   print ('External- direct HTML link'+';'+str(http_status)+';'+link, file = log)

When I execute the code I get the following error:

    return self._store[key.lower()][1]
    KeyError: 'content-length'

The header output is as follows:

CaseInsensitiveDict({'status': '200', path=/; HttpOnly, shpuvid=rBBcnFJUTliSHV+hA5lLAg==; expires=Thu, 08-Oct-15 18:26:32 GMT;'connection': 'keep-alive', 'cache-control': 'max-age=0, private, must-revalidate', 'date': 'Tue, 08 Oct 2013 18:26:32 GMT', 'content-type': 'text/html; charset=utf-8', 'x-rack-cache': 'miss'})

I know that the error exists because the header output has no key "content-length", but when if condition does not satisfy it has to jump to next elif condition which does not happen, rather stops the code execution throwing the above error.

Any suggestions? Might be a silly question but a good thing for a beginner to learn.

bcollins · Accepted Answer · 2013-10-08 18:39:12Z

10

Instead of using the bracket notation, use r.headers.get('content-length') from the dictionary which will not throw the key error but simply return None.

Its nice that you can use either notation to retrieve values from a dictionary. Many times you want that key error to be thrown as not to let a problem go unnoticed. In this case, it appears that dictionary.get() is what you want.

answered Oct 8, 2013 at 18:39

bcollins

3,4794 gold badges23 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Kobi K · Accepted Answer · 2013-10-08 18:40:09Z

3

Key error generally means the key doesn't exist.

I'm gessing that self._store[key.lower()][1] is not valid (doesn't exist)

From official python docs:

exception KeyError

Raised when a mapping (dictionary) key is not found in the set of existing keys.

answered Oct 8, 2013 at 18:40

Kobi K

7,9376 gold badges47 silver badges88 bronze badges

Comments

Andrew Milne · Accepted Answer · 2019-07-27 19:07:50Z

0

The problem may be inside requests itself.

There is some asynchronous behavior in the requests package that causes problems when multiple calls to requests.get() are directed to the same server.

https://blog.petrzemek.net/2018/04/22/on-incomplete-http-reads-and-the-requests-library-in-python/

answered Jul 27, 2019 at 19:07

Andrew Milne

1

Collectives™ on Stack Overflow

python dictionary key error cannot resolve

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related