Reading text from website using sockets Python

Question

I am trying to go to http://www.py4inf.com/code/romeo.txt, read the contents of romeo.txt and print them back out, am using python 3.6.1.

import socket

mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysock.connect(('www.py4inf.com', 80))
mysock.send('GET http://www.py4inf.com/code/romeo.txt HTTP/1.0\n\n'.encode("utf8"))

while True:
    data = mysock.recv(512)
    if ( len(data) < 1 ) :
        break
    print (data.decode("utf8"))

mysock.close()

instead of the contents of the page it prints out

TTP/1.1 404 Not Found
Server: nginx
Date: Wed, 21 Jun 2017 03:00:15 GMT
Content-Type: text/html
Content-Length: 162
Connection: close
 <html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html

Why is this? Thanks in advance

Mac's answer solves your problem, but I would really recommend you take a look at requests (docs.python-requests.org/en/master). Makes your life so much easier. — f0xdx
– f0xdx, Commented Jun 21, 2017 at 3:32

Mac · Accepted Answer · 2017-06-21 03:19:58Z

2

In theory, the Host header is only mandatory from HTTP 1.1 onwards, but it appears that particular server requires the Host header to be present, even for HTTP 1.0. I'm not sure if that's the default behaviour of Nginx, or whether the server admin's explicitly configured it that way.

In any case, try changing your request to the following:

mysock.send('GET http://www.py4inf.com/code/romeo.txt HTTP/1.0\nHost: www.py4inf.com\n\n'.encode("utf8"))

I can understand your confusion - IMHO, it should be returning 400 not 404 if it is insisting on the Host header being provided (since it's a client request issue, not a matter of the resource not existing).

edited Jun 21, 2017 at 3:19

answered Jun 21, 2017 at 3:09

Mac

14.8k11 gold badges65 silver badges83 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Reading text from website using sockets Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related