python socket http library post request not working when data is passed

Question

So as my title said, Im making a small http library with basic get and post requests. I have the get request done, but my post request is acting weird. Anytime I pass data to the post, it doesn't get a response. It seems to work just fine when I don't pass the body attribute (last one). Im not trying to use the requests library, i'm trying to make one myself.

Can anyone see what I'm doing wrong? I feel like I am missing a header but I don't know which.

import socket
from urllib.parse import urlparse
import json

TCRLF = "\r\n\r\n"
CRLF = "\r\n"



def post(url, headers=None, data=""):

    connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    request = build_request("POST", url, headers, data)
    print("request: " + request)
    print("url:" + url)
    print("headers :" + str(headers))

    url = urlparse(url)

    connection.connect((url.hostname, url.port or 80))
    connection.sendall(request.encode("UTF-8"))
    response = connection.recv(4096).decode("UTF-8")
    print("response: " +  response)
    #
    # (response_header, response_body) = response.split(TCRLF)
    # response_body = json.loads(response_body)
    #
    # print(response_header)
    # print(response_body)

    return None

def build_request(req_type, url, headers=None, body=""):

    url = urlparse(url)

    hostname = url.hostname
    path = url.path or "/"
    query = url.query
    port = url.port or 80

    print("hostname" + hostname)
    print("path : " + path)
    print("query :" + query)
    print("port :" + str(port))
    print("body:" + str(body))

    uri = "{}?{}".format(path, query) if query else path

    formatted_headers = "".join(
        "{}:{}\r\n".format(k, v) for k, v in headers.items())


    print("formatted: "+ formatted_headers)

    requestGet = "GET " + uri + " HTTP/1.0" + CRLF + "Host:" + hostname + CRLF + formatted_headers + TCRLF

    ##TODO body failing when passing object
    requestPost = "POST " + uri + " HTTP/1.0" + CRLF + "Host: " + hostname + CRLF + "Content-Length: " + str(
        len(body)) + CRLF + formatted_headers + TCRLF + body + TCRLF

    print("Request" + requestPost)

    if req_type=="POST":
        return requestPost
    else:
        return requestGet


def main():
    post("http://httpbin.org/post", {"Content-Type": "application/json" }, "{'Assignment': 1}")

if __name__ == '__main__':
    main()

Steffen Ullrich · Accepted Answer · 2018-10-01 06:00:39Z

I have no idea where you've got your knowledge of HTTP from. But neither your GET nor your POST request are proper requests according to the HTTP standard. For GET you are adding an additional \r\n at the end (i.e. GET / HTTP/1.0\r\nHost: example.com\r\n\r\n\r\n - the last \r\n being too much) and for POST you do the same and also add some invalid data (TCRLF) after the request body.

This additional \r\n you add after the request header is counted against the Content-length you gave which means that the HTTP body as read by the server is two bytes shorter than the body you've intended. Together with the TRCLF you send after the body the actual HTTP body you send is 6 bytes longer than the value you give in the Content-Length header.

Please refer to the HTTP standard for how HTTP should look like - that's what standards are for. Do not try to get a proper understanding of how HTTP works by just looking at traffic or code samples you'll find on the internet. Specifically simple code samples are often wrong or incomplete and might work with some servers or in some use cases but not in general.

Anytime I pass data to the post, it doesn't get a response.

I cannot reproduce the claim that no response at all is returned. When taking your unmodified code I actually get a response from the server which includes among others the servers view of your response body:

...    
"data": "\r\n{'Assignment': ", 
...

This is your intended body prefixed with your wrong \r\n and shortened by two bytes to match the claimed Content-length. When modifying your code to do proper requests without the wrong \r\n on various places I get the expected:

...
"data": "{'Assignment': 1}", 
...

Yep, right you are. Removed the extra \r\n 's for the formatting . As well, it seems like the network I was at was causing some issues. The same code works now.

fcdt · Accepted Answer · 2020-07-16 10:31:14Z

1

body = '{"name":"mandar"}'
length = len(body)
#request = "GET / HTTP/1.1\r\nHost:%s\r\nUser-Agent: Mandar\r\n\r\n" % serverHost
p = HttpParser()
request = "POST / HTTP/1.1\r\nHost:{0}\r\nUser-Agent:Mandar\r\nContent-Type: application/json\r\nContent-Length: {1}\r\n\r\n{2}\r\n".format(serverHost, length, body)

edited Jul 16, 2020 at 10:31

fcdt

2,5215 gold badges17 silver badges27 bronze badges

answered Jul 16, 2020 at 9:42

user13941141

111 bronze badge

Collectives™ on Stack Overflow

python socket http library post request not working when data is passed

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related