Python requests module hangs on socket connection but cURL works

Question

I'm trying to make GET/POST requests between docker containers.

In one of them (the others work fine), when doing the following:

ADDR = 'http://172.20.0.2:5002/see'
r = requests.post(ADDR, data=data, headers=headers)

the command hangs forever (or until timeout, if one is defined) on trying to connect. It never even establishes the connection:

^CTraceback (most recent call last):
  File "/workspace/master_server.py", line 35, in <module>
    r = requests.post(ADDR, data=data, headers=headers)
  File "/usr/local/lib/python3.5/dist-packages/requests/api.py", line 112, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.5/http/client.py", line 1106, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1102, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)

If I run the exact same request using curl from bash, it works effortlesly:

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '"[1215 2589 1392 3019]"' 'http://172.20.0.2:5002/see'

Any other containers that I try to use in the exact same way work with either the requests module or curl. Any help on how to fix it or at least debug the problem effectively?

I have the exact same problem right now. It worked on windows but ubuntu hangs now (using python3, robobrowser) — juleslasne
– juleslasne, Commented Nov 6, 2018 at 12:37
same problem, works locally on windows and hangs on ubuntu ec2 server — jugovich
– jugovich, Commented Jul 26, 2019 at 20:49

Ben Elliott · Accepted Answer · 2020-06-25 22:09:17Z

8

I recently ran into the same issue locally, and like some comments on the original question, my code worked in windows, but not linux. After a few hours found the culprit for my case: IPv6.

The python module I was using attempts to make a socket connection with a server and was by default using IPv6. After turning IPv6 off in my linux environment (Ubuntu 18.04) with the following steps:

Open terminal
Enter: sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
Enter: sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1

The module worked as expected**.

**note that these steps will likely vary per environment

answered Jun 25, 2020 at 22:09

Ben Elliott

1692 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

M. Yuri · Accepted Answer · 2020-03-17 11:19:11Z

2

I faced the same problem and lost a lot of time ... In my case, the problem was as follows (guniconf.py):

workers = 1

change to:

workers = 10  # any number is greater than 1

More: There is a Flask application that serves some REST API. The application has a REST API method (let's call it A-method), which makes inside itself a call to another REST API method (let's call it B-method) using requests.post(). When the B-method is called from A-method, app hangs. As I understand it, when calling the A-method, the gunicorn worker processes it, but with the B-method, there are no more workers.

answered Mar 17, 2020 at 11:19

M. Yuri

213 bronze badges

Comments

gawbul · Accepted Answer · 2020-10-08 10:00:11Z

1

I had the same issue on macOS (Catalina) recently and it turned out to be due to IPv6.

I had to go to System Preferences -> Network -> Wi-Fi -> Advanced -> TCP/IP and change Configure IPv6 to Link-local only (there wasn't an "Off" option in the list, though this would also be applicable). Then click OK and Apply.

You can also disable altogether using:

sudo networksetup -setv6off Wi-Fi

answered Oct 8, 2020 at 10:00

gawbul

1,21713 silver badges20 bronze badges

1 Comment

Scott Boston Over a year ago

Yet another Stack Overflow save. Thank you for your input here. You just solved my problem on MacOS. +1

Parham · Accepted Answer · 2023-03-03 18:15:58Z

0

I encountered this error when running my Debian container with docker compose, on a MacOS host. Based on @Ben Elliot's answer, I disabled IPv6 by adding this to the containers config in my docker-compose.yaml:

sysctls:
- net.ipv6.conf.all.disable_ipv6=1

Example:

services:
  srv_a:
    build: ...
    sysctls:
    - net.ipv6.conf.all.disable_ipv6=1
    ...

answered Mar 3, 2023 at 18:15

Parham

3,5326 gold badges34 silver badges46 bronze badges

Collectives™ on Stack Overflow

Python requests module hangs on socket connection but cURL works

4 Answers 4

Comments

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related