Python server program has high cpu usage

Question

I am experimenting with a python tcp server (I've just recently discovered the awesomeness that is python). Anyway, the server runs just fine and prints data the way I would expect, but when I look at the CPU usage in the windows task manager it shows that python.exe is consuming 97%-99% of the CPU.

Just to see what would happen, I ran it again on a different computer and it only used about 50% of the cpu.

Here is what I am wondering:

Why is the cpu usage so high?
Why would I see a difference running on two different machines (one is Windows 7 the other is server 2008, could that matter)?
I am creating a new thread for each connnection, and running a while loop that is always true, but I have a "break" when there is no longer a connection. Is this thread getting destroyed correctly?

Thanks in advance for any help!

import socket
import threading
import logging
import time

TCP_IP = "127.0.0.1"
TCP_PORT = 10000
BUFFER_SIZE = 1024
SOCKET_TIMEOUT = 2


def handler(conn):
    while 1:
        try:
            data = conn.recv(BUFFER_SIZE)
            if data:
                dataS = data.decode(encoding = 'UTF-8')
                print ("received data: ")
                print (dataS)
                logging.getLogger("TestLogger").critical(dataS)

        except socket.error as e:
            print("connection closed")
            conn.close()
            break

try:    
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind((TCP_IP, TCP_PORT))
    s.setblocking(0)
    s.listen(5)

except Exception as e:
    print(e)


while 1:
    try:
        conn, addr = s.accept()
    except Exception as e:
        continue
    print ('Connection address:')
    print (addr)
    conn.settimeout(SOCKET_TIMEOUT)
    connThread = threading.Thread(target = handler, args = (conn,))
    connThread.start()

You should post this on code review, as it appears to be functional code. — Sterling Archer
– Sterling Archer, Commented May 23, 2014 at 20:25
I wasn't aware of code review. Should I remove this post then? — jteezy14
– jteezy14, Commented May 23, 2014 at 20:27
I am sure that the hardware properties would be different on Win 7 machine and the server. That's why they have different performances! — ρss
– ρss, Commented May 23, 2014 at 20:30
I tested your code on Windows 8 64-bit, core i5 6GB ram, the CPU usage never goes above 32% in my case! — ρss
– ρss, Commented May 23, 2014 at 20:32

pmoleri · Accepted Answer · 2014-05-23 20:38:37Z

6

You are using setblocking(0), that means your socket is not blocking. What you are doing is called polling and will use as much CPU as it can.

When you use threading you don't need to poll. Just remove the setblocking(0) line and it should work properly.

answered May 23, 2014 at 20:38

pmoleri

4,5181 gold badge20 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

jteezy14 Over a year ago

You are right. Thank you so much! The cpu usage is negligible now.

rsy · Accepted Answer · 2017-01-19 19:11:34Z

4

The problem is the infinite while loop (while 1). Please include a few microsecond delay (time.sleep(0.001)), this brings the CPU usage drastically down.

answered Jan 19, 2017 at 19:11

rsy

1922 silver badges10 bronze badges

Comments

Robert Špendl · Accepted Answer · 2018-11-19 17:35:17Z

2

To low reputation to just comment on rsy's answer. In your particular case, when you receive data with "conn.recv", just setting setblocking(1) solves the problem, as pmoleri has answered.

But if you do polling in a loop with "select" statement, just setblocking(1) has no effect and rsy's solution with a short delay (time.delay(0.01)) is the way to go.

answered Nov 19, 2018 at 17:35

Robert Špendl

1721 silver badge4 bronze badges

Collectives™ on Stack Overflow

Python server program has high cpu usage

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related