0

I am doing a little bit of python multithreading programming and found the result of my code vary strange, not parallel at all (8 cores, 8 threads, 13% cpu utilization). Then I have found a python GIL term and these slides (http://www.dabeaz.com/python/GIL.pdf). Is it real that python is not parallelizable? Will the multiprocessing module help to utilize computational resources or there is another performance issue with that?

Improve this question
6
  • Which multithreading library did you use? Commented Dec 24, 2014 at 10:26
  • stackoverflow.com/questions/6821477/… Commented Dec 24, 2014 at 10:28
  • @Nebril I use standard threading module can you recommend something else. Because spawning new processes does not really fit me. Commented Dec 24, 2014 at 10:30
  • @itun as Reut wrote: try multiprocessing docs.python.org/2/library/multiprocessing.html Commented Dec 24, 2014 at 10:33
  • 1
    Praise the docs then! Commented Dec 24, 2014 at 10:36

1 Answer 1

Reset to default
2

From the docs:

CPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). If you want your application to make better use of the computational resources of multi-core machines, you are advised to use multiprocessing. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously.

A a general rule of thumb:

When your bottleneck is I/O (like writing to disk...) - consider threading which allows the program to keep running elsewhere when something is blocking an execution path.

When your bottleneck is cpu power, consider multiprocessing which allows cpu utilization.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

what should I do if I cannot spawn new processes? Is there any 3rd party library to use threads in a usual sense.
You should drop by the python room for advices on what to do, instead of starting an opinion-based discussion here. The chat room: chat.stackoverflow.com/rooms/6/python

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.