7

I'm building a Python API around a black box .NET DLL using Python .NET. The DLL is only doing networking operations. The DLL require me to run a windows message pumping loop, otherwise the network operations get stuck after a while. I run the windows message loop using System.Windows.Forms.Application.Run() in the main thread. This works fine for only receiving data. My program starts to behave weirdly though when I start to do calls from an other Python thread to the DLL. I think it is related to threading since the problems are accuring very irregular - networking events disappear or come in very late. As far as I know Python and C# are thread safe, but maybe because of the multiple layers of wrapping something goes wrong.

So I have a couple of questions:

  • Is it a bad idea to do calls to the DLL's from multiple Python theads? Or does this depend on the DLL internals? I thought from the DLL's perspective the Python threads are seen as one single agents due to the GIL.

  • Does every Python thread using this DLL need a message pump?

  • It is probably a good idea to keep all DLL interactions in one thread. I have difficulties accomplishing this, since I give control to the message pump loop in the main thread. My naive approach would be to put new outgoing network messages generated in my worker thread on a Python queue, create a custom message pump loop in the main thread that does the windows event handling but also monitors my queue and if there is message would do a call to the DLL. But this all feels quite clunky and a lot of work for such a simple task. Is this the right approach?

  • Is there an other way to put a function in the main windows event loop that monitors the previously described queue and takes action? Or should I dive into the .NET specifics and start using .NET events or dispatchers?

2 Answers 2

10

I have found answers to most of my own questions.

  • I think we can't assume the DLL is thread safe, so probably best to isolate all interaction with the DLL to one thread.

  • It looks like the Windows message system is per thread and not per process, so yes, every thread using the Windows message system is required to have a Windows message processing loop.

  • One can insert execution in the a windows event loop using Form.Invoke. running a non-ui main loop, you can get a dispatcher using Dispatcher.CurrentDispatcher which can used for 'invoking' a function. The invoked function is then executed on the thread where the dispatcher is created. The called function needs to be delegated though, which is a C# specific thing to pass references to functions.

in the end I did something like this for a non-ui main loop:

import clr
import threading

# we need to get access to the threading assembly
clr.AddReference("c:\\Program Files\\Reference Assemblies\\Microsoft\\Framework\\v3.0\\WindowsBase.dll")

import System
from System.Windows.Threading import Dispatcher
from System.Windows.Forms import Application

# now get the dispatcher for the current thread
dispatcher = Dispatcher.CurrentDispatcher

def display():
    print(threading.current_thread())

# now make this a deligate
deligate = System.Action(display)

def other_thread(dispatcher):
    # now you can use the dispatcher from the main thread to
    # to schedule a run from an other thread
    dispatcher.Invoke(deligate)

thread = threading.Thread(target=other_thread, args=(dispatcher,))
thread.start()
Application.Run()

some related links:

Sign up to request clarification or add additional context in comments.

1 Comment

I fixed the problems I had when using python threads in winforms applications with pythonnet thanks to this answer. Many thanks!
0

I have had a similar problem: using a pythonnet library DLL (import clr, et.) from multiple Python threads at the same time.

I had errors when multiple threads were communicating with the same functions (managing a hardware device) in parallel.

Solution:

  1. create a global lock:

    lock = threading.Lock()
    
  2. replace each API call e.g. device.move(x=1234) by:

    with lock:
        device.move(x=1234)
    

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.