How to use COM from python win32com or comtypes to access an IRTDServer?

Question

Environment

Windows 10 + python 3.6.3 64 bit (also tried 32 bit). I am a python developer trying to use COM for (nearly) the first time and hit this huge blocker.

Problem

I have had various errors when trying to use an IRTDServer implemented in a dll (not written by me), via either win32com or comtypes. Using win32com turned out to be more difficult. I have an included an example unittest for both libraries below.

Accessing the server from Excel 2016 works as expected; this returns the expected value:

=RTD("foo.bar", , "STAT1", "METRIC1")

Code using win32com library

Here is a simple test case which should connect to the server but doesn't. (This is just one version, as I have changed it many times trying to debug the problem.)

from unittest import TestCase

class COMtest(TestCase):
    def test_win32com(self):
        import win32com.client
        from win32com.server.util import wrap

        class RTDclient:
            # are these only required when implementing the server?
            _com_interfaces_ = ["IRTDUpdateEvent"]
            _public_methods_ = ["Disconnect", "UpdateNotify"]
            _public_attrs_ = ["HeartbeatInterval"]

            def __init__(self, *args, **kwargs):
                self._comObj = win32com.client.Dispatch(*args, **kwargs)
            def connect(self):
                self._rtd = win32com.client.CastTo(self._comObj, 'IRtdServer')
                result = self._rtd.ServerStart(wrap(self))
                assert result > 0

            def UpdateNotify(self):
                print("UpdateNotify() callback")
            def Disconnect(self):
                print("Disconnect() called")
            HeartbeatInterval = -1

_rtd = RTDclient("foo.bar")
_rtd.connect()

Result:

Traceback (most recent call last):
  File "env\lib\site-packages\win32com\client\gencache.py", line 532, in EnsureDispatch
    ti = disp._oleobj_.GetTypeInfo()
pywintypes.com_error: (-2147467263, 'Not implemented', None, None)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test\test.py", line 23, in test_win32com
    _rtd.connect()
  File "test\test.py", line 16, in connect
    self._rtd = win32com.client.CastTo(dispatch, 'IRtdServer')
  File "env\lib\site-packages\win32com\client\__init__.py", line 134, in CastTo
    ob = gencache.EnsureDispatch(ob)
  File "env\lib\site-packages\win32com\client\gencache.py", line 543, in EnsureDispatch
    raise TypeError("This COM object can not automate the makepy process - please run makepy manually for this object")
TypeError: This COM object can not automate the makepy process - please run makepy manually for this object

Following those directions, I ran the makepy script successfully:

> env\Scripts\python.exe env\lib\site-packages\win32com\client\makepy.py "foo.bar"
Generating to C:\Users\user1\AppData\Local\Temp\gen_py\3.5\longuuid1x0x1x0.py
Building definitions from type library...
Generating...
Importing module

(I replaced the UUID on stackoverflow for privacy. This UUID is the same as the typelib UUID for "foo.bar".)

The generated file contains various the function and type definitions of both IRtdServer and IRTDUpdateEvent. But in this file, both interfaces are subclasses of win32com.client.DispatchBaseClass, while according to OleViewDotNet, they should be subclasses of IUnknown?

However, when I attempted to run the unittest again, I received the exact same error as before. It is as if the lookup mechanism is not finding the generated module?

Also, GetTypeInfo returning Not implemented is alarming me. From my understanding, win32com uses that method (part of IDispatch COM interface) to determine the argument and return types for all other functions in other interfaces, including IRtdServer. If it's not implemented, it would be unable to determine the types correctly. Yet, the generated file seems to include this information, which is also perplexing.

Code using comtypes library

from unittest import TestCase

class COMtest(TestCase):
    def test_comtypes(self):
        import comtypes.client

        class RTDclient:
            # are these for win32com only?
            _com_interfaces_ = ["IRTDUpdateEvent"]
            _public_methods_ = ["Disconnect", "UpdateNotify"]
            _public_attrs_ = ["HeartbeatInterval"]

            def __init__(self, clsid):
                self._comObj = comtypes.client.CreateObject(clsid)
            def connect(self):
                self._rtd = self._comObj.IRtdServer()
                result = self._rtd.ServerStart(self)
                assert result > 0

            def UpdateNotify(self):
                print("UpdateNotify() callback")
            def Disconnect(self):
                print("Disconnect() called")
            HeartbeatInterval = -1

_rtd = RTDclient("foo.bar")
_rtd.connect()

Result:

  File "test\test.py", line 27, in test_comtypes
    _rtd.connect()
  File "test\test.py", line 16, in connect
    self._rtd = self._comObj.IRTDServer()
  File "env\lib\site-packages\comtypes\client\dynamic.py", line 110, in __getattr__
    dispid = self._comobj.GetIDsOfNames(name)[0]
  File "env\lib\site-packages\comtypes\automation.py", line 708, in GetIDsOfNames
    self.__com_GetIDsOfNames(riid_null, arr, len(names), lcid, ids)
_ctypes.COMError: (-2147352570, 'Unknown name.', (None, None, None, 0, None))

Some other solutions I've tried

(Based on googling and answers in the comments below)

(Re-)Registered the DLL
Registered the 32 bit version of the DLL and tried python 32 bit
Set compatibility mode of python.exe to Windows XP SP3
Tried not instantiating IRtdServer, that is, replacing these two lines:
```
self._rtd = self._comObj.IRtdServer()
result = self._rtd.ServerStart(self)
```
with:
```
result = self._comObj.ServerStart(self)
```
The error this time is:
```
TypeError: 'NoneType' object is not callable
```
That would seem to indicate that the ServerStart function exists, but is undefined? (Seems really weird. There must be more to this mystery.)

Tried passing interface="IRtdServer" parameter to CreateObject:

def __init__(self, clsid):
    self._comObj = comtypes.client.CreateObject(clsid, interface="IRtdServer")
def connect(self):
    result = self._comObj.ServerStart(self)
    ...

The error received is:

  File "test\test.py", line 13, in __init__
    self._comObj = comtypes.client.CreateObject(clsid, interface="IRtdServer")
  File "env\lib\site-packages\comtypes\client\__init__.py", line 238, in CreateObject
    obj = comtypes.CoCreateInstance(clsid, clsctx=clsctx, interface=interface)
  File "env\lib\site-packages\comtypes\__init__.py", line 1223, in CoCreateInstance
    p = POINTER(interface)()
TypeError: Cannot create instance: has no _type_

Tracing code in the comtypes library, that would seem to indicate that the interface parameter wants an interface class, not a string. I found various interfaces defined in the comtypes library: IDispatch, IPersist, IServiceProvider. All are subclasses of IUnknown. According to OleViewDotNet, IRtdServer is also a subclass of IUnknown. This leads me to believe that I need to similarly write an IRtdServer class in python in order to use the interface, but I don't know how to do that.

I noticed the dynamic parameter of CreateObject. The code indicates this is mutually exclusive to the interface parameter, so I tried that:
```
def __init__(self, clsid):
    self._comObj = comtypes.client.CreateObject(clsid, dynamic=True)
def connect(self):
    self._rtd = self._comObj.IRtdServer()
    result = self._rtd.ServerStart(self)
```
But the error is the same as my original error: IRtdServer has _ctypes.COMError: (-2147352570, 'Unknown name.', (None, None, None, 0, None))

Any help or clues would be greatly be appreciated. Thank you in advance.

(Not really knowing what I'm doing,) I tried to use OleViewDotNet to look at the DLL:

I'm not sure here, but is self._comObj.IRtdServer(self) you really need? Did you tryed just call self._comObj.ServerStart(self)? — Peter Taran
– Peter Taran, Commented Dec 27, 2017 at 4:14
Another option is to put interface you need in 4th parameter of CreateObject: comtypes.client.CreateObject(clsid, None, None, IRtdServer). It may be IRtdServer or "IRtdServer" or it's GUID — comtypes documentation is not pretty clear here. — Peter Taran
– Peter Taran, Commented Dec 27, 2017 at 4:19
I tried several permutations of what you said in my latest edit, under "Some other solutions I've tried". Still not working. I think I need to use dynamic=True to avoid writing my own IRtdServer class though. — mach
– mach, Commented Dec 27, 2017 at 5:22
So you're asking us 'why I can't access the service without creating a class ID' (register). — dsgdfg
– dsgdfg, Commented Dec 27, 2017 at 5:32
I ended up taking a different approach with this project by using win32com to talk to Excel and fill cells with =RTD(...) formulas, then reading the values from the cells. It's an extra step but it works. I had no trouble using win32com for that. — mach
– mach, Commented Jan 10, 2018 at 23:55

Aincrad · Accepted Answer · 2018-10-24 13:00:24Z

5

I ran into same problem.

I also tried using win32com to get excel run that for me, that's a bit unstable to be honest...I cannot even touch my Excel.

Therefore I spent some time looking into this. The problem lies with CastTo. Think that COM object you (and I) loaded just does not contain enough information to be casted (some methods like GetTypeInfo are not implemented etc...)

Therefore I created a wrapper that makes methods of those COM objects callable...not obvious. And this seems working for me.

Client code is modified from a project called pyrtd, which didn't work for various reasons (think due to change of RTD model...return of RefreshData is just completely different now).

import functools

import pythoncom
import win32com.client
from win32com import universal
from win32com.client import gencache
from win32com.server.util import wrap


EXCEL_TLB_GUID = '{00020813-0000-0000-C000-000000000046}'
EXCEL_TLB_LCID = 0
EXCEL_TLB_MAJOR = 1
EXCEL_TLB_MINOR = 4

gencache.EnsureModule(EXCEL_TLB_GUID, EXCEL_TLB_LCID, EXCEL_TLB_MAJOR, EXCEL_TLB_MINOR)

universal.RegisterInterfaces(EXCEL_TLB_GUID,
                             EXCEL_TLB_LCID, EXCEL_TLB_MAJOR, EXCEL_TLB_MINOR,
                             ['IRtdServer', 'IRTDUpdateEvent'])


# noinspection PyProtectedMember
class ObjectWrapperCOM:
    """
    This object can act as a wrapper for an object dispatched using win32com.client.Dispatch
    Sometimes the object written by 3rd party is not well constructed that win32com will not be able to obtain
    type information etc in order to cast the object to a certain interface. win32com.client.CastTo will fail.

    This wrapper class will enable the object to call its methods in this case, even if we do not know what exactly
    the wrapped object is.
    """
    LCID = 0x0

    def __init__(self, obj):
        self._impl = obj  # type: win32com.client.CDispatch

    def __getattr__(self, item):
        flags, dispid = self._impl._find_dispatch_type_(item)
        if dispid is None:
            raise AttributeError("{} is not a valid property or method for this object.".format(item))
        return functools.partial(self._impl._oleobj_.Invoke, dispid, self.LCID, flags, True)


# noinspection PyPep8Naming
class RTDUpdateEvent:
    """
    Implements interface IRTDUpdateEvent from COM imports
    """
    _com_interfaces_ = ['IRTDUpdateEvent']
    _public_methods_ = ['Disconnect', 'UpdateNotify']
    _public_attrs_ = ['HeartbeatInterval']

    # Implementation of IRTDUpdateEvent.
    HeartbeatInterval = -1

    def __init__(self, event_driven=True):
        self.ready = False
        self._event_driven = event_driven

    def UpdateNotify(self):
        if self._event_driven:
            self.ready = True

    def Disconnect(self):
        pass


class RTDClient:
    """
    Implements a Real-Time-Data (RTD) client for accessing COM data sources that provide an IRtdServer interface.
    """

    MAX_REGISTERED_TOPICS = 1024

    def __init__(self, class_id):
        """
        :param classid: can either be class ID or program ID
        """
        self._class_id = class_id
        self._rtd = None
        self._update_event = None

        self._topic_to_id = {}
        self._id_to_topic = {}
        self._topic_values = {}
        self._last_topic_id = 0

    def connect(self, event_driven=True):
        """
        Connects to the RTD server.

        Set event_driven to false if you to disable update notifications.
        In this case you'll need to call refresh_data manually.
        """

        dispatch = win32com.client.Dispatch(self._class_id)
        self._update_event = RTDUpdateEvent(event_driven)
        try:
            self._rtd = win32com.client.CastTo(dispatch, 'IRtdServer')
        except TypeError:
            # Automated makepy failed...no detailed construction available for the class
            self._rtd = ObjectWrapperCOM(dispatch)

        self._rtd.ServerStart(wrap(self._update_event))

    def update(self):
        """
        Check if there is data waiting and call RefreshData if necessary. Returns True if new data has been received.
        Note that you should call this following a call to pythoncom.PumpWaitingMessages(). If you neglect to
        pump the message loop you'll never receive UpdateNotify callbacks.
        """
        # noinspection PyUnresolvedReferences
        pythoncom.PumpWaitingMessages()
        if self._update_event.ready:
            self._update_event.ready = False
            self.refresh_data()
            return True
        else:
            return False

    def refresh_data(self):
        """
        Grabs new data from the RTD server.
        """

        (ids, values) = self._rtd.RefreshData(self.MAX_REGISTERED_TOPICS)
        for id_, value in zip(ids, values):
            if id_ is None and value is None:
                # This is probably the end of message
                continue
            assert id_ in self._id_to_topic, "Topic ID {} is not registered.".format(id_)
            topic = self._id_to_topic[id_]
            self._topic_values[topic] = value

    def get(self, topic: tuple):
        """
        Gets the value of a registered topic. Returns None if no value is available. Throws an exception if
        the topic isn't registered.
        """
        assert topic in self._topic_to_id, 'Topic %s not registered.' % (topic,)
        return self._topic_values.get(topic)

    def register_topic(self, topic: tuple):
        """
        Registers a topic with the RTD server. The topic's value will be updated in subsequent data refreshes.
        """
        if topic not in self._topic_to_id:
            id_ = self._last_topic_id
            self._last_topic_id += 1

            self._topic_to_id[topic] = id_
            self._id_to_topic[id_] = topic

            self._rtd.ConnectData(id_, topic, True)

    def unregister_topic(self, topic: tuple):
        """
        Un-register topic so that it will not get updated.
        :param topic:
        :return:
        """
        assert topic in self._topic_to_id, 'Topic %s not registered.' % (topic,)
        self._rtd.DisconnectData(self._topic_to_id[topic])

    def disconnect(self):
        """
        Closes RTD server connection.
        :return:
        """
        self._rtd.ServerTerminate()

answered Oct 24, 2018 at 13:00

Aincrad

511 silver badge2 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

Mayank Wadhwa Over a year ago

hey can please share how to implement this code. may be a repo showing a example. thank you

Aincrad Over a year ago

you can check pyrtd project, the above is just to make that project work again.

Mayank Wadhwa Over a year ago

Hey man i tried what you said and i am getting the following error

TypeError: Objects for SAFEARRAYS must be sequences (of sequences), or a buffer object. pythoncom error: Failed to call the universal dispatcher

the error is happening on the register_topic() method

Aincrad Over a year ago

I am not able to replicate the error. It works for the RTDServer I am connecting to. Are you passing a tuple as argument for register_topic()? And it also really depends on the implementation of RTDServer...

Mayank Wadhwa Over a year ago

No i am sending a string which is converted to a tuple inside the method. if i need to send a tuple what should be it content beside topic

|

kunif · Accepted Answer · 2017-12-27 10:28:48Z

2

There seems to already be both server/client for Excel 2002.
pyrtd

Looking at that source, once you create a dispatch object, then it seems to be cast to IRtdServer.
Extract related parts, it becomes below.

from win32com import client, universal
from win32com.server.util import wrap

def __init__(self, classid):
    self._classid = classid
    self._rtd = None

def connect(self, event_driven=True):
    dispatch = client.Dispatch(self._classid)
    self._rtd = client.CastTo(dispatch, 'IRtdServer')
    if event_driven:
        self._rtd.ServerStart(wrap(self))
    else:
        self._rtd.ServerStart(None)

Please refer to client.py and examples/rtdtime.py of the following sources.
pyrtd - default
pyrtd/rtd/client.py
pyrtd/examples/rtdtime.py

edited Dec 27, 2017 at 10:28

answered Dec 27, 2017 at 10:23

kunif

4,4002 gold badges13 silver badges35 bronze badges

6 Comments

mach Over a year ago

I've also seen that project, but that code produces various errors. I remember I had trouble with getting Dispatch function to work. I believe I came across an article somewhere which said the RTD or COM model has changed somewhere between Office 2003/2007. That could be the reason, or could be something else... I will include win32com examples in my question soon.

mach Over a year ago

Updated with a win32com example

kunif Over a year ago

Your win32com sample does not match the pyrtd source. The place to call Dispatch(), the variable(_comObj) to store, and the variable(dispatch) specified by CastTo() are different.

mach Over a year ago

(A) Calling Dispatch() whether from __init__() or connect() gave same error. (It would be awfully bad practice if win32com assumed that certain COM methods were only called from inside certain python functions.) (B) I initially directly specified the name of the COM library passed to Dispatch(), but then changed it to use *args unpacking to allow more flexible testing. It's functionally equivalent and also gave the same error either way. (C) The dispatch variable was a copy+paste error - corrected.

kunif Over a year ago

Although this may not be the problem, there was a problem with Excel 2016 's RTD Interface, there was an article that it was able to be avoided by specifying the server name by the RTD().

|

tifoji · Accepted Answer · 2025-01-17 21:38:49Z

1

I have released a version using comtypes. @mach

https://github.com/tifoji/pyrtdc/

win32com also works using an approach similar to the wrapper suggested by @Aincrad

answered Jan 17 at 21:38

tifoji

112 bronze badges

Collectives™ on Stack Overflow

How to use COM from python win32com or comtypes to access an IRTDServer?

Environment

Problem

Code using win32com library

Code using comtypes library

Some other solutions I've tried

3 Answers 3

10 Comments

6 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Environment

Problem

Code using win32com library

Code using comtypes library

Some other solutions I've tried

3 Answers 3

10 Comments

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related