4

I have a C# application that will continously allocate memory for data stored in byte arrays. I have another process written in python that will read from these arrays once instantiated. Both processes will be running on a ubuntu machine.

The obvious solution seems to be to share memory between the processes by passing a pointer from the C# process to the python process. However, this has turned out to be difficult.

I've mainly looked at solutions proposed online. Two notable ones are named pipes and mapped memory files. I read the following posts:

Sharing memory between C and Python. Suggested to be done via named pipes: Share memory between C/C++ and Python

The C# application will neither read nor write from the array and the python script will only read from the array. Therefore, this solution doesn't satisfy my efficiency requirements and seems to be a superfluous solution when the data is literally stored in memory.

When i looked at memory mapped files, it seemed as if though that we would allocate memory for these memory files to write the data to. However, the data will already be allocated before the mapped file is used. Thus, it seems inefficient as well.

The second post: https://learn.microsoft.com/en-us/dotnet/standard/io/memory-mapped-files?redirectedfrom=MSDN

The article says: "Starting with the .NET Framework 4, you can use managed code to access memory-mapped files in the same way that native Windows functions access memory-mapped files". Would an ubuntu machine run into potential problems when reading these files in the same way that windows would? And if not, could someone give either a simple example of using these mapped files between the program languages mentioned above as well as pass a reference to these mapped files between the processes, or give a reference to where someone has already done this?

Or if someone knows how to directly pass a pointer to a byte array from C# to python, that would be even better if possible.

Any help is greatly appreciated!

1
  • 1
    Skipt the topics of C# and python. That is unimportant. The topic is interprocess-communication. Memory Mapped Files map hard disk files, that is an option - two processes can communication by accessing the same file, no matter if memory mapped or not. Are you sure, you want to force your applications on the same machine ? If you treat them as on two machines, they rather communicate with remote procedure calls, passing their data in arguments. That's simpler and cleaner, but might be slower. A client-server approach is the usual way to deal with such things. Commented Nov 1, 2019 at 17:40

1 Answer 1

1

So after coming back to this post four months later, i did not find a solution that satisfied my efficiency needs.

I had tried to find a way to get around having to write a large amount of data, already allocated in memory, to another process. Meaning i would have needed to reallocate that same data taking up double the amount of memory and adding additional overhead even though the data would be read-safe. However, it seemed as though the proper way to solve this, in this case, for two processes running on the same machine would be to use named pipes as they are faster than i.e. sockets. As Holger stated, this ended up being a question of interprocess-communication.

I ended up writing the whole application in python which happened to be the better alternative in the end anyways, probably saving me a lot of headache.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.