how to share an array by pointer address in python3 between two different process?

Question

I have a program that reads serial port and plot graph with python 3 in raspbery pi. I get 50 bytes per packet and 1000 packet/s so it is a little time consuming work to get and parse those packets. thus I decide to implement it in two different processes using multiprocessing module and use a core for each process. to do this I defined two buffers one is being filled with read_process and the other is being displayed by display_process and using semaphore to synchronise them not to corrupt data.

however I found out that multiprocessing module sharing data is not what I expected as a real sharing memory, as it says this module pickles data and sends it to each process which means it is copying all data instead of sending pointer of that data which reduces speed alot. I just want to know is it possible to send a pointer from one process to another process and how?

By processes do you mean: 2 separate programs; or 2 threads in 1 program? The latter should be easy. The former requires creation of a shared resource: The simplest is a pipe, and that might be good enough if you just want one of your programs to act as a serial buffer, but you can also create a shared memory block. — Gem Taylor
– Gem Taylor, Commented Nov 6, 2019 at 10:46
@GemTaylor I mean two separate process not thread, shared memory in multiprocessing is copying all data so for two different process you have two copies of data in different memory places ,I need to share one memory place so when filled a buffer in read process I could access it on display process instead of copying it which is time consuming — david
– david, Commented Nov 6, 2019 at 10:50
What you do with shared memory is use the OS calls to set up your shared memory between the processes, then coerce your language to use that piece of memory as the particular object's storage. This memory then really is shared - there is no copying (though there are caveats). I can't help you with the latter, as python isn't a language I know well enough, but I know that there are mechanisms. Pipes are easy on unix in almost all languages but it is effectively a double-copy through the O/S handle buffer. — Gem Taylor
– Gem Taylor, Commented Nov 6, 2019 at 14:06
I guess the other question is "what benefit does two processes give you over 2 threads?" It gives you isolation, which you don't want here, but everything else is still isolated of course. — Gem Taylor
– Gem Taylor, Commented Nov 6, 2019 at 14:08
@GemTaylor thank you I will consider OS, and Thread is running in just a core and one python Interpretter means when I am getting data from serialPort my GUI is stopped due to GIL rule in python while with process I have two independent interpretter both process working at the same time in 2 different cores and I have 4 cores — david
– david, Commented Nov 6, 2019 at 14:32

david · Accepted Answer · 2019-11-06 21:33:36Z

1

I could find my answer and fortunately in python version 3.8 they add a new module called shared_memory

and as they said that in what's new page so I build python3.8 for my rpi

answered Nov 6, 2019 at 21:33

david

4755 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

how to share an array by pointer address in python3 between two different process?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related