Connecting a C++ program to a Python script with shared memory

Question

I'm trying to connect a C++ program to python using shared memory but I don't know how to pass the name of the memory segment to python.

Here is my C++ code:

key_t key = ftok("address", 1);
int shm_o;
char* msg = "hello there";
int len = strlen(msg) + 1;
void* addr;

shm_o = shmget(key, 20, IPC_CREAT | 0600);
if(shm_o == -1)
{
    std::cout << "Failed: shmget.\n";
    return 1;
}

addr = shmat(shm_o, NULL, 0);
if(addr == (void*) -1)
{
    std::cout << "Failed: shmat.\n";
    return 1;
}

std::cout << "Shared memory segment created successfully with id: " << shm_o;
memcpy(addr, msg, len);

getchar();
return 0;

I'm trying to get python to read from the shared memory segment like so:

shm_a = shared_memory.SharedMemory(name="address", create=False, size=20)

print(bytes(shm_a.buf[:11]))

but it throws an exception saying there is no file or directory called 'address'.

Am I going about this correctly or is there another way to attach python to the shared memory segment?

Any help would be much appreciated.

Python documentation states that shared_memory refers to "System V style” shared memory blocks (though is not necessarily implemented explicitly as such)". Try creating a shmem process with python and see if it is visible in ipcs -m. — treuss
– treuss, Commented Sep 14, 2022 at 15:24
@treuss I tried it. The memory segment doesn't show up. does this mean you can't share memory between python and c++? — kasra
– kasra, Commented Sep 14, 2022 at 15:30
@kasra: Afaics, on Posix, Python's multiprocessing module uses Posix shared memory, not SysV shared memory (despite the documentation's claim that it's "System V style" shared memory). On Windows, it uses a different system. Interfacing with C++ (or C) would require the same dual implementation, I think. But shmget is not part of either of them. — rici
– rici, Commented Sep 14, 2022 at 15:50
Try changing your c++ code to use POSIX shared memory segements, i.e. shm_open and shm_unlink from <sys.mman.h> — treuss
– treuss, Commented Sep 14, 2022 at 16:02

treuss · Accepted Answer · 2022-09-14 17:45:41Z

Taking the liberty to post a working example here for POSIX shared memory segments, which will work across C/C++ and Python on Linux/UNIX-like systems. This will not work on Windows.

C++ code to create and write data into a shared memory segment (name provided on command line):

#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>

#include <iostream>
#include <string>

int main(int argc, char * argv[])
{
    if (argc != 2) {
         std::cerr << "Argument <shmem_name> required" << std::endl;
         return 1;
    }
    const char * shmem_name = argv[1];
    size_t shm_size = 4096;
    int shmem_fd = shm_open(shmem_name, O_CREAT|O_RDWR, S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP);
    if (shmem_fd == -1) {
         perror("shm_open");
         return 1;
    }
    std::cout << "Shared Memory segment created with fd " << shmem_fd << std::endl;
    if (ftruncate(shmem_fd, shm_size) == -1) {
        perror("ftruncate");
        return 1;
    }
    std::cout << "Shared Memory segment resized to " << shm_size << std::endl;
    void * addr = mmap(0, shm_size, PROT_WRITE, MAP_SHARED, shmem_fd, 0);
    if (addr == MAP_FAILED) {
        perror("mmap");
        return 1;
    }
    std::cout << "Please enter some text to write to shared memory segment\n";
    std::string text;
    std::getline(std::cin, text);
    while (! text.empty()) {
        strncpy((char *)addr, text.data(), shm_size);
        std::cout << "Written '" << text << "' to shared memory segment\n";
        std::getline(std::cin, text);
    }
    std::cout << "Unlinking shared memory segment." << std::endl;
    shm_unlink(shmem_name) ;
}

Python code to read any string from the beginning of the shared memory segment:

import sys
from multiprocessing import shared_memory, resource_tracker

if len(sys.argv) != 2:
    print("Argument <shmem_name> required")
    sys.exit(1)

shm_seg = shared_memory.SharedMemory(name=sys.argv[1])
print(bytes(shm_seg.buf).strip(b'\x00').decode('ascii'))
shm_seg.close()
# Manually remove segment from resource_tracker, otherwise shmem segment
# will be unlinked upon program exit
resource_tracker.unregister(shm_seg._name, "shared_memory")

Collectives™ on Stack Overflow

Connecting a C++ program to a Python script with shared memory

1 Answer 1

C++ code to create and write data into a shared memory segment (name provided on command line):

Python code to read any string from the beginning of the shared memory segment:

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

C++ code to create and write data into a shared memory segment (name provided on command line):

Python code to read any string from the beginning of the shared memory segment:

Comments

Your Answer

Sign up or log in

Post as a guest

Related