Passing byte string from Python to C

Question

I am writing a python extension in C and I am trying to pass a bytes object to my function. Obviously the 's' token is for strings; I have tried 'O', 'N', and a few others with no luck. Is there a token I can use to parse a bytes object? If not is there an alternative method to parse bytes objects?

static PyObject *test(PyObject *self, PyObject *args)
{
    char *dev;
    uint8_t *key;

    if(!PyArg_ParseTuple(args, "ss", &dev, &key))
        return NULL;

    printf("%s\n", dev);

    for (int i = 0; i < 32; i++)
    {
          printf("Val %d: %d\n", i, key[i]);
    }

    Py_RETURN_NONE;
}

Calling from python: test(b"device", f.read(32)).

If you use Python 3, the first argument in the call of your test function is an unicode string, not a bytes string. You need to write b"device" instead. — Laurent LAPORTE
– Laurent LAPORTE, Commented Feb 12, 2020 at 22:33
No, the first arg is supposed to be a Unicode string containing the device name. The second arg is a crypto key read in from a file as a bytes object. — Daniel
– Daniel, Commented Feb 12, 2020 at 22:37

ShadowRanger · Accepted Answer · 2020-02-13 00:49:05Z

1

If you read the parsing format string docs, it's pretty clear.

s is solely for getting a NUL terminated UTF-8 encoded C-style string from a str object (so it's appropriate for your first argument, but not your second).

y* is specifically called out in the docs with (emphasized in original text):

This is the recommended way to accept binary data.

y# would also work, at the expense of requiring the caller to provide immutable bytes-like objects, excluding stuff like bytearray and mmap.mmaps.

answered Feb 13, 2020 at 0:49

ShadowRanger

158k12 gold badges221 silver badges314 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Daniel Over a year ago

Firstly, this answered my question; however, it terminates after the loop with Process finished with exit code 139 (interrupted by signal 11: SIGSEGV). Any quick reason you see why this is?

ShadowRanger Over a year ago

@DanielCopley: After the loop? Your code as written could conceivably seg fault if you read outside the bounds of the input array, but it's unlikely. I have no idea what changed once you used buffers though; you're basically asking for psychic debugging here. Segfaults mean you tried to read or write to addresses that aren't mapped into your memory space, but it doesn't say much else.

Daniel Over a year ago

I think I am possibly using the Py_buffer type incorrectly? I switched y* to y (which returns a character pointer instead of Py_buffer) and this seems to have resolved my issue. Not sure if there are any pitfalls with this approach.

ShadowRanger Over a year ago

@DanielCopley: Raw binary data is apt to contain NUL bytes. If you don't use y* or y#, you have no idea what the real length of the data is, and may stop before you reach the end.

ShadowRanger Over a year ago

@DanielCopley: The fact that "positively knowing the length" and getting it wrong means you segfault, or produce total gibberish, rather than being able to raise a helpful exception is a problem. The instant you rely on caller data, you shouldn't be writing code that makes assumptions that can crash your program with no useful diagnostics. It's also misleading to maintainers, who can and will reasonably assume y means "NUL-terminated byte string", because that's the only legal use of it.

|

Collectives™ on Stack Overflow

Passing byte string from Python to C

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related