2

http://docs.python.org/2/c-api/buffer.html

int ndim

The number of dimensions the memory represents as a multi-dimensional array. If it is 0, strides and suboffsets must be NULL.

What's the real world usage for this? Is it used for scatter gather vector buffers?

1 Answer 1

3

Using ndim and shape is primarily for multidimensional fixed-shape arrays. For example, if you wanted to build something like NumPy from scratch, you might build it around the buffer API. There are also variations to make things easy for NumPy, PIL, and modules that wrap typical C and Fortran array-processing libraries.

If you read a bit further down, the next two values both say "See complex arrays for more information." If you click that link, it gives you an example of doing something like NumPy, and describes how it works.

Also see PEP 3118 for some rationale.

It's not (primarily) for jagged-shaped arrays, like the scatter/gather use. While you can use PIL-style suboffsets for that, it's generally simpler to just use a list or array of buffers (unless you're trying to interface with PIL, of course).

(The old-style buffer API did support a mode designed specifically for scatter/gather-like use, but it was dropped in Python 3.x, and deprecated in 2.6+ once the 3.x API was backported, basically because nobody ever used it.)

Sign up to request clarification or add additional context in comments.

8 Comments

thanks a million! I am noob to scatter/gather, any recommend/related info to get started with? (preparing to write network/file IO tasks for exercise.)
@est: Python 2.x doesn't provide any scatter/gather APIs. Python 3 has recvmsg_into and sendmsg on its sockets, but they work by just providing an iterable of 1D buffers (a list of bytearrays works fine), so there's no need for anything more fancy. If you're using some third-party library, or ctypes-ing direct to platform-specific functions like sendv, then… well, each one will have different requirements.
@est: For example, on most *nix platforms with sendv and/or writev, you have to create an array of struct iovec objects, each of which has a buffer pointer and a length. No Python type has exactly that shape, so you have to build it out of a ctypes.Structure.
Could you explain how to implement jagged-shaped arrays using suboffsets? The length of the second dimension is constant even if you have PIL-style pointer list in the first dimension isn't it?
@molnarg: Here is an implementation of the relevant parts (with incomplete error handling—e.g., if you run out of memory halfway through allocating the Jagged, you'll leak all the buffers allocated so far).
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.