4
$\begingroup$

I have some rendered or processed pixel data in a bgl.Buffer() object and need it in a numpy array but that proccess seems to take very long. Here is what I have tried so far:

buffer = bgl.Buffer(bgl.GL_BYTE, WIDTH * HEIGHT * 4)
# render something into it
imageDataNp = np.empty(WIDTH * HEIGHT * 4, dtype=np.float32)

# now some benchmarks
imageDataNp = np.array(buffer, dtype=np.float32) # 7.05s
imageDataNp = np.asarray(buffer, dtype=np.float32) # 7.00s
imageDataNp = np.fromiter(buffer, dtype=np.float32) # 3.29s

imageDataNp = np.array(buffer.to_list(), dtype=np.float32) # 4.69s
imageDataNp = np.asarray(buffer.to_list(), dtype=np.float32) # 4.71s
imageDataNp = np.fromiter(buffer.to_list(), dtype=np.float32) # 2.80s

# create an image datablock as secondary buffer
if not IMAGE_NAME in bpy.data.images:
    bpy.data.images.new(IMAGE_NAME, WIDTH, HEIGHT, float_buffer=True)
image = bpy.data.images[IMAGE_NAME]
image.scale(WIDTH, HEIGHT)

image.pixels.foreach_set(buffer)
image.pixels.foreach_get(imageDataNp) # both lines together take only 2.05s

# answer from stackexchange by Sanoronas
buffer_list = bytes(buffer.to_list())
imageDataNp = np.frombuffer(buffer_list, dtype=np.float32) # both lines together take 1.28s

It appears like even setting and getting to and from a Blender image datablock as a "secondary" buffer ist faster than going directly from bgl.Buffer() to a numpy array.

One possible reason I can think about is that bgl.Buffer() is at the time of 2.92 still not supporting the numpy buffer proctocol as described in this post.

Is there a way to achieve something similar using only Python? I see that the Cookie Cutter addon by CG Cookie ships with an file bgl_ext.py which includes a function def np_array_as_bgl_Buffer(array) which appears to do exactly the complement to what I am trying to achieve.

An example file with the full code from above can be found here.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
7
$\begingroup$

I already talked to Gottfried directly, but I am sure that will be interesting for others as well: The fastest way to copy the image data for now seems to take a slight detour over the OpenGL calls. Here is a complete benchmarking example, which does the following (you need to provide an image yourself):

  1. Create a numpy array of the appropriate size
  2. Create a bgl.Buffer using the numpy array as a template
  3. Copy the pixel data via OpenGL from the image's OpenGL bindcode into the Buffer.
  4. Get a flipped view of the numpy array, which now contains the image data which was copied into the Buffer object.
  5. Save the numpy array via PIL to a PNG image in the blend-files directory. This step takes long and is only to demonstrate that the numpy array really contains the data.

This approach takes about 12 ms for a 4096 x 4096 RGBA image. The same approach can be taken with the GPUOffscreen.color_texture, if you rendered to an offscreen.

import bpy, bgl, os
import numpy as np
import timeit
from PIL import Image

# returns a flipped view into the given numpy array
def from_image_to_numpy(image, ndarray):
    
    # prepare image for OpenGL use
    if image.gl_load(): raise Exception()

    # set image texture to active texture
    bgl.glActiveTexture(bgl.GL_TEXTURE0)
    bgl.glBindTexture(bgl.GL_TEXTURE_2D, image.bindcode)
    
    # then we pass the numpy array to the bgl.Buffer as template,
    # which causes Blender to write the buffer data into the numpy array directly
    buffer = bgl.Buffer(bgl.GL_BYTE, ndarray.shape, ndarray)
    bgl.glGetTexImage(bgl.GL_TEXTURE_2D, 0, bgl.GL_RGBA, bgl.GL_UNSIGNED_BYTE, buffer)
    bgl.glBindTexture(bgl.GL_TEXTURE_2D, 0)
    
    # start time measurement
    start = timeit.default_timer()
    
    # this is needed to flip the image in Y direction due to the differently
    # defined origins of OpenGL and BPY/PIL (top-left vs. bottom-left)
    # NOTE: Could also use np.flip, but that is a bit slower (still < 0.1 ms though)
    flipped_ndarray = ndarray[::-1, :, :]

    # return the flipped array
    return flipped_ndarray

# SOME TESTING CODE
# +++++++++++++++++++++++++++++++++
if __name__ == '__main__':

    # PREPARE THE IMAGE AND NUMPY ARRAY
    # +++++++++++++++++++++++++++++++++
    # get an image from the Blender data blocks
    IMAGE_NAME = "Untitled"
    image = bpy.data.images[IMAGE_NAME]

    # create the numpy array
    ndarray = np.empty((image.size[1], image.size[0], image.channels), dtype=np.uint8)

    # SOME BENCHMARKING OF THE FUNCTION
    # +++++++++++++++++++++++++++++++++
    # define number of repeated executions to take the timing from
    EXEC_NUMBER = 100
    
    # test code to get the average execution time for from_image_to_numpy()
    print("Execution of from_image_to_numpy() takes %.3f ms (avergage of %i executions)" % (timeit.timeit("from_image_to_numpy(image, ndarray)", number=EXEC_NUMBER, globals=locals()) / EXEC_NUMBER * 1000, EXEC_NUMBER))

    # COPY FROM IMAGE TO NUMPY ARRAY
    # +++++++++++++++++++++++++++++++++
    # load the image data into a numpy array
    ndarray = from_image_to_numpy(image, ndarray)

    # SAVE THE ARRAY TO FILE
    # +++++++++++++++++++++++++++++++++
    # directory path of the blend file
    path = bpy.path.abspath(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))

    # start time measurement
    im = Image.fromarray(ndarray)
    im.save(path + "/bpy_image_to_numpy.png", format="PNG", quality=90, optimize=True)```
Improve this answer
$\endgroup$
6
$\begingroup$

You can use the function np.frombuffer. However you first have to convert the bgl Buffer to a bytes list.

buffer_list = bytes(buffer.to_list())
imageDataNp = np.frombuffer(buffer_list, dtype=np.float32)

With your example file this method took 1.69s in contrast to 1.8s for your workaround.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, this is already a big improvement. $\endgroup$ Commented May 3, 2021 at 7:50

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.