Capture Webcam image using CV2 and Pyglet in Python

While this works, I found that loading images from numpy arrays was pretty slow when the image was high resolution. pygarrrayimage, a python module on github, can load numpy arrays directly into the video card without making a copy:

https://github.com/motmot/pygarrayimage

This kept my python application which is loading images from high res videos from lagging up. Check out the example folder for how to blit images to the screen fast.

There are a number of problems with your approach, but the trickiest thing is converting numpy arrays to textures. I use the approach below, which I discovered at some point elsewhere on SO. In short, you have to utilize the ctypes types and structures exposed by pyglet.gl in order to generate an array of GLubytes, and then put the contents of the image (a numpy array) into it. Then, because you have a 1-d array of values, you have to specify how Pyglet should make the image, pImage here, by specifying the pixel format and pitch.

If you get the example below working, you should be able to get pImg to update on each call of on_draw, and you should be done.

import pyglet
from pyglet.gl import *
from pyglet.window import key
import cv2
import numpy
import sys

window = pyglet.window.Window()

camera=cv2.VideoCapture(0)

retval,img = camera.read()
sy,sx,number_of_channels = img.shape
number_of_bytes = sy*sx*number_of_channels

img = img.ravel()

image_texture = (GLubyte * number_of_bytes)( *img.astype('uint8') )
# my webcam happens to produce BGR; you may need 'RGB', 'RGBA', etc. instead
pImg = pyglet.image.ImageData(sx,sy,'BGR',
       image_texture,pitch=sx*number_of_channels)

@window.event
def on_key_press(symbol, modifiers):
    if symbol == key.ESCAPE:
        print 'Application Exited with Key Press'
        window.close()

@window.event
def on_draw():
    window.clear()
    pImg.blit(0,0)

pyglet.app.run()

You can convert each opencv image to a pyglet image by using the ImageData constructor. The idea is to convert the opencv image to a PIL array, which in turn is converted to a string of bytes, and then passed as the raw data to the constructor.

from PIL import Image
def cv2glet(img):
    '''Assumes image is in BGR color space. Returns a pyimg object'''
    rows, cols, channels = img.shape
    raw_img = Image.fromarray(img).tobytes()

    top_to_bottom_flag = -1
    bytes_per_row = channels*cols
    pyimg = pyglet.image.ImageData(width=cols, 
                                   height=rows, 
                                   format='BGR', 
                                   data=raw_img, 
                                   pitch=top_to_bottom_flag*bytes_per_row)
    return pyimg

import pyglet
import cv2
window = pyglet.window.Window()

video = cv2.VideoCapture(0)
def takepicture(dt):
    num = 0

    ret,frame = video.read()
    cv2.imwrite(str(num)+'.jpg',frame)
    print("Image_Captured")

@window.event
def on_draw():
    window.clear()
    image = pyglet.image.load('0.jpg')
    image.blit(0,0)

pyglet.clock.schedule_interval(takepicture, 0.001)


pyglet.app.run()