If your data is small enough, just load everything right into memory but be sure to cull objects that are not within the dimensions of the screen. This can be as simple as a rectangle on rectangle intersection test. Doing this will save precious time in the graphics pipeline. OpenGL must do quite a bit of work before it can safely discard invisible shapes, it's best to do this work before sending the vertices to OpenGL.
Another option is to load chunks of your level into memory, discard chunks that are no longer visible. Chunks should be at least the size of your screen to be effective (going any smaller would probably be overkill). Culling here is also important; even if you're chunk is on screen, that doesn't mean all of it will be.
You would want some sort of method of detecting when a chunk is close to being in view (for example being close to the edge of a screen, but not on-screen yet). Load in a chunk when it is close enough and unload when it is far enough. Exact implementation is up to you. You could use something like a quad tree if you plan on moving on more than 1 axis (ie left-right, up-down).
Hope that helps.
