OpenGL uses a left-handed coordinate system, so the camera's Z axis is always pointing "into" the scene.
What you need to do is rotate the camera's Z axis into object space, which is actually quite simple:
Say the upper 3x3 of your model-view matrix is orthogonal (a scaled rotation). Use the normalization of the 3rd row of it as the direction vector.
For a general model space matrix, you need to do this:
objZ = (inverse(MV)*vec4(0,0,1,0)).xyz;
Then normalize.