EDIT
Okay, I finally figured it out! (Note: Since I'm new here (longtime lurker!) I can't answer my own question for a few hours, so as soon as I can I'll move this into an actual answer to the question)
I changed
Matrix.orthoM(projection, 0, 0, windowWidth, 0, windowHeight, -1, 1);
to
float ratio = windowWwidth / windowHeight;
Matrix.orthoM(projection, 0, 0, ratio, 0, 1, -1, 1);
I then had to scale my projection matrix to make it a lot smaller with Matrix.scaleM(projection, 0, 0.05f, 0.05f, 1.0f);.
I then added an offset to the modelview translations to simulate a camera so that I could center on my action (so Matrix.translateM(modelview, 0, quadX, quadY, 0); was changed to Matrix.translateM(modelview, 0, quadX + camX, quadY + camY, 0);)
Thanks for the help, all!