It depends on what you're doing.

All game programmers should know at least high school algebra. That means:

• A good sense of numbers
• Comfort with basic math (add, sub, mult, divide)
• Trigonometry (sin and cos)
• Vectors, angle between vectors, dot product, cross product

Your graphics programmers must know:

• All the above, and
• Matrix algebra (for transformation matrices)
• Polygon intersection methods / ray-polygon (see a book like Real-time collision detection

Your physics programmers must know:

• All of the above and
• Differential equations
• Numerical methods