Have you checked what-math-should-all-game-programmers-know?what-math-should-all-game-programmers-know?
If you want to get into 3D then you will need to understand vector operations such as dot-product and cross-product. You will also need to understand matrix basis representation and representations such as the world-matrix. The easiest way is to follow a tutorial for some "My first 3D game" and try to understand the math as you go along.