"position" probably refers to the upper left corner of your entity.
if (x < tileBounds.Right)
position.X = tileBounds.Right;
The above will move the entity's left edge to the colliding tile's right edge. That's good.
if (x > tileBounds.Left)
position.X = tileBounds.Left;
However, this will move the entity's left edge to the colliding tile's left edge. That's bad - it places your entity squarely inside the collidee. After that, since the entity is still intersecting, on the next loop through the code, that left check passes (because you compare with > instead of >=, and you just set X = Left), but the right check hits and moves the entity to the right.
Your top and left checks need to take into account the width and height of your entity.