So I'm storing my MySQL database connection login info in a .php file that's above the web root.
But if someone gains access to the whole server, they could potentially open that file up
What else can I do to protect the info?
You cannot do anything against the person who got the access to the server.
Besides the obvious answers, you should ensure your site is safe from directory traversal attacks that may read your PHP file above the root.
echo file_get_contents($_GET['page']);
(better example than previous)