The % (percent) symbol is special to both (La)TeX and Lua, but in very different ways. In (La)TeX, it's the default comment character. In Lua, it's one of the "magic" characters in pattern matching operations. (Lua's other "magic" characters are ^$().[]*+-?.)
The \directlua directive is expandable, in the TeX-specific sense of the word. This means that TeX scans the argument of \directlua for any macros; if macros are found, TeX will try to expand them. In your second example, when (La)TeX scans
\directlua{tex.sprint(string.format(' %.3f', math.pi))}
it doesn't come across any LaTeX macros, but it does encounter the % character, which it interprets as a comment character. Hence, everything through the end of the input line is ignored, including the two closing parentheses and the single closing curly brace. That's why you're getting an error message.
How to keep these two rather incompatible uses of % from interfering with each other? I can think of five methods. The first two assure that LaTeX never gets to "see" and act on the % character to begin with. The next two work by "smuggling" some form of % past LaTeX's prying eyes. The final method employs Lua's string.char function to generate a % symbol inside a Lua code block, thereby also keeping LaTeX from doing anything untoward.
Load the Lua code in such a way that it's not "seen" by LaTeX.
Place the Lua code in an external file (generally with extension .lua, e.g., file.lua) and load the code into LuaTeX with a \directlua{dofile("file.lua")} instruction.
In your main tex file, load the luacode package, which provides environments called luacode and luacode*, and place your Lua code in either a luacode or a luacode* environment.
Instead of \directlua, use an instruction which can handle \% as a substitute for % while passing control to Lua.
Employ the macro \luaexec (provided by the luacode package) as a substitute for \directlua. Unlike \directlua, \luaexec lets you "escape" % by prefixing it with a backslash, i.e., by writing \%. E.g.,
\luaexec{tex.sprint(string.format('\%.3f', math.pi))}
Define a macro called, say, \percentchar which expands to % without this character being interpreted by TeX as a comment character. This allows continued use of \directlua. (Many thanks to @JosephWright for providing this suggestion and to @egreg for providing a pointer to an even more parsimonious definition of \percentchar.)
Place the following instruction in the preamble to make its scope global:
\makeatletter\let\percentchar\@percentchar\makeatother
%% or: \begingroup\catcode`\%=12\relax\gdef\percentchar{%}\endgroup
Then, execute this instruction in the body of the document:
\directlua{tex.sprint(string.format('\percentchar.3f', math.pi))}
Recall that \directlua is expandable, i.e., it'll expand \percentchar to %, which is the character Lua needs to see in the string.format function. And, because this form of % has been given catcode 12 ("other"), it won't be misinterpreted as a comment character by LaTeX -- and all is well. :-)
Use Lua to generate the % character. (Many thanks to @AlainMatthes for suggesting this method.)
Instead of inputting some form of the % character directly, use Lua's string.char function -- specifically, run string.char(37) -- to generate the % character. ("37" is the position of the % character in the ASCII table.)
\directlua{
local pcc = string.char(37)
tex.sprint(string.format(pcc .. '.3f', math.pi))
}
While methods 4 and 5 may seem a bit cumbersome at first, they enjoy the undeniable advantage of being implementable without having to load any packages (such as the luacode package) or having to place the Lua code in an external file.
Which of these five methods one should use depends crucially on what one needs to accomplish when making a Lua call. If it's just a single instance of a simple instruction, methods 3 thru 5 may be easiest. If the Lua code is long and complex, methods 1 and 2 are almost certainly to be preferred as you (the programmer) will get to "see" the % characters directly.
All five possibilities are illustrated in the following sample document.

% !TEX TS-program = lualatex
\RequirePackage{filecontents}
%% Create an external file to store a Lua function
\begin{filecontents*}{aux.lua}
function printpi (n) -- n: number of decimal digits to be shown
tex.sprint ( string.format ( '%.' .. n .. 'f' , math.pi ))
end
\end{filecontents*}
\documentclass{article}
\directlua{ dofile ( "aux.lua" ) } % load Lua code from external file
\usepackage{luacode} % for 'luacode' environment and '\luaexec' macro
\begin{luacode}
function pi3() -- this function doesn't take an argument
tex.sprint ( string.format ( '%.3f' , math.pi ) )
end
\end{luacode}
\begin{document}
% Method 1
$\pi \approx \directlua{printpi(3)}$
% Method 2
$\pi \approx \directlua{pi3()}$
% Method 3
$\pi \approx \luaexec{tex.sprint(string.format('\%.3f', math.pi))}$
% Method 4
\makeatletter\let\percentchar\@percentchar\makeatother
$\pi \approx \directlua{tex.sprint(string.format('\percentchar.3f', math.pi))}$
% Method 5
\directlua{
local pcc = string.char(37)
tex.sprint('$\\pi\\approx' .. string.format(pcc .. '.3f', math.pi) .. '$')
}
\end{document}