TemporaryDirectory is an object, that is initially designed to be used in a context (with statement) to avoid having to take care of the deletion manually. It has a cleanup method that can be called manually to delete the temporary folder, and an attribute name that returns the full path of the directory. So you could either call temp_dir.cleanup() directly, or os.removedirs(temp_dir.name).
If you really want to manage the lifetime of the folder by yourself, you should rather use tempfile.mkdtemp.
For more information, Here another thread about this same topic. To wrap up, keep in mind that TemporaryDirectory has been designed to make your life easier and safer by deleting the temporary folder automatically for you behind the scene:
with tempfile.TemporaryDirectory() as temp_dir_fullpath:
# Beware temp_dir_fullpath != tempfile.TemporaryDirectory() !!!
print('created temporary directory:', temp_dir_fullpath)
It is very convenient for short-living directory, restricted to a single context (namely not passing the path to other methods or classes...). However, if you need to manager it by yourself (mainly to reuse it later in different contexts or passing it to other methods), in my opinion you are better off doing the job by yourself and just using plain tempfile.mkdtemp method, which is what tempfile.TemporaryDirectory is probably doing under the hood, adding extra sugar for automatic deletion at the cost of returning an object instead of a plain string. This way it is clear that you are handling the deletion by yourself rather than a mistake.