There are multiple ways to reload classes of a Java application while it is running.
To put it simply, the application can unload itself using ClassLoaders or external tools (like debuggers) could communicate with the JVM and tell it to redefine classes.
Spring restarts (or restarts performed by the application)
Spring Boot devtools supports an automatic restart:
Applications that use spring-boot-devtools automatically restart whenever files on the classpath change. This can be a useful feature when working in an IDE, as it gives a very fast feedback loop for code changes. By default, any entry on the classpath that points to a folder is monitored for changes.
So essentially, Spring is able to detect changes to the classpath, stop everything related to Spring and start it again. This works because Spring has its own lifecycle logic (see also this) and its components can shut down and can then be recreated. Spring actually does that as you can see in the docs:
The restart technology provided by Spring Boot works by using two classloaders. Classes that do not change (for example, those from third-party jars) are loaded into a base classloader. Classes that you are actively developing are loaded into a restart classloader. When the application is restarted, the restart classloader is thrown away and a new one is created.
Once the application is fully stopped, it can close the ClassLoader, make sure everything is actually stops and load everything again. it is also possible to use a parent ClassLoader for library classes that can be assumed to not change and a child ClassLoader for application code that may be reloaded.
If you take a look at an application using Spring devtools with the restart logic enabled, you might see that the classes are actually loaded by a RestartClassLoader which is a subclass of URLClassLoader. You can check the source code here.
Certain parts of Spring applications like static files can also be hot swapped without restarting the context.
IDE reloads
While the restart/reload logic can be part of the application, it can also be provided by IDEs or other tools outside of it. This can be done using the debugging APIs or via agents/instrumentation.
For example, if you start a Java application in Eclipse in Debug mode and change some classes, it automatically replaces these classes during execution. Functionality for redefining classes is provided by the debugging APIs (see also the JVM TI docs on redefining classes).
This doesn't unload the class and load a new class but it replaces the code in loaded methods and does (normally) not work if method signatures or fields change.
If we take a look at the source code of Eclipse JDT-debug, we can see that it listens to changed classes, checks whether everything can be hot-swapped and then performs hot swapping here.
If everything works out, it exits out of the methods if the code to replace is currently being executed, tells the JVM to redefine the classes and finally goes back into the redefined methods.
Other tools
Apart from that, there are also other tools like JRebel that uses instrumentation/agents and doesn't need to be run in debug mode. Note that this is a commercial tool which I have never used so I don't know how exactly that works. However, I think it's still worth mentioning due to its different approach. This may also use something like JVM TI for that.
On the topic of IDE restarts, a project called DCEVM even allows reloading classes whens signatures changed. You can find publications clarifying how that works here.
Affiliation Disclaimer: At the time of writing this, I am studying at JKU where this was originally developed but I was not involved in that project in any way.
Hot deploy in production
As you also asked
is there any way to achieve hot reload/deploy in production in Java
I recommend not using these approaches in production as it may result in inconsistencies.
For example, with debugging/instrumentation-based hot code replace, you could have assigned something to a field but then you replace the code in some method that's now assuming a different assignment of the field or it might also result in some code getting executed multiple times. These inconsistencies are fine for development but are dangerous in production. Hot code replace (or whatever your tool of choice calls it) is a feature for development.
The Spring approach may be more robust but it may also result in some inconsistencies if a library you are using started things that aren't stopped correctly or similar.
However, with many application servers or with OSGi or similar solutions as mentioned in this other answer by Dudi Boy, you can use similar approaches that are built for production. Many application servers are build for running multiple applications in the same JVM and allow stopping, starting and redeploying some of them without restarting the whole JVM.
If you don't use anything like that, just restart the application. There are also some ways for an application to restart itself. If startup time is an issue, you could use things like AppCDS (see this usability improvement) to improve on it.