0

I'm learning java concurrency under the hood and I've read some java article and videos regarding multi threading and concurrency and just can't seem to put them all together

Here is the gist of what i've understand so far (Please correct me if im wrong!) :

  1. CPU has physical core and logical core. Logical core in essence is physical core * the number of threads that can run on each cores
  2. AFAIK Spring Web MVC has a Request per Thread model, therefore from my understanding each request will take 1 of these logical core
  3. Hence if the cpu has 8 logical core, spring can run 8 process parallelly (not concurrent!)
  4. Concurrency refers to a process where a single logical core can handle multiple request, assume a request will do a blocking I/O call, rather than waiting for the I/O call, the thread can be freed to be used to handle another request. this is what it meant to be concurrent

With all that in mind

Now i guess this is where its gets confusing Suppose i have 8 logical core, and a spring web mvc server that can receive api call. In this api call, we will do an never ending blocking I/O operation Does this mean that

  1. I can only hit at maximum 8 times before another request will be queued / delayed since all the threads are used?

The next one would be an improvement on the spring web mvc, rather than doing the I/O operation on the main thread, i want to do it in a non blocking way. which means i will spawn a new thread for each of the api call. Does this mean that

  1. I can only hit at maximum 4 times before another request will be queued / delayed since all the threads are used? (4 main thread, 4 from the blocking I/O call)
Thread t1 = new Thread(() -> {
   // Blocking I/O call
})
t1.start();

Taking a step further, rather than doing the I/O call on the new thread in a blocking manner, i will now use CompletableFuture / Future so that it's asynchronous and non-blocking.

x = CompletableFuture.supplyAsync(() -> {
    // Long I/O call 
})
// do some operation
x.join();

My question here will be

  1. After the supplyAsync part, the process will continue until we come to x.join(); while waiting for x.join() will the thread be blocking? or will the thread be freed and context switching will happen so that the thread can be used to serve another request?

If the answer to the question number 3 is the thread will be blocking, doesn't that mean the only difference from the new thread I/O blocking call (refer to the 2nd improvement) is that when we use completableFuture we can // do some operation before the blocking call?

For the question number 3 i assume that once we get to the blocking part of the code (.get() for Future / .join() completableFuture) the thread will be freed and can handle another request.

Note : Sorry for my bad english!

Improve this question
4
  • 3
    Keep in mind that the thread scheduler of a multitasking operating system (e.g., Windows, macOS, Linux, etc.) is constantly preempting threads to allow other threads to run. So, even when a thread is performing a CPU-bound task, it is not monopolizing a core. Commented May 4, 2024 at 6:02
  • complementing last comment: a Java Thread runs on one (logical) core, but it is not one fixed bound core, the scheduler can change cores. ||| If a Java operation is blocked, probably the scheduler will swap out the specific Java Thread and run a different one on a given core (part of concurrency) ||| Since Java 21, preview since 19, there are Virtual Threads: JEP 444 Commented May 4, 2024 at 6:14
  • Okay, so to clarify no matter how cpu-bound task it is, it will not monopolize a core and If a Java operation is blocked a scheduler will swap out the specific Java Thread and run a different one on a given core. Then when that java operation completes another thread will come to continue the rest of the process? Commented May 4, 2024 at 6:31
  • 1
    Don't get too caught up with Java specifically. There are likely a few hundred processes running on your computer (some might be Java; most are something else). Each process will have one or more threads, meaning there are likely a few thousand threads competing for CPU time. That is many more threads than cores. The scheduler's job is to context switch between threads according to some algorithm. This happens "behind the scenes". Your program simply starts a thread and has it do something; the thread scheduler handles how and when that thread actually executes. Commented May 4, 2024 at 19:44

2 Answers 2

Reset to default
4

CPU has physical core and logical core.

Not always. Only CPU chips with a simultaneous multithreading (SMT) technology like Intel’s Hyper-Threading will appear to offer two logical cores per real physical core.

Basically this technology involves two sets of registers that hold the data and instructions currently executing. The CPU is able to switch between each set of registers quite rapidly, greatly reducing the cost of context switches.

Only one set of registers is in use at a time, but the host operating system sees two cores instead of the one physical core. Some, if not all, such chips allow a sysadmin to disable the hyper-threading feature, thereby cutting in half the number of apparent cores.

Logical core in essence is physical core * the number of threads that can run on each cores

No. Unless I’ve not kept up with industry developments, hyper-threading creates the illusion of only a second logical core per physical core. So a machine with 12 physical cores with hyper-threading disabled will appear to have 24 cores after enabling hyper-threading.

Not all chips have hyper-threading technology. For example, Apple Silicon chips M1, M2, and M3 in modern Macs do not.

Java has a Request per Thread model

I do not know what you mean. Java does not have “requests”. Edit your Question to clarify.

If you meant Web server requests, that is up to the particular web server implementation. The programmers for a basic web server might choose to launch a thread per request. But platform threads are quite expensive in terms of CPU and memory. So a high-performance web server would instead choose to use one fresh new virtual thread per request, or create their own technology for juggling multiple requests.

Hence if the cpu has 8 logical core, jvm can run 8 process parallelly (not concurrent!)

In theory, yes, a machine with 4 physical cores and 8 logical cores can be running 8 threads (not processes) for a JVM at a time but not literally simultaneously because only half the logical cores are ever actually executing.

But keep in mind that dozens, or likely hundreds, of other processes are running threads too. So the CPU is unlikely to be using all its cores for the JVM for anything more than brief moments.

If a machine were often heavily loaded with processes running CPU-bound tasks, then the tiny amount of context-switching overhead cost in hyper-threading may not be so tiny in total. In this scenario, a sysadmin may consider disabling hyper-threading.

Concurrency refers to a

Be aware that the terms multi-threading, parallel computing, concurrent computing, and concurrency are often used loosely and interchanged.

When in doubt, consult Wikipedia, define your own meaning, and ask an author/speaker to clarify theirs.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Hi, thanks for taking the time to answer my questions. I've edited the question, what i meant by java is a spring web mvc program.
Sorry if my questions are confusing. Yes I too found that it doesn't make sense if the CPU were to use all its cores just for the JVM. but then i just can't seem to grasp how does cpu cores and its thread relates to Spring Web MVC threads per request model (prior to Virtual threads) @basil-bourque
@javanoob I do not use Spring, so I can’t address that half of your Question. I do recommend you learn about virtual threads, as you’ll learn much about platform threads too. See video talks by Alan Bateman, Ron Pressler, and José Paumard. And read the JEP I linked.
okay, I'll do that. Thanks you so much! may i still leave this question open? i've added the spring-web-mvc tag in hopes someone that uses it can maybe help to clarify
0

Threads not mean that they would run per core.

Improve this answer

3 Comments

As far as i know threads run on the logical core, so if we have 1 logical core, and a task that runs forever, this logical core will forever be occupied and can no longer be used
@javanoob Everybody is telling you different, so you don’t ‘know’ that at all.
@javanoob That is not how it works at all, the OS thread scheduler will switch between threads when needed, and a thread has no choice in that. At this time, my 16 core machine has 3051 threads active according to Task Manager, if you were correct, 3035 of those would never be able to do any work.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.