0

When i use worker_threads to handle a lot of complex logic that irrelevant with the main thread, i found that memory on the server very high.

Below is part of my simplified code.

main.js

const worker = new Worker(process.cwd() + "/worker.js")

// My business will repeat cycle infinitely, this code just an example
for (let i = 0; i < 1000000; i++) {
  worker.postMessage(i)    
}

woker.js

parentPort.on("message", async data => {
  // a log of logic....
})

When I run this scripts on the server, the memory will keep increasing, Maybe the thread is not shut down?

I tried using "worker.removeAllListeners()", resourceLimits option and third party library "workerpool", but still didn't solve my problem.

what should i do or using other method to solve this problem? Thanks for answering me!

Improve this question
4
  • We would have to see more of your real code to understand what might be going on. The for loop you show in main.js would immediately throw a million items in the message queue which is not what you should be doing. I know you said that's not your real code, but we need to see the real code to see how you're actually doing things there. Commented Jan 15, 2023 at 8:40
  • When I've written workerThread code, I don't just send jobs to work threads whenever. Instead, I send a job and wait for the workerThread to tell me its done (with some sort of completion message back to the parent) before sending it another job. That allows me to manage the jobs in the main thread and each worker just does what it's told. It's also possible that whatever you're doing in the workers is just using a lot of memory. Commented Jan 15, 2023 at 8:42
  • WorkerThreads don't shut down on their own unless they crash. They should, however garbage collect on their own memory that is no longer in use. You sound like you're expecting the threads to shut-down on their own. That's not how they work. You would have to call worker.terminate() to stop it. Commented Jan 15, 2023 at 8:43
  • thanks for your reply! i understand how to resolve this problem after I read your comment! Commented Jan 15, 2023 at 9:01
Related questions
0
Split concatenated field into separate parts
2
Extract data from the last two square brackets SQL
1
SQL: Extract every substring found between every brackets in that string using a SELECT statement
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.