Skip to main content
We’ve updated our Terms of Service. A new AI Addendum clarifies how Stack Overflow utilizes AI interactions.
Software Engineering

Questions tagged [parallel-programming]

Ask Question

For questions about programming in such a way that a task can be divided among multiple threads/workers which work simultaneously ('in parallel') on the subtasks, leading to performance gains.

74 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
0 votes
0 answers
86 views

I am working on a project where I need to transfer thousands of files (each sized between 50-60 MB) every hour from an SFTP server to local storage or AWS S3. I am using Apache Spark 3.5 with Scala 2....
Abhishek 's user avatar
  • 11
1 vote
1 answer
476 views

In a distributred system that uses parallel processing to complete tasks and I want to determine when all sub tasks have been completed, what design method or principals can i apply ? I am using ...
Manse's user avatar
  • 149
0 votes
0 answers
45 views

I want to run a complex algorithm consisting of many subtasks in parallel on a number of processors. These subtasks depend on one another, so that this algorithm is characterized by a DAG (Directed ...
Zur Luria's user avatar
  • 113
-2 votes
1 answer
93 views

I am supervising a university project on the topic of race detection for multi-threaded processes with shared memory. We have a complex algorithm consisting of tens of thousands of subtasks, and we ...
Zur Luria's user avatar
  • 113
0 votes
0 answers
106 views

I have a situation where I want to run a complex algorithm consisting of many subtasks in parallel on a number of processors. These subtasks depend on one another, so that this algorithm is ...
Zur Luria's user avatar
  • 113
0 votes
1 answer
158 views

I'm not sure if my answers to the following exercise is correct: Suppose the program is run with two threads. Further suppose that the following sequence of events occurs: Time Thread 0 ...
Lilian Shi's user avatar
  • 11
1 vote
1 answer
222 views

I am trying to create a simple demonstration of using 'parallel LINQ' (PLINQ). I have two versions of my task, in C#: var result = Enumerable.Range(1,1000000).Where(x => IsPrime(x)).ToList(); ...
Richard Pawson's user avatar
  • 113
3 votes
1 answer
169 views

I have 1-20 agents that issue requests over time with roughly ~50 in total in flight at any given time. Many of these requests are identical. The tasks are idempotent, but if two identical requests ...
Unknown1987's user avatar
  • 141
-3 votes
2 answers
141 views

In Practical Foundation of Programming Languages 38 Futures and Speculations A future is a computation that is performed before it is value is needed. Like a suspension, a future represents a value ...
Tim's user avatar
  • 5,555
2 votes
2 answers
611 views

In Fundamentals of Parallel Multicore Architecture, by Yan Solihin, p304 defines sequential consistency memory model: Overall, we can express programmers’ implicit expectation of memory access ...
Tim's user avatar
  • 5,555
1 vote
2 answers
1k views

https://en.wikipedia.org/wiki/Memory_coherence says: Memory coherence is an issue that affects the design of computer systems in which two or more processors or cores share a common area of memory.[1]...
Tim's user avatar
  • 5,555
0 votes
2 answers
1k views

I have a reinforcement learning project. For this I created a vectorized environment in C++, which is a handler for multiple instances of a simple game. It is highly parallelizable. Each worker can ...
Dudly01's user avatar
  • 137
0 votes
0 answers
57 views

Let's say that there is a simple sequential algorithm that performs a linear search of a unique key in an array: public class SearchSeq { public static int search(int[] a, int key) { for (int i ...
Konstantinos Loizas's user avatar
  • 11
-2 votes
1 answer
872 views

I have a .net web application, let's say A and a .net console application, let's say B.In A, I have a queue with multiple jobs to be done by B.Once I select the jobs from A, a queue gets created. Each ...
mrinali's user avatar
  • 23
3 votes
3 answers
2k views

So I am trying to speed up my program by using concurrency and/or multi-threading and/or process parallelism. The topics are pretty complex and I am sort of new to them so I am still trying to figure ...
tera_789's user avatar
  • 263
3 votes
1 answer
11k views

I have a website which offers pages in the format of https://www.example.com/X where X is a sequential, unique number increasing by one every time a page is created by the users and never reused even ...
nicktheone's user avatar
  • 41
4 votes
3 answers
2k views

This is a question I constantly ask myself when designing a data intensive application: When is it appropriate to use stream() over parallelStream()? Would it make sense to use both? How do I quantify ...
user0000001's user avatar
  • 263
0 votes
2 answers
77 views

I'm wondering, in general, if a part of a program can be safely parallelized with significant performance gains, should we? Or do we need consider the importance of this work because it can affect ...
roverred's user avatar
  • 397
10 votes
1 answer
452 views

Tiled images Tiled images are large images that have been split in smaller square tiles. There are several tiled image formats, with different ways of organizing the tile files. A tiled image on ...
lovasoa's user avatar
  • 209
0 votes
1 answer
368 views

I am writing a C library with routines I want to call from an external driver program using many OpenMP threads. The library will have a main container called lib_workspace, which will be used to do ...
vibe's user avatar
  • 101
1 vote
0 answers
61 views

When using derived types in MPI for communication of data which is not contiguous in memory, the sequence of datatype-displacement pairs defining the derived type seems to be assumed to be the same ...
K. Mylonakis's user avatar
  • 11
0 votes
1 answer
2k views

I have the need to distribute a set of long running jobs across a cluster of containers in ECS. These jobs essentially would need to open a socket connection with a remote server and begin streaming ...
remarkpk's user avatar
  • 9
0 votes
1 answer
188 views

I am fairly new to Docker and Kubernetes and I have a question for which I could not figure out the answer myself. I am working on an application that does string matching on data extracted from ...
calin.bule's user avatar
  • 3
0 votes
3 answers
207 views

I have been interested in parallel computing lately, and I just wanted to check if there's some sort of standard or workflow for designing a parallel architecture. In particular, I am interested in ...
anonuser01's user avatar
  • 117
2 votes
1 answer
165 views

I have known about the Flynn taxonomy for a while (SIMD, MIMD, etc.), as well as models of concurrency (actor model, petri nets, etc.). However I am wondering now how they overlap and interrelate. If ...
Lance Pollard's user avatar
  • 2,757
4 votes
1 answer
210 views

The application I'm developing requires that some data is obtained through different channels. As this is a network process, I have decided to use async programming. I have designed the following ...
Daniel García Rubio's user avatar
  • 151
7 votes
3 answers
2k views

How can I tell in an application if there are parts that will benefit from using the C# Task library (I believe that is the Parallel processing library.) Given other optimizations have been done, how ...
johnny's user avatar
  • 3,679
1 vote
2 answers
920 views

Using Functional language, How can 2 different parties achieve the result of increment/decrement operations concurrently? For the below scenario, Let's say, I've 2 quantities in stock and 2 users in ...
Vicky's user avatar
  • 399
7 votes
1 answer
874 views

While working on an inherited project, I noticed the original dev(s) created many asynchronous functions that never seem to take advantage of being, well, asynchronous. For example: // The async ...
8protons's user avatar
  • 1,389
3 votes
2 answers
500 views

I'm implementing an algorithm that needs to Sort elements, in parallel (this will be done on a GPU, but that doesn't matter much) Compute a comparison metric for every pair of elements. This ...
Olumide's user avatar
  • 251
1 vote
2 answers
308 views

I have general parallel programming question. Suppose there is a directed graph with cycles. Let’s assume that each node has fairly small amount of incoming edges ~ from 0 to 20 and potentially ...
EugeneL's user avatar
  • 19
0 votes
1 answer
159 views

I'm new to actors and just playing around in ruby using Celluloid. I have this code below module Enumerable # Simple parallel map using Celluloid::Futures def pmap(&block) futures = map { ...
Grimbox's user avatar
  • 377
0 votes
1 answer
284 views

How would one implement an actor-based concurrent language in ruby? My thought is that only the correct way of creating programs is using actors, but I'm not entirely sure on how this could be done, ...
Grimbox's user avatar
  • 377
26 votes
10 answers
7k views

I'm currently creating a programming language for fun where the idea is that every function call/new block (if clauses, loops etc) will work in a separate thread. Instead of creating new Threads the ...
Grimbox's user avatar
  • 377
1 vote
0 answers
91 views

List specific programming concepts that should code adhere to that it will run in parallel. For example, if a block of code does not change shared state, it should be able to be done on another thread....
disassemble-number-5's user avatar
  • 135
0 votes
1 answer
265 views

I'm developing a framework with C++, and it contains three layers: Low level functions which do the hard work An upper layer which uses the low level functions to accomplish tasks (functions at this ...
bayindirh's user avatar
  • 101
3 votes
2 answers
3k views

When doing (possibly heavy) pixel processing on a large image, multithreading becomes a must. The standard practice is to initiate a loop whose indices are partitioned into multiple threads within a ...
MathuSum Mut's user avatar
  • 219
2 votes
2 answers
148 views

I have a doubt with threading data service calls within a foreach loop. Here it goes: Say you need to request data from a service and then process that data, for this example, let's say data request ...
barclow's user avatar
  • 161
1 vote
1 answer
1k views

I am starting to learn parallel programming in c++, and I have a program like this: for i = 1:N do something time and memory intensive end If I program this in parallel, and my processors have ...
ckoe's user avatar
  • 111
0 votes
3 answers
1k views

I have inherited an application which performs approximately 100,000 executions in a C# for-loop against SQL Server. for (int i=0; i<100000; i++) { //Execution can take 0.250 to 5 seconds to ...
TheLegendaryCopyCoder's user avatar
  • 101
1 vote
4 answers
2k views

I define an algorithm to be best if it minimizes the total run time over commodity hardware (e.g. normal desktop and server PCs). I have sets A and B. I also have function f that works like f(a, b, n)...
caveman's user avatar
  • 73
-1 votes
2 answers
158 views

The title may have been a little vague... I am working on a piece of software that is designed to perform one task. I would like this task to work in parallel, allowing for multiple asynchronous ...
user0000001's user avatar
  • 263
1 vote
1 answer
446 views

I was reading about implementation of producer-consumer problem with one producer and one consumer. Looking at this paragraph I see the implementation where shared resource access is not synchronized ...
ctomek's user avatar
  • 212
-2 votes
1 answer
69 views

Running a ray tracing program as a single thread on an iMac quad-core processor the CPU utilisation is 12%. OK so one of the Intel hyper-threads has been 100% allocated to the process. When GCD is ...
Mike Kidd's user avatar
  • 109
5 votes
1 answer
957 views

I need to implement a producer-consumer pattern for reading, processing and saving electrical values. I have to implement this in C# .NET 4.6.1. I try to describe this in great detail, so that there ...
John's user avatar
  • 358
2 votes
2 answers
10k views

When I write... l2 = [my_computation(x) for x in l] ..., I wonder if python is applying my_computation to every x using all cores of my CPU. If not, why ? Is there a simple way to make python ...
ofaurax's user avatar
  • 473
0 votes
1 answer
209 views

I am wanting to have/ or make a program that runs on a (raspberry pi)computer cluster with one pi executing only video content while the other only handles music, etc under a main program like an AI. ...
Chris Manning's user avatar
  • 101
1 vote
1 answer
2k views

I'm learning Python, more specially parallel programming using Python Parallel Programming Cookbook by Giancarlo Zaccone. At the time the book was published async/await was still in the beta version ...
user avatar
50 votes
5 answers
19k views

I've been reading over and over that functional languages are ideal (or at least very often useful) for parallelism. Why is this? What core concepts and paradigms are typically employed and which ...
Louis Thibault's user avatar
  • 2,208
1 vote
4 answers
1k views

We have an app that is using a fairly simple stack (Linux, PHP, Oracle, Shell Scripts, etc). We have a series of scripts that need to be executed: /scr/app1/start.sh /scr/app2/start.php /scr/app3/...
Avindra Goolcharan's user avatar
  • 119

1
2