How to pass 2D array in MPI and create a dynamic tag value using C language?

Question

I am new to MPI programing. I have a 8 by 10 array that I need to use to find the summation of each row parallely. In rank 0 (process 0), it will generate the 8 by 10 matrix using a 2 dimensional array. Then I would use tag number as the first index value(row number) of the array. This way, I can use a unique buffer to send through Isend. However, it looks like my method of tag number generation for Isend is not working. Can you please look in to the following code and tell me if I am passing the 2D array correctly and tag number. When I run this code, it stop just after executing rannk 1 and waits. I use 3 process for this example and use the command mpirun -np 3 test please let me know how to tackle this problem with an example if possible.

#include "mpi.h"
#include <stdio.h>
#include <stdlib.h>

int main (int argc, char *argv[])
{
        MPI_Init(&argc, &argv);
        int world_rank;
        MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
        int world_size;
        MPI_Comm_size(MPI_COMM_WORLD, &world_size);        
        int tag = 1;        
        int arr[8][10]; 
        MPI_Request request;
        MPI_Status status;
        int source = 0;
        int dest;

        printf ("\n--Current Rank: %d\n", world_rank);

        if (world_rank == 0)
        {
            int i = 0;
            int a, b, x, y;

            printf("* Rank 0 excecuting\n");

            for(a=0; a<8/(world_size-1); a++)//if -np is 3, this will loop 4 times
            {           
                for(b=0; b<(world_size-1); b++)//if -np is 3, this loops will loop 2 times
                {//So, if -np is 3, due to both of these loops, Isend will be called 8 times
                    dest = b+1;     
                    tag = a+b;//create a uniqe tag value each time, which can be use as first index value of array
                    //Error: This tag value passing to Isend doesn't seems to be workiing
                    MPI_Isend(&arr[tag][0], 10, MPI_INT, dest, tag, MPI_COMM_WORLD, &request);  
                }
            }

            for(x=0; x<8; x++)//Generating the whole 8 by 10 2D array
            {   
                i++;
                for ( y = 0; y < 10; y++ )
                {
                    arr[x][y] = i; 
                }   
            }               
        }
        else 
        {
            int a, b;                   
            for(b=1; b<=8/(world_size-1); b++)
            {
                int sum = 0;
                int i;
                MPI_Irecv(&arr[tag][0], 10, MPI_INT, source, tag, MPI_COMM_WORLD, &request);
                MPI_Wait (&request, &status);               
                        //Error: not getting the correct tag value
                for(i = 0; i<10; i++)
                {   
                    sum = arr[tag][i]+sum;
                }
                printf("\nSum is: %d at rank: %d and tag is:%d\n", sum, world_rank, tag);
            }           
        }
        MPI_Finalize();
}

Clarissa G · Accepted Answer · 2015-09-19 04:50:13Z

The tag issue is because of how the tag is computed (or not) on different processes. You're initializing the tag values for all processes as

int tag = 1;

and later, for process rank 0 you set the tag to

tag = a+b;

which, for the first time this is set, will set tag to 0 because both a and b start out as zero. However, for processes with rank above 0, the tag is never changed. They will continue to have the tag set to 1.

The tag uniquely identifies the message being sent by MPI_Isend and MPI_Irecv, which means that a send and its corresponding receive must have the same tag for the data transfer to succeed. Because the tags are mismatched between processes for most of the receives, the transfers are mostly unsuccessful. This causes processes with rank higher than 0 to eventually block (wait) forever on the call to MPI_Wait.

In order to fix this, you have to make sure to change the tags for the processes with rank above zero. However, before we can do that, there's a few other issues worth touching up on.

With the way you've set your tag for the rank 0 process right now, tag can only ever have values 0 to 4 (assuming 3 processes). This is because a is limited to the range 0 to 3, and b can only have values 0 or 1. The maximum possible sum of these values is 4. This means that when you access your array using arr[tag][0], you will miss out on a lot of the data, and you'll re-send the same rows several times. I recommend changing the way you approach sending each subarray (which you're currently accessing with tag) so that you have only one for loop to determine which subarray to send, rather than two embedded loops. Then, you can calculate the process to send the array to as

dest = subarray_index%(world_size - 1) + 1;

This will alternate the desitnations between the processes with rank greater than zero. You can keep the tag as just subarray_index. On the receiving side you'll need to calculate the tag per process, per receive.

Finally, I saw that you were initializing your array after you sent the data. You want to do that beforehand.

Combining all these aspects, we get

#include "mpi.h"
#include <stdio.h>
#include <stdlib.h>

int main (int argc, char *argv[])
{
        MPI_Init(&argc, &argv);
        int world_rank;
        MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
        int world_size;
        MPI_Comm_size(MPI_COMM_WORLD, &world_size);        
        int tag = 1;        
        int arr[8][10]; 
        MPI_Request request;
        MPI_Status status;
        int source = 0;
        int dest;

        printf ("\n--Current Rank: %d\n", world_rank);

        if (world_rank == 0)
        {
            int i = 0;
            int a, b, x, y;

            printf("* Rank 0 excecuting\n");
            //I've moved the array generation to before the sends.
            for(x=0; x<8; x++)//Generating the whole 8 by 10 2D array
            {   
                i++;
                for ( y = 0; y < 10; y++ )
                {
                    arr[x][y] = i; 
                }   
            }

            //I added a subarray_index as mentioned above.
            int subarray_index;
            for(subarray_index=0; subarray_index < 8; subarray_index++)
            {
                dest = subarray_index%(world_size - 1) + 1;     
                tag = subarray_index;
                MPI_Isend(&arr[subarray_index][0], 10, MPI_INT, dest, tag, MPI_COMM_WORLD, &request);
            }

        }
        else 
        {
            int a, b;                   
            for(b=0; b<8/(world_size-1); b++)
            {
                int sum = 0;
                int i;
                //We have to do extra calculations here. These match tag, dest, and subarray.
                int my_offset = world_rank-1;
                tag = b*(world_size-1) + my_offset;
                int subarray = b;
                MPI_Irecv(&arr[subarray][0], 10, MPI_INT, source, tag, MPI_COMM_WORLD, &request);
                MPI_Wait (&request, &status);               
                for(i = 0; i<10; i++)
                {   
                    sum = arr[subarray][i]+sum;
                }
                printf("\nSum is: %d at rank: %d and tag is:%d\n", sum, world_rank, tag);
            }           
        }
        MPI_Finalize();
}

There's a one thing that still seems a bit unfinished in this version for you to consider: what will happen if your number of processes changes? For example, if you have 4 processes instead of 3, it looks like you may run into some trouble with the loop

for(b=0; b<8/(world_size-1); b++)

because each process will execute it the same number of times, but the amount of data sent doesn't cleanly split for 3 workers (non-rank-zero processes).

However, if that is not a concern to you, then you do not need to handle such cases.

Gilles · Accepted Answer · 2015-09-19 05:23:03Z

Aside from the obvious question: "why on earth would you want to do that?", there are so many problems here that I'm not sure I'll be able to list them all. I'll give it a try though:

Tag: it seems that the bulk of your method is to use the tag as an indicator of where to look for the receiver. But there are (at least) two major flaws here:
1. Since tag isn't know before reception, what is &arr[tag][0] supposed to be?
2. Tags in MPI are messages "identifier"... On normal circumstances a given communication (send and matching receive) should have a matching tag. This can be alleviated by using MPI_ANY_TAG special tag on the receiving side, and retrieving its actual value using the MPI_TAG field of the reception's status. But that's another story.
Bottom line here is that the method isn't such a good one.
Data initialisation: one of the major principles of non-blocking MPI communications is that you should never modify a buffer you used for a communication between the post of the communication (the MPI_Isend() here) and its finalisation (which is missing here). Therefore, your data generation must happen before the attempts to communicate the data.
Speaking of which, communication finalisation: you have too finalise your sending communications. This can be done using either a wait-type call (MPI_Wait() or MPI_Waitall()), or an "infinite" loop of test-type calls (MPI_Test() and such)...
The MPI_Irecv(): why are you using a non-blocking receive when the very next call is MPI_Wait()? If you want a blocking receive, just call MPI_Recv() directly.

So fundamentally, what you try to do here doesn't look right. Therefore, I'm very reluctant in trying to propose you a corrected version since I don't understand the actual problem you try to solve. Is this code a reduced version of a bigger real one (or an initial version of something supposed to grow), or just a toy example meant for you to learn how MPI send/receive works? Is ther any fundamental reason why you're not using a collective communication such as MPI_Scatter()?

Depending on your answer on these questions, I can try to produce a valid version.

Collectives™ on Stack Overflow

How to pass 2D array in MPI and create a dynamic tag value using C language?

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related