Collective operation

A collective operation is a concept in parallel computing in which data is simultaneously sent to or received from many nodes.

Common examples of collective operations are gather (in which data is collected from all nodes), scatter (in which a set of data is broken up into pieces, and a different piece is sent to all nodes), and broadcast (in which the same data is sent to all nodes).

Parallel computing frameworks with collective operations

Collective operations are present in the following parallel programming frameworks:

References

  1. ^ Intercommunicator Collective Operations. The Message Passing Interface (MPI) standard, chapter 7.3.1. Mathematics and Computer Science Division, Argonne National Laboratory.