Skip to main content
more options

Position

Data replication

  • A few program parameters need to be replicated, but not the amplitude array, which has the bulk of the data. Therefore, data replication is not a problem here.

Load balancing

  • All points require equal work, so the points should be divided equally amongst the processes.
  • Cyclic? Block?
    Both types of decomposition split the work load evenly, +/- one point.
  • Cyclic and block do equally well for load balancing.

Communication

  • Cyclic?
    Cyclic decomposition will deal out the points so that neighboring points will always be found on different processes. This isn't what we want!
  • Block?
    Block decomposition leaves contiguous data points on the same process. Only the point at each end of the block requires communication, so the larger the block size, the smaller the percentage of communication overhead.
  • Cyclic does too much communicating compared to block.

Conclusion: domain decomposition should be block by position

ProcessorsPlot

Data replication

  • Some program parameters are replicated, but not the amplitude array
    In general, if large amounts of data are replicated, this will limit the size of the subset of the domain that can be stored on one process, and thus the largest problem size that can be solved. This is not a problem for the wave code.

Load balancing

  • All points require equal work, so the points should be divided equally amongst the processes.
  • Cyclic?
    A cyclic decomposition would have each point given to the available nodes in turn until all are distributed, like dealing cards to the players. In this distribution, the work load is identical for each set of points +/- one point.
  • Block?
    • A block decomposition would have the work split into the number of nodes, leaving contiguous data points on the same node. In this distribution, the work load identical for each block of points +/- one point.
  • Cyclic and block do equally well for load balancing.

Communication

  • Cyclic?
    Neighboring points are all assigned to different processes, therefore each point would require communication.
  • Block?
    • Only the point at each end of the block requires communication, so the larger the block size, the smaller the percentage of communication overhead.
  • Cyclic does too much communicating compared to block.

Conclusion: domain decomposition should be block by position

Each colored area represents a different process. Boxes at the edges of each colored area indicate the endpoints that will need to be communicated between neighboring processes at each time step.

ProcessorsPlot