The parallel hardware targeted by Project Sumatra can readily be programmed in the data parallel style.

Data parallel programs tend to exhibit these features:

  • large-scale SIMD: one stream (or a few streams) of instructions applied in parallel to many processors
  • arrays or tables: homogeneous data easily split across processors (easily = sub-linear work to decide partitions)
  • FORALL loops: large-scale program structure of: partition input; do local processing; collect results (and repeat)
  • reduction: inter-processor communication via meshes or trees, producing reductions or scans

Useful links:

Nested Data Parallelism

Flat data parallel languages are not very flexible. In practice programmers prefer to request lots of small tasks rather than one big SIMD task. The limit case is lots of random loops performing scalar operations, which is hard to convert back to any data-parallel form. A compromise is to allow the programmer to specify groupings or nestings on data sets for which an underlying homogeneity can be recovered and exploited by the system.

References (more recent first):

  • No labels


Terms * Trademarks * License: GPLv2 * Contact Us * Privacy