High-Level Specification of Data Layout in a Distributed Nested Data Parallel Computation
Abstract
In previous work, we have presented a novel virtual machine for
executing Nested Data-Parallel (NDP) languages on distributed memory
architectures and described how a simple data-parallel functional
language, Adl, can be realized upon this architecture.
Within our model of NDP execution, declarations of data layout take the
form of arbitrary partitioning functions, mathematical mappings from
an index of the aggregate to the identity of its owner (the processing
element which stores the value of that index). Each aggregate is associated
with one such function at the time of its creation.
PFN is an interactive visual tool which affords a higher-level view of
design process of specifying the mapping of data to nodes. Programmers
assign aggregate elements to a chosen processing node by gesture
rather than by writing code. The executable partitioning functions
implementing the PFN specification are generated by PFN according to
an optimized set of templates provably free of logical errors. We
describe our data placement model; a framework for for abstract
specification of partitioning; and the PFN tool itself, illustrating
its use in defining partitioning schemes.