Run-Time Support for Asynchronous Parallel Computations

Abstract

The notion of a global memory space is a useful abstraction, that provides a natural way of writing portable parallel applications. However, the implementation of this concept on nonshared memory architectures often forces applications to behave in a suboptimal way, due to decisions and policies that are actually fixed during the design phase. In order to fully exploit the underlying machine, the application must be able to control the detailed behavior of the system during the execution; these decisions are potentially dependent on the particular tradeoffs of the target architecture, and on the problem-specific information available to the programmer.

This work describes the motivations and major design aspects of a distributed environment, dubbed DREAM, that provides support for parallel computations on asynchronous multiprocessors. DREAM's facilities allow the development of programs that incur a low amount of overhead and possibly make use of dynamic load-balancing techniques. Among the possible optimizations achievable by using DREAM, we can consider overlapping of communication with computation, message vectorization for network transactions, and efficient support for on-demand dynamic communication patterns. We also report on the performance results of a first prototype of the system, currently running on a network of IBM RS/6000 workstations.

Full version (PDF format)