Parallel MIMD Computation: HEP Supercomputer and its Applications

421 Pages·1985·14.78 MB·English

by  Janusz S. Kowalik

Parallel MIMD Computation
MIT Press Series in Scientific Computation
Dennis B. Gannon, editor
The Massively Parallel Processor, edited by J. L. Potter, 1985
Parallel MIMD Computation: The HEPSupercomputerand Its Applications,
edited by J. S. Kowalik, 1985
Synchronization ofParallel Programs, by F. Andre, D. Herman, and
J.-P.Verjus, 1985
PARALLEL MIMD COMPUTATION:
THE HEP SUPERCOMPUTER AND ITS APPLICATIONS
edited by J. S. Kowalik
The MITPress
Cambridge, Massachusetts
London, England
PUBLISHER'S NOTE
Thisformat isintended to reduce the costof publishing certain works in book form and
to shorten the gap between editorial preparation and final publication. Detailed editing
and composition have been avoided byphotographing the text of this book directly from
the authors' prepared copy.
© 1985 by The Massachusetts Institute of Technology
All rights reserved. No part of this book may be reproduced in any form byany electronic
or mechanical means (including photocopying, recording, or information storage and
retrieval) without permission in writing from the publisher.
Printed and bound inthe United States of America
Library of CongressCataloging in Publication Data
Main entry undertitle:
Parallel MIMD computation.
(The MIT Press seriesin scientific computation)
Includesbibliographical references and index.
1.Denelcor HEP (Computer) 2. Parallel processing (Electronic computers) I. Kowalik,
JanuszS. II.Title: Parallel MIMD computation. III.Series.
QA76.8.D436P37 1985 001.64 85-39
ISBN 0-262-11101-2
Contents
Series Foreword
Preface
J.S. Kowalik
Contributors
PART 1 ARCHITECTURE
1.1 HEP Architecture, Programming and Performance
Harry F. Jordan
1.2 The Architecture of HEP 41
Burton Smith
PART 2 PERFORMANCE
2.1 Performance Characterization of the HEP 59
R. 111I. Hackney
PART 3 PROGRAMMING ANDLANGUAGES
3.1 A Brief Survey of Implicit Parallelism Detection 93
KarlJ. Ottenstein
3.2 HEP SISAL: Parallel Functional Programming 123
S. 1.Allan and R. R. Oldehoeft
3.3 Execution Support for HEP SISAL 151
R.R.Oldehoeft andS. J.Allan
3.4 Logic Programming on the HEP 181
John Gabriel, Tim Lindholm, E.L Lusk, andR.A. Overbeek
3.5 Programming the HEPwith Large-Grain Data Flow Techniques 203
R. G. Babb /I
PART4 APPLICATIONS
4.1 Solving Ordinary Differential Equations on the HEPComputer 231
Swam P. Kumar and Robert E. Lord
4.2 A Parallel Linear Algebra Libraryfor the Denelcor HEP 275
J.J. Dongarra and D. C.Sorensen
4.3 Parallel Algorithms for Recurrence and Tridiagonal Equations 295
J. S. Kowalik andS. P. Kumar
4.4 Hydrocodes on the HEP 309
D. L.Hicks
4.5 LosAlamos Experiences with the HEP Computer 331
O. M. Lubeck, P. O. Frederickson, R.E.Hiromoto, andJ. W Moore
4.6 Solution of Boundary-Value Problems on HEP 349
M. R.Scott
APPENDIX
Useof Monitors inFORTRAN: A Tutorial on the Barrier,
Self-scheduling DO-Loop, and Askfor Monitors 367
E. L. Lusk andR.A. Overbeek
SERIES FOREWORD
It isoften the case that the periods of rapid evolution in the physical sciences occur
when there isa timely confluence of technological advances and improved experi
mental technique. Many physicists, computer scientists, and mathematicians have
saidthat sucha period of rapid change isnow underway. VJeare currently under
going a radical transformation in the way we view the boundaries of experimental
science. It has become increasingly clear that the use of large-scale computation
and mathematical modeling isnow one ofthe most importanttools in the scientific
and engineering laboratory. We have passed the point of viewing the computer
as a device for tabulating and correlating experimental data; we now regard it as
a primary vehicle for testing theories for which no practical experimental apparatus
can be built. NASA scientists speak of "numerical" wind tunnels, and physicists
experiment with the large-scale structure of the universe.
The major technological change accompanying this new view of experimental
science isa blossoming of new approaches to computer architecture and algorithm
design. By exploiting the natural parallelism in scientific applications, new com
puter designs show the promise of major advances in processing power. When
coupled with the current biennial doubling of memory capacity, supercomputers
are on their way to becoming the laboratories of much of modern science.
In this series we hope to focus on the effect these changes are having on the
design of mathematical and scientificsoftware. In particular, we plan to highlight
many major new trends in the design of numerical algorithms and the associated
programming and software tools that are being driven by the new advances in
computer architecture. Of course, the relation between algorithm design and
computer architecture is symbiotic. New views on the structure of physical pro
cesses demand new computational models, which then drive the design of new
machines. We can expect progress inthis area for manyyears, asour understanding
of the emerging science of concurrent computation deepens.
In Parallel MIMD Computation, Janusz Kowalik has brought together a wide
ranging group of computer scientists and physicists whose recent research shares
an important common thread. Each has been involved in the development or the
pioneering use of the HEPsupercomputer and has participated in the design of the
software tools and programming techniques that have become part of the foun
dation of parallel computation. Basedon a radical new model of parallel execu
tion, the HEP has stimulated a substantial amount of new research in parallel
algorithm and software design. In particular,this machine hasprovided researchers
with their first large-scale MIMD computer. This architectural feature opened a
new dimension for experimental algorithm design. Although this book isa collec
tion of original contributions, it succeedsin presenting a remarkably coherent view
of the potential of the HEPmodel of computation.
Dennis B. Gannon
PREFACE
The demand for more powerful computers israpidly growing and can only be met
by a radical change in computer architecture. The demand for massiveamounts of
computing powerarisesfromtwo broad classesofproblems:
a. problems requiring numerical computation, such as aerodynamics, computer
graphics,weatherforecasting, oiIreservoirsimulation,and
b. problems requiring symbolic processing or a combination of symbolic and
numeric computation, suchasexpertsystems,computer vision, robotics, natural
languageunderstandingand otherapplicationsofartificialintelligence.
The first class of problems is historically earlier and has attracted much more
attention from the computer designers and vendors. Almost all of the
supercomputers representing the state-of-the-art in computing machinery have
been designed primarily for numerical computing. The main architectural feature
of the most successful commercial high-performance machines is arithmetic
pipeliningthatallowsconcurrentprocessing ofoverlappedoperations. Thesesuper
computers are very effective for applications that have a high percentage of
vectorisable operations. They are notsuitablefor problems dominated by scalar or
non-numerical computation. The pipelined supercomputers such as CRAY-l and
CYBER 205 can be considered asrelatively modest extensions of the conventional
single-processor mainframes. To increase computer performance bythefactorsofa
hundred or athousand we mustlookto machineswith multiple processors,capable
oftrulyparallel execution.
Currentdesign effortsin parallel architectureindicatethreepotential directions:
1. array processorsworking in lock-step,
2. multiprocessorswhich typicallyshareresourcesandworkasynchronously,
3. massively parallel aggregates of processors that have either a flexible
configurationoraspecialized configuration tosolve limited classesofproblems.
Inthe second categorywe can identifyavery general class ofmachines capable of
processing multiple streams of instruction on multiple streams of data (MIMD). So
faronly afew MIMDcomputers have been built and used. In the United States the
best known MIMD computers are: CRAY X-MP, CRAY-2, 5-1, and DENELCOR
HeterogeneousElementProcessor(HEP).
The MIMD computer architecture employs tightly coupled high-performance
processorswhose numbertypicallycanvary from 2to 16. The questionofextending
these systems to a much higher number of processors is still open. Using many
processors to solve one partitionable problem would require small granularity of
the computationally independent subtasks and this implies extensive
communication between processors and memory banks. This need for
communication may siqnifically reduce systems high-performance. In this regard
the HEP multiprocessoroffers remarkable originality in design. The computational
synchronization isimplemented in hardwarewith alowoverhead allowing efficient
usage of small granularity. Also, the memory latency which can slow-down any
multiprocesorsystemisovercome bythe useofpipelining. Acomputational process
waiting for data isqueued up until its memory access issatisfied, thus permitting
other processesto execute. These two important features of HEPsuggest that the
HEP architecture may be extendable and also be a basis for highly efficient and
powerful future MIMDmachines.
The purpose ofthis book ISto introduce MIMD computing to awider audience of
researchers and potential users. This volume is a collection of several research
papers on the MIMD computation and the HEP machine which have been
contributed bythe HEPdesignerand by scientists who have experimented with this
system. Mostofthese computational experiments are related to numerical analysis
type problems. Can HEP also be used efficiently for nonnumeric problems, for
example, logic programming and symbolic processing? This is certainly a
challenging question whose answer (particularily if the answer is: yes)would be of
interest to the fast growing community of developers and users of the Artificial
Intelligencetechnology.
We have found the HEPmultiprocessorto be an excellenttool for experimenting
with the MIMD computation and programming. Being an innovative general
purpose machine and the first commercial MIMD machine in the world, the HEP
system has provided uswith a timely opportunity to getseriously acquainted with
parallel computing and haveaglimpseofthe futurecomputation.