Compiling for Multi, Many, and Anycore. Rudi Eigenmann Purdue University

Transcription

1 Compiling for Multi, Many, and Anycore Rudi Eigenmann Purdue University

2 Compilation Issues in Multi, Many, Anycore and sample contributions Multicore: shared-memory issues revisited => Cetus: parallelizing compiler for multicore Manycore: message-passing programming? => Shared-memory model for message-passing architectures Anycore: heterogeneity will increase complexity by an order of magnitude => Moving compile-time decisions into runtime R. Eigenmann, Compiling for Multi, Many, Anycore 2

3 Compiler Infrastructure for Multicore Compilation Cetus Successor to Polaris Source-to-source translator for C, C++, Java Supported by the U.S. National Science Foundation Written in Java cetus.ecn.purdue.edu R. Eigenmann, Compiling for Multi, Many, Anycore 3

4 Manycore: Shared-memory model for message-passing architectures Translating OpenMP to MPI

5 OpenMP for Message-Passing Architectures 25 2 Our OpenMP to MPI translator Hand-coded MPI Speedup CG EP FT LU IS ART EQUAKE From NAS Translated Par. Benchmarks OpenMP Hand-coded MPI From SPEC OMP21 R. Eigenmann, Compiling for Multi, Many, Anycore 5

6 OpenMP for Message-Passing Architectures: Comparison with High-Peformance Fortran Execution Time (secs) CG (CLASS B) on Ethernet LU (CLASS A) on Ethernet Number of Processors MPI Execution Time OpenMP Execution Time HPF Execution Time MPI Scalability OpenMP Scalability HPF Scalability Number of Processors Execution Time (secs) Scalability A good case for HPF Scalability A bad case for HPF MPI Execution Time OpenMP Execution Time HPF Execution Time MPI Scalability OpenMP Scalability HPF Scalability R. Eigenmann, Compiling for Multi, Many, Anycore 6

7 OpenMP for Message-Passing Architectures: How does it Work? All shared data is conceptually replicated. Iterations are statically scheduled onto processors. For j=1,n a[e1]= RHS End for M P M P M P RHS data are local reads M P For j=1,n =a[e2] End for Compiler sends data produced (a[e1]) to all future consumer processors. RHS data (a[e2]) are local reads, again R. Eigenmann, Compiling for Multi, Many, Anycore 7

8 Anycore: Moving Compile-time Decisions into Runtime R. Eigenmann, Compiling for Multi, Many, Anycore 8

9 Is there Potential? You bet! Imagine you (the compiler) had full knowledge of input data and execution platform of the program Performance 1 1 You are here Amdahl s law of dynamic optimization 1 1% knowledge R. Eigenmann, Compiling for Multi, Many, Anycore 9

10 Early Results Relative performance improvement percentage (%) ammp applu apsi Whole-program tuning train data set Subroutine-level tuning train data set Whole-program tuning ref data set Subroutine-level tuning ref data set art equake mesa Tuning Goal: determine the best combination of GCC options mgrid R. Eigenmann, Compiling for Multi, Many, Anycore 1 sixtrack swim wupwise geometric mean

11 Reduction of Tuning Time through Subroutine-level Tuning Whole-program tuning Subroutine-level tuning 12. Normalized tuning time ammp applu apsi art equake mesa mgrid sixtrack swim wupwise GeoMean R. Eigenmann, Compiling for Multi, Many, Anycore 11

12 Ongoing Work Beyond autotuning of compiler options New applications of the tuning system MPI parameter tuning Tuning library selection - (ScalaPack,...) OpenMP to MPI translator Best allocation of computation on host vs. GPGPU R. Eigenmann, Compiling for Multi, Many, Anycore 12

13 Runtime Work Distribution on GPU vs. Host Speedups over CPU version Speedups over CPU version Speeuup CPU GPU Tuned Speedup CPU GPU Tuned Dimension size (WA = HA = WB) Dimension size (WA = HA = WB) Arbitrary data dimensions Data dimensions optimal for GPU R. Eigenmann, Compiling for Multi, Many, Anycore 13

14 Conclusions Compilation issues for (today s) multicores: same as for SMPs but has the importance changed? Manycore, anycore: We need high-level programming models E.g., shared-memory models advanced optimizations E.g., runtime decision support R. Eigenmann, Compiling for Multi, Many, Anycore 14