igh erformance omputing

Transcription

1 igh erformance omputing 2012/11/19 Xavier Vigouroux Business Development Manager 1 The views expressed are those of the author and do not reflect the official policy or position of Bull

2 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull 2

3 Yes! It s worth to invest 3

4 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull 4

5 1996 PhD The views expressed are those of the author and do not reflect the official policy or position of Bull Who am I 5

6 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull 6

7 Bull: European leader in mission-critical digital systems recognized worldwide in secure systems OPERATING IN 50 COUNTRIES 1.3bn REVENUES Efforts in research in % +29% growth in profitability in 1 st quarter ,6% growth in

8 The views expressed are those of the author and do not reflect the official policy or position of Bull 8

9 How large is a machine? top500. m. Linpack Ax = b Flops/s 9

10 Top Tflops/s Tflops/s PowerPC cores Tflops/s Tflops/s cores AMD cores GPU Tflops/s Tflops/s sparc cores 10

11 TOP500 1,E+09 1,E+08 1,E+07 1,E+06 1,E+05 1,E+04 1,E+03 1,E+02 1,E+01 1,E Rpeak Rmax 11

12 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull 3 petaflop-scale systems TERA 100 CURIE IFERC 1.25 PetaFlops Xeon cores 256 TB memory 30 PB disk storage 500 GB/s IO throughput 580 m² footprint 2 PetaFlops Xeon cores GPU cores 360 TB memory 10 PB disk storage 250 GB/s IO throughput 200 m² footprint 1,5 PetaFlops Xeon cores 280 TB memory 15 PB disk storage 120 GB/s IO throughput 200 m² footprint 12

13 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull 3 large GPU based systems #7 green500 TERA 100 GPU-based extension 198 bullx B505 accelerator blades 396 NVIDIA Tesla M2090 GPU processors 202,752 GPU cores CURIE GPU-based extension 144 bullx B505 accelerator blades 288 NVIDIA Tesla M2090 GPU processors 147,456 GPU cores BSC GPU-based system 126 bullx B505 accelerator blades 252 NVIDIA Tesla M2090 GPU processors 129,024 GPU cores 13

14 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull Source: list nov 12 14

15 The views expressed are those of the author and do not reflect the official policy or position of Bull 15

16 Load the supercomputer 16

17 Ramp up the users Tier 0 1 Petaflop/s cores European, PRACE TGCC, IFERC Tier Teraflop/s 5000 cores National Meso Centres 10 Teraflop/s 500 cores Regional 17

18 Cooling 12kW/rack 6 C 30KW 30 C 80KW 18

19 Cooling Water has closed as possible to the heat source Water can be hotter (as delta T is key) Room can be hotter (remove CRAC) But, maintainability is key! No change in maintenance process CPU can be changed, DIMM can be changed Blades can be removed 30 C 80KW 19

20 % The views expressed are those of the author and do not reflect the official policy or position of Bull Power Feeding 15% 10% PSU Efficiency (240V) Load (V) 20

21 Efficient Hardware 2 x CPUs 2x GPUs/Xeon Phis 21

22 J/s Consumed Energy (J) The views expressed are those of the author and do not reflect the official policy or position of Bull Bull - Energy aware batch scheduler Fix the CPU frequency $# srun --cpu-freq= resv-ports -N2 -n64./cg.c.64& $# sacct -j 58 -format=jobid,elapsed,avecpufreq,consumedenergy JobID Elapsed AveCPUFreq ConsumedEnergy :00: Effective CPU Frequency Job Power consumption freq Duration (s) 22

23 Use efficient Library Courtesyof Jack DONGARRA MAGAM + starpui 23

24 The views expressed are those of the author and do not reflect the official policy or position of Bull 24

25 2 Key elements Computing the function Managing Data Data function Results 25

26 Data Movement and Floating point FPS-164 and VAX (1976) 11 Mflop/s; transfer rate 44 MB/s Ratio of flops to bytes of data movement: 1 flop per 4 bytes transferred Nvidia Fermi and PCI-X to host 500 Gflop/s; transfer rate 8 GB/s Ratio of flops to bytes of data movement: 62 flops per 1 byte transferred Flop/s are cheap, so are provisioned in excess 26

27 [DATA] Non Volatile RAM Source: ITRS 27

28 [DATA] 3D Wioming from CEA leti Wide IO Memory Interface Next Generation Courtesy of CEA leti 28

29 [DATA] Hybrid Memory Cube Wide I/O standard comes from mobile world True «3D» (with Through Silicon Vias») Bandwidth Energy DDR-3 10,66 GB/s 50-70pJ/bit HMC 128GB/s 8 pj/bit Source:

30 [DATA] Photonics Photonic is getting closer to the CPU Basic blocks are in place: Modulator Multiplexer Routing Receiver Off chip Energy cost very close to On chip Energy cost To go deeper: Silicon Photonics for Exascale Computing Keren Bergman, Columbia universisty Lightwave Research Laboratory 30

31 [FUNCTION] Near threshold Voltage Drawback: surface is increasing 31

32 Frequency (MHz) The views expressed are those of the author and do not reflect the official policy or position of Bull [FUNCTION] CPU frequency ? ? 0 Source: top500 - mean freq 32

33 Technology node size (nm) The views expressed are those of the author and do not reflect the official policy or position of Bull [FUNCTION] Mask reduction

34 The views expressed are those of the author and do not reflect the official policy or position of Bull 34

35 TOP500 1,E+09 1,E+08 1,E+07 1,E+06 1,E+05 1,E+04 1,E+03 1,E+02 1,E+01 1,E Rpeak Extrap Peak Rmax Extrap Max 35

36 Is it a long way to Exascale Jun 97 Nov 08 Nov 12 Jun 19 asci red roadrunner Titan Eflop Sandia ORNL ORNL??? MW 0,50 2,48 8,3 20 Eflop max 1,45E-06 0,001 0,027 1 nodes racks U/rack Concur M O(10 9 ) Year factor Year factor Year factor Gflop/W 2,91E-03 1,549 0,4 1,651 3,3 1,519 50,0 Tflop/node 2,00E-04 1,798 0,2 1,708 1,5 1,441 15,6 W/node 68,8 1, ,3 1, ,3 0, ,5 nodes/rack 69,8 0,904 21,9 1,437 93,4 1, ,3 W/rack 4807,7 1, ,1 1, ,0 1, ,7 Tflop/U 4,66E-04 1,579 0,09 2,455 3,23 1,637 79,37 W/U 160,3 1, ,8 1, ,3 1, ,3 36

37 Technology node size (nm) The views expressed are those of the author and do not reflect the official policy or position of Bull Node evolution : «à la Phi» Gflop/W 50,0 Tflop/node 15,6 1-2 Phi W/node 312,5 nodes/rack 213,3 Tflop/U 79,37 10 Phi W/U 1587,3 60 => ~500 1 => ~8 8nm With µarch improvement, it should be less! Phi power consumption will have to be around 150W-300W Between 200 and 400 Phi per Rack 37

38 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull [#freq] x [#flop/cycle] x [#cores] 1050 MHz 700 MHz

39 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull Gene Amdahl John Gustafson 39

40 Stack Application Libraries Runtime Supercomputer Data Center Hurry UP Speed UP Keep the Pace 40

41 Strong Integration Ressource Manager Data exchange Library Code Resiliency Topology See Fastforward approach See MAGMA [UTK] with StarPU [INRIA] 41

42 Supercomputer Suite COMPLETE OPEN FLEXIBLE All functionalities o Cluster management o Development factory o Execution environment o Data storage and access All sizes o From department o to Top 10 Best of breed o Linux, o OpenMPI, o HPC Toolkit, o Nagios, o OFED, o Slurm, o Lustre, o Shine,... + Bull added value Modular: Get what you need when you need INTEGRATED Installed, deployed and operated as a single software 42

43 Applications and Performances team Electro-Magnetics Computational Chemistry Quantum Mechanics Computational Chemistry Molecular Dynamics Computational Biology Structural Mechanics Implicit Structural Mechanics Explicit One Bull expert (PhD) for each segment Seismic Processing Computational Fluid Dynamics Reservoir Simulation Rendering / Ray Tracing Climate / Weather Ocean Simulation Data Analytics 43

44 44

45 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull 45

46 The views expressed The views are expressed those of are the those author of the author and and do do not not reflect the official policy policy or position or position of Bull of Bull 46

47 For any question 47