European Scientific Infrastructure Learn from the CERN experience?? The Mission of CERN (1954): The Organization shall provide for collaboration among European States in nuclear research of a pure scientific and fundamental character, and in research essentially related thereto. 20 Member States (2000) The organization shall have no concern with work for military requirements and the results of its experimental and theoretical work shall be published or otherwise made generally available. 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 1
LEP in Year 2000 STANDARD MODEL HIGGS BOSON + e + e Z +... (with b tag) m H > 113.3 GeV/c 2 at 95%C.L. (20 July 2000) 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 2
CERN, evolution of resources MCHF, 2000 prices CERN paid (2750 staff, 800 fellows, paid associates) Unpaid visiting scientists - Users 7000 6000 5000 4000 3000 2000 1000 0 1400 1200 1000 800 600 400 Year 1956 CERN Paid Total Users Expenditure at 2000 prices 2M&BEBC 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 ISR 300 GeV ppbar Energie Exploitation Personnel 1982 1984 1986 1988 1990 1992 1994 1996 1998 LEP 1 LEP 2 LHC 1400 1200 1000 800 600 400 900 800 700 600 500 400 300 200 100 0 1400 1200 1000 800 600 400 200 Age distribution >5500 visitors, 2750 staff, >1200/year turnover (3.99) Excellent education 20 25 30 35 40 45 50 55 60 65 70 75 80 Contributions CONTRIBUTIONS TO THE CERN PROGRAMMES, (MCHF, 2000 PRICES) Staff Users 200 SC PS 200 0 0 0 52-54 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 Years 1980 1982 1984 1986 1988 1990 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 3 1992 1994 1996 1998 2000
Education ~1200 people rotating/year Example LEP-Opal Collaboration 0 900 800 700 600 500 400 300 200 100 20 25 30 35 40 45 50 55 60 65 70 75 80 Staff Users 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 4
1954 2000 CMS Alice LHCb Atlas 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 5 PS aéroport Genève
Evolution of CERN Particle Accelerators PS, 1959, ~1 GeV (cm) ISR, 1971, ~9 GeV (cm) SPS, 1976,~4.5 GeV (cm) LEP/LHC SPS Isolde p ACOL p (antiproton) p (proton) ion e + (positon) e (électron) Booster PS EPA LIL e + e linacs Spp bar S 1981 ~90 GeV (cm) LEP, 1989, 80-209(?) GeV (cm) LHC, 2005, ~2'000 GeV (cm) ("cm" center of mass of partons) Linacs protons et ions LEAR s.c.cavity 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 6
40 Years of Particle Detectors 81 cm Bubble Chamber at PS 1962-68 Big European Bubble Chamber At PS and SPS 1974-1985 Charm Search at ISR 1975-1980 UA1 at Spp bar S 1981-1989 ALEPH at LEP 1989-2000 ATLAS at LHC, 2005-2020 150*10 6 sensors; 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 7
16-Sep-00 H. F. Hoffmann, CERN-DG/DI 8
Experimental Apparatus BEBC photo, v-beam, ~30 tracks, semi-automatic scanning, very sophisticated tracking and analysis codes-->computer literacy resolution ~ 100 µ, < 1 event/sec, 20m 3 liquid superheated hydrogen, 2 Tesla S.C. magnet 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 9
Experimental Apparatus, continued LHC collisions, 10 9 events/s Complexity of data: ~250SPECint95 * sec/event 1 / 10 13 selectivity Pixel detector 50*100µ/pixel, 140million channels General Purpose detector, Multiple, simultaneous detection modes, OO programming, millions of lines of code 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 10 Basic building block in full pixel readout chip : 8 pixels/12 000 transistors in 400 by 425 mm 2
300 ATLAS Collaboration 250 200 150 100 50 0 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 11 Austria Czech Republic Denmark Finland France Germany Greece Italy Netherlands Norway Poland Portugal Slovac Republic Spain Sweden Switzerland United Kingdom CERN Armenia Australia Azerbaijan Republic Republic of Belarus Brazil Canada China PR Republic of Georgia Israel Japan Morocco Romania Russia JINR Dubna Slovenia Taiwan Turkey 1800 physicists,150 institutes; 35 countries R&D, proposal, design, reviews, approval: 1988-1996 Construction, installation 1996-2005, operation 2005-2020 Material Cost 300 M Euro, CERN part:20% United States
Some typical features of such collaborations Open, global collaboration of critical mass, able to deal with all problems posed, together with CERN and collaborating institutes "Lean, bottom-up" self-organisation; success based on experienced collaborators, eager young people, common goals and competition MoU, best intentions but not legally binding Free choice of collaborating institutes to participate -or not Clear common long-term mission, clear objectives, Free exchange of ideas, technologies, R&D results Often best people in the field of interest External peer reviews; elaborate internal reviews and QA Good record of achievements in terms of delivery to specs, schedules, budgets 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 12
CERN's Network in the World 267 institutes in Europe, 4603 users 208 institutes elsewhere, 1632 users some points = several institutes 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 13
Virtual Room Videoconferencing System 3267 machines reg. 1994 people in 52 countries 182 institutes http://vrvs.cern.ch/ Bandwidth >256 Kbps --> >10 frames/sec Virtual Rooms Concept Enter a Virtual Room Through Your Nearest Reflector 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 14 9
Trends Better resolution,smaller cross-sections --> higher energy, intensity accelerators, more complex, selective, multi-purpose experiments Each generation of accelerators -->another "cultural revolution" for accelerator staff and experimentalists Experimental "facilities", small specialised experiments --> self-organised, bottom-up "virtual, big science laboratories" More elaborate peer reviews (scientific excellence!!),more project management European --> Global Organised interdisciplinary TT with excellent partners 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 15
From Particles to Petabytes: Challenges in High Throughput Computing Example: (Data-) Grid, a global Science Infrastructure 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 16
On-line System Multi-level trigger Filter out less interesting Reduce data volume 24 x 7 operation 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 17 Data Recording & Offline Analysis Level 1 - Special Hardware Level 2 - Embedded Processors Level 3 Farm of commodity CPU 75 KHz (75 GB/sec)fully digitised (75 GB/sec)fully digitised 100 Hz (100 MB/sec) (100 MB/sec) Digital telephone 1-2 KB/sec 40 MHz (1000 TB/sec) equivalent) (1000 TB/sec) equivalent) 5 KHz (5 GB/sec) (5 GB/sec)
How Much Data is Involved? Level 1 Rate (Hz) 10 6 1 billion people 10 5 surfing the Web 10 4 10 3 10 2 High Level-1 Trigger (1 MHz) LHCB KLOE HERA-B H1 ZEUS UA1 LEP CDF ATLAS CMS CDF II NA49 10 4 10 5 10 6 ALICE High No. Channels High Bandwidth (500 Gbit/s) High Data Archive (PetaByte) 10 7 Event Size (bytes) 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 18
LHC Will Accumulate Multi-Petabytes TeraBytes Long Term Tape Storage Estimates 14'000 12'000 10'000 8'000 6'000 LHC 4'000 2'000 0 Current Experiments COMPASS 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 Year 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 19
Complex Queries = More CPU Per Byte Thousands of SPECint 95 1'000 900 800 700 600 500 400 300 200 100 0 Evolution of Computing Capacity CERN 1999: 3.5K SI95 900 CPUs LHC COMPASS Others? 1997 1998 1999 2000 2001 2002 2003 2004 2005 Year 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 20
CERN Computer Center Today --> Commodity 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 21
CERN Computer Center Today No longer aligned with supercomputing philosophies Require many small independent problem solutions High Throughput Computing processing click-like interactions in parallel A marriage of supercomputer storage systems with supermarket commodity CPU Disk access layer (hw+sw) sandwiched between Middle-ware on network layer important Therefore, our natural affinity has shifted from supercomputers towards ISPs, e-commerce and data marketers 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 22
Grids: Next Generation Web Web: Uniform access to HTML documents http:// http:// Grid: Flexible, highperformance access to all significant resources Software catalogs Colleagues On-demand creation of powerful virtual computing and data systems Computers Sensor nets Data Stores Web-sites 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 23
0 Grids: Next Generation Web Web: Uniform access to HTML documents http:// http:// Grid: Flexible, highperformance access to all significant resources Software catalogs 300 250 200 150 100 50 Colleagues Computers Sensor nets Data Stores On-demand creation of powerful virtual computing and data systems 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 24 Austria Czech Republic Denmark Finland France Germany Greece Italy Netherlands Norway Poland Portugal Slovac Republic Spain Sweden Switzerland United Kingdom CERN Armenia Australia Azerbaijan Republic Republic of Belarus Brazil Canada China PR Republic of Georgia Israel Japan Morocco Romania Russia JINR Dubna Slovenia Taiwan Turkey United States
LHC Vision: Data Grid Hierarchy Bunch crossing per 25 nsecs; 100 triggers per second. Event is ~1 MByte in size Experiment Tier 1 ~PByte/sec Online System FNAL Center Italy Center UK Center Tier 3 Physics data cache ~0.6-2.5 Gbits/sec ~622 Mbits/sec Workstations Institute Institute ~0.25TIPS Institute 100-1000 Mbits/sec Tier 0 +1 Tier 2 Tier 4 Institute ~100 MBytes/sec Offline Farm, CERN Computer Ctr > 20 TIPS German Centre ~2.5 Gbits/sec Tier2 Center Tier2 Center Tier2 Center Tier2 Center Tier2 Center Physicists work on analysis channels Each institute has ~10 physicists working on one or more channels 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 25
The Grid Middleware Services Concept Standard services that Provide uniform, high-level access to a wide range of resources (including networks) Address interdomain issues: security, policy Permit application-level management and monitoring of end-to-end performance Broadly deployed, like Internet Protocols Enabler of application-specific tools as well as applications themselves 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 26
EU-Grid Project Work Packages Work Package Number Work Package title WP1 Grid Workload Management INFN Lead contractor WP2 Grid Data Management CERN WP3 Grid Monitoring Services PPARC WP4 Fabric Management CERN WP5 Mass Storage Management PPARC WP6 Integration Testbed CNRS WP7 Network Services CNRS WP8 High Energy Physics Applications CERN WP9 Earth Observation Science Applications ESA WP10 Biology Science Applications CNRS WP11 Dissemination and Exploitation INFN WP12 Project Management CERN 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 27
Participants Main partners: CERN, INFN(I), CNRS(F), PPARC(UK), NIKHEF(NL), ESA-Earth Observation Other sciences: Earth Observation, Biology, Medicine Industrial participation: CS SI/F, DataMat/I, IBM/UK Associated partners: Czech Republic, Finland, Germany, Hungary, Spain, Sweden (mostly computer scientists) Work with US: Underway; Two Particle Physics Data-Grid projects, several other grid projects; Formal collaboration being established Industry and Research Project Forum with representatives from: Denmark, Greece, Israel, Japan, Norway, Poland, Portugal, Russia, Switzerland Many other interested sciences in many European Countries 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 28
GEANT, necessary infrastructure Minimum bandwidth of 2.5 Gbps between core nodes, possibility of starting with some 10Gbps (STM- 64/OC-768c) circuits is not excluded. Connection to other World Regions in principle via core nodes only, They will, together, form a European Distributed Access (EDA) point conceptually similar to the STAR TAP. 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 29
GRIDs In 2000: Summary Grids are already changing the way we do science and engineering --> e-science Key services and concepts have been identified, and development has started Transition of services and applications to production use is about to occur in significant tests In future more sophisticated integrated services and tool-sets could drive advances in many fields of science and engineering High Energy Physics, facing the need for Petascale Data, is an early adopter and leading Data Grid developer Hope to be approved on 25.10.00 Further applications should be possible in 2001!!! 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 30
The Web, a historical case study Invented at CERN in 1989 as application layer on top of the internet infrastructure Development started in Europe (small) and US (big) 80% of the most visited sites: US, <10% Europe Web Site Servers 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 31
Why Science, why European Research Area &FP5 Opens new possibilities: Creates tools to exploit them: Aids their efficient utilisation: Brings nations together, science technology economics acts globally,.. : science The scientific method plays a more general role: question received wisdom, formulate hypotheses, devise tests, conduct experiments, observe their results, draw conclusions, understand the present, predict the future 16-Sep-00 H. F. Hoffmann, CERN-DG/DI 32