SimMillennium99

SimMillennium:Computer Systems, Computational Science and Engineering in the Large: 

SimMillennium: Computer Systems, Computational Science and Engineering in the Large Jim Demmel, David Culler E. Brewer, J. Canny, A. Joseph, J. Landay, S. McCanne A. Neureuther, C. Papadimitrou, C. Sequin, K. Yelick EECS, UC Berkeley www.millennium.berkeley.edu NSF CISE EIA RI and MII PI’s Workshop Aug 7-9 1999

Project Goals: 

Project Goals Enable major advances in Computational Science and Engineering Simulation, Modeling, and Information Processing becoming ubiquitous Many participants outside CS Explore novel design techniques for large, complex systems Fundamental Computer Science problems ahead are problems of scale Use Capitalism, not Socialism (i.e. not Computer Center) Develop fundamentally better ways of assimilating and interacting with large volumes of information and with each other Explore emerging technologies networking, OS, devices

Outline: 

Outline Background on UC Berkeley Millennium infrastructure description Other infrastructure contributions Systems research Networking Computational Economy Applications List of all participants A few highlights Conclusions

Background at UC Berkeley: 

Background at UC Berkeley Mammoth NSF RI (1988-1993) CM-5 Titan NSF RI (1994-1999) Culler, spoke yesterday NOW = Network of Workstations Curriculum CS 267 - Applications of Parallel Computing MS in Comp Sci & Eng Curriculum Proposed Comp Eng Sci undergrad program NERSC at LBNL National Energy Research Scientific Computing Center Supercomputer center next to campus SimMillennium (1998-2003)

Planned Millennium Infrastructure: 

Planned Millennium Infrastructure

The Community: 

The Community School of Info. Mgmt and Sys. Computer Science Electrical Eng. Mechanical Eng. BMRC Nuclear Eng. IEOR Civil Eng. Inst. Of Transport Business Chemistry Astro Physics Biology Economy Math Geo

NT Workstations for Sci. & Eng. : 

NT Workstations for Sci. & Eng. SIMS C.S. E.E. M.E. BMRC N.E. IEOR Civil Eng Transport Business Chemistry Astro Physics Biology Economy Math Geo

SMP => storage, small-scale parallelism: 

SMP => storage, small-scale parallelism SIMS C.S. E.E. M.E. BMRC N.E. IEOR Civil Eng Transport Business Chemistry Astro Physics Biology Economy Math Geo

Group Cluster of SMPs => Parallelism: 

Group Cluster of SMPs => Parallelism SIMS C.S. E.E. M.E. BMRC N.E. IEOR Civil Eng NERSC Transport Business Chemistry Astro Physics Biology Economy Math Geo

Campus Cluster => large-scale Parallelism: 

Campus Cluster => large-scale Parallelism SIMS C.S. E.E. M.E. BMRC N.E. IEOR Civil Eng NERSC Transport Business Chemistry Astro Physics Biology Economy Math Geo

Gigabit Ethernet Connectivity: 

Gigabit Ethernet Connectivity Gigabit Ethernet SIMS C.S. E.E. M.E. BMRC N.E. IEOR Civil Eng NERSC Transport Business Chemistry Astro Physics Biology Economy Math Geo

Physical Connectivity: 

Physical Connectivity

Visualization and Novel User Interfaces: 

Visualization and Novel User Interfaces

Current Infrastructure (by end of 8/99): 

Current Infrastructure (by end of 8/99) All 195 desktops and 20 SMPs delivered All 18 16-(or smaller)-processor clusters (8 dual SMPs) Mostly running Myrinet interconnects, some 100Mb Ethernet NOW functioning as large central cluster Cluster build service Millennium wide services .5 Tbyte file server distributed system services and software for UNIX and NT Optical fiber for Gigabit in place 10 Gigabit switches purchased 2 Vision Maker Digital Desks purchased

A Millennium Cluster: 

A Millennium Cluster 16x2 Processor 400 MHz Pentium II 100 MHz Memory Bus 33 MHz 32-Bit PCI 100BaseTX Ethernet Myrinet M2F Windows NT 4.0 or LINUX Terminal Server Edition

Industrial / Academic Collaboration: 

Industrial / Academic Collaboration Computers via Intel Technology 2000 grant 200 NT desktops 16 department 4-way SMPs 10+ 8x2 Group Clusters, 1 ~200x2 Campus Cluster PPro => Pentium II => Pentium III Additional storage via IBM SUR grant 0.5 TB this year => 4 TB NT tools via Microsoft grant Solaris x86 tools via SMCC grant Nortel discounts the gigabit Ethernet 70% Campus provides 3 technical staff, fiber Research provides the prog. and system support 200 Gflop/s 150 GB memory 8 TB disk

What NSF is paying for: 

What NSF is paying for Fast internal networks for clusters Gigabit ethernet switches Interesting I/O devices Large displays 3D glasses Haptic mice Position sensors One staff person

Primary Faculty Participants - 1: 

Primary Faculty Participants - 1 CS D. Culler, J. Demmel, E. Brewer, J. Canny, A. Joseph, R. Katz, J. Landay, S. McCanne, C. Papadimitriou, C. Sequin, R. Wilensky, K. Yelick Systems, Numerical Methods, Services, HCI, Networking, Computational Economics, Digital libraries, Parallel languages EE A. Neureuther Technology CAD for EBEAM Lithography Civil Engineering S. Govindjee, G. Fenves Earthquake Engineering, Finite Element Modeling Physics B. Price, J. Wurtele, D. Lowder Processing neutrinos and muons at South Pole SIMS H. Varian, R. Larson, M. Hearst Computational Economics, User Interfaces

Primary Faculty Participants - 2: 

Primary Faculty Participants - 2 Astronomy J. Arons, C. McKee. P. Marcus Star Formation, Geophysical Turbulence Transportation Studies S. Sastry, A. Kanafani Redesign of Nation’s Air Traffic Control System

Secondary Faculty Participants - 1 : 

Secondary Faculty Participants - 1 Geology/Geophysics M. Richards, D. Dreger Mantle modeling Math D. Eisenbud, B. Poonen, A. Grunbaum, T. Slaman, B. Sturmfels, P. Vojta Crystal growth modeling, tomography, symbolic computing Berkeley Multimedia Research Center L. Rowe Video effects processing Mechanical Engineering V. Carey, M. Frenklach, A. Packard, P. Papadopoulos, P. Marcus Modeling Automated Highways, Material Processing Biology D. Lindberg, S. Brenner Reconstruct Phylogenetic Tree of Life, Genome studies Nuclear Engineering J. Vujic Planning radiotherapy for Brain Tumors

Secondary Faculty Participants - 2: 

Secondary Faculty Participants - 2 NERSC W. Saphir Business N. Hakansson Computational Finance Laboratory Chemistry K. Durkin, D. Chandler, D. Harris, W. Lester, W. Miller, R. Stevens, B. Whaley Computational Chemistry, Molecular Dynamics Visualization Economics A. Nevo Market Modeling

The CS Research Agenda: 

The CS Research Agenda High Performance Cluster Computing Environment Fast communication on Clusters of SMPs Compiler Techniques for Performance and Ease of use Numerical Techniques and Solvers Particles, FFT, AMR, Multigrid, Sparse and Dense Lin. Alg. Novel System Design Techniques Clusters of clusters Computational Economy Open infrastructure services Novel modes of interacting with large amounts of data User interfaces, Digital Libraries

Communication Interface Revolution: 

Communication Interface Revolution Low Overhead Communication “Happens” Academic Research put it on the map Active Messages (AM), FM, PM, … Unet Memory Messaging (Get/Put, Reflective, VMMC, Mem. Chan.) Intel / Microsoft / Compaq recognized it Virtual Interface Architecture 1.0 released 12/16/97 Berkeley VIA over Myrinet released on NT and Linux

World-Record Datamation Sort: 

World-Record Datamation Sort Old Record (NOW)

Computational Economy Approach: 

Computational Economy Approach System has a supply of various resources Demand on resources revealed in price distinct from the cost of acquiring the resources User has unique assessment of value Client agent negotiates for system resources on user’s behalf submits requests, receives bids or participates in auctions selects resources of highest value at least cost

Advantages of the Approach: 

Advantages of the Approach Decentralized load balancing according to user’s perception of what is important, not system’s own metric adapts to system and workload changes Creates incentive to adopt efficient modes of use exploit under-utilized resources maximize flexibility (e.g., migratable, restartable applications) Establishes user-to-user feedback on resource usage basis for exchange rate across resources Powerful framework for system design Natural for client to be watchful, proactive, and wary Generalizes from resources to services Rich body of theory ready for application

Current Prototype: 

Current Prototype Specify #procs p and value on job rexec -n 16 -value 20 fft.mpi Market-based Proportional Sharing Bidder i gets fraction bi / Sk bk of resource If one bidder, no cost to use resource Resource may be CPU, Memory, Network, I/O Existing OS mechanisms/policies insufficient New proportional CPU schedule for LINUX New page replacement policy Game theoretic analysis Preliminary experience in CS267 Students trusted system to allocate fairly, so they did not try to flood system with jobs Future work other mechanisms and analysis (Vikrey auction, batch vs interactive) package up and market services (make, popular simulators, DB search)

Application Highlights: 

Application Highlights PEER - NSF Earthquake Engineering Center FE modeling of Bay Area during Big One Need better parallel sparse linear system solvers EBEAM - Electron Beam Lithography Simulate next generation chip manufacturing Need better parallel N-body force calculation AMANDA Antarctic Muon and Neutrino Detector Array Need to process many detector tracks for events Web Page Design Better human interfaces using novel devices Digital Library Support access to large active document collection

Earthquake Modeling: 

Earthquake Modeling PEER = Pacific Earthquake Engineering Research Center UC, Caltech, Stanford, USC, U Washington Model behavior of civil infrastructure in Big One Improve earthquake resistant designs Requires large scale FE models Buildings, roads, bridges, etc. coupled to ground Simulate effects of earthquakes Requires solution of very large sparse linear systems Collaboration on software G. Fenves - OO Finite Element Modeling System J. Demmel - direct and iterative parallel linear equation solvers Prometheus - a multigrid solver for FE problems (M. Adams) SuperLU - sparse Gaussian elimination (X. Li) Port from Cray T3E to Millennium

SuperLU Scales well on Millennium: 

SuperLU Scales well on Millennium

Millennium sometimes beats a Cray T3E: 

Millennium sometimes beats a Cray T3E

EBEAM - Lithography Simulation: 

EBEAM - Lithography Simulation A. Neureuther and J. Demmel Simulate future chip manufacturing devices which will use electron beams instead of light Computational Bottleneck: computing electrostatic forces on electrons Pbody (D. Blackston) Parallel O(N) or O(N log N) N-Body code Incorporates Barnes-Hut / Fast Multipole Method / Anderson in unified framework Portable across many platforms Easy to tune for accuracy and performance Now used in production runs Will be packaged as Web service on Millennium, with other TCAD tools

Performance of Pbody: 

Performance of Pbody Over 90% efficient on 4 Millennium procs 61 secs for 200K electrons on 1 proc (vs 46 secs for Cray T3E) 500x faster than direct O(N2) method

AMANDA: 

AMANDA Antarctic Muon And Neutrino Detector Array International project to detect particles in 1 km3 of ice at South Pole 98 scientists, 15 universities, 4 countries Millennium uses Machine at South Pole for data collection, webcast, teleconference Used in PBS broadcast ``Passport to Knowledge: Live from the Pole” in 1998 linking schoolchildren in Mississippi to South Pole crew Simulation of AMANDA events and calculating optical properties of the ice

Better Interfaces for Web Site Design: 

Better Interfaces for Web Site Design J. Landay, Raecine Sapien (SUPERB student) Most web designers do not like to program, edit Provide a more natural user interface that matches their style Exploit large displays, position trackers, vision and gesture recognition to make design easier and faster Prototype (minus vision) built over summer, evaluated on a group of designers

Web Site Design Issues: 

Web Site Design Issues Taken from Contextual Design Beyer & Holtzblatt

System Components: 

System Components Physical components Vision system CrossPad Digital Desk Command Area

Second User Tests: 

Second User Tests

Digital Libraries: Rethinking Scholarly Information Dissemination and Use: 

Digital Libraries: Rethinking Scholarly Information Dissemination and Use Robert Wilensky Principal Investigator David Forsyth Co-principal Investigator The UC Berkeley Digital Library Team

Goal: Complete Rethinking of How we Use Information: 

Goal: Complete Rethinking of How we Use Information Must support entire “information cycle”: creation, dissemination and collaboration in addition to organization, access, presentation and preservation non-textual material (photos, video, maps) in addition to text-based content primary data sources, informal “publication” as well as traditional archival product radically new modes of use Scholarly information use is an especially attractive place to start.

GIS Viewer: Streetfinder example: 

GIS Viewer: Streetfinder example

GIS Viewer Example: 

GIS Viewer Example

Conclusions:What is Millennium About?: 

Conclusions: What is Millennium About? An experiment in large-scale system design Advance the state of computational science and engineering Exploring novel design techniques Exploring important new technologies NSF support essential