wilhelmson

MEAD: 

MEAD Co-Leads Sara Graves Bob Wilhelmson http://www.ncsa.uiuc.edu/expeditions/MEAD/

MEAD Meeting 9/18-19/2002: 

MEAD Meeting 9/18-19/2002

MEAD Workshop Presentations: 

MEAD Workshop Presentations Expedition Overview (Wilhelmson) Scientific Scenarios (Jewett) Ensemble Evaluation (Ramamurthy) DTF and ETF (Towns) WRF/IA64 Update (Straka) ROMS/Coupling (Haidvogel) ROMS/SMS (Schaffer/Henderson) Performance/Scheduling (Mandal/Fowler) Prophesy: Performance Analysis & Modeling (Taylor) The Model Coupling Toolkit (Jacob) WRF architecture (Michalakes) HDF5 (McGrath/Yang/Folk) NCOMMAS (Anderson) Fault Tolerant Computing (Woodward) Portal Interface (Weber) Workflow (Rossi) Data Management (Hampton) Petascale Data Quest Expedition (Hefedova/Foster) ESML/Feature Detection (Graves) D2K (Welge) ADaM (Graves) High Dimensional Machine Learning (York/Isbell/Husbands) Volume Visualization (Porter) Wall/Stereo/Visbench Visualization (Semeraro) NCL/NCAR Graphics (Don Middleton) Education (Bramer) Collaborative Science (Stevens)

Scientific Scenarios: Storm Interaction Study: 

Scientific Scenarios: Storm Interaction Study Jacksonville, IL tornado (F2) Real World Model Real World 1 km MM5, real data initialization Jewett and Wilhelmson April 1996 Tornado Outbreak

Simulation Suite: 

Simulation Suite Looking north 1 km MM5 232 simulations: storm positions This suite explores only one set of parameters defining cell interaction. Not: Cell age Cell size Cell intensity Cell rotation secondary cell position varies as shown

Visual Comparison of Pairs of Simulations: 

Visual Comparison of Pairs of Simulations Looking north 1 km MM5 2 km rainwater (fill), vorticity (contour)

Composite Visualization of Simulations: 

Composite Visualization of Simulations Looking north 1 km MM5

Hurricane Ensemble (Suite) Visualization: 

Hurricane Ensemble (Suite) Visualization

Community Components: 

Community Components WRF Model (Weather Research and Forecasting) ROMS Model (Regional Ocean Modeling System) Coupled WRF/ROMS Model D2K (Data to Knowledge) ADaM (Algorithm Development and Modeling System) Visualization Engines

MEAD Organization & Working Groups: 

MEAD Organization & Working Groups Modeling coupling and I/O performance/fault tolerance Workflow and portal interface itineraries (fixed, changeable, push and pull) user functionality (e.g. setting model parameters) Metadata and data management finding data publishing data handling file formats Ensemble services data mining and machine learning collaborative analysis/visualization Education Working groups are underlined

Metadata and Data Management: 

Metadata and Data Management

Data From a Nested Grid Storm Model Suite: 

Data From a Nested Grid Storm Model Suite All variables in a tile (full and partial tiles) – 12 or more All tiles at a given time – 16 to 64 All nests - 3 All times – 200 to 500 All runs - 100 Selected visualizations/animations - ? Selected analysis - ? Additional documentation and use published analysis, datamining results, visualizations, and documentation (e.g., a scientific notebook) 1 domain – 256 x 256 x 50 x 12 = 39 M Words All runs with 3 nests and 200 saves = 2.34 TeraWords Metadata entries = 64 x 3 x 200 x 100 = 3.8 M

Data From a Hurricane Ensemble: 

Data From a Hurricane Ensemble 600 gigabytes per storm (with a grid spacing of 20 km) In 3-5 years, most hurricane ensemble predictions will use an inner-grid resolution of 1-2 km, resulting in several tens of terabytes of data!

Data from a Single Large Simulation: 

Data from a Single Large Simulation Billion Zone Simulation 109 x 12 variables x 1000 times x 4 bytes is 48 Terabytes plus analysis, visualization, and scientific notebook

M&DM – Issues: 

M&DM – Issues How to store and access (including finding relevant info) the large data volumes (e.g. use DODS,…..) Interactive distributed (?) parallel metadata searches through and across hugh metadata databases What databases to use? Organization of runs, projects, groups, … Secure user publishing and access (more than a pipeline) including “user notebook” Replica management Movement of data chunks between sites for post pipeline services (analysis, data mining, and vis) Stable, portable middleware and M&DM related software on platforms starting with the TeraGrid

The Portal: 

The Portal

Grid Computing Environment Client: 

Grid Computing Environment Client The GCE client at SC2002 will be used to set up computational jobs, submit the jobs to be run on grid-enabled hosts, and monitor running jobs. It is designed to launch transient web services, coined “Application Manager Web Services”, as the environment for the job to be run on a grid enabled host. It will eventually allow specification of suite analysis/data mining/visualization activities based on access to data on disk.from the relevant software tools evoked via event notification. Thus, a scientist can submit a suite of simulations today on the grid and come back tomorrow to look at and share analysis/data mining/visualization results (having peacefully slept all night!).

Collaborative Analysis/Visualization: 

Collaborative Analysis/Visualization

SWOF – Visualization/Collaboration Issues: 

SWOF – Visualization/Collaboration Issues Collaborative settings Presentations One on one interactive analysis/visualization (much closer to screen) Group to group interactive analysis/visualization Collaborative decision making into portal (see driver screen) D2K / ADaM data exploration Visualization tools NCAR (vector) Graphics HVR (Porter) Visbench Vis5D/VisAD High resolution displays including the Wall Displaying animations Integration with the Earth System Grid effort which are focused on climate data