EU Astrophysics Network Overview : EU Astrophysics Network Overview Ed Seidel
Albert Einstein Institute
Principal Network Coordinator
Scientific and technological reasons for carrying out research in our field : Scientific and technological reasons for carrying out research in our field Exploring Einstein’s General Relativity
Want to develop theoretical lab to probe this fundamental theory
Fundamental theory of Physics (Gravity)
Among most complex equations of physics
Predict black holes, gravity waves, want much more
Exciting new field: Gravitational Wave Astronomy
LIGO, VIRGO, GEO, LISA, … ~ $1 Billion worldwide!
Fundamentally new information about Universe
A last major test of Einstein’s theory: do GWs exist?
A century later, both of these developments happening at the same time: very exciting coincidence!
But, the community needed to carry out theoretical work is lacking…
Not enough people or training
Slide3 : This EU Network Astrophysics
10 EU Institutions, 3 years, €1.5M
Continue these problems
Entire Community becoming Grid enabled NSF Black Hole Grand Challenge
8 US Institutions, 5 years, $4M
Solve problem of colliding black holes (try…) NASA Neutron Star Grand Challenge
5 US Institutions, 3years, $1.4M
Solve problem of colliding neutron stars (try…) NSF ASC Project
5 US Institutions, 3 years, $2.2M
Develop “Collaboratory” EU GridLab
10 EU Institutions, 3 years, €5M
Develop Grid Tech
For these projects Grand Challenge Collaboratories
Building Communities to Solve These Problems
Socio-Economic Aspects of our Project for the EU… : Socio-Economic Aspects of our Project for the EU… Actually came up with an answer to this one!
We work hard together, but…
We have fun, too.
Actively building our Network community
Probably first Network to support socio-economic tourism with new €
Network research objectives : Network research objectives The formation of a close alliance among the different expert groups to solve urgent problems required for GWA, too large and complex for any single group.
Such a Network was needed, but did not exist.
The development and training of a young community of researchers to populate this emerging research area of GWA.
Strong, but somewhat isolated European groups already in relevant areas
Need coordinated effort to build up the broad-based expertise required for a systematic attack on these problems.
The development of a community simulation code for relativistic astrophysics
Simulation tools are essential for the success of this research area.
Cactus is an important part of the training program for this Network.
Slide6 : The development of appropriate approximation tools to provide better physical understanding and confirmation of numerical simulations.
Perturbation theory
Post-Newtonian Theory
The application of these numerical and approximation tools to a set of core astrophysics problems
GW signals from NS, BS, ???
Most promising sources for LIGO/GEO/VIRGO, …
Good focus problems for training a community
General View of Network : Enablers Focus Physics Areas Outreach /Collab GridLab ZIB/Garching
LBL/NCSA/ANL NCSA/WashU Cactus
Community NSF KDI 3D BH Collisions Other (strange stars,
Core collapse, etc) 3D NS processes and
mergers World General View of Network
Our Team 7 Focus areas, links created, strengthened by Network : Our Team 7 Focus areas, links created, strengthened by Network Valencia Ports SOTON Meudon Jena Palma Trieste AUTH Rome WashU Potsdam Jena
Project View: Code Leverage : Project View: Code Leverage NS Studies Meudon BBH Meudon BNS Hydra Projects extremely Collaborative
Build on each other
Share Modules or Build them together
Cactus an important collaborative technology EE’s MOL Horizon Finders Wave Extractors Lazarus Elliptics Parallel I/O Gauges AMR EE’s Community Workshop
this spring/fall
improves EE’s,
all simulations Vacuum BH Studies
Developing a Community: People Leverage : Developing a Community: People Leverage Actual Network funding rather small
10-12 positions (some positions converted to 2 for shorter times)
6-7 postdocs, 4-5 predocs
Very Strong Leverage! 6:1 ratio of paid/volunteer effort
10 coordinators
65 others officially working on the project at least part-time
Other sites in Europe joining in
Is it possible to get additional funding to help them?
Other projects in US and EU adding value
Attending this meeting
~65!!
This project has seeded a much larger effort
Now
Even more later
Community Code: Advantages : Community Code: Advantages Sharing of Expertise: No single group or community can do these problems
Sharing of Code among projects
Infrastructure
BH routines apply directly to NS work, etc…
But free to keep routines in group as long as desired
Better Code
Open source encourages people to be more careful in coding!
Encourages documentation
Encourages deeper thinking about how it interfaces to another code
More trusted code, results
When code becomes open, and people can run it for themselves, they will begin to believe the results
Improvements propagate quickly though community
Not well accepted yet, but we are starting a good trend…
Scientific Highlights of Work Thus Far : Scientific Highlights of Work Thus Far The Network is working
Community developing
Exciting meetings
Tools being used
Extraordinary new BH evolution results between AEI/Meudon groups
New Hydro Effort between AEI/SISSA/Thessaloniki/Valencia
…
Pre-Network 3D BH Results : Pre-Network 3D BH Results NSF Grand Challenge 1994
Equal mass, no spin, no J
Head on Collision in 3D
Code crashes by t=40M
See New Diener Result!! Grazing Collision 1999
Big Advance: 3D, unequal mass, spin, J
Physics!
Code crash by 40M
New 3D Black Hole Evolutions : New 3D Black Hole Evolutions New Techniques
Excision
Gauges, formulations
Baumgarte initial data
Estimated Innermost Stable Circular Orbit (ISCO)
Other circular orbits, too (“Pre-ISCO sequence: PI-1-10) OP Group has generated most astrophysically relevant binary BH data to date: now in Cactus!
High Profile These simulations will break ground in various ways : High Profile These simulations will break ground in various ways We feel we are about to crack (not solve…) this problem
Stumped community for 4 decades
$4.5M NSF Grand Challenge project helped lay groundwork
No other group close to doing a BBH orbit
US Dept of Energy
Special allocation for us to do work on 4.5TB machine
Very unusual: DOE HQ approval given!
NCSA
Special allocation
Public Outreach
Discovery Channel Movie for June
Scientific American Cover in April
EU Hydro Code Effort AEI/Valencia/Thessaloniki/Trieste : EU Hydro Code Effort AEI/Valencia/Thessaloniki/Trieste Originally a small collaboration between AEI and WashU
NASA GC project
Several papers published jointly, code developed/tested jointly
Hydro code itself primarily written by Mark Miller at WashU
Decided in Dec, 2001 to develop independent EU code, based on original lessons learned from old code
Visit of Baiotti and Montero to AEI, with Ian Hawke, many others contributing, critical to Network Hydro code
From start to finish, about 3 months
Initial Applications
Mergers: NS-NS, strange-strange, accretion
Later Applications
NS-BH, etc…
Stellar Core Collapse (w/Dimmelmeier)
The methodological approach and work plan : The methodological approach and work plan Scientifically organized into Fully Numerical, Post-N, and Perturbation Theory
Management: Project and Institutional tracks
Each subarea has person clearly assigned to shepard activities
Facilitating Connections
Phone calls, conference calls, Emails, etc
Each Institute has Local Coordinators assigned to manage Institutional matters
Project Coordinator tries to have global view of things
Advisory Board gives feedback
Students feedback solicited, incorporated
Peter Diener assigned to be student Ombudsperson…
Advisory Board : Advisory Board Bernard Schutz
AEI Director
Heavily involved in GEO, other GW Detector Efforts
Tsvi Piran, Hebrew University
Pioneer in numerical relativity
Very familiar with both relativity and astrophysics problems we face
Karsten Danzmann
AEI Director
Leads GEO Project
Jean-Pierre Lasota
Astrophysicist/Relativist
A Leader of Meudon group, expert in many problems of interest to us
Wai-Mo Suen
Close colleague, involved in many similar projects
Leader of some major US-based projects (ASC, NASA, others)
My Favorite Advisory Board Member : My Favorite Advisory Board Member “Sathya”
Expert in many things
Very polite
A true gentleman
Work Plan : Work Plan Cactus Nonlinear BH
simulations Hydrodynamics Post-Newtonian
theory Stellar
perturbation
theory Perturbative time
evolutions Characteristic
Methods Training complete,
AMR tested 12 months 24 months 36 months Remote/distributed
Simulations, input
from projects below Addressing Efficacy
Of Characteristic
Hydro Full community
Code release, with
documentation Merger simulations
with studies of waves Continually refined as techniques developed Completed binary
NS Initial
Data Module Simulations refined as
techniques developed Accretion
Code PN GW, reaction
expressions
Post-N.Hydro Code,
Post-N Initial Data
Module Simulations
continue Pert Eqs. for rotating
NS, nonlinear results Modes computed, valuable
input to num. simulations Close limit module,
time evolution for
rotating NSs Perturbative evolutions,
Input to num. simulations Nonlinear Numerical Codes,
Simulations Pert. Techniques,
codes, Time
How the network functions and how the partners collaborate : How the network functions and how the partners collaborate Our Website for communications:
http://www.eu-network.org/
Notes
Codes
Workplans
Results
Newsletter
Visits
Exchanges
AEI Visitor Program
Conference calls
All Pis every 2 months
Various subgroups at regular intervals
Video Conferencing: (Apply for a supplement??)
Electronic Collaborations : Electronic Collaborations CVS for common source code, reports, etc
Central repository at AEI
All partners in Network (and worldwide) can update local installations via simple commands (cvs update)
Email
Cactus
Collaborative
Modular
Testsuites!
Very advanced computational tools, developed worldwide, used, tested by our group
Leveraging Technologies of ASC, TiKSL, GridLab projects
Leveraging Other Projects : Leveraging Other Projects Cactus
NSF KDI-ASC Project
Suen (WashU), Norman, Shalf (LBL),Seidel (AEI/NCSA), Foster (Argonne), Parashar (Rutgers)
Norman’s Zeus Astrophysics Suite
GrACE: Parallel AMR for Many Applications
Portal: Global Resource Management, building generic GUI interfaces for resource selection
Viz Tools
German DFN TiKSL/GriKSL projects
GridLab
Slide24 : The Astrophysical Simulation Collaboratory (ASC) $2.2M NSF project (Suen, Seidel, Shalf, Foster, Norman, Parashar,…) 1. User has science
idea... 3. Selects Appropriate Resources... 5. Collaborators log in to monitor... 4. Steers simulation,
monitors performance... 2. Composes/Builds Code Components w/Interface... Want to integrate and migrate this technology to the generic user…
Cactus Our collaborative Infrastructure : Cactus Our collaborative Infrastructure Core “Flesh” Plug-In “Thorns”
(modules) driver input/output interpolation SOR solver coordinates boundary conditions black holes equations of state remote steering wave evolvers multigrid parameters grid variables error handling scheduling extensible APIs make system ANSI C Fortran/C/C++
Cactus User Community Using and Developing Physics Thorns : Cactus User Community Using and Developing Physics Thorns Goddard Penn State Wash U AEI TAC Tuebingen Southampton SISSA Thessaloniki Meeting Next Year:
Theoretical Foundations of Einstein’s Equations for Numerical Relativity
(Using Cactus to investigate and compare different implementations) Climate Modeling
(NASA,+) Chemical Engineering
(U.Kansas) Bio-Informatics
(Canada) Geophysics
(Stanford) Astrophysics
(Zeus) Plasma
(Princeton) EU Astrophysics
Network NASA Neutron Star
Grand Challenge Early Universe
(LBL) Numerical Relativity Other Applications Portsmouth RIKEN Impact on other
Projects!
Cactus ASC Portal www.ascportal.org (prototype/works now) : Cactus ASC Portal www.ascportal.org (prototype/works now) Part of NSF KDI Project
(Astrophysics Simulation Collaboratory)
Use any Web Browser !!
Portal (will) provides:
Single access to all resources
Locate/build executables
Central/collaborative parameter files, thorn lists etc
Job submission/tracking
Access to new Grid Technologies
GridLab: www.GridLab.org Enabling Dynamic Grid Applications : GridLab: www.GridLab.org Enabling Dynamic Grid Applications EU Project ~€5M
AEI, ZIB, PSNC, Lecce, Athens, Cardiff, Amsterdam, SZTAKI, Brno, ISI, Argonne, Wisconsin, Sun, Compaq
Grid Application Toolkit (APIs/Tools)
Develop new grid scenarios for 2 main apps:
Numerical relativity
Grav wave data analysis
Dynamic Grid Computing : Physicist has new idea ! Free CPUs!! Add more
resources Clone job with
steered parameter Queue time over,
find new machine Dynamic Grid Computing
Connections to industry : Connections to industry We are connected to industry through computational science projects
Microsoft
Sun
Compaq
Intel
GridLab
Grid Systems
No direct coupling through our science (that I know of)
The training programme : The training programme Feedback on talks, practice talks, etc
Review Talk topics selected by YR’s
School planned for 2003
Training sessions (twice), extensive online tutorials
High performance computing,
Cactus (our code framework)
Visualization tools,
Code maintenance systems (CVS), etc
Want to give Video courses on scientific topics!
Not able so far
Access Grid funding with NSF?
Use of the Budget : Use of the Budget Midterm report shows basically on track
Most of the money (1.5M€) spent on YR
About 1/3 of funding spent to date
Roughly proportionate for Networking and salary costs
Will accelerate spending as more YRs hired
Portsmouth, Valencia, AUTH made late hires
UIB just now making hires
Any proposed revision to the contract?
No major changes forseen at this time, but questions
Young Researcher Issues : Young Researcher Issues Special attention paid to their training and well being…
Any difficulties encountered? Peter will report!
Responsibilities given to YRs
Peter Diener’s role as YR coordinator
Helping to prepare reports!
Developing Web sites, documentation
Building Central Codes
Developing “soft skills”
Collaboration/Communication built in to this project
Slide34 : Seek Input
Actively seek YR input into all activities of the Network,
Owe several beers: Dorota, Johannes, Reinhard
1.5 cases to Denis, Werner…
topics for review talks, suggestions for improving their training
Anonymous FAQ on web site
Give Input/Advice
Comment constructively on the talks
Last time had rule that senior researchers could not ask questions until after YRs did: wait and prod until they do!
General Problems Encountered : General Problems Encountered Recruitment difficult
Restrictive rules
No existing EU community
Travel budgets not enough
Restrictions on overseas travel!
Insufficient EU travel/subsistence funding for visits
No real budget to support extended visits we are supposed to have
Bureaucracy
Reports, regulations far more than any NSF, NASA, or DFN grant I know!
Computing
No central EU computing facilities
We are quite lucky and unique due to US connections
Specific Questions from/for EC : Specific Questions from/for EC Staffing issues/Collaborations
Students finishing after Network ends
Must care for them!
No-Cost Extension?
New Project/Continuation
Status of UIB
Strong historical connection to AEI
Postdocs hired beginning next month and month after
Generally plan to fulfill person months
Could use some supplement for extended visits
Role of WashU group
Slide37 : Strengthening Ties with Other Projects
US groups
WashU
NSF projects, comoputing facilities, etc
New NSF, NASA projects? Joint funding?
GridLab, other EU projects
GridLab officially connected through Pis, technologies
Enabling other EU groups to join as “Associate Members”
Various other groups want to join
A small amount of funding could go a long way
Various issues to discuss later…