ggf3 final

Slide1 : Development of GRID environment for interactive applications J.Marco (CSIC) GGF3 (Frascati,Rome) 9-X-2001


The CROSSGRID project : The CROSSGRID project EU Vth Framework Programme (IST) 3 year project 2002-2004 6M € proposal, negotiation by end of October Objectives: Final user interactive applications Extend GRID into 11 EU countries in close collaboration with DATAGRID 22 Partners: COORD: CYFRONET (PL, coordinator M.Turala) CR: UvA (N), PSNC(PL), FZK (D) ,CSIC (E) Companies: Algosystems(GR) + DATAMAT (I) ICM,INP,IPJ (PL) + IISAS (SK) + USTUT+TUM (D) + ULINZ (A) + TCD(Ir) UAB+USC+UAM (E) + LIP (P) + Demo+AuTH (Gr) + UCY (Cy)


CROSSGRID Collaboration : CROSSGRID Collaboration Poland: Cyfronet andamp; INP Cracow PSNC Poznan ICM andamp; IPJ Warsaw Portugal: LIP Lisbon Spain: CSIC Santander Valencia andamp; RedIris UAB Barcelona USC Santiago andamp; CESGA UAM Madrid Ireland: TCD Dublin Italy: DATAMAT Netherlands: UvA Amsterdam Germany: FZK Karlsruhe TUM Munich USTU Stuttgart Slovakia: II SAS Bratislava Greece: Algosystems Demo Athens AuTh Thessaloniki Cyprus: UCY Nikosia Austria: U.Linz


Project Structure : Project Structure Workpackages: Applications (P.Sloot, UvA) Programming Environment (H.Marten, FZK) New Services andamp; tools (N.Meyer, PSNC) Testbed (J.Marco, CSIC) Dissemination (Y.Perros, Algosystems) Management (M.Turala) Architecture Team Lead by M.Bubak, Cyfronet


Applications : Applications Final user interactive applications: Response loop in the range of seconds to minutes Computing andamp; Data intensive Architecture basis: GLOBUS + DATAGRID Multidisciplinary: Simulation andamp; visualisation for surgical procedures Air pollution combined with weather forecasting Distributed Data Analysis in HEP Flooding crisis team decision support system


Tools & Services : Tools andamp; Services GRID Application Programming Environment Verification of MPI use Performance prediction Monitoring Services andamp; Tools User friendly portals Roaming access Efficient distributed data access Specific resource management Open Architecture approach


Testbed : Testbed Testing and validation for Applications Programming environment New services andamp; tools Emphasis on collaboration with DATAGRID + extension to DATATAG Extension of GRID across Europe


Extension of GRID : Extension of GRID


Application development : Application development Common requirements: Near real-time performance across the network Efficient access to large distributed databases Large scale parallel simulations Data discovery Data mining Visualisation Deployment on the CROSSGRID testbed will test the applications in the final user environment and provide feedback WP2 WP3 MPI Adressed in collaboration between WP1 tasks


Interactive Simulation & Visualisation of a Biomedical System : Interactive Simulation andamp; Visualisation of a Biomedical System pre-treatment planning in vascular interventional and surgical procedures through real-time interactive simulation of vascular structure and flow Distributed real-time simulation environment The user can change the structure of the arteries, thus mimicking an interventional or surgical procedur Developped by University of Amsterdam, in collaboration with the Leiden University Medical Center Distributed Visualisation System GVK (University of Linz) GRID


Flooding Crisis Team Support : Virtual organisation for flood prevention and protection Key points: Numerical flood modelling Near real-time response Tasks: Distributed data collection Distributed simulation andamp; data analysis Distributed access, support for Virtual Organisation System integration Leadership: II SAS Bratislava Flooding Crisis Team Support


Flooding Crisis Team Support Model : Flooding Crisis Team Support Model


Distributed Physics Analysis in HEP : Distributed Physics Analysis in HEP Final user applications for interactive physics analysis in a GRID aware environment for LHC experiments Acess to large O/R DBMS through middleware server employing XML for query/data format description Distributed data-mining techniques (mainly Neural Networks) Integration of user-friendly interactive acess via portals CSIC + UAB + FZK + INP + ICM + IPJ


Evolution in HEP Interactive Physics Analysis : Evolution in HEP Interactive Physics Analysis From LEP2 era (CERN 1995-2000, the Higgs hunt): Physics analysis on final 'ntuples', 10-100 GB size/channel Processed by teams/individually: Preselection cuts + b-tagging + kinematics fixed. Multidimensional techniques not 'real-time': Neural Networks took 10 minutes – several hours CL estimations not simultaneously optimized


Evolution in HEP Interactive Physics Analysis : Evolution in HEP Interactive Physics Analysis ...To LHC era (CERN 2003 (Physics TDR)-...): High rate of complex pp collisions ( 109 interactions/second) Optimize High Level Trigger Work on Physics TDR: channels to define data subsets Interactive physics analysis will run on distributed databases much larger than at LEP o(PB ) Need for interactive friendly user applications in a GRID environment, supporting powerful analysis techniques


Weather forecast and air pollution modelling : Weather forecast and air pollution modelling ICM (Warsaw) + CSIC (Santander+Valencia) + USC (Santiago) (strong links to Meteo community) Integration of distributed databases into GRID: Operational data Reanalysis DB (numerical weather prediction model integrated over decades) Migration of data-mining algorithms: Linear and non-linear correlation methods Neural networks Air pollution model coupled to weather prediction: Based on photochemical grid model (STEM II) Visualisation using GVK Testing and demonstration


WP2: GRID Application Programming Environment : WP2: GRID Application Programming Environment Facilitate development and tuning of parallel distributed interactive applications on GRID: MPI profiling and verification (UST+CSIC) Metrics and benchmarks (UCY,TUM) Interactive performance evaluation tools (CYFRONET+USC) Integration and coordination: FZK


WP3: new GRID services & tools : WP3: new GRID services andamp; tools User friendly GRID (PSNC + UCY+ DATAMAT + ALGO) Portals Roaming access GRID resource management (UAB, CSIC) Based on self-adaptive scheduling agents Take into account dynamic information from monitoring Monitoring (TCD+CYFRONET) Service managers + local monitors + application monitors Optimisation of data access (CYFRONET) Expert system proposing data migration


WP3support for other WP : WP3 support for other WP


WP4: International Testbed Organisation : WP4: International Testbed Organisation Objective: Provide a distributed resource facility Support testbed sites across Europe, integrating national facilities Assure interoperability with DATAGRID testbed 16 testbed sites across 9 countries GEANT for network connectivity Effort coordinated by CSIC, benefiting from experience in DATAGRID testbeds also in TCD, UAB, LIP


Testbed MAP : Testbed MAP Auth Thessaloniki UvA Amsterdam FZK Karlsruhe TCD Dublin UAB Barcelona LIP Lisbon CSIC Valencia UAM andamp; CSIC Madrid USC Santiago CSIC Santander DEMO Athens UCY Nikosia CIFRONET Cracow II SAS Bratislava PSNC Poznan ICM andamp; IPJ Warsaw


Links with GRID projects : Links with GRID projects Strong link with DATAGRID: Complementary middleware Extension of testbed: interoperability Support to prepare project! Thanks! National GRID initiatives: Poland, Ireland, Spain... DATATAG: Support in applications with possible collaboration in USA Presence in GRIDSTART Participation in GGF


Current Activities & Plans : Current Activities andamp; Plans Exploratory work on Distributed O/R DBMS use Data-mining techniques (NN) parallelization using MPI Collaborative tools setup Videoconf meetings Web Support Documentation Preparing Negotiation (AnnexI) Start beginning 2002 !