wp6 crossgrid status

Slide1: 

Development of GRID environment for interactive applications J.Marco (CSIC) DATAGRID WP6 MEETING (PARIS) 5-III-2002

The CROSSGRID project: 

The CROSSGRID project EU Vth Framework Programme (IST) 3 year project, starting date 1st March 2002 4.86 M € contract signed last week! Objectives: Final user interactive applications Extend GRID into 11 EU countries in close collaboration with DATAGRID 21 Partners: COORD: CYFRONET (PL, coordinator M.Turala) CR: UvA (N), PSNC(PL), FZK (D) ,CSIC (E) Companies: Algosystems(GR) + DATAMAT (I) ICM,INP,IPJ (PL) + IISAS (SK) + USTUT+TUM (D) + ULINZ (A) + TCD(Ir) UAB+USC (E) + LIP (P) + Demo+AuTH (Gr) + UCY (Cy)

CROSSGRID Collaboration: 

CROSSGRID Collaboration Poland: Cyfronet andamp; INP Cracow PSNC Poznan ICM andamp; IPJ Warsaw Portugal: LIP Lisbon Spain: CSIC Santander Valencia andamp; RedIris UAB Barcelona USC Santiago andamp; CESGA Ireland: TCD Dublin Italy: DATAMAT Netherlands: UvA Amsterdam Germany: FZK Karlsruhe TUM Munich USTU Stuttgart Slovakia: II SAS Bratislava Greece: Algosystems Demo Athens AuTh Thessaloniki Cyprus: UCY Nikosia Austria: U.Linz

Project Structure: 

Project Structure Workpackages: Applications (P.Sloot, UvA) Programming Environment (H.Marten, FZK) New Services andamp; tools (N.Meyer, PSNC) Testbed (J.Marco, CSIC) Management (M.Turala) Architecture Team Lead by M.Bubak, Cyfronet

Applications: 

Applications Final user interactive applications: Response loop in the range of seconds to minutes Computing andamp; Data intensive Architecture basis: GLOBUS + DATAGRID Multidisciplinary: Simulation andamp; visualisation for surgical procedures Air pollution combined with weather forecasting Distributed Data Analysis in HEP Flooding crisis team decision support system

Tools & Services: 

Tools andamp; Services GRID Application Programming Environment Verification of MPI use Performance prediction Monitoring Services andamp; Tools User friendly portals Roaming access Efficient distributed data access Specific resource management Open Architecture approach

Testbed: 

Testbed Testing and validation for Applications Programming environment New services andamp; tools Emphasis on collaboration with DATAGRID + extension to DATATAG Extension of GRID across Europe

Extension of GRID: 

Extension of GRID

Application development: 

Application development Common requirements: Near real-time performance across the network Efficient access to large distributed databases Large scale parallel simulations Data discovery Data mining Visualisation Deployment on the CROSSGRID testbed will test the applications in the final user environment and provide feedback WP2 WP3 MPI Adressed in collaboration between WP1 tasks

Distributed Physics Analysis in HEP: 

Distributed Physics Analysis in HEP Final user applications for interactive physics analysis in a GRID aware environment for LHC experiments Acess to large O/R DBMS through middleware server vs access to catalog + distributed root data files Distributed data-mining techniques (mainly Neural Networks, also self organizing maps, ...) Integration of user-friendly interactive acess via portals CSIC + UAB + FZK + INP + ICM + IPJ

HEP Interactive Physics Analysis: 

HEP Interactive Physics Analysis MPI Master Scalability: OK in local cluster…

WP4: International Testbed Organisation: 

WP4: International Testbed Organisation Objective: Provide a distributed resource facility Support testbed sites across Europe, integrating national facilities Assure interoperability with DATAGRID testbed 15 testbed sites across 9 countries ~16 FTE working on them Start points: Join DataGRID testbed gradually Use GEANT for network connectivity Extend testbed with incremental releases as applications and middleware demand and provide feedback Effort coordinated by CSIC, will benefit from experience in DATAGRID testbed also in TCD, UAB, LIP

Testbed MAP: 

Testbed MAP Auth Thessaloniki UvA Amsterdam FZK Karlsruhe TCD Dublin UAB Barcelona LIP Lisbon CSIC IFIC Valencia CSIC Madrid USC Santiago CSIC IFCA Santander DEMO Athens UCY Nikosia CIFRONET Cracow II SAS Bratislava PSNC Poznan ICM andamp; IPJ Warsaw

Slide14: 

Network (Geant) setup CrossGrid WP4 - International Testbed Organisation

Slide15: 

Tasks in WP4 4.0 Coordination and management (task leader: J.Marco, CSIC, Santander) Coordination with WP1,2,3 Collaborative tools (web+videoconf+repository) Integration Team 4.1 Testbed setup andamp; incremental evolution (task leader:R.Marco, CSIC, Santander) Define installation Deploy testbed releases Trace security issues CrossGrid WP4 - International Testbed Organisation Testbed site responsibles: CYFRONET (Krakow) A.Ozieblo ICM(Warsaw) W.Wislicki IPJ (Warsaw) K.Nawrocki UvA (Amsterdam) D.van Albada FZK (Karlsruhe) M.Kunze IISAS (Bratislava) J.Astalos PSNC(Poznan) P.Wolniewicz UCY (Cyprus) M.Dikaiakos TCD (Dublin) B.Coghlan CSIC (Santander/Valencia) S.Gonzalez UAB (Barcelona) G.Merino USC (Santiago) A.Gomez Demo (Athenas) C.Markou AuTh (Thessaloniki) D.Sampsonidis LIP (Lisbon) J.Martins

Slide16: 

Tasks in WP4 4.2 Integration with DATAGRID (task leader: M.Kunze, FZK) Coordination of testbed setup Exchange knowledge Participate in WP meetings 4.3 Infrastructure Support (task leader: J.Salt, CSIC, Valencia) Fabric management HelpDesk Provide Installation Kit Network support 4.4 Verification andamp; quality control (task leader: J.Gomes, LIP) Feedback Improve stability of the testbed CrossGrid WP4 - International Testbed Organisation

Slide17: 

Workflow and interfaces CrossGrid WP4 - International Testbed Organisation Setup report on requirements andamp; planning 4.0 Coordination and management (PM1-36) Plan Evolution: Prototype 0 FINAL WP 1,2,3 WP 1,2,3 WP 1 requirements from 1st prototype releaset final testbed 3 15 21 33 36 D4.5 D4.1 M4.1 30 D4.7 testbed setup on selected sites 6 D4.2 D4.4 Feedback from Feedback mainly from PU PU M4.3 PU internal progress report CO CO D4.6 internal progress report CO M4.4 final demo + report PU PU First testbed Support Prototype 1 Integration Team 10 internal progress report DATAGRID testbed Feedback from D4.3 CO M4.2

Questions…: 

Questions… Interactive use of resources: Priorities/Allocation/Booking policies How to allocate 500 machines for 5 minutes of interactive use? 'Connection': Concept of Interactive Session? Use of Permanent and Volatile storage Scheduler? Strongly data-location aware Dynamic versioning/installation? Software configuration for different applications

Testbed Organization:: 

Testbed Organization: Use national certificates as in DataGrid: Spain, Portugal, Ireland, Germany, Poland Greece, Cyprus, Slovakia… V.O.: Define CROSSGRID testbed V.O. Join DataGrid HEP V.O. (Alice,Atlas,CMS,LHCb) Define new V.O. for other applications: Med V.O. Flood Prevention V.O. Meteo V.O.

Testbed Organization:: 

Testbed Organization: Kick-off meeting in Krakow (17th March) First three months: Review and setup local infraestructure Gather requirements from applications/middleware Setup HelpDesk, define Integration Team Reinforce connection with Datagrid: Extend testbed (Germany,Poland,Portugal,Spain) Find topics for contribution