011116 dg coord price

Datagrid Coordination Meetings4 March 2001, Amsterdam23 June 2001, Rome7 October 2001, Rome: 

Datagrid Coordination Meetings 4 March 2001, Amsterdam 23 June 2001, Rome 7 October 2001, Rome Summary for HEP-CCC Meeting 16 November 2001, CERN L. Price, ANL

Goals of DG Coordination: 

Goals of DG Coordination Define jointly the architecture of the data grid to avoid incompatible architectures from different projects. Ensure that tools developed by all projects will interoperate to make as much as possible a seamless data grid system for LHC and our other scientific participants. Partition major tasks among the projects in order to use scarce resources to best advantage and avoid having each project invent only the top priority 1/3 of what we ultimately need.

Participants: 

Participants

Benefits and Costs of Coordination: 

Benefits and Costs of Coordination Benefits Effective reach of projects Common infrastructure Larger effort improves visibility, attracts funding Costs More interactions More meetings More coordination More reporting More B.S. From Paul Avery

Many Flavors of Joint Projects: 

Many Flavors of Joint Projects GriPhyN, PPDG and EU DataGrid are already joint projects Interconnected management structures Each serving needs of multiple constituencies GriPhyN: CS, HEP/LHC, gravity waves, Digital astronomy PPDG: CS, HEP (LHC + current expts), Nuc. Phys., networking DataGrid: CS, HEP, earth sensing, biology, networking From Paul Avery

Joint Projects: The Reality: 

Joint Projects: The Reality From Richard Mount

Discussion Points: 

Discussion Points Common testbeds are necessary as a means of strongly encouraging compatibility, as well as the usual practical benefits of testbeds, namely that any problem of the system in the real world will be revealed. Support of shared code base. In addition to the overlapping goals of the datagrid projects, it was noted that the projects all rely on tools developed outside of the projects, including Globus, Condor, and others. Many practical and legal issues are raised by the need to make program available to and usable by other parts of the overall datagrid community. Open source and other sharing concepts were discussed in some detail. Desirability of working toward a common or at least compatible architecture. This is so partly because the LHC experiments, at least, are connected with all of the projects and will need to use parts of each project. It also permits separately developed modules to interoperate with less effort than otherwise.

Common Architecture: 

Common Architecture A unifying structure & approach Standard protocols, APIs, policies Reflects pragmatic choices corresponding to best available technology Documented (and mostly implemented) Certainly a work in progress Requires Some structure (“architect”? “arch board”?) Procedures for collecting input from the user community and guiding development From Ian Foster

Main coordination issues: 

Main coordination issues Compatibility of DataGrid architecture with PPDG and GriPhyN Synchronisation of deliverables among the different projects Common test beds across Atlantic Globus support and development in US and EU From Fabrizio Gagliardi

Other Points of Discussion : 

Other Points of Discussion Mechanisms for coordination. Ian Foster and Carl Kesselman are already members of the EU DataGrid Architecture Task Force Coordinating council concept, consisting of management representatives from the projects. Technical steering group consisting of technical representatives of the projects. Synchronization of deliverables is also an issue, since some experiments are connected to multiple grid projects. Dedicated facilities. Dedicated bandwidth for tests, expecially across oceans. Grid Operating Center will probably be needed.

Grid activities in Japan: 

Grid activities in Japan Ninf – ETL/TIT Is a frontier developing “Network enabled servers” Has been collaborating with NetSolve, UTK Grid RPC – APM WG Metacomputing – TACC/JAERI MPI for vectors, STAMPI ApGrid A regional testbed across the Pacific Rim Globus promotion  Firewall compliance extension Others ITBL – MEXT (ex-STA part) Tsukuba-One (DWDM ring) Super SINET IPv6 promotion From Satoshi Sekiguchi

2nd Meeting: Rome 23 June 2001: 

2nd Meeting: Rome 23 June 2001 Focus on organization Relationship to CERN “LHC Grid Project” Licensing and code sharing proposal Ian Foster: Consortium for Open Grid Software International Data Grid Coordination Organization

Consortium for Open Grid Software: 

Consortium for Open Grid Software From Ian Foster

Shared Development of Grid Software:A Proposal: 

Shared Development of Grid Software: A Proposal Long-term success of Grid concept requires establishment of a framework for shared development of Grid software, that Addresses legal/social issues associated with many contributors and users Addresses governance issues Is connected with “standards” processes Builds credibility for open architecture, open source Grid approach COGS: Consortium for Open Grid Software From Ian Foster

International Data Grid Coordination Organization: 

International Data Grid Coordination Organization Extended discussion, resulting in following organization plan: InterGrid Management Board Start from the ad hoc group that has met twice Consider small number of additions to be sure all players are represented Board represents managements of other projects; does not manage the projects itself! Meet 3 times per year, usually face to face Joint Technical Board Membership: 6 from North America; 6 from Europe; 2 Asia Meet monthly by telephone or video conference Face to face when needed

3rd Meeting: Rome 7 Oct 2001: 

3rd Meeting: Rome 7 Oct 2001 Update on LHC Grid Computing from Hoffmann (in person) Agreement that LHC project will be part of HENP Intergrid Coordination group (new name) Initial presentation of potential joint testbeds Transatlantic testbeds or even better worldwide testbeds are considered to be the heart of the grid coordination process. Interoperability will be required for any success. After initial coordination of grid architectures, testbeds will generally be done by groups from international physics experiments associated with the grid projects, so that practical benefit will be obtained. Refinement of COGS plan License draft generally agreed to with some comments Governance and organization less clear—but considered not to be required

Getting going on joint testbeds: 

Getting going on joint testbeds Tasks: (a)    identify and agree on the initial sites to be included in the pilot testbed, with 3 or 4 sites from each side of the Atlantic; (b)   agree with the collaborations on the target applications and an initial data challenge programme; (c)    define the resources, environment, software and services to be provided at each of the component sites; (d)   specify the Grid protocols, interfaces and standards necessary to ensure inter-working and a satisfactory application environment; (e)    prepare a plan for building and operating the testbed; (f)     prepare a plan for initial usage of the testbed by selected experiments.

Participating Data Grid Projects: 

Participating Data Grid Projects PPDG GriPhyN DataGrid National Projects: Italy, France, UK, Holland,…. DataTAG iVDGL GridPP LHC Grid Computing Project Good liaison with experiments: BaBaR, D0, ATLAS, CMS, ALICE especially

Licensing: Goals: 

Licensing: Goals Allow anybody to use Grid software for any purpose No constraints on intended purpose Can include things you don’t like! Allow anybody to contribute to Grid sofware Get credit for their work As few bureaucratic barriers as possible Two goals are not necessarily compatible! From Ian Foster

Globus Toolkit as a Vehicle: 

Globus Toolkit as a Vehicle We offer the Globus Toolkit as a vehicle for those who want to use it Believe that there are advantages: Strong “brand” with considerable acceptance Considerable code base already, of core code and contributed modules Make commitment to community to support this process as best we can From Ian Foster

First Meeting of Joint Technical Board: 8 Oct 2001: 

First Meeting of Joint Technical Board: 8 Oct 2001 Of membership of 14, 5 (+LEP) were together at Frascati, 6 were on a conference phone call, and 2 were not appointed yet. General discussion of practical organization and goals of testbed projects. Beyond general agreement to “Keep it Simple”, there was not time to conclude the discussion. Presentations of status, architecture plans, and near term milestones for each data grid project Agreement to meet first Monday of each month at 7 am in California/4 pm in middle European time by phone or possibly video.

Summary : 

Summary Datagrid projects have individual identities and responsibilities to funding agencies and cannot be managed as a single project Neverthless close coordination is important for success The adopted compromise is to organize so the technical leaders of the different projects talk frequently (JTB) The managements also need to work together, but meetings are less frequent (IGMB) A legal basis for joint software development is important and being worked on. Common testbeds are very important.