20041007 umwlap002 04 barberis

Computing plans and outlook : 

Computing plans and outlook Dario Barberis CERN & Genoa University

ATLAS Computing Timeline: 

ATLAS Computing Timeline

Miscellaneous News: 

Miscellaneous News Computing Model: convergence to be reached soon (including data sizes) document to be submitted by end 2004 for LHCC review in Jan’05 Computing TDR: still due in June 2005 (we have to get going with it!) LCG MoU and ATLAS Computing Addendum to M&O MoU: documents being worked on will be finalized by the end of this year submitted to the C-RRB and ATLAS RRB in April 2005 to be signed by Funding Bodies before the end of 2005 hopefully effective in 2006 ATLAS Core Computing Project: will be defined in the C-MoU – draft being discussed with management

Plans for 2005: 

Plans for 2005 Consolidation most required functionalities are now in place or being actively worked on some of the developments in 2004 were targeted at specific clients: DC2, CTB, HLT... all have to be brought together coherently Robustness both of the software base and the computing environment (production and generally job submission tools) Optimization speed of data access and algorithm execution, as well as memory size, has to become a priority for next year Usability easing the threshold for newcomers has to be a priority

Reviews in 2005: 

Reviews in 2005 We plan a complete set of reviews of all software development and deployment activities: Grid production tools Software Infrastructure Framework and EDM Detector Description and Graphics Simulation Calibration & Alignment Tracking (InDet & Muons) Calorimetry (LAr & Tile) ESRAT (including Physics Analysis Tools) Database & Data Management Distributed Analysis Reviews will take place between January and July 2005

The “final” prototype: DC3: 

The “final” prototype: DC3 We should consider DC3 as the “final” prototype, for both software and computing infrastructure Current tentative schedule is Q1-2006 to end Q2-2006 Schedule is highly constrained DC2 Phase III (tests of Distributed Analysis) will probably run during Q1-2005 We agreed to have a “season” of reviews during the first half of 2005 Developers will need some time between DC2 and DC3 for necessary work: completion of the system with the inclusion of missing functionality integration of components developed in the context of the CTB implementation of review recommendations Cosmic run will be later in 2006 This will be the last large scale exercise before real data After DC3 surely we will be forced to sort out problems day-by-day, as the need arises, for real, imperfect data coming from the DAQ: no time for more big developments

DC3 aims: 

DC3 aims We should agree unambiguously on the aims of this operation DC1 and DC2 started with 1-2 main objectives but many other clients showed up with sometimes negative interferences We need from time to time large scale tests (increasing scale tests) of both software and computing infrastructure We should build on the lessons of DC2 and not repeat blindly the same exercise example: running ATLAS Geant4 simulation is easy (everything works well), using the current set of Grid tools is another story Main aim of DC3 should be to test the software and computing infrastructure that we will need at the beginning of 2007: calibration and alignment procedures and conditions DB Tier-0 reconstruction and data distribution distributed access to the data for analysis

DC3 Software (1): 

DC3 Software (1) Geometry: propose initial geometry updated version with representation of really built detectors some time-varying misalignments and miscalibrations must be introduced (this part was descoped from DC2) Simulation: good performance in DC2 Geant4 production continuous production with DC2/initial geometry will start soon should use “as-built” geometry as soon as available all available simulated data at the end of 2005 can be used as input for DC3 Event Data Model: reviews in early 2005 and feedback from the Rome Physics Workshop will assess the current EDM for reconstruction and analysis

DC3 Software (2): 

DC3 Software (2) Trigger: we should run L1 trigger as part of digitization routinely for L2 and EF, we could think of running them as a “pre-challenge” together with the HLT: let’s open the discussion Calibration & Alignment: was descoped from DC2 but developed with the CTB this winter we have to merge these developments back into the main stream we could envisage two Cal/Al passes: fast, run in a few hours, to provide parameters for Tier-0 reconstruction slow, with more precise results, to provide parameters for Tier-1 reprocessing within a few months of data taking we would like to exercise both in DC3

DC3 Software (3): 

DC3 Software (3) Reconstruction: reviews in early 2005 will assess the current algorithms for reconstruction and analysis Physics Analysis Tools: the Rome Physics Workshop will assess the current functionality Data Management: we’ll build on the experience of DC2 (intensive part is still to come!) DM review is foreseen towards the middle of 2005 Production tools: the review of the production system used for DC2 will be at the beginning of 2005 this leaves ~1 year to set up the system to be used for DC3 there was also 1 year to develop the DC2 ProdSys but clearly it was not enough one the other hand, this time we are not starting from zero again

Preliminary plan for DC3 (1): 

Preliminary plan for DC3 (1) Aim is a sustained 50% data flow test: we need at least 50M triggered events if we want to run 10 days intensively remember: the “nominal” raw data rate is 200 Hz and each day has 50k “active” seconds: 10M events/day Simulation: use all production done in any case in the 2nd half of 2005 supplement by missing channels to reach a reasonably representative mixture of event types we have to decide on pile-up levels Pre-DC3: run trigger (if appropriate), bring events to CERN, mix, put on Castor disk pool as if they were coming from online

Preliminary plan for DC3 (2): 

Preliminary plan for DC3 (2) Real DC3: run “fast” Cal/Al procedures reconstruct on Tier-0 distribute output according to the Computing Model run analysis jobs in many sites run “slow” Cal/Al procedures reprocess at Tier-1’s and redistribute the data Manpower: need to plan shifts in advance DC3 should be treated as a test beam operation timescales to be followed both for developments and productions We need a “DC3 Coordinator” soon!

After DC3: 

After DC3 We should keep the DC3 infrastructure working “forever” after DC3 we can use both simulated and cosmic ray data as event source Any modification to software, infrastructure or data distribution must be planned at that point well in advance no “here is the solution!” untested bug fix in the production environment We must run the production in a more professional way effectively from early 2006 for many many years production tool developer team production operation team adequate monitoring tools people on shift (also remotely) to follow and monitor the production running offline shifts will be a task for the whole Collaboration