PAGIS

PAGIS: An Architecture for Programming on the Grid: 

PAGIS: An Architecture for Programming on the Grid Andrew Wendelborn Distributed & High Performance Computing Group Department of Computer Science, University of Adelaide Adelaide SA 5005, Australia Email: andrew@cs.adelaide.edu.au December 2002

Background: 

Background PAGIS DHPC propject Processing of satellite imagery Started as a compositional process-oriented model for GIS applications Now an architecture for building grid applications Outline Base-level API and GUI Reflective (metalevel) programming for grid applications Framework for defining grid behaviours Adaptive software: safe reconfiguration An adaptive resource broker

Principles: 

Principles High-level computational model Programmers shouldn’t need to be grid experts Reason about behaviour at an abstract level Simple, intuitive interface Following compositional component-based paradigm Access to system Programmer abstracts over low-level details … BUT Wants to tune aspects of system critical to grid application We provide this via a separate interface

Process Networks: 

Process Networks General, clearly defined semantics Separates meaning from implementation Different strategies produce equivalent results Compositional Reconfigurable Highly applicable to grid programming Process1 Process3 Process2 Process4

Slide5: 

Evaluator Client PAGIS overview GUI PN structure

Slide6: 

Evaluator Scheduler Client

Slide7: 

Client Evaluator GRAM - GRIS GRAM - GRIS GIIS Scheduler Grid enabled Globus toolkit Distributed JVMs

Tuning and Behaviour: 

Tuning and Behaviour We have seen how to build base-level application How to fine-tune its implementation and behaviour Types of tuning Replacing code for PN process Allocation and re-allocation Monitoring, and other non-application (meta) concerns “Actors” observe and influence behaviour Human observation and intervention Other software systems Change how the computation occurs, not the result of the computation

A Reflective Approach: 

A Reflective Approach Separate concerns into two programs Functional concerns: baselevel program Non-functional concerns: metalevel program (meta-behaviour) Reflects separation in process network model Reify certain aspects of baselevel Metalevel programmer reflects upon selected aspects to control or customize aspects critical to application Software engineering for the grid

Method Reification: 

Method Reification Constructs an explicit representation of a method invocation at the metalevel Provides hooks to control and customize method invocation process We have designed a customization framework, Enigma ObjectA ObjectB MetaA MetaB

Grid Behaviours: 

Grid Behaviours Enigma gives us a model for describing and implementing interesting grid behaviours Locality Migrating the computation and its parts Allocation and Adaptation Computation meta-attributes to drive allocator Periodically reassess allocation Performance Monitoring Intercept invocations to monitor progress

PAGIS as a Grid Programming Model: 

PAGIS as a Grid Programming Model API provides baselevel interface to system PN provides semantics and safety of adaptation Enigma: hooks to observe / influence behaviour Software engineering of adaptive applications

The System: 

The System Allocation Manager Channel Service Process Service Distribution Tools Globus Plugin Broker COG MDS GRAM Gatekeeper Job Manager Worker JVM fork GRAM data RSL Client Worker LDAP query Client Plugin Java implementation Globus toolkit Java CoG Coglets for peer-to-peer Java / Globus

Current Implementation: 

Current Implementation Building a PN Java; CoG; Globus toolkit Operations for simple image processing Inspection of meta-level attributes Changing meta-level attributes Influences allocation May lead to migration Underway: integration with DARC

PAGIS and DARC: 

DARC provides data abstractions PAGIS provides process abstractions Host A Host B Host C Node Node Node DR DR DR DR DR DR TCP PAGIS and DARC

Slide16: 

Client Evaluator GRAM - GRIS DARC GIIS Scheduler DARC provides data sources Powerful abstraction over Globus toolkit

Adaptive Resource Broker Project: 

Adaptive Resource Broker Project Testbed for different RB strategies Emulator (NISTNet) and simulators NWS (Network Weather Service) Performance database Forecasts Detection of anomalous conditions Generate threshold events RB adapts its behavior

Possible emulation framework: 

Possible emulation framework

Grid Simulator: 

Grid Simulator Simple discrete event simulation package Builds on GridSim and SimGrid Java, multi-threaded Can specify a full network topology Heterogeneous resources of any kind modelled easily Arbitrary information can be published about resources Allows study of RB techniques on an arbitrary grid monitored by NWS threshold events feedback to RB

Next Steps: 

Next Steps Java peer-to-peer environment Coglets for peer-to-peer Java / Globus DARC integration Adaptive middleware for mobile computing Experimental testbed for Enigma-based behaviours Adaptive RB framework XML intermediate form for PN structure … applications …