Slide1 : © 2006, iTKO, Inc. | All rights reserved. Achieving 99.99% Quality in SOA Applications – The “Headless” Dilemma
John Michelsen
StickyMinds Webinar November 7, 2006
Agenda : Agenda SOA development and QA trends
Strategy/best practices - the Three C’s
“Headless” Testing Fundamentals
LISA Intro, and LISA WS-Testing Offer
Who are we? : Who are we? Entered the market to deliver an SOA testing solution, LISA
Current version 3.5 (2006)
Key value propositions:
Increase Test Coverage
Lower Cost per Test
Faster Discovery & Resolution Everyone should own quality™.
Platform Evolution Causes Testing Evolution : Platform Evolution Causes Testing Evolution
The Promise of SOA : The Promise of SOA Reduce integration cost
Through loosely-coupled forms of integration and industry standards
Increase asset reuse
Build new business workflows from existing Services to form composite applications
Increase business agility
Better control business process definition and management to meet customer needs
Reduce business risk
Governance, compliance, and risk reduction through increased business visibility and agility
…before SOA : Server
(database) Browser UIs Standard
enterprise apps Client
UIs …before SOA You had a standard delivery platform.
And a standard infrastructure.
SOA promises a composite workflow : Transaction service Data
warehouse BI tools Core app Partner Reseller Customer buyer Business rules Order management Outsourced
supplier Transaction provider Partner Reseller Customer
buyer Your extended enterprise Division 1 Division 2 SOA promises a composite workflow SOA was supposed to simplify extending workflows.
But heterogeneous technology is complex : Transaction service Data
warehouse BI tools Your App Workflow Channel Partner RMI objects Customer company CRM Web App Business Rules Outsourced firm Legacy App SOAP objects Web Services .NET Ordering Service Content Database Legacy App Web interface Messaging service File System Legacy Data Division2 Financials Mainframe Division 1 Your Company ESB But heterogeneous technology is complex
Why A New Strategy? : Why A New Strategy? Few changes at the database level, but multiplies issues downstream
For every middle tier change, there are hundreds of front-end impacts possible at the UI layer
Limitless front end options
Any client, multiple customers
Mix of technologies – XML/RPC, Swing, AWT, JS, DOM, now AJAX
Services consume business logic Database/ mainframe
= 1 change UI level
=1000x Middle-tier level
= 100x
Best Practices: “Three C’s” of SOA Testing : Best Practices: “Three C’s” of SOA Testing Complete Testing
Every heterogeneous layer of architecture
Test UI verify in system of record
Reuse functional test for performance testing
Collaborative Testing
Test early before UIs are created
Not just dev, business analysts and QA should verify processes
Continuously Test
Regress on existing functionality
Add testing of new services to existing testing workflows Complete >
single component all technologies Continuous >
Phased tests constant testing Collaborative >
Dev/QA test silos everyone tests, every phase
Complete: SOA Testing across every layer : Data / Applications Process and services layer Complete: SOA Testing across every layer Presentation layer Integration layer Swing CORBA ESB BPM
Typical Trading System Example : Additional Apps Now
& Future Additional Apps Now
& Future Additional Apps Now
& Future
CORBA App Typical Trading System Example
Ordering
Swing App ESB
Processing
Web App Business Process
App A Business Process App B Additional Apps Now
& Future
DB RMI Data is correlated
in here. 1 9 App Server
EJB, WS, POJO
Collaborative: SOA Requires Agile : Collaborative: SOA Requires Agile QA expands dev tests and create functional and load tests Complete Test Coverage
Lower Cost per Test
Faster Discovery & Resolution Developers unit test and jump-start QA Business Analysts outline a business process and ‘not implemented’ test cases Production test their implementation and report issues to support Support rapidly debug issues and communicates appropriately
Collaborative Testing in an SOA environment : Collaborative Testing in an SOA environment BPM Validation Orchestration Tests Service Validation Component Tests Service 3 Dev/QE Place Order Service 1 Service 3 Service 2 QE QA Continuous SOA Test Platform
Continuous: SOA is never done : Continuous: SOA is never done SOA development is constantly evolving Project-based development had a “test phase” Database Create components Create UI Acceptance Test Deploy
App Unit test Integration
Continuous: Minimize Unintended Consequences : ESB Continuous: Minimize Unintended Consequences Database Registry MDM Transactions Internal Service Legacy App Partner Service ? ? ? ? ? ? ? ? NEW
Continuous: Minimize Unintended Consequences : ESB Continuous: Minimize Unintended Consequences Database Registry MDM Transactions Internal Service Legacy App Partner Service A data service is currently failing due to a change in the internal service
Continuous: SOA at 5-9’s : Continuous: SOA at 5-9’s Business Investment Availability 100% 99.999% system
uptime Physical
Infrastructure Scalability 100% 99.999% application
availability Integrity 100% ?????% functional integrity Application
Infrastructure Business Applications Fault Tolerant HW
Diesel Power
Redundant Bandwidth App Servers
Transaction Monitors
Clustered Services SOA-aware Dev/QA Process
Comprehensive Testing Tools
Continuous Testing Service Availability to the business is the product of all three
Continuous Levels: Build & Deploy : Continuous Levels: Build & Deploy Component/ Service Teams Development QA/QE Pre-production Production product component product component Dev build test bed Staging Production Dev build test bed Continuous SOA Test Platform Deployment Level Build
Level
So how do we defeat the “Headless” dilemma? : So how do we defeat the “Headless” dilemma?
Requirements are King -- On-line Music Scenario : Requirements are King -- On-line Music Scenario Use Case: (UC1) Customer with existing account does a search for ‘Rush’, selects the Grace Under Pressure tour DVD and uses the 1-click purchase feature to buy.
Business: (BR1) We need a flexible mechanism to change these criteria at our whim. After discussion: for now partners get 20% of net proceeds on their product sold on the first $100,000 / mo, 10% thereafter.
PSR: (PSR1) Search can’t take any longer than 3 seconds for a user on a typical broadband connection.
Policy: Punt for now. This is security, interop standards, corp dev standards, etc.
Functional: (FR1) Search must include results from CDs, DVDs, Books, and Memorabilia, with CDs on the top (UC1).
Technical: (TR1) Create a search service that aggregates CD, DVD, Book, and Mem data (FR2). (TR2) Search service execution must be under 2 seconds (PSR1).
Service Level Guidelines : Service Level Guidelines Service Validation Component Tests Service 3 Dev/QE UC & BR: Validation mechanism
PSR: Load/performance test at the service level
FR: Likely 80% of your requirements can be nailed right here
TR: Dev test at the component level UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Orchestration Level Testing Guidelines : Orchestration Level Testing Guidelines UC & BR: Not usually
PSR: Often the best way to prove PSR
FR: Invoke a behavior, validate outcomes
TR: Simulate unstable/unavailable services Orchestration Tests Place Order Service 1 Service 3 Service 2 QE QA UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Composite/BPM Level Testing Guidelines : Composite/BPM Level Testing Guidelines UC: Primary focus, performs a cause that we validate at the service/component level
BR: Many times same as UC, sometimes not relevant
PSR: A few PSR scenarios proved at this level, but continuous testing is the key here
FR & TR: Hopefully very few BPM Validation UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements
Typical SOA Test Map -- Dramatically Simplified : Typical SOA Test Map -- Dramatically Simplified Presentation
Layer Content
Mgmnt Offers &
Specials Search Order
Mgmnt Commission
Rules Svc CRM System Media Delivery CRM
Facade Batch Feeds UC1 UC1 UC1 UC1: Search & Buy
Typical SOA Test Map -- Dramatically Simplified : Typical SOA Test Map -- Dramatically Simplified Presentation
Layer Content
Mgmnt Offers &
Specials Search Order
Mgmnt Commission
Rules Svc CRM System Media Delivery CRM
Facade Batch Feeds BR1* BR1 BR1: Commissions
Typical SOA Test Map -- Dramatically Simplified : Typical SOA Test Map -- Dramatically Simplified Presentation
Layer Content
Mgmnt Offers &
Specials Search Order
Mgmnt Commission
Rules Svc CRM System Media Delivery CRM
Facade Batch Feeds TR2* PSR1 PSR1: 3 sec search
Typical SOA Test Map -- Dramatically Simplified : Typical SOA Test Map -- Dramatically Simplified Presentation
Layer Content
Mgmnt Offers &
Specials Search Order
Mgmnt Commission
Rules Svc CRM System Media Delivery CRM
Facade Batch Feeds FR1: Search is aggregate, CDs top results FR1
Typical SOA Test Map -- Dramatically Simplified : Typical SOA Test Map -- Dramatically Simplified Presentation
Layer Content
Mgmnt Offers &
Specials Search Order
Mgmnt Commission
Rules Svc CRM System Media Delivery CRM
Facade Batch Feeds TR1 & 2 TR2: 2 sec search TR1: Create search on 4 repos TR1
The Three Dimensions of Quality maturity : The Three Dimensions of Quality maturity
LISA meets SOA Testing Challenges : LISA meets SOA Testing Challenges LISA is Complete - Test heterogeneous SOA
LISA can test every tier of an application, regardless of location
LISA can test every service technology with one tool, one test
LISA’s framework approach creates bridges to legacy services that means ‘you can get there from here’
LISA is Collaborative – Everyone owns quality
LISA shines at testing all components before UI even exists, as well as integrating with the team’s ALM lifecycle and process
Unit, regression, system, load, and monitoring in one tool
Leverages diverse skill sets in Dev, QA/QE, and BA roles
LISA is Continuous – Enables Agile SOA
Tests constantly evolving, interdependent systems during and after deployment
Traps for the “what-if” unintended system-wide consequences of making changes or corrections to components.
LISA WS-Testing $5,000 Off for next 50k users : LISA WS-Testing $5,000 Off for next 50k users LISA WS-Testing Edition is THE tool for Web Services testing
All our web services goodies are in there:
WSDL-based, point and click
WS-I Interop testing
Functional testing and service simulation
UI and Command Line execution
Thank you for your interest! : Thank you for your interest! ? ? ? iTKO Web Site Resources
URL: http://www.itko.com or http://www.ws-testing.com
White papers
Support FAQs, Tech Notes, and Forums (once registered)
Download WS-Testing or start a larger evaluation
More from me
See above
Blog: http://itko.blogspot.com
Email: Start with info@itko.com; I’m john@itko.com