IBM Software Development PlatformSoftware QualitySales 201 Training: IBM Software Development Platform Software Quality Sales 201 Training 2005


Agenda: Agenda Strategy IBM Software Development Platform e-Business on demand Overview of IBM Software Quality products and offerings IBM Rational Functional Tester IBM Rational Robot IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational PurifyPlus IBM Rational Test RealTime Exercises Additional References


Imperatives in Today’s World: Innovation & Productivity: Imperatives in Today’s World: Innovation & Productivity Improving the productivity of resources deployed. Innovating the business to capture new value. IT Business


IT is the lever to maximize flexibility and responsiveness while containing cost: IT is the lever to maximize flexibility and responsiveness while containing cost An on demand operating environment is an integrated infrastructure aligned to business goals and processes in a resilient and secure manner On Demand Operating Environment On Demand Operating Environment All designed with an architecture that allows you to manage services as components Results: Simplification and Optimization of IT to meet the needs of the business responsively Business-Driven Development Infrastructure Management Integration


The challenge: Poor visibility, lack of cohesion across business and technology domains: Operations Application Developers Business Analysts Business View Poor visibility and governance over IT investments Lack of actionable information Blind decision-making Application Development View The challenge: Poor visibility, lack of cohesion across business and technology domains Operations View Overwhelming complexity Relentless time-to-market pressure Uncontrolled change Inadequate service levels Inability to rapidly deploy applications Complex, multi-tier operating environments


The solution: Breaking down the silosA shared view of the development lifecycle: Business Analysts Operations Application Developers Business View Clear view of technology ROI Top-down and bottom-up visibility into technology projects Objective decision-making support Application Development View The solution: Breaking down the silos A shared view of the development lifecycle Operations View Rapid application development and deployment Improved collaboration Asset reuse Improved service and quality compliance Predictable deployments Accelerated diagnosis and repair


Transform and simplify software development: Transform and simplify software development Higher productivity - 50% + increase in developer productivity Improved quality - 80% fewer bugs Greater predictability Business-driven development Align Technology and Business priorities Improve efficiency and responsiveness Create innovative products An integrated approach to software development that aligns line-of-business, development and operations teams to improve business performance Software development becomes a driver of competitive advantage


Discover, develop, and deploy assets: Operations Business Discover, develop, and deploy assets Discover business & technology assets Business priorities Requirements Middleware and software assets Develop at the speed of business Rapid application development Model-driven architecture Asset-based development Direct-to-middleware productivity Deploy to closed-loop environments Automated applications deployment Streamlined composite application management Direct-to-operations productivity Prioritize Plan Manage Measure Optimize Iterate


The business-driven development lifecycle: The business-driven development lifecycle Executive Project Manager Business Development Operations Manage change & assets Prioritize Plan Manage Measure Optimize Iterate Govern


The IBM Software Development PlatformA complete, open, modular, and proven solution: Analyst Architect Developer Tester Rapidly construct, transform, integrate and generate code Design, create, and execute tests Model, simulate, assemble, and monitor processes Visually model applications and data Provision, configure, tune and troubleshoot applications Follow a common process Manage and measure projects and portfolios Manage requirements Manage change and assets Manage quality Project Manager Executive Align investments with business objectives Analyze and monitor project portfolios Deployment Manager The IBM Software Development Platform A complete, open, modular, and proven solution


Agenda: Agenda Strategy IBM Software Development Platform e-Business on demand Overview of IBM Software Quality products and offerings IBM Rational Functional Tester IBM Rational Robot IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational PurifyPlus IBM Rational Test RealTime Exercises Additional References


Focus # 1: Discover AssetsOptimize Business Investments: Focus # 1: Discover Assets Optimize Business Investments Identify and analyze Business processes Requirements Software Assets Packaged applications Existing applications Middleware Impact of potential change Business process optimization is the #1 priority in 2004 — Information Week Study 2Q04 — 400 business-technology executives DISCOVER IBM SOFTWARE DEVELOPMENT PLATFORM


Focus # 2: Develop AssetsImprove Time-to-Value: Rapid application development Model-driven architecture Asset based development Direct-to-middleware productivity Service-oriented architecture Tightly integrated run-time test and deploy Anytime, anywhere, team collaborative environments Focus # 2: Develop Assets Improve Time-to-Value Dramatically accelerate the development cycle IBM SOFTWARE DEVELOPMENT PLATFORM


Focus # 3: Deploy AssetsDeliver Business Results: Focus # 3: Deploy Assets Deliver Business Results Unify development and operations teams Accelerate deployment and provisioning Automate applications monitoring Monitor business performance and analyze ROI Verify and manage change Enable closed-loop deployment IBM SOFTWARE DEVELOPMENT PLATFORM


A team-based approachAll tools and team members support all development imperatives: A team-based approach All tools and team members support all development imperatives Analyst Architect Developer Tester Rapidly construct, transform, integrate and generate code Design, create, and execute tests Model, simulate, assemble, and monitor processes Visually model applications and data Provision, configure, tune and troubleshoot applications Follow a common process Manage and measure projects and portfolios Manage requirements Manage change and assets Manage quality Project Manager Executive Align investments with business objectives Analyze and monitor project portfolios Deployment Manager


IBM Software Quality Offerings: Capabilities Product Description Runtime analysis Component, functional and performance testing Cross-platform test execution support Test & planning reporting IBM Software Quality Offerings Ensures reliability, functionality, scalability Reduces cycle time Accommodates multiple skill sets, roles Shares responsibility for quality across team


From the Industry AnalystsIBM Rational Software Quality Products: From the Industry Analysts IBM Rational Software Quality Products Ovum "IBM Rational has been focusing on strategic integration of the ASQ* toolset with the complete IBM Software Development Platform, including the use of the Eclipse framework for software tool integration. We expect over the next 18 months IBM Rational's ASQ tools will again provide best-of-breed, single-point test tools, which will excel by operating within a formidably integrated environment across the development, test and delivery lifecycle. As a bonus, the fact that the platform is based on Eclipse open standards means that you will not be locked into IBM Rational-only tools.“ Source Paul Herzlich (Ovum) 08.04 - Ovum Evaluates: Software Testing Tools IDC - Noted Rational’s “extremely well established and complementary product set to IBM’s.” Source: Worldwide Distributed Automated Software Quality Tools 2004-2008 Forecast and 2003 Vendor Shares M. Webster (IDC) 08.04 * Automated Software Quality


Agenda: Agenda Strategy IBM Software Development Platform e-Business on demand Overview of IBM Software Quality products and offerings IBM Rational Functional Tester IBM Rational Robot IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational PurifyPlus IBM Rational Test RealTime Exercises Additional References


Challenges of Functional Testing: Challenges of Functional Testing Control cost of test automation Inexperienced testers face steep learning curve for tools Test maintenance grows with project Testing Tools Lack Flexibility Power users hands are tied by tool limitations “I need one tool for both my expert and novice testers.” “Our script language limits our ability to overcome some automation obstacles.”


Functional Testing with IBM Rational Functional Tester Test automation for the novice and the professional: IBM Rational Functional Tester Automated regression testing for Web, J2EE, .Net and terminal-based applications Key Benefits Minimize test maintenance with scripts resilient to application changes Wizard enhanced automation to speed test creation for the new user Powerful scripting language and IDE for the professional tester Supports Team oriented parallel development Functional Testing with IBM Rational Functional Tester Test automation for the novice and the professional


Challenge 1: Controlling the Cost of Automation: Challenge Overcome the complexity of test automation to ensure teams spend more time on high value quality assessment rather than on low return test maintenance. Resolution Provide technologies to enable novice testers to craft complex tests with no programming Data driven wizard Dynamic data verification Reduce script maintenance Wizard driven global updates to centralized object map Challenge 1: Controlling the Cost of Automation


Challenge 2: Tools Lack Flexibility: Challenge Fulfill both standard test needs as well as expert requirements to ensure usage limited only by the imagination of the project team. Resolution Provide two test script development languages Java Visual Basic .NET Give professional testers the flexibility to overcome testing challenges in the language of their choice Challenge 2: Tools Lack Flexibility


Challenge 3: Test Team Integration: Challenge 3: Test Team Integration Challenge Accommodate the requirements of both dedicated, independent QA teams as well as integrated project teams to ensure high operational integrity. Resolution Provide version control support in the box with ClearCase LT Provide integrations for standard version control features Check In/Out Compare with Previous Versions Show change history


What is Automated Testing?: What is Automated Testing? Using automation tools to record actions taken against an application. Test scripts are derived from these recordings which are then used for subsequent test execution. Record Enhance Execute


Challenge of Manual Testing with Short Test Cycles: Rerun the first tests and... Time Tests Challenge of Manual Testing with Short Test Cycles Manual testing alone can’t keep pace with the rapid development of applications Quality will be sacrificed to keep on schedule


Effective Test AutomationCreating an automated test is a three steps process: Effective Test Automation Creating an automated test is a three steps process Script Recording Capture user interactions with the system under test Record


Effective Test AutomationCreating an automated test is a three step process: Effective Test Automation Creating an automated test is a three step process Script Recording Capture user interactions with the system under test Script Enhancing Modify scripts for situation specific challenges Enhance Record


Effective Test AutomationCreating an automated test is a three step process: Execute Effective Test Automation Creating an automated test is a three step process Script Recording Capture user interactions with the system under test Script Enhancing Modify scripts for situation specific challenges Script Execution Ensure reliable playback for local or remote execution Enhance Record


IBM Rational Functional Tester Overview: IBM Rational Functional Tester Linux-based test authoring ClearCase LT IBM Rational Functional Tester Overview Manual Tester Eclipse Java Development Tools Wizard and table-assisted data-driven tests Tests Java, .NET, Web and terminal emulator-based apps Java and VB.NET-based test scripts Test Script Language User has choice of two test script languages – Java and VB.NET Use of Java implies editing and debugging in Eclipse/RAD/RSA-based editor Use of VB.NET implies usage of Microsoft VS.NET-based editor/debugger Identical capabilities in both editors GUI Technologies Automates regression testing of Java, .NET, Web and terminal emulator-based applications Data-driven Testing Simplified data-driven testing enabled by wizards and Excel-like data tables Improved Linux support Users can author and modify test scripts on Linux (no script recorder) Scripts can also be executed on Linux Eclipse availability When using Java scripting, runs inside Eclipse/RSA/RAD shell Full Eclipse JDT features are accessible Accessible to developers from within their IDE Manual Tester in the box Customers receive one Manual Tester license per Functional Tester license Improve return on manual testing efforts while adopting automated test methods ClearCaseLT in the box Supports parallel development of manual tests Enables remote test access for distributed teams Automated functional regression testing


IBM Rational Functional Tester New Positioning : IBM Rational Functional Tester New Positioning Lowering the Cost of Functional Testing Supports Integration of Test Team with Rest of the Project Team Offers Extensive Customization Options Broader environment support Adds support for testing VS.NET Winforms applications Adds ability to edit and playback scripts on Linux (no recording) Advances in ease of use Choice of script languages Wizard for data-driven testing Improved object map maintenance capabilities Ships with full-use copies of IBM Rational Manual Tester and IBM Rational ClearCaseLT and customers can download Rational TestManager What’s New in Atlantic


Rational Functional Tester – Key Features and Benefits : Rational Functional Tester – Key Features and Benefits Test Java, Web & VS.NET Winform apps Lower TCO with a single tool supporting a broad array of industry standard application development technologies Java or VB.NET scripting language Easier adoption of the tool – Teams select the language they are most familiar with. Helps sell to Microsoft customers Add-on for Terminal-based application testing Central QA and IT teams can support client-server, web and legacy application testing with single tool Wizards for data-driven testing Eliminates manual coding required to vary data in test script Expanded dynamic verification Ability to use regular expressions in data pools created with new wizard for data-driven testing Advanced Object Map Maintenance Global Find and Modify tools further reduce test maintenance


Rational Functional Tester – Key Features and Benefits : Rational Functional Tester – Key Features and Benefits Version Control “in-the-box” Collocated teams can support multiple versions of test scripts and parallel development paths with no additional tools to buy. Distributed teams can upgrade to ClearCase to facilitate remote testing, whether off-site or off-shore. Ability to install, edit and execute tests on Linux Developers and testers in diverse, heterogeneous environments benefit from a common toolset


Summarizing Key Features in IBM Functional Tester: Broad Environment Support Test Web, Java. .net and terminal-based applications Create data driven tests without coding Static data and properties verification Dynamic data validation without coding Flexible and Extensible Scripts Flexible coding language Powerful, professional debugger Object map editing flexibility Version control of scripts Script Execution Options Reliable playback with ScriptAssure Remote and local playback on various platforms Summarizing Key Features in IBM Functional Tester


Reduce Test Script MaintenanceReliable Playback with Script Assure: Reduce Test Script Maintenance Reliable Playback with Script Assure Version 1.0 Version 2.0 No User Intervention Required With ScriptAssureTM


Case StudySAS Institute, Inc.: Case Study SAS Institute, Inc. Challenge Improve communication between developers and testers Solution IBM Rational Functional Tester Enables best-of-breed automated functional and regression testing from within an Eclipse environment, including testing from within IBM WebSphere Studio and Rational XDE Developer Benefits Productivity and quality improvements Efficiency has gone up and the quality of software delivery has gone up Bring quality issues to the surface faster fix these issues and deliver software on time Minimize script maintenance through resilient reusable test scripts Achieved better results by using a consistent methodology More information http://www-306.ibm.com/software/success/cssdb.nsf/CS/JENS-5WVSPY?OpenDocument&Site=software


Competitive AdvantagesIBM Rational Functional Tester: Competitive Advantages IBM Rational Functional Tester Part of a proven, complete, open and modular software development platform Single vendor solution Integration with other development tools Provides a shared development/test environment and common language on application development projects using Eclipse or Visual Studio.net Choice of a Java or Visual Basic.net scripting language Allows testers to leverages existing Java or VB.net skills on program teams Product wizards to help build complex tests without coding Provides a single tool to test Java, Web, .net and terminal-based applications ScriptAssure technology for building robust, resilient test scripts


ResourcesIBM Rational Functional Tester: Resources IBM Rational Functional Tester IBM Rational Functional Tester Web page http://www-306.ibm.com/software/awdtools/tester/functional/index.html Rational Functional Tester evaluation copy http://www14.software.ibm.com/webapp/download/preconfig.jsp?id=2005-01-19+04%3A01%3A18.402144R&S_TACT=104CBW71&S_CMP= Introduction to IBM Rational Functional Tester http://www-128.ibm.com/developerworks//rational/library/04/r-3228/ developerWorks testing resources http://www-130.ibm.com/developerworks/rational/products/xdetester Success stories http://www.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=rational Training http://www-306.ibm.com/software/sw-training/ Events http://www-306.ibm.com/software/sw-events/


Agenda: Agenda Strategy IBM Software Development Platform e-Business on demand Overview of IBM Software Quality products and offerings IBM Rational Functional Tester IBM Rational Robot IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational PurifyPlus IBM Rational Test RealTime Exercises Additional References


Effective Test AutomationCreating an automated test is a three step process: Execute Effective Test Automation Creating an automated test is a three step process Script Recording Capture user interactions with the system under test Script Enhancing Modify scripts for situation specific challenges Script Execution Ensure reliable playback for local or remote execution Enhance Record


Recording Scripts: Recording Scripts Create a Test Script that captures all system-user interaction Test scripts are recorded on the fly, as user navigates application Verification points are inserted to validate system response Key considerations when recording scripts Environment Support Flexible verification ability


Environment Support for Recording Scripts: Environment Support for Recording Scripts IBM Rational Robot has special environment support for key development languages Java, HTML, .NET, Visual Basic, MFC PowerBuilder, Delphi, Oracle Forms Robot also has a general recording engine which can be used against other windows and terminal emulation environments


Verification Points in Recording Scripts: Verification Points in Recording Scripts Automated Validation Robot captures data and properties that can be invisible to users During script execution, current results are compared to stored baselines Discrepancies are flagged and reported to user 13 different verification points are available to validate system behavior including Rational SiteCheck for web site link checking


Enhancing Scripts: Enhancing Scripts Enhancing Scripts with basic coding extends their value and reach SQA Basic code is added to perform a variety of functions Typical Modifications: Conditional branching, datapooling, refactoring Key considerations when enhancing scripts Flexible coding language Extensibility of language and playback engine


Flexible Coding Language for Enhancing Scripts: Flexible Coding Language for Enhancing Scripts IBM Rational Robot uses the SQA Basic Language Uses Visual Basic syntax Augments VB commands with test specific functions Easy to read language very suitable to those will little programming experience Flexible power to enhance scripts Programmatic access to all GUI objects Datapool facility enables data driven tests Programmatic access to Windows API Programmatic access to COM objects


Solving Custom Problems with Extensibility: Solving Custom Problems with Extensibility Robot has an extensibility interface to allow support for custom controls Development partners have been trained Can manufacture support for ActiveX, .NET Windows Forms, Java & Win32 including MFC and ATL GUI controls Eliminates the work around Allowing testers to focus on the task of testing


Executing Tests: Executing Tests Scripts are executed, discrepancies are noted Scripts are executed and test logs created Test logs are highlight differences between actual and expected results Key considerations when executing scripts Multiple execution points Coordination of scripts


Automated Testing: Automated Testing Script Execution Options Scripts can be run from Robot GUI Scripts can be executed from Command line Scripts can be run from TestManager Offers ability to run scripts on remote machines


Effective Test Automation: Effective Test Automation Powerful and flexible scheduling TestManager will coordinate timing and dependencies Schedule execution on remote computers from central control station TestManager is configuration aware, i.e.: Tests designated as Windows XP will only be run on XP machines


Summarizing Key Features in IBM Rational Robot: Broad Environment Support Support for all major development environments Flexible and Extensible Scripts Simple, yet powerful, scripting language for enhancing scripts Support available for custom control Script Execution Options Run scripts from Robot, TestManager or command line Summarizing Key Features in IBM Rational Robot


Case StudyCorporate Express : Case Study Corporate Express Challenges Cut testing time and challenges in manual testing process manual regression testing found to be tedious, time-consuming and error-prone Improve repeatability of tests Improve confidence in software quality at release Solution Automated the manual process with IBM Rational Robot Benefits Cut testing time from days to weeks, over 75% 31/2 weeks of testing in 3 days Team confidence that each regression test is executed accurately and with a consistent process from one release to the next More information http://www-306.ibm.com/software/success/cssdb.nsf/CS/JENS-5WPKR2?OpenDocument&Site=software.


Resources: Resources IBM Rational Robot Web page http://www-306.ibm.com/software/awdtools/tester/robot/index.html IBM developerWorks page for IBM Rational Robot http://www-136.ibm.com/developerworks/rational/products/robot Success stories http://www.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=rational Training http://www-306.ibm.com/software/sw-training/ Events http://www-306.ibm.com/software/sw-events/


Agenda: Agenda Strategy IBM Software Development Platform e-Business on demand Overview of IBM Software Quality products and offerings IBM Rational Functional Tester IBM Rational Robot IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational PurifyPlus IBM Rational Test RealTime Exercises Additional References


What is Manual Testing?: What is Manual Testing? Using human labor to validate the business use cases and fitness for use of an application, system or device Includes labor to plan, design, execute and evaluate the status of the application or device under test Experience indicates over 75% of the testing done today is performed manually


Why Teams use Manual Testing?: Why Teams use Manual Testing? Recruitment of subject-matter experts and end-users to test Poor or non-existent test automation-ready interfaces Strict regulatory or process requirements aren’t met with automation tools


Rational Manual Tester  Positioning: Rational Manual Tester Positioning Advances Manual Testing Simplifies Through Assistance Satisfies Diverse Needs Supports manual testing done by analysts, subject-matter experts and technical/non-technical testers Develop detailed, easy to follow manual tests Minimize test maintenance and ensure test consistency Reduce human error during data entry and data verification Support diverse usage models and processes Ships with full copy of ClearCaseLT for version control of tests Key Capabilities


Challenges of Manual Testing: Challenges of Manual Testing Labor Intensive Too much time is spent creating and maintaining manual test documentation “More time is spent maintaining test documentation than on executing tests.” “We struggle to document even a minimum set of tests.”


Challenges of Manual Testing: Challenges of Manual Testing Labor Intensive Too much time is spent creating and maintaining manual test documentation Unreliable Inadvertent human-error often skews test results “Our results are unreliable due to human error.” “Our tests aren’t detailed enough for the testers to consistently execute them.”


Challenges of Manual Testing: Challenges of Manual Testing Labor Intensive Too much time is spent creating and maintaining manual test documentation Unreliable Accidental human-error often skews test results Resistant to Change Each project has diverse processes, tester skill sets and types of applications and devices under test “Each project has different standards, which are hard to manage manually.” “No single tool or process works for the different software and hardware systems we must test.”


Challenge 1: Labor Intensive: Challenge 1: Labor Intensive Challenge Creating and maintaining manual tests is labor intensive and difficult with tools such as Microsoft Word or Excel. Resolution Implement a solution that promotes the best practice of modular tests which can share common content Leverage existing Word or Excel test documents to quick-start test development


Challenge 2: Unreliable: Challenge Human error often skews tests results Resolution Assist testers with the tasks most prone to error – data entry and data verification Build manual tests that are clear, concise and easy for the tester to follow Challenge 2: Unreliable


Challenge 3: Resistant to Change: Challenge Each project has diverse processes, tester skill sets and types of applications and devices under test Resolution Provide a solution that testers of all skills levels can leverage Customize the tool to fit each project’s process metrics and values Leverage a solution that is independent of the software or hardware under test Challenge 3: Resistant to Change For all skill levels For any target of test For any process


IBM Rational Manual Tester Overview: IBM Rational Manual Tester ClearCase LT Test Import / Results Export IBM Rational Manual Tester Overview Data entry & validation assistance Custom test fields Test step reuse library Rich test editor Customizable Add new data fields to individual test steps Apply internally-defined naming conventions Tool Assistance Automated data entry Automated data comparison and results storage Reduces opportunity for manual error during test execution Import/Export Imports manual test source files from Microsoft Word, Microsoft Excel and Rational TestManager Exports test results to CSV format for additional analysis Manual test authoring and execution ClearCaseLT in the box Supports parallel development of manual tests Enables remote test access for distributed teams Rich Text Editor Modify test font, size, color; attach images and files Supports creation of both test steps and verification points Ensures clarity of test step direction and consistency of test execution Reuse Library Library for common test procedures Drag-and-drop construction of new tests Automatic global update when modifying any one instance of linked content


Rational Manual Tester – Key Features and Benefits : Rational Manual Tester – Key Features and Benefits Rich Text Editor Reduces the ambiguity that often plagues other methods used to define and deliver test step directions Import From Word, Excel or TestManager Easier adoption of the tool Multiple Validation Points Allows test results to be reported and analyzed at either a high level or a granular level Reuse Palette Promotes shared content which results in time & cost saving in test maintenance & greater control over test quality and consistency Assisted Data Entry and Data Verification Reduce the likelihood of human error in the areas most prone to mistakes Custom Data Fields Easily adapts to existing processes and metrics collection standards


Rational Manual Tester – Key Features and Benefits : Rational Manual Tester – Key Features and Benefits Translucent Execution UI The translucent window used to display test instructions makes test completion faster by eliminating the need to move the test instruction window in order to see the application under test Supports Team Use Supports individuals, teams, and mobile works through private and shared test data stores – ensuring test assets remain easily accessible and reportable – ClearCaseLT in the box Generates Spreadsheet-Ready Results Testers can use any reporting tool to analyze results - ensuring that Rational Manual Tester can easily work with a team’s processes and standards for metrics collection and reporting


Summarizing Key Features in Manual Tester: Summarizing Key Features in Manual Tester Advances manual testing Rich text editor for test authoring Test step reuse palette Test script import from Microsoft Word or Excel Satisfies diverse needs Supports usage by distributed teams Generates spreadsheet-ready results data Multiple validation points Custom data fields Simplifies through assistance Assisted data entry and data verification during test execution


ResourcesIBM Rational Manual Tester: Resources IBM Rational Manual Tester IBM Rational Manual Tester Web page http://www-306.ibm.com/software/awdtools/tester/manual/index.html Rational Manual Tester evaluation copy http://www14.software.ibm.com/webapp/download/preconfig.jsp?id=2005-01-16+03%3A32%3A04.405210R&S_TACT=104CBW71&S_CMP=&s= Rational Manual Tester evaluation guide http://www-106.ibm.com/developerworks/edu/i-dw-r-manualtest-i.html Article A simple tool to manage manual testing http://www-128.ibm.com/developerworks/rational/library/04/r-3232/ developerWorks testing resources http://www-128.ibm.com/developerworks/rational/products/manualtester/ Success stories http://www.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=rational Training http://www-306.ibm.com/software/sw-training/ Events http://www-306.ibm.com/software/sw-events/


Agenda: Agenda Strategy IBM Software Development Platform e-Business on demand Overview of IBM Software Quality products and offerings IBM Rational Functional Tester IBM Rational Robot IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational PurifyPlus IBM Rational Test RealTime Exercises Additional References


IBM Rational Performance Tester Overview: IBM Rational Performance Tester Graphical test scheduler and user profiler ClearCase LT IBM Rational Performance Tester Overview Eclipse Java Development Tools High scalability hyper-threaded execution architecture Visual test construction and editing HTTP/S load testing Visual Test Editor Test viewer requires no coding for test modification and execution Automated identification and variation of dynamic data Insertion of Java code is supported for advanced customization needs New load testing technology New load test recorder and execution engine Supports HTTP/S; additional protocol support (e.g. ERP) planned for Q2 2005 Windows and Linux load generation support High scalability High scalability, low footprint due to hyper-threaded architecture Significant improvement over Robot technology Test scheduler Graphical user profiling and transaction modeling Eases modeling of user populations Platform Support Multiple platform support for load generation Eclipse-based Runs inside Eclipse/RSA/RAD shell Accessible to developers from within their IDE Load and performance testing ClearCaseLT in the box Supports parallel development of manual tests Enables remote test access for distributed teams


IBM Rational Performance Tester New Positioning: IBM Rational Performance Tester New Positioning Built for Day One Productivity Lowers Cost of Performance Testing Provides Advanced Data Capabilities Completely re-architected performance testing solution for testing complex web application Designed for novice and advanced performance testers Real-time monitoring for immediate feedback during test execution Easy to build data driven tests Eclipse-3 integration What’s New in Atlantic


Rational Performance Tester – Key Features and Benefits : Rational Performance Tester – Key Features and Benefits Visual test editor Easy for a novice to understand, with drill-down to detailed data for more expert users. Simplifies test creation and maintenance Automatic identification of variable data Allows tests to automatically re-use data sent by server and to prepare for data-driven testing Built-in datapool editor for data-driven testing Easily vary data for more realistic load tests Rich scheduling editor Novice users can easily assemble and schedule tests Eclipse based Provides a single environment for development as well as testing activities


Rational Performance Tester – Key Features and Benefits : Rational Performance Tester – Key Features and Benefits Java execution engine Tests may call custom Java provided by the user. Provides highly-customizable and flexible performance testing Real-time reporting Know immediately about performance problems High Scalability Low memory and processor footprint required to generate large multi-user loads. Allows larger tests with less hardware Remote system monitoring Collect and visualization of remote server information. Find resource constraints that cause poor performance


ResourcesIBM Rational Performance Tester: Resources IBM Rational Performance Tester IBM Rational Performance Tester Web page http://www-306.ibm.com/software/awdtools/tester/performance/index.html Success stories http://www.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=rational Training http://www-306.ibm.com/software/sw-training/ Events http://www-306.ibm.com/software/sw-events/


Agenda: Agenda Strategy IBM Software Development Platform e-Business on demand Overview of IBM Software Quality products and offerings IBM Rational Functional Tester IBM Rational Robot IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational PurifyPlus IBM Rational Test RealTime Exercises Additional References


The Problem with Trying to Pinpoint Bugs: The Problem with Trying to Pinpoint Bugs Crash Crash


What Makes Bugs Difficult to Find?: What Makes Bugs Difficult to Find? Many memory-related bugs have no immediate visible symptoms Often the symptoms are difficult, if not impossible, to reproduce Manually tracing symptoms back to problems in the source code can be a daunting task


Why Is My Program So Slow?: Why Is My Program So Slow? Do you deliver slow code that’s “good enough?” Is it too difficult to find performance problems? Can you get repeatable, useful performance data? Can you quantify the effect of performance fixes? Does your performance profiling data help you manage your development project? What about Predictability at runtime?


Code Coverage Challenge: Code Coverage Challenge Can you guarantee the code that you deliver is reliable? How complete is your test suite? Are you sure that you have tested the entire application? Can you understand the summary of several tests? How Reliable is Untested Code?


IBM Rational PurifyPlus: Developing Fast, Reliable Code: Instantly become more productive IBM Rational PurifyPlus: Developing Fast, Reliable Code To develop fast, reliable code, you need a tool that Automatically pinpoints hard-to-find bugs Highlights performance bottlenecks Keeps you from shipping untested code You need a multi-platform and multi-language tool for Unix, Windows, Linux, Java, .NET, VB6, C/C++, and more IBM Rational PurifyPlus combines three industry leaders into one box: IBM Rational Purify IBM Rational Quantify IBM Rational PureCoverage The x-ray for software 


Detect Memory Problems: Rational Purify: Detect Memory Problems: Rational Purify Automatically identifies the problem while the application is running! Finds errors in third-party code, even without source Bugs in libraries that you deliver are still bugs in your application! No recompile required Much faster and easier to use Incorporate into your existing project quickly and easily Test what you are delivering No need to maintain multiple source baselines QE knows that they are testing what you are delivering Detects memory corruption and memory leaks


Improve Code Performance: Rational Quantify: Improve Code Performance: Rational Quantify Leading product that provides repeatable performance data Build better quality C/C++, Java, Visual Basic 6.0, and .NET software Get your team solving the correct problems Fast and easy to use Provides concise, graphical views of performance data “River of Time” – a straightforward way to view and understand performance profiling data Easily integrates into your nightly builds Collects performance data from code you intend to deliver No need to maintain multiple source baselines


Identify Untested Code: IBM Rational PureCoverage: Identify Untested Code: IBM Rational PureCoverage Helps developers deliver higher quality code You know the unit testing is complete Helps identify “dead code” or unexercised paths in legacy code Helps improve your QA process Easily see what has and has not been tested Validate the quality and completeness of your test suite Easy to learn and easy to use out of the box With Rational PureCoverage, you know that you are covered! Avoid shipping untested code


IBM Rational PurifyPlus: IBM Rational PurifyPlus Key components of IBM Rational PurifyPlus Provides run-time data collection and analysis capabilities Key for proactive debugging Supports Unix, Windows, Java, .NET and Linux environments Linux RedHat C/C++ Windows VB6 C/C++ HP-UX Full 64bit C/C++ Solaris Java C/C++ SGI-IRIX Full 64bit C/C++ Not Available Yet Quantify PureCoverage Full 64bit Java .NET Full 64bit


IBM Rational PurifyPlus Summary: Instantly become more productive IBM Rational PurifyPlus Summary To develop fast, reliable code, you need IBM Rational PurifyPlus that: Automatically pinpoints hard-to-find bugs Highlights performance bottlenecks Keeps you from shipping untested code Powerful tool for developers and testers Strong ROI when used as “medicine” Even better ROI when used as “vitamin” Used as part of the development process


Case Study Ensemble Systems: Case Study Ensemble Systems Challenge Ensemble Systems was contracted to develop an easy-to-maintain software system to perform data analysis twice as fast as an existing solution, while significantly reducing analysis errors. Solution Rational Rose, Rational Purify and Rational Quantify able to effectively model the system, detect and fix memory leaks and enhance the overall performance of the application under development. Benefits Reduced the time the system required to analyze defects by 85%, and reduced the system's error rate by 90%. Quickly pinpointed memory leaks and identified opportunities to improve performance More Information http://www-306.ibm.com/software/success/cssdb.nsf/CS/JENS-5PFQXR?OpenDocument&Site=software


Case StudySAS Institute, Inc. : Case Study SAS Institute, Inc. Challenge Improve software quality Solution Rational Purify and Rational Quantify to help find performance bottlenecks and memory problems in the software Integrated Rational Purify and Rational Quantify into testing process through product APIs Benefits Achieve higher level of confidence in the quality of software releases More information http://www-306.ibm.com/software/success/cssdb.nsf/CS/JENS-5WVSWK?OpenDocument&Site=software


Competitive AdvantagesIBM Rational PurifyPlus: Competitive Advantages IBM Rational PurifyPlus Part of a family of tools that comprise the IBM Software Development Platform Integration across tools Can form a more complete development and testing solution Multilanguage support Java, but also Visual C++, Visual Basic, Visual Basic.NET and Visual C#.NET in one package Module and test level coverage Offline analysis of data Merging test results


ResourcesIBM Rational PurifyPlus: Resources IBM Rational PurifyPlus IBM Rational PurifyPlus Web page http://www-306.ibm.com/software/awdtools/purifyplus/ developerWorks: Rational http://www.ibm.com/developerworks/ Getting Started Guides Training – Web-based or Instructor-led Articles - whitepapers – forums Success stories http://www.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=rational Training http://www-306.ibm.com/software/sw-training/ Events http://www-306.ibm.com/software/sw-events/


Agenda: Agenda Strategy IBM Software Development Platform e-Business on demand Overview of IBM Software Quality products and offerings IBM Rational Functional Tester IBM Rational Robot IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational PurifyPlus IBM Rational Test RealTime Exercises Additional References


IBM Rational Test RealTime Fix your code before it breaks: IBM Rational Test RealTime Fix your code before it breaks Rational Test RealTime is a cross-platform testing solution designed for those who write code for real-time and embedded software products Automates testing, analysis and bugs solving early during development cycle Provides a low-overhead technology for enabling target-independent tests Extends model-driven development to include runtime analysis capabilities Fix the defects, enhance your tests and move on!


Embedded and real-time software testing challengesEmbedded and real-time systems are complex by nature: Embedded and real-time software testing challenges Embedded and real-time systems are complex by nature Application Complexity Strong timing constraints Low memory footprints Concurrent/Distributed/Networked Environment Complexity Multiple RTOS/IDE/Chips vendors Limited host-target connectivity Low built-in debugging capabilities Process Complexity Requirements Design translation errors Difficult to maintain Poor performance


Embedded and real-time software testing challengesIt’s hard to set up and deploy an effective testing framework: Embedded and real-time software testing challenges It’s hard to set up and deploy an effective testing framework Are you frustrated by manual testing because of its tendency to be Labor-intensive? Resistant to reuse? Are you concerned that test creation and execution will Pull you out from your comfort zone? Add overhead to an already crowded schedule? Require to much efforts to ensure quality at both source code and system level? Are you convinced that your embedded environment is too specific for a commercial testing solution because of Design constraints? Heterogeneous build/execution tools? Strict testing techniques and activities recommended by certification standards?


Embedded and real-time software testing challengesYou need a way to test on the host AND on the actual target!: Embedded and real-time software testing challenges You need a way to test on the host AND on the actual target! If your target platform is not available, you can test on the host To simulate the operational environment and discover generic bugs But tests must be performed in the final target environment as well Since some errors are only detected under the operational conditions Cross-platform testing is difficult because the system conception… Is frequently based on multiple chips for different execution machines or boards Combines more than one programming language Presents very constrained architectural characteristics


Embedded and real-time software testing challengesHow to bring runtime analysis information at the UML level?: Embedded and real-time software testing challenges How to bring runtime analysis information at the UML level? You want to use the Unified Modeling Language (UML) to visually lay out your application's design and tests but you still don’t know how State machines and code coverage of model-level execution are related? If there are any memory leaks in the hand-coded parts of the application? To measure the performance profile of the generated code?


How can your team overcome these challenges?Would this accelerate quality?: How can your team overcome these challenges? Would this accelerate quality? Automated component testing at all levels of complexity Full automation of the creation and deployment of test scripts, stubs and drivers for the simplest functions up to distributed systems Faster debugging thanks to extensive runtime analysis capabilities Memory and performance profiling, code coverage and runtime tracing features Make testing an integral part of development Dynamic links between tested code, test results and potentially visual models Prioritize your tests Code complexity and static metrics calculation Fully repeatable test sessions Complete automation of regression testing


How can your team overcome these challenges?Would this solve the verification constraints in the real target?: How can your team overcome these challenges? Would this solve the verification constraints in the real target? An integration framework flexible enough to accommodate your selection of compiler, linker, debugger, and target architecture To generate test harnesses executable in the real operational conditions To avoid changing test scenarios when target environment does Automatic test download, execution and results upload on target To free developers from target management restraints To help your development team concentrating primarily on test design Low-overhead and minimally intrusive verification probes and drivers To cope with low memory footprints and limited CPU resources To encourage in-situ verification of the software despite target constraints


How can your team overcome these challenges? Would this empower your model-centric development process?: How can your team overcome these challenges? Would this empower your model-centric development process? Develop, debug and analyze at the model level Graphically visualize reached and un-reached application states and transitions Track memory errors and performance in the hand-coded portions of the application while staying in the modeling environment Trace application execution flow in a UML-compliant graphical format


Test, analyze and resolve during development: Test as you code Automatic component test template and data generation Black- and white-box testing All levels of complexity: from single functions to distributed systems Static metric calculation for tests prioritization complexity estimation Full regression testing Test, analyze and resolve during development


Test, analyze and resolve during development: Test as you code Analyze while you test Code coverage analysis Memory profiling Performance profiling Runtime tracing Thread profiling Test, analyze and resolve during development


Test, analyze and resolve during development: Test, analyze and resolve during development Test as you code Analyze while you test Resolve what you have uncovered Test execution integrated with your debugger Consolidated, detailed to-the-point test reporting Test data hyperlinked to runtime analysis results and code Fix the defects, enhance your tests and move on!


Test, analyze and resolve during developmentOverview of IBM Rational Test RealTime features: Built to achieve standards compliance DO-178B (Avionics), MISRA (Automotive), DEF STAN 00-55 (Defense) Test, analyze and resolve during development Overview of IBM Rational Test RealTime features


Test, analyze and resolve during developmentTest RealTime provides a mean for achieving pervasive quality: Test, analyze and resolve during development Test RealTime provides a mean for achieving pervasive quality Flexible to minimally impact preexisting development processes Powerful and source-code aware test languages drive robust data-driven tests Process agnostic, so you can stay in your comfort zone Shortens time-to-problem-resolution, maximizes time-to-code White-box and black-box testing for all your software modules Component testing of C/C++, Ada or Java (J2ME/J2SE) functions and classes Message-based testing of C threads, tasks, processes and system nodes Full set of runtime analysis features to enhance your debugging capabilities Enables quality verification from a single and unified testing tool Handles all test enablement activities so you test early and often Creates shared debug and test vocabulary for your team Produces testing information required for certification processes conformance


Test and debug both host and targetTarget Deployment Port technology: Test and debug both host and target Target Deployment Port technology A versatile feature which fully automates Test harness building and downloading on the target Test execution and monitoring Test results uploading on the host A low-overhead feature which leverages Your choice of compiler/linker/debugger Your host-target connectivity (serial link, Ethernet connection, JTAG probe…)


Test and debug both host and target: Test and debug both host and target


Test and debug both host and targetTarget Deployment Port provides full target independence: Test and debug both host and target Target Deployment Port provides full target independence Customizable to support a complete range of embedded targets Assures tool adoption Reduces your ramp-up time when target configuration changes Guarantees the reuse of test assets despite target constraints Host, build and target environment agnostic Ensures portability of test and runtime analysis processes Simplifies multiple team deployment Optimizes ROI in comparison to home-grown test solutions Size and speed optimized to limit target impact Enables full control to minimize instrumentation overhead  Frees your tests from having to compensate for target restraints Avoids overtasking your system


Harness the power of model-driven testingIBM Rational Test RealTime - IBM Rational Rose RealTime integration: Harness the power of model-driven testing IBM Rational Test RealTime - IBM Rational Rose RealTime integration Move from code-centric to model-centric development Perform test and runtime analysis at the model level Rose RealTime sequence diagrams for test scripting and Quality Architect RealTime for test generation Test RealTime plug-in for the runtime analysis of your C++ models From within Rose RealTime for C++, activate the following runtime analysis features State-machine coverage Memory and performance profiling UML runtime tracing


Harness the power of model-driven testingFrom within IBM Rational Rose RealTime: Harness the power of model-driven testing From within IBM Rational Rose RealTime UML SequenceDiagram as atest case Covered UML state Uncovered UML transition


Harness the power of model-driven testingFrom within IBM Rational Rose RealTime: Harness the power of model-driven testing From within IBM Rational Rose RealTime Develop and debug at the model level Seamlessly integrate Test RealTime runtime analysis capabilities Rose RealTime application execution Visualize covered states/transitions in your UML state machines


SummaryFix your code before it breaks with IBM Rational Test RealTime: Summary Fix your code before it breaks with IBM Rational Test RealTime A unique cross-platform solution for component testing and runtime analysis Automated functional testing at all levels of complexity to ensure your software reliability Memory and performance profiling, code coverage measurement and runtime tracing abilities to diagnose run-time issues Dynamic links between code, test assets and visual models to make testing an integral part of the development cycle Low-overhead technology for enabling target-independent tests and runtime analysis All in one tool – for any IDE and hosted on any target!


Case StudyAlcatel Space : Case Study Alcatel Space Challenge When building a communication system for a satellite project, Alcatel needed to shorten testing time yet meet stringent testing standards and overcome significant testing challenges. Solution Rational Test RealTime enabled them to successfully achieve these objectives. Benefits Increased development productivity as a result of validation testing improvements Heightened confidence in the overall quality of the application under test Improved the predictability of the application's performance further ensuring customer satisfaction More information http://www-306.ibm.com/software/success/cssdb.nsf/CS/JENS-5MHK6Z?OpenDocument&Site=software


Competitive Advantages IBM Rational Test RealTime: Competitive Advantages IBM Rational Test RealTime Full and flexible solution in a single tool Integrated with lifecycle development tools IBM Rational ClearCase IBM Rational ClearQuest IBM Rational Test Manager IBM Rational Rose RealTime Rational Unified Process Integrated with a UML model-driven development solution Test deployment at all levels of verifications and granularity Component testing and runtime analysis capabilities at the source code level Test RealTime Target Deployment Port feature Deploy tests on any target from 4-bit to 64-bit architecture


Resources: Resources Rational Test RealTime Web page http://www-306.ibm.com/software/awdtools/test/realtime/index.html Real-Time and embedded solutions webinars http://www.ibm.com/software/rational/info/literature/whitepapers.jsp Rational Developer Network http://www-136.ibm.com/developerworks/rational/products/testrealtime Success stories http://www.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=rational Training http://www-306.ibm.com/software/sw-training/ Events http://www-306.ibm.com/software/sw-events/


Quiz: Quiz Which roles on the software development team can benefit from IBM Software Quality products and offerings? What are the benefits of an automated functional testing product such as IBM Rational Functional Tester? Name three of the major features in Rational Functional Tester. Which development languages does Rational Functional Tester support for creating test scripts? What is the function of IBM Rational Robot? Name three major features of Rational Manual Tester. What does the assisted data verification feature in Rational Manual Tester do? What does the visual test editor in Rational Performance Tester do? What are the three areas of code analysis in IBM Rational PurifyPlus? What are two of the competitive advantages of IBM Rational PurifyPlus? What does Target Deployment Port technology do in Rational Test RealTime?


Slide113: © Copyright IBM Corporation 2004. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, the e-business logo and other IBM products and services are trademarks or registered trademarks of the International Business Machines Corporation, in the United States, other countries or both. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries or both. Microsoft, Windows, Windows NT and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries or both. All other trademarks, company, products or service names may be trademarks, registered trademarks or service marks of others Disclaimer: NOTICE – BUSINESS VALUE INFORMATION IS PROVIDED TO YOU 'AS IS' WITH THE UNDERSTANDING THAT THERE ARE NO REPRESENTATIONS OR WARRANTIES OF ANY KIND EITHER EXPRESS OR IMPLIED. IBM DISCLAIMS ALL WARRANTIES INCLUDING, BUT NOT LIMITED TO, IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. IBM DOES NOT WARRANT OR MAKE ANY REPRESENTATIONS REGARDING THE USE, VALIDITY, ACCURACY OR RELIABILITY OF THE BUSINESS BENEFITS SHOWN.. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGES, INCLUDING THOSE ARISING AS A RESULT OF IBM'S NEGLIGENCE.WHETHER THOSE DAMAGES ARE DIRECT, CONSEQUENTIAL, INCIDENTAL, OR SPECIAL, FLOWING FROM YOUR USE OF OR INABILITY TO USE THE INFORMATION PROVIDED HEREWITH OR RESULTS EVEN IF IBM HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. THE ULTIMATE RESPONSIBILITY FOR ACHIEVING THE CALCULATED RESULTS REMAINS WITH YOU.