software testing

Tips for Success whenSelecting Testing Tools : 

Tips for Success whenSelecting Testing Tools Marie Ackley ackley @ promail.com SASQAG panel discussion October 18, 2001

Different Categories of Testing Tools : 

2 Different Categories of Testing Tools Requirements Management Configuration Management Test Planning and Test Case Management Test Evaluation - Coverage Analysis Automated Testing Systems - Front End testing Load Simulators and Performance Measurement Tools Defect Tracking others: Cyclomatic Complexity; Bounds Checkers; Comparators - Files, Screens, Reports; Data Generators - Combinatoric, Cause / Effect Node Graphing

Leading Cause of SHELF WARE in Testing Tools : 

3 Leading Cause of SHELF WARE in Testing Tools is Putting the CART before the HORSE Don't invest in Tools without having good QA Methods and Practices. A good tool will compliment and augment good processes but not even the greatest tool can overcome process chaos.

Requirements Management : 

4 Requirements Management Supports multi levels of linking and tracing: Requirements to Specifications to Test Cases, including hierarchies within those groups Provides traceability - both forward and backward Provides coverage analysis Provides multi relationships: one-to-many, many-to-one and coverage is tracked for partials when relationship is one-to-many Provides Change Management of requirements along with audit history and maybe security privileges for modifications Import and export features so stuff came go and come between the tool and word processors or spreadsheets

Requirements Management : 

5 Requirements Management Links to tool comparisons the "Volere" Requirements Method - by Atlantic Systems Guild http://www.systemsguild.com/GuildSite/Robs/Template.html Automating Requirements Management - by Karl E. Wiegers Detailed feature comparisons of many requirements management tools and guidance on selecting a requirements management tool http://www.processimpact.com/articles/rm_tools.html

Configuration Management : 

6 Configuration Management "To change and to change for the better are two different things" - German Proverb Provides Identifications (indexing of artifacts) Controls not only software code but documentation, hardware, firmware, and system environment configuration Provides audit trail and history tracking Provides customizable reports High ease of use for View Only functions Sufficient security levels for the Promote functions Accounting and reporting for different ”Status Levels" - WIP, Waiting Room, Release/Production

Configuration Management : 

7 Configuration Management Links to tool comparisons Configuration Management Vendor Links - by UCM Central http://www.snuffybear.com/ucmcentral_new_vendorlinks.htm config-mgmt Newsgroup http://www.faqs.org/faqs/by-newsgroup/comp/comp.software.config-mgmt.html

Test Planning and Test Case Management : 

8 Test Planning and Test Case Management Import test cases from other file formats such as word processing or spreadsheet Customizable reporting Tracks history and status of test cases Allows file attachments Integration - can tool link to other tools such as your Requirements tool and your Automated Testing tool Provide a Test Execution Results repository Maintains Master Set of test cases and then can build “subsets” for project specific needs

Test Planning and Test Case Management : 

9 Test Planning and Test Case Management Links for tool comparisons Testing and Test Management Tools http://www.methods-tools.com/tools/frames_testing.html Information About Testing Tools - http://www.cs.queensu.ca/home/shepard/testing.dir/under.construction/tool_list.html

Test Evaluation - Coverage Analysis : 

10 Test Evaluation - Coverage Analysis Identifies what "unique" data combinations are needed Uses an appropriate measurement technique Decision; Condition/decision; Path; Predicate Types of coverage included in the reporting Branch; Loop; Multi; Operator; Operand Integration - is it compatible with your other tools such as Automated Test Execution tools Implementation - is the tool "invasive" (must be compiled along with the source code)

Test Evaluation - Coverage Analysis : 

11 Test Evaluation - Coverage Analysis Links to tool comparisons Experience With the Cost of Different Coverage Goals for Testing - by Brian Marick http://www.testing.com/writings/experience.pdf Coverage Analysis - by Steve Cornett http://www.bullseye.com/webCoverage.html

Automated Testing Systems - Front End testing : 

12 Automated Testing Systems - Front End testing Maintenance - object recognition when the application changes Ease Of Use - complexity matches the programming talent of QA persons Tool language and syntax matches what company uses: Visual Basic or C Extensible Language - can use external subroutines or DLLs Integration with other tools - Code Coverage, Test Case Management, Defect Tracking Compatibilty - platforms (Client/Server, Web, Mainframe) Operating Systems(Windows, Mac, Unix, OS/2, Mainframe) and GUI Development (PowerBuilder, C, Visual Basic, Galaxy

Automated Testing Systems - Front End testing : 

13 Automated Testing Systems - Front End testing Supports Data Driven inputs from flat files, spreadsheet files, and database connections For Evaluation/Proof of Concept - include automating a Level One test Links to tool comparisons Making the Right Choice - by Elisabeth Hendrickson http://www.qualitytree.com/feature/mtrc.pdf Short Sharp Advice about how to choose a testing tool - Mark Fewster and Dorothy Graham http://www.grove.co.uk/Tool_Information/Choosing_Tools.html Tool Comparison between Rational, Mercury, Segue, and Compuware - by Ray Robinson http://www.qacity.com/testtype/AUTOMATION_TOOLS.doc

Automated Testing Systems : 

14 Automated Testing Systems Links to articles Lessons in Test Automation - by Elfriede Dustin http://www.sqa-test.com/articles.html The Truth About Automated Test Tools - by Linda Hayes http://www.worksoft.com/resources/4-97.htm http://www.worksoft.com/resources/5-97.htm Test Automation - by Bret Pettichord http://www.io.com/~wazmo/qa/#test_automation Testers' Network - VeriTest website http://www.veritest.com/testers'network Test Automation Frameworks - by Carl J. Nagle http://members.aol.com/sascanagl/FRAMESDataDrivenTestAutomationFrameworks.htm

Load Simulators and Performance Measurement : 

15 Load Simulators and Performance Measurement Reporting includes graphical charts Debugging features for script creation Reporting includes graphical charts and logs of performance for server speed and responsiveness Quantity of simulated users Variations in session-specific items such as cookies, usernames, passwords, proxies and ASP-session IDs Support for your middleware - SQLnet, DBLib or CBLib, SQL Server, ODBC, Telnet Challenges include SSL sessions, cookies, forms, proxies, password authentication, dynamic HTML, full HTTP, SSL, XML, Javascript, ActiveX, WAP, Java Platforms and protocols

Load Simulators and Performance Measurement : 

16 Load Simulators and Performance Measurement Link to tool comparisons Load and Performance Test Tools - by Rick Hower http://www.softwareqatest.com/qatweb1.html Software Research - TestWorks http://www.methods-tools.com/tools/testing.html

Defect Tracking : 

17 Defect Tracking Determine how many current users you have and consider and scaling for future users Web based or client-server Features: Customizable Fields; Customizable Reports; File Attachments; Audit Trail; Email linking; Security Levels Provides customizable rules for the workflows, security privileges and notifications Integration - does it link to your other tools

Defect Tracking : 

18 Defect Tracking Links to tool comparisons Tracking Down a Defect Management Tool - by Hung Quoc Nguyen http://www.qacity.com/testgen/DefectTrackingNguyenSTQE701.pdf The Problem with Problem Tracking - by Linda Hayes http://www.worksoft.com/resources/3-99.htm

Websites for Tool Information : 

19 Websites for Tool Information Vendor contact information http://www.testingfaqs.org/tools.htm http://www.ondaweb.com/sti/stivend.htm#L http://www.dbmsmag.com/pctest.html#P0890 Bulletin board forums and articles http://www.qaforums.com http://community.sqatester.com http://www.qualitytree.com/links/links.htm http://www.stqemagazine.com