200105Craft

SQA In Review: 

SQA In Review SEPG Conference March 12 - 15, 2001 Terrence W. Craft First Data Corporation

Outline: 

Outline SQA Basics & the 11 Steps Lessons Learned SQA Audit Process

Slide3: 

When the thinking changes, the organization changes, and vice versa. - Gerald Weinberg

SQA Basics: 

SQA Basics SQA at Level 2 SQA support is critical success factor for improvement directly contributes to discipline/maturity level of the organization included as verification feature in each KPA

SQA Basics, cont.: 

SQA Basics, cont. Software Quality Assurance - monitors execution of software engineering activities; ensures officially established corporate and organizational processes are being implemented and followed an appropriate development methodology is in place projects use standards and procedures independent reviews and audits are conducted and reported documentation is produced to support maintenance

SQA Basics, cont.: 

SQA Basics, cont. changes to project requirements and deliverables are controlled testing is performed adequately against established standards to highlight high risk areas deviations from standards and procedures are identified immediately and corrected ensure project auditability (external, corporate) SQA plan and project plan are compatible

11 Steps: 

11 Steps 1. Gauge the organization 2. Obtain Senior Mgt commitment 3. Establish the SQA function 4. Gather development practices 5. Separate routine from dynamic activities 6. Establish relationships 7. Integrate with the project teams 8. Perform reviews, maintain quality records 9. Link to s/w process improvement 10.Communicate to the organization 11.Continually add value

Cost to Start: 

Cost to Start Staffing $55k - $155k total compensation/yr/FTE Industry standard is 4-7% of development organization actual is 3-4% of technical staff Level 4 & 5 sites are 5-10% Training Approx $8k-$15k/yr/FTE during initial start-up Classes Conferences Professional associations dues/fees

Outline: 

Outline SQA Basics & the 11 Steps Lessons Learned SQA Audit Process

Slide10: 

All models are wrong; some models are useful - George Box

Lessons Learned/Responses: 

Lessons Learned/Responses Lesson Learned: Rome wasn’t built in a day… Focus on providing value to the projects and improving the organization and project processes. Response: Focus on critical, visible, and/or receptive projects initially and expand the scope gradually. Limit SQA criteria to the most critical process and/or product requirements initially and increase as warranted by the results. Establish thresholds and sampling techniques for SQA activities to ensure that small-scale projects are not over-burdened. Revisit the scope of the SQA program periodically.

Lessons Learned/Responses: 

Lessons Learned/Responses Lesson Learned: Rome wasn’t built in a day… (continued) Response: Ensure non-compliance and issue escalation procedures and thresholds are clear and concise. Ensure deviation approval/disapproval guidelines are clear and concise. Provide thresholds for non-compliance handling procedures based on the severity of the non-compliance and criticality of the project. Provide thresholds for issue handling procedures based on the severity of the issue and the criticality of the project.

Lessons Learned/Responses: 

Lessons Learned/Responses Lesson Learned: Fair is fair… Objectivity, consistency, and efficiency are crucial for success. Response: Keep the SQA criteria in sync with the organization’s processes, and ensure the SQA criteria are based directly on documented process requirements. Pilot SQA criteria, audits, and reviews to eliminate subjectivity and ambiguity. Continually review and improve the SQA criteria and processes to increase objectivity and effectiveness. Utilize organizational change control procedures when implementing changes to the SQA program and/or criteria.

Lessons Learned/Responses: 

Lessons Learned/Responses Lesson Learned: Fair is fair … (continued) Response: Pair up SQA staff members when conducting audits and reviews. This doubles the experience, expertise, and objectivity the SQA staff brings to the review and/or audit. Document and communicate lessons learned, conduct procedures, and identification and resolution of ‘gray’ areas related to SQA activities. Strive for consistency in criteria and audit/review type to allow for comparison of results over time.

Lessons Learned/Responses: 

Lessons Learned/Responses Lesson Learned: We’re here to help you… Provide the project team with practical, usable examples and support from project management subject matter experts. Response: Provide the project teams and SQA representatives with templates/examples of project plans tailored to their project work types. Provide the project teams and SQA representatives with templates and/or examples of project work breakdown structures with SQA audit and review tasks built in.

Lessons Learned/Responses: 

Lessons Learned/Responses Lesson Learned: We’re here to help you… (continued) Response: Include experienced, respected project managers on the SQA staff to provide coaching and mentoring. Ensure close involvement with the customer SQA group. This can significantly benefit the project team and leadership by saving them considerable time, trouble, and duplication of effort.

Lessons Learned/Responses: 

Lessons Learned/Responses Lesson Learned: Eat your own dog food… If an organizational SQA group exists, use the organization’s processes to manage the SQA project. This provides the SQA group with tremendous insight into the processes, and also increases the credibility of the SQA group with the project teams. Response: Ensure the organizational SQA plan includes consideration for all of the typical project planning activities. Use the organization’s project planning, project tracking, requirements management, and configuration management processes to execute the SQA project.

Lessons Learned/Responses: 

Lessons Learned/Responses Lesson Learned: Eat your own dog food… (continued) Response: Conduct SQA audits/reviews on the SQA project itself. These audits/reviews may be performed by project team SQA representatives, process improvement group representatives, or past SQA staff members to increase objectivity.

Outline: 

Outline SQA Basics & the 11 Steps Lessons Learned SQA Audit Process

Slide20: 

Progress will accelerate if you perform the improvement activities and create the momentum and driving force for change. Progress will decelerate if you do nothing to raise confidence in the change. - Kim Caputo

Slide21: 

The only one who likes change is a wet baby. Corollary to Caputo’s quote: - author unknown

Why Audit?: 

Why Audit? Simply stated: “Only processes actually followed contribute to process improvement.” SQA Audit Purpose: Provide management appropriate visibility so that approved processes are enforced.

SQA Goals: 

SQA Goals Assure SQA activities are planned Objectively verify that software products and activities adhere to applicable standards, procedures, and requirements. Inform applicable groups and individuals of SQA activities and results. Assure senior management addresses noncompliance issues that cannot be resolved within the software project.

An Affordable Process: 

An Affordable Process General Rule: do not audit your own project (violates objectivity). Auditor Model Choices: Centralized audit group Audit teams using software professionals from other areas of the company. Management teams perform audits (e.g., Juran model) Decentralized audit: buddy system of software professionals auditing each other’s projects.

SQA Audit Steps: 

SQA Audit Steps Prepare for the audit Conduct the audit Follow up the audit

Prepare for the Audit: 

Prepare for the Audit Ensure SQA audits are planned in each Software Project Management Plan. Prepare an audit checklist from approved polices, standards, and processes. Obtain project artifacts such as project plans and software requirement specifications to verify products and processes. Prepare the audit reporting forms. Set the meeting with the Project Manager. Record labor expended on these activities.

SQA Checklist Example: 

SQA Checklist Example Software project’s purpose, scope, goals, and objectives. Selection of a software lifecycle. Identification of the selected procedures, methods, and standards for developing and/or maintaining the software. Size estimates of the software work products and any changes to the software work products. Estimates of the software project’s effort and costs. Estimated use of critical computer resources. Software project schedules, including identification of milestones and reviews. Identification and assessment of the project’s software risks. Plans for the project’s software engineering facilities and support tools.

Conduct the SQA Audit: 

Conduct the SQA Audit Meet with the Project Manager and others. Find evidence that work followed process, policies, and standards. Make note of work done well to compliment project team. Describe and classify non-conformities. Review findings with Project Manager and correct statements appropriately. Record labor expended.

Follow Up the SQA Audit: 

Follow Up the SQA Audit Discuss with Project Manager observations and non-conformities. Set date for corrective action plans. Both sign the audit report. Verify and approve corrective action plans. Present audit report to department management. Verify corrective actions are complete and effective. Present non-resolved non-conformities to senior management. Record labor expended.

Staffing: 

Staffing About 150 projects Durations ranging from 1 month to several years Technology staff is about 800 SQA staff - 1 manager 9 analysts 3 auditors

Slide31: 

Quality Is Not Free… …but quality is cheaper than the alternatives - SEI