Software Quality Assurance - Outline : November 15, 1997 2 Software Quality Assurance - Outline What is Software Quality assurance(SQA)?
Software Quality Assurance Activities.
Software Reviews and their importance
ISO 9000 approach to SQA What is SQA? : November 15, 1997 3 What is SQA? Software Quality Assurance is an umbrella activity that is applied throughout the software process... It encompasses.. : November 15, 1997 4 It encompasses.. A quality management approach Effective software engineering technology Formal technical reviews that are applied throughout the software process A multitiered testing strategy Control of software documentation and changes to it A procedure to assure compliance with software development standards Measurement and reporting techniques Quality ??? : November 15, 1997 5 Quality ??? Quality refers to any measurable characteristics such as correctness, maintainability, portability, testability, usability, reliability, efficiency, integrity, reusability and interoperability. Quality Concepts : November 15, 1997 6 Quality Concepts Quality of Design refers to the characteristics that designer’s specify for an item. Quality Control is the series of inspections, reviews and tests used throughout the development cycle to ensure that each work product meets the requirements placed upon it. Quality of Conformance is the degree to which the design specifications are followed during manufacturing. (cont'd)... : November 15, 1997 7 (cont'd)... Quality policy refers to the basic aims and objectives of an organization regarding quality as stipulated by the management. Quality assurance consists of the auditing and reporting functions of management. Cost of Quality includes all costs incurred in the pursuit of quality or in performing quality related activities such as appraisal costs, failure costs and external failure costs. (cont'd)... : November 15, 1997 8 (cont'd)... Quality planning is the process of assessing the requirements of the procedure and of the product and the context in which these must be observed. Quality testing is assessment of the extent to which a test object meets given requirements Quality assurance plan is the central aid for planning and checking the quality assurance. Quality assurance system is the organizational structure, responsibilities, procedures, processes and resources for implementing quality management. Relative cost of correcting an error : November 15, 1997 9 Relative cost of correcting an error Defn. of Software Quality Assurance : November 15, 1997 10 Defn. of Software Quality Assurance Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software. SQA Group Plan : November 15, 1997 11 SQA Group Plan Evaluations to be performed
Audits and reviews to be performed
Standards that are applicable to the project
Procedures for error reporting and tracking
Documents to be produced by the SQA group
Amount of feedback provided to software project team SQA Group Activities : November 15, 1997 12 SQA Group Activities Participates in the development of the projects software process description
Reviews software engineering activities to verify compliance with the defined software process.
Audits designated software work products to verify compliance with those defined as part of the software process. (cont'd)... : November 15, 1997 13 (cont'd)... Ensures that deviations in software work and work products are documented and handled according to a document procedure.
Records any non-compliance and reports to senior management. Software Reviews : November 15, 1997 14 Software Reviews ‘Filter’ for the software engineering process
‘Purify’ the software work products that occur as a result of analysis, design, and coding.
Achieve technical work of more uniform, greater and more predictable quality.
Detect errors and problems at the earliest possible time. Formal Technical Reviews : November 15, 1997 15 Formal Technical Reviews To uncover errors in function, logic, or implementation for any representation of the software
To verify that software meets its requirements
To ensure that software representation meets predefined standards
To achieve software development in a uniform manner
To make projects more manageable Defect Amplification Model : November 15, 1997 Defect Amplification Model Defect Amplification with Reviews : November 15, 1997 Defect Amplification with Reviews Cost Comparison of Error Repair : November 15, 1997 Cost Comparison of Error Repair Review Guidelines.. : November 15, 1997 16 Review Guidelines.. Review the product, not producer
Set an agenda and maintain it
Limit the debate
Enunciate problem areas, not to solve every problem noted
Take written notes
Allocate resources and time schedule for FTR’s Limit the number of participants and insist upon advance preparation
Develop a checklist for each work product to be reviewed
Training for all reviewer’s
Reviewing earlier reviews Additional Structures : November 15, 1997 Additional Structures Requirements Control Board
All requirement changes must be formally reviewed and approved Software Control Board
All design changes must be formally reviewed and approved Interface Control Board Statistical Quality Assurance : November 15, 1997 17 Statistical Quality Assurance Implies information about software defects is collected and categorized
An attempt is made to trace each defect to its underlying cause
Isolate the vital few causes of the major source of all errors
Then move to correct the problems that have caused the defects Categories of Errors : November 15, 1997 Categories of Errors Incomplete or erroneous specification (IES)
Misinterpretation of customer comm (MCC)
Intentional deviation from specification (IDS)
Violation of programming standards (VPS)
Error in data representation (EDR)
Inconsistent module interface (IMI)
Error in design logic (EDL) Categories of Errors (cont'd) : November 15, 1997 Categories of Errors (cont'd) Incomplete or erroneous testing (IET)
Inaccurate or incomplete documentation (IID)
Error in programming lang. Translation (PLT)
Ambiguous or inconsistent human-computer interface (HCI)
Most often IES, MCC and EDR are the vital few causes for majority of errors. Definitions : November 15, 1997 18 Definitions Ei = the total number of errors uncovered during the ith step in the software engineering process
Si = the number of serious errors
Mi = the number of moderate errors
Ti = the number of minor errors
PS = size of the product (LOC, design statements, pages of documentation) error index : November 15, 1997 19 error index Phase index for each step and then error index is calculated
PIi = ws(Si/Ei)+wm(Mi/Ei)+wt(Ti/Ei)
Formula: Software Reliability : November 15, 1997 20 Software Reliability Defined as the probability of failure free operation of a computer program in a specified environment for a specified time.
It can measured, directed and estimated
A measure of software reliability is mean time between failures where
MTBF = MTTF + MTTR
MTTF = mean time to failure
MTTR = mean time to repair Software Availability : November 15, 1997 21 Software Availability Availability =MTTF/(MTTF + MTTR) * 100%
Software availability is the probability that a program is operating according to requirements at a given point in time Software Safety : November 15, 1997 Software Safety Processes that help reduce the probability that critical failures will occur due to SW Hazard analyses Identify hazards that could call failure Develop fault tree Identify all possible causes of the hazard Formally review the remedy for each Redundancy Require a written software safety plan Require independent verification & validation Example Fault Tree -- Thermal : November 15, 1997 Example Fault Tree -- Thermal Loss of heat Power failure Computer failure Incorrect
input SW failed to throw switch ... Computer failure SW failed to throw switch ... Logic reversed Software Safety : November 15, 1997 Software Safety Redundancy
Replicated at the hardware level
Similar vs.. dis-similar redundancy Verification
Assuring that the software specifications are met Validation
Assuring that the product functions as desired Independence Overview of SQA Plan : November 15, 1997 22 Overview of SQA Plan Purpose of Plan
Standards, Practices and Conventions
Reviews and Audits
Problem Reporting and Corrective action Tools, Techniques and Methodologies
Records Collection, Maintenance and Retention
Risk Management ISO 9000 Quality Standards : November 15, 1997 23 ISO 9000 Quality Standards ISO 9000 describes quality assurance elements in generic terms that can be applied to any business.
It treats an enterprise as a network of interconnected processes.
To be ISO-complaint processes should adhere to the standards described.
Elements include organizational structure, procedures, processes and resources.
Ensures quality planning, quality control, quality assurance and quality improvement. ISO 9001 : November 15, 1997 24 ISO 9001 An international standard which provides broad guidance to software developers on how to Implement, maintain and improve a quality software system capable of ensuring high quality software
Consists of 20 requirements...
Differs from country to country.. ISO 9001 (cont'd)..requirements : November 15, 1997 25 ISO 9001 (cont'd)..requirements Management responsibility
Document and data control
Purchasing Control of customer supplied product
Product identification and traceability
Inspection and testing
Control of inspection, measuring and test equipment ISO 9001 (cont'd).. : November 15, 1997 26 ISO 9001 (cont'd).. Inspection and test status
Control of non-confirming product
Corrective and preventive action
Handling, storage, packaging, preservation and delivery Control of quality records
Internal quality audits
Statistical techniques Summary- : November 15, 1997 27 Summary- SQA must be applied at each step
SQA might be complex
Software reviews are important SQA activities
Statistical SQA helps improve product quality and software process
Software Safety is essential for critical systems
ISO 9001 standardizes the SQA activities