A collection of Software Testing Metrics

Process Metrics

• Test Coverage = Number of units (KLOC/FP) tested / total size of the system. (LOC represents Lines of Code)

• Number of tests per unit size = Number of test cases per KLOC/FP (LOC represents Lines of Code).

• Acceptance criteria tested = Acceptance criteria tested / total acceptance criteria

• Defects per size = Defects detected / system size

• Test cost (in %) = Cost of testing / total cost *100

• Cost to locate defect = Cost of testing / the number of defects located

• Achieving Budget = Actual cost of testing / Budgeted cost of testing

• Defects detected in testing = Defects detected in testing / total system defects

• Defects detected in production = Defects detected in production/system size

• Quality of Testing = No of defects found during Testing/(No of defects found during testing + No of acceptance defects found after delivery) *100

• Effectiveness of testing to business = Loss due to problems / total resources processed by the system.

• System complaints = Number of third party complaints / number of transactions processed

• Scale of Ten = Assessment of testing by giving rating in scale of 1 to 10

• Source Code Analysis = Number of source code statements changed / total number of tests.

• Effort Productivity = Test Planning Productivity = No of Test cases designed / Actual Effort for Design and Documentation

• Test Execution Productivity = No of Test cycles executed / Actual Effort for testing

• Functional Test Coverage = FE/FT, Where,FE is the number of test requirements that are covered by test cases that were executed against the software & FT is the total number of test requirements

• SMI = Mt – ( Fa + Fc + Fd)/Mt

Where

SMI is the Software Maturity Index value

Mt is the number of software functions/modules in the current release

Fc is the number of functions/modules that contain changes from the previous release

Fa is the number of functions/modules that contain additions to the previous release

Fd is the number of functions/modules that are deleted from the previous release


• Reliability = 1 – Number of errors (actual or predicted)/Total number of lines of executable code

This reliability value is calculated for the number of errors during a specified time interval.

Three other metrics can be calculated during extended testing or after the system is in production. They are:

• MTTFF (Mean Time to First Failure)

MTTFF = The number of time intervals the system is operable until its first failure (functional failure only).


• MTBF (Mean Time Between Failures)

MTBF = Sum of the time intervals the system is operable


• MTTR (Mean Time To Repair)

MTTR = sum of the time intervals required to repair the system

The number of repairs during the time period


IEEE SOFTWARE QUALITY METRICS METHODOLOGY Std 1061-1998

• Conformance: An application of a software quality metrics methodology conforms to this standard if all required provisions, identiÞed by the use of the verb shall,are implemented.

• Attribute: A measurable physical or abstract property of an entity.

• Critical range:Metric values used to classify software into the categories of acceptable, marginal, or unacceptable.

• Critical value: Metric value of a validated metric that is used to identify software that has unacceptable quality.

• Direct metric: A metric that does not depend upon a measure of any other attribute.

• Direct metric value:A numerical target for a quality factor to be met in the Þnal product. For example,mean time to failure (MTTF) is a direct metric of final system reliability.

• Measure:

(A) A way to ascertain or appraise value by comparing it to a norm.

(B) To apply a metric.

• Measurement: The act or process of assigning a number or category to an entity to describe an attribute of that entity. A Þgure, extent, or amount obtained by measuring.

• Metrics framework: A decision aid used for organizing, selecting, communicating, and evaluating the required quality attributes for a software system. A hierarchical breakdown of quality factors, quality subfactors, and metrics for a software system.

• Metrics sample: A set of metric values that is drawn from the metrics database and used in metrics validation.

• Metric validation:The act or process of ensuring that a metric reliably predicts or assesses a quality factor.

• Metric value: A metric output or an element that is from the range of a metric.

• Predictive metric: A metric applied during development and used to predict the values of a software quality factor.

• Predictive metric value:A numerical target related to a quality factor to be met during system development.

• Process metric:A metric used to measure characteristics of the methods, techniques, and tools employed in developing, implementing, and maintaining the software system.

• Product metric:A metric used to measure the characteristics of any intermediate or final product of the software development process.

• Quality attribute: A characteristic of software, or a generic term applying to quality factors, quality subfactors, or metric values.

• Quality factor:A management-oriented attribute of software that contributes to its quality.

• Quality requirement:A requirement that a software attribute be present in software to satisfy a contract,standard, speciÞcation, or other formally imposed document.

• Quality subfactor:A decomposition of a quality factor or quality subfactor to its technical components.

• Software component:A general term used to refer to a software system or an element, such as module,unit, data, or document.

• Software quality metric:A function whose inputs are software data and whose output is a single numerical value that can be interpreted as the degree to which software possesses a given attribute that affects its quality.

• Validated metric: A metric whose values have been statistically associated with corresponding quality factor values.

Note: Collection of Metrics from online resources and presented above as a notes for quick read