Metrics - Test Results

Next up in the metrics posts are the metrics for Test Results, where we will look at metrics for defect Accuracy, Test Efficiency and Technical Debt.

I have included some recommendations to targets to achieve, risks you face by not monitoring this as inspiration.

Happy testing!

/Nicolai

  

Metric: Defect Accuracy

Measured as: 

·         Defects that are or has been in status Reopen, rejected or duplicate.

Purpose / Objective:

·         This metric aims to pinpoint waste in the defect management life cycle. Defects that are not accurate will waste resources in the defect management lifecycle.

Recommended targets:

·         More than 15% inaccurate defects require actions from defect manager to ensure that efforts are not wasted.

Measurement Frequency:

·         Weekly while test is running and prior to test readiness reviews

Risks if not monitored:

·         Waste of resources.

·         Poor defect management.

Metric: Test Efficiency

Measured as: 

·         Defect Detection Ratio (DDT) vs. previous test phase - Number of defects found 14 days after the end of a test divided by the number of defects found in the test level under assessment.

Purpose / Objective:

·         This metric aims to give an indication on how efficiently the recently conducted test was.

·         Low DDT ratios indicate that project is subject to cost escalation due to defects being found too late in the process.

Recommended targets:

·         80% DDT for the later test levels as minimum, to reduce cost escalation. 85%+ for UAT test level.

Measurement Frequency:

·         Should be executed 14 days after each test level & 14 days after go-live to prod.

Risks if not monitored:

·         Inability to see value for money for each test phase.

·         Inability to prioritize test effort for new releases / iterations.

Metric: Technical debt

Measured as: 

·         Age of defects that are not closed.

Purpose / Objective:

·         This metric serves as input to defect prioritization and management of the defects found as part of any test.

Recommended targets:

·         No defects older than 6 months.

·         No high severity defects older than 1 month.

Measurement Frequency:

·         Weekly or in agreement with stakeholders

Risks if not monitored:

·         Unknown technical debt.

·         Problems scoping new releases / iterations.