Catalog
concept#Quality Assurance#Observability#Reliability#Software Engineering

Quality Metrics

Measurable indicators to assess and steer software and process quality.

Quality metrics are measurable indicators to assess software and process quality.
Established
Medium

Classification

  • Medium
  • Technical
  • Design
  • Intermediate

Technical context

Prometheus / metric pipelinesCI/CD systems (e.g. Jenkins, GitHub Actions)Issue trackers (e.g. Jira, GitHub Issues)

Principles & goals

Metrics are goal-oriented and aligned with business objectives.Data sources and calculations must be documented transparently.Metrics are iteratively validated and adapted to context.
Run
Team, Domain

Use cases & scenarios

Compromises

  • Focusing on easily measurable rather than relevant goals.
  • Gaming or manipulating metrics to improve appearances.
  • Overloading with KPIs without actionable guidance.
  • Focus on a few meaningful indicators.
  • Link metrics to concrete actions.
  • Regular validation of metric assumptions.

I/O & resources

  • Issue tracker data (bugs, incidents)
  • CI/CD and test pipeline metrics
  • Production telemetry and logs
  • Dashboards with SLI/SLO overview
  • Reports for release and management decisions
  • Alerting and runbook actions

Description

Quality metrics are measurable indicators to assess software and process quality. They enable objective evaluation of defect density, reliability, maintainability and test coverage and support data-driven improvement decisions. Their use includes data collection, instrumentation, trend analysis and dashboards for continuous monitoring. Metrics should be transparent, contextualized and aligned with business objectives.

✔

  • Objective basis for improvement decisions.
  • Early detection of quality regressions.
  • Measurable verification of improvement efforts.

✖

  • Metrics can be misleading without context.
  • Collection and maintenance incur ongoing effort.
  • Not all quality aspects are quantifiable.

  • Defect Density

    Number of defects per LOC or function size; indicates code quality and release readiness.

  • MTTR (Mean Time To Recovery)

    Average time to recover after failure; central for reliability assessments.

  • Test Coverage

    Percentage of code paths covered by tests; helps reduce regressions.

Defect density reporting in an e-commerce system

Weekly reports combined issue data and release tags to prioritize hotfixes.

SLO dashboard for API latency

Monitoring dashboard with SLI trends, alerts and responsible runbooks.

Test coverage measurement in CI pipeline

Automated coverage reports during pull request checks to prevent regressions.

1

Define goals and relevant quality dimensions.

2

Document metrics with standardized calculation formulas.

3

Introduce instrumentation and automated data collection.

4

Establish dashboards, alerts and regular review cycles.

⚠️ Technical debt & bottlenecks

  • Outdated or inconsistent metric definitions across repositories.
  • Monolithic dashboards without versioning or tests.
  • Missing automation for data quality checks.
Instrumentation gapsData qualityDashboard performance
  • Sprint team measures only number of closed tickets instead of quality.
  • Using test coverage as the sole quality criterion.
  • Misusing metrics as personal performance measures for developers.
  • Forgetting to adapt metrics to changed contexts.
  • Insufficient documentation of calculation logic.
  • Overreliance on aggregated indicators without sampling.
Basics of metric definition and statisticsExperience with observability and monitoring toolsAbility to contextualize technical indicators
Measurability of SLIs and SLOsData integrity and consistencyScalable telemetry pipelines
  • • Privacy and compliance requirements
  • • Limited resources for metric maintenance
  • • Heterogeneous tool landscape