Catalog
concept#Quality Assurance#Software Engineering#DevOps#Reliability

Test Levels

Classification of test types by purpose and scope (unit, integration, system, acceptance) to structure test strategies and responsibilities.

Test Levels describes the hierarchy of tests (unit, integration, system, acceptance) for systematic software quality assurance.
Established
Medium

Classification

  • Medium
  • Technical
  • Design
  • Intermediate

Technical context

CI servers (e.g. Jenkins, GitHub Actions)Test frameworks (e.g. JUnit, pytest)Mocking and stubbing tools

Principles & goals

Separate test levels by granularity and scopeProvide fast feedback close to implementationAutomate where repeatability and stability matter
Build
Team, Domain

Use cases & scenarios

Compromises

  • Overfocusing one level while neglecting others
  • Insufficient test data and environment specifications
  • Increased CI times due to lack of parallelization
  • Keep tests fast and deterministic at unit level
  • Isolate integration and E2E tests in dedicated environments
  • Use test data management and mocking for stability

I/O & resources

  • Requirements and acceptance criteria
  • Test environments and test data
  • CI/CD integration and test frameworks
  • Test reports and failure feedback
  • Regression protection via automated suites
  • Release criteria based on test results

Description

Test Levels describes the hierarchy of tests (unit, integration, system, acceptance) for systematic software quality assurance. It defines purpose, scope and responsibilities at each level and guides test strategy, automation effort and fault localization. It also supports prioritization and allocation of testing resources.

✔

  • Improved fault localization through clear responsibility boundaries
  • More efficient testing strategy and resource usage
  • Scalable automation paths per level

✖

  • Overspecifying levels can increase effort
  • Flaky tests at integration or E2E level are hard to debug
  • Not all faults are visible at unit level

  • Average test runtime

    Average duration of full test runs in the CI pipeline.

  • Fault localization time

    Time from fault occurrence to identification of the affected component.

  • Test coverage ratio

    Proportion of source code covered by automated tests.

Unit testing in Microservice A

A set of fast unit tests to secure service logic and DTO transformations.

Integration tests for payment processing

Runs against simulated payment gateways to validate end-to-end integrations.

Acceptance test for web checkout

Business-driven scenarios validate checkout flows with realistic test data.

1

Analyze existing test coverage and identify gaps

2

Define goals per test level (e.g. runtime, coverage)

3

Automate prioritized tests and integrate into CI

4

Monitor test metrics and continuously improve

⚠️ Technical debt & bottlenecks

  • Slow, non-parallel test pipelines
  • Missing or unreliable test data knowledge
  • High legacy test maintenance burden
Long test runtimesFlaky testsComplex test environments
  • E2E tests for UI details that change frequently
  • Using mocks at incorrect integration boundaries
  • Using unit tests as a substitute for integration validation
  • Unclear responsibilities for test types
  • Neglecting maintenance of outdated tests
  • Overcomplicated test data preparation
Test design and test case developmentTest automation developmentFault diagnosis and debugging
Fast feedback cycles for developersScalable CI/CD pipelinesIsolability of components for fault diagnosis
  • • Limited CI resources
  • • Legacy code without tests
  • • Restricted access to external integrations during tests