Catalog
method#Quality Assurance#Reliability#Observability

Non-Functional Testing

A structured approach to plan and execute tests for non-functional requirements such as performance, scalability, security and reliability.

Non-functional testing is a structured method to evaluate system attributes such as performance, scalability, reliability, security and maintainability.
Established
Medium

Classification

  • Medium
  • Technical
  • Architectural
  • Intermediate

Technical context

CI/CD pipelines (e.g. Jenkins, GitLab CI)Monitoring tools (e.g. Prometheus, Grafana)Load testing tools (e.g. JMeter, k6)

Principles & goals

Define non-functional requirements early and explicitlyUse measurable metrics instead of vague goalsIntegrate automation and embed tests into CI/CD
Build
Enterprise, Domain, Team

Use cases & scenarios

Compromises

  • Incorrect load profiles lead to misleading results
  • Over-focusing on single metrics neglects overall behavior
  • Test environments with production data carry compliance risks
  • Run tests with production-like data and topologies
  • Repeat scenarios regularly and maintain baselines
  • Integrate tests into CI/CD and monitoring pipelines

I/O & resources

  • Non-functional requirements and acceptance criteria
  • Requirement profiles and load models
  • Test environments, measurement tools and monitoring
  • Test reports with metrics, thresholds and recommended actions
  • Concrete optimization and capacity recommendations
  • Regression tests and baselines for subsequent releases

Description

Non-functional testing is a structured method to evaluate system attributes such as performance, scalability, reliability, security and maintainability. It defines test scenarios, environments and metrics to validate non-functional requirements and supports architectural and operational decisions. It integrates with CI/CD and monitoring to enable continuous validation and regression control.

✔

  • Early detection of performance and reliability issues
  • Improved decision basis for architecture and capacity
  • Reduced production outages through validated operational assumptions

✖

  • High effort for realistic test environments
  • Result interpretation requires domain and infrastructure knowledge
  • Not all aspects can be fully reproduced in test environments

  • Throughput (requests/s)

    Measures number of successful requests per second under defined load.

  • 95th percentile latency

    Indicates how long most requests wait at most; important for SLAs.

  • Error rate

    Share of failed or rejected requests during tests.

Scaling test of a payments API

Load tests revealed DB contention; sharding and connection pooling improved throughput.

Resilience exercise after cloud migration

Chaos tests exposed missing timeouts; automatic circuit-breaker integration reduced failure impact.

Performance regression during release

Continuous performance tests detected a regression; hotfix and cache strategy optimization resolved the issue.

1

Capture and prioritize non-functional requirements

2

Define test objectives, metrics and acceptance criteria

3

Create representative load profiles and scenarios

4

Automate tests and integrate into CI/CD

5

Analyze results, derive actions and update baselines

⚠️ Technical debt & bottlenecks

  • Unchecked legacy components without performance profiles
  • Missing automated tests for critical paths
  • Outdated baselines that no longer reflect real load
Database contentionNetwork latencyThread and connection pooling
  • Performance engineers ignore functional changes and analyze only raw data
  • Tests run only in small, non-representative environments
  • Results published without concrete actions or responsibilities
  • Wrong assumptions about user behavior in load modeling
  • Insufficient isolation between test and production environments
  • Lack of consideration for external integrations and third parties
Knowledge in load and performance testingUnderstanding of infrastructure, networking and databasesAbility to analyze and interpret metrics
Performance requirements of business processesAvailability and SLA targetsSecurity and compliance requirements
  • • Availability of realistic test data
  • • Limited test infrastructure and costs
  • • Time constraints in the release cycle