Loading content
Automation bias weakens human oversight when AI output is treated as the default truth. High output volume, time pressure, or unclear accountability can allow flawed recommendations to enter systems without review.
Use this profile to understand the building block briefly, place it in the model, and open related building blocks.
Theoretical construct: explains a term, principle, or mental model.
Concrete cog in the system that works inside larger relationships.
Automation bias is the tendency to weight automated recommendations too heavily and defer one’s own review. It matters most when AI output becomes the default and human oversight turns formal rather than effective.
The term comes from human-factors and decision-support research: once systems produce recommendations, people tend to treat them as the default assumption and check counterevidence less often. In AI governance, that becomes a control risk when high output volume, time pressure, or unclear accountability allow flawed suggestions to be accepted without review. NIST’s AI RMF frames such risks as part of trustworthiness and evaluation.
Think of it as a three-gate review path: recommendation, countercheck, release. Automation bias appears when gate 2 is too weak and the first system output is effectively treated as the decision. Countermeasures slow the path on purpose: gather independent evidence, require justification for exceptions, and assign clear roles so the human does not merely confirm, but actually decides.
A second look or second source should confirm the suggestion before it takes effect.
Trust in the system should match task criticality, error likelihood, and uncertainty.
Humans intervene at defined points in recommendation, approval, or learning.
Plausible but wrong outputs increase the chance that review is skipped.
Systematic assessments expose risk points where unreviewed acceptance is likely.
The concept is useful in approval flows, triage, review processes, and agentic workflows where AI creates recommendations with immediate effect. It matters most when errors are costly, hard to see, or difficult to undo. More control adds time and can reduce throughput; too little control increases the risk of silent acceptance.
Where this building block is located in the topic model.
No structure path available.
Explore how this building block connects to concepts, methods, technologies, and tools.
These sources establish the term and its professional meaning.
All direct connections of the current building block in a compact text view.
This classification shows where the building block typically matters, how demanding it is, and what kind of impact it has in the model.
The level within the organization (enterprise, domain, team) at which the AssetBlock is applied.