Evaluator-Optimizer

A generator produces a result, an evaluator scores it against specific criteria, and the generator optimizes it based on the feedback. Typical conditions for use: Quality criteria can be explicitly formulated; Iterative improvement is measurable. The central trade-off: Better output quality is gained against additional evaluation complexity and latency.

Use this profile to understand the building block briefly, place it in the model, and switch to the 360° assessment when needed.

Content type

Concept

Theoretical construct: explains a term, principle, or mental model.

Classification level

Structure

What organizes, connects, or makes decisions possible.

Available perspectives

Open this building block directly in an already curated view.

360°

In brief

Evaluator-Optimizer in brief

An evaluator-optimizer has a generator create a draft and a separate evaluator improve it repeatedly through feedback.

Where does it come from?

The pattern comes from iterative optimization and review processes and has been described for language models as an evaluator-optimizer workflow. It revives the older idea of coupling production and quality checking through feedback instead of accepting a one-shot draft unchecked.

The mental model

The optimizer produces an output; the evaluator checks it against criteria and returns actionable feedback. The loop ends at sufficient quality, an abort condition, or an exhausted budget.

Key building blocks

Core idea

An evaluator-optimizer has a generator create a draft and a separate evaluator improve it repeatedly through feedback. Conceptual starting point.

Mechanism

The optimizer produces an output; the evaluator checks it against criteria and returns actionable feedback. The loop ends at sufficient quality, an abort condition, or an exhausted budget. Practical guiding question.

Practice context

The pattern improves consistency and verifiability when criteria are observable. It wastes effort when evaluation is vague or the evaluator repeats the generator’s errors.

When is it useful?

The pattern improves consistency and verifiability when criteria are observable. It wastes effort when evaluation is vague or the evaluator repeats the generator’s errors.

Anthropic – Building effective agents Shinn et al. – Reflexion