Proposed specificationSource-located exemplarH2 H3

Evaluator-optimizer

Revise an artifact using iterative feedback.

JOIN POINTgenerate_evaluate_reviseRECORD STATUSCandidate · v0.1.0

UML class view

The proposed contract drawn as participant roles, received calls, and numbered dependencies.

Object interactions

The same roles and calls as a class collaboration and a numbered message sequence.

Object interaction diagram for Evaluator-optimizer, showing role collaboration and a numbered message sequence — FIG. W05 / OBJECT INTERACTIONS
Open full SVG Download PDF

Pointcut

Where the mechanism applies. All four dimensions belong to the proposed match, not just its vocabulary.

Context: An artifact is improved through explicit evaluation and revision.
Trigger: A candidate receives a score, failure, or critique.
Structure: Generator, evaluator, revision input, and stopping rule.
Effect: A revised candidate is selected with recorded evaluation evidence.

Protected contract

The obligation that clients, operators, and auditors should be able to rely on.

Requires: Evaluation criteria are observable and revision budget is bounded.
Ensures: Selected output maps to a passing or best-recorded evaluation.
Invariant: Final independent evaluation is not merely the optimizer's repeated selector.
Violation: Evaluator errors, ties, or exhaustion terminate with an explicit status.

Negative control

One model call that says it improved itself is not enough.

Variation & lifecycle

The hot spot is admitted only where the protected semantics survive.

Permitted variation

Rubric evaluator, model judge, best-of-N, and iterative revision.

Test obligation

Test improvement, stagnation, evaluator exception, and held-out evaluation.

Evolution question

Did evaluation separate from generation and gain stopping rules?

Prompt + code design

An illustrative prompt and a small design sketch for the proposed contract. Names and APIs are invented; this is not a repository excerpt.

PROMPT SKETCH

Generate candidates, evaluate them against declared criteria, and revise only within the revision budget. Select the passing candidate or the best recorded one, with its evaluation attached.

CODE DESIGN / PYTHON

def optimize(task, generator, evaluator, budget):
    best = None
    for revision in range(budget):
        candidate = generator.generate(task, feedback=best.feedback if best else None)
        score = evaluator.evaluate(candidate, task.criteria)
        best = Candidate(candidate, score, revision)
        if score.passed:
            break
    return Selection(best, criteria=task.criteria, revisions=revision + 1)

Illustrative design sketch. The selected output must map to an evaluation record, not an unstated preference. This example is a teaching aid, not implementation evidence or a conformance claim.

Detection & matching

The frozen rule that retrieves this record, the canonical source exemplar, and the control that should stay unmatched.

Join point: generate_evaluate_revise
Source locator: See the source record.
Lexical cue: (?i)\b(?:evaluator[_-]?optimizer|optimizer|evaluation[_-]?feedback|revision[_-]?loop)\bFrozen expression, v0.1.0. A match is a retrieval cue, not a confirmed implementation.
Aliases: None
Canonical exemplar: openai/openai-agents-pythonexamples/agent_patterns/llm_as_a_judge.py:47Support: examples/agent_patterns/README.md:1
Negative control: One model call that says it improved itself is not enough.

Scope-anchored review. pohe-agent analyze <repo> --pattern W05 anchors the cue to its enclosing declaration and checks a separate operational signature for all four pointcut dimensions. Missing roles and negative-control risks remain visible; a structural match is not E2/E3. Calibration report ↗

Repository files

Up to five public files retrieved by the frozen cue and pinned to the audited commit. A match is a place to inspect, not a confirmed implementation.

01openai/openai-agents-pythonexamples/agent_patterns/llm_as_a_judge.py:47CANONICAL SOURCE 02affaan-m/ECCskills/manim-video/assets/network_graph_scene.py:6RETRIEVAL CUE 03browser-use/browser-usebrowser_use/llm/mistral/schema.py:1RETRIEVAL CUE 04bytedance/deer-flowbackend/app/gateway/routers/input_polish.py:41RETRIEVAL CUE 05crewAIInc/crewAIlib/crewai/src/crewai/rag/qdrant/client.py:80RETRIEVAL CUE

Evidence & provenance

Each layer answers a different question. Do not substitute retrieval for review.

Source-located exemplar

A concrete source anchor is available for this proposed record. It is one exemplar, not evidence that all repositories—or every contract field—conform.

Review boundary: this candidate and its specification come from an AI-assisted study. Independent human validation remains open.

ANY-EVIDENCE FAMILY COVERAGE17/36

47.2% of repository families in the frozen panel have at least one current cue, historical cue, or canonical source layer. This is not confirmed prevalence.

Current lexical-cue repositories: 17 / 41

Canonical source anchoropenai/openai-agents-python 86bd29878b30examples/agent_patterns/llm_as_a_judge.py:47

Supporting surfaceexamples/agent_patterns/README.md:1

Generator-evaluator loop revises until pass.

Exploratory retrieval cue Regex / v0.1.0

(?i)\b(?:evaluator[_-]?optimizer|optimizer|evaluation[_-]?feedback|revision[_-]?loop)\b

This expression is a discovery instrument only. A match is a retrieval cue—not a confirmed implementation, pointcut match, or contract proof.