Paul Bergeron.
Menu

Work / Agent evaluation

Did revision actually fix the answer?

Inspect a draft, a critique and a revision, with a bounded read-only tool loop.

Scope: original public experiment. I built the application workflow and interface.

Use it to explore:

  • Examine whether a revision responds to a specific criticism.
  • Separate tool evidence from a model's own interpretation of its answer.
  1. Draft with tools
  2. Critique
  3. Revise
  4. Inspect

What happens inside

The draft phase can use Wikipedia search and a bounded arithmetic tool. A critique identifies possible gaps, and a revision responds to it. The stages are streamed so the change in the answer is visible.

Decisions and tradeoffs

Bound tool use

The draft has a finite turn budget. Tools retrieve public snippets or evaluate restricted arithmetic; they do not provide unrestricted code execution.

Expose the revision

A reader can compare draft, critique and final answer instead of seeing only the final wording.

Do not assume self-correction worked

The same model can repeat its errors. The revised result still needs external verification.

Illustrative example

An authored revision example

Draft: ‘A lower training loss proves a model generalizes.’ Critique: ‘That confuses training fit with performance on unseen data.’ Revision: ‘Use held-out evaluation to examine generalization, and account for distribution differences.’ This illustration is written to explain the pattern; it is not model output.

What this does not establish

Critique is not independent evaluation. A more polished response may still be wrong. The experiment does not establish measured quality improvement over a single answer.

Try a live run

Live requests send your prompt to OpenRouter and the selected model provider. Use public, non-sensitive material. Read how demo data is handled.

Checking live availability…

Technical walkthrough reviewed September 2026. Examples do not imply client adoption or production maturity.