Scope: original public experiment. I built the application workflow and interface.
Use it to explore:
- Explore a draft set of study questions before human review.
- Make a quality threshold and a duplicate rule explicit enough to challenge.
- Generate
- Grade
- Deduplicate
- Filter
What happens inside
The generator produces up to six items. A grader scores correctness, specificity and clarity from 1 to 5. Items need a mean score of at least 3.5 and must not match a previous item's word set at Jaccard similarity 0.8 or higher.
Decisions and tradeoffs
Expose the quality rule
A fixed rubric and threshold make the filtering behavior understandable. Passing the rule does not independently establish correctness.
Use a transparent duplicate check
Word-set Jaccard similarity is small and inspectable. It does not reliably catch semantic duplicates or paraphrases.
Keep validation outside generation
Malformed or out-of-range scores cannot quietly become successful results. Human review remains necessary for educational or specialist use.
Illustrative example
Inspect a deterministic duplicate fixture
These sentences contain the same words in a different order. The duplicate rule ignores order: it compares unique word sets.
A cell contains genetic material and uses energy.
A cell uses energy and contains genetic material.
Jaccard similarity: 1.0 — 8 shared unique words / 8 total unique words. The second item is a duplicate under the 0.8 threshold.
Inspect the sets
a · and · cell · contains · energy · genetic · material · uses
Punctuation and word order are ignored. Repeating a word does not add a new set element.
Engineering walkthrough / Actual filter execution
A quality score is only one gate.
The filter needs two separate answers: does the item meet the score threshold, and does it repeat material already seen? A high model score cannot override the duplicate rule.
valid rubric scores → unrounded mean ≥ 3.5 → Jaccard < 0.8 → keep| Item | Input | Mean | Duplicate | Result |
|---|---|---|---|---|
| 1 | A cell contains genetic material and uses energy. | 4.33 | No | Keep |
| 2 | A cell uses energy and contains genetic material. | 4.33 | Yes | Reject |
| 3 | A mitochondrion is a city-sized rock. | 2.33 | No | Reject |
A boundary worth testing
A mean of 3.499 rounds to 3.50 for display, but it must still fail the 3.5 threshold. The implementation compares the unrounded mean. Display formatting is not the decision rule.
Malformed grading is a failed run
Missing, nonnumeric or out-of-range scores reject the response. Treating missing scores as zero would silently conflate a provider failure with a low-quality item.
What this still needs at scale
Human calibration, lineage, semantic duplicate detection and persistence are separate engineering problems. This example shows the local filtering contract; it does not claim to implement a production data platform.
Download the filter fixture and result (JSON)What this does not establish
The model is generating and assessing content. Scores need calibration against independent judgments. This six-item experiment has no delivery queue, persistent lineage or production curation claim.
Try a live run
Live requests send your prompt to OpenRouter and the selected model provider. Use public, non-sensitive material. Read how demo data is handled.
Checking live availability…
Technical walkthrough reviewed September 2026. Examples do not imply client adoption or production maturity.