No AI Slop: evidence-led editing, packaging and evaluation
Evaluate No AI Slop: retained facts, revision effort and model cost
Design a clean baseline for editing quality and account for self-review, output length and human repair without inventing a benchmark score.
What you will learn
- Choose an outcome the reader actually needs
- Use a clean baseline and independent inspection
- Count the whole edit cycle
Before you start
- A draft whose facts can be checked
- Basic understanding of assistant instructions and plugin scope
Inspect pattern evidence, preserve meaning and distinguish package checks from unmeasured editing outcomes.
Key takeaways
- Preserved facts outrank shorter output.
- A self-check file is not a published benchmark.
- Measure retries and human repair, not just one model response.
Choose an outcome the reader actually needs
The goal can be clearer reading, recognizable voice or fewer unsupported claims. These are different outcomes from fewer words. Define protected facts and a small set of voice traits before comparing drafts. A candidate that removes a real caveat fails even if a reviewer prefers its rhythm or it uses fewer tokens.
The repository’s eval.md provides questions for self-review, not an observed benchmark table. No quality percentage, latency result or savings estimate follows from its presence. Our package fixtures measure validation behavior and contribute no evidence about reader comprehension or the model’s editing skill.
Use a clean baseline and independent inspection
Give baseline and candidate the same draft, audience, format and preservation constraints in separate clean conversations. Pin host and model versions and save prompts and outputs. If the baseline already inherits the candidate skill, the comparison cannot isolate the skill’s contribution. Repeating one attractive example is insufficient to cover voice, quotation and factual uncertainty.
Include a release note, a personal account, technical terminology and quoted material. Have a reviewer compare preserved facts before judging style, preferably without knowing which condition produced each text. Record every omission, invention and necessary human repair. This is a proposed evaluation design; we did not recruit readers or run model trials.
Count the whole edit cycle
Track input context, generated output, retries and reviewer time. Loading rules and a checklist adds context; repair loops may add more work. Whether the host caches instructions or bills additional reasoning depends on that service. Keep unmeasured token and cost fields unknown instead of promising cheaper editing.
A useful decision can be narrow: adopt for a particular kind of internal draft where fact retention and review effort meet your criteria, while excluding quotations or specialized material. Publish the failed cases alongside successes. An average preference score should not hide a single invented customer claim that would block your actual release.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Define protected facts and voice traits.
- 2
Compare clean baseline and candidate conversations.
- 3
Inspect omissions before style preference.
- 4
Record model usage and human repair separately.
Copy-ready example
{
"draftId": "synthetic-release-note",
"protectedFacts": 3,
"factsRetained": null,
"unsupportedAdditions": null,
"inputTokens": null,
"outputTokens": null,
"humanRepairMinutes": null,
"modelTrialExecuted": false
}Frequently asked questions
What quality improvement was measured here?
None. We provide an evaluation design and explicitly separate it from package tests.
Can the tool guarantee lower token cost?
No. Rules, self-review and retries may add cost; the configured host and measured workflow decide.
Sources
- No AI Slop / skills/no-ai-slop/eval.mdSource checked 2026-09-14
- No AI Slop / skills/no-ai-slop/SKILL.mdSource checked 2026-09-14