i-have-adhd: readable responses, adapters and evidence
Build a response-readability evaluation that can disagree with your expectations
Design a small, auditable project with clean conditions, blinded review and blocker-first decisions; distinguish the proposal from a shipped tool.
What you will learn
- Start with cases that can actually be completed
- Keep evidence and judgement separate
- Make the next experiment answer a narrow question
Before you start
- Basic Git and command-line knowledge
- Ability to separate observed behavior from an untested claim
Explain the inspected mechanism, plan a reversible trial and interpret evidence without treating style as correctness.
Key takeaways
- A case must be feasible under its runner permissions.
- Save raw answers before assigning scores.
- A proposed experiment is not an established result.
Start with cases that can actually be completed
A useful learning project is a response evaluation notebook, not another unsupported score badge. Include a direct answer, a multi-step task, a conceptual explanation, an ambiguous failure and an explicit complete-list request. Each case needs required facts, permitted tools and a clear success criterion that the chosen runner can satisfy.
The upstream tool-disabled action case illustrates why this matters: a grader cannot fairly demand a repository edit when the runner has no editing tools. Either provide an isolated fixture workspace and authorized tools or grade only the explanation the runner can produce. Decide that before collecting answers, and keep the same contract for both conditions.
Keep evidence and judgement separate
Store task ID, revision, model, host version, condition, trial and raw answer before adding any score. Present reviewers with blinded labels and keep the label mapping outside their prompt. Audit whether required facts survived, whether unknown causes stayed uncertain and whether an offered command matches the stated environment. A model judge is a reviewer, not ground truth.
Use the upstream rubric as a starting point for correctness, autonomy, actionability, safety and concision. Add human checks for your actual reading intention and preserve disagreement. Do not convert an editorial self-assessment into a claimed user study. If participating readers supply real work examples, obtain permission and remove secrets before sharing or sending them to a model.
Make the next experiment answer a narrow question
A practical extension would compare two error-reporting rules: one that asks for cause and fix immediately, and one that first distinguishes observation from hypothesis. Hold the task set and model constant, then inspect unsupported causal claims and follow-up task completion. This is a proposed experiment, not an improvement already demonstrated by this series.
A future interactive view could show the same task, both answers and highlighted omissions. It does not require a 3D scene; an accessible side-by-side comparison may better serve reading. The current deliverable is articles, SVG diagrams and fourteen offline adapter cases. No interactive evaluator, paid generation run or clinical effectiveness experiment is shipped here.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Define required facts and allowed tools for each case.
- 2
Capture clean baseline and candidate answers.
- 3
Blind conditions and review omissions and blockers.
- 4
Publish limitations alongside any aggregate result.
Copy-ready example
{
"caseId": "complete-list",
"allowedTools": [],
"requiredFacts": [
"retain all requested items",
"mark unknown values"
],
"conditions": [
"baseline",
"candidate"
],
"trialsPlanned": 3,
"executed": false
}Frequently asked questions
Is there already a Three.js evaluator on this page?
No. The interactive comparison is a proposal; these pages use static, accessible SVG diagrams.
Can simulated reader scores prove a benefit?
No. They are editorial checks, not independent observed outcomes.
Sources
- i-have-adhd / evals/README.mdSource checked 2026-09-12
- i-have-adhd / evals/RESULTS.mdSource checked 2026-09-12
- i-have-adhd / evals/rubric.mdSource checked 2026-09-12