Context Mode
Build a Context Mode evidence-recall lab with replayable fixtures
Design a small project that tests retrieval correctness, byte boundaries, source updates and session scope without turning speculative improvements into release claims.
What you will learn
- Version expected facts and supporting passages, not just prompts.
- Derive fixtures from actual branches and byte boundaries.
- Label proposed improvements and unmeasured results honestly.
Before you start
- Basic JSON and MCP concepts
- An isolated test client and invented data
Explain the chapter’s actual implementation boundary and verify it with an explicit evidence fixture.
Key takeaways
- Version expected facts and supporting passages, not just prompts.
- Derive fixtures from actual branches and byte boundaries.
- Label proposed improvements and unmeasured results honestly.
Make the expected answer a versioned artifact
A useful extension project is an evidence-recall lab: a repository of invented documents, questions and expected supporting facts. Store a content hash and a manifest beside each fixture. Include a release threshold, a rare identifier, an explicit negation and a question with no answer. This makes future package upgrades testable against a stable task instead of relying on a memorable demonstration.
Separate the answer from its evidence requirement. For example, the expected timeout must include its unit and source revision, while an absent migration decision must remain unknown. A model that repeats the right number from a previous conversation has not proven that retrieval worked. Use fresh test sessions and compare the actual returned passage with the fixture manifest.
Turn source-code observations into experiments
Create dense single-line, CJK and emoji fixtures around the store’s default byte cap. Add repeated headings to explore the source/title fusion key, plus a typo query with no initial match and another with a weak initial match. These cases follow concrete implementation branches inspected in this series. They are proposed experiments, not claims that the project currently fails them.
Add dated events for timeline ordering, an edited file-backed source, and two isolated project markers. Record whether the result is current, complete and correctly scoped. A source-error scenario should be visible in the report even if other stores still return passages. Keep session cleanup an explicitly reviewed test operation rather than silently purging a developer’s working history between runs.
Produce a report readers can reproduce
Each run should name the source commit or package version, host, runtime, fixture hash, query mode and all unmeasured fields. Keep raw evidence local under an appropriate retention policy and publish only the harmless fixture results. A compact HTML table or SVG showing pass, fail and unknown states is more useful here than an animated dashboard that conceals missing evidence.
Possible future improvements include exposing incomplete-source status, distinguishing fallback timestamps and offering clearer candidate-budget diagnostics. These are editorial proposals derived from the inspected code, not an upstream roadmap or a promise of implementation. The first deliverable is a repeatable correctness report; richer visualizations should make a specific failure easier to understand rather than inflate the project’s apparent maturity.
Implementation steps
- 1
Create an invented corpus and an expected-evidence manifest.
- 2
Add ranking, multibyte, freshness and scope fixtures.
- 3
Run in an isolated host with explicit cleanup decisions.
- 4
Publish a reproducible pass/fail/unknown report without sensitive logs.
Copy-ready example
{"fixture":"release-larch","sourceHash":"record before run","queryMode":"relevance","expected":["18 seconds","rollback above 2 percent"],"observed":null,"status":"not run","upstreamRoadmap":false}Frequently asked questions
Are these future features announced by the maintainer?
No. They are clearly labeled editorial proposals and test ideas based on the inspected implementation.
Does the lab need Three.js?
Not initially. A readable evidence table and a focused diagram are enough unless an interactive view clarifies a real ranking or state-transition problem.
Sources
- README.mdSource checked 2026-09-07
- package.jsonSource checked 2026-09-07
- LICENSESource checked 2026-09-07
- src/store.tsSource checked 2026-09-07
- src/search/unified.tsSource checked 2026-09-07
- src/server.tsSource checked 2026-09-07
- src/executor.tsSource checked 2026-09-07
- src/security.tsSource checked 2026-09-07
- src/session/purge.tsSource checked 2026-09-07
- tests/store-bytecap.test.tsSource checked 2026-09-07