Context Mode
Measure Context Mode: response bytes, answer recall, latency and stored data
Build a controlled evaluation that does not confuse advertised context reductions with token billing, retrieval quality or end-to-end task speed.
What you will learn
- Count all calls needed for the answer, not one selected response.
- Measure tokens separately from UTF-8 bytes.
- Report recall and cold/warm latency beside storage size.
Before you start
- Basic JSON and MCP concepts
- An isolated test client and invented data
Explain the chapter’s actual implementation boundary and verify it with an explicit evidence fixture.
Key takeaways
- Count all calls needed for the answer, not one selected response.
- Measure tokens separately from UTF-8 bytes.
- Report recall and cold/warm latency beside storage size.
Define a denominator that matches the question
For a byte comparison, retain the original tool-output bytes and the complete bytes returned across every Context Mode call needed for the answer. Include indexing metadata, searches, retries and follow-up retrievals. A reduction percentage based on one tiny search response and a large original file is not an end-to-end task measurement when several other calls were required.
Model token counts and billing need their own observation method. UTF-8 bytes do not map to a fixed number of tokens across English, Chinese, code and JSON. Host-side caching or unrelated prompts can further change billed input. Record those variables separately and leave unmeasured fields null. The project’s advertised percentage should remain a labeled upstream example, not a prefilled result in your own report.
Pair efficiency with evidence recall
Use a fixed corpus containing common facts, a rare identifier, a negation and a value close to a chunk boundary. Define the expected facts before asking the assistant questions. Measure how many required facts are supported by retrieved passages, including units and conditions. A shorter answer that silently reverses a negation or loses a rollback threshold fails the task regardless of its byte savings.
The inspected store uses a default 4096-byte chunk cap, and its regression tests cover dense lines, blank-line sections, CJK text and emoji. These are upstream test cases we inspected, not newly executed benchmark results. Add equivalent shapes to your evaluation because a character-based size assumption can understate multibyte storage and change the number of chunks searched.
Separate cold work from repeated questions
Measure first indexing time, first query latency and repeated query latency separately. Also record database size and the total number of tool round trips. Comparing an already indexed fixture against a baseline that must reload everything can be useful for repeated work, but it should not be presented as a fair first-use comparison. Repeated indexing can itself change stored state and needs a controlled reset procedure.
Include a source edit and a failed retrieval in the run matrix. File-backed refresh work can move cost into a later search, while recovery calls add latency and context. Make the final decision from task completion, evidence recall and operational effort together. This chapter supplies a measurement design; no universal speedup, price saving or production latency result is claimed.
Implementation steps
- 1
Freeze the corpus, questions and expected facts.
- 2
Measure a direct-output baseline and a fresh index path.
- 3
Repeat with a warm index, edited file and missing fact.
- 4
Reject savings claims that omit incorrect or unsupported answers.
Copy-ready example
{"scenario":"cold-index","rawBytes":null,"allReturnedBytes":null,"modelInputTokens":null,"requiredFacts":4,"supportedFacts":null,"indexMs":null,"queryMs":null,"databaseBytes":null}Frequently asked questions
Can byte reduction be used as a token-billing reduction?
Not directly. Measure the actual tokenizer or provider usage and account for every necessary call and host-side behavior.
Were the upstream chunk tests run for this article?
No. Their source was inspected to design relevant fixtures; the full upstream test suite was not executed.
Sources
- README.mdSource checked 2026-09-07
- package.jsonSource checked 2026-09-07
- LICENSESource checked 2026-09-07
- src/store.tsSource checked 2026-09-07
- src/search/unified.tsSource checked 2026-09-07
- src/server.tsSource checked 2026-09-07
- src/executor.tsSource checked 2026-09-07
- src/security.tsSource checked 2026-09-07
- src/session/purge.tsSource checked 2026-09-07
- tests/store-bytecap.test.tsSource checked 2026-09-07