AI Agent Book: a study map for agent engineering
Reading agent.py and the chapter-1 experiment runner
Inspect the mechanisms behind the book’s first reproducibility lesson.
What you will learn
- Start with the enum and messages
- Follow evidence generation
- Watch the edge cases
Before you start
- Python environment basics
- Access to a supported model provider for live experiments
Turn one context lesson into a documented experiment and design decision.
Key takeaways
- An enum label needs message-level verification.
- An evidence hash is provenance, not correctness.
- Educational code execution needs isolation.
Start with the enum and messages
ContextMode in agent.py names five conditions. Follow the method that assembles messages for each turn, then compare actual provider requests rather than inferring behavior from the enum name.
The ToolRegistry includes an educational code interpreter. Treat any path that runs generated code as untrusted until it is isolated; do not point this experiment at personal files.
Follow evidence generation
The experiment runner defines contract checks for the context modes, computes outcomes and token usage, and writes evidence plus a SHA-256 record. Read how it distinguishes missing data from a failed trial.
A hash protects the recorded artifact from unnoticed edits; it does not prove the model answer was correct or the comparison was statistically sound.
Watch the edge cases
A provider may omit reasoning fields, return a tool error or vary across repeated calls. The runner’s assertions and null values should be reflected in your report rather than silently converted into success.
We inspected two Python files, not every chapter implementation or external dependency.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Locate the mode enum and message builder.
- 2
Read evaluate_context_contract and outcome calculation.
- 3
Identify how unavailable provider data is represented.
Copy-ready example
from agent import ContextMode
# inspect the actual request messages for ContextMode.NO_TOOL_RESULTSFrequently asked questions
Does the runner establish causality?
It can check intended message differences; repeated controlled trials are still needed for a causal claim.
Is the code interpreter a production sandbox?
Do not assume that. Review and isolate it before executing untrusted code.
Sources
- AI Agent Book / chapter1/context/agent.pySource checked 2026-09-26
- AI Agent Book / chapter1/context/run_experiment_1_1.pySource checked 2026-09-26
- AI Agent Book / chapter1/context/README.mdSource checked 2026-09-26