AI Agent Book: a study map for agent engineering
A four-week learning project from AI Agent Book
Turn one context lesson into a documented experiment and design decision.
What you will learn
- Week one: read and inspect
- Weeks two and three: run controls
- Week four: write a decision
Before you start
- Python environment basics
- Access to a supported model provider for live experiments
Turn one context lesson into a documented experiment and design decision.
Key takeaways
- Reproducibility starts before the first call.
- Failed arms must stay visible.
- Learning value does not require a positive result.
Week one: read and inspect
Read the opening chapter and context guide. Trace how full and no_tool_results modes assemble messages. Write a hypothesis about one small calculator task before invoking a provider.
Freeze the book commit, package lock and model ID so a later reader can tell which setup produced each observation.
Weeks two and three: run controls
Use synthetic tasks, repeat each mode, archive raw requests and record contract checks, correctness, cost and failures. Keep the code interpreter disabled or isolated unless it is essential to the task.
If an arm fails because of provider access or missing evidence, mark it unevaluable. Do not substitute a mocked pass for a live result.
Week four: write a decision
Summarize when tool results helped, when they did not and what the sample cannot establish. Decide whether your own agent needs the same context retention policy.
This is a proposed study, not a completed four-week experiment. A good result may be a well-documented negative or inconclusive finding.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Fix the revision, provider and hypothesis.
- 2
Run repeated synthetic comparisons with raw traces.
- 3
Publish a bounded conclusion and unresolved questions.
Copy-ready example
study:
question: do_tool_results_help
modes: [full, no_tool_results]
data: synthetic
repeats: record-count
conclusion: bounded_to_tasks_and_modelFrequently asked questions
Must the project last exactly four weeks?
No. The schedule is a pacing device; keep the evidence requirements.
Can I claim a universal context rule?
A small study supports only a bounded conclusion for its tasks and model.
Sources
- AI Agent Book / README.mdSource checked 2026-09-26
- AI Agent Book / chapter1/context/README.mdSource checked 2026-09-26
- AI Agent Book / chapter1/context/agent.pySource checked 2026-09-26
- AI Agent Book / chapter1/context/run_experiment_1_1.pySource checked 2026-09-26