TesterArmy e2e
TesterArmy e2e explained: agent steps inside an ordinary test
Understand natural-language actions, explicit assertions and recorded replay
What you will learn
- Start with the test contract
- Understand the replay claim
- Bound the evidence
Before you start
- A disposable app or test account
- A web or mobile engine
- A deliberate model-provider choice for agent steps
Use two runs and one controlled UI mutation before trusting cached actions
Key takeaways
- Agent steps and locator assertions can coexist.
- Replay is conditional on a verified trace.
- No model call is needed for tests without agent steps.
Start with the test contract
TesterArmy e2e runs end-to-end tests for web and mobile apps. A test can mix a natural-language `agent.act` step, a model-judged `agent.assert`, and locator assertions such as `expect(screen.getByRole(...))`.
The natural-language step lets a model choose actions against the observed app. A conventional locator assertion remains useful for a stable invariant. The README explicitly says tests without agent steps make no model calls.
Understand the replay claim
When a later assertion verifies an agent step, the framework can record its actions and replay them on a subsequent run until the app changes. This is a conditional cache path, not proof that every run is free of model calls.
The pinned source implements trace recording and a replay route with its own action budget and end-state check. A changed screen, missing target or unsafe trace can stop replay and return control to the live step.
Bound the evidence
We inspected README, setup and security documents plus runner, agent and replay source at commit `aff6f34`. We did not execute a browser or mobile test.
This series treats the documented workflow as a design to evaluate. It does not convert repository examples into measured reliability, cost savings or compatibility guarantees.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Choose one disposable web or mobile target.
- 2
Write one agent step followed by a checkable assertion.
- 3
Run twice and inspect whether the second run replays.
Copy-ready example
await agent.act('open the account settings');
await agent.assert('the billing panel is visible');
await expect(screen.getByRole('heading')).toContainText('Billing');Frequently asked questions
Does every test require an AI provider?
The README says tests without agent steps need no model.
Did this series reproduce replay?
No. It analyzes pinned documentation and source without executing a test.
Sources
- TesterArmy e2e / README.mdSource checked 2026-10-14
- TesterArmy e2e / packages/e2e/src/agent/act.tsSource checked 2026-10-14
- TesterArmy e2e / packages/e2e/src/agent/replay.tsSource checked 2026-10-14
- TesterArmy e2e / packages/e2e/src/cache/recorder.tsSource checked 2026-10-14