TesterArmy e2e
TesterArmy e2e versus scripted E2E tests
Use adaptive steps where locators are brittle, and keep deterministic checks where they work
What you will learn
- Preserve strong assertions
- Compare engine surfaces
- Make adoption measurable
Before you start
- A disposable app or test account
- A web or mobile engine
- A deliberate model-provider choice for agent steps
Use two runs and one controlled UI mutation before trusting cached actions
Key takeaways
- Adaptive navigation has a model cost.
- Locator assertions remain valuable.
- Web and mobile engines need separate validation.
Preserve strong assertions
A conventional test specifies locators and expected values. TesterArmy e2e lets that style coexist with agent steps, so a stable invoice amount or status can remain an explicit assertion.
Natural language helps with variable navigation but introduces a model route and more ambiguity. Ask whether the workflow actually changes enough to justify that cost.
Compare engine surfaces
The README lists web engines through Playwright and mobile engines through simulators or emulators. They share the high-level test API but differ in available actions, observation quality and environment setup.
A single test plan does not prove equal coverage across platforms. Keep platform-specific fixtures and test the accessibility surface the engine uses.
Make adoption measurable
Run the same user journey with scripted steps and with one agent step. Compare valid passes, false passes, runtime, maintenance edits and provider cost across app revisions.
We did not rank TesterArmy e2e against Playwright, Appium or other agentic frameworks. The decision should follow your own app changes and observed failure modes.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Keep explicit assertions for stable invariants.
- 2
Try an agent step on a genuinely variable path.
- 3
Compare failures and maintenance across revisions.
Copy-ready example
stable path -> scripted action + explicit assertion
variable path -> agent action + explicit assertion
compare -> pass quality + runtime + upkeepFrequently asked questions
Should all locator steps become natural language?
No. Keep stable checks deterministic and reserve adaptive actions for steps that benefit.
Is replay a replacement for assertions?
No. A later assertion is part of the documented trace-verification contract.
Sources
- TesterArmy e2e / README.mdSource checked 2026-10-14
- TesterArmy e2e / docs/quickstart.mdxSource checked 2026-10-14
- TesterArmy e2e / docs/reference/engine.mdxSource checked 2026-10-14
- TesterArmy e2e / docs/reference/runner.mdxSource checked 2026-10-14
- TesterArmy e2e / packages/e2e/src/agent/replay.tsSource checked 2026-10-14