AutoHedge
AutoHedge cost and backtests: why flat prices can still show simulated gains
Separate token and tool costs from terminal previews, then inspect the experimental paired-fill assumption before interpreting any return number.
What you will learn
- Measure the whole task, not the panel
- Inspect the experimental fill model
- Design a more informative evaluation
Before you start
- Basic Python, Git and dependency-management knowledge
- A fictional evidence task with no wallet or signing authority
Explain the inspected implementation and its counterexamples without mistaking a simulation or generated text for a verified financial outcome.
Key takeaways
- Display truncation does not cap provider cost.
- The experimental backtest assumes paired fills around each close.
- Toy accounting gain is not evidence of real profitability.
Measure the whole task, not the panel
The REPL truncates a displayed result to 2,000 characters, but only after the director returns. That does not limit model input/output tokens. The Exa helper requests two results and a 20-character context field while also requesting text and a structured summary; it serializes the returned object, not a guaranteed 20-character total response.
Worker max_loops=1 settings do not establish exactly one provider request for the whole system. Handoffs, tools and dependency behavior need observation. Track total attempts, failures, provider usage and the number of accepted research artifacts. Cost per accepted artifact must include unsuccessful work; no measured latency, price or token-saving claim is supplied here.
Inspect the experimental fill model
experimental/market_making.py contains a simplified backtest separate from the default agent path. For each close price it calculates a bid below the close and an ask above it, buys if quote balance allows, then sells if base inventory allows. It assumes both fills within that row without requiring high/low crossing, queue position, fees or slippage evidence.
The exact backtest function was selected by AST and given two invented rows with the same close of 100 through a fake pandas-like object. With the source example parameters, it produced four simulated fills and a final accounting value approximately 10000.2 from 10000. This is an illustration of the fill assumption, not a historical backtest, expected return or investment result.
Design a more informative evaluation
The teaching arithmetic below shows how one same-row buy/sell pair creates 0.1 of accounting gain even when the observed close does not move. Changing the assumed fill mechanism changes the result. A positive printout can therefore reveal a simulator convention rather than an exploitable market opportunity.
A future evaluation should separate no-fill, partial-fill, adverse movement and fee cases, and preserve unavailable data rather than substitute a confident answer. Compare research evidence quality independently from any simulated strategy metric. Real performance, predictive calibration and a deployable trading policy remain unmeasured.
Implementation steps
- 1
Record complete provider/tool usage and accepted artifacts.
- 2
Inspect simulator fill conditions before reading its return number.
- 3
Add no-fill, fees and adverse-case fixtures.
- 4
Keep simulated accounting separate from real market results.
Copy-ready example
const close = 100, spreadFraction = 0.001, quantity = 1;
const assumedBuy = close * (1 - spreadFraction / 2);
const assumedSell = close * (1 + spreadFraction / 2);
console.log(Number(((assumedSell - assumedBuy) * quantity).toFixed(6)));
// 0.1: invented paired fills, not observed market profitFrequently asked questions
Was a profitable strategy demonstrated?
No. Two invented flat-price rows demonstrate a simulator assumption; no real dataset, fees, live orders or investment returns were tested.
Does the terminal’s short result prove low token cost?
No. Truncation happens after the result is produced and does not constrain provider usage.
Sources
- autohedge/cli.pySource checked 2026-09-08
- autohedge/workers.pySource checked 2026-09-08
- autohedge/tools/exa_search_tool.pySource checked 2026-09-08
- experimental/market_making.pySource checked 2026-09-08