Archify
Archify Performance and Cost: Benchmark Validation, Rendering, and Exports
Design a fair Archify benchmark for typed-IR validation, layout, rendering, source tracing, and export formats without confusing visual polish with correctness.
What you will learn
- Design fixed Archify benchmark fixtures
- Separate phase latency and operating cost
- Validate optimizations against evidence and accessibility
Before you start
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
Key takeaways
- Separate validation, layout, rendering, tracing, and export phases.
- Use fixed fixtures and p50/p95 measurements with output-size and resource data.
- Optimize cost per accepted diagram without removing evidence or accessibility checks.
Measure the full diagram pipeline
Archify work is more than rendering pixels. Fix the skill/compiler revision, input JSON size, node and edge counts, diagram type, theme, motion setting, source-link count, host, and export format. Measure parse, schema validation, layout, source-trace preparation, render, and export separately so a slow PNG step is not blamed on the validator.
Use three fixtures: a tiny flow for feedback latency, a representative architecture for normal use, and a stress graph with many nodes or long labels. Keep the authored facts constant across runs; otherwise a larger input can masquerade as a tool regression.
Report p50, p95, and output size
Record p50/p95 wall time, CPU and memory, node/edge counts, validation errors, output bytes, and cache state across repeated runs. Compare cold and warm executions, but never hide a validation failure behind a faster render. If source tracing or route interactions add work, measure them as explicit phases.
For visual diffs, normalize intentional metadata such as timestamps or generated IDs. A checksum is useful for identical self-contained output; a structural or perceptual diff is more useful when the renderer embeds non-semantic metadata.
Budget CI and human review
A CI job also pays for agent context, runner minutes, artifact storage, and reviewer attention. Track changed diagrams versus full rebuilds, cache hit rate, artifact retention, and time spent reviewing a candidate/last-good diff. The practical metric is cost per accepted, source-traceable diagram.
Do not optimize away source links, validation diagnostics, or accessibility checks. A smaller artifact that cannot explain an edge is cheaper only on paper; it shifts cost to debugging and review.
Safe optimization loop
Start with bounded JSON, deterministic IDs, and caching of immutable source extraction. Then test layout pruning, incremental rendering, or export parallelism behind the same fixture suite. Accept an optimization only when topology, source traces, export correctness, and visual readability remain within the team's rubric.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Pin compiler, fixture, diagram type, theme, motion, host, and export.
- 2
Measure cold/warm p50/p95 time, CPU, memory, errors, and bytes.
- 3
Track CI minutes, cache, storage, and reviewer time.
- 4
Re-run source-trace, visual, export, and accessibility checks after changes.
Copy-ready example
{
"fixture": "architecture-small.json",
"nodes": 3,
"edges": 2,
"p95_ms": { "validate": 0, "render": 0, "export": 0 },
"output_bytes": 0,
"review_minutes": 0
}Frequently asked questions
Is render time the main Archify cost?
Not necessarily. Agent context, validation, source tracing, export, CI minutes, storage, and human review can dominate; measure each phase.
Can a checksum prove visual equivalence?
Only for identical normalized bytes. For intentional metadata or layout changes, combine structural and perceptual diffs with source-trace checks.
Sources
- Archify README (captured 2026-08-31)Source checked 2026-08-31
- Archify repositorySource checked 2026-08-31