Octop explained: a self-hosted assistant for users, agents and channels
Measuring Octop: separate channel delay, model work and operator effort
Evaluate accepted tasks without assuming one process makes every workflow inexpensive.
What you will learn
- Define a useful completed task
- Account for the actual services
- Report scope and limits
Before you start
- Basic Python and service administration
- An isolated test environment
A proposed read-only dashboard makes unverified capabilities visible before use.
Key takeaways
- Delivery is not answer acceptance.
- Self-hosting still has service costs.
- Concurrency claims need measurements.
Define a useful completed task
Choose a fixed synthetic request and an acceptance checklist. Record receipt time, first response, final response and whether the output is correct. A delivered message is not necessarily an accepted answer.
Keep web and IM trials separate initially. Network delivery and provider latency can dominate different parts of the experience, so a single end-to-end number may hide the cause.
Account for the actual services
Record model usage, embedding work, storage and reviewer effort where observable. Local deployment does not remove model-provider bills, and a knowledge-base import adds work beyond ordinary chat.
Track retries and scheduled jobs explicitly. A failed task that runs again can consume tokens or send duplicate messages; measure accepted outcomes rather than only successful HTTP requests.
Report scope and limits
Repeat comparable trials with the same provider, model, corpus and concurrency. Include rejected answers and timeouts; do not select only the fastest warm response.
No Octop performance or cost benchmark was run here. The worksheet is a measurement plan, and the single-process architecture is not evidence of a particular throughput or percentage saving.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Fix a task and acceptance checklist.
- 2
Separate delivery, model and review time.
- 3
Repeat and include failures.
Copy-ready example
trial,surface,provider,model,received_at,first_response,accepted,retries,review_minutes
example,web,,,,,,,Frequently asked questions
Can I infer cost from process count?
No. Model, embedding, storage and review costs need separate observation.
Are the fields below measured?
No. They are a future trial record.
Sources
- Octop / README.mdSource checked 2026-09-23
- Octop / docs/architecture.mdSource checked 2026-09-23