OpenAI Skills
OpenAI Skills cost and latency: measure retrieval, context and accepted outcomes
Separate archive retrieval, host context and task execution costs. Build a measurement plan without inventing token savings, success rates or benchmark results.
What you will learn
- Measure the step that is actually slow
- Distinguish context policy from observed savings
- Price failures and maintenance honestly
Before you start
- Basic Git, Python and command-line concepts
- An explicit boundary for permitted repository inspection and changes
Explain the chapter’s source behavior and apply its acceptance checklist without confusing a catalog with its host.
Key takeaways
- Selected-folder size is not archive download cost.
- A short diagnostic snippet does not imply a small log transfer.
- Measure cost per accepted task, including correction and maintenance.
Measure the step that is actually slow
The download installer fetches a whole repository ZIP even when only one nested skill is selected. The response is read into memory before being written to a temporary archive. Sparse Git retrieval takes a different path with filtered objects and selected directories. Neither implementation makes the selected folder’s byte size equal to the total transfer or peak memory cost.
Listing also has its own semantics: list-skills.py requests one GitHub contents endpoint, keeps directory entries and sorts their names. Its installed annotation checks directory names beneath the installer’s local skills root. It is not a scan of every skill source that a host might discover, nor does it prove that the installed directory matches the requested revision.
Distinguish context policy from observed savings
Current official documentation describes progressive disclosure: a host initially uses skill names and descriptions, then loads the selected instructions. That is a context-management mechanism, not a measured token-saving percentage for this repository. This review does not benchmark a model or infer a cost reduction from that design alone.
For a real evaluation, keep the task, model, host build, tool access and acceptance criteria fixed. Record selected skills, loaded material, tool calls, elapsed time and whether the output was accepted without repair. A smaller prompt that selects the wrong workflow or repeatedly retries a command can cost more per useful result than a slightly larger prompt that succeeds.
Price failures and maintenance honestly
The CI helper can download a complete run log and only afterward select a short snippet. Reducing the visible report does not necessarily reduce upstream log-transfer volume. Its max-lines and context settings bound the extracted presentation window, not all network traffic or all information exposed to the local process.
Use separate totals for retrieval, local processing, model usage, external-service charges and human correction. Include migration work because the source repository is deprecated. The measurement record below intentionally leaves timings and costs null: the isolated probes established function behavior, not real installation latency, live diagnosis quality or production operating cost.
Implementation steps
- 1
Choose one fixed task and define acceptance before measuring.
- 2
Separate retrieval, helper, model and review timings.
- 3
Record misses, retries and unauthorized-action attempts.
- 4
Publish distributions only after real controlled runs.
Copy-ready example
{"sourceRevision":"49f948f","realTasks":0,"archiveBytes":null,"peakMemoryBytes":null,"acceptedTaskCost":null,"diagnosisP95Ms":null,"measuredTokenSavings":null}Frequently asked questions
How much money does this skill catalog save?
No savings figure was measured. Compare accepted tasks under a controlled host, model and tool configuration before making that claim.
Does --max-lines limit the downloaded log?
In the inspected helper it controls the extracted report window after log retrieval, not a network byte limit.
Sources
- README.mdSource checked 2026-09-08
- skills/.system/skill-installer/SKILL.mdSource checked 2026-09-08
- skills/.system/skill-installer/LICENSE.txtSource checked 2026-09-08
- skills/.system/skill-installer/scripts/install-skill-from-github.pySource checked 2026-09-08
- skills/.system/skill-installer/scripts/github_utils.pySource checked 2026-09-08
- skills/.system/skill-installer/scripts/list-skills.pySource checked 2026-09-08
- skills/.system/skill-creator/scripts/quick_validate.pySource checked 2026-09-08
- skills/.system/skill-creator/references/openai_yaml.mdSource checked 2026-09-08
- skills/.curated/gh-fix-ci/SKILL.mdSource checked 2026-09-08
- skills/.curated/gh-fix-ci/scripts/inspect_pr_checks.pySource checked 2026-09-08
- skills/.curated/gh-fix-ci/agents/openai.yamlSource checked 2026-09-08
- skills/.system/plugin-creator/SKILL.mdSource checked 2026-09-08
- Current official skills guideSource checked 2026-09-08