OpenAI Plugins: package examples, portable formats and connection boundaries
Plugin performance and cost: separate package loading, model work and service calls
Design an evidence-based plugin trial without assigning one price or speed claim to a heterogeneous catalogue of skills and connected services.
What you will learn
- A catalogue cannot have one meaningful runtime cost
- Measure one package against one acceptance task
- Optimize the failing phase rather than removing safeguards
Before you start
- Basic JSON and directory-path knowledge
- Understanding of skills and external-service permissions
Distinguish source examples from current format guidance and inspect a package without confusing metadata with runtime proof.
Key takeaways
- Different packages do not share one runtime cost model.
- Measure final acceptance, not directory loading alone.
- Call reduction is useful only when evidence and authorization remain intact.
A catalogue cannot have one meaningful runtime cost
The repository contains different kinds of packages. A skills-only workflow and a connected design workflow exercise different resources. Package metadata does not establish model token usage, remote-service charges, quota behavior or review time. No repository-wide cost or speed number follows from the manifest format.
Separate discovery and loading from actual task execution. An entry appearing quickly in a directory says nothing about the latency of its external tools. A workflow that produces an answer faster can still require more human repair if it misses required evidence or takes an unauthorized action.
Measure one package against one acceptance task
Use a synthetic task with a fixed expected result and record the host version, plugin revision, selected model and connection state. Measure time to first useful result, final acceptance time, tool calls, retries and human corrections. Record actual usage from the relevant host or service rather than inferring it from the package name.
For a skills-only notes trial, check decisions, owners and unanswered questions. For a connected design trial, define the required source evidence and permitted operations before measuring. These tasks are not interchangeable benchmarks. This review ran no live model or remote-service performance trial.
Optimize the failing phase rather than removing safeguards
If the wrong skill is selected, improve its trigger description and trial inputs. If a remote call dominates latency, inspect service behavior and request scope. If repeated review dominates, strengthen acceptance criteria and evidence capture. Those are diagnostic hypotheses, not measured claims about the Figma service or any host implementation.
Avoid treating fewer tool calls as the sole success metric. Skipping a necessary source inspection can lower call count while making the output less reliable. Keep authorization and factual acceptance fixed when comparing configurations, and report failures alongside timings rather than removing them from the sample.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Choose one package and synthetic acceptance task.
- 2
Record host, package, model and connection versions.
- 3
Measure execution, retries and human correction.
- 4
Compare configurations without dropping failed trials.
Copy-ready example
{
"trialProposal": true,
"packageRevision": null,
"hostVersion": null,
"accepted": false,
"toolCalls": null,
"retries": null,
"modelUsage": null,
"humanRepairMinutes": null,
"benchmarkExecuted": false
}Frequently asked questions
Are all plugins free because the examples are public?
Public example files do not establish the pricing or quotas of models and connected services.
Does fewer tool calls prove a better workflow?
No. Required evidence and accepted output must remain equivalent.
Sources
- OpenAI Plugins / README.mdSource checked 2026-09-14
- OpenAI Plugins / plugins/figma/README.mdSource checked 2026-09-14
- OpenAI — Package your plugin (checked 2026-09-14)Source checked 2026-09-14