Heretic
Heretic 对比与选型：abliteration、微调还是评估实验室
比较自动/手工消融、微调、现成模型和托管选项的证据、控制、成本与风险。
你将学会
- Explain the KL/refusal optimization objective
- Run a pinned model experiment
- Design broader safety evaluation
开始前需要
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
先看结论
- Select a method based on the research question and governance burden.
- Use an equal prompt and capability fixture for comparisons.
- Fewer refusals alone do not establish a better model.
Compare the actual decision
用同一模型族、提示清单和能力套件比较拒答、保真度、事实性、成本、许可证和运维；减少拒答不等于更安全或更有用。
Use the same model family, prompt manifest, capability suite, and publication policy for a fair bake-off. Compare refusal behavior, KL or other fidelity measures, factuality, coding/math tasks, calibration, latency, cost, and human judgments.
Make trade-offs explicit
Heretic's strengths are automation, flexible kernels, and reproducible CLI runs. Its boundaries include supported architectures, GPU/CPU cost, metric sensitivity, and the responsibility to evaluate transformed artifacts. A hosted option may win on operations; a manual method may win when a researcher needs fine-grained hypotheses.
Do not equate fewer refusals with better alignment or usefulness. Document what each candidate cannot answer, how it handles unsafe requests, and how quickly a team can revoke or replace it.
A selection record should name the owner, evaluation window, release policy, and rollback trigger so a later team can understand why the method was chosen.
A reviewable next step
Use the pinned source evidence, record versions and assumptions, and ask a qualified reviewer to validate the result before adopting it.
Use the pinned source evidence, record versions and assumptions, and ask a qualified reviewer to validate the result before adopting it. Keep the fixture, output, and decision together so another developer can reproduce the conclusion.
如何选择
| 比较维度 | 方案 A | 方案 B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
实施步骤
- 1
Define behavior, fidelity, cost, license, and risk criteria.
- 2
Run equal baseline and candidate evaluations.
- 3
Add human review and operational tests.
- 4
Record decision, limits, and reevaluation triggers.
可复制示例
question -> methods
model/prompt fixture -> refusal + fidelity + capability
ops + license + review -> decision + exit plan常见问题
Is Heretic a replacement for fine-tuning?
No. It targets a different intervention surface; choose based on whether directional weight edits answer your research question.
How do I compare an existing decensored checkpoint?
Use the same model family, prompts, software context where possible, and report provenance plus human evaluation.
资料来源
- Heretic README (captured 2026-08-31)来源核查 2026-08-31
- Heretic repository来源核查 2026-08-31