Heretic
Heretic 架构：残差方向、权重核与 Optuna
从提示残差到矩阵正交化、参数搜索和研究可视化。
你将学会
- Explain the KL/refusal optimization objective
- Run a pinned model experiment
- Design broader safety evaluation
开始前需要
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
先看结论
- Prompt residuals and weight surgery are distinct stages.
- Float direction indices and flexible kernels enlarge the search space.
- The research visualization path is separate from model transformation.
A layered mental model
将 Heretic 拆成残差采集、方向构造、注意力/MLP 权重编辑、Optuna 目标和研究图五层，理解 per-layer 与浮点方向索引的搜索空间。
The special `per layer` direction index selects each layer's direction, while a float index interpolates between neighboring directions. This turns a discrete set of measured vectors into a larger search space, but also makes experiment provenance essential.
The ablation kernel
`max_weight`, `max_weight_position`, `min_weight`, and `min_weight_distance` describe a flexible weight shape over layers. Attention and MLP components receive separate parameters because the README reports that MLP interventions can damage capabilities more.
The optimization objective is a co-minimization: fewer refusals on harmful prompts and low KL divergence on harmless prompts. Neither term is a complete model-quality objective; add capability, calibration, bias, and misuse evaluations around it.
Research branch
Optional research features compute residual geometry, use PaCMAP to project layers, align consecutive projections, and emit PNG/GIF artifacts. Keep this CPU-heavy analysis off the critical transform path and preserve the source arrays or hashes used to create plots.
如何选择
| 比较维度 | 方案 A | 方案 B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
实施步骤
- 1
Map residual collection and direction construction.
- 2
Trace matrix orthogonalization by component and layer.
- 3
Inspect Optuna parameters and objective metrics.
- 4
Separate plots, evaluation, and export from the transform core.
可复制示例
harmful/harmless prompts
-> first-token residuals
-> layer directions
-> attention/MLP matrix orthogonalization
-> Optuna TPE (refusals + KL)
-> checkpoint + evaluation常见问题
Does Heretic edit every transformer layer equally?
No. It supports layer-specific directions and flexible, separately optimized weight kernels.
Is KL divergence a safety metric?
No. It measures deviation from the original model on a chosen prompt set; pair it with broader behavioral and human review.
资料来源
- Heretic README (captured 2026-08-31)来源核查 2026-08-31
- Heretic repository来源核查 2026-08-31