Heretic
Heretic Features and Quickstart: Run a Reproducible Abliteration Study
A guarded walkthrough of Heretic's model support, evaluation, quantization, research plots, and evidence capture.
What you will learn
- Explain the KL/refusal optimization objective
- Run a pinned model experiment
- Design broader safety evaluation
Before you start
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
Key takeaways
- A safe quickstart starts with a frozen baseline and authorized model.
- Evaluation, upload, chat, and serving are separate gates.
- Research plots and metrics need versioned prompts and hardware context.
Define the experiment before the command
Choose a model revision, license, hardware budget, prompt groups, and a private output location before installing. The useful question is not simply ‘does it refuse less?’ but whether behavior changes are acceptable across harmless, safety-sensitive, factuality, and capability prompts.
Keep the original model immutable. Record the model identifier, revision, Python/PyTorch versions, Heretic configuration, seed if applicable, and the exact evaluation prompts so a later comparison has a stable baseline.
Use the documented interfaces
The README documents the `heretic` CLI, `--help`, configuration through `config.default.toml`, optional `research` extras, `--evaluate-model`, `--plot-residuals`, and `--print-residual-geometry`. Start with one small dense model, then add quantization or research plots only when their trade-offs are understood.
After optimization, inspect the offered save, upload, chat, and benchmark actions as separate decisions. A generated checkpoint should not be uploaded or served until its provenance, license, and human review are complete.
Make results legible
Report refusal counts and KL divergence together with prompt-set composition, model revision, software, GPU, quantization, runtime, and warnings. The README explicitly notes that values can vary by platform and hardware.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Freeze model, prompts, license, and output policy.
- 2
Install pinned Python/PyTorch/Heretic dependencies.
- 3
Run baseline, transform, and built-in evaluation.
- 4
Review results and provenance before any export or serving.
Copy-ready example
python3.10 -m venv .venv
. .venv/bin/activate
pip install -U heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507
heretic --model Qwen/Qwen3-4B-Instruct-2507 --evaluate-model <checkpoint>Frequently asked questions
Which model should I start with?
Use a small, supported model you are licensed to download, then scale only after the evaluation harness is stable.
What does `--plot-residuals` add?
It projects per-layer residuals with PaCMAP and writes plots plus an animation; it is CPU-expensive research output, not a safety certificate.
Sources
- Heretic README (captured 2026-08-31)Source checked 2026-08-31
- Heretic repositorySource checked 2026-08-31