Heretic
Heretic: quickstart de un estudio de abliteration reproducible
Modelos, evaluación, cuantización, gráficas de investigación y registro de evidencia.
Qué aprenderás
- Explain the KL/refusal optimization objective
- Run a pinned model experiment
- Design broader safety evaluation
Antes de empezar
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
Conclusiones clave
- A safe quickstart starts with a frozen baseline and authorized model.
- Evaluation, upload, chat, and serving are separate gates.
- Research plots and metrics need versioned prompts and hardware context.
Define the experiment before the command
Fija revisión, licencia, prompts y política de salida; ejecuta baseline, transformación y evaluación con la CLI y conserva hardware, configuración y revisión humana.
Keep the original model immutable. Record the model identifier, revision, Python/PyTorch versions, Heretic configuration, seed if applicable, and the exact evaluation prompts so a later comparison has a stable baseline.
Use the documented interfaces
The README documents the `heretic` CLI, `--help`, configuration through `config.default.toml`, optional `research` extras, `--evaluate-model`, `--plot-residuals`, and `--print-residual-geometry`. Start with one small dense model, then add quantization or research plots only when their trade-offs are understood.
After optimization, inspect the offered save, upload, chat, and benchmark actions as separate decisions. A generated checkpoint should not be uploaded or served until its provenance, license, and human review are complete.
Make results legible
Report refusal counts and KL divergence together with prompt-set composition, model revision, software, GPU, quantization, runtime, and warnings. The README explicitly notes that values can vary by platform and hardware.
Cómo elegir
| Criterio | Opción A | Opción B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Pasos de implementación
- 1
Freeze model, prompts, license, and output policy.
- 2
Install pinned Python/PyTorch/Heretic dependencies.
- 3
Run baseline, transform, and built-in evaluation.
- 4
Review results and provenance before any export or serving.
Ejemplo para copiar
python3.10 -m venv .venv
. .venv/bin/activate
pip install -U heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507
heretic --model Qwen/Qwen3-4B-Instruct-2507 --evaluate-model <checkpoint>Preguntas frecuentes
Which model should I start with?
Use a small, supported model you are licensed to download, then scale only after the evaluation harness is stable.
What does `--plot-residuals` add?
It projects per-layer residuals with PaCMAP and writes plots plus an animation; it is CPU-expensive research output, not a safety certificate.
Fuentes
- Heretic README (captured 2026-08-31)Fuente verificada 2026-08-31
- Heretic repositoryFuente verificada 2026-08-31