Heretic
Heretic: futuro y proyecto de un laboratorio de edición auditable
Manifiestos, puertas de evaluación, visualización residual y publicación responsable.
Qué aprenderás
- Explain the KL/refusal optimization objective
- Run a pinned model experiment
- Design broader safety evaluation
Antes de empezar
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
Conclusiones clave
- An evidence manifest turns a model edit into a reproducible claim.
- Capability profiles should be explicit for each architecture and quantization.
- Interactive visuals are optional; accessible text remains canonical.
The project brief
Diseña un laboratorio privado que guarde revisiones, prompts, configuración, hardware, métricas, hashes y decisiones; Markdown/SVG es canónico y Three.js opcional.
Start with one supported dense model and one local GPU. Add multimodal, MoE, quantized, and research-plot variants only as explicit capability profiles, because the README does not promise universal architecture support.
Evidence manifest
Record model and tokenizer revisions, license, dependency lock, hardware, configuration, prompt hashes, direction settings, Optuna budget, refusal/KL/capability results, output hashes, reviewer, and publication decision. Keep raw prompts and weights in controlled storage; publish sanitized fixtures and hashes.
Use Markdown and local SVG as the canonical report. A Three.js view can optionally explore layers and residual trajectories, but it must not hide the textual metrics, warnings, or accessibility path.
Roadmap and gates
Phase one is trace-only reproducibility; phase two adds advisory quality and safety checks; phase three can enforce release gates for license, provenance, regression, and human sign-off. Future research can explore better objectives, architecture coverage, and interpretability while retaining rollback and quarantine controls.
Cómo elegir
| Criterio | Opción A | Opción B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Pasos de implementación
- 1
Define model, prompt, run, metric, artifact, and review fields.
- 2
Add offline fixtures and baseline/candidate parity checks.
- 3
Gate publication on license, provenance, safety, and capability review.
- 4
Promote trace-only checks to advisory and enforced release policies.
Ejemplo para copiar
run:
model_revision: <sha>
prompt_manifest_sha256: <hash>
config: config.default.toml
hardware: <gpu-pytorch-cuda>
metrics: { refusal: <n/total>, kl: <value> }
checkpoint_sha256: <hash>
reviewer: <id>
release: holdPreguntas frecuentes
Should the lab store model weights in Git?
No. Keep large or sensitive artifacts in controlled storage and commit only sanitized fixtures, hashes, and manifests.
Is Three.js required for the project?
No. Add it only if a layer/residual exploration view helps; preserve accessible text, SVG, and downloadable evidence.
Fuentes
- Heretic README (captured 2026-08-31)Fuente verificada 2026-08-31
- Heretic repositoryFuente verificada 2026-08-31