Heretic
Heretic: seguridad y operaciones de artefactos y evaluaciones
Gobierno de pesos, prompts, secretos, checkpoints, uploads, red y aprobación humana.
Qué aprenderás
- Explain the KL/refusal optimization objective
- Run a pinned model experiment
- Design broader safety evaluation
Antes de empezar
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
Conclusiones clave
- Weights and prompts deserve the same care as production data.
- Save, upload, chat, and serving need separate approvals.
- Incident recovery includes revoke, quarantine, restore, and deletion.
Protect the experiment surface
Aísla y cifra modelos y prompts, separa guardar/subir/chat/servir y exige revisar licencia, hashes, privacidad, métricas y aprobación antes de publicar.
Treat downloaded models and generated checkpoints as untrusted artifacts until licenses, hashes, provenance, and evaluation results are reviewed. Do not reuse a transformed checkpoint across projects without carrying its manifest forward.
Constrain external actions
The CLI can offer save, upload, chat, and benchmark actions after a run. Separate those permissions, require an approval gate for Hugging Face uploads or network serving, and keep tokens in a secret manager. A local chat smoke test should not silently expose the worker to the internet.
Prompt content and model outputs may include unsafe instructions. Sanitize reports, restrict logging, and do not let an agent widen evaluation scope or publish claims automatically.
Operate and recover
Monitor GPU jobs, download failures, disk growth, anomalous output, and evaluation drift. Rehearse revoke, quarantine, restore, deletion, and incident notification with a known-good baseline and a clear owner.
Cómo elegir
| Criterio | Opción A | Opción B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Pasos de implementación
- 1
Inventory models, prompts, credentials, outputs, and owners.
- 2
Apply isolation, encryption, redaction, retention, and least privilege.
- 3
Gate uploads and network serving with review and budgets.
- 4
Monitor, quarantine, restore, and validate cleanup.
Ejemplo para copiar
licensed model + hash -> isolated worker
prompt/output -> redacted evidence store
save/upload/chat -> explicit approval + scoped secret
incident -> revoke -> quarantine -> restore/deletePreguntas frecuentes
Can I upload every generated model to Hugging Face?
No. Verify license, provenance, evaluation, privacy, and publication approval first.
Should prompts be kept in logs?
Only when necessary and authorized; prefer hashed or redacted manifests with controlled raw storage.
Fuentes
- Heretic README (captured 2026-08-31)Fuente verificada 2026-08-31
- Heretic repositoryFuente verificada 2026-08-31