Desplegar Heretic: GPU, cuantización y procedencia del modelo

Treat deployment as an experiment runner

Heretic es un runner de experimentos, no un servicio multi-tenant; fija Python/PyTorch/uv, revisión del modelo, volumen de salida y pruebas de recuperación.

Python 3.10+ and PyTorch 2.2+ are the baseline. The README warns that some models need later PyTorch features, such as `torch.accelerator` for MXFP4 gpt-oss models; pin the exact stack instead of assuming the minimum is sufficient.

Capacity and artifacts

The program benchmarks the machine to choose batch size. On the documented RTX 3090 example, Qwen3-4B takes roughly 20–30 minutes, but treat that as a workload-specific reference. Bitsandbytes `bnb_4bit` can reduce VRAM while adding a quantization variable to the evidence.

Persist the source model revision, lockfile, TOML configuration, hardware inventory, logs, evaluation outputs, checkpoint hash, and license. Keep upload credentials outside the worker and make export an explicit, reviewed step.

Recovery

Use a disposable output directory, checkpoint space and a known-good baseline. Test interrupted runs, insufficient VRAM, corrupted downloads, failed uploads, and cleanup before scheduling batches.