MiniMind
MiniMind Deployment: From Checkpoint and GPU Training to an OpenAI-Compatible Service
A practical MiniMind deployment guide covering Python/PyTorch environments, single- and multi-GPU training, checkpoint management, API serving, and evaluation.
What you will learn
- Design a pinned MiniMind deployment
- Choose single- or multi-GPU topology
- Serve and recover a verified checkpoint
Before you start
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
Key takeaways
- Pin environment, data, seed, hardware, tokenizer, and checkpoint before scaling.
- Separate single-process smoke tests from DDP/DeepSpeed production jobs.
- Serve read-only verified checkpoints on localhost first and preserve rollback evidence.
Pin the experiment before scaling
A reproducible MiniMind deployment starts with a fixed repository revision, Python/PyTorch environment, dataset, tokenizer, seed, GPU, and checkpoint source. Keep inference, pretraining, SFT, post-training, and serving as separate jobs so a checkpoint's role is clear.
The README's headline of two hours and three yuan qualifies a specific one-epoch SFT measurement on one NVIDIA 3090. Treat it as a reference point, not a universal deployment budget for every dataset, stage, or accelerator.
Choose the training topology
Use one process for a smoke test, then move to DDP or DeepSpeed only when data, checkpoint, and evaluation paths are stable. Record batch size, sequence length, gradient accumulation, precision, worker count, and checkpoint cadence. A multi-GPU run with different effective batch or data order is a different experiment.
Keep data and checkpoints on explicit volumes with enough space for resume and rollback. Verify that interrupted jobs leave a complete checkpoint and that a resumed run records the exact step and optimizer state.
Serve a verified checkpoint
The repository documents a lightweight `scripts/serve_openai_api.py` service and a Streamlit `web_demo.py`. Bind the API to localhost during evaluation, expose a read-only model directory, and test a basic request before streaming, `reasoning_content`, or `tool_calls` extensions.
Record model revision, tokenizer, decoding settings, request ID, latency, token counts, and errors without persisting prompts unless policy allows it. An OpenAI-compatible schema is an integration convenience, not a guarantee of frontier quality or safe tool execution.
Operate and recover
Monitor GPU memory, disk, checkpoint age, request latency, error rates, and evaluation drift. Keep a known-good checkpoint and environment lockfile, rehearse rollback, and document whether a model is for education, research, or a constrained internal service. Validate licensing and data provenance before serving beyond a private lab.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Create a pinned Python/PyTorch environment and record the fixture.
- 2
Run one-process inference/training, then scale with explicit DDP/DeepSpeed settings.
- 3
Serve a verified checkpoint locally and test basic plus optional response fields.
- 4
Monitor resources, evaluate drift, and rehearse checkpoint/environment rollback.
Copy-ready example
python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
torchrun --nproc_per_node 1 train_full_sft.py
cd scripts && python serve_openai_api.pyFrequently asked questions
Can the README's two-hour figure size a production job?
No. It describes a specific SFT measurement on one 3090; measure your data, stage, hardware, and review workload.
Should the API bind to all interfaces by default?
Start on localhost with a read-only model directory, then add authentication and network controls before any wider exposure.
Sources
- MiniMind README (captured 2026-08-31)Source checked 2026-08-31
- MiniMind repositorySource checked 2026-08-31