MiniMind 源码分析：从数据加载到 Loss、Checkpoint 与推理

Begin with a tiny JSONL fixture

源码阅读从一个小 JSONL 夹具开始：跟踪 tokenizer/template、batch、attention/forward、loss、梯度、optimizer、checkpoint 保存恢复，再连接 Streamlit 或 serve_openai_api.py。文章强调用固定 seed、数据和硬件做可复现对照，并把高层框架兼容层与核心原生实现分开。

The README presents MiniMind as a from-scratch teaching codebase, so prefer its native PyTorch path for source study before comparing wrapper integrations. Keep the fixture synthetic or openly licensed and record its hash.

Trace forward, loss, and optimizer

Follow batch collation into embeddings, attention, MLP or expert routing, logits, labels, masking, and loss. Record tensor shapes, dtype, device, and whether gradients are enabled. Then trace optimizer construction, scheduler, gradient accumulation, clipping, logging, and checkpoint cadence.

A one-batch forward/loss test catches tokenizer, dimension, mask, and device errors earlier than a multi-hour run. For MoE or long-context experiments, add a separate fixture so routing and sequence-length assumptions are visible.

Inspect save/resume and inference seams

Locate the checkpoint fields for model, optimizer, scheduler, step, and configuration, and test an interrupted run followed by resume. Compare generated text only with fixed decoding settings; also compare structural fields such as tool calls or reasoning markers when the script exposes them.

Trace `serve_openai_api.py` and `web_demo.py` from request parsing to tokenizer, generation, streaming, and error response. Keep serving code read-only and separate from training writes so an API request cannot mutate a checkpoint.

Turn observations into contributions

Good first contributions include a tokenizer/template regression, a deterministic save/resume fixture, a clearer tensor-shape assertion, an evaluation configuration lock, or a redacted API error test. State whether the behavior is observed in source, documented in the README, or still a hypothesis, and measure any optimization with the same fixture.