MiniMind 架构：Tokenizer、Transformer、训练阶段与服务接口

Six layers to keep distinct

MiniMind 可拆成 tokenizer 与模板、Dense/MoE 模型、数据清洗与 JSONL、Pretrain/SFT/LoRA/RL/Agentic RL、评测、checkpoint 和 API/WebUI 六层。每层有不同输入输出和实验假设，只有固定配置、数据、硬件与模板后，跨阶段比较才有意义。

Each layer has a different contract. A tokenizer change alters sequence statistics, a dataset change alters supervision, a LoRA run changes parameter ownership, and a serving wrapper changes the integration surface. Do not attribute a result to the model architecture until those variables are fixed.

Follow one token through the model

Start with a text sample and trace normalization, special markers such as `<think>` or tool-call templates, token IDs, embeddings, attention/MLP blocks, logits, and loss. For MoE variants, record routing configuration and effective active parameters rather than treating the checkpoint as a generic Dense model.

The training loop adds batching, masking, mixed precision, optimizer updates, logging, and checkpoint cadence. Capture where each field is created and consumed; this is more reliable than inferring internals from a high-level framework command.

Separate learning and serving graphs

Pretraining, SFT, DPO, PPO/GRPO/CISPO, tool use, agentic RL, and distillation reuse pieces but ask different data and evaluation questions. Checkpoint names and directories should make the stage explicit, and evaluation must load the matching tokenizer and prompt template.

Serving wraps a checkpoint with generation settings and an API or Streamlit UI. Keep the model directory read-only, isolate tool calls, and record `reasoning_content`, `tool_calls`, or `open_thinking` fields as optional protocol features rather than assumptions about model cognition.

Architecture review fixtures

Use a tiny JSONL fixture to test tokenizer coverage, a one-batch forward/loss pass, save/resume, one evaluation prompt, and one API request. Add a MoE or tool-call fixture only after the dense path is understood. Contributions should include expected tensor shapes, checkpoint fields, and a reproducible command.