PyTorch explained: tensors, automatic differentiation and your model
PyTorch quickstart: verify a scalar gradient before training
Use a tiny CPU calculation to distinguish installation, forward computation and backward behavior.
What you will learn
- Choose the correct package
- Check a known answer
- Observe accumulation deliberately
Before you start
- Basic Python and calculus
- An isolated environment for later exercises
A learning exercise compares analytical derivatives and framework outputs without hiding failed checks.
Key takeaways
- Installation and accelerator readiness differ.
- Known derivatives make debugging concrete.
- Clear gradients intentionally.
Choose the correct package
The repository README directs binary users to the official installation instructions. Select the build for your OS and accelerator rather than copying a CUDA command from an unrelated machine.
Use an isolated environment and record Python, torch and device information. Installing the library does not verify that your accelerator driver or a downloaded model is compatible.
Check a known answer
Set x to 3 with requires_grad enabled and compute x squared. The derivative is analytically 2x, so the expected gradient is 6. This simple case needs no external dataset or pretrained weights.
Use floating-point input for this exercise and start on CPU. If the result differs, inspect the executed code and environment before introducing batches, mixed precision or a GPU.
Observe accumulation deliberately
Recompute the forward expression before a second backward call and leave x.grad unchanged. The expected accumulated value is 12; setting x.grad to None before another fresh pass starts a new accumulation.
These are analytical expected values, not measured outputs from this session. The default graph is generally released after backward, so do not repeat backward on the same saved result as a shortcut.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Select an official build and record versions.
- 2
Check the scalar derivative on CPU.
- 3
Recompute forward to study accumulation.
Copy-ready example
import torch
x = torch.tensor(3.0, requires_grad=True)
(x * x).backward()
(x * x).backward()
print(x.grad) # Analytical expectation: 12
x.grad = None
(x * x).backward()
print(x.grad) # Analytical expectation: 6Frequently asked questions
Do I need a GPU for this exercise?
No. A scalar CPU example is sufficient.
Why recompute y?
The default backward path releases its graph; a fresh forward creates a new one.
Sources
- PyTorch / README.mdSource checked 2026-09-23
- PyTorch / torch/autograd/__init__.pySource checked 2026-09-23