All the demos
Every explainer here is built show-first, and this index collects the showings. Each entry deep-links to the demo where it lives, in reading order. They are all client-side and self-contained: sliders recompute the mathematics live, and the readouts check the measured value against the predicted one wherever a closed form exists.
What is a neuron
- Perceptron Device
- Pixel Deviceunder “A neuron over an image”
- Network Viewunder “Stacking neurons into a network”
- Loss Landscapeunder “From knobs to a landscape”
- Solution Setunder “Many weights, one output”
Why neural networks are singular
What is a dead direction
- Relu Rescale
- Attention Rotationunder “The same symmetry, all over the network”
- Unit Duplicationunder “The same symmetry, all over the network”
- Singular Setunder “The singular set”
Reading the order of a dead direction
- Fisher Decayunder “Two views of the same number”
- Multiplicityunder “Many dead directions at once”
Why singular models generalize
- Occam Volumeunder “The price of a model”
- Singular Fluctuationunder “The fluctuation that measures overfitting”
Reading the order one layer at a time
- Kfac Factorunder “The Fisher of a layer factorises”
- Kfac Dualityunder “Forward death and backward death”
- Kfac Layersunder “The same reading, layer by layer”
Reading a dead direction from both sides
- Both Sidesunder “Two reads, one from each side”
- Both Sides Backwardunder “Two reads, one from each side”
Counting dead directions
- Counting Volumeunder “Volume counts them”
- Volume Slopesunder “What the count is, exactly”
A floor that survives depth
- Sigma Min Depthunder “The identity path holds the floor up”
Reading a dead direction off the weights
- Ln Kernelunder “What centring does to a direction”
- Ln Kernel Galleryunder “RMSNorm has no such direction”
The cost of reading a dead direction
- Cost Ladderunder “The ladder, rung by rung”
Why the dead direction depends on the optimizer
- Rotation Gaugeunder “Two kinds of gauge: scaling and rotation”
- Gauge Orbitunder “The function lives in the quotient; a measurement feels the gauge”
The symmetries are groups
- Group Orderunder “When the order of two moves matters”
Along the orbit and across it
How an optimizer measures distance
Invariant loss, equivariant optimizer
An optimizer that respects the gauge
A sharper minimum, and a readable rate
- Rate Readabilityunder “A rate you can read”
Why grokking is slow then sudden
- Grokking Plateauunder “The plateau and the door”
- Grokking Seedsunder “The seeds tell it”
Nothing to read yet
- Formation Phasesunder “Two phases”
Reading at a standstill
- Frozen Scanunder “Off the axes, where it counts”
Every way a direction dies
- Dead Zoounder “The kind that does not”
Adding up the orders
- Assemble Lambdaunder “Sum versus crossing”
- Nu Universalityunder “The last constant: fixed by the order, absorbed by the network”
Geometric singular learning
- Fisher Decayunder “The signature demo”
Dead-direction signatures
- Volume Slopesunder “The signature demo”
Algebraic dead directions
- Ln Kernel Galleryunder “The signature demo”
Dead-direction conditioners
- Grokking Seedsunder “The signature demo”
Measuring dead directions
- Frozen Scanunder “The signature demo”