Reading at a standstill
Everything so far read the order from motion. The decay-rate reading#4 watched the Fisher shrink as training approached the singular set, and the gauge arc#12 worked hard to keep that motion legible. Both efforts share two preconditions: a run that is actually descending, and coordinates that line up with the geometry. Drop both. A trained network arrives as a single frozen checkpoint, trained by whoever, with whatever optimizer, its dead structure parked in whatever basis the run happened to end in. The order is still there, and it can still be read.1
The trick is that motion was never essential, only displacement. The rate law says the directional Fisher grows as $u^\top F(\theta_0 + t\,u)\,u \propto t^{2(k-1)}$ as you move distance $t$ away from the singular point along the dead direction $u$. Training supplied that path for free; at a standstill you synthesise it, evaluating the Fisher at a handful of points along a straight line you choose. One forward-and-backward sweep per point, a log–log fit, and the slope hands back $k$. No descent, no learning, the checkpoint never changes.
That leaves two real problems: finding $u$, and trusting the fit. Each has a mechanism.
Construct, do not search
The natural first idea is an eigensolve: the dead direction should be the Fisher's smallest eigenvector, so take the bottom of the spectrum. On a real network the bottom is exactly the wrong place to look. It is crowded with directions that are flat for boring reasons: the within-layer reparametrisations the layered structure leaves flat (a full $(L-1)\,d^2$ of them on a depth-$L$, width-$d$ linear stack), the gauge#12 directions and normalisation kernels, and, when samples run short, a sampling null space besides. A high-dimensional floor of symmetry and noise. The order-carrying mode sits above that floor, and an eigensolve pointed at the bottom returns a gauge every time.
So the read does not search the spectrum; it constructs the direction. A layer's Fisher factorises into the two K-FAC factors#6, the forward $A$ and the backward $G$, and a genuine dead unit announces itself in both: its input direction starves in $A$ and its output direction starves in $G$. Pairing the two, the joint mode of the factors, pins the direction outright, and the duality#7 covers the layers where one factor is uninformative, reading whichever side separates the structure more cleanly. The candidate is built from the layer's own structure, then verified by the scan: if the constructed direction were wrong, the slope would not settle on a clean power law.
A window you can trust
The second problem is contamination. A recovered direction is never perfect; it is the true dead mode plus a small admixture $\varepsilon$ of something livelier. Along the synthetic path the mixture behaves like two power laws in competition, and below a crossover scale, roughly $t \lesssim \varepsilon^{2/\Delta\alpha}$ where $\Delta\alpha$ is the gap between the two exponents, the contaminant's contribution swamps the signal. Fit the whole range and you fit garbage.
The cure is the purity-matched window: sweep the fit window and keep the one where a single power law holds cleanest, at the highest $r^2$ above a hard admissibility bar. The selector optimises the fit alone; it never sees the architecture's predicted order, so the prediction stays an independent check on what comes out. And the read is built to refuse: a scan whose slope will not settle, or that never clears the contamination, comes back as a rejection with the diagnosis attached rather than a number. It never mints a wrong exponent.
Off the axes, where it counts
Here is the read earning its keep on a real network. Train two grokking transformers identically except for the activation. The squared-ReLU model happens to leave its dead subspace on the coordinate axes, so even a naive scan along each axis reads its order, $k = 3$, correctly. The gelu model's optimizer leaves the same kind of structure rotated off the axes. The axis-by-axis scan now cuts across the dead subspace at an angle and comes back with a slope of $1.29$, an exponent that corresponds to no order at all. The constructed joint mode follows the structure to where it actually lies and reads $k = 2.0$ at $r^2 = 1.000$: the activation's predicted order, recovered exactly where the rotation had hidden it.1
Plant an order, then rotate it. At zero rotation both reads agree. Turn the structure off the axes and the per-coordinate scan drifts to a deviant slope while the constructed mode holds the true order; add contamination and watch the trustworthy window shrink from below until the read starts refusing. The three mechanisms, construction, the window, and the refusal, are all visible in one plot.
What this changes
The reading no longer belongs only to carefully-instrumented runs. Any checkpoint, trained under any optimizer, is now a legible object: the equivariant optimizer#17 makes the structure cleaner and the trajectory readable in real time, but a run trained without it is not lost, and even the drifted, rotated solutions Adam leaves behind can be read after the fact. What the frozen read returns per direction, the order, the type, the count, is the raw material that adding up the orders#23 assembles into the model's global complexity.
Still open / underspecified. The read sees what its detectors surface: the K-FAC joint mode, the convolutional channel factor, and the algebraic LayerNorm kernel cover the structure this program has catalogued, and the coverage residual, dead structure none of the detector family exposes, is unmeasured by construction. Two taxonomy rows (attention-head death, expert death in mixtures of experts) have detectors in hand but no verified read yet. And where an optimizer leaves the structure too diffuse to carry any order at all, there is genuinely nothing to recover, a fact about the solution rather than the instrument.
- The descent-free alignment-free read, the dual-factor construction, the purity-matched window, and the rotated-grokking demonstration are from Tejas Pradeep Shirodkar, Measuring Dead Directions: Decomposing and Classifying Singular Structure off Canonical Alignment, arXiv:2607.00603 (2026). ↩︎