A substrate-first framework for understanding why some interpretability methods work locally, why they fail globally, and why semantic manifolds are not required.
The preceding document, Twelve Structural Falsifications of the Manifold Hypothesis, establishes that strong semantic manifold assumptions are structurally incompatible with transformer inference.
A natural question follows:
If smooth semantic manifolds are false, why do linear probes, sparse autoencoders, steering vectors, and related methods work at all?
This document provides a direct answer.
Context is not something a model “adds,” “infers,” or “moves through.”
Context is the total boundary condition under which computation is evaluated.
Context includes:
Semantic manifold narratives implicitly exclude these features.
Transformer computation does not.
Strong semantic manifolds require:
As shown in twelve.html, transformer inference violates these requirements structurally and operationally.
Context can be fractured, aliased, saturated, or undefined.
Semantic manifolds cannot carry that burden globally.
Transformer computation is best understood as algebra operating over a context substrate.
More precisely:
The model supplies structured substrate; the observer supplies the rendering relation.
Algebra does not require smooth space.
It requires only that certain operations commute, compose, or remain invariant within a restricted regime.
Linear probes, sparse autoencoders, and steering methods succeed when:
They work when the sampled regime behaves like a temporary algebraic island.
A probe does not certify a concept.
It discovers a hyperplane that remains approximately invariant under a restricted family of contexts.
That is algebra, not semantics.
Context Integrity is the study of which discontinuities are:
Rather than denying fractures, Context Integrity treats them as first-class objects to be mapped, tested, and monitored.
Algebraic Interpretability identifies the regularities; Context Integrity tests where those regularities survive, fracture, or fail.
Stress-prompt effects, for example, are not semantic jumps.
They are context substitutions that move computation into a different algebraic regime.
Algebraic Interpretability does not claim:
It claims only that:
This framework is not a replacement for existing interpretability tools.
It is a correction to their explanatory stories.
Only their explanatory labels need to be rebadged.
The falsification groundwork is laid in twelve.html.
This document explains what survives.