Claims, evidence, fractures, and certification
Mechanistic interpretability should be audited as a measurement science. Its methods may recover information, alter behavior, expose causal participation, support control, or improve monitoring. These are real achievements. They are not interchangeable achievements.
Mechanistic interpretability studies internal model states through probes, sparse decompositions, feature dictionaries, Jacobians, steering directions, circuit analysis, causal interventions, chain-of-thought inspection, activation patching, and runtime monitors. The audit begins after a method appears to work.
A successful decoder may show that a target is recoverable. A steering vector may show that a direction influences behavior. A circuit may mediate part of a task. A monitor may reduce operational risk. None of these conclusions automatically establishes a native semantic variable, a complete mechanism, or a safety guarantee.
The audit therefore examines five distinct objects: the model computation, the interpretability method, the recovered object, the stated claim, and the proposed application. Errors arise when evidence about one object is promoted into a claim about another.
This audit takes no position on the global geometry of model state. It assumes no theory of representation, requires no commitment to any interpretability program, and endorses no framework of its own. Its standards are empirical and its verdicts are about evidence.
The audit is therefore usable by anyone, including researchers who disagree with every structural claim made elsewhere on this site. That is intentional. An instrument that only certifies its owner's results is not an instrument.
| Class | Name | What it establishes | What remains unearned |
|---|---|---|---|
| O | Observation | An internal quantity covaries with a condition | Representation or cause |
| D | Decoding | A target is recoverable under a stated measurement procedure | That the model uses the decoded variable |
| I | Intervention | A modification changes a measured outcome | Semantic identity or clean control |
| C | Causal attribution | A component or state participates causally in a bounded computation | Completeness or uniqueness |
| M | Mechanism recovery | A structured causal process predicts computation over a declared domain | Validity outside that domain |
| U | Operational utility | A method improves monitoring, debugging, control, or auditing | Mechanistic truth |
Class U deserves particular care. Utility is the most defensible achievement in the field and the most frequently exchanged for a claim it did not buy. A method that reliably reduces operational risk has earned deployment. It has not thereby earned an ontology.
The ladder is not a schedule. It carries no presumption that a result at one rung will eventually reach the next, and no presumption that failure to climb indicates a temporary shortage of effort. Some results stop where they stop because that is where the evidence stops. Others stop because the higher rung does not exist for that object.
A representation need not be a neuron, direction, sparse feature, or fixed coordinate. It may be a subspace, relation, trajectory, transition rule, distributed configuration, context-indexed state, or equivalence class. Stability applies to the role or transformation law, not necessarily to one coordinate.
A representation claim should establish identifiability, observer robustness, domain stability, temporal availability, model availability, and causal relevance. A signal that can be decoded only by one instrument, one labeling scheme, or one narrow prompt set is better described as an observer-conditioned measurement than as a stable internal object.
Interpretability never begins without a target vocabulary, decoder class, dataset, decomposition, labeling procedure, and evaluation regime. The recovered object is jointly conditioned by model state, measurement method, observer ontology, and data distribution.
This does not make interpretation arbitrary. It changes the burden of proof. Alternative labels, bases, decoder families, seeds, widths, penalties, and decompositions become competing measurements. Structure that persists across admissible measurements has a stronger claim to model-native status.
The audit also asks what the instrument could not have found. A vocabulary that contains no term for a phenomenon will not report its absence. The negative space of a method is part of its result and should be stated, not left to the reader.
A signal may be recoverable before a decision, during its construction, after it as a trace, or later as a reportable summary. A binary before-or-after test is insufficient for distributed computation.
| Question | Audit target |
|---|---|
| First recoverability | When does the information become decodable? |
| First causal efficacy | When does intervention first change the result? |
| Peak mediation | Where does the proposed state carry the largest causal effect? |
| Persistence | Does the state remain stable, rotate, split, or disappear? |
| Revision | Is the state updated or overwritten by later computation? |
| Re-expression | Does a later signal reconstruct an earlier computation mainly for report? |
Binary necessity and sufficiency are too coarse for distributed models. A valid mechanism may be contributory but not individually necessary, conditionally sufficient but not globally sufficient, or one member of a substitutable family.
| Role | Meaning | Required evidence |
|---|---|---|
| Contributory | Changing the object alters the computation | Controlled intervention with matched damage |
| Conditionally necessary | The object is required under stated background conditions | Ablation across controlled contexts |
| Conditionally sufficient | The object recovers the target under stated background conditions | Insertion or restoration with controls |
| Redundant | Alternative paths can substitute for the object | Joint ablation and pathway substitution |
| Mediating | The proposed path carries a measurable portion of the effect | Mediation analysis and path intervention |
Behavioral restoration shows that an intervention recovers the measured output. Process restoration additionally recovers intermediate dependencies, timing, sensitivity, and error structure. A restored answer may be produced by an artificial shortcut. Strong mechanism claims require evidence that the repaired model recovers the relevant computational trajectory rather than only the final answer.
Restoration evidence should state which of the two was achieved. A report that says only that performance returned has reported behavior and should be classified as such, however the repair was motivated.
Density alone does not determine whether an intervention is valid. A rare state may be naturally reachable, while a high-density state may violate learned relations among layers, positions, or residual components. State-validity evidence may include reachability from actual inputs, preservation of learned conditional relations, agreement with naturally occurring states, downstream calibration, reversibility, and absence of unrelated competence collapse.
Selectivity does not require zero downstream change. It requires that the intended causal family changes disproportionately relative to unrelated damage and matched interventions.
Every substantive claim should declare the transformations under which it is expected to remain valid. Fracture tests should be derived from those implied invariances. A claim with no declared invariance is not conservative. It is untestable.
| Claim | Expected invariance | Failure meaning |
|---|---|---|
| Token-specific detector | May vary under retokenization | Usually a domain boundary |
| Concept representation | Should survive meaning-preserving paraphrase and hard negatives | Construct failure or overnarrow domain |
| Language-independent concept | Should survive translation, tokenizer, and lexical variation | Claim falsification |
| Stable semantic direction | Should survive admissible basis, context, layer, and checkpoint changes | Coordinate dependence or locality |
| Global mechanism | Should survive relevant context, position, model-state, and distribution changes | Global claim falsification |
| Safety monitor | Should survive the declared adaptive threat envelope | Deployment claim failure |
Fractures occur when nearby, equivalent, or claim-related states produce abrupt failures of interpretation, recovery, intervention, or correspondence. The register below lists the standard pressures. It is an inventory of tests, not a theory of the model. Each pressure is neutral with respect to what the failure means; the meaning is fixed by the invariance the claim declared.
| Pressure | Applied by | Invariance typically implied |
|---|---|---|
| Tokenization | Retokenization, spelling, whitespace, segmentation | Concept identity independent of surface segmentation |
| Formatting | Markup, casing, delimiters, presentation protocol | Identity independent of presentation |
| Syntax | Meaning-preserving rewrite, voice, clause order | Semantic rather than lexical identity |
| Context length | Extension, truncation, distant dependency | Stability of the state under accumulation |
| Layer and position | Cross-layer and cross-position transfer | A static coordinate rather than a moving one |
| Normalization | Rescaling, gain, residual composition | Metric relations preserved under rescaling |
| Projection | Alternative decoder, dictionary, basis, seed, width | Object exists independent of the instrument |
| Saturation | Magnitude sweep, dose-response, sign reversal | Smooth response over the operating range |
| Finite precision | Quantization, reduced precision, rounding | Structure survives discretization |
| Adversarial pressure | Optimized inputs, adaptive attack | Local continuity of the claimed relation |
| Model version | Fine-tune, checkpoint, retrain | Durable identity of the coordinate |
| Composition | Concept combination, intervention ordering | Independence and commutation of the parts |
A failure outside the declared domain is a boundary. A failure inside the domain is a failed replication. A failure against an implied invariance is claim falsification or demotion. A failure caused by an invalid intervention is inconclusive. A failure revealing an alternative path reduces completeness. A failure appearing only under one instrument is an observer-dependence warning.
Three reporting faults recur and should be recorded by name whenever they are found.
| Fault | Description |
|---|---|
| Unauthorized promotion | A claim is stated at a rung above the one its evidence reached |
| Unreported exclusion | A phenomenon the instrument could not have found is not listed as unfound |
| Unlicensed extension | A local result is presented as holding over a domain that was never tested |
None of the three requires bad faith and none is rare. Each is a defect of reporting, correctable by reporting.
Absolute completeness is rarely measurable because the full denominator is not known. Completeness should therefore be reported relative to explicit targets: fraction of behavior recovered, task loss recovered, causal effect mediated, state variation explained, prompt classes covered, or known failure modes predicted.
The residual is not noise by default. It may contain redundant traces, downstream records, alternative mechanisms, missing features, or safety-relevant capacity. Residual information must be audited through the same ladder: decoding, temporal precedence, intervention, mediation, and restoration.
Different bases, decompositions, circuits, or causal graphs may explain the same observed behavior. This may reflect inadequate experiments, true model symmetry, or multiple valid descriptive levels. When explanations are observationally and causally equivalent, the audit should report the equivalence class and identify the invariant structure shared across its members.
Disagreement about feature counts is not by itself scientific disagreement. Two decompositions that preserve the same causal transitions may be one result described twice.
A successful linear probe establishes linear recoverability on a defined activation distribution. It should report held-out and shifted-distribution performance, calibration, nuisance controls, matched-capacity baselines, cross-layer and cross-position transfer, intervention effects, and stability after fine-tuning.
For activation edits, test dose-response, sign reversal, matched random directions, unrelated-task degradation, state validity, layer and position dependence, reversibility, restoration, and vector composition.
Report reconstruction error, loss recovered, downstream behavior recovered, sparsity distribution, dead and rare features, dictionary-width sensitivity, cross-run feature matching, causal quality, missing known concepts, and information retained in the residual. A concept that was expected and not found should be reported as not found, with the same prominence as a concept that was found.
A circuit claim requires an explicit task, causal paths, conditional necessity or sufficiency, redundancy analysis, restoration, bounded completeness, alternative-circuit comparison, and transfer beyond discovery prompts.
Chain of thought may be useful without being a faithful transcript of internal computation. Audit retrospective faithfulness separately from prospective causal use through truncation, substitution, false reasoning, irrelevant insertions, answer preservation, hidden-state evidence, and monitor-aware training.
A safety claim must state its threat model, attacker knowledge, monitor visibility, optimization budget, false-positive and false-negative costs, distribution shift, adaptive pressure, fallback defenses, and containment after failure. Robustness is always relative to an explicit threat envelope.
Benchmark detector, held-out detector, adaptive detector, monitor-aware detector, deployment defense, and defense system are different claims. Survival at one rung does not certify the next.
Every arrow requires additional evidence. The audit should compare the paper, abstract, laboratory post, podcast, press coverage, policy briefing, and product claim. Caveats retained in the paper do not automatically survive public translation. A caveat that appears only in the supplementary material has been preserved, filed, and removed from the claim.
The audit records the exact point of unauthorized promotion, and by whom it was performed. The paper, the abstract, and the press release may each have a different answer. A technically faithful satire may afterwards dramatize that promotion. The technical judgment is established first; the satire may only repeat it.
These standards are themselves subject to the ladder. They can be misused in three ways, and the misuses should be named as plainly as the ones they detect.
Result: ______________________________
Classification: Observation / Decoder / Intervention / Monitor / Local causal model / Partial mechanism / Mechanistic account
Target construct: ______________________________
Competing construct excluded: ______________________________
Declared domain: ______________________________
Claim-invariance contract: ______________________________
Temporal role: ______________________________
Causal role: Contributory / Conditionally necessary / Conditionally sufficient / Redundant / Mediating
State-validity evidence: ______________________________
Restoration: None / Behavioral / Process
Completeness denominator: ______________________________
Recovered fraction: ______________________________
Pressures applied: ______________________________
Fractures found: ______________________________
Expected and not found: ______________________________
Unexplained residual: ______________________________
Threat envelope: ______________________________
Strongest justified claim: ______________________________
Typical unauthorized promotion: ______________________________
Explicitly unearned claims: ______________________________
Mechanistic interpretability should be audited with the same seriousness applied to the models it examines. Its tools can expose structure, recover information, alter behavior, diagnose failures, support control, and provide inexpensive monitoring. The objective is not to demand total understanding before partial methods may be used. It is to state exactly what each method has earned.