Harmful Demo Count Flips Refusal Before SAE Features Disappear
Packing many harmful demos into long context lifts attack success. Refusal features may stay SAE-readable even when the model complies.
AI interpretability · Jacobian Lens · alignment
Research explainers on J-Space and the internal workspaces where modern LLMs stage reportable reasoning.
How safety teams use interpretability to catch behaviors that tests miss — and make model behavior operational.
Read featured explainer →Deep dives on Anthropic’s interpretability work, internal workspaces in LLMs, and what J-Space reveals about model behavior.
Packing many harmful demos into long context lifts attack success. Refusal features may stay SAE-readable even when the model complies.
Sampled tokens can look aligned while residual Jacobians have already reoriented. Run JVP estimators and a five-stage protocol for NIST and EU GPAI logs.
Global workspace theory ignites then broadcasts under capacity limits. Scaled dot-product attention only routes over token positions.
Residual streams analogize capacity-limited broadcast better than attention maps. Softmax routing stays graded and content-addressable, not ignited.
GWT treats conscious access as competition for a limited broadcast. Transformers use graded QKV routing on a residual stream, not ignition.
QKV attention routes tokens rather than igniting broadcast. Decoder-only transformers fail GWT; the 2017 paper dropped recurrence. Use indicator tests.
J-Space tracks the sparse internal workspace where language models stage reportable reasoning. The Jacobian Lens makes that workspace measurable — so safety, alignment, and interpretability research can move from speculation to evidence.