Subtext

What is Subtext?

Subtext is a free, open-source LLM interpretability tool that turns a language model's hidden reasoning into something you can watch as it happens. During a live chat with a local model, it reads the Jacobian lens at nine layer depths on every token — while the model reads your message and while it writes its reply — and renders those internal activations on an interactive canvas.

The lens, a method from Anthropic's interpretability research, transports a residual-stream activation into the model's final-layer basis and decodes it through the model's own unembedding, revealing which vocabulary words an internal state is disposed to produce — now or several tokens later.

Key features

• Real-time reasoning visualization: see verdicts form during reading, before any output token is generated.
• Nine-depth lens readouts: vertical position maps to layer, so you can watch a concept climb the stack toward emission.
• Per-token ledger and word inspector: hover for a word's per-layer activation profile, or click to open peak strength, mean depth, and strength history.
• Scrubbable playback: pause, step, and seek to any token in a response; the canvas and rankings reconstruct that exact moment.
• Trace view: plot one word's readout strength across layers and tokens to see structure the instantaneous view can't show.
• Export and replay: save a full session as JSON and replay it in any browser — no GPU required.

Why it's useful

The value of Subtext is the gap between a model's internal state and its visible text. It surfaces judgments that are internally settled before they're spoken, plans the model holds while emitting unrelated tokens, and the unspoken middle term of two-hop questions. It reproduces reporting and planning phenomena from Anthropic's paper on an open 4B model running on consumer hardware.

How it works

A single HTML front end connects over WebSocket to a Python server running Qwen3.5-4B with a pre-fitted Neuronpedia Jacobian lens. Each exchange has a reading phase (a prefill pass over your message) and a thinking phase (token-by-token generation with a KV cache), with the lens read at the newest position each step. Because the lens adds only a per-layer matrix-vector product and an unembedding per token, streaming runs at the model's native generation speed.

Who it's for

Subtext is built for interpretability researchers, ML engineers, and developers who want a concrete, watchable window into how transformer models reason. Running it live needs an NVIDIA GPU (~10 GB VRAM) or an Apple Silicon Mac with 16 GB+ unified memory and Python 3.11+; a CPU fallback works for smoke tests, and exported replays run in any browser with no GPU at all.

Pricing

Subtext is free and open-source. Clone the repository, install the requirements, and run the server to start watching your model think.