Transformers, from the inside · Chapter 3

The MLP

By Tomáš Sikora · July 3, 2026

An interactive tour of the other two thirds of a transformer: the SwiGLU block, RMSNorm and pre- vs post-norm, MLPs as key-value memories, where facts live, how small arithmetic is computed, and why single neurons stop making sense.

The MLP block holds two thirds of the parameters yet gets a fraction of the attention. This chapter opens it up: normalization and its consequences for interpretability, neurons read as key-value memories, causal tracing of factual recall, arithmetic probes, layer ablation, and the polysemanticity that breaks single-neuron analysis. All quantities are precomputed from forward passes of Qwen3-1.7B.

The other two thirds

The SwiGLU block — gate, up and down projections and the SiLU nonlinearity.

Normalization, the part everyone skips

RMSNorm, pre- vs post-norm, and what normalization means for interpretability.

MLPs as key–value memories

Neurons as pattern → update pairs: what each key detects and each value writes.

Where facts live

Severed causal tracing and the three-step recall of factual associations.

How the model computes 8 − 6

Per-depth answer probes: where small arithmetic crystallises, MLP vs attention.

Ablation, again

Knocking out MLP layers to establish which are load-bearing.

Neurons: mostly not what you would hope

A census of neuron roles — a few clean detectors in a sea of polysemanticity.

The wrong basis

Why the neuron basis fails, and superposition and transcoders as the way out.

This is an interactive article — the figures are explorable and require JavaScript to load.

More in this series: The Residual Stream · Attention · all posts