https://sikoratom.com/blog Interactive write-ups on transformer internals and mechanistic interpretability. en https://sikoratom.com/blog/attention https://sikoratom.com/blog/attention Mon, 13 Jul 2026 12:00:00 GMT An interactive tour of attention beyond the basic heatmap: information-flow arcs, the causal mask, softmax/temperature, induction heads, attention sinks, a taxonomy of every head, the QK/OV split, and head ablation. https://sikoratom.com/blog/residual-stream https://sikoratom.com/blog/residual-stream Tue, 03 Jun 2025 12:00:00 GMT An interactive study of the residual stream, the shared working state a transformer maintains across its layers.