Transformers & Perception

Self-attention and ViT derived from first principles — then pushed into the physical world. Attention, positional encodings, Swin, DETR, DINO, MAE, and transformers for 3D vision and video.

Code: transformers_for_perception

Self-Attention From First Principles

A geometric route from similarity to self-attention

Any digital image is just a matrix of numbers. Consider a satellite image of a region. Pick any pixel- say, position \((100, 100)\). Its value is \(142\). What does it mean?…

Madhav Prashanth Ramachandran

Self-Attention From First Principles — Part 2

Scaled dot-product attention, multi-head attention

Part 1 ended with self-attention almost fully derived — queries, keys, a similarity score, and a weighted average. Almost, because we left two things unfinished.

Madhav Prashanth Ramachandran