This poster includes a walkthrough of DeepSeek’s multihead latent attention with detailed captions, and a comparison between various forms of attention, including the required sizes of kv caches, and the 3d model of each type of attention head.

The matrix images are actually from the real deepseek model - mostly the weights from the first layer of DeepSeek V3.

Download is a vector poster pdf.