🎮 Matrix/Attention (Practice)

Companion to the Matrix Ops & Attention guide. Implement scaled dot-product attention — the heart of transformers. Edit and run the code right here — it executes in your browser.

🐍 Your turn

Loading editor…

🧠 Challenge yourself

1. Print the attention weight matrix and confirm rows sum to 1.
2. Why the 1/sqrt(d) scaling? (gradient stability)
3. Where does the O(n^2) cost come from, and how do sparse formats help?

---

Continue the learning path → Beam Search