gpu
FlashAttention: making attention fast by respecting the memory hierarchy Lesson
FlashAttention is an exact attention algorithm that runs much faster and uses far less memory by tiling the computation to keep it in the GPU's fast on-chip memory - never writing the giant attention matrix to slow memory - which is why it became the default attention kernel in modern transformers.
Nvidia is now backstopping the sales of its own chips News
Nvidia formalized a financing program that guarantees a revenue floor for cloud firms buying its GPUs in exchange for a cut of their sales - a move analysts say could help drive AI debt past $7 trillion by 2029.
FpSan (Floating-Point Sanitizer) Tool
Open-source correctness checker for Triton GPU kernels, and the tool OpenAI says it used to validate the production kernels GPT-5.6 Sol rewrote. It compares symbolic computation under its own payload algebra rather than simulating IEEE floating point, so results should be compared only against other FpSan runs. Useful for anyone writing or generating custom kernels who needs to catch numerical breakage before it reaches production.