Archives

How Close Is FlashAttention to the Limit? Understanding Attention Through Data Movement

Beyond FLOPs: How COSMA Builds Parallel Matrix Multiplication from Communication Bounds

The Red-Blue Pebble Game: Why Faster Processors Still Have to Move Data