Archives

How Close Is FlashAttention to the Limit? Understanding Attention Through Data Movement

Beyond FLOPs: How COSMA Builds Parallel Matrix Multiplication from Communication Bounds

The Red-Blue Pebble Game: Why Faster Processors Still Have to Move Data

Git Needs a Trash Can

RoPE: Properties, Patterns, and Long-Context Behavior

CS336 Assignment 1: Large Language Model Training and Inference

CS231n Lecture Note: Generative Models

CS231n Lecture Note: Self-Supervised Learning

CS231n Lecture Note: Large Scale Distributed Training

自動微分 | DIY 實現自己的 PyTorch