DISTILLATIONS Dec 3, 2025 9 min read Exploring Linear Attention Variants Can we have capable limitless-context foundation models? Read Article
DISTILLATIONS Nov 25, 2025 7 min read Why Small Models Get Stuck: Exploring the Softmax Bottleneck Learn about the "Stupidity Wall" and how to parameterize your small models efficiently. Read Article
