architectures
Everything on Ground Truth tagged “architectures” — 2 items.
Linear Attention Lesson
Linear attention rewrites the transformer's attention math so its cost grows in a straight line with sequence length instead of exploding as the square, letting models handle very long inputs cheaply, at the price of a compressed memory of the past.
RNNs and LSTMs: How Neural Networks Learned to Remember, Before Transformers Lesson
Recurrent neural networks process sequences one step at a time while carrying a memory, and LSTMs added gates to fix their forgetfulness; together they powered a decade of sequence modeling and set up exactly the problem transformers later solved.