systems
OpenAI Rebuilt Voice So the Model Itself Decides When to Talk News
OpenAI's engineering posts on GPT-Live describe removing the separate turn detector from the audio path entirely and cutting session startup from six network round trips to one, treating a voice conversation as a live media system rather than a model feature.
Offloading and streaming: running a model bigger than your memory Lesson
Offloading and streaming let a machine run a model far larger than its memory by keeping only the parts needed right now in fast memory and fetching the rest from system RAM or disk on demand, trading speed for capacity.
Kimi K3 runs in 8 gigabytes of RAM, at 33 seconds per token News
A hand-written C engine generates text with Moonshot's 2.8-trillion-parameter Kimi K3 using a peak of 8.24 gigabytes of RAM and no GPU, by reading the model's four-bit experts directly off disk, at a rate of roughly one token every 33 seconds.
LongStraw Makes Million-Token RL Training Possible on 8 GPUs - But Its Code Doesn't Run Yet News
A new systems paper called LongStraw shows reinforcement-learning post-training can execute on prompts beyond 2 million tokens on a fixed 8-GPU budget by scoring the shared prompt once without gradients and backpropagating only through the short generated response, though the authors call this proof of execution capacity rather than full training correctness, and the public code is not yet runnable.
FlashAttention: making attention fast by respecting the memory hierarchy Lesson
FlashAttention is an exact attention algorithm that runs much faster and uses far less memory by tiling the computation to keep it in the GPU's fast on-chip memory - never writing the giant attention matrix to slow memory - which is why it became the default attention kernel in modern transformers.
An AI wrote a working operating-system kernel from scratch in 38 minutes News
A blow-by-blow log shows one of the now-suspended models building bootable low-level systems code from an empty folder -- the kind of feat that made regulators nervous.