Learn · Intermediate

Federated learning

Federated learning trains one shared model across many devices or organizations without ever collecting their data centrally. Instead of copying everyone's photos, messages, or patient records into a data center, the current model is sent out to each participant, trained locally on data that never moves, and only the resulting updates come back to be averaged. Google introduced the approach in 2016, and the phone in your pocket has almost certainly participated in it.

The motivating problem is simple to state. The most valuable training data for many applications is the data you are least able to gather: what people type, what a hospital knows about its patients, what a bank sees in its transactions. Regulation, contracts, and basic decency all say that data should not be pooled. The conventional pipeline requires pooling. Federated learning is the workaround.

The round

The mechanism runs in rounds, and the core algorithm, federated averaging, was described by Brendan McMahan and colleagues at Google in Communication-Efficient Learning of Deep Networks from Decentralized Data.

A central server holds the current model and selects a subset of available clients, typically devices that are idle, charging, and on unmetered wifi. Each selected client downloads the model, trains it for a few passes over its own local data, and sends back only the change in weights. The server averages those updates, weighted by how much data each client had, applies the result to the shared model, and starts the next round. Repeat for thousands of rounds.

The analogy that fits is a recipe circulated among a hundred home cooks. You do not ask them to ship you their groceries. You send everyone the recipe, they each adjust it based on their own kitchen and ingredients, they send back a note describing their adjustments, and you fold all the notes into a new version of the recipe. Nobody's groceries left the house, and the recipe still got better.

Why it is harder than ordinary distributed training

It would be easy to mistake this for distributed training, where a job is split across many GPUs. The differences are what make federated learning a distinct field.

The data is not identically distributed. A GPU cluster shuffles one dataset across workers, so each worker sees a representative sample. In federated learning, one person's phone contains one person's writing. Client updates therefore pull in conflicting directions, and naive averaging can drift or stall in ways that never happen in a data center. This is the central technical difficulty, usually written as the non-IID problem.

Communication is the bottleneck, not compute. Sending a full set of model weights over a mobile connection is expensive, so a large body of work exists purely on compressing, quantizing, and sketching updates before transmission.

Clients are unreliable and enormously numerous. Devices drop out mid-round, have wildly different amounts of data, and cannot be scheduled. The system has to tolerate clients vanishing without corrupting the round.

Privacy is not automatic

The most important thing to understand about federated learning is that keeping data on the device is not the same as keeping it private. A model update is a function of the data that produced it, and functions leak. The paper Deep Leakage from Gradients showed that under some conditions an attacker holding the gradients can reconstruct the actual training examples, pixel for pixel.

Two defenses are layered on top in practice. Secure aggregation uses cryptography so the server can compute the sum of all client updates without seeing any individual one, which means no single participant's contribution is ever visible. Differential privacy adds calibrated noise and clips update sizes so that no single client's data can measurably change the final model, giving a formal, quantifiable guarantee rather than an intuition.

There is also a security question running the other way. If anyone can join a training round, anyone can send a poisoned update designed to install a backdoor in the shared model. That threat is the federated version of data poisoning and backdoor attacks, and defending against it means checking incoming updates for anomalies, which sits in direct tension with secure aggregation's goal of making individual updates invisible.

Where it actually gets used

Mobile keyboards were the first large deployment, learning next-word prediction from what people type without that text leaving the phone. Healthcare is the other natural fit, where several hospitals can jointly train a diagnostic model that none of them has enough data to train alone, and none of them is permitted to share. Financial fraud detection across institutions follows the same shape.

For large language models the picture is different. Full pretraining is not federated, because the compute and communication costs are prohibitive. The active area is federated fine-tuning, where only a small adapter is trained and exchanged, which makes the update small enough to ship over a consumer connection. See fine-tuning and LoRA for why that is now practical, and encrypted inference for the related problem of using a model privately rather than training one.

Ground Truth. (2026, August 16). Federated learning. Ground Truth. https://groundtruth.day/learn/federated-learning.html

@misc{groundtruth:federated-learning,
  title  = {Federated learning},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/learn/federated-learning.html}
}

Topics: privacy · distributed-training · on-device · fundamentals · security