Federated Averaging keeps raw data on each client, performs several local training passes, and sends weights to a central server for averaging; this sharply reduces communication rounds relative to synchronized SGD in the reported tasks. It is an important boundary case for Open Epistemic Intelligence. Coordination Systems can inherit local custody and heterogeneous data from it, but not claim fully decentralized coordination while aggregation still has a privileged center.
The paper that named federated learning. Each client trains on its own data for several local passes, sends the resulting weights, and a server averages them. Raw data never moves.
The efficiency comes from doing more work per round on each device. Naive synchronized SGD sends a message per minibatch; FederatedAveraging cuts the number of communication rounds by orders of magnitude on the same task.
It holds up on data that is unbalanced and not identically distributed across clients, which is what real devices produce.