Skip to content

Communication-Efficient Learning of Deep Networks from Decentralized Data (Federated Averaging)

Authors: H. Brendan McMahan, Erika Moore, Daniel Ramage, Seth Hampson, Blaise Agüera y Arcas Year: 2017 (arXiv 2016) Venue: AISTATS 2017 Citations: 10,000+

Summary

Introduces Federated Averaging (FedAvg), an algorithm for training models across decentralized data. Clients perform local SGD updates, server averages models. Enables privacy-preserving and communication-efficient distributed learning.

Key Concepts

  • Decentralized Learning: Training without centralizing data
  • Federated Averaging: Average client model updates at server
  • Communication Efficiency: Reduce communication rounds via multiple local epochs
  • Privacy Preservation: Data never leaves devices
  • Non-IID Data: Clients have different data distributions
  • Robustness: Works despite system heterogeneity

Impact

  • 10,000+ citations
  • Foundation of federated learning research
  • Important for privacy-preserving machine learning
  • Applications in mobile devices, IoT, healthcare
  • Inspired FedProx, FedAdam, and variants
  • Critical for privacy-regulation compliance (GDPR, etc.)
  • Growing importance in production ML systems

Key Results

  • Convergence on non-IID data
  • Reduced communication vs. centralized learning
  • Practical algorithm for mobile devices
  • Better privacy without sacrificing accuracy
  • Federated Optimization (Konečný et al., 2016)
  • FedProx (Li et al., 2020)
  • FedAvg with Momentum (Wang et al., 2020)
  • Differential Privacy (Dwork et al., 2006)