ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever et al.
1.5k
Citations
103
Influential Citations
IEEE Transactions on Wireless Communications
Venue
2020
Year
In this article, the problem of training federated learning (FL) algorithms over a realistic wireless network is studied. In the considered model, wireless users execute an FL algorithm while training their local FL models using their own data and transmitting the trained local FL models to a base station (BS) that generates a global FL model and sends the model back to the users. Since all training parameters are transmitted over wireless links, the quality of training is affected by wireless factors such as packet errors and the availability of wireless resources. Meanwhile, due to the limited wireless bandwidth, the BS needs to select an appropriate subset of users to execute the FL algorithm so as to build a global FL model accurately. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize an FL loss function that captures the performance of the FL algorithm. To seek the solution, a closed-form expression for the expected convergence rate of the FL algorithm is first derived to quantify the impact of wireless factors on FL. Then, based on the expected convergence rate of the FL algorithm, the optimal transmit power for each user is derived, under a given user selection and uplink resource block (RB) allocation scheme. Finally, the user selection and uplink RB allocation is optimized so as to minimize the FL loss function. Simulation results show that the proposed joint federated learning and communication framework can improve the identification accuracy by up to 1.4%, 3.5% and 4.1%, respectively, compared to: 1) An optimal user selection algorithm with random resource allocation, 2) a standard FL algorithm with random user selection and resource allocation, and 3) a wireless optimization algorithm that minimizes the sum packet error rates of all users while being agnostic to the FL parameters.
This paper addresses a critical gap in federated learning (FL) research: most prior work assumes ideal communication channels, ignoring the realities of wireless networks such as packet errors, limited bandwidth, and the need to select a subset of users. As FL is increasingly deployed on mobile and IoT devices, understanding and mitigating the impact of wireless impairments is essential for practical performance. The authors propose a joint framework that co-designs learning and communication, moving beyond treating them as separate problems.
The significance lies in its holistic optimization: rather than optimizing wireless metrics (e.g., packet error rate) independently, the framework directly minimizes the FL loss function. This aligns communication decisions with the ultimate learning objective, leading to measurable accuracy gains. The paper also provides a theoretical convergence rate that quantifies how wireless factors degrade FL, offering a foundation for future work.
The proposed joint framework achieves identification accuracy improvements of:
These gains demonstrate that co-optimizing learning and communication yields tangible benefits, especially when wireless resources are scarce. The results also highlight the importance of considering FL-specific metrics (e.g., loss function) rather than generic wireless metrics.
This work has broad implications for deploying FL in real-world wireless systems, such as edge computing, autonomous vehicles, and smart cities. By providing a principled framework that integrates communication constraints into the learning loop, it enables more reliable and accurate FL under practical conditions. The theoretical convergence analysis also serves as a tool for future research on communication-efficient FL. The paper has been highly cited (1499 citations), reflecting its impact on the intersection of wireless communications and machine learning.
Alex Krizhevsky, Ilya Sutskever et al.
Ashish Vaswani, Noam Shazeer et al.
Douglas M. Bates, Martin Mächler et al.
Diederik P. Kingma, Jimmy Ba