Speaker
Description
Machine learning is today’s fastest-developing and most powerful computer technology, finding applications in
nearly every domain of science and industry, such as natural language processing, visual object detection, autonomous driving, stock market prediction, medical applications, and many more. For machine learning to be effective, a large amount of high-quality training data is essential. At the same time, the number of Internet of Things (IoT) devices such as smart watches, voice control systems, air quality monitors, surveillance cameras, and much more is increasing rapidly. These devices, as well as smartphones, are equipped with high-quality sensors generating an enormous amount of real-world data which has the potential to feed machine learning models. Due to the high privacy sensitivity and velocity of the data, classical offline training in data centers is not possible. To tackle these challenges, Google proposed a new approach to distributed machine learning called federated learning in 2016. In federated learning, clients train a global model on local data and upload only their model parameter deltas to a server where they get merged into a global model while keeping their local data private. While this approach holds great opportunity for privacy-preserving AI solutions e.g. for medical applications, several challenges limiting its efficiency and effectiveness remain, namely communication cost, the dependency on a trusted central server, and the parameter divergence in the presence of non-IID training data.
In this paper, we first analyze the state of the art with respect to the aforementioned challenges.
Considering the characteristics of modern wireless networks, we notice that there is a huge opportunity for leveraging wireless broadcasting in a hybrid system architecture comprising peer-to-peer subgroups and hierarchical servers. We design a new protocol for federated learning where clients asynchronously share gradient updates with other nearby clients via wireless broadcasting and globally exchange gradient updates via a hierarchical server network. This way, we benefit from the efficiency of wireless broadcasting to increase communication efficiency and decrease server involvement. Furthermore, the frequent exchange of gradient updates between clients allows us to better cope with non-IID training data and allows for higher privacy guarantees.
Our protocol serves as a framework that can easily be enhanced using many recently published contributions in federated learning. The impact of our work is expected to become even stronger in future 5G+ networks because it benefits from high device density and mobility.
