Date of Award
5-2024
Document Type
Thesis
Degree Name
Master of Science (MS)
Department
Computer Science
Committee Chair/Advisor
Carlos Toxtli Hernandez
Committee Member
Rong Re
Committee Member
Nina Hubig
Committee Member
Mitch Shue
Abstract
Federated learning has emerged as a solution to the challenges faced by traditional centralized machine learning approaches, such as data privacy, security, ownership, and computational bottlenecks. However, federated learning itself introduced new challenges, including system heterogeneity and scalability. Existing federated learning approaches, such as hierarchical and heterogeneous federated learning, address some of these challenges but have limitations in real-world scenarios where multiple issues coexist, particularly in large-scale, heterogeneous environments like mobile applications and IoT devices. This work proposes a new federated learning architecture that combines heterogeneous federated learning and hierarchical federated learning into a unified architecture. The proposed approach aims to address the limitations of existing architectures by building clusters of models based on their types and usage of in-cluster models’ weights averaging to create an ensemble of heterogeneous models for further knowledge distillation into student models of different types that are to be distributed into respective clusters to continue training. The created implementation of the proposed architecture showed accuracy comparable with accuracy of FedDF chosen as a baseline heterogeneous federated learning architecture within the same environment with slight advantage in convergence speed in some cases.
Recommended Citation
Lukyanov, Dmitry, "Heterogeneous Federated Learning at Scale" (2024). All Theses. 4211.
https://open.clemson.edu/all_theses/4211
Author ORCID Identifier
0009-0002-4529-6012