Federated and multi-agent learning provide complementary frameworks for building intelligent systems that operate across distributed, heterogeneous, and often partially cooperative entities. Federated learning focuses on collaboratively training a shared global model while data remain decentralized, enabling organizations, devices, or users to benefit from collective knowledge without exposing sensitive information. Multi-agent learning, in contrast, studies how multiple autonomous agents learn policies or representations through interaction, coordination, negotiation, or competition within a shared environment. When combined, these paradigms enable systems in which agents not only learn from their own experiences but also participate in privacy-preserving knowledge sharing that supports collective performance. This integration is particularly powerful in domains such as robotics, healthcare, environmental monitoring, and large-scale socio-technical systems, where agents must adapt locally, respect autonomy constraints, and still contribute to shared global objectives.
Federated learning (FL) in heterogeneous environments remains challenging because client models often differ in both architecture and data distribution. While recent approaches attempt to address this challenge through client clustering and knowledge distillation, simultaneously handling architectural and statistical heterogeneity remains difficult. We introduce COSMOS, a model-agnostic framework that enables server-side personalization using only pseudo-label communication. Clients train local models and predict on the public data; the server clusters clients by prediction similarity, trains a cluster-specific model for each group using its own compute, and distills the resulting models back to clients. We provide the first theoretical analysis showing that distillation from the learned cluster models can yield exponential personalization risk contraction, going beyond the convergence-to-stationarity guarantees typically provided in model-agnostic FL. Experiments across benchmarks demonstrate that COSMOS consistently outperforms all model-agnostic FL baselines while remaining competitive with state-of-the-art personalized FL methods. More broadly, our results highlight personalized server-side learning with pseudo-labels as a promising paradigm for scalable and model-agnostic federated learning in highly heterogeneous environments.
@inproceedings{conf/ecml/RachmutGZYV26,author={Rachmut, Ben and Ge, Luise and Zhang, Ning and Yeoh, William and Vorobeychik, Yevgeniy},title={{COSMOS}: Model-Agnostic Personalized Federated Learning with Clustered Server Models and Pseudo-Label-Only Communication},booktitle={European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},pages={to appear},year={2026},}
ACL
Protecting Language Models Against Unauthorized Distillation through Trace Rewriting
Xinhang Ma, William Yeoh, Ning Zhang, and
1 more author
In Annual Meeting of the Association for Computational Linguistics, 2026
Knowledge distillation is a widely adopted technique for transferring capabilities from LLMs to smaller, more efficient student models. However, unauthorized use of knowledge distillation takes unfair advantage of the considerable effort and cost put into developing frontier models. We investigate methods for modifying teacher-generated reasoning traces to achieve two objectives that deter unauthorized distillation: (1) anti-distillation, or degrading the training usefulness of query responses, and (2) API watermarking, which embeds verifiable signatures in student models. We introduce several approaches for dynamically rewriting a teacher’s reasoning outputs while preserving answer correctness and semantic coherence. Two of these leverage the rewriting capabilities of LLMs, while others use gradient-based techniques. Our experiments show that a simple instruction-based rewriting approach achieves a strong anti-distillation effect while maintaining or even improving teacher performance. Furthermore, we show that our rewriting approach also enables embedding watermarks that can be reliably detected with essentially no false alarms. Our code is available at https://github.com/xhOwenMa/trace-rewriting.
@inproceedings{conf/acl/MaYZV26,author={Ma, Xinhang and Yeoh, William and Zhang, Ning and Vorobeychik, Yevgeniy},title={Protecting Language Models Against Unauthorized Distillation through Trace Rewriting},booktitle={Annual Meeting of the Association for Computational Linguistics},pages={11307--11324},year={2026},}
FAccT
Learning Fairness in Multi-agent Systems with Distributed-Evaluation, Centralized-Allocation
Ashwin Kumar and William Yeoh
In ACM Conference on Fairness, Accountability, and Transparency, 2026
A wide variety of resource allocation problems operate under resource constraints that are managed by a central arbitrator, with agents who evaluate and communicate preferences over these resources. We formulate this broad class of problems as Distributed Evaluation, Centralized Allocation (DECA) problems and propose methods to learn fair and efficient policies in centralized resource allocation. Our methods are applied to learning long-term fairness in a novel and general framework for fairness in multi-agent systems. We show three different methods based on Double Deep Q-Learning: (1) A joint weighted optimization of fairness and utility, (2) a split optimization, learning two separate Q-estimators for utility and fairness, and (3) an online policy perturbation to guide existing black-box utility functions toward fair solutions. Our methods outperform existing fair MARL approaches on multiple resource allocation domains, even when evaluated using diverse fairness functions, and allow for flexible online trade-offs between utility and fairness.
@inproceedings{conf/fat/KumarY26,author={Kumar, Ashwin and Yeoh, William},title={Learning Fairness in Multi-agent Systems with Distributed-Evaluation, Centralized-Allocation},booktitle={ACM Conference on Fairness, Accountability, and Transparency},pages={4133--4161},year={2026},}
EMNLP
EcoLoRa: Communication-Efficient Federated Fine-Tuning of Large Language Models
Han Liu, Ruoyao Wen, Srijit Nair, and
6 more authors
In Conference on Empirical Methods in Natural Language Processing, 2025
To address data locality and privacy restrictions, Federated Learning (FL) has recently been adopted to fine-tune large language models (LLMs), enabling improved performance on various downstream tasks without requiring aggregated data. However, the repeated exchange of model updates in FL can result in prohibitively high communication costs, hinder- ing the distributed learning process. To address this challenge, we propose EcoLoRa, a novel communication-efficient federated fine-tuning framework for LLMs. Leveraging the modular structure, we propose a round-robin segment sharing scheme, where each client uploads only a complementary LoRA segment per round to reduce network bandwidth. It is further combined with adaptive sparsification methods tailored to LoRA’s training dynamics and lossless encoding techniques. We conduct extensive evaluations on both question-answering and value-alignment tasks across multiple datasets and models. The results show that EcoLoRa significantly reduces communication overhead without compromising performance. For in- stance, it reduces communication time by up to 79% and total training time by up to 65%.
@inproceedings{conf/emnlp/LiuWNLLZTVZ25,author={Liu, Han and Wen, Ruoyao and Nair, Srijit and Liu, Jia and Lou, Wenjing and Zhang, Chongjie and Yeoh, William and Vorobeychik, Yevgeniy and Zhang, Ning},title={{EcoLoRa}: Communication-Efficient Federated Fine-Tuning of Large Language Models},booktitle={Conference on Empirical Methods in Natural Language Processing},pages={20743--20757},year={2025},}
DAC
Resilient Federated Learning on Embedded Devices
with Constrained Network Connectivity
Federated learning enables decentralized model training while preserving data privacy. However, since the learning process overlays the physical network infrastructure, the efficiency of learning can be impacted by network connectivity. In this work, we conducted extensive experiments to empirically characterize the impacts and leverage the insights to propose an adaptive federation framework, where clients with limited bandwidth are only prompted to transmit adaptively compressed gradient updates when the gradient similarity score is similar between the local and global models. Our evaluation in simulated environments and on real hardware devices shows bandwidth savings of 60% to 78% compared to state-of-the-art methods.
@inproceedings{conf/dac/LiLLCVYLZ25,author={Li, Zihan and Liu, Han and Li, Ao and Chan, Ching-hsiang and Vorobeychik, Yevgeniy and Yeoh, William and Lou, Wenjing and Zhang, Ning},title={Resilient Federated Learning on Embedded Devices
with Constrained Network Connectivity},booktitle={{ACM/IEEE} Design Automation Conference},pages={1--7},year={2025},}