Federated learning has emerged as a leading approach for decentralized machine learning, enabling multiple clients to collaboratively train a shared model without exchanging private data. While federated learning enhances data privacy, it remains vulnerable to inference attacks, such as gradient inversion and membership inference, during both the training and inference phases. Specifically, QuanCrypt-FL is designed to defend against adversaries aiming to reconstruct sensitive client data through gradient leakage or membership inference in FL settings. Homomorphic Encryption provides a promising solution by encrypting model updates to protect against such attacks, but it introduces substantial communication overhead, slowing down training and increasing computational costs. To address these challenges, we propose QuanCrypt-FL, a novel algorithm that integrates fully homomorphic encryption with low-bit quantization and pruning techniques to enhance security against inference attacks while significantly reducing computational overhead, making federated learning more efficient and resilient. Further, we propose and implement mean-based clipping to mitigate quantization overflow or errors. By integrating these methods, QuanCrypt-FL creates a communication-efficient FL framework that ensures privacy protection with minimal impact on model accuracy, thereby improving both computational efficiency and attack resilience. We validate our approach on MNIST, HAM10000, CIFAR-10, and CIFAR-100 datasets, demonstrating superior performance compared to state-of-the-art methods. QuanCrypt-FL consistently outperforms existing method and matches Vanilla-FL in terms of accuracy across varying clients. Further, QuanCrypt-FL achieves up to 9× faster encryption, 16× faster decryption, and 1.5× faster inference compared to existing method, with training time reduced by up to 3×.
