FedBNN trains one-bit neural networks across federated devices using rotation-aware binarization, cutting FLOPs up to 58x and memory 32x with competitive accuracy.
-
FedBNN trains binary neural networks directly during federated training rather than binarizing afterward.
-
Rotation-aware binarization is adapted to federated learning via server-client exchange of rotation info.
-
Delivers up to 58x FLOP and 32x memory reductions versus full-precision federated models.
-
Accuracy stays competitive with FedAvg across FMNIST, SVHN, CIFAR10, TinyImageNet, FEMNIST.
-
Tested on CNN4, ResNet10, ResNet18, and ConvNeXt-Tiny under IID and non-IID client splits.
-
Paper on arXiv ; no public code release linked yet. Researchers at IIT Madras and UIUC have proposed FedBNN, a framework that trains binary neural networks within the federated learning loop. The paper reports reductions of up to 58 times in floating-point operations and 32 times in model memory, while retaining accuracy close to full-precision federated baselines across several image-classification benchmarks.
Federated learning reduces the movement of raw data by training across participating devices, but the resulting model must still fit each device’s memory, compute, and energy budget. FedBNN targets that deployment constraint by representing weights and activations with one bit during inference. Model updates can still reveal information, so secure aggregation, differential privacy, and related protections remain separate concerns.
Full-precision training leaves an inference problem #
Common FedAvg deployments train models whose weights are stored as 32-bit floating-point values. That format works well on servers, but it can strain phones, sensors, and microcontrollers that must run the model repeatedly under tight power and memory limits.
Binary neural networks compress weights and activations to values such as plus one and minus one. Packed binary kernels can replace floating-point multiplication with XNOR operations, which compare bits, and population-count operations, which count matching results. Post-training binarization often loses substantial accuracy because rounding a trained floating-point model onto that two-value grid introduces large errors.
This story is for Pro members #
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.