arXiv:2611.17576v1 Announce Type: new Abstract: Binary neural networks (BNNs) reduce model storage and replace many arithmetic operations with efficient bitwise computations, but the single-sign activation limits their representational capacity. Each preactivation direction is compared with one threshold and reduced to one of two values, discarding variation within each sign region. We propose \textbf{BiTE} (Branch-Preserving Binary Threshold Expansion), which evaluates each feature direction against $K$ ordered thresholds and retains the resulting binary responses as separate inputs to the following learned operator. BiTE can distinguish up to $K+1$ ordered activation states while allowing the following operator to learn a separate transformation for each threshold response. It also enables a controllable trade-off between the number of independently constructed feature directions and the number of ordered responses extracted from each direction. On BinaryViT, BiTE improves mean final-epoch Top-1 accuracy from $74.60\pm0.30%$ to $75.83\pm0.15%$ on CIFAR-10 and from $42.80\pm0.46%$ to $44.07\pm0.32%$ on CIFAR-100, with nearly unchanged trainable parameter counts. A matched uniform-collapse control reaches $75.33\pm0.15%$ on CIFAR-10, supporting the value of retaining the threshold responses separately relative to the evaluated scalar collapse. Binary VGG-16 experiments further show a favorable observed parameter--accuracy trade-off, including a configuration that achieves $0.64$ percentage points higher mean accuracy with $21.8%$ fewer parameters than conventional width scaling. Overall, these results support branch-preserving threshold expansion as an effective way to improve binary representations and the observed parameter--accuracy trade-off.
Beyond a Single Sign: BiTE for Branch-Preserving Binary Threshold Expansion
A new arXiv paper (2611.17576v1) proposes BiTE (Branch-Preserving Binary Threshold Expansion), a method that evaluates each feature direction against K ordered thresholds and keeps the resulting binary responses as separate inputs to the next learned operator, letting binary neural networks distinguish up to K+1 ordered activation states instead of a single sign. On BinaryViT, BiTE raised mean final-epoch Top-1 accuracy from 74.60±0.30% to 75.83±0.15% on CIFAR-10 and from 42.80±0.46% to 44.07±0.32% on CIFAR-100 with nearly unchanged trainable parameter counts, while a matched uniform-collapse control reached 75.33±0.15% on CIFAR-10. Binary VGG-16 experiments included a configuration achieving 0.64 percentage points higher mean accuracy with 21.8% fewer parameters than conventional width scaling.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.