# Native binary embeddings experiment: curious about your thoughts

> Source: <https://discuss.huggingface.co/t/native-binary-embeddings-experiment-curious-about-your-thoughts/177107#post_1>
> Published: 2026-06-23 15:34:41+00:00

I spent a few days testing a simple hypothesis: does training a binary embedding model natively (with a binary loss) produce better retrieval than just binarizing a float model post-hoc?

The setup is deliberately small : bert-mini (~11M params), CPU-only training on a Mac Mini M4 Pro, NLI 550k pairs, 3 epochs.

Key results on SciFact Recall@10:

And at 1M vectors with FAISS (AVX2+POPCNT on x86):

The three things that made the binary model actually converge:

Models and code on GitHub / HuggingFace (korben99/bne-binary-2048).

Happy to hear if you’ve seen similar or contradictory results, especially at larger scales or with bigger backbones. Also curious whether the 2048-dim sweet spot holds with e.g. MiniLM.