Bonsai has discovered a pervasive issue with vector search across the entire ecosystem, impacting millions of implementations, and present in official vendor SDKs, documentation, tutorials, and articles.
Most embedding models return vectors as float32, but many clients cast and serialize them as float64. That doubles the precision, which doubles the disk and network cost. The extra digits add no accuracy, so you're paying to store and move noise.
We call this problem "Float Bloat"
We estimate this problem globally at over 20 Petabytes of unnecessary disk storage overhead.
What does the problem look like? Suppose you get a vector from your favorite model, embeddings API, or inference provider. It will return a list of float32s as the vector:
[ -0.011625106, 0.014652754, 0.0172214, -0.0177951529, 0.027116421, 0.06390719, 0.0082179, ... ]
But when the client casts and serializes the embedding, it raises the values' precision to float64 and adds meaningless digits to every dimension:
[ -0.011625106446444988, 0.014652754180133343, 0.017221400514245033, -0.017795152962207794, 0.02711642161011696, 0.063907191157341, 0.008217900060117245, ... ]
The added precision is just a side effect of floating point conversion (known as widening). It is not more accurate, and the additional digits take up disk space and network bandwidth. Depending on the vector database and search algorithm used, this can also result in additional CPU overhead when calculating vector similarity.
How often does it happen? #
At Bonsai, we sampled 18 diverse vector search clusters across all tiers, and found that 12 out of those 18 contained float bloat. All the way from sandbox through enterprise.
We also found it in the main branch of the world's most popular embedding vendor SDK, and in the public documentation of the world's largest cloud companies. It's present in hundreds of blogs and tutorials, and in numerous open source repos.
Nobody does this on purpose. It's the default behavior in several popular languages used for vector search. Take this Python example. You have an embedding stored in an object and you need to serialize it, either for transfer or storage:
embedding = my_numpy_vec.tolist() #<-- this is the culprit
json.dumps(embedding)
The above will provide float64 widened from float32.
In Python, The fix is cryptic and must be done explicitly, which explains the high prevalence of the problem:
values = my_numpy_vec.tolist()
embedding = [float(f"{value:.9g}") for value in values]
json.dumps(embedding)
Unless care is taken, the problem surfaces often during binary to JSON conversions, conversion to base64 and back, and when the incorrect numeric type is used in the client.
A float32 has nine digits and a float64 has seventeen. #
A float32
has a 24-bit mantissa and at most 9 significant digits. Cast it to float64
and the value is unchanged, but it now lives on a far finer grid that needs up to 17 digits. The default serializer will then cast and print all 17.
The serializer usually gets the blame, but the extra digits come from the cast. Most encoders will print a genuine float32
correctly; the value just tends to get promoted to float64
before it ever reaches them.
| Real dtype | Mantissa bits | Round-trip digits | Format |
|---|---|---|---|
| bfloat16 | 8 | 4 | %.4g |
| float16 | 11 | 5 | %.5g |
| float32 | 24 | 9 | %.9g |
| float64 | 53 | 17 | shortest |
Estimating impact #
We serialized the same 768-dim vector across five languages. Widened JSON runs ~1.8× the shortest-float32 text and ~5× the raw float32 binary. This is about 8 wasted bytes per value, and it repeats on every stored copy and every network hop. A re-index, replica, snapshot, and client cache are four copies and four hops, each carrying the widened precision.
| Corpus (768-dim) | Widened JSON | Shortest text | float32 binary | Text fix saves | Binary saves |
|---|---|---|---|---|---|
| 1M vectors | 15.2 GB | 8.6 GB | 3.1 GB | 6.6 GB | 12.1 GB |
| 10M vectors | 151.8 GB | 86.0 GB | 30.7 GB | 65.8 GB | 121.1 GB |
Use this handy calculator to estimate how much of your overhead is waste.
Ready to power your search with AI?
Launch a fully managed Elasticsearch or OpenSearch cluster, with built-in vector search and AI capabilities.
Create an AI Ready Search Cluster
Find and Fix It #
In the languages with no float32
scalar (JavaScript, Python, Ruby), widening is forced the instant a value leaves the typed array, so the fix is to format the digits yourself. In the ones that keep a real float (Java, C#, Rust), the fix is simpler: delete the up-cast and let the native encoder see the float32
. Every fix below is lossless.
JSON.stringify([...f32arr])
// a Float32Array element
// reads back as float64
js
'[' + Array.from(f32arr,
x => x.toPrecision(9)
).join(',') + ']'
json.dumps(vec.tolist())
// .tolist() promotes f32
// to a Python float (double)
'[' + ','.join(
'%.9g' % x for x in vec
) + ']'
JSON.generate(vectors)
// Ruby Float is always
// 64-bit; no f32 exists
'[' + vectors.map { |x|
'%.9g' % x
}.join(',') + ']'
temp.add((double) v[y][j]);
// double[] → Jackson
// prints 17-digit doubles
float[] embedding = v[y];
// Jackson emits
// shortest-float32
double[] Embedding { get; }
Serialize(embedding);
// store truncates to f32 anyway
float[] Embedding { get; }
Serialize(embedding);
// or ReadOnlyMemory<float>
json!(vec_f32)
// serde's json! macro
// widens during serialize
to_string(&vec_f32)
// serde (ryu) emits
// shortest-float32
We've also released a new agent skill bonsai-fix-float-bloat
, available in the Claude Marketplace as part of omc/search-skills
that can find and fix this issue for you. See it in our Search Skills repository on Github.
It's almost never your embedding service #
We surveyed OpenAI, Voyage, Cohere, Jina, Google, AWS, and Huggingface inference endpoints on float32
models. Every native wire we could sample emits shortest-float32 decimals. If your stored vectors are seventeen digits long, look at your client because that's probably the problem.
Where it actually enters
SDK .tolist()
calls, OpenAI-compatible wrapper shims, framework serializers, and “save embeddings to JSON” tutorials. OpenAI’s own SDK even requests compact base64 float32 bytes, then throws the win away with .tolist()
.
The cure already shipped
Cohere, Voyage, and Jina expose int8
, binary
, and base64
output types. A 1024-d binary vector is 128 bytes versus ~11 KB of widened JSON. Most tutorials ignore them and hand-roll json.dumps
instead.
The fix, in order of preference #
The bug needs two things on the storage path: a promotion to float64
, and writing it as decimal text. Break either link and the bloat is gone.
Serialize at the real precision.%.9g
(Python/C), f32ryu
(Rust),strconv.AppendFloat(b, x, 'g', -1, 32)
(Go),toPrecision(9)
(JS).Don't leave binary in the first place. If both ends are yours, ship base64 float32 bytes, Arrow, npy, or protobufrepeated float
.Pass through without re-serializing. If you're only relaying already-correct text, stream the bytes; don't parse-then-re-encode.
Also, talk to us at Bonsai if you're interested in seeing how we can help scale up your hybrid and vector search needs.
Ready to power your search with AI? #
Launch a fully managed Elasticsearch or OpenSearch cluster, with built-in vector search and AI capabilities.
Learn how a managed service works and why it’s valuable to dev teams
You won’t be pressured or used in any manipulative sales tactics
We’ll get a deep understanding of your current tech stack and needs
Get the information you need to decide whether to go with Bonsai
Create an AI Ready Search Cluster
Or, schedule a consultation: