{"slug": "float-bloat-vector-serialization-gone-wrong", "title": "Float Bloat: vector serialization gone wrong", "summary": "Bonsai, a vector search company, has identified a pervasive issue it calls 'Float Bloat' where embedding vectors are cast from float32 to float64 during serialization, doubling storage and network costs without adding accuracy. The company estimates this problem causes over 20 petabytes of unnecessary disk storage overhead globally, and it found the issue in 12 of 18 sampled vector search clusters, as well as in the main branch of the most popular embedding vendor SDK and documentation from major cloud companies.", "body_md": "Bonsai has discovered a pervasive issue with vector search across the entire ecosystem, impacting millions of implementations, and present in official vendor SDKs, documentation, tutorials, and articles.\n\nMost embedding models return vectors as float32, but many clients cast and serialize them as float64. That doubles the precision, which doubles the disk and network cost. The extra digits add no accuracy, so you're paying to store and move noise.\n\nWe call this problem \"Float Bloat\"\n\n**We estimate this problem globally at over 20 Petabytes of unnecessary disk storage overhead.**\n\nWhat does the problem look like? Suppose you get a vector from your favorite model, embeddings API, or inference provider. It will return a list of float32s as the vector:\n\n```\n[ -0.011625106, 0.014652754, 0.0172214, -0.0177951529, 0.027116421, 0.06390719, 0.0082179, ... ]\n```\n\nBut when the client casts and serializes the embedding, it raises the values' precision to float64 and adds meaningless digits to every dimension:\n\n```\n[ -0.011625106446444988, 0.014652754180133343, 0.017221400514245033, -0.017795152962207794, 0.02711642161011696, 0.063907191157341, 0.008217900060117245, ... ]\n```\n\nThe added precision is just a side effect of floating point conversion (known as widening). It is not more accurate, and the additional digits take up disk space and network bandwidth. Depending on the vector database and search algorithm used, this can also result in additional CPU overhead when calculating vector similarity.\n\n## How often does it happen?\n\nAt Bonsai, we sampled 18 diverse vector search clusters across all tiers, and found that 12 out of those 18 contained float bloat. All the way from sandbox through enterprise.\n\nWe also found it in the main branch of the world's most popular embedding vendor SDK, and in the public documentation of the world's largest cloud companies. It's present in hundreds of blogs and tutorials, and in numerous open source repos.\n\n# How does it happen?\n\nNobody does this on purpose. It's the default behavior in several popular languages used for vector search. Take this Python example. You have an embedding stored in an object and you need to serialize it, either for transfer or storage:\n\n```\n# Python with NumPy\nembedding = my_numpy_vec.tolist() #<-- this is the culprit\njson.dumps(embedding)\n```\n\nThe above will provide float64 widened from float32.\n\nIn Python, The fix is cryptic and must be done explicitly, which explains the high prevalence of the problem:\n\n```\n# Python with NumPy\nvalues = my_numpy_vec.tolist()\nembedding = [float(f\"{value:.9g}\") for value in values]\njson.dumps(embedding)\n```\n\nUnless care is taken, the problem surfaces often during binary to JSON conversions, conversion to base64 and back, and when the incorrect numeric type is used in the client.\n\n## A float32 has nine digits and a float64 has seventeen.\n\nA `float32`\n\nhas a 24-bit mantissa and at most **9 significant digits**. Cast it to `float64`\n\nand the value is unchanged, but it now lives on a far finer grid that needs up to **17 digits**. The default serializer will then cast and print all 17.\n\nThe serializer usually gets the blame, but the extra digits come from the cast. Most encoders will print a genuine `float32`\n\ncorrectly; the value just tends to get promoted to `float64`\n\nbefore it ever reaches them.\n\n| Real dtype | Mantissa bits | Round-trip digits | Format |\n|---|---|---|---|\n| bfloat16 | 8 | 4 | %.4g |\n| float16 | 11 | 5 | %.5g |\n| float32 | 24 | 9 | %.9g |\n| float64 | 53 | 17 | shortest |\n\n## Estimating impact\n\nWe serialized the same 768-dim vector across five languages. Widened JSON runs **~1.8×** the shortest-float32 text and **~5×** the raw float32 binary. This is about 8 wasted bytes per value, and it repeats on every stored copy and every network hop. A re-index, replica, snapshot, and client cache are four copies and four hops, each carrying the widened precision.\n\n| Corpus (768-dim) | Widened JSON | Shortest text | float32 binary | Text fix saves | Binary saves |\n|---|---|---|---|---|---|\n| 1M vectors | 15.2 GB | 8.6 GB | 3.1 GB | 6.6 GB | 12.1 GB |\n| 10M vectors | 151.8 GB | 86.0 GB | 30.7 GB | 65.8 GB | 121.1 GB |\n\nUse this handy calculator to estimate how much of your overhead is waste.\n\n### Ready to power your search with AI?\n\nLaunch a fully managed Elasticsearch or OpenSearch cluster, with built-in vector search and AI capabilities.\n\nCreate an AI Ready Search Cluster\n\n## Find and Fix It\n\nIn the languages with no `float32`\n\nscalar (**JavaScript, Python, Ruby**), widening is forced the instant a value leaves the typed array, so the fix is to format the digits yourself. In the ones that keep a real float (**Java, C#, Rust**), the fix is simpler: delete the up-cast and let the native encoder see the `float32`\n\n. Every fix below is lossless.\n\n```\nJSON.stringify([...f32arr])\n// a Float32Array element\n// reads back as float64\njs\n'[' + Array.from(f32arr,\n  x => x.toPrecision(9)\n).join(',') + ']'\njson.dumps(vec.tolist())\n// .tolist() promotes f32\n// to a Python float (double)\n'[' + ','.join(\n  '%.9g' % x for x in vec\n) + ']'\nJSON.generate(vectors)\n// Ruby Float is always\n// 64-bit; no f32 exists\n'[' + vectors.map { |x|\n  '%.9g' % x\n}.join(',') + ']'\ntemp.add((double) v[y][j]);\n// double[] → Jackson\n// prints 17-digit doubles\nfloat[] embedding = v[y];\n// Jackson emits\n// shortest-float32\ndouble[] Embedding { get; }\nSerialize(embedding);\n// store truncates to f32 anyway\nfloat[] Embedding { get; }\nSerialize(embedding);\n// or ReadOnlyMemory<float>\njson!(vec_f32)\n// serde's json! macro\n// widens during serialize\nto_string(&vec_f32)\n// serde (ryu) emits\n// shortest-float32\n```\n\nWe've also released a new agent skill `bonsai-fix-float-bloat`\n\n, available in the Claude Marketplace as part of `omc/search-skills`\n\nthat can find and fix this issue for you. See it in our [Search Skills](https://github.com/omc/search-skills) repository on Github.\n\n## It's almost never your embedding service\n\nWe surveyed OpenAI, Voyage, Cohere, Jina, Google, AWS, and Huggingface inference endpoints on `float32`\n\nmodels. Every native wire we could sample emits shortest-float32 decimals. If your stored vectors are seventeen digits long, look at your client because that's probably the problem.\n\n#### Where it actually enters\n\nSDK `.tolist()`\n\ncalls, OpenAI-compatible wrapper shims, framework serializers, and “save embeddings to JSON” tutorials. OpenAI’s own SDK even requests compact base64 float32 bytes, then throws the win away with `.tolist()`\n\n.\n\n#### The cure already shipped\n\nCohere, Voyage, and Jina expose `int8`\n\n, `binary`\n\n, and `base64`\n\noutput types. A 1024-d binary vector is **128 bytes** versus ~11 KB of widened JSON. Most tutorials ignore them and hand-roll `json.dumps`\n\ninstead.\n\n## The fix, in order of preference\n\nThe bug needs two things on the storage path: a promotion to `float64`\n\n, *and* writing it as decimal text. Break either link and the bloat is gone.\n\n**Serialize at the real precision.**`%.9g`\n\n(Python/C), f32`ryu`\n\n(Rust),`strconv.AppendFloat(b, x, 'g', -1, 32)`\n\n(Go),`toPrecision(9)`\n\n(JS).**Don't leave binary in the first place.** If both ends are yours, ship base64 float32 bytes, Arrow, npy, or protobuf`repeated float`\n\n.**Pass through without re-serializing.** If you're only relaying already-correct text, stream the bytes; don't parse-then-re-encode.\n\nAlso, talk to us at Bonsai if you're interested in seeing how we can help scale up your hybrid and vector search needs.\n\n## Ready to power your search with AI?\n\nLaunch a fully managed Elasticsearch or OpenSearch cluster, with built-in vector search and AI capabilities.\n\nLearn how a managed service works and why it’s valuable to dev teams\n\nYou won’t be pressured or used in any manipulative sales tactics\n\nWe’ll get a deep understanding of your current tech stack and needs\n\nGet the information you need to decide whether to go with Bonsai\n\nCreate an AI Ready Search Cluster\n\nOr, schedule a consultation:", "url": "https://wpnews.pro/news/float-bloat-vector-serialization-gone-wrong", "canonical_source": "https://bonsai.io/blog/float-bloat/", "published_at": "2026-08-27 17:12:50+00:00", "updated_at": "2026-08-27 19:51:26.991620+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools"], "entities": ["Bonsai"], "alternates": {"html": "https://wpnews.pro/news/float-bloat-vector-serialization-gone-wrong", "markdown": "https://wpnews.pro/news/float-bloat-vector-serialization-gone-wrong.md", "text": "https://wpnews.pro/news/float-bloat-vector-serialization-gone-wrong.txt", "jsonld": "https://wpnews.pro/news/float-bloat-vector-serialization-gone-wrong.jsonld"}}