Float Bloat: vector serialization gone wrong Bonsai, a vector search company, has identified a pervasive issue it calls 'Float Bloat' where embedding vectors are cast from float32 to float64 during serialization, doubling storage and network costs without adding accuracy. The company estimates this problem causes over 20 petabytes of unnecessary disk storage overhead globally, and it found the issue in 12 of 18 sampled vector search clusters, as well as in the main branch of the most popular embedding vendor SDK and documentation from major cloud companies. Bonsai has discovered a pervasive issue with vector search across the entire ecosystem, impacting millions of implementations, and present in official vendor SDKs, documentation, tutorials, and articles. Most embedding models return vectors as float32, but many clients cast and serialize them as float64. That doubles the precision, which doubles the disk and network cost. The extra digits add no accuracy, so you're paying to store and move noise. We call this problem "Float Bloat" We estimate this problem globally at over 20 Petabytes of unnecessary disk storage overhead. What does the problem look like? Suppose you get a vector from your favorite model, embeddings API, or inference provider. It will return a list of float32s as the vector: -0.011625106, 0.014652754, 0.0172214, -0.0177951529, 0.027116421, 0.06390719, 0.0082179, ... But when the client casts and serializes the embedding, it raises the values' precision to float64 and adds meaningless digits to every dimension: -0.011625106446444988, 0.014652754180133343, 0.017221400514245033, -0.017795152962207794, 0.02711642161011696, 0.063907191157341, 0.008217900060117245, ... The added precision is just a side effect of floating point conversion known as widening . It is not more accurate, and the additional digits take up disk space and network bandwidth. Depending on the vector database and search algorithm used, this can also result in additional CPU overhead when calculating vector similarity. How often does it happen? At Bonsai, we sampled 18 diverse vector search clusters across all tiers, and found that 12 out of those 18 contained float bloat. All the way from sandbox through enterprise. We also found it in the main branch of the world's most popular embedding vendor SDK, and in the public documentation of the world's largest cloud companies. It's present in hundreds of blogs and tutorials, and in numerous open source repos. How does it happen? Nobody does this on purpose. It's the default behavior in several popular languages used for vector search. Take this Python example. You have an embedding stored in an object and you need to serialize it, either for transfer or storage: Python with NumPy embedding = my numpy vec.tolist <-- this is the culprit json.dumps embedding The above will provide float64 widened from float32. In Python, The fix is cryptic and must be done explicitly, which explains the high prevalence of the problem: Python with NumPy values = my numpy vec.tolist embedding = float f"{value:.9g}" for value in values json.dumps embedding Unless care is taken, the problem surfaces often during binary to JSON conversions, conversion to base64 and back, and when the incorrect numeric type is used in the client. A float32 has nine digits and a float64 has seventeen. A float32 has a 24-bit mantissa and at most 9 significant digits . Cast it to float64 and the value is unchanged, but it now lives on a far finer grid that needs up to 17 digits . The default serializer will then cast and print all 17. The serializer usually gets the blame, but the extra digits come from the cast. Most encoders will print a genuine float32 correctly; the value just tends to get promoted to float64 before it ever reaches them. | Real dtype | Mantissa bits | Round-trip digits | Format | |---|---|---|---| | bfloat16 | 8 | 4 | %.4g | | float16 | 11 | 5 | %.5g | | float32 | 24 | 9 | %.9g | | float64 | 53 | 17 | shortest | Estimating impact We serialized the same 768-dim vector across five languages. Widened JSON runs ~1.8× the shortest-float32 text and ~5× the raw float32 binary. This is about 8 wasted bytes per value, and it repeats on every stored copy and every network hop. A re-index, replica, snapshot, and client cache are four copies and four hops, each carrying the widened precision. | Corpus 768-dim | Widened JSON | Shortest text | float32 binary | Text fix saves | Binary saves | |---|---|---|---|---|---| | 1M vectors | 15.2 GB | 8.6 GB | 3.1 GB | 6.6 GB | 12.1 GB | | 10M vectors | 151.8 GB | 86.0 GB | 30.7 GB | 65.8 GB | 121.1 GB | Use this handy calculator to estimate how much of your overhead is waste. Ready to power your search with AI? Launch a fully managed Elasticsearch or OpenSearch cluster, with built-in vector search and AI capabilities. Create an AI Ready Search Cluster Find and Fix It In the languages with no float32 scalar JavaScript, Python, Ruby , widening is forced the instant a value leaves the typed array, so the fix is to format the digits yourself. In the ones that keep a real float Java, C , Rust , the fix is simpler: delete the up-cast and let the native encoder see the float32 . Every fix below is lossless. JSON.stringify ...f32arr // a Float32Array element // reads back as float64 js ' ' + Array.from f32arr, x = x.toPrecision 9 .join ',' + ' ' json.dumps vec.tolist // .tolist promotes f32 // to a Python float double ' ' + ','.join '%.9g' % x for x in vec + ' ' JSON.generate vectors // Ruby Float is always // 64-bit; no f32 exists ' ' + vectors.map { |x| '%.9g' % x }.join ',' + ' ' temp.add double v y j ; // double → Jackson // prints 17-digit doubles float embedding = v y ; // Jackson emits // shortest-float32 double Embedding { get; } Serialize embedding ; // store truncates to f32 anyway float Embedding { get; } Serialize embedding ; // or ReadOnlyMemory