# Private AI Inference with Homomorphic Encryption: A Practical Guide to Computing on Encrypted Data > Source: > Published: 2026-08-15 12:23:41+00:00 In 2009, Craig Gentry proved that it is possible to compute on encrypted data without ever decrypting it, and the result was widely treated as a theoretical curiosity. Sixteen years later, homomorphic encryption has crossed from conference papers into production pipelines: banks screen transactions against encrypted watchlists, hospitals run diagnostic models on data that never leaves their custody, and in August 2026 Google announced private AI features built on the same primitives. The gap between "possible in theory" and "usable in practice" is still wide, but it is no longer an argument against trying. This guide walks through what homomorphic encryption actually computes, how the CKKS scheme turns encrypted vectors into a workable substrate for machine learning, and the cost model that decides whether a private inference pipeline is worth building at all. Ordinary encryption has a hard property: a ciphertext reveals nothing about the plaintext. AES-CTR, ChaCha20, RSA — all of them scramble data so thoroughly that an attacker holding the ciphertext and a supercomputer cannot recover the message without the key. That property is also the problem. If a server stores customer data encrypted at rest, every query requires shipping the data (or the key) somewhere a human or a process can read it. The moment the data is decrypted for computation, the confidentiality boundary moves from the storage layer to the memory of whatever process is doing the work. Homomorphic encryption changes the terms. A homomorphic scheme is one where operations on ciphertexts correspond to operations on plaintexts: `Enc(a) ⊕ Enc(b) = Enc(a + b)` . A server can add, multiply, and combine encrypted values and return the encrypted result, and the client — the only party holding the key — decrypts the final answer. The server learns nothing about the inputs, the intermediate values, or the output. For inference, this is the entire ballgame: the model owner never exposes weights, and the data owner never exposes the query. To see why this is hard, consider what AES does to a single byte. The S-box substitution and the ShiftRows/MixColumns rounds mix the input so completely that flipping one plaintext bit changes roughly half the output bits. That avalanche effect is what makes AES secure, and it is exactly what makes it unusable for computation. There is no way to run `a + b` on AES ciphertexts because the algebraic structure of the ciphertext has no relationship to the algebraic structure of the plaintext. Fully homomorphic encryption takes the opposite approach. Instead of starting with a scrambling cipher and hoping arithmetic survives, it starts with an algebraic structure that naturally supports both operations. The classic construction works over polynomial rings: plaintexts are small polynomials, ciphertexts are pairs of larger polynomials, and addition and multiplication in the ring map to addition and multiplication of the encrypted messages. Security comes not from destroying structure but from noise — every operation grows a random error term that must stay small enough for decryption to still recover the message. Multiply too many times and the noise drowns the signal. Gentry's 2009 breakthrough had two parts. The first was the observation that a scheme with bounded noise can still be made unbounded: if the decryption circuit is shallow enough, the server can evaluate it homomorphically, producing a fresh ciphertext with reset noise. That process, bootstrapping, was the theoretical missing piece. The second part was a working scheme with a decryption circuit shallow enough to bootstrap. The catch was performance — early bootstrapping took minutes per operation. The decade after Gentry produced a family of practical schemes. BGV and BFV handle encrypted integers with exact arithmetic. TFHE (CGGI) works over encrypted bits and is fast enough for small circuits like database lookups and comparisons. And CKKS, published by Cheon, Kim, Kim, and Song in 2017, introduced approximate arithmetic over encrypted real numbers, with a noise budget that behaves like floating-point error. That last property is what made machine learning viable: neural networks already tolerate small numerical errors, so a scheme that treats noise as precision loss fits the workload instead of fighting it. CKKS also inherits a trick called SIMD packing from its predecessors. A single ciphertext in CKKS is not one number but a vector of hundreds of slots, and operations apply element-wise across the whole vector. A dot product — the inner loop of almost every inference step — becomes one ciphertext multiplication and a few rotations instead of hundreds of separate operations. The construction is a polynomial ring `R = Z[X] / (X^N + 1)` , typically with `N = 4096` or `8192` . Plaintexts are polynomials of degree less than `N` whose coefficients are the numbers you actually care about, scaled by a large factor and rounded. The scale factor is the scheme's version of fixed-point arithmetic: it sets how many bits of fractional precision survive a multiplication. After each multiply, the scale of the result grows, and the scheme provides a rescaling operation that brings it back down. Noise is the real constraint. Each addition adds noise linearly; each multiplication roughly squares it. The scheme allocates a noise budget at encryption time, and every operation spends some of it. When the budget hits zero, decryption produces garbage. This is why parameter selection matters more in FHE than in any other part of a machine learning stack: the ring degree sets the maximum vector size and the top of the noise budget, the scale factor sets precision, and the multiplication depth you need sets how many levels the chain must provide before bootstrapping becomes necessary. A minimal encrypted vector in Python, using a research-oriented binding, looks like this: ``` python from openfhe import CKKSRNS, SecurityLevel, ScalingTechnique # A fresh CKKS context with 40 levels of multiplicative depth params = CKKSRNS() params.SetMultiplicativeDepth(40) params.SetScalingModSize(50) params.SetSecurityLevel(SecurityLevel.HEStd_128_classic) cc = CKKSRNS.GenCryptoContext(params) cc.Enable(PKE) cc.Enable(KEYSWITCH) cc.Enable(LEVELEDSHE) cc.Enable(ADVANCED) keys = cc.KeyGen() cc.EvalMultKeyGen(keys.secretKey) # allow encrypted multiplication cc.EvalRotateKeyGen(keys.secretKey, [1, 2, 4, 8]) plain = cc.MakePackedPlaintext([0.5, 1.5, 2.5, 3.5]) ct = cc.Encrypt(keys.publicKey, plain) ``` That is the entire setup. `ct` is now a ciphertext the server can store, transform, and return without ever seeing the values inside. The classic private inference flow is split between a client that owns the data and a server that owns the model. The client encrypts its input, the server evaluates the model homomorphically, and the client decrypts the result. For a logistic regression — a single affine transform followed by a sigmoid — the encrypted computation is small enough to show in full: ``` python def encrypted_predict(ct_x, ct_w, ct_b, cc, keys): # Encrypted dot product: one multiply, one add over packed slots ct_z = cc.EvalAdd(cc.EvalMult(ct_x, ct_w), ct_b) # Approximate the sigmoid with a low-degree polynomial # because division and exp are not directly supported sigmoid_approx = ( 0.5 + 0.197 * ct_z - 0.004 * cc.EvalMult(ct_z, ct_z) * ct_z ) return sigmoid_approx ``` Three details matter here. First, the sigmoid must be replaced by a polynomial approximation such as a Taylor or Chebyshev expansion, because CKKS supports only addition and multiplication. Second, the approximation degree is a direct trade against the noise budget: every extra multiply spends a level. Third, the whole vector of slots is processed in parallel, so one encrypted call classifies an entire batch of inputs, not a single sample. For deeper networks, the same recipe repeats layer by layer: convolution becomes a sum of shifted and multiplied ciphertexts, ReLU becomes a polynomial like `x^2` -based approximation or a TFHE-style comparison, and pooling becomes rotations plus additions. The engineering problem is not expressing the model — it is keeping the depth inside the budget. Homomorphic inference is slow, and the honest framing is to say how slow and why. Ciphertexts are two or three orders of magnitude larger than the plaintexts they hold. Multiplication on a packed ciphertext is roughly a thousand to ten thousand times more expensive than the equivalent plaintext float operation, depending on ring degree and security level. Bootstrapping, when the noise budget runs out, costs on the order of tens of milliseconds to seconds per ciphertext — cheap enough to amortize over a packed batch, ruinous if applied per element. The practical consequence is that private inference shifts the bottleneck from model quality to arithmetic budget. A model that runs in 2 milliseconds on plaintext floats can take seconds homomorphically, and the gap is dominated by the number of multiplications per slot, not the model's parameter count. Architectures that are friendly to FHE are the ones that keep multiplicative depth low: shallow MLPs, quantized networks, models with polynomial activations. Deep transformer stacks with attention and softmax are the worst case, because attention is a softmax followed by matrix products — softmax division is not a native operation, and its polynomial replacement is expensive. The honest answer is that homomorphic inference earns its cost in a narrow but real set of situations. The first is regulated data with a shared computation: hospitals collaborating on a model where patient records cannot leave each institution's boundary, or banks running joint fraud models over accounts they are legally barred from sharing. The second is API-based inference where the query itself is the secret — legal research, medical symptom triage, proprietary financial signals — and the client does not want the server to see what it is asking. The third is the emerging pattern behind the August 2026 announcements: consumer AI where the provider wants to process a user's data without being able to read it, as a product differentiator rather than a regulatory requirement. What homomorphic encryption does not buy you is protection against a malicious server. A server that controls the evaluation can still drop requests, return garbage, or measure timing and access patterns. FHE guarantees confidentiality of the data against the server's curiosity, not integrity of the result against the server's malice. Teams that need both must add zero-knowledge proofs or commit-and-reveal protocols on top. The ecosystem has consolidated around a few serious options. Microsoft SEAL is the reference implementation of BFV and CKKS in C++, with Python bindings via the pybind11-based extensions; it is battle-tested but expects the caller to manage parameters. OpenFHE is the community successor, actively maintained, and adds BGV, TFHE, and a unified API across schemes. TenSEAL, built on SEAL, provides a Pythonic API for CKKS over PyTorch tensors, though its maintenance has slowed. Zama's Concrete implements TFHE with a compiler that takes plain Python functions and emits bootstrapped circuits, which suits smaller integer workloads like lookups and decision trees. For a new project, the practical default is OpenFHE for custom CKKS work, and Concrete when the target workload is small and integer-shaped. ``` # OpenFHE exposes the same context pattern across schemes, # so the pipeline above ports to BGV with two line changes: params = BGVRNS() params.SetMultiplicativeDepth(20) cc = BGVRNS.GenCryptoContext(params) ``` The parameter selection itself is a black art that libraries are only beginning to automate. The rule of thumb is: choose the ring degree from the vector size and security level, choose the scaling modulus from the precision you need, then count the multiplicative depth of your exact model graph and add headroom for the approximations. Getting this wrong shows up not as a crash but as silent precision loss at the output — which makes an end-to-end test with known plaintexts the most important step in any FHE project. A useful heuristic before committing engineering time: if the plaintext model fits on a single machine and the deployment is a one-off computation, homomorphic encryption is probably the wrong tool — shipping the data under an agreement is simpler. If the computation is recurring, the inputs are sensitive, and the participants do not trust each other enough to share plaintexts, the calculus flips. Start with a single layer, measure the noise budget after every operation, and instrument the number of slots used per ciphertext. Most teams discover that the bottleneck is not the cryptographic primitives but the model's activation functions, and that replacing one ReLU with a polynomial buys more than a faster library ever could. The field is also moving faster than its reputation suggests. Ciphertext compression, GPU kernels for CKKS multiplication, and programmable bootstrapping have each cut the effective cost by an order of magnitude within the last few years. The 2009 result was a proof that encrypted computation is possible; the current state of the art is a proof that it is affordable for the workloads where confidentiality is actually worth money. For anyone building AI products on other people's data, that is a gap worth watching — and a pilot worth running. Originally published on [Dispatch](https://dispatch-blog.hashnode.dev/private-ai-inference-with-homomorphic-encryption-a-practical-guide-to-computing-on-encrypted-data).