Google is introducing HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler and development toolchain designed to make encrypted computation easier to deploy. In particular, HEIR can compile pre-trained AI models built for conventional, unencrypted inputs so they can instead operate on encrypted data.
According to Google homomorphic encryption (HE) addresses several challenges with end-to-end encryption, including enabling services such as spam and virus detection to operate on encrypted user data, mitigating the risk of exposing proprietary AI models when they are deployed on users' devices, and more.
A solution to these issues is homomorphic encryption, a rapidly maturing technology that fundamentally alters this trade-off by allowing computations to be performed directly on encrypted data. Servers can process ciphertexts and return encrypted results without exposing any underlying information.
In this context, HEIR is an open-source compiler toolchain and development platform for HE that can adapt pre-trained AI models designed to operate on unencrypted data to instead process encrypted inputs. Google has used HEIR to enable private content recommendations without exposing user data, detect credit card fraud while protecting sensitive financial information, identify network intrusions without revealing packet contents to the service provider, and recognize hotwords in audio streams without disclosing the underlying recordings.
HEIR uses an intermediate representation as an abstraction layer to "represent and scale complex models across diverse dialects". According to Google, application developers can write a program in Python, annotate which data types should be encrypted, and then use HEIR to compile the code for homomorphic execution.
Google provides a set of demos showcasing how pre-trained PyTorch models can be compiled to FHE. Currently the process is not yet automated, and involves a few steps, beginning with using torch_mlir to export the PyTorch model to MLIR.
One concern raised by several Hacker News readers is performance. Sabretooth1405
noted that "from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable". dhx
provided some concrete figures, noting that a 64 bit equality operation takes 80ms, addition and subtraction 100ms, division as much as 8 seconds. However, odo1242
pointed out that the outlook for LLMs may be more favorable, since their computations rely heavily on addition and multiplication, while branching, among the operations that FHE handles most poorly, is not entirely relevant.
User patters
offered another perspective on the performance question, noting that "1000x overhead would make some image classification tasks go from 1ms to 1s. That’s viable for some applications". Bringing the discussion back to the broader privacy trade-off, meindnoch
argued that "the most private AI is the one running on my own hardware, not in some giant data center" while Chris2048
similarly suggested that "when you want to keep a query/data private it's maybe worth the extra $ KW" required for a local deployment.
While Google includes benchmarking code in HEIR's repo, it has not provided any figures about its relative speed when applied to LLMs.