Google's HEIR Aims to Make Homomorphic-Encrypted Inference a One-Click Capability

Google has introduced HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler toolchain that enables pre-trained AI models to operate on encrypted data, aiming to make homomorphic-encrypted inference a one-click capability. The toolchain has been used by Google for private content recommendations, credit card fraud detection, network intrusion detection, and hotword recognition without exposing sensitive data. However, performance concerns remain, with Hacker News users noting high overheads (e.g., a 64-bit equality operation takes 80ms, division up to 8 seconds), though some argue that for certain applications like image classification, a 1000x overhead may still be viable.

Google is introducing HEIR Homomorphic Encryption Intermediate Representation https://blog.google/security/how-google-is-making-private-ai-practical-with-homomorphic-encryption/ , an open-source compiler and development toolchain designed to make encrypted computation easier to deploy. In particular, HEIR can compile pre-trained AI models built for conventional, unencrypted inputs so they can instead operate on encrypted data. According to Google homomorphic encryption https://github.com/google/fully-homomorphic-encryption HE addresses several challenges with end-to-end encryption, including enabling services such as spam and virus detection to operate on encrypted user data, mitigating the risk of exposing proprietary AI models when they are deployed on users' devices, and more. A solution to these issues is homomorphic encryption, a rapidly maturing technology that fundamentally alters this trade-off by allowing computations to be performed directly on encrypted data. Servers can process ciphertexts and return encrypted results without exposing any underlying information. In this context, HEIR https://github.com/google/heir is an open-source compiler toolchain and development platform for HE that can adapt pre-trained AI models designed to operate on unencrypted data to instead process encrypted inputs. Google has used HEIR to enable private content recommendations without exposing user data, detect credit card fraud while protecting sensitive financial information, identify network intrusions without revealing packet contents to the service provider, and recognize hotwords in audio streams without disclosing the underlying recordings. HEIR uses an intermediate representation as an abstraction layer to "represent and scale complex models across diverse dialects". According to Google, application developers can write a program in Python, annotate which data types should be encrypted, and then use HEIR to compile the code for homomorphic execution. Google provides a set of demos showcasing how pre-trained PyTorch models can be compiled to FHE https://github.com/google/fully-homomorphic-encryption/tree/main/demos readme . Currently the process is not yet automated, and involves a few steps, beginning with using torch mlir https://github.com/google/fully-homomorphic-encryption/tree/main/demos exporting-torch-to-mlir to export the PyTorch model to MLIR. One concern raised by several Hacker News readers is performance. Sabretooth1405 noted https://news.ycombinator.com/item?id=49301386 that "from my understanding HE and other techniques have very high overheads ~10^3 on inference tasks and thus aren't very commercially viable". dhx provided some concrete figures https://news.ycombinator.com/item?id=49302507 , noting that a 64 bit equality operation takes 80ms, addition and subtraction 100ms, division as much as 8 seconds. However, odo1242 pointed out that the outlook for LLMs may be more favorable https://news.ycombinator.com/item?id=49305109 , since their computations rely heavily on addition and multiplication, while branching, among the operations that FHE handles most poorly, is not entirely relevant. User patters offered another perspective https://news.ycombinator.com/item?id=49308478 on the performance question, noting that "1000x overhead would make some image classification tasks go from 1ms to 1s. That’s viable for some applications". Bringing the discussion back to the broader privacy trade-off, meindnoch argued https://news.ycombinator.com/item?id=49303443 that "the most private AI is the one running on my own hardware, not in some giant data center" while Chris2048 similarly suggested https://news.ycombinator.com/item?id=49309858 that "when you want to keep a query/data private it's maybe worth the extra $ KW" required for a local deployment. While Google includes benchmarking code in HEIR's repo https://github.com/google/heir/tree/main/benchmark , it has not provided any figures about its relative speed when applied to LLMs.