Hotword detection in FHE on GPU in 350ms

Belfort Labs launched a live demo at hotword.belfortlabs.com running a fully homomorphic encryption (FHE) hotword detector on its proprietary GPU platform, with server-side evaluation completing in 350ms, according to HEIR compiler creator Jeremy Kun. The round trip breaks down as 15ms for client-side encryption, 350ms for server-side evaluation, and 600ms for network latency, totaling under one second, though one-time setup encryption keys still take about a minute to generate in the browser and upload. The demo, compiled with the HEIR homomorphic encryption compiler, encrypts spoken audio client-side and returns an encrypted result that the client decrypts to update a word-connection game, and Belfort has opened an early access program for its FHE-based cloud platform.

A quick update on our homomorphic encryption compiler HEIR https://heir.dev : our partners at Belfort have a hotword detector in FHE that runs in 350ms. More specifically, they launched a live demo at hotword.belfortlabs.com https://hotword.belfortlabs.com/ , that has a little game inspired by the NY Times Connections puzzle. Instead of typing in your guess for the word that connects a group of clues, you say it out loud. Your audio is encrypted under FHE, transitted to their server, and a hotword detector runs on it to determine which word you said still encrypted . The client receives the encrypted result, decrypts it, and updates the game state. Not only was this demo compiled with HEIR to target Belfort’s proprietary GPU platform, but the inference step runs in 350ms. More specfically, the breakdown of runtime is: 15ms for client-side encryption, 350ms for server-side evaluation, 600ms for network latency my Portland-based laptop to Belfort’s servers which are likely in Belgium for a total round-trip of less than one second. The one caveat is that they pre-load the one-time setup encryption keys, which currently take a minute or so to generate in the browser and upload to the server. So if you want to play the game securely, you need that extra step. Of course, this could all be done locally, but it’s an impressive showcase of how fast FHE has become. And to my delight, how useful our compiler has become for compiling FHE workloads There’s still a lot more work to do, but this milestone makes me proud of the community of academics and practitioners we’ve grown around this compiler. Hooray for collaboration Belfort also has an early access program for their FHE-based cloud platform which you can sign up for here http://eml.belfortlabs.com/ . You can forgive me for the advertisement, but they intend to use HEIR as a primary tool. So if you use their platform, you’ll be helping find bugs and improvemenets in the HEIR compiler Want to respond? Send me an email mailto:mathintersectprogramming@gmail.com , post a webmention https://webmention.io/www.jeremykun.com/webmention , or find me elsewhere on the internet https://www.jeremykun.com/about/ . This article is syndicated on: