{"slug": "show-hn-video-analysis-with-homomorphic-encryption", "title": "Show HN: Video Analysis with Homomorphic Encryption", "summary": "Belfort Labs released BlindSight, an open-source demo that runs a person-detection model on fully homomorphic encryption (FHE) ciphertext so a webcam stream is never decrypted server-side. The detector, PersonNet, is a four-layer convolutional network with about 60k parameters that classifies 64×64 images as containing a person near the camera, and it was trained on Harvard-Edge's Wake Vision dataset with \"near person\" labels derived from Open Images bounding boxes. The demo runs on Belfort's eML server, ships trained weights at model/training/runs/baseline/best.pt under Apache-2.0, and requires macOS 14 or newer on Apple Silicon, Linux (x86-64 or ARM), or Windows with WSL 2, plus uv and a C++20 compiler; its developer mode lets the gateway hold the key seed, which Belfort warns is \"for development only.", "body_md": "BlindSight detects presence of a person in webcam stream without ever exposing the video frames. Client displays the video, captures frames, and encrypts them using FHE (fully homomorphic encryption). Belfort eML server performs inference directly on these encrypted frames, and returns an encrypted verdict. End-to-end encrypted; the server never sees the images. Only the client holds the secret key and can decrypt the verdict.\n\nBlindsight is built with Belfort eML ([https://eml.belfortlabs.com](https://eml.belfortlabs.com)).\n\nWhat happens to one frame:\n\n1. `client.py` grabs the newest frame from the webcam it is showing.\n2. It converts the frame to RGB, centre-crops and resizes it to the model's input size, and encrypts it with the secret key.\n3. The deploy runs the model on the ciphertext and returns encrypted scores.\n4. `client.py` decrypts the scores and shows the verdict.\n\nCapturing, encrypting and decrypting all happen in the one process on your laptop, so the image never leaves it unencrypted.\n\nBefore the first frame can be classified, the deploy needs evaluation keys. These are derived from the same seed as the client's secret key, and there are two ways to get them onto the GPU:\n\n- **Developer mode** (`provider.py serve --dev` ), which this README uses: the gateway chooses\nthe seed and the GPU generates the evaluation keys from it. The provider hands the seed to`client.py` together with the client token, so the client uploads nothing and is ready\nwithin minutes.**For development only:** the gateway knows the seed, so Belfort could\ndecrypt everything the deploy serves.\n- **Your own keys** (`provider.py serve` ):`client.py` draws the seed itself, derives the\nsecret key and the evaluation keys from it, and uploads the evaluation keys. Only the client\never has the seed, which is the point of the demo, but the upload can take tens of minutes.\nStep 5 of[`eml/README.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/eml/README.md#5-your-own-keys-serve-without---dev) shows this\nflow.\n\nEither way the keys are made once per client token, not once per frame.\n\nThe detector is PersonNet, a small convolutional network: four layers, about 60k parameters,\nbuilt only from operations that are cheap under FHE. Given a 64×64 image, it says whether\nthere is a person close to the camera. It was trained on\n[Wake Vision](https://huggingface.co/datasets/Harvard-Edge/Wake-Vision), with \"near person\"\nlabels derived from Open Images' bounding boxes.\n\nThe trained weights, `model/training/runs/baseline/best.pt`, are in the repository, so you don't\nneed to train anything to run the demo. If you want to change the model or the data,\n[`model/README.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/model/README.md) explains how to retrain.\n\n| Path | What | \n|---|---|\n| [`eml/`](https://github.com/belfortlabs/blindsight-demo/blob/main/eml/README.md) | the Belfort ML example (eML is short for encrypted ML). `provider.py` compiles and serves the model.`client.py` encrypts images and decrypts verdicts: from the live webcam in a window (macOS, Linux and WSL 2), from sample frames (`--samples` ) or from your own images (`--images` ). | \n| [`model/`](https://github.com/belfortlabs/blindsight-demo/blob/main/model/README.md) | PersonNet: its preprocessing, training and plaintext inference, with the trained weights. | \n| [`dataset/`](https://github.com/belfortlabs/blindsight-demo/blob/main/dataset/README.md) | Wake Vision, fetched and labelled: the sample frames for the Belfort ML example, and the training data. | \n| [`INSTALL.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/INSTALL.md) | the installation steps for macOS, Linux and WSL 2. | \n| `assets/` | the architecture diagram: `architecture_diagram.drawio` and its SVG export, shown above. | \n| `LICENSE` ,`NOTICE` | Apache-2.0, and the attributions for the data the model was trained on (see *Licence* ). | \n\n- \n**macOS 14 or newer on Apple Silicon, Linux (x86-64 or ARM), or Windows with WSL 2.**\n- \n**uv and a C++20 compiler.** Every part of the repository runs with`uv run` , which sets up\nits environment the first time, and the Belfort ML client builds its encryption code with\nthe compiler the first time it runs.[`INSTALL.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/INSTALL.md) has the steps for\neach platform; on a Mac that is two commands, on Linux and WSL 2 a few more.\n- \n**A Belfort eML account and an API key.** Sign up on the Belfort eML console at[eml.belfortlabs.com](https://eml.belfortlabs.com) with your email address; the console emails\nyou a link to activate the account, and a new account comes with 10 credits, or 100 minutes\nof GPU time. Once signed in, open[API keys](https://eml.belfortlabs.com/api-keys) and create\na key. The console shows it (`blf_live_...` ) only once, so copy it then; an account can hold\nup to five active keys. Keep the key out of source code: the provider reads it from the`BELFORT_API_KEY` environment variable, which*Run it* below sets with`export` . The[eML documentation](https://eml.belfortlabs.com/docs) explains accounts, keys, deploys and\ncredits in more detail.\n- \n**Sample frames.** Two labelled sets drawn from Wake Vision. The calibration frames are what\nthe model provider calibrates the compiled model on, and the test frames are for checking it\nafterwards. One command downloads and prepares both (about 1.1 GB, a few minutes with a\nHugging Face login). You need nothing else from the dataset, because the weights are already\ntrained:\n\n``` php\ncd dataset\nuv run prepare_dataset.py                     # -> dataset/samples/test.npz, calibration.npz\ncd -                                          # back to the repository root\n```\n\n [`dataset/README.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/dataset/README.md) covers the Hugging Face login and the details.\n\nYou need two terminals, both started from the repository root.\n\n```\ncd eml\nexport BELFORT_API_KEY=blf_live_...           # your API key\nuv run provider.py compile                    # once per model; reused after\nuv run provider.py serve --dev                # development only: starts a GPU deploy that makes the keys\n```\n\n`compile` needs the calibration frames because under FHE every ReLU is replaced by a\npolynomial, and a polynomial is only accurate over a limited range. Running the 512 frames in\n`dataset/samples/calibration.npz` through the model tells the compiler what those ranges are.\nThe test frames that `client.py --samples` sends later come from a different split of the\ndataset (`dataset/samples/test.npz`), so the model is checked on frames it was not tuned on.\n\nIf `compile` fails with `name_taken`, something changed since the model was last compiled: the\nweights, the calibration frames, the compile options or the Belfort release. Run\n`uv run provider.py delete` and compile again.\n\n`serve --dev` starts a GPU deploy in developer mode: the gateway chooses the secret seed and\nthe GPU generates the evaluation keys from it. Once the deploy is ready, `serve` writes the\nclient token to `eml/.client-token` and the seed to `eml/.secret-seed-*` for terminal 2, and\nprints both as `export` lines in case the client runs elsewhere. **Developer mode is for\ndevelopment only**: the gateway knows the seed, so it could decrypt what the deploy serves. To\nkeep the keys to yourself, serve without `--dev`, as step 5 of\n[`eml/README.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/eml/README.md#5-your-own-keys-serve-without---dev) describes.\n\n**Expect 5 to 10 minutes before the deploy is ready.** The gateway starts a fresh GPU instance,\nand the GPU then generates the evaluation keys. The wait is not charged; credits count from the\nmoment the deploy is ready.\n\nLeave the terminal as it is while you use the client. Press Enter to stop the deploy; it stops by itself after 30 minutes anyway. You are charged credits for the time it ran, rounded up to 10 minutes.\n\nSteps 1 and 2 of [`eml/README.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/eml/README.md#1-compile) go into more detail, including\nthe compile options, and its [*Belfort release*](https://github.com/belfortlabs/blindsight-demo/blob/main/eml/README.md#belfort-release) section says\nwhich release the packages are pinned to.\n\n```\ncd eml\nuv run client.py                              # reads the client token terminal 1 wrote\n```\n\nIf the client runs on another machine, first run the two `export` lines that terminal 1\nprinted, `BELFORT_CLIENT_TOKEN` and `BELFORT_CLIENT_SECRET_SEED`.\n\n`client.py` starts by building its encryption code, which takes a few minutes the first time,\nthen makes its key pair from the seed terminal 1 handed over. The GPU already holds the\nevaluation keys, so nothing is uploaded, and the window opens once the client is ready to\nclassify.\n\n**Expect several seconds per frame.** Most of that is sending the ciphertexts between your\ncomputer and the server: the encrypted image going up and the encrypted verdict coming back. The\nencrypted inference, also called FHE evaluation, on the GPU only takes a few hundred\nmilliseconds. The status line shows both numbers, the round-trip time and the FHE evaluation\ntime, so you can see the split. In the\n[Blindsight blog post](https://belfortlabs.com/blog/encrypted-ai-for-aiot), Belfort has shown further\noptimizations that bring the whole round trip, from sending the encrypted image to receiving the\nencrypted verdict, under one second. Those optimizations are being integrated into eML step by\nstep, so the time per frame will keep dropping.\n\nThe menus let you pick a camera and a format. Any format will do; the frame is cropped and shrunk to the model's size anyway. The status line under the picture counts the frames classified on its left and shows the last verdict on its right: the class, the two scores, the round-trip time and the FHE evaluation time. Along the bottom of the picture, an animation follows each frame on its trip. On the left, the centre square that the network sees is snapped out of the picture and turns into noise as it is encrypted. In the middle it waits at the deploy, inside a black dashed box. On the right it is decrypted again and gets a coloured box, and that colour is the verdict: dark green means a person is near, dark red means nobody is, orange means the model is unsure, with the colours blending in between. The terminal prints the Belfort ML client's status line for every inference. While a frame is being classified, newer frames are dropped, so the next one sent is always the latest.\n\nOn macOS, the first run asks for camera access for your terminal app. If you said no, you can\nallow it later in System Settings > Privacy & Security > Camera. On WSL 2 the camera has to be\nattached first; see [`INSTALL.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/INSTALL.md#webcam-on-wsl-2).\n\nStep 3 of [`eml/README.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/eml/README.md#3-classify-a-live-webcam) describes the client and\nthe animation in more detail.\n\nYou can also check the compiled model without a camera: run `uv run client.py --samples` in\nterminal 2. It encrypts the 64 labelled test frames from `dataset/samples/test.npz`, has the\ndeploy classify them, decrypts the verdicts and compares each with its label. About 55 of 64\nshould be right, which is what the model scores in plaintext too. To classify your own photos instead, run\n`uv run client.py --images photo.jpg other.png`; any size works. Step 4 of\n[`eml/README.md`](https://github.com/belfortlabs/blindsight-demo/blob/main/eml/README.md#4-or-check-the-compiled-model-with-sample-frames-or-your-own-images)\nhas the details.\n\nClose the window, press Cmd-Q (macOS) or Ctrl-Q (Linux) in it, or press Ctrl-C in terminal 2;\nany of these stops `client.py` cleanly. Stopping the client does not stop the deploy. Press\nEnter in terminal 1 for that; the deploy costs credits for as long as it runs, up to 30\nminutes.\n\nThe code, the documentation and the trained weights in `model/training/runs/` are under the\n[Apache License 2.0](https://github.com/belfortlabs/blindsight-demo/blob/main/LICENSE). The weights were trained on\n[Wake Vision](https://huggingface.co/datasets/Harvard-Edge/Wake-Vision) (Harvard Edge Computing\nLab, CC BY 4.0), with labels derived from\n[Open Images](https://storage.googleapis.com/openimages/web/index.html) bounding-box\nannotations (Google, CC BY 4.0). [`NOTICE`](https://github.com/belfortlabs/blindsight-demo/blob/main/NOTICE) has the attributions and citations; keep\nit with the weights if you pass them on. No dataset images are in the repository: they are\nFlickr photographs, each under its own licence, and the scripts in `dataset/` download them\nfrom their publishers.", "url": "https://wpnews.pro/news/show-hn-video-analysis-with-homomorphic-encryption", "canonical_source": "https://github.com/belfortlabs/blindsight-demo", "published_at": "2026-11-10 09:12:08+00:00", "updated_at": "2026-11-10 09:22:59.102006+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "ai-safety", "ai-tools"], "entities": ["Belfort Labs", "BlindSight", "PersonNet", "Belfort eML", "Wake Vision", "Open Images", "Harvard-Edge"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-video-analysis-with-homomorphic-encryption", "markdown": "https://wpnews.pro/news/show-hn-video-analysis-with-homomorphic-encryption.md", "text": "https://wpnews.pro/news/show-hn-video-analysis-with-homomorphic-encryption.txt", "jsonld": "https://wpnews.pro/news/show-hn-video-analysis-with-homomorphic-encryption.jsonld"}}