I’ve built a custom home surveillance system based around Frigate. Frigate does the person detection part and feeds Casa Segura for finer detection. It’s like a funnel for intrusion detection. Frigate’s algorithm is based on machine learning models such as TensorFlow Lite or OpenVINO, using models like MobileNet or YOLO for person detection. The system is urrently using Gemini-3.1-Flash-Lite which works well, but there is an problem with false positives, basically my dogs triggers the system detected as persons. I don’t want to hear an alarm at 3AM. That’s the main point on lowering the FP.
Recently Gemini-3.5-Flash-Lite was introduced, there is 50% bump in pricing, which I find it odd, I would expect that the same tier of model continue to have the same pricing. Maybe they improved the model a lot, and that’s the justification of the price bump. I’m a bit tired of going through reading every new card of a model introduction.
I was excited to test it because I’m waiting for a model that can lower the FP on zero shot. I’ve a small benchmark with clips from previous FP events, which are real world, currently the benchmark is managed by kimi k3 (it was build with Claude Opus 4.5 and succesors models).
18 camera clips, verified frame by frame by “hand” (10 without humans — dogs day/night, dog lying down, dog far away; 8 with humans, including person + dogs in scene).
When the model mistakes a dog for a person.
| Model | FP (30 runs) | Miss (24 runs) |
|---|---|---|
| gemini-3.1-flash-lite (production) | 0 | 0 |
| gemini-3.5-flash-lite (the “new and improved”) | 1 (dark dog, far away, at night; conf 0.90) | 0 |
| gemini-3.1-pro-preview (the “pro”) | 0 | 0 |
Gemini 3.1 Pro is pretty expensive. 3.1 is still 40% cheaper than 3.5.
| Model | Confidence | Price in/out ($/M) |
|---|---|---|
| gemini-3.1-flash-lite (production) | 0.95–1.00 | $0.25 / $1.50 |
| gemini-3.5-flash-lite (the “new and improved”) | 0.90–0.95 | $0.30 / $2.50 |
| gemini-3.1-pro-preview (the “pro”) | 0.95–1.00 | ~8× flash-lite |
Latency by model #
Gemini 3.1 and 3.5 are very similar, but 3.1 wins. Gemini 3.1 Pro, is really slow.
| Model | Latency | Cost (54 calls) | Verdict |
|---|---|---|---|
| gemini-3.1-flash-lite (production) | ~2.9 s (median) | ~$0.04 (est.) | Kept |
| gemini-3.5-flash-lite (the “new and improved”) | ~3 s | $0.059 (measured) | Discarded |
| gemini-3.1-pro-preview (the “pro”) | ~10 s (3.4×) | ~$0.30 (est.) | Discarded: same accuracy, 8× cost |
3 runs per clip, same prompt and transcode as production (6s, 2 fps, 768px), temperature 0.2.
According to Kimi: The single FP of the 3.5 came with confidence 0.90 — exactly the kind of tail hallucination that no model eliminates. Against that, a more expensive model doesn’t win; a second query is needed.
Sadly the new model was marginally worse than the one I already had (Gemini 3.1 Flash Lite); what fixed the false positives wasn’t switching models, but asking the same model to check again. I feel that is hackish, not perfect and adds latency.