Meet the AI that skips the server bill #
Server bills for AI inference? Optional. Desert Ant Labs’ approach to on-device AI delivers small, specialized models run directly on your hardware. Instead of routing every audio or vision task through a large cloud model, these tiny, task-specific models execute locally, cutting latency and keeping sensitive data private.
This shift appeals to app builders and users alike. Developers minimize inference costs, while end-users benefit from near-instant processing and enhanced privacy. These beta models, part of Desert Ant Labs’ SDK v4, represent a new frontier for efficient, secure AI.
Meet the two new offerings: Who and Eye. Who handles speaker diarization, identifying who spoke when in audio tracks, down to 10-millisecond precision. It processes a 50-minute interview in about 3 seconds, operating at ~950x real-time on Apple hardware.
Eye provides on-device vision capabilities. It tags, scores, and sorts photos without cloud uploads, performing tasks like:
- Classifying images (screenshot, receipt, document)
- Tagging content across 558 categories
- Assessing aesthetic quality
- Identifying smiling faces or closed eyes
- Grouping near-duplicate shots and selecting the best
Neither model is a general-purpose assistant. Who doesn't transcribe speech or identify names, only speaker turns. Eye recognizes a receipt but doesn't perform OCR. They are complementary tools, optimized for specific, high-performance edge cases.
Who finds every voice—but not every name #
Desert Ant Labs’ Who model specializes in speaker diarization: it identifies which numbered speaker talked when, reporting turn boundaries to 10 milliseconds. Overlapping speech is attributed to both speakers. Crucially, Who does not transcribe audio or identify speakers by name.
Speed claims are aggressive. Desert Ant Labs clocks Who at roughly 950x real-time on an M5 Pro. A 57-minute NASA podcast processed in 6.4 seconds, including MP3 decoding. A video demo showed an 11.9-second run on an older M2 Max.
Benchmarks reveal nuanced performance. On the AMI meeting dataset, Who posted a 12.6% error rate, narrowly beating pyannote’s 13.1%. However, it struggles with earnings calls featuring up to 20 speakers, where its 27.7% error rate lags pyannote's 7.3%.
Who shines with close-mic audio from podcasts and calls. It’s less suited for crowded, far-field recordings. This specific model optimizes for defined use cases, not broad application.
Eye turns a messy photo folder into a library #
Eye model’s one-call photo analysis transforms messy folders into organized libraries. It classifies images as screenshots, receipts, documents, or photos. Tags across 558 categories provide granular context. It also scores aesthetics, flags faces with closed eyes or smiles, and groups near-duplicates to suggest the best shot.
Crucially, Eye recognizes a receipt but doesn't read its text—OCR is not part of this model. Confidence scores require thresholds: Desert Ant Labs recommends 0.5 for a positive match, or a stricter 0.9 for sensitive automatic actions like hiding private images.
Better Stack’s browser demo provided a reality check. Eye sorted 105 photos in 16.1 seconds, averaging about 6.5 images per second. While the demo surfaced useful categories, subjective judgments like blur and aesthetic scores can be inconsistent. This tiny model runs on your phone, not the cloud, making it a powerful local tool for photo management.
Enjoying this? Get one like it in your inbox each morning.
one email a day · unsubscribe in two clicks · no third-party tracking
Local AI still comes with caveats #
Local AI still has quirks. Eye runs in a browser using WebGPU, falling back to WebAssembly; Who pairs with a separate speech recognizer for speaker-labeled transcripts. Neither offers a fully automatic, set-it-and-forget-it workflow.
Processing media stays on-device, ensuring privacy. However, the source-available SDK performs a monthly license check. The free tier covers up to 100,000 monthly active devices—it’s not open source.
Who and Eye show promise for:
- Podcast workflows
- Meetings with dedicated microphones
- Photo library cleanup
Test on representative data before relying on them. Who struggles with large groups (20+ speakers) or far-field microphones; Eye's confidence scores require tuning for automatic sensitive-image handling.
Frequently Asked Questions #
What does Desert Ant Labs’ Who model do?
Who identifies when different speakers talk in an audio recording. It assigns numbered speakers and timestamps, but does not transcribe speech or identify people by name.
Can Eye read text on receipts?
No. Eye can classify an image as a receipt, but it does not perform OCR or extract the receipt’s text.
Do Who and Eye send recordings or photos to the cloud?
The models process media locally on supported devices or in the browser. The SDK does make a monthly license check, which the company says does not include audio or photos.
Are Desert Ant Labs’ Who and Eye models open source?
No. They use Desert Ant Labs’ source-available license, which offers free use up to 100,000 monthly active devices, subject to its terms.