{"slug": "wildlens-an-offline-field-notebook-that-tells-you-what-you-just-found-outside", "title": "WildLens: an offline field notebook that tells you what you just found outside", "summary": "A developer built WildLens, an offline-first progressive web app that identifies plants, birds, animals and insects from a phone photo entirely on-device. The app runs the open-weight BioCLIP vision model exported to ONNX and executed in the browser via ONNX Runtime Web (WebAssembly), comparing image embeddings against precomputed text embeddings for species looked up on GBIF, with no server or API key. It reports its top three guesses rather than a single answer, flags hazardous finds, and stores observations in a local IndexedDB field journal.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass](https://dev.to/challenges/hacktoberfest-week1-2026-10-05)*\n\nWildLens is a small field notebook that lives in your phone's browser. Point the camera at a plant, bird, animal or insect and it tells you what it thinks you're looking at. The identification runs entirely on your device, so your photos are never uploaded, and once the model has downloaded the app is built to keep working with no signal.\n\nI'm a developer, so I'm well aware of the irony of building an app to get people off screens. That's why I wanted the screen to be the shortest part of the experience: you spot something on a walk, take a photo, get a name in a few seconds, and put the phone back in your pocket. Every result also comes with a short \"Look closer\" prompt, a small observation task that turns the photo into a reason to keep looking at the real thing.\n\nI didn't want it to bluff, either. When WildLens isn't sure, it says so. It shows its top three guesses instead of one confident-sounding answer, and if it can't find a living subject in the photo, it tells you that. Anything hazardous gets a safety note, and every plant carries one fixed rule: never eat anything based on a photo.\n\nWhat you find can be saved to a local field journal with your own notes and, if you choose, an approximate location. It stays on your device, and you can export or clear it whenever you like.\n\nI gave it a quiet look on purpose: a parchment background, muted greens, a serif for scientific names, no gradients. It should feel like a notebook, not another app begging for attention.\n\nIt's for anyone who has stood on a trail or in a garden and wondered \"what is that?\"\n\nLive app: [https://wildlens-57g9.onrender.com/](https://wildlens-57g9.onrender.com/)\n\nIt works best on a phone. Open it, allow camera access, and on first launch let it download the model (about 330 MB) over Wi-Fi. That's a one-time download, and the model is kept in your browser's cache afterwards.\n\n**How it went outside:** \n\nI tried to identify the plants in my garden and a sweet little cat that always comes to my garden. Then, when I went outside, I tried to identify the flowers that I saw there and kept them in my journal, it worked quite nicely and was amazing.\n\nPoint your phone at a plant, bird, animal or insect and find out what it is. An open-weight vision model runs inside your browser, so your photos never leave your device.\n\nWildLens is a mobile-first, offline-first PWA with a calm \"field notebook\" look. It is built for the DEV \"Touch Grass\" challenge: an excuse to go outside, look closely at what is living around you, and keep notes.\n\nWildLens is a React, TypeScript, Vite and Tailwind app, installable as a PWA, with the journal stored in IndexedDB.\n\nThe AI at its core is **BioCLIP**, an open-weight vision model from the Imageomics Institute (MIT licensed) that was trained on biodiversity images across the tree of life. I exported its image encoder to ONNX once, in Google Colab, and the app runs it in the browser with ONNX Runtime Web (WebAssembly). There's no server and no API key.\n\nIdentification works like this. The photo is center-cropped and normalised, the encoder turns it into a vector, and that vector is compared with precomputed vectors for each species. BioCLIP responds best to full taxonomic names, so a small build script looks up every species on GBIF, and I embed \"a photo of\" plus the taxonomic string. Those text embeddings ship as a small file, so the phone only ever runs the image half of the model. A softmax turns the similarities into probabilities, and a set of \"not nature\" prompts catches photos that shouldn't become a species (a living room should never be reported as a gecko). When no single species wins clearly, the scoring adds up probabilities by genus, family, order and class, so it can lean on \"probably this family\" instead of a wrong species.\n\nThe model file is far too big for GitHub, so it lives in a Hugging Face model repo. The app downloads it once and keeps it in the browser cache. The app itself is a static site on Render.\n\nThe first version didn't work well. I started with OpenAI's CLIP (ViT-B/32, 8-bit) through Transformers.js, and it was wrong almost every time. I grew the label list, rewrote the prompts and tried prompt ensembling to separate a cat from a cheetah. Each tweak helped a little, but none fixed the real problem: CLIP is a general-purpose model, and BioCLIP actually knows biology.\n\nWhat saved me was keeping the whole UI behind one small `NatureIdentifier` interface. Switching models meant adding a new identifier class and updating the scoring, with no changes to any screen. After the swap, the answers finally started to make sense.\n\nIt's honest about its limits. It knows a few hundred species, not every organism on earth. Look-alikes (small brown birds, frogs, grasses) are hard for any model. And it's a notebook companion, not a safety tool.\n\n**Credits:** BioCLIP (Imageomics Institute), OpenCLIP, ONNX Runtime Web, GBIF for taxonomy, and Hugging Face for hosting the model files.\n\nI started with the trail. If you're hiking with no signal, a cloud vision API is just a spinner. Because the model weights are open, I could put the model on the phone itself: no connection needed, no per-photo cost, no API key to leak and no rate limit to hit.\n\nPrivacy mattered just as much. A wildlife photo carries a time and often a place, which is exactly the kind of data I didn't want sitting on a server I don't control. With a local model, the photo never leaves the device.\n\nOpen also meant I wasn't locked in when my first choice turned out to be wrong. With a closed API, \"this model isn't good at biology\" is a support ticket. With open weights it was a change I could make myself. Species are just text too, so adding a bird means adding a line, not retraining anything.\n\nWildLens is mostly other people's open work, glued together carefully: BioCLIP, ONNX Runtime, GBIF's taxonomy, Hugging Face's hosting. A closed API would have given me a black box that needed internet. Open innovation gave me a notebook that works at the edge of the trail.\n\nI used an AI coding assistant throughout, for scaffolding, for debugging the in-browser WebAssembly setup, and for the one-time model export. The decisions about what to build and what to keep were mine, and I tested the results myself.\n\n**Best Use of Render:** WildLens is deployed on Render as a static site, which serves the PWA front end. The model itself runs on the user's device.\n\nThanks for reading. If you try it, I'd love to hear what you found out there. Now go outside.", "url": "https://wpnews.pro/news/wildlens-an-offline-field-notebook-that-tells-you-what-you-just-found-outside", "canonical_source": "https://dev.to/aditi_2609/wildlens-an-offline-field-notebook-that-tells-you-what-you-just-found-outside-3j57", "published_at": "2026-10-10 04:25:41+00:00", "updated_at": "2026-10-10 04:30:37.519587+00:00", "lang": "en", "topics": ["computer-vision", "ai-tools", "ai-products", "developer-tools", "artificial-intelligence"], "entities": ["WildLens", "BioCLIP", "Imageomics Institute", "ONNX Runtime Web", "GBIF", "Hacktoberfest", "React", "IndexedDB"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/wildlens-an-offline-field-notebook-that-tells-you-what-you-just-found-outside", "markdown": "https://wpnews.pro/news/wildlens-an-offline-field-notebook-that-tells-you-what-you-just-found-outside.md", "text": "https://wpnews.pro/news/wildlens-an-offline-field-notebook-that-tells-you-what-you-just-found-outside.txt", "jsonld": "https://wpnews.pro/news/wildlens-an-offline-field-notebook-that-tells-you-what-you-just-found-outside.jsonld"}}