{"slug": "introducing-semantic-search", "title": "Introducing Semantic Search", "summary": "Foxglove announced Semantic Search, a feature that lets robotics teams search image and video topics in recordings using plain-language text prompts, powered by vector search over frame embeddings and NVIDIA Cosmos open world models for action search. The feature is part of Foxglove's agentic data platform, with Smart Search and Agentic Search available today and Semantic Search available upon request. Foxglove will demo the features live on September 2, 2026.", "body_md": "# Introducing Semantic Search\n\nSearch the image and video topics in your recordings with a plain-language prompt.\n\nRobotics teams spend their time on rare cases because that’s where the learning is. But the same rarity that makes these moments so valuable is also what makes them hard to retrieve. There isn’t an event type for a scenario nobody predicted, and your metadata cannot accurately describe what the camera saw. So, engineers end up scrubbing timelines and manually inspecting image topics.\n\nToday, as part of our [agentic data platform](/blog/introducing-the-agentic-data-platform-for-physical-ai), we are announcing Semantic Search to solve this problem. It lets you search for objects or scenes in the image and video topics in your recordings, using plain-language text prompts. It comes with several other improvements to make the search experience more intuitive.\n\n## Semantic Search\n\n### What can you do with Semantic Search?\n\nOn the search page, you can now search using a visual description instead of specifying exact metadata. Results come back in a gallery view, so you can preview them and open only the ones that matter.\n\nYou can also search for a scene that unfolds over time, not just a static object in a frame:\n\n### How does Semantic Search work?\n\nSemantic Search is powered by vector search over frame embeddings. While ingesting data, Foxglove decodes your image and video topics, samples frames (a frame per topic per second), and runs them through an image embedding model. The vectors are stored in Lance tables in [your object storage](/blog/bring-your-own-storage-is-generally-available).\n\nYour prompt goes through the same model, and results are ranked by cosine similarity. Structured conditions are applied first: device, time range, and topic narrow the candidate set, and then the vector search orders what’s left. Semantic Search understands two levels of context. Object and scene search finds matches within a single frame. Action search, powered by NVIDIA Cosmos open world models, reasons across sequences of frames to recognize motion and events that no single image can capture.\n\nTo learn more about the supported formats and encodings, see the [Semantic Search docs](https://docs.foxglove.dev/docs/data/search#visual-search).\n\nSupport for images and videos is the first step in Semantic Search. Our goal is to extend the same concept (the ability to search based on description) to all other data types that robots emit. Our vision is that robotics teams should spend less time becoming data mining experts and more time building autonomous robots.\n\n## Smart Search\n\nThe new search experience also makes it easier to write structured queries with auto-complete in the search box. You can even combine structured and visual search criteria into a single query:\n\nSometimes, exploring data visually gives you ideas for conditions you want to search on elsewhere. You can now select aspects of your data in a visualization panel and turn them into search queries.\n\n## Agentic Search\n\nLike all other aspects of our platform, search is fully integrated with the [Foxglove Agent](/blog/foxglove-goes-agentic). From within the search experience, you can write a plain-language request for what you want to search. Your request is routed to the agent, which converts the request into a proper query, performs the search, and returns the results. Here is an example of an agent-assisted search where a plain-language prompt does what previously required a complicated query:\n\nSimilarly, you can initiate the search in plain language from within the agent sidebar. The agent will understand the search intent and take you to the new search experience.\n\n## Try the new search experience\n\nSmart Search and Agentic Search are available today. To request access to Semantic Search, please [get in touch with us](https://foxglove.dev/contact?reason=sales).\n\nOur engineers will demo these features live on September 2, 2026. [Register now](https://luma.com/foxglove-agentic-workflows).", "url": "https://wpnews.pro/news/introducing-semantic-search", "canonical_source": "https://foxglove.dev/blog/introducing-semantic-search", "published_at": "2026-08-18 06:41:58.716167+00:00", "updated_at": "2026-08-18 06:42:00.365838+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "ai-products", "ai-tools", "ai-infrastructure"], "entities": ["Foxglove", "NVIDIA Cosmos"], "alternates": {"html": "https://wpnews.pro/news/introducing-semantic-search", "markdown": "https://wpnews.pro/news/introducing-semantic-search.md", "text": "https://wpnews.pro/news/introducing-semantic-search.txt", "jsonld": "https://wpnews.pro/news/introducing-semantic-search.jsonld"}}