Search the image and video topics in your recordings with a plain-language prompt.
Robotics teams spend their time on rare cases because that’s where the learning is. But the same rarity that makes these moments so valuable is also what makes them hard to retrieve. There isn’t an event type for a scenario nobody predicted, and your metadata cannot accurately describe what the camera saw. So, engineers end up scrubbing timelines and manually inspecting image topics.
Today, as part of our agentic data platform, we are announcing Semantic Search to solve this problem. It lets you search for objects or scenes in the image and video topics in your recordings, using plain-language text prompts. It comes with several other improvements to make the search experience more intuitive.
Semantic Search #
What can you do with Semantic Search?
On the search page, you can now search using a visual description instead of specifying exact metadata. Results come back in a gallery view, so you can preview them and open only the ones that matter.
You can also search for a scene that unfolds over time, not just a static object in a frame:
How does Semantic Search work?
Semantic Search is powered by vector search over frame embeddings. While ingesting data, Foxglove decodes your image and video topics, samples frames (a frame per topic per second), and runs them through an image embedding model. The vectors are stored in Lance tables in your object storage.
Your prompt goes through the same model, and results are ranked by cosine similarity. Structured conditions are applied first: device, time range, and topic narrow the candidate set, and then the vector search orders what’s left. Semantic Search understands two levels of context. Object and scene search finds matches within a single frame. Action search, powered by NVIDIA Cosmos open world models, reasons across sequences of frames to recognize motion and events that no single image can capture.
To learn more about the supported formats and encodings, see the Semantic Search docs.
Support for images and videos is the first step in Semantic Search. Our goal is to extend the same concept (the ability to search based on description) to all other data types that robots emit. Our vision is that robotics teams should spend less time becoming data mining experts and more time building autonomous robots.
Smart Search #
The new search experience also makes it easier to write structured queries with auto-complete in the search box. You can even combine structured and visual search criteria into a single query:
Sometimes, exploring data visually gives you ideas for conditions you want to search on elsewhere. You can now select aspects of your data in a visualization panel and turn them into search queries.
Agentic Search #
Like all other aspects of our platform, search is fully integrated with the Foxglove Agent. From within the search experience, you can write a plain-language request for what you want to search. Your request is routed to the agent, which converts the request into a proper query, performs the search, and returns the results. Here is an example of an agent-assisted search where a plain-language prompt does what previously required a complicated query:
Similarly, you can initiate the search in plain language from within the agent sidebar. The agent will understand the search intent and take you to the new search experience.
Try the new search experience #
Smart Search and Agentic Search are available today. To request access to Semantic Search, please get in touch with us.
Our engineers will demo these features live on September 2, 2026. Register now.