{"slug": "search-is-how-agents-see-the-world", "title": "Search Is How Agents See the World", "summary": "Large language model agents rely on search as their primary interface to the world, making search freshness a correctness problem rather than a mere UX issue. If search results are stale, agents cannot accurately observe the consequences of their actions, leading to stalls, duplicate work, or incorrect inferences. Search documents are computed entities that aggregate fields like pricing, inventory, and ratings from multiple sources, and agents use both keyword and vector search to retrieve them.", "body_md": "# Search Is How Agents See the World\n\nJune 29, 2026\n\n## Search maintenance used to be a UX problem. For agents, it is a correctness problem.\n\nLarge language models reason in language. So the natural way for an agent to ask the world for information is in words: describe the thing it needs, then search for it.\n\nKeyword search and vector search are both versions of this. Keyword search lets the agent ask by exact terms: this order ID, this SKU, this customer name. Vector search lets the agent ask by meaning: products like this, tickets related to this failure mode, policies relevant to this request.\n\nThat makes search the agent’s interface to the world.\n\nIf search results are stale, humans can compensate. If a result looks wrong, they can refresh the page, check the source system, ask a coworker, or use judgment. Stale search may be frustrating, but the human still has ways to recover.\n\nAn autonomous agent operates in a tighter loop. It uses search as its eyes into the world: it looks up context, decides what to do, takes an action, and then searches again to observe what changed. That final search is important because it enables the agent to reason about the consequences of its action. Is it closer to its goal? Or should it change course? Search is how it finds out.\n\nThat loop only works if search reflects the current state of the world. If the agent updates a price, changes an order, opens a ticket, grants a permission, or writes a row, it needs to see the consequence of that action before it can decide what to do next.\n\nIf it cannot see the consequence, it has bad options. It can stall because it cannot confirm progress. It can retry and risk doing the same thing twice. Or it can infer that something happened because it expected it to happen, not because it observed it.\n\nFor a human, stale search is often annoying. For an agent, it's game breaking.\n\n## The agent sees computed entities\n\nImagine an agent that helps manage a commerce catalog.\n\nIt can place orders, adjust pricing based on demand or policy, resolve inventory issues, reorganize products into collections, and take corrective actions without human intervention. To do that work, it searches the catalog by exact terms and by meaning. It might look up a product by SKU, or it might search for “cozy gear for a snowy cabin trip.”\n\nThe agent is not searching raw tables. It is searching product-shaped documents.\n\n`1` | \n\n```\n{\n```\n\n |\n`2` | \n\n```\n  \"id\": \"product_123\",\n```\n\n |\n`3` | \n\n```\n  \"name\": \"Cast Iron Camp Skillet\",\n```\n\n |\n`4` | \n\n```\n  \"description\": \"A durable skillet for open-fire cooking.\",\n```\n\n |\n`5` | \n\n```\n  \"brand\": \"North Trail\",\n```\n\n |\n`6` | \n\n```\n  \"price\": 42,\n```\n\n |\n`7` | \n\n```\n  \"in_stock\": true,\n```\n\n |\n`8` | \n\n```\n  \"avg_rating\": 4.7,\n```\n\n |\n`9` | \n\n```\n  \"review_count\": 128,\n```\n\n |\n`10` | \n\n```\n  \"collections\": \"Winter Cabin Getaway: warm, rugged gear for snowy trips\",\n```\n\n |\n`11` | \n\n```\n  \"search_blob\": \"Cast Iron Camp Skillet A durable skillet for open-fire cooking. Winter Cabin Getaway: warm, rugged gear for snowy trips\",\n```\n\n |\n`12` | \n\n```\n  \"embedding\": [0.12, -0.08, 0.44, \"...\"]\n```\n\n |\n`13` | \n\n```\n}\n```\n\n |\n\nThis document is what the agent sees. It may look simple, but it is the product of a lot of computation.\n\nSearch documents are usually built around derived core entities: a product, customer, order, ticket, policy, or account in the shape an application or agent needs to retrieve. Each document gathers the fields that describe the entity, but it also resolves the logic that makes those fields meaningful.\n\nFor a product, `avg_rating`\n\nis computed from reviews. `price`\n\nmay reflect base price, promotions, customer segment rules, region, contract terms, and availability. `in_stock`\n\nmay depend on warehouse inventory, reservations, supplier feeds, and fulfillment rules. `collections`\n\nmay combine merchandising data with product relationships. `search_blob`\n\nis intentionally composed from the fields that should affect semantic retrieval. The embedding is then derived from that text.\n\nThe result is a computed entity: a product-shaped record assembled from operational data and business logic.\n\nThat shape is right for search. [Elastic’s own performance guidance](https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/search-speed#_document_modeling) says documents should be modeled to make search-time operations cheap, and that avoiding joins by denormalizing documents can produce significant speedups. Search systems are fast because they search this kind of precomputed shape.\n\nBut those documents only look simple because the computation has already happened. The hard part is keeping each computed entity correct as the underlying systems change.\n\nA useful mental model looks like this:\n\n`1` | \n\n```\nproducts ───────┐\n```\n\n |\n`2` | \n\n```\nbrands ─────────┤\n```\n\n |\n`3` | \n\n```\ninventory ──────┤\n```\n\n |\n`4` | \n\n```\nreviews ────────┤──> product_doc ───────────────> keyword index\n```\n\n |\n`5` | \n\n```\ncollections ────┘        │\n```\n\n |\n`6` | \n\n```\n                         └──> embedding text ──> vector store\n```\n\n |\n\nThe index is not the source of truth. The source systems are. But the thing search needs is the computed entity: the final product-shaped record after joins, aggregates, rollups, pricing logic, permissions, and embedding text have all been resolved.\n\nThat is where search maintenance becomes hard.\n\n## The hard part is maintaining the computed entity\n\nPutting a document into a search index is not the hard part. The hard part is keeping the right computed entity current when the data and business logic behind it change. And that's because the thing that changed is not always the thing you index.\n\nSearch indexes store product documents, while operational systems update catalog rows, inventory, reviews, collections, permissions, and more. The challenge is translating those source-level changes into updates to the computed entity.\n\nA single change can ripple through the system in different ways. Updating a description affects one product’s text and embedding. Changing a price rule may update prices across many products without touching embeddings. A new review shifts a product’s rating, while a collection edit can update search text for every product it contains. Some changes are localized; others fan out across large portions of the dataset.\n\nThese fan-out cases are the hardest. A collection description change can require re-embedding every associated product. A permission update can alter which documents are visible without modifying any product rows. Pricing logic changes may depend on region, customer segment, inventory, or promotions, affecting many entities at once.\n\nWhat matters is not where the change originates, but how it affects the computed entity. This makes the problem one of view maintenance, not simple synchronization.\n\nCDC captures input-level changes: a row updated in `collections`\n\n, a new review, an inventory adjustment. But search needs to understand the output-level impact. Which product documents changed? Which fields within them? Did the embedding text change, or only a filterable attribute?\n\nCDC tells you what happened to an input. It can tell you that a row changed in `collections`\n\n, or that a row changed in `reviews`\n\n, or that an inventory record was updated. But the search system needs to know the output-level consequence.\n\nWhich product documents changed? Which fields changed inside those documents? Did the text used for embedding change? Did only a filterable attribute change? Should the vector be regenerated, preserved, or deleted?\n\nThose are entity-level questions.\n\nThis is where most approaches start to break down. The system has to translate low-level source changes into precise updates to computed entities, and that translation is where complexity accumulates.\n\n## Batch and DIY streaming both fall short\n\nOne common approach is to periodically recompute the index in batch. Reprocess large swaths of data, doing more work than necessary but guaranteeing you process anything that may have changed. They eventually produce the correct index, but introduce lag between the operational system and what search reflects.\n\nThe natural next step is to try to make things more real-time by building a pipeline by hand, whether that’s a streaming system or a set of microservices reacting to changes.\n\nThat sounds better. Source changes flow through CDC or APIs. Services pick up updates. Workers patch the index. Embedding jobs refresh vectors. With enough code, the index can become fresher.\n\nBut now the application owns the hard part.\n\nIt has to join source changes into computed entities. It has to maintain aggregates. It has to apply pricing logic. It has to handle fan-out. It has to decide which document fields changed. It has to know when to call the embedding model and when to preserve the existing vector. It has to handle deletes, retries, ordering, schema changes, backfills, and operational failures.\n\nIn other words, the team has built a custom view-maintenance system.\n\nThat system is usually brittle because the logic is spread across services. One service knows about source events. Another knows about entity assembly. Another knows about pricing or permissions. Another knows about embeddings. Another writes to the index. The rules that define “what the agent sees” are no longer in one place.\n\nThis is why batch and raw CDC both miss the core problem.\n\nBatch is too stale. Raw CDC is too low-level. And building your own pipelines means solving a complex problem from scratch. What you need is a way to define the computed entity once, then have the system keep it current as the source data changes.\n\n## Define the computed entity once, as SQL\n\nMaterialize lets you define the search document as SQL and keep it current as source systems change.\n\nThe important idea is simple: the search document should not be a batch artifact or a pile of application code. It should be a maintained view of a computed entity.\n\nStart with the pieces. Reviews become a product-level aggregate:\n\n`1` | \n\n```\nCREATE VIEW reviews_agg AS\n```\n\n |\n`2` | \n\n```\nSELECT\n```\n\n |\n`3` | \n\n```\n  product_id,\n```\n\n |\n`4` | \n\n```\n  AVG(stars) AS avg_rating,\n```\n\n |\n`5` | \n\n```\n  COUNT(*) AS review_count\n```\n\n |\n`6` | \n\n```\nFROM reviews\n```\n\n |\n`7` | \n\n```\nGROUP BY product_id;\n```\n\n |\n\nCollections become a product-level rollup:\n\n`1` | \n\n```\nCREATE VIEW collection_text AS\n```\n\n |\n`2` | \n\n```\nSELECT\n```\n\n |\n`3` | \n\n```\n  pc.product_id,\n```\n\n |\n`4` | \n\n```\n  string_agg(c.name || ': ' || c.description, ' | ') AS text\n```\n\n |\n`5` | \n\n```\nFROM product_collections pc\n```\n\n |\n`6` | \n\n```\nJOIN collections c\n```\n\n |\n`7` | \n\n```\n  ON c.collection_id = pc.collection_id\n```\n\n |\n`8` | \n\n```\nGROUP BY pc.product_id;\n```\n\n |\n\nThen those pieces become the final product document:\n\n`1` | \n\n```\nCREATE MATERIALIZED VIEW product_doc AS\n```\n\n |\n`2` | \n\n```\nSELECT\n```\n\n |\n`3` | \n\n```\n  p.product_id,\n```\n\n |\n`4` | \n\n```\n  p.name,\n```\n\n |\n`5` | \n\n```\n  p.description,\n```\n\n |\n`6` | \n\n```\n  b.name AS brand,\n```\n\n |\n`7` | \n\n```\n  inv.price,\n```\n\n |\n`8` | \n\n```\n  inv.in_stock,\n```\n\n |\n`9` | \n\n```\n  rev.avg_rating,\n```\n\n |\n`10` | \n\n```\n  rev.review_count,\n```\n\n |\n`11` | \n\n```\n  col.text AS collections,\n```\n\n |\n`12` | \n\n```\n  CONCAT_WS(' ', p.name, p.description, col.text) AS search_blob\n```\n\n |\n`13` | \n\n```\nFROM products p\n```\n\n |\n`14` | \n\n```\nJOIN brands b\n```\n\n |\n`15` | \n\n```\n  ON b.brand_id = p.brand_id\n```\n\n |\n`16` | \n\n```\nJOIN inventory inv\n```\n\n |\n`17` | \n\n```\n  ON inv.product_id = p.product_id\n```\n\n |\n`18` | \n\n```\nLEFT JOIN reviews_agg rev\n```\n\n |\n`19` | \n\n```\n  ON rev.product_id = p.product_id\n```\n\n |\n`20` | \n\n```\nLEFT JOIN collection_text col\n```\n\n |\n`21` | \n\n```\n  ON col.product_id = p.product_id;\n```\n\n |\n\nThis view is the product document. It joins catalog, brand, inventory, reviews, and collections into one product-shaped record that the agent can search. In a real system, this view might also include pricing rules, permissions, regional availability, personalization features, or any other business logic needed to produce the entity the agent should see.\n\nThe important part is not only that the transformation is written in SQL. It is that Materialize maintains the result.\n\nWhen inputs change, Materialize updates only the affected `product_doc`\n\nrows. A new review updates one product’s rating. A promotion updates prices for the relevant products. A collection edit updates search text for its products.\n\nDownstream systems don’t need to recompute relationships or figure out what changed. Materialize converts source changes into entity-level updates and the search document is continuously maintained.\n\n## Send entity-level changes downstream\n\nOnce Materialize maintains `product_doc`\n\n, the search index does not need to understand the whole operational graph or how the entity is computed. It only needs to consume changes to the computed entity.\n\nIf a price changes, the downstream update can look like this:\n\n`1` | \n\n```\n{\n```\n\n |\n`2` | \n\n```\n  \"before\": {\n```\n\n |\n`3` | \n\n```\n    \"id\": \"product_123\",\n```\n\n |\n`4` | \n\n```\n    \"price\": 42,\n```\n\n |\n`5` | \n\n```\n    \"search_blob\": \"Cast Iron Camp Skillet ... Winter Cabin Getaway: warm, rugged gear for snowy trips\"\n```\n\n |\n`6` | \n\n```\n  },\n```\n\n |\n`7` | \n\n```\n  \"after\": {\n```\n\n |\n`8` | \n\n```\n    \"id\": \"product_123\",\n```\n\n |\n`9` | \n\n```\n    \"price\": 39,\n```\n\n |\n`10` | \n\n```\n    \"search_blob\": \"Cast Iron Camp Skillet ... Winter Cabin Getaway: warm, rugged gear for snowy trips\"\n```\n\n |\n`11` | \n\n```\n  }\n```\n\n |\n`12` | \n\n```\n}\n```\n\n |\n\nThe price changed. The embedded text did not. The index should update the price, but it should not call the embedding model.\n\nIf the description changes, the update looks different:\n\n`1` | \n\n```\n{\n```\n\n |\n`2` | \n\n```\n  \"before\": {\n```\n\n |\n`3` | \n\n```\n    \"id\": \"product_123\",\n```\n\n |\n`4` | \n\n```\n    \"price\": 39,\n```\n\n |\n`5` | \n\n```\n    \"search_blob\": \"Cast Iron Camp Skillet A durable skillet for open-fire cooking...\"\n```\n\n |\n`6` | \n\n```\n  },\n```\n\n |\n`7` | \n\n```\n  \"after\": {\n```\n\n |\n`8` | \n\n```\n    \"id\": \"product_123\",\n```\n\n |\n`9` | \n\n```\n    \"price\": 39,\n```\n\n |\n`10` | \n\n```\n    \"search_blob\": \"Cast Iron Camp Skillet A lightweight skillet for open-fire cooking...\"\n```\n\n |\n`11` | \n\n```\n  }\n```\n\n |\n`12` | \n\n```\n}\n```\n\n |\n\nNow the embedded text changed. The vector has to change too.\n\nBecause Materialize produces full before-and-after images of the computed entity, the downstream pipeline can trivially determine what changed. Materialize has already answered the hard question of which entities changed, so the search system only needs to answer the local question of what fields changed inside this entity.\n\nThat distinction keeps the architecture simple. The search system does not need to understand joins, aggregates, pricing logic, or fan-out relationships. It only reacts to changes in the final entity shape.\n\nIt also enables precise embedding updates. The pipeline can compare the fields that feed the embedding and act accordingly:\n\n- If\n`price`\n\nchanged, update the price. - If\n`in_stock`\n\nchanged, update availability. - If\n`avg_rating`\n\nchanged, update the rating. - If\n`search_blob`\n\nchanged, regenerate the embedding. - If nothing relevant changed, do nothing.\n\nAt this point, the remaining problem is mechanical: compare before and after, and only re-embed when the text that drives meaning actually changed.\n\nWe already ship this as a small, focused piece of infrastructure. [Perfect Embedding](https://github.com/materializeinc/perfect-embedding) is a Kafka Connect Single Message Transform (SMT) that sits in the pipeline between Materialize and your search index. It inspects the before-and-after images of each document, checks the fields you’ve configured as embedding inputs, and only calls the embedding service when those fields change.\n\nOut of the box, it integrates with OpenAI-compatible embedding APIs, but the interface is pluggable so you can use whatever embedding provider you prefer.\n\n## Build search your agents can trust\n\nAgents do not need a stale copy of operational data. They need a maintained interface to your business.\n\nThat interface is the search document: the product, ticket, customer, policy, or order shape the agent retrieves before it acts. If that document lags behind the source system, the agent reasons from a stale model. If the document stays current, the agent can observe the result of its actions and keep going.\n\nMaterialize maintains that document as SQL. It turns source-level changes into entity-level changes, including joins, aggregates, rollups, fan-out updates, business logic, and deletes. The search index receives the changes it needs. [Perfect Embedding](https://github.com/materializeinc/perfect-embedding) ensures embeddings stay in sync without unnecessary recomputation.\n\nThe result is a search layer that moves with the operational system instead of catching up to it later.\n\nMaterialize has built a solution around these ideas, making it easy to define, maintain, and serve real-time computed entities for search and beyond. If you're building agents, or any system that depends on fresh, correct search, [check out our website ](/)to see how it works and get started.", "url": "https://wpnews.pro/news/search-is-how-agents-see-the-world", "canonical_source": "https://materialize.com/blog/agent-search/", "published_at": "2026-06-29 00:00:00+00:00", "updated_at": "2026-06-30 16:34:56.767740+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-products"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/search-is-how-agents-see-the-world", "markdown": "https://wpnews.pro/news/search-is-how-agents-see-the-world.md", "text": "https://wpnews.pro/news/search-is-how-agents-see-the-world.txt", "jsonld": "https://wpnews.pro/news/search-is-how-agents-see-the-world.jsonld"}}