# From Exception Report to AI Agent: How MariaDB Combines SQL and Vector Search

> Source: <https://mariadb.com/resources/blog/from-exception-report-to-ai-agent-how-mariadb-combines-sql-and-vector-search/>
> Published: 2026-11-19 16:32:25+00:00

**Recommendation:** Keep the same embedding model for indexing and querying, since vectors from different models can't be compared, and template each exception into a search question before drafting the reason for human review.
More in the playbook "Retrieval and RAG": <https://wpnews.pro/playbooks/rag-retrieval>

## Key takeaways

- **Reviewers stop hunting for files.** An AI agent can pull the numbers behind a flagged exception, find the reason in contracts or filings, and cite both. Reviewers then check citations instead of searching documents by hand.
- **One database holds both your numbers and your documents.** MariaDB Cloud stores the structured financial data and the document knowledge base in the same service, so there’s no second vector database and no sync job between them.
- **People keep the decisions that matter.** The agent handles lookups and drafts the explanation, but a person still decides what to do with every exception.
- **You can start small, at no cost.** MariaDB AI Retrieval’s Public Preview includes 100 MCUs per organization each month. Loading 11 annual reports used about 61.

If you build business applications on MariaDB, you probably own a report like this one. Once a month, a SQL query lists the accounts, invoices, or borrowers that broke a business rule. For example, an invoice more than 90 days overdue or a borrower whose debt exceeds the limit in its loan agreement.

Finance teams call this an exception report. Each row on that list is an exception. The query tells you which rows broke the rule, but not why. The reason is in a document, such as a contract, an annual filing, or a support email, and a reviewer has to find it.

This article follows one exception report from manual review to a working agent on MariaDB and MariaDB Cloud AI Retrieval. The agent took the least effort to build. Most of the effort went into the data that the agent reads.

## From Tasks to a Workflow

Before any AI is involved, a person handles every exception. The reviewer pulls the numbers behind the flag, finds the documents for that period, works out the cause, and decides what to do with the exception: close it, call the customer, or escalate it. A month with many exceptions needs more reviewers, or the review finishes late. A late review delays the decisions it’s meant to support, such as calling a customer about an unpaid invoice.

The first application most teams build runs the same steps for every exception. The usual design is retrieval-augmented generation, or RAG. RAG gives a language model passages from your own documents, so that it answers from those passages instead of from what it learned in training. A RAG application works in two phases. It indexes the documents once ahead of time, then retrieves passages and generates an answer for each question.

In the indexing phase, code splits each document into short passages called chunks, usually a few paragraphs long. A text embedding model converts each chunk into an embedding, a vector of several hundred numbers that represents the text’s meaning. Chunks with similar meanings get vectors that are close together. The application stores each chunk’s text and vector in a database with a vector index that quickly finds the closest vectors.

In the query phase, the application converts the question into a vector with the same embedding model. It must be the same model, because vectors from different models can’t be compared. The search then returns the few chunks whose vectors are closest to the question’s vector. This is vector search, a type of semantic search, because it compares meanings instead of words. A question about “the change in net income” can find a paragraph about “lower earnings,” even though the two share no words. Finally, the application adds the retrieved chunks to the prompt alongside the question, and the language model generates the answer from them.

For the exception report, a fixed SQL query pulls the numbers, and a template turns each exception into a search question, such as “reasons for the change in net income in fiscal 2022.” A model drafts the reason from the numbers and the chunks, and a person reviews every draft.

Anthropic’s guide [Building effective agents](https://www.anthropic.com/research/building-effective-agents) calls an application like this one a workflow:

Workflows are systems where LLMs and tools are orchestrated through predefined code paths.

A workflow removes the clerical work from the review, so the reviewer spends time on decisions instead of on finding files. Its limit is the fixed search. Suppose a company’s profit fell because it took out a new loan, and its interest expense grew faster than its revenue. A search for “reasons for the change in net income” can miss the passage about the loan, and then the reviewer searches the documents by hand.

## From a Workflow to an Agent

An experienced reviewer reads the numbers first and uses them to decide which document to open. An agentic application lets the model make that decision. The same guide describes it this way:

Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.

The model gets tools instead of a fixed path. It decides what to look up, reads the result, chooses the next lookup, and stops when it has enough. Its answer cites a source for every number and every reason, so the reviewer checks citations instead of searching documents.

With each stage, the team gives the application more of the routine decisions. One decision stays with a person at every stage: what to do with an exception.

## The Application: One Company, Two Kinds of Data

The exception comes from the SEC filings of a small public company. The report rule flags any year in which a company’s operating income grew, but it still reported a net loss, and this company broke that rule in 2022 and 2023. The numbers show that something changed, but only a document explains what changed, so a good answer needs both sources.

I put both kinds of data in one [MariaDB Cloud service.](https://mariadb.com/cloud-get-started) I loaded 11 years of financial data into a schema named `financials`, and the company’s 11 annual reports into a knowledge base in [MariaDB AI Retrieval](https://mariadb.com/docs/mariadb-cloud/cloud-ai/ai-retrieval), a MariaDB Cloud service that parses documents, splits them into chunks, embeds each chunk, and searches them. AI Retrieval split the reports into 3,369 chunks and stored them next to the financial data, so the application needs no second database and no job to keep two databases in sync. If you prefer to build the search yourself, MariaDB Server has had a built-in `VECTOR` data type and vector index since version 11.7.

The agent has two tools. The `run_sql` tool runs one read-only `SELECT` statement on the financial data. The `search_documents` tool sends a query to AI Retrieval, which finds and ranks chunks in two stages.

The first stage is a hybrid search, which combines keyword search with vector search and returns the 25 best chunks. Vector search is fast because the embedding model encodes each chunk once, at indexing time, and encodes only the question at search time. An embedding model used this way is called a bi-encoder, because it encodes the question and the chunk separately. The cost of that speed is that the search compares the two texts only through their vectors, so a chunk that is loosely related to the question can still rank high.

The second stage reranks those 25 chunks with a reranking model, or reranker. A reranker is usually a cross-encoder, a model that reads the question and one chunk together and scores how well the chunk answers the question. That comparison is usually more accurate than comparing two vectors, but it’s too slow to run on every chunk in the knowledge base, so the reranker scores only the candidates from the first stage. The tool keeps the top 6.

Every result from either tool comes back with a short reference, called a ref, that the answer can cite. For example, `SQL 1` is the first query result, and `fy2022#322` is chunk 322 of the fiscal 2022 annual report.

Here is the core of the agent, condensed from the application’s Python code:

```
# The system prompt and the question start the conversationllm.ask(question)# One model call per step, up to 8 stepsfor step in range(MAX_STEPS):    # The model replies with tool calls, or with its answer    reply = llm.step()    # A reply without tool calls is the answer    if not reply["calls"]:        return reply["text"]    # Run each tool, and send the results back to the model    results = []    for call in reply["calls"]:        result = tools.run(call["name"], call["args"])        results.append((call, result))    llm.add_results(results)
```

The full loop is about 25 lines, and the lines I left out handle errors and progress updates. The application has no planner, router, or memory store. For the questions I asked, the model chose to make 2 to 5 tool calls. Because the model chooses its own path, you debug this application differently from a workflow. In a workflow, a wrong answer usually traces back to a line of code, such as the search template, and you fix that line. Here the model chose the search, so there’s no line to fix. Instead, you check what the model had to work with: the prompt, the tools, and the data.

## One Question, Two Sources

I asked the agent the question that a reviewer would ask about this exception: “The company’s operating income grew from 2021 to 2023, yet it reported net losses in 2022 and 2023. Why?”

In its first step, the model wrote the plan below and requested one SQL query and two searches. A 10-K is the annual report that a US public company files with the SEC.

I’ll pull operating income, interest expense, and net income for 2021–2023 from the SEC data, and search the 10-Ks for why the company lost money.

The query showed that interest expense grew from $2.11 million to $12.68 million in two years, and that from 2022 on, it exceeded operating income. That explains the losses, but the numbers don’t say why the interest expense grew. The two searches looked for that reason in the annual reports. In its second step, the model wrote this answer. Plain words in brackets replace names, dates, and amounts, and refs such as `[SQL 1]` are the model’s citations:

Interest expense rose much faster than operating income, and that turned operating profits into net losses.

**Operating income, interest and net result** [SQL 1]

| **Year** | **Operating income** | **Interest expense** | **Net income (loss)** | 
|---|---|---|---|
| 2021 | $6.09 million | $2.11 million | $2.95 million | 
| 2022 | $8.50 million | $10.37 million | $(2.48) million | 
| 2023 | $9.42 million | $12.68 million | $(2.53) million | 

- **Where the debt came from:** The company signed the [new] Credit Agreement [in late 2021]. It replaced cheaper loans […] and carried a higher interest rate, plus more amortization of loan fees [and other costs] [fy2021#339]. 2022 was the first full year of this loan. However, none of the passages I found explains the jump in 2022 interest specifically.
- **2023:** The 2023 10-K says interest expense rose $2.31 million because interest rates on its borrowings were much higher. Interest income of $0.85 million and an income tax expense of $0.11 million partly offset each other [fy2023#278]. The same report says 2022 had a tax benefit of $0.29 million, which made the 2022 loss smaller [fy2023#278].
- **What came next:** [Later,] the company paid off the [credit agreement] with a [new] term loan from [another bank] [fy2024#455]. The 2025 10-K says interest expense then fell [by more than half] [fy2025#308].

Every number and every reason cites its source. The first bullet also says which part the model couldn’t explain: why interest jumped in 2022. The prompt asks the model to report what it couldn’t find, and an answer that does is easier to trust.

Neither source could answer the question on its own. The financial data showed the trend without the reason, and no single report covered all three years. The model chose which numbers to pull and what to search for, as a reviewer would.

## Describe Your Data in the Prompt

Each decision that the model makes used to depend on what the reviewer knew. Before you let the model make a decision, you have to put that knowledge into the prompt and the data, and most of the work on this application went there.

If you build only one part of this application for your own data, build a description of that data. A model that sees only table and column names has to guess what the values mean, as a new analyst would on day one. The description fills about a third of this application’s system prompt and gives the model five kinds of information:

1. **The tables and their columns.** Name each table and view, list its columns, and say what one row holds. For example, the`annual_values` view holds 12-month values only.
2. **What coded values mean.** In the`period_months` column, 3 means a quarter, 6 or 9 means year to date, 12 means a full year, and`NULL` means a balance on one date.
3. **Rules for values that the data doesn’t store.** Companies report the fourth quarter only as part of the annual report, so the description says that the fourth quarter is the annual value minus the 9-month value.
4. **The stored name of each business term.** The financial data comes from XBRL, the tagged format of SEC filings, in which each measure has a standard name called a concept. Revenue is stored as the concept`RevenueFromContractWithCustomerExcludingAssessedTax` , a name that the model is unlikely to guess. Interest expense has two names in this data, depending on the year.
5. **Example queries.** Give one query for a common question, such as quarterly revenue for one year, and one that finds names the description leaves out, such as`SELECT concept, label FROM concepts WHERE label LIKE '%interest%'` .

To test the description, ask the questions that your users ask and read the SQL that the model writes. When a query is wrong, add the fact that the model was missing.

The `run_sql` tool also sets limits that the prompt can’t enforce. It accepts one `SELECT` statement, runs it in a read-only transaction with a 15-second limit set by the MariaDB `max_statement_time` variable, and returns at most 50 rows.

## Grounding the Answer

The agent saves the reviewer time only if its citations are correct. Grounding means tying an answer to sources the application supplies, such as SQL results and retrieved chunks, rather than to what the model learned in training. The system prompt asks the model to cite a ref for every number and every fact, and to state a cause only when a chunk states it. A citation only shows that the model named a source, though. The model can name a source that doesn’t contain the fact, or a ref that neither run_sql nor search_documents returned. So before the reviewer sees an answer, something has to test its grounding, and that test is often called a groundedness check.

I wrote the check in about 100 lines of Python that run after every answer, with no model call. It confirms that every ref came from a tool call in the same run. It also matches each dollar amount against the sources that its line cites, and it rejects a line that cites only SQL but states an amount from a chunk. When an answer fails, the loop sends the failures back to the model once, and the web page flags a second failure for the reviewer.

In one test run, the check caught a Claude Opus answer about operating margin that read well and gave correct revenue figures for each of the company’s regions. The answer cited [SQL 1] for those figures, but that query hadn’t returned any regional figures. The figures came from a chunk of an annual report, so a reviewer who opened the SQL result to confirm them would have found nothing there.

The check can’t tell whether a chunk supports the cause that it’s cited for. Most groundedness tools test that by asking a model whether each cited source supports each claim, and they report the share of supported claims as faithfulness. This check makes no model call, so that judgment stays with the reviewer.

## Agent Design Matters

The first bullet of the answer above says that none of the passages the model found explains why interest expense jumped in 2022. The annual reports do explain it. One chunk says that the new loan was large and that the company spent most of it on a legal settlement. The model searched for the loan and its refinancing, not for a settlement, so the answer missed how large the loan was and what it paid for. I ran the same question six times, and only one run found that chunk. Its search named the other party to the settlement before the model had seen any data.

The cause is in the first step. In the run above, the model requested its searches in the same step as the SQL query, before it had seen any numbers, so it had to guess what to search for from the question alone. That first guess determines whether the answer finds the whole cause.

So a team can let an agent make the lookups when a question needs about two steps, such as one query and one search. When a question needs a number before the agent can choose the next search, move that step back into the workflow as code. At every stage, a person decides what to do with each exception.

## Five Steps for Your First Agentic Application

The diagram below shows the finished application. Most of the work went into the parts around the agent, not into the agent itself.

Before you take a proof of concept like this one further, look at the task and sort the questions your structured data can answer from the ones only your documents can answer. Then follow these five steps:

1. **List the decisions that your reviewers make,** and move them to the application one at a time, starting with the lookups.
2. **Describe your data in the system prompt,** with the stored name of each business term and the rules that a new analyst would need.
3. **Make every tool return a ref that names the value your users ask about,** so the model and the reviewer can tell which source a fact came from. Here it’s the fiscal year. In your data, it might be a region, a product, or a contract.
4. **Limit SQL in the database session, not in the prompt.** A read-only transaction and a statement timeout apply even when the model ignores a rule.
5. **Check citations in code before a person sees them.** Start with the checks that need no model call.

The agent loop is about 25 lines. The quality of its answers depended on the financial data, the documents, and the rules around them, and MariaDB stored the data and the documents in one service.

## Build Your Own Document Knowledge Base

You don’t need an agent to start. Build a knowledge base from your own documents with the default settings. MariaDB AI Retrieval is in Public Preview, with no separate sign-up. You need a MariaDB Cloud account with a role above Viewer, because the Viewer role can’t create knowledge bases. In the MariaDB Cloud portal, AI Retrieval is in the AI & Automation section of the left navigation, and the [AI Retrieval quickstart](https://mariadb.com/docs/mariadb-cloud/cloud-ai/ai-retrieval/quickstart) shows how to create a knowledge base, upload a file, and run a first query.

You can try this on your own documents now. The Public Preview includes 100 MariaDB Compute Units (MCUs) per organization each month, and the 11 annual reports in this article took about 61 MCUs to load, leaving room to ask your knowledge base questions.

Pick the exception from last month’s report that took your reviewer the longest to explain. Upload the supporting documents, such as the contracts behind a flagged invoice, and ask the question your reviewer asked. If the results include the passage that your reviewer found by hand, the search works on your data, and you can add the SQL tool and the agent after it.

## Frequently Asked Questions

An exception report is a recurring query that lists records that broke a business rule, such as invoices more than 90 days overdue or borrowers over a loan limit. Each row is an exception. The query shows which records broke the rule, but the reason is usually in a document, such as a contract or filing, that a reviewer has to find.

A RAG workflow runs the same fixed steps for every question: it builds a search from a template, retrieves passages, and has a model draft an answer. An AI agent gets tools instead of a fixed path. The model decides what to look up, reads the result, and chooses its next step, much as an experienced reviewer would.

Yes. MariaDB Server has had a built-in VECTOR data type and vector index since version 11.7, so you can store embeddings and run similarity searches with SQL. MariaDB Cloud also offers AI Retrieval, a managed service that handles document parsing, chunking, embedding, and search for you.

[MariaDB AI Retrieval](https://mariadb.com/resources/blog/mariadb-cloud-ai-retrieval-managed-rag-over-documents-and-databases/) is a MariaDB Cloud service that turns your documents into a searchable knowledge base. It parses documents, splits them into chunks, embeds each chunk, and searches them using hybrid search and reranking. It stores the knowledge base alongside your relational data in the same MariaDB Cloud service.

Structured data shows what changed, and documents explain why. In this example, SQL showed that interest expense outgrew operating income, causing net losses, but only the annual reports explained the new loan behind it. Neither source could answer the question alone.

Hybrid search combines keyword search with vector search to find the 25 most relevant chunks quickly. A reranker, usually a cross-encoder, then reads the question and each candidate chunk together and scores how well it answers the question. This two-stage approach is more accurate than vector search alone and still fast enough for large knowledge bases.

Check citations in code before a person sees the answer. In this example, a check of about 100 lines of Python confirms that every cited reference came from a tool call in the same run, and that every dollar amount appears in the source it cites. Failed answers go back to the model once, and a second failure is flagged for the reviewer.

Enforce limits in the database session, not just in the prompt. Here, the SQL tool accepts only one SELECT statement, runs it in a read-only transaction, sets a 15-second limit with MariaDB’s max_statement_time variable, and returns at most 50 rows. These limits apply even if the model ignores its instructions.

Agents work well when a question needs about two steps, such as one query and one search. When the agent needs to see a number before it can choose the right search, move that step into the workflow as code. Otherwise, the agent may guess its search terms before seeing the data and miss part of the answer.

AI Retrieval is in [Public Preview in MariaDB Cloud](https://mariadb.com/products/cloud/), with no separate sign-up. You need a MariaDB Cloud account with a role above Viewer. Find it under AI & Automation in the portal, and use the quickstart to create a knowledge base, upload a file, and run a first query. The preview includes 100 MCUs per organization each month.
