{"slug": "choosing-the-right-database-for-ai-agents-llm-generated-sql", "title": "Choosing the Right Database for AI Agents: LLM Generated SQL", "summary": "Jai, writing on the Substack blog, argues that configurable AI agents that let end users upload their own structured data need databases supporting flexible schemas, multi-tenancy, and reliable LLM-generated SQL, and that while PostgreSQL's JSONB offers flexibility, it complicates LLM-generated SQL because models are more reliable with ordinary column-based SQL than with JSONB path expressions and type casts.", "body_md": "- Published on\n\n# Choosing the Right Database for AI Agents: LLM Generated SQL\n\n- Authors\n- Name\n- Jai\n[@jkntji](https://twitter.com/jkntji)\n\nAI agents are rapidly evolving from simple chatbots into configurable systems that can adapt to different domains and data sources. One particularly interesting use case is an agent that users can configure with their own structured data.\n\nAn agent might work with store inventory data for one user and customer records, product catalogs, or entirely different datasets for another. In these systems, the structure of the data is not defined by the developer in advance; it is defined by the end user at runtime.\n\nThis creates a unique set of challenges for the underlying database. The system needs to support flexible schemas, strong multi-tenancy so that users' data remains isolated, easy ingestion and querying of files such as CSVs, and, increasingly, reliable LLM-generated SQL.\n\nIn this article, we'll explore these requirements and look at the database architectures that make the most sense for configurable AI agents.\n\nThe Core Requirements of Configurable Agents\n\nUnlike traditional applications, where developers define the database schema, configurable agents need to work with structured data provided by the end user. This creates several important requirements:\n\n**Schema flexibility:** Different instances of the same agent should be able to work with completely different data structures without requiring application code changes or migrations.**Queryability:** The data must remain fully searchable and filterable so the agent can answer questions or take actions based on it.**Multi-tenancy:** Data belonging to different users or organizations must remain strictly isolated.**Tabular data ingestion:** Users may upload CSVs containing hundreds or thousands of rows, and those rows need to become queryable records.**Compatibility with LLM-generated SQL:** Many agentic systems rely on an LLM to generate the SQL needed to retrieve data. The simpler and more standard the SQL is, the more reliably the LLM can generate it.\n\nThese requirements create an interesting trade-off between flexibility, isolation, and query reliability.\n\nWhy Traditional Relational Schemas Can Be Difficult\n\nA traditional relational database with predefined columns works extremely well when the developer controls the schema. It becomes more difficult when the end user defines the structure.\n\nYou cannot define every possible column a user might need in advance. One dataset might contain `price`\n\n, `category`\n\n, and `stock`\n\n, while another contains `customer_name`\n\n, `email`\n\n, and `subscription_status`\n\n.\n\nThis is why document-oriented and semi-structured approaches such as JSONB initially appear attractive.\n\nPostgres + JSONB: Flexibility with Caveats\n\nPostgreSQL's JSONB data type provides an appealing middle ground. You can store arbitrary structured data in a JSONB column while still benefiting from Postgres's mature ecosystem, transactions, indexing, and security features.\n\nJSONB is fully queryable using operators such as `->`\n\n, `->>`\n\n, and `@>`\n\n, and it supports powerful GIN indexes.\n\nA typical design might look like this:\n\n```\nCREATE TABLE dataset_rows (\n  id UUID PRIMARY KEY,\n  tenant_id UUID NOT NULL,\n  dataset_id UUID NOT NULL,\n  row_data JSONB NOT NULL\n);\n```\n\nEach row of a user-uploaded CSV becomes a row in this table, with the CSV columns stored inside the JSONB document.\n\nMulti-tenancy can be handled using a `tenant_id`\n\ncolumn together with Row Level Security (RLS). When configured correctly, the database itself can enforce tenant isolation rather than relying entirely on application-level filtering.\n\nThis architecture works well from a storage perspective. However, it introduces additional complexity when the primary way of querying the data is through LLM-generated SQL.\n\nLLMs are generally more reliable when generating ordinary column-based SQL than when they need to produce JSONB path expressions, type casts, and containment operators. Every additional layer of syntax gives the model another opportunity to generate an invalid or incorrect query.\n\nMulti-Tenancy Is Solvable; Dynamic Queries Are Inevitable\n\nThe possibility of data from different users mixing in the same table is a legitimate concern, but it is a solvable one.\n\nA `tenant_id`\n\nor `agent_id`\n\ncolumn combined with Row Level Security is a mature approach for multi-tenant applications. Stronger forms of isolation, such as schema-per-tenant or database-per-tenant, are also possible depending on the application's requirements.\n\nDynamic queries, however, are unavoidable.\n\nBecause the fields are defined by the end user, neither the application nor the LLM can rely on a fixed set of column names known at development time. The system needs to understand the schema of each dataset and construct queries against it at runtime.\n\nThis challenge exists regardless of whether the underlying data is stored in Postgres JSONB, relational tables, or a document database.\n\nHandling User-Uploaded CSVs\n\nConsider a user uploading a CSV containing 1,000 product records.\n\nThe system needs to turn that tabular data into something the agent can efficiently query. After ingestion, the agent may need to answer questions such as:\n\n- Which products cost less than ₹500?\n- Which electronics products are currently in stock?\n- Which brands have more than 20 products?\n- What is the average price within a particular category?\n\nAnother user's CSV may contain completely different columns.\n\nAt the scale of a few thousand rows per dataset, performance is unlikely to be the deciding factor. Most modern databases can comfortably handle that volume.\n\nThe more important considerations are schema flexibility, isolation, operational simplicity, and how easily an LLM can generate correct queries against the data.\n\nThe Decisive Factor: LLM-Generated SQL\n\nOnce an LLM is responsible for generating queries, the evaluation criteria change.\n\nConsider asking an LLM to generate this:\n\n``` php\nWHERE (row_data->>'price')::numeric < 500\n  AND row_data->>'category' = 'electronics'\n```\n\nNow compare it with:\n\n```\nWHERE price < 500\n  AND category = 'electronics'\n```\n\nThe second version is much simpler. It uses ordinary column names and standard SQL patterns that language models encounter frequently.\n\nThis matters because the goal is not merely to choose a database that *can* execute the query. The goal is to create an environment in which the LLM can generate the correct query as reliably as possible.\n\nThat changes the architectural decision considerably.\n\nRecommended Architectures for Configurable Agents\n\nGiven the combination of user-defined schemas, multi-tenancy, CSV ingestion, and LLM-generated SQL, three approaches stand out.\n\n**1. Create relational tables from uploaded CSVs**\n\nWhen a CSV is uploaded, the system can inspect its headers and create an actual table whose columns correspond to the CSV columns. The rows are then inserted as ordinary relational records.\n\nThe LLM can be given the resulting schema and generate standard SQL against it.\n\nThis provides excellent queryability and keeps the SQL surface simple. If Postgres is already the primary database, this approach allows teams to continue using its mature ecosystem.\n\nHowever, dynamically creating large numbers of tables introduces its own operational considerations, particularly as the number of users and datasets grows. Table lifecycle, naming, migrations, permissions, and cleanup all need to be managed carefully.\n\n**2. Use DuckDB for user-defined datasets**\n\nDuckDB is particularly interesting for this use case.\n\nIt provides a clean SQL interface, excellent performance on analytical and tabular data, and first-class support for formats such as CSV and Parquet.\n\nInstead of forcing arbitrary user-defined datasets into the application's primary relational database, each agent or dataset can have its own DuckDB database file. The agent can then query ordinary tables using standard SQL.\n\nThis creates a useful separation:\n\n**Application data -> Postgres****User-defined agent data -> DuckDB**\n\nFor agentic applications, this can be a very natural architecture. Each agent effectively gets its own isolated, queryable database while the main application database continues to manage users, agents, permissions, billing, configuration, and other application-level data.\n\n**3. Use SQLite per agent or dataset**\n\nSQLite offers a similar file-based isolation model.\n\nA dedicated SQLite database can be created for each agent or dataset, giving each one its own schema and physical database file. The LLM still gets a clean, standard SQL interface.\n\nSQLite is simple, mature, and extremely reliable. For applications that primarily need transactional reads and writes against relatively small datasets, it can be an excellent choice.\n\nDuckDB becomes more attractive when the workload is heavily oriented toward querying, filtering, aggregating, and analyzing tabular data.\n\nApproaches to Use More Cautiously\n\n**Postgres + JSONB** remains a viable option, particularly when keeping everything inside one database is operationally valuable.\n\nHowever, if LLM-generated SQL is central to the product, JSONB adds unnecessary query complexity. Strong prompting, schema information, query validation, and retries can compensate for this, but they also add complexity elsewhere in the system.\n\n**MongoDB** provides excellent schema flexibility and works naturally with arbitrary document structures. However, it does not use SQL. If the architecture is specifically designed around LLM-generated SQL, MongoDB requires either a different query-generation strategy or an additional translation layer.\n\nComparison Summary\n\n| Approach | Schema Flexibility | Multi-Tenancy | CSV Ingestion | LLM SQL Reliability | Overall Fit |\n|---|---|---|---|---|---|\n| Postgres + relational tables | High | Excellent | Excellent | Excellent | Excellent |\n| DuckDB | High | Excellent with file-level isolation | Excellent | Excellent | Excellent |\n| SQLite per agent/dataset | High | Excellent with file-level isolation | Good | Excellent | Strong |\n| Postgres + JSONB | High | Excellent | Good | Medium | Good |\n| MongoDB | High | Good | Good | Not SQL-based | Weak for SQL-first agents |\n\nConclusion\n\nBuilding an AI agent that can work with arbitrary structured data supplied by the end user places unusual demands on the database layer.\n\nSchema flexibility alone is not enough. The architecture also needs strong isolation, efficient ingestion and querying of tabular data, and, when LLMs generate the queries, a simple and predictable query surface.\n\nPostgres remains an excellent foundation for application data. But user-defined agent data does not necessarily need to live in the same database.\n\nFor systems built around user-uploaded tabular data, engines such as DuckDB and, in some cases, SQLite offer an interesting alternative. Each agent can effectively have its own isolated database, with its own schema, while still exposing ordinary SQL to the LLM.\n\nThat combination is particularly powerful for agentic systems: **flexible schemas for users, physical isolation between agents, and simple SQL for the LLM.**\n\nThe database decision is therefore no longer just about storage, scalability, or query performance. In an LLM-powered system, it is also about creating a data environment that the model can understand and query reliably.\n\nAnd that may ultimately be one of the most important database design considerations for the next generation of configurable AI agents.", "url": "https://wpnews.pro/news/choosing-the-right-database-for-ai-agents-llm-generated-sql", "canonical_source": "https://predictabledialogs.com/learn/ai-stack/choosing-database-configurable-ai-agents", "published_at": "2026-08-15 05:53:34+00:00", "updated_at": "2026-08-15 06:11:59.765594+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-infrastructure"], "entities": ["Jai", "PostgreSQL", "JSONB"], "alternates": {"html": "https://wpnews.pro/news/choosing-the-right-database-for-ai-agents-llm-generated-sql", "markdown": "https://wpnews.pro/news/choosing-the-right-database-for-ai-agents-llm-generated-sql.md", "text": "https://wpnews.pro/news/choosing-the-right-database-for-ai-agents-llm-generated-sql.txt", "jsonld": "https://wpnews.pro/news/choosing-the-right-database-for-ai-agents-llm-generated-sql.jsonld"}}