Connect your AI agents to Datalake to read and store data Datalake-MCP is a .NET 9 Model Context Protocol server that connects AI agents to Azure Data Lake Storage Gen2 accounts, letting MCP-capable clients such as Claude Desktop and Claude Code navigate filesystems, read CSV, Parquet, XML, text and markdown files, and write text, markdown, CSV, Parquet and PDF files. The server runs as a .NET Generic Host over stdio with no HTTP endpoint to deploy, and reads files lazily via DataLakeFileClient.OpenReadAsync, using Parquet row-group reads with column projection and pagination through truncated and nextOffset fields to work around the roughly 1 MB tool-call size limit. Its filter parameter on read_csv, read_parquet and read_xml supports eq, ne, gt, gte, lt, lte, contains, startswith, endswith, isnull and isnotnull operators combined with AND. How Datalake-MCP turns an Azure Data Lake Storage Gen2 account into a set of tools that Claude can navigate, query and write to Most of the data a company cares about does not live in a database an LLM can talk to. It lives in a data lake: thousands of CSV and Parquet files, organised in folders by domain, year or source system. Getting an AI agent to use that data usually means writing a custom pipeline, exporting samples by hand, or pasting rows into a chat window. The Model Context Protocol MCP gives us a cleaner option. If the data lake is exposed as an MCP server, any MCP-capable client Claude Desktop, Claude Code, your own agent can discover and use it as a set of typed tools. That is what Datalake-MCP does. It is a .NET 9 MCP server that connects to an Azure Data Lake Storage Gen2 account and lets an agent: - N avigate filesystems and directories, - R ead CSV, Parquet, XML, text and markdown files, with filters, column projection and pagination, - S ave text, markdown, CSV, Parquet and PDF files, with a searchable index of everything that has been written. This post covers how it works internally and what it makes possible. User <- Claude Desktop / Code ← MCP over stdio → Datalake.Mcp.Server .NET 9 The server runs as a .NET Generic Host and talks to the client over stdio . The client launches it as a child process, so there is no HTTP endpoint to secure and nothing to deploy. Connection settings AccountName , AccountKey , DefaultFileSystem come from .NET user-secrets in development, or from environment variables when the server is registered in the client. The code is split into three layers: - Tools DataLakeTools : the MCP surface. Each method is a tool with a description and typed parameters that the model sees. - Services : format-specific readers, the writer, the index and the row filter. They contain no Azure-specific code. - DataLakeService : a thin wrapper over Azure.Storage.Files.DataLake , the only class that talks to the storage account. The tools are designed to be called in sequence. A typical exploration goes list filesystems , then list directory , then get parquet schema , then read parquet . The agent sees column names and types before it asks for rows, so it can write sensible filters instead of guessing. A data lake file can be gigabytes in size, and an LLM can only consume a tiny fraction of it. The server is built around that mismatch. Lazy remote reads. Files are opened with DataLakeFileClient.OpenReadAsync , which returns a stream that downloads only the bytes actually requested. Nothing is fetched up front. - CSV, text and markdown are read line by line and the read stops at the requested limit. - XML is read element by element tabular XML: a root with repeated record elements . DTD processing and external entity resolution are disabled, which prevents XXE attacks. - Parquet is read one row group at a time, with column projection : if you ask for three columns, the others are never decoded. This is where Parquet’s columnar layout pays off. Pagination that does not re-read. Every read tool returns truncated and nextOffset . When a result is cut off, either by maxRows or by the tool-call size limit about 1 MB , the agent calls the tool again with offset = nextOffset and gets the next page. The model walks a large file in chunks without any hidden state in the server. The most useful feature for analytical work is the filter parameter on read csv , read parquet and read xml . It is a list of conditions combined with AND: json {“column”: “stato”, “operator”: “eq”, “value”: “APERTO”}, {“column”: “importo”, “operator”: “gt”, “value”: “1000”} Supported operators are eq , ne , gt , gte , lt , lte , contains , startswith , endswith , isnull and isnotnull . Comparison semantics: - numeric when both the cell and the value parse as numbers invariant format, 12.5 , - timestamp-aware for Parquet date columns when the value is a date, - case-insensitive text otherwise. Two design decisions matter here: 1. The filter runs inside the streaming loop, before offset and maxRows . The pagination counters therefore count only matching rows. “Give me the next 100 open orders over 1000” works as expected, and non-matching rows never reach the model’s context window. 2. Filters can reference columns that are not projected. You can filter on stato and return only id and importo . A wrong column name or operator returns an error that lists the allowed values, so the agent can correct itself in the next call without human help. A request like ”show me the open orders above 1000" becomes: look at the schema, build a filter, read with a projection, and summarise the result. The model never sees the other 99% of the file. Reading is only half of the story. Agents are far more useful when they can leave something behind: a report, a cleaned dataset, a summary of an analysis. Write access to shared storage is also where things go wrong, so the write path was designed conservatively. The save tools take a path folders and subfolders are created if missing , a fileSystem , a description and an overwrite flag. No silent overwrite. Uploads are conditional If-None-Match: . If the file already exists and overwrite is not true , the call fails and the existing file is untouched. An agent cannot destroy data by accident, only by being explicitly told to. Validation before upload. - The path extension must match the format. - CSV must have the same number of fields on every row. - Parquet is built from a list of rows column to value objects , with types inferred bool, integer, decimal, string . - PDF is received as base64, and the %PDF- header is checked. Each successful save is recorded in an index file , a CSV at the filesystem root configured with DataLake:SavedFilesIndexPath , default saved-files-index.csv with the columns: path, name, format, sizeBytes, savedAt, description Saving the same path again updates its row. Because several agents or sessions may write at the same time, the index is updated with optimistic concurrency : read the index and its ETag, apply the change, write back with If-Match: