Local AI Is Not Private AI

"Local AI" and "private AI" are used as synonyms in vendor copy, in procurement questionnaires and in architecture reviews. They are not synonyms. Local AI is a statement about one component: the model runs on hardware you control. Private AI is a statement about the whole system: nothing the work produces reaches a third party unless someone with the authority to decide has decided it, and what comes out can be checked. A team can run a model locally and still send everything that matters somewhere else.

This post separates the two terms with the specifics: what local buys, where local deployments quietly stop being private, why contractual privacy is not the same thing as architectural privacy, and the four properties we use to decide whether a deployment deserves the word. It closes with where LM-Kit stands and what we do not claim. Every external fact carries its source in the list at the end.

What running locally actually buys #

Local inference delivers four things, all real and all worth having.

• Latency. No network round trip per request. For interactive work on a document or a transcript this is the difference between a tool people use and a tool they wait for.
• Availability. No provider outage, no rate limit, no regional incident. The service is up when your machine is up, and it works on a site with no connection at all.
• Cost shape. Capacity you plan replaces a meter that scales with exactly the volume you automated. Thecost comparison covers the arithmetic.
• Stability. A model file you hold does not change. A hosted model does: OpenAI's deprecation schedule, read on September 18, 2026, retires thegpt-5-2025-08-07 ando3 snapshots on December 11, 2026 (announced June 11) andwhisper-1 with thegpt-4o transcription models on February 26, 2027 (announced August 26) [1]. Six months' notice is generous by industry standards, and it still means every pipeline built on those models re-validates on the provider's calendar, not yours.

Notice what is missing from that list. Nothing in "the model runs here" says where the document went before the model saw it, where the index built from it lives, where the logs of the conversation are kept, or what the assistant that called the model was allowed to read. Local is a property of inference. Private is a property of the data path.

Five ways a local deployment stops being private #

These are the patterns we see most often in architectures presented as local. Each one keeps the model on-premises and moves something else out.

1. Embeddings from a hosted API. The chat model is local; the embedding model is a cloud endpoint. Every chunk of every document is sent out in plain text to be turned into a vector. The corpus has left the building, one paragraph at a time.
2. A managed vector database. Vectors are stored with their source text so results can be shown. The retrieval layer is now a SaaS holding a searchable copy of the documents.
3. Tracing and observability as a service. Prompts, retrieved context and completions are logged in full to a third-party tool "for debugging". This is the entire data path, retained externally, under the log-retention policy of a company that is not your processor of record.
4. A hosted assistant with a local file tool. The tool runs on your machine; the model reasoning over its output does not. If the tool returns the document, the document has been disclosed. Governance lives in what the tool is allowed to return, not in where the tool executes.
5. Models that update themselves. A runner that pulls the latest tag on restart changes behaviour underneath a validated pipeline. This is a control failure rather than a disclosure, and it fails the same test: the organization no longer decides what the system does.

None of these is a mistake made by careless people. Each is the default of a popular tool. The point is that "we run the model locally" is compatible with all five.

A contract about processing is not a property of the system #

The usual answer to the leaks above is a contract: a data processing agreement, a zero-retention tier, a region selection. Those instruments matter and we recommend reading them. They are also legal promises, and legal promises can be overridden by other legal instruments. Four documented cases.

Default retention is a policy, not an absence. OpenAI's data-usage documentation states that "by default, abuse monitoring logs are generated for all API feature usage and retained for up to 30 days", that zero data retention applies only to eligible endpoints, and that the state held by the conversations, assistants and threads endpoints is retained "until deleted" [2]. This is a reasonable policy, published clearly. It is still a policy that the provider, or a court, can change.

A court did change it. On May 13, 2025, in The New York Times v. OpenAI, Magistrate Judge Ona T. Wang ordered OpenAI to "preserve and segregate all output log data that would otherwise be deleted on a going forward basis", which covered conversations users had deleted. The order applied to ChatGPT Free, Plus, Pro and Team users and to API traffic without a zero-retention agreement; it was lifted, with exceptions, on October 9, 2025 [3][4]. For five months, a 30-day deletion promise made to millions of customers was suspended by a proceeding none of them were party to. The customers who were exempt were exempt because of the tier they had bought, not because of a property they held.

Region is not residency. The United States CLOUD Act of 2018 added 18 U.S.C. § 2713, which requires a provider of remote computing services to preserve and disclose customer records within its "possession, custody, or control, regardless of whether such communication, record, or other information is located within or outside of the United States" [5]. Choosing a European region for a US provider's service changes where the disk is. It does not change who can be compelled to read it.

Contract clauses have already lost to surveillance law once. On July 16, 2020, the Court of Justice of the European Union invalidated the EU-US Privacy Shield in Schrems II, and held that standard contractual clauses can only be relied on where the destination country's law does not allow authorities to access the data in ways the clauses cannot prevent [6]. The ruling is the clearest statement a court has made that a contract about processing cannot substitute for control over processing.

        A promise about what happens to your data is a legal instrument. Other legal instruments can override it. Where the data physically is, and who holds the only copy, cannot be overridden by a filing.

The four properties that make AI private #

This is the test we apply, and the one our definition of private AI is built on. A deployment is private when it holds all four. Running the model locally settles the first half of the first row and nothing else; the table shows what each property looks like once the word "local" has done all it can.

The fourth property surprises people in a privacy conversation. It belongs there for a practical reason: a system whose output cannot be checked forces a person to re-read the source document to trust the result, which means the sensitive document is opened again, by more people, more often. A redaction you cannot prove is not a redaction. An extracted total with no confidence figure is a number someone will re-key by hand. Verification is what lets private data stay closed.

Why the distinction has a date on it #

The general obligations of the EU AI Act have applied since August 2, 2026, six weeks before this post. The Act entered into force on August 1, 2024; obligations for general-purpose model providers applied from August 2, 2025; some high-risk categories were extended to December 2, 2027 and systems embedded in regulated products to August 2, 2028 [7]. The deployer duties in the text, logging, human oversight, record keeping, are all easier to demonstrate with records you hold than with records you can only request.

The organizational response to the alternative is already on record. In April 2023 Samsung engineers pasted internal material into ChatGPT; on May 1, 2023 the company restricted generative AI on company devices until it could "create a secure environment for safely using generative AI", as reported by Bloomberg [8]. That response, prohibition, is what most organizations chose, and it is what private AI exists to replace with something people can actually use.

The cost of getting it wrong keeps climbing. IBM's 2026 Cost of a Data Breach report puts the global average at USD 4.99 million, a 12% increase in a year and a record, and reports a 56% increase in AI-driven attacks [9].

**30** days of default API abuse-log retention [2]

**149** days a court suspended that deletion policy [3][4]

2 Aug 2026 general application of the EU AI Act [7]

$4.99M average breach cost, 2026 [9]

Private AI does not mean no cloud model #

The property is governed egress, not geography. A fully local deployment is one setting of that control; an air gap is the strictest setting. A third is common in practice: an external assistant, say a frontier model your team already uses, is given governed tools through the Model Context Protocol. The sensitive work, reading the document, searching the index, extracting the fields, runs locally. The assistant receives the result of the tool it was allowed to call and nothing else. The source document is never disclosed; the derived result is, and an administrator decided which results are allowed.

That arrangement is private by our definition because the organization decides what leaves. It would not be private if the assistant held a tool that returned the file. The distinction is the whole discipline: who decides, and whether the decision is enforced by the system or hoped for in a policy document. The Trust Center states both modes and what each one discloses.

Where LM-Kit stands #

We build for private AI, not for local AI, and the difference is the product. A model runner is one component of the stack that touches your data. The leaks above come from the others: embeddings, the index, retrieval, tracing, the tools an agent may call. LM-Kit One puts those components inside the same perimeter as the model, developed and released as one server: OCR and layout, schema-constrained extraction, a search engine with citations, agents under permission policies, the OpenAI, Anthropic, Ollama and MCP dialects, audit and a console. There is no hosted embedding endpoint in the path, because the embedding model runs beside the chat model. There is no external tracing service, because the request history is a table on your server. LM-Kit.NET compiles the same engine into a .NET application, in process; its required cloud calls are zero.

The commitments are checkable rather than promised: no account, no license key, no activation and nothing verified at runtime; telemetry export off until an operator enables it; signed packages with SHA-256 checksums and an SBOM per release; air-gapped operation with the documentation shipped inside the server. Outputs carry their evidence, a confidence score per extracted field, document and page citations on answers, and abstention when the corpus does not support a claim. LM-Kit is a French company under French law, part of the Calico IIM Group, whose companies have managed enterprise content for more than forty years; the sovereignty page gives the facts rather than the flag.

What we do not claim: that open-weight models match the largest hosted models on every task. They are smaller, and the stack around them is what closes the gap on the tasks that matter for documents: layout-aware reading, constrained extraction, retrieval that cites, review in front of uncertain results. Capability is measured on your hardware and your documents before you commit, which is why both products are free to build and evaluate with no time limit. When a task needs frontier reasoning, the governed-tools mode above exists, with the boundary explicit.

Eight questions before you call a deployment private

• Which model computes the embeddings, and where does it run?
• Where is the vector index stored, and does it hold the source text?
• Which service receives prompts and completions for logging or tracing?
• Can any component fetch a model or an update without an operator's action?
• If an external assistant is connected, which tools can it call, and what do they return?
• Who can change the list of allowed tools, and is the change recorded?
• For each extracted field, is there a confidence figure a reviewer can act on?
• For each answer, can a reader open the cited page without asking the system again?

A deployment that answers all eight with something you can point at is private. One that answers "the model runs on our server" is local, and that is a good start.

Run the whole stack where the data lives

The definition of private AI in full, and two ways to deploy it. Both free to build and evaluate, nothing to activate.

Sources #

1. OpenAI, Deprecations , read September 18, 2026.
2. OpenAI, Your data , data retention and zero data retention eligibility, read September 18, 2026.
3. OpenAI, How we're responding to The New York Times' data demands in order to protect user privacy , June 2025.
4. Simon Willison, OpenAI slams court order to save all ChatGPT logs, including deleted chats , June 5, 2025, quoting the May 13, 2025 order of Magistrate Judge Ona T. Wang; the order was lifted with exceptions on October 9, 2025.
5. 18 U.S.C. § 2713 , added by the CLOUD Act, Public Law 115-141, Division V, March 23, 2018.
6. Court of Justice of the European Union, Press Release No 91/20, judgment in Case C-311/18 , July 16, 2020.
7. European Commission, AI Act: regulatory framework for AI , application timeline of Regulation (EU) 2024/1689.
8. TechCrunch, Samsung bans use of generative AI tools like ChatGPT after April internal data leak , May 2, 2023, citing Bloomberg.
9. IBM, Cost of a Data Breach Report 2026 .