Private AI or local AI? What differs, and why both terms exist.

Start with what is not there. Neither "local AI" nor "private AI" has a formal definition, and we know of no standard that defines either term. The NIST AI Risk Management Framework, released January 26, 2023, treats privacy as one of its characteristics of trustworthy AI, "privacy-enhanced", and does not define "private AI" or "local AI" as categories [10]. Both terms are vendor and community usage, and they are often used interchangeably in product copy and in architecture discussions.

What exists instead is consistent usage, and it points at two different questions. In practice "local AI" is used descriptively: the model weights are on hardware you control and inference runs there. "Private AI" is used for a property of the whole system: who can reach the data, and everything derived from it, and whether that can be verified. Both terms are needed because the two questions have independent answers: a team can run a model locally and still send everything that matters somewhere else, and, as one well-documented case below shows, a system can be private without being local.

This post works through the specifics: what local buys, where local deployments quietly stop being private, why contractual privacy is not the same thing as architectural privacy, how the vendors who use each term define it, the four combinations that follow, and the four properties we require before we use the word ourselves. It closes with where LM-Kit sits and what we do not claim. Every external fact carries its source in the list at the end.

What running locally actually buys

Local inference delivers four things, all real and all worth having.

• Latency. No network round trip per request. For interactive work on a document or a transcript this is the difference between a tool people use and a tool they wait for.
• Availability. No provider outage, no rate limit, no regional incident. The service is up when your machine is up, and it works on a site with no connection at all.
• Cost shape. Capacity you plan replaces a meter that scales with exactly the volume you automated. The cost comparison covers the arithmetic.
• Stability. A model file you hold does not change. A hosted model does. A snapshot of OpenAI's deprecation schedule as read on September 18, 2026, which moves as models retire: it lists the gpt-5-2025-08-07 and o3 snapshots on December 11, 2026 (announced June 11) and whisper-1 with the gpt-4o transcription models on February 26, 2027 (announced August 26) [1]. Six months' notice in each case, and every pipeline built on those models still re-validates on the provider's calendar, not yours.

Notice what is missing from that list. Nothing in "the model runs here" says where the document went before the model saw it, where the index built from it lives, where the logs of the conversation are kept, or what the assistant that called the model was allowed to read. Local is a property of inference. Private is a property of the data path.

Five ways a local deployment stops being private

These are the patterns we see most often in architectures presented as local. Each one keeps the model on-premises and moves something else out.

1. Embeddings from a hosted API. The chat model is local; the embedding model is a cloud endpoint. Every chunk of every document is sent out in plain text to be turned into a vector. The corpus has left the building, one paragraph at a time.
2. A managed vector database. Vectors are typically stored with their source text so results can be shown. The retrieval layer is now a SaaS holding a searchable copy of the documents.
3. Tracing and observability as a service. Prompts, retrieved context and completions are logged in full to a third-party tool "for debugging". This is the entire data path, retained externally, under the log-retention policy of a company that is not your processor of record.
4. A hosted assistant with a local file tool. The tool runs on your machine; the model reasoning over its output does not. If the tool returns the document, the document has been disclosed. Governance lives in what the tool is allowed to return, not in where the tool executes.
5. Models that update themselves. A runner that pulls the latest tag on restart changes behaviour underneath a validated pipeline. This is a control failure rather than a disclosure, and it fails the same test: the organization no longer decides what the system does.

None of these is a mistake made by careless people. Each is an ordinary configuration in widely used tooling. The point is that "we run the model locally" is compatible with all five.

A contract about processing is not a property of the system

The usual answer to the leaks above is a contract: a data processing agreement, a zero-retention tier, a region selection. Those instruments matter and we recommend reading them. They are also legal promises, and legal promises can be overridden by other legal instruments. Four documented cases.

Default retention is a policy, not an absence. OpenAI's data-usage documentation states that "by default, abuse monitoring logs are generated for all API feature usage and retained for up to 30 days", that zero data retention applies only to eligible endpoints, and that the state held by the conversations, assistants and threads endpoints is retained "until deleted" [2]. This is a reasonable policy, published clearly. It is still a policy that the provider, or a court, can change.

A court did change it. On May 13, 2025, in The New York Times v. OpenAI, Magistrate Judge Ona T. Wang ordered OpenAI to "preserve and segregate all output log data that would otherwise be deleted on a going forward basis", which covered conversations users had deleted. By OpenAI's own account the order applied to ChatGPT Free, Plus, Pro and Team users and to API customers without a zero data retention agreement, and not to ChatGPT Enterprise, ChatGPT Edu or zero-retention API customers; it was lifted, with exceptions, on October 9, 2025 [3][4]. For nearly five months, a 30-day deletion policy was suspended by a proceeding its customers were not party to. The customers who were exempt were exempt because of the tier they had bought, not because of a property they held.

Region is not residency. The United States CLOUD Act of 2018 added 18 U.S.C. § 2713, which requires a provider of remote computing services to preserve and disclose customer records within its "possession, custody, or control, regardless of whether such communication, record, or other information is located within or outside of the United States" [5]. Choosing a European region for a US provider's service changes where the disk is. It does not change who can be compelled to read it.

Contract clauses have already lost to surveillance law once. On July 16, 2020, the Court of Justice of the European Union invalidated the EU-US Privacy Shield in Schrems II, and held that standard contractual clauses can only be relied on where the destination country's law does not allow authorities to access the data in ways the clauses cannot prevent [6]. The ruling is the clearest statement a court has made that a contract about processing cannot substitute for control over processing.

A promise about what happens to your data is a legal instrument. Other legal instruments can override it. Where the data physically is, and who holds the only copy, cannot be overridden by a filing.

Two different questions

With no formal definitions to lean on, the best evidence of what the terms mean is how the vendors who use them define them, and those definitions point at different questions. "Local AI" and "on-device AI" describe topology: the model weights are downloaded and inference runs on hardware the user controls, without a call to an external inference service. "Private AI" entered enterprise vocabulary when VMware announced it under that name at VMware Explore in August 2023, and its May 2024 white paper defines it as "an architectural approach that aims to balance the business gains from AI with the practical privacy and compliance needs of the organization" [7]; VMware positions the same approach across data centers, virtual private clouds, public clouds and edge sites, so the definition is deliberately independent of where the hardware sits. "Sovereign AI", in Red Hat's words, "represents a shift from renting AI to owning AI" and is about "owning technology, keeping data local, and making sure your AI systems reflect your values and legal requirements" [8]: a claim about ownership and jurisdiction.

So local answers where the computation happens. Private answers who can reach the data and everything derived from it, for how long, for what purpose, and whether that can be verified. Two independent questions give four combinations, and all four exist in production today.

The bottom-left cell is the interesting one, because it shows that private without local is possible. It is also the hardest cell to occupy. Apple can claim it because it publishes the measurements of every software image running in production to an append-only log and invites researchers to verify that the running code matches the inspected code [9]. A hosted provider that offers a retention promise and a region selector is not in that cell; it is in the cell to its right, with a contract. For an organization that cannot audit its provider's runtime, which is nearly every organization, the one route into the private column that it can verify itself is the top-left cell: hold the infrastructure. That is why our definition of private AI requires control of the infrastructure, and why we treat local as necessary in practice and never sufficient.

Written as an equation, and valid under exactly that condition:

Private AI = local AI for the whole data path, not just the model, + models your operators pin + egress your operators govern + output you can verify.

Every term after the first is something the word "local" does not promise. The first term is where the word is most often stretched: the model is local and the embeddings, the index or the logs are not.

The four-property test

Since no standard defines the term, we state the bar we hold ourselves to. This is LM-Kit's working definition, the one on our private AI page, and not a claim that everyone uses the word this way. A deployment meets it when it holds all four, whatever the model's address.

• Residency. Every artifact, from the source file to the output, including embeddings, the index, logs and conversation state, stays on machines you administer.
• Model control. Models are chosen, pinned and versioned by your operators. Nothing changes without an operator's action.
• Governed egress. Every outbound call is an explicit, logged, administrator-set decision, and the default is none.
• Verifiable output. A confidence score per extracted field, citations on answers, and document changes checkable against the original.

The fourth property surprises people in a privacy conversation. It belongs there for a practical reason: a system whose output cannot be checked forces a person to re-read the source document to trust the result, which means the sensitive document is opened again, by more people, more often. A redaction you cannot prove is not a redaction. An extracted total with no confidence figure is a number someone will re-key by hand. Verification is what lets private data stay closed.

Private AI does not mean no cloud model

The property is governed egress, not geography. A fully local deployment is one setting of that control; an air gap is the strictest setting. A third is common in practice: an external assistant, say a frontier model your team already uses, is given governed tools through the Model Context Protocol. The sensitive work, reading the document, searching the index, extracting the fields, runs locally. The assistant receives the result of the tool it was allowed to call and nothing else. The source document is never disclosed; the derived result is, and an administrator decided which results are allowed.

That arrangement is private by our definition because the organization decides what leaves. It would not be private if the assistant held a tool that returned the file. The distinction is the whole discipline: who decides, and whether the decision is enforced by the system or hoped for in a policy document. The Trust Center states both modes and what each one discloses.

Where LM-Kit stands

LM-Kit is both, and the distinction is why the product has the shape it has. It is local AI by construction: models, embeddings, indexes, agents and tools run on hardware you control, as a server you deploy with LM-Kit One or inside your own .NET process with LM-Kit.NET, whose required cloud calls are zero. It is built for the top-left cell of the matrix, which means the four properties are engineered rather than assumed.

• Residency. A model runner is one component of the stack that touches your data; the leaks above come from the others. LM-Kit One keeps them in the same perimeter as the model, developed and released as one server: OCR and layout, schema-constrained extraction, a search engine with citations, agents, the OpenAI, Anthropic, Ollama and MCP dialects, audit and a console. The embedding model runs beside the chat model on the same hardware, so nothing in the default path calls a hosted embedding endpoint, and request history stays in the server's own store, so no external tracing service is needed to see what happened.
• Model control. Models are chosen, imported and pinned by your operators. Nothing changes underneath a validated pipeline because a provider shipped an update.
• Governed egress. Fully local operation is the default and air-gapped operation is supported. Connecting an external assistant through MCP is an administrator's decision, and the assistant receives only the results of the tools it is allowed to call.
• Verifiable output. A confidence score per extracted field, document and page citations on answers, and abstention when the corpus does not support a claim.

Given that, why do we describe LM-Kit as a private AI company rather than a local AI one, when every LM-Kit deployment is local? Because the two words describe different things about the same product, and only one of them is the promise. Nuance by nuance:

• Local is a location; private is an obligation. "Local" states where the software runs, which is a fact about a deployment. "Private" states what the software owes the owner of the data: nothing leaves without a decision, and what comes out can be checked. We prefer to be judged on the obligation.
• Local describes one component; private describes the whole path. The category people call "local AI" is populated by model runners, and the five leaks above all happen outside the runner. LM-Kit One ships the document pipeline, the search engine, the agents and the serving layer together because that is where the data actually travels; a word that names the runner alone undersells the part that matters.
• Local is a geography; private includes governed disclosure. An LM-Kit deployment can connect an external assistant through MCP and let it call tools whose results an administrator has allowed. That mode is not "local" in any useful sense, and it is private by our definition. The word has to cover both modes.
• Local says nothing about verification. Confidence scores, citations and abstention are central to how LM-Kit is built, and they are privacy features as much as quality features: output that can be checked is output that does not send a person back to the sensitive source.
• Local is a download; private is a discipline. Anyone who downloads open weights has local AI by lunchtime. Private AI is the identity, access policies, audit, egress control and pinned models around it, which is what LM-Kit One adds to a runner and why it exists.
• We do not disown local. Local execution is how LM-Kit achieves residency, and it is a property of every deployment. For us, privacy is a commitment. Local is how we keep it.

The commitments around it are checkable rather than promised: no account, no license key, no activation and nothing verified at runtime; telemetry export off until an operator enables it; signed packages, with SHA-256 checksums and an SBOM available per release; the documentation shipped inside the server for disconnected sites. LM-Kit is a French company under French law, part of the Calico IIM Group, whose companies have managed enterprise content for more than forty years; the sovereignty page gives the facts rather than the flag.

What we do not claim: that open-weight models match the largest hosted models on every task. They are smaller, and the stack around them is what closes the gap on the tasks that matter for documents: layout-aware reading, constrained extraction, retrieval that cites, review in front of uncertain results. Capability is measured on your hardware and your documents before you commit, which is why both products are free to build and evaluate with no time limit. When a task needs frontier reasoning, the governed-tools mode above exists, with the boundary explicit.

A deployment that answers all eight with something you can point at meets our definition of private. One that answers "the model runs on our server" is local, and that is a good start.

Sources

1. OpenAI, Deprecations, read September 18, 2026.
2. OpenAI, Your data, data retention and zero data retention eligibility, read September 18, 2026.
3. OpenAI, How we're responding to The New York Times' data demands in order to protect user privacy, June 2025.
4. Simon Willison, OpenAI slams court order to save all ChatGPT logs, including deleted chats, June 5, 2025, quoting the May 13, 2025 order of Magistrate Judge Ona T. Wang; the order was lifted with exceptions on October 9, 2025.
5. 18 U.S.C. § 2713, added by the CLOUD Act, Public Law 115-141, Division V, March 23, 2018.
6. Court of Justice of the European Union, Press Release No 91/20, judgment in Case C-311/18, July 16, 2020.
7. VMware by Broadcom, VMware Private AI: Privacy and Security Best Practices, technical white paper, May 2024.
8. Red Hat, What is sovereign AI?
9. Apple Security Research, Private Cloud Compute: A new frontier for AI privacy in the cloud, June 10, 2024.
10. NIST, AI Risk Management Framework (AI RMF 1.0), released January 26, 2023; "privacy-enhanced" is one of its characteristics of trustworthy AI.