ONNX Runtime GenAI is Microsoft's on-device generate loop: tokenize, decode, sample, stream. LM-Kit.NET is the embedded private AI runtime .NET applications build on: the loop, plus documents, RAG, agents, and speech.
Both put local generative AI inside a C# application. They disagree about how much of the application they should carry.
ONNX Runtime GenAI
MIT-licensed generative extensions for ONNX Runtime: pre and post processing, sampling, KV cache, and grammar-based tool calling, with a C# API and execution providers from CPU and CUDA to DirectML, OpenVINO, and QNN. It powers Foundry Local, Windows ML, and the VS Code AI Toolkit.
LM-Kit.NET
One NuGet package where the generate loop is the foundation, not the product: document intelligence, structured extraction with confidence, RAG with citations, governed agents, and speech-to-text ride the same engine, entirely in process.
The rows that decide real projects, not a feature checklist.
| Dimension | LM-Kit.NET | ONNX Runtime GenAI |
|---|---|---|
| Primary job | A complete AI runtime: inference plus the application capabilities on top | The generative loop over ONNX models, done well |
| Models | Curated open-weight catalog downloaded on demand, or import your own; quantization built in | ONNX graphs, prepared per model with the model builder or pulled pre-converted |
| Hardware | CPU-first; CUDA, Vulkan, and Metal acceleration | Execution providers: CPU, CUDA, DirectML, TensorRT, OpenVINO, QNN, WebGPU |
| Platforms | Windows, Linux, macOS | Windows, Linux, macOS, Android; iOS in development |
| Document intelligence | Parsing, OCR, layout, Markdown, splitting, extraction with confidence, redaction, signatures, PDF/A | Not in scope; parsing, OCR, and pipelines are yours to assemble |
| RAG and search | Embeddings, built-in vector store, hybrid retrieval, reranking, citations | Embedding models runnable; the retrieval stack is yours |
| Agents and tools | Agent orchestration, 70+ built-in tools, permission policies, memory, MCP | Grammar-based tool calling inside the loop; orchestration lives elsewhere |
| Speech | Local speech-to-text with streaming, VAD, and translation | Speech models runnable; no speech pipeline |
| A serving twin | The same engine ships as LM-Kit One, a server other applications call | Foundry Local serves ONNX models locally as a separate product |
| Licensing | Free to build and evaluate; Professional for larger production use | MIT open source |
ONNX Runtime GenAI evolves quickly and this table reflects our reading of its public documentation at publication. Corrections are welcome through contact.
One question settles most cases: are you building the AI stack, or building the application that needs one?
Choose ONNX Runtime GenAI
You want Microsoft's MIT-licensed loop over ONNX models, target NPUs and mobile through execution providers, and your team assembles everything above the loop.
Choose LM-Kit.NET
The product needs documents understood, answers cited, agents governed, and speech transcribed, in process, this quarter, without assembling and maintaining that stack yourself.
The two also meet in the middle: Foundry Local, built on ONNX Runtime GenAI, is compared separately.
The full comparison grid, plus the capability pages most relevant to this one. The same engine ships two ways: embedded as LM-Kit.NET, or served as LM-Kit One, the Private AI Application Server.
LM-Kit.NET
