LoraFinetuning
Trainer with iteration loop, progress events, checkpointing.
Solutions · Local Inference · LLM fine-tuning
Specialise any model on your data.
LoraFinetuning trains LoRA adapters inside the same runtime
that serves your inference: no Python environment, no training server,
no export step between the model you train and the model you ship.
Datasets load from the standard formats (chat JSONL, ShareGPT, Alpaca,
plain text, ZIP archives); text and vision models train through the same
API; the output is a GGUF adapter or a merged model, and nothing ever
leaves your machine. This is the capability that compounds: every
interaction your app collects becomes data that makes your next model
better, locally.
LoraTrainingParameters
Rank, alpha, target modules, epochs, learning-rate schedule.
TrainingDataset
Build and export datasets in ShareGPT format.
ShareGptExporter
Export production conversations as a ShareGPT-format dataset.
Training and serving are one system.
LoRA (Low-Rank Adaptation) trains a small pair of matrices next to each frozen weight, so adapting a model costs a fraction of retraining it: a typical adapter is 10 to 100 MB and trains on consumer hardware. What makes the LM-Kit take different is where that training happens: inside the inference stack itself, against the same weights, templates, and pipelines that will serve the result.
One stack, no second system
The package that serves your app also trains it. No Python environment, no training cluster, no conversion scripts: LoraFinetuning runs in-process, next to the code that consumes the result, GPU-accelerated with a CPU fallback.
Train exactly what you serve
Training renders conversations through the same chat template, tokenizer, and vision pipeline inference uses, on the same GGUF weights, quantized bases included. The first live request behaves like the last training sample: nothing drifts in an export step, because there is none.
Your data never leaves the machine
Datasets, checkpoints, and artifacts stay on your hardware. That is what makes production data usable as training data: customer conversations, internal documents, and proprietary logs can teach the model precisely because nothing is uploaded to do it.
Every pipeline is a data source
Conversations feed the trainer directly as ChatHistory; extraction configurations generate supervised pairs from your documents; any dataset exports to ShareGPT for review and versioning. The application you already run produces the dataset that improves it.
Artifacts that deploy like models
One run yields a small adapter to hot-swap per tenant or task, or one merged GGUF that loads like any other model, optionally re-quantized to serving precision. Version them, ship them, apply them at load time: no serving-side training dependency.
Measured, not assumed
Live loss, per-epoch validation, and dataset diagnostics surface problems before compute is spent, and every shipped sample measures held-out quality before and after training so the improvement is a number, not an impression.
A full training run in 30 lines.
FineTune.cs
using LMKit.Model; using LMKit.Finetuning; using LMKit.TextGeneration.Chat; // 1. Load the base model (it stays frozen during training). var model = LM.LoadFromModelID("qwen3.5:0.8b"); // 2. Configure the trainer. using var finetuning = new LoraFinetuning(model); finetuning.Parameters.Rank = 16; finetuning.Parameters.Alpha = 32; finetuning.Parameters.TargetModules = LoraTargetModules.AttentionAndFeedForward; finetuning.Parameters.Epochs = 4; finetuning.Parameters.LearningRate = 1e-4f; finetuning.Parameters.Schedule = LearningRateSchedule.Cosine; // 3. Stream live loss and accuracy. finetuning.FinetuningProgress += (s, e) => { if (!e.IsValidation) Console.WriteLine($"epoch {e.Epoch + 1}/{e.TotalEpochs} step {e.Step}/{e.TotalSteps} loss={e.Loss:F4}"); }; // 4. Add training data as conversations; assistant turns are supervised. foreach (var (message, label) in tickets) { var sample = new ChatHistory(model); sample.AddMessage(AuthorRole.User, message); sample.AddMessage(AuthorRole.Assistant, label); finetuning.AddTrainingData(sample); } // 5. Train. Output is a GGUF adapter (or TrainToModel for a merged model). finetuning.TrainToAdapter("adapters/ticket-router.gguf"); // 6. Apply and serve, or hot-swap at runtime. model.ApplyLoraAdapter(new LoraAdapterSource("adapters/ticket-router.gguf"));
Build training data from production.
Most real fine-tuning failures stem from poor data, not poor hyperparameters. The SDK ships first-class tools for building, filtering, exporting, and versioning training datasets directly from running applications.
AddDatasetFile
Load the formats teams already have: chat JSONL (the OpenAI shape), ShareGPT, Alpaca, plain text, or a ZIP archive mixing them. Auto-detected, one call. Image references ride along for vision fine-tuning.
AddTrainingData / AddRawText
Add a ChatHistory as one or more conversations (assistant turns supervised, system and user turns masked), or raw text with every token supervised for continued-pretraining corpora.
TrainingDataset · ShareGptExporter
Build and export datasets in the standard ShareGPT format. Share with your team or version-control alongside your code.
SampleCount / SampleMinLength / SampleMaxLength
Inspect the loaded corpus before training so you catch outliers that would skew the loss curve.
AssistantLossOnly
Compute loss only on assistant tokens, the standard setup for instruction tuning. On by default for chat data.
UnmaskedSampleCount
Signals a chat-template mismatch: if the assistant span cannot be located, masking degrades to full-sequence loss and the count rises, so data problems surface instead of hiding.
The knobs that matter, sane defaults.
A focused parameter set: enough control for real runs, defaults calibrated so a first fine-tune just works.
LoRA structure
Rank and Alpha set adapter capacity and scaling; UseRsLora keeps high ranks at a usable effective scale. FirstLayer/LastLayer restrict adapters to a block range. Seed makes initialization reproducible.
Target modules
TargetModules: Attention, AttentionAndFeedForward, or All (including MoE experts). Spend capacity where the task needs it.
Schedule
LearningRate, Schedule (constant, cosine, cosine with restarts, linear, polynomial), MinLearningRate, and WarmupRatio. LoraPlusRatio trains the B matrices at a boosted rate.
Throughput & memory
SequencePacking shares training windows across short samples; GradientAccumulation grows the effective batch without more memory; MicroBatchSize is the lever when a run does not fit the device.
Supervision & regularizers
AssistantLossOnly trains only on assistant tokens, with a mismatch signal so bad data surfaces. ValidationSplit reports held-out loss per epoch; NeftuneAlpha counters overfitting on small sets.
Checkpointing & resume
CheckpointDirectory and CheckpointSaveSteps write resumable step checkpoints; ResumeOptimizerPath continues an interrupted run; RequestStop cancels and still keeps the adapter trained so far.
Where local fine-tuning pays off.
Domain language
Adapt a base model to your domain's vocabulary: legal, medical, financial, scientific. Improve named-entity accuracy and instruction following without prompt engineering.
House style
Train an adapter that emits writing in your brand voice, with your editorial conventions, terminology, and tone.
Customer-facing chat
Fine-tune on your support transcripts for grounded, on-brand replies. Hot-swap a freshly-trained adapter daily without redeploying the base model.
Tool-call accuracy
Train against a corpus of (intent, function-call) pairs so the model rarely hallucinates tool names or arguments. Pair with grammar-constrained decoding for ironclad output.
Compliance training
Train on your team's redaction policies, privacy rules, and disclosure templates. Run entirely on premises so the policies themselves never leak.
Multi-tenant SaaS
Train a per-customer adapter on their data. Load adapters dynamically per request so each tenant gets a model that knows their patterns.
API reference.
LoraFinetuning
Main trainer. Load data, set the training window and micro-batch, checkpoint and resume. Subscribe to FinetuningProgress for live metrics.
LoraTrainingParameters
Rank, alpha, target modules, epochs, learning-rate schedule, warmup, validation split, and assistant-only loss.
TrainingDataset
Loads chat JSONL, ShareGPT, Alpaca, plain text, and ZIP archives with format auto-detection; exports back to ShareGPT.
ShareGptExporter
Convert collected ChatTrainingSamples into a ShareGPT-format JSON file for sharing or archival.
After training, see LoRA Integration for runtime hot-swap and Model Quantization to compress for deployment.
Build it. Read it. Try it.
Working console demos on GitHub, step-by-step how-to guides on the docs site, and the API reference for the classes used on this page.
How-to guide
Fine-tune a model with LoRA
The full training loop: data, hyperparameters, live metrics, checkpointing, artifacts.
Read the guide → How-to guideFine-tune vision models on image data
Train on labeled images through the model's vision path.
Read the guide → How-to guidePrepare training datasets for LoRA finetuning
Dataset formats, building, export, and quality controls for LoRA training.
Read the guide → SampleText-to-SQL fine-tuning (CLI)
Teach a model your database schema from a dataset file, then merge one deployable model.
Read on docs → SampleVision display reader fine-tuning (CLI)
Train a vision-language model to read seven-segment displays from labeled images.
Read on docs → How-to guideLoad and merge LoRA adapters
Hot-swap adapters at runtime without reloading the base model.
Read the guide →Train where the data lives.
No cloud uploads. No GPU rental. Your data, your model, your hardware.