Fine-tune → GGUF pipeline
The CLI walks a base model through five stages, each a Screen in the TUI:
- LoRA fine-tune — a hand-written trainer (candle) trains LoRA A/B
adapters on the q/v projections of a Qwen model against your dataset,
writing
lora_adapter.safetensors. - Merge — folds the adapter into the base weights
(
W + scale·(B@A)), writingmodel.safetensorsplus the config and tokenizer alongside it. - GGUF export — a hand-rolled GGUF writer (no llama.cpp dependency) quantizes and packages the merged model.
- Local chat test — runs the exported GGUF through the same inference
engine (
onde::mistralrs) the SDKs use, so you can sanity-check the model before publishing it. - Publish — uploads the GGUF to Hugging Face and assigns it to an app through GresIQ. See Publish & deploy.
Fine-tuning always runs on CPU
Metal’s backward pass has been observed to produce corrupt gradients on Apple silicon, so LoRA training runs on CPU regardless of platform. This is a deliberate correctness tradeoff, not a missing feature.
Per-project workspaces
Each fine-tune run lives in its own workspace under ~/.onde, so you can have
several in-flight projects (different datasets, different base models)
without them colliding.