Skip to Content
CLIFine-tune → GGUF pipeline

Fine-tune → GGUF pipeline

The CLI walks a base model through five stages, each a Screen in the TUI:

  1. LoRA fine-tune — a hand-written trainer (candle) trains LoRA A/B adapters on the q/v projections of a Qwen model against your dataset, writing lora_adapter.safetensors.
  2. Merge — folds the adapter into the base weights (W + scale·(B@A)), writing model.safetensors plus the config and tokenizer alongside it.
  3. GGUF export — a hand-rolled GGUF writer (no llama.cpp dependency) quantizes and packages the merged model.
  4. Local chat test — runs the exported GGUF through the same inference engine (onde::mistralrs) the SDKs use, so you can sanity-check the model before publishing it.
  5. Publish — uploads the GGUF to Hugging Face and assigns it to an app through GresIQ. See Publish & deploy.

Fine-tuning always runs on CPU

Metal’s backward pass has been observed to produce corrupt gradients on Apple silicon, so LoRA training runs on CPU regardless of platform. This is a deliberate correctness tradeoff, not a missing feature.

Per-project workspaces

Each fine-tune run lives in its own workspace under ~/.onde, so you can have several in-flight projects (different datasets, different base models) without them colliding.