Skip to Content
Introduction

Onde Inference

Run LLMs on-device. Onde is an inference engine with first-class support for Apple silicon , built so chat runs fully on-device — no server round trip, no user data leaving the device.

This site is the technical documentation: the CLI pipeline for fine-tuning and publishing your own models, the SDKs you embed in an app, and how a published model gets deployed to end users through GresIQ. For product marketing and per-platform quickstarts, see ondeinference.com/sdk .

Where to go next

Fine-tune a LoRA adapter, merge it, export GGUF, test it locally, and publish it.

Onde CLI

Embed the inference engine in a Rust, Swift, Kotlin Multiplatform, or Flutter app.

SDK

How a model in the catalog gets assigned to an app and picked up by the SDK.

GresIQ & deployment

The pieces

  • onde — the inference engine crate (vendored mistral.rs ) and the source of truth for every SDK.
  • onde-cli — a Rust TUI (onde binary) for the local pipeline: LoRA fine-tune → adapter merge → GGUF export → local chat test → Hugging Face upload → model assignment to an app. Also runnable as an MCP server (onde --mcp).
  • SDKs — thin, idiomatic wrappers around the engine for Swift , Kotlin Multiplatform (Maven Central ), Flutter , and React Native .
  • GresIQ — the model catalog and app-assignment layer. Publishing a model makes it available in the catalog; assigning it to an app is what an end app’s SDK actually downloads.

License

Onde is dual-licensed under MIT and Apache 2.0 — use it under either license at your option. Models you load have their own, separate licenses; see the onde repository  for the current list and their commercial-use conditions.

Overview