SaaS Metrics
Gemini Executive Synthesis
Feature request for integrated local training capabilities (`ollaya train`) for System 1 decision models, specifically using LoRA, to allow users to fine-tune models on proprietary datasets.
Technical Positioning
Ollaya maintains its core identity as an "ultra-fast, calibrated in-process runtime" for inference, strictly avoiding bloat from training dependencies. The platform aims to facilitate the *running* of custom-trained models by defining an import path for LoRA adapters and calibration parameters, rather than integrating the training process itself. This ensures a lightweight core while supporting user customization.
SaaS Insight & Market Implications
This discussion highlights a fundamental tension in B2B SaaS: balancing core product focus with user demand for extended capabilities. Developers face significant pain points in training specialized models for proprietary data, currently relying on complex external pipelines. The proposal for `ollaya train` directly addresses this, aiming for rapid, local fine-tuning. However, Ollaya's executive decision to reject integrated training, prioritizing a "zero bloat" inference core, reveals a strategic commitment to its "ultra-fast, calibrated in-process runtime" identity. This stance, while potentially frustrating for users seeking an all-in-one solution, ensures the core product remains lean, performant, and stable. The market implication is a clear delineation: Ollaya will be the premier *runtime* for decision models, while training remains a separate, albeit supported, ecosystem. This forces users to integrate multiple tools but guarantees Ollaya's core value proposition.
Proprietary Technical Taxonomy
Raw Developer Origin & Technical Request
GitHub Issue
Sep 29, 2026
Repo: ollaya-dev/ollaya
FEATURE-REQUEST about integrated tooling for training (LoRa)
Today, Ollaya is an ultra-fast, calibrated in-process runtime for **System 1 decision models** (Laya, Winnow, Kev, Decider).
While Ollaya allows creating custom presets with `Modelfile` (`QUESTIONS ...`), users currently rely on pre-trained upstream models. If a company or developer wants to classify internal tickets, detect specific security vulnerabilities, or route proprietary domain logs, their only option today is to train a model externally using complex pipelines (Unsloth, Axolotl, PyTorch) and manually convert the weights.
**The Proposal:** Introduce a first-class, lightweight local training capability (**`ollaya train`**) allowing users to fine-tune a compact base model (0.4B–2B) on their own local dataset in **under 2 minutes** on a single GPU (NVIDIA CUDA or AMD ROCm), automatically fit the calibration parameters, and register the resulting model directly in `~/.ollaya/models/`.
With LLM is more complicated but for decisional we can use LoRa with little dataset .
I thinks that can help the comunity to create and share their own model, and maybe trained on big internal dataset.
i can provide PR if you are inerrested by this way.
Developer Debate & Comments
To follow up on this proposal, here are some thoughts on how the architecture could be cleanly designed **without bloating the core Ollaya runtime**: ### 1. Absolute Rule: Zero Bloat in Core Ollaya (`ollaya` & `ollaya-runner`) The core inference daemon must stay 100% lightweight, pure, and dependency-free: - **No heavy autograd frameworks (libtorch, PyTorch, CUDA/ROCm compilers)** linked into the release binaries. - Users who only run inference download the exact same compact binary and packages as today. ### 2. Proposed Decoupled Architecture We can keep training completely out-of-process, mirroring the clean pattern already established by `convert/` and `scripts/llama-cpp.sh`: * **Option A (Thin CLI Dispatcher + UV):** - The `ollaya` CLI only provides a thin command parser for `ollaya train`. - It delegates out-of-process to an isolated, UV-managed Python toolchain (`convert/ollaya_train/`). - When a user runs `ollaya train`, `uv` resolves the training dependencies (PyTorc...
@solarpush, thanks for the proposal and the follow-up on keeping the core lean. Sorry it took a while to answer. We have decided not to put training into Ollaya itself. Ollaya runs authors' models unmodified and gates each one on parity against the author's own code. A training loop is a different product, with its own dependencies, hardware matrix and support load, even as a separate binary. What fits is the other half: making it easy to run a model you trained elsewhere. `kev` already works this way: Jared Palmer's LoRA adapters run on the unmodified Qwen3.5 base. An import path for a LoRA adapter (plus its head or label tokens and its calibration) on a base Ollaya already runs would let any training tool (yours, Unsloth, Axolotl) end in a model Ollaya can load and check. If you would like to build the training side as its own project that targets such a format, we would be glad to link it from the docs, and the import format is something we can design together in this issue.
Adjacent Repository Pain Points
Other highly discussed features and pain points extracted from ollaya-dev/ollaya.
Extracted Positioning
Request for aarch64 CUDA support for Ollaya, specifically for NVIDIA GB10 GPUs on DGX Spark platforms.
Ollaya aims to provide GPU acceleration across diverse architectures, but prioritizes verification on real hardware. It leverages existing `llama.cpp` CUDA builds for GGUF models (e.g., `winnow`) as a realistic first step, while acknowledging the greater effort required for ONNX models due to lack of official aarch64 CUDA builds from Microsoft. The core principle is rigorous "parity gate" testing before shipping.
Top Replies
Good question. There are two engines, and aarch64 CUDA differs between them: - **GGUF models on llama.cpp** (`winnow`, `jevk5`): llama.cpp b11146, the build Ollaya pins, ships `ubuntu-cuda-13.4-arm...
I also have a DGX Spark (aarch64, GB10) and would be happy to help test the ARM64 CUDA pack if needed. Share the build and parity-check commands, and I can run them and report the results and logs.
Thanks @drsect0r and @Xe, that helps a lot. Here's the parity check for `winnow:e4b` on llama.cpp's CUDA backend on a GB10. It's the same gate every device passes: Ollaya's runner against a stock `...
Extracted Positioning
Integration of `snap1-2b` (MiniCPM5-2B + LoRA) into Ollaya, leveraging GGUF format for efficient deployment.
Achieving high accuracy (0.655-0.660) with a compact 2B parameter model, optimized for one-forward-pass decision-making. Strict adherence to prompt versioning and byte-for-byte parity checks ensures consistent, reliable model behavior across different environments. Prioritizing Q8_0 quantization for broader compatibility while offering Q4_K_M for smaller machines.
Top Replies
@emnlmn, thanks, this is a clear proposal and it fits the GGUF path well. Like Cygnet, it reads letter logits after one prompt, so it is one forward pass per question. We would like to add it. How ...
@cobanov thanks, this plan works for me. Updates since your reply: - **Quantization**: we measured Q8_0 against Q4_K_M on typed-decisions (snap 0.5.0, M1 Max): 0.655 vs 0.654 accuracy, so Q4_K_M lo...
@emnlmn, thanks for the numbers and the v0.5.0 notes. Plan: - **Prompt.** We port v0.5.0 (`PROMPT_VERSION` 6). - **Quantization.** The default tag will be Q8_0: our rule is Q8_0 wherever it fits, a...
Extracted Positioning
Integration of Decima, a 122M parameter System One decision model, into the Ollaya library.
Targeting low-end CPU inference with high performance (~20 ms per decision on one x86 core, int8), supporting TypeSafe API for `choice`, `noul`, and `score` questions. Emphasizes local, efficient decision model serving.
Frequently Asked Questions
Market intelligence mapped to Feature request for integrated local training capabilities (`ollaya train`) for System 1 decision models, specifically using LoRA, to allow users to fine-tune models on proprietary datasets..
What problem does Feature request for integrated local training capabilities (`ollaya train`) for System 1 decision models, specifically using LoRA, to allow users to fine-tune models on proprietary datasets. solve?
Based on our AI analysis of the original developer request, its primary technical positioning is: Ollaya maintains its core identity as an "ultra-fast, calibrated in-process runtime" for inference, strictly avoiding bloat from training dependencies. The platform aims to facilitate the *running* of custom-trained models by defining an import path for LoRA adapters and calibration parameters, rather than integrating the training process itself. This ensures a lightweight core while supporting user customization.
Are engineers actively discussing Feature request for integrated local training capabilities (`ollaya train`) for System 1 decision models, specifically using LoRA, to allow users to fine-tune models on proprietary datasets.?
Yes, we have tracked 2 direct responses and active debates regarding this specific topic originating from GitHub Issue.
Which technical concepts are associated with Feature request for integrated local training capabilities (`ollaya train`) for System 1 decision models, specifically using LoRA, to allow users to fine-tune models on proprietary datasets.?
Our proprietary extraction maps Feature request for integrated local training capabilities (`ollaya train`) for System 1 decision models, specifically using LoRA, to allow users to fine-tune models on proprietary datasets. to adjacent architectural concepts including System 1 decision models, in-process runtime, Modelfile, pre-trained upstream models.
Engagement Signals
Cross-Market Term Frequency
Quantifies the cross-market adoption of foundational terms like calibration and fine-tune by tracking occurrence frequency across active SaaS architectures and enterprise developer debates.