← Back to AI Insights
Gemini Executive Synthesis

Integration of Decima, a 122M parameter System One decision model, into the Ollaya library.

Technical Positioning
Targeting low-end CPU inference with high performance (~20 ms per decision on one x86 core, int8), supporting TypeSafe API for `choice`, `noul`, and `score` questions. Emphasizes local, efficient decision model serving.
SaaS Insight & Market Implications
This request highlights a critical market demand for highly efficient, specialized decision models deployable on minimal hardware. The Decima model, at 122M parameters and 20ms inference on a single x86 core, directly addresses the pain point of resource-constrained environments and edge deployments. Its TypeSafe API compatibility ensures seamless integration into existing enterprise systems. The explicit mention of "not Laya-compatible" architecture indicates potential fragmentation or specialized requirements within the decision model ecosystem, forcing developers to manage diverse model types. Ollaya's strategy to support such compact, performant models positions it as a key enabler for localized, real-time decision-making, crucial for applications where latency and infrastructure costs are paramount.
Proprietary Technical Taxonomy
System One model 122M parameters x86 core int8 TypeSafe PyTorch checkpoint safetensors fine-tuned encoder

Raw Developer Origin & Technical Request

Source Icon GitHub Issue Oct 3, 2026
Repo: ollaya-dev/ollaya
Support Decima (amyrmahdy/decima-small): a 122M System One model for CPU

Hi! I'd like to request support for **Decima**, a small open decision model I trained. It answers TypeSafe `choice`, `noul` and `score` questions and is meant for the low end of the library: 122M parameters, ~20 ms per decision on one x86 core (int8).

**Upstream**
- Weights: [huggingface.co/amyrmahdy/decima-small](huggingface.co/amyrmahdy/decima-... tag `v1.1.1` (commit `2e7f4d0`). The PyTorch checkpoint is in `pytorch/`: the fine-tuned encoder (`encoder/model.safetensors` + config + tokenizer), the scoring head (`head.safetensors`) and `decima.json`, which holds the layout and the fitted temperature.
- Reference code: [github.com/amyrmahdy/decima](github.com/amyrmahdy/decima (tag `v1.1.1`).
- `decima/model.py` is the PyTorch reference.
- `decima/systemone.py` is the TypeSafe mapping, also served at `/v1/systemone` by `python -m decima.serve`.
- License: Apache-2.0. Base encoder: `intfloat/multilingual-e5-small` (MIT).

**Architecture (not Laya-compatible)**
- A late-interaction scorer. The state (with the question) is encoded once. Each option is encoded separately and cross-attends to the state's token states.
- One logit per option, and the options never see each other, so permuting them permutes the probabilities exactly (0 % flips).
- `score` uses an ordinal head. `p = softmax(s / T)` with a single temperature.
- Budgets: state 512 tokens, option 64 tokens. A state over budget returns 422 STATE_TRUNCATED rather than being truncated.

**Measured*...

Developer Debate & Comments

No active discussions extracted for this entry yet.

Adjacent Repository Pain Points

Other highly discussed features and pain points extracted from ollaya-dev/ollaya.

Extracted Positioning
Request for aarch64 CUDA support for Ollaya, specifically for NVIDIA GB10 GPUs on DGX Spark platforms.
Ollaya aims to provide GPU acceleration across diverse architectures, but prioritizes verification on real hardware. It leverages existing `llama.cpp` CUDA builds for GGUF models (e.g., `winnow`) as a realistic first step, while acknowledging the greater effort required for ONNX models due to lack of official aarch64 CUDA builds from Microsoft. The core principle is rigorous "parity gate" testing before shipping.
Top Replies
cobanov • Sep 29, 2026
Good question. There are two engines, and aarch64 CUDA differs between them: - **GGUF models on llama.cpp** (`winnow`, `jevk5`): llama.cpp b11146, the build Ollaya pins, ships `ubuntu-cuda-13.4-arm...
drsect0r • Sep 29, 2026
I also have a DGX Spark (aarch64, GB10) and would be happy to help test the ARM64 CUDA pack if needed. Share the build and parity-check commands, and I can run them and report the results and logs.
cobanov • Sep 29, 2026
Thanks @drsect0r and @Xe, that helps a lot. Here's the parity check for `winnow:e4b` on llama.cpp's CUDA backend on a GB10. It's the same gate every device passes: Ollaya's runner against a stock `...
Extracted Positioning
Integration of `snap1-2b` (MiniCPM5-2B + LoRA) into Ollaya, leveraging GGUF format for efficient deployment.
Achieving high accuracy (0.655-0.660) with a compact 2B parameter model, optimized for one-forward-pass decision-making. Strict adherence to prompt versioning and byte-for-byte parity checks ensures consistent, reliable model behavior across different environments. Prioritizing Q8_0 quantization for broader compatibility while offering Q4_K_M for smaller machines.
Top Replies
cobanov • Oct 2, 2026
@emnlmn, thanks, this is a clear proposal and it fits the GGUF path well. Like Cygnet, it reads letter logits after one prompt, so it is one forward pass per question. We would like to add it. How ...
emnlmn • Oct 2, 2026
@cobanov thanks, this plan works for me. Updates since your reply: - **Quantization**: we measured Q8_0 against Q4_K_M on typed-decisions (snap 0.5.0, M1 Max): 0.655 vs 0.654 accuracy, so Q4_K_M lo...
cobanov • Oct 3, 2026
@emnlmn, thanks for the numbers and the v0.5.0 notes. Plan: - **Prompt.** We port v0.5.0 (`PROMPT_VERSION` 6). - **Quantization.** The default tag will be Q8_0: our rule is Q8_0 wherever it fits, a...
Extracted Positioning
Feature request for integrated local training capabilities (`ollaya train`) for System 1 decision models, specifically using LoRA, to allow users to fine-tune models on proprietary datasets.
Ollaya maintains its core identity as an "ultra-fast, calibrated in-process runtime" for inference, strictly avoiding bloat from training dependencies. The platform aims to facilitate the *running* of custom-trained models by defining an import path for LoRA adapters and calibration parameters, rather than integrating the training process itself. This ensures a lightweight core while supporting user customization.
Top Replies
solarpush • Sep 29, 2026
To follow up on this proposal, here are some thoughts on how the architecture could be cleanly designed **without bloating the core Ollaya runtime**: ### 1. Absolute Rule: Zero Bloat in Core Ollaya...
cobanov • Oct 2, 2026
@solarpush, thanks for the proposal and the follow-up on keeping the core lean. Sorry it took a while to answer. We have decided not to put training into Ollaya itself. Ollaya runs authors' models ...

Frequently Asked Questions

Market intelligence mapped to Integration of Decima, a 122M parameter System One decision model, into the Ollaya library..

What is the technical positioning of Integration of Decima, a 122M parameter System One decision model, into the Ollaya library.?
Based on our AI analysis of the original developer request, its primary technical positioning is: Targeting low-end CPU inference with high performance (~20 ms per decision on one x86 core, int8), supporting TypeSafe API for `choice`, `noul`, and `score` questions. Emphasizes local, efficient decision model serving.
How is the developer community reacting to Integration of Decima, a 122M parameter System One decision model, into the Ollaya library.?
Yes, we have tracked 1 direct responses and active debates regarding this specific topic originating from GitHub Issue.
Which technical concepts are associated with Integration of Decima, a 122M parameter System One decision model, into the Ollaya library.?
Our proprietary extraction maps Integration of Decima, a 122M parameter System One decision model, into the Ollaya library. to adjacent architectural concepts including System One model, 122M parameters, x86 core, int8.

Engagement Signals

1
Replies
open
Issue Status

Cross-Market Term Frequency

Quantifies the cross-market adoption of foundational terms like MIT license and int8 by tracking occurrence frequency across active SaaS architectures and enterprise developer debates.