← Back to AI Insights
Gemini Executive Synthesis

The Strata inference engine's stability and parameter handling, particularly for multimodal (image) inputs, when running Qwen model quantizations.

Technical Positioning
Strata aims for one-click installation and OpenAI/Anthropic API compatibility, supporting Qwen models with optional image input. The implicit positioning is reliable, performant local inference, including multimodal capabilities.
SaaS Insight & Market Implications
The Strata inference engine exhibits critical stability issues, manifesting as 'thinking loops' and excessive token generation in 25-33% of prompts. This problem is acutely pronounced with image inputs, where the engine's internal logic silently discards crucial sampling parameters (e.g., `temperature`, `top_p`) when requests carry embeddings. This design flaw prevents the application of recommended Qwen model settings, directly causing erratic and repetitive model outputs. While a specific `IQ3_S` quantization is a contributing factor, the fundamental issue for multimodal inputs lies in Strata's failure to honor user-configured generation parameters. This undermines Strata's value proposition as a reliable, user-friendly solution for Qwen models, especially for multimodal applications. The silent failure to apply critical settings represents a significant reliability and developer experience defect.
Proprietary Technical Taxonomy
thinking loops quant llama.cpp experimental-speed-projection RP system prompt Qwen recommended thinking settings temperature top_p

Raw Developer Origin & Technical Request

Source Icon GitHub Issue Sep 28, 2026
Repo: Niko1221/Strata
Frequent thinking loops.

Does anyone else experience this? I'm seeing a whole lot of thinking loops occur, and i mean, literally the same block of text repeated over and over. It's not a rare event either, so far, maybe 1/4 - 1/3 of my prompts end in a loop. That's with standard settings, with and without "experimental-speed-projection".

I've been running 3.8 next for some time now, in llama.cpp, and i haven't seen it loop once until now.

I'm not sure if it's an issue with Strata or the quant though. I'm running ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S version. In llama.cpp, i used unsloth's UD_Q3_K_XL and UD_IQ4_XS, currently using AtomicChat's 4.27bpw one and [this one](huggingface.co/Navin-Models/Qwen... haven't had issues with any of them.

The loops seemed to happen mostly when i was asking to describe or ask a question about images. I'm trying regular textual prompts right now, they complete correctly more frequently, but still loop sometimes.

I also tried to use an RP system prompt, and the responses looked really odd, just a constant spam of "Need maybe ... need maybe ... need maybe ..." in the thought process, and the final answers also felt quite a bit different from what i was seeing from this model before. Though that was with experimental-speed-projection thingy turned on for refusal reduction, so maybe it is to blame. With no prompt, or without projection, thinking looks normal (even if it loops).

I'll try to check ISTA-DASLab's...

Developer Debate & Comments

hendrikp • Sep 28, 2026
are you using the qwen recommended thinking settings? maybe also see , i saw too repetions in output without the recommended thinking setup - but they were also present in my llamacpp comparision (so not inference engine issue).
izMoff • Sep 28, 2026
> are you using the qwen recommended thinking settings? Which settings? I don't know, i'm using whichever settings Strata sets by default. Honestly, i'm not even sure how to change sampling parameters or anything here, i don't see them in the config.
hendrikp • Sep 28, 2026
They are in the gear symbols in the webui chat - a side menu will open These are the qwen recommended settings for thinkings: ``` { "temperature": 1.0, "top_p": 0.95, "top_k": 20, "min_p": 0.0, "presence_penalty": 0.0, "frequency_penalty": 0.0, "repetition_penalty": 1.0, } ``` And if youre using the api then inside the `/v1/chat/completions`
izMoff • Sep 28, 2026
Well, maybe the issue is with the quant, hard to say. I tried to load it up in llama.cpp, asked the same question about the same image, which sent Strata into loops multiple times, and although in llama.cpp it didn't loop exactly, it did spent 40k tokens thinking about more or less the same things, over and over. Which is also not something i had happen with aforementioned quants, at least not for so long, and not on a simple question like that.
jleaders • Sep 28, 2026
Two things I confirmed in the engine that may matter for the image-prompt loops, since image requests only ever take the text path's settings today: 1. A request with an image decodes **greedy, always**. `serve/server.py:279` drops the sampling keys whenever a request carries embeddings, and the engine's `GENI` parser skips everything but `cvec=` (`src/program/generate.cpp:2651`). So `temperature` / `top_p` / `top_k` on a vision request are accepted and then silently discarded — no warning, no `ERR`. 2. That also means the recommended Qwen thinking settings cannot be applied to image requests at all, which is consistent with the loops showing up mainly there. Not claiming it is the cause — the IQ3_S quant is still an untested suspect — but the fix on the engine side is small: parse and honor the same sampling keys on `GENI`, or answer `ERR` naming the unsupported field so it is never silently ignored. For reference, I measured the throughput side of the recommended set on the IQ1_M ...

Adjacent Repository Pain Points

Other highly discussed features and pain points extracted from Niko1221/Strata.

Top Replies
FF-AnthonyKaul • Sep 28, 2026
What is your CPU usage when this happens?
shpati • Sep 28, 2026
Interestingly half the CPU threads (8 of 16) work at 95-99%, whereas the other half are used very little or not at all by the engine.
FF-AnthonyKaul • Sep 28, 2026
That is the desired outcome since using all the threads ends up with a spin lock and would actually make things slower. You should be seeing higher than 30tps on your machine. Post your strata-q3_s...
Top Replies
Niko1221 • Sep 28, 2026
Thanks @jeremiahritchey - yes, I'm interested, and the numbers (49 s -> 2 s on the return to A) are exactly the problem agent users hit. There are now three efforts on the same state: **** (@midhat...
midhatn • Sep 28, 2026
@Niko1221 @jeremiahritchey @maedoc — yes, I am up for converging on this. One capture/restore core, 's RAM policy, and an optional NVMe tier is the direction I support. I am happy for to be superse...
maedoc • Sep 28, 2026
> would you be up for converging on this? yes ofc 👍 the design and most of implementation in was done with strata itself (a bit of help from glm-5.3), the design is a bit simpler minded than that,...
Top Replies
itguy327 • Sep 28, 2026
similar issue here
technerd-official • Sep 28, 2026
same here have a single 3090 with 64gb system ram and still cant get it to run. (ExpertCache: cudaMalloc failed: out of memory).
Niko1221 • Sep 28, 2026
Thanks @shanoor - this is fixed as far as the engine can fix it, in [v0.1.19](https://github.com/Niko1221/Strata/releases/tag/v0.1.19). **The cause** (not the 32 GB shared-memory figure; the engine...
Top Replies
Niko1221 • Sep 28, 2026
Thanks for the careful write-up - and good to hear about 2x decode on the 5080. **The selection budget is fixed, by the model, not by Strata.** `idx_top_k = 2048` comes from the GGUF (the model was...
enkynakamura • Sep 28, 2026
Sounds good, i will work on it and send a PR to judge. I will write my own needle harness rather than bother you for tools/needle_bench.py, which I could not find in the repo, and I will document t...
Niko1221 • Sep 28, 2026
@enkynakamura sorry, that harness was in my development tree only (and its prompts contain private notes, so they can't be published). [v0.1.17](https://github.com/Niko1221/Strata/releases/tag/v0.1...
Top Replies
itguy327 • Sep 28, 2026
fixed here https://github.com/Niko1221/Strata/issues/51
sal7g86 • Sep 28, 2026
Is it possible to use tensor parallelism or layer splitting on two GPUs?
itguy327 • Sep 28, 2026
sorry about this misunderstanding. I was having a separate multi gpu issue. +1 to this idea

Frequently Asked Questions

Market intelligence mapped to The Strata inference engine's stability and parameter handling, particularly for multimodal (image) inputs, when running Qwen model quantizations..

How is The Strata inference engine's stability and parameter handling, particularly for multimodal (image) inputs, when running Qwen model quantizations. positioned in the market?
Based on our AI analysis of the original developer request, its primary technical positioning is: Strata aims for one-click installation and OpenAI/Anthropic API compatibility, supporting Qwen models with optional image input. The implicit positioning is reliable, performant local inference, including multimodal capabilities.
Are engineers actively discussing The Strata inference engine's stability and parameter handling, particularly for multimodal (image) inputs, when running Qwen model quantizations.?
Yes, we have tracked 7 direct responses and active debates regarding this specific topic originating from GitHub Issue.
What are the foundational technologies related to The Strata inference engine's stability and parameter handling, particularly for multimodal (image) inputs, when running Qwen model quantizations.?
Our proprietary extraction maps The Strata inference engine's stability and parameter handling, particularly for multimodal (image) inputs, when running Qwen model quantizations. to adjacent architectural concepts including thinking loops, quant, llama.cpp, experimental-speed-projection.

Engagement Signals

7
Replies
open
Issue Status

Cross-Market Term Frequency

Quantifies the cross-market adoption of foundational terms like API and tokens by tracking occurrence frequency across active SaaS architectures and enterprise developer debates.