SaaS Metrics
GitHub Issue
Low t/s speed on a NVIDIA RTX 3080 with 16GB VRAM
Fist of all, thank you so much for the great job!
I am experiencing low inference speeds of around 17 t/s on the machine with the below specs and wonder if there are any settings that can be adjusted to increase the speed.
Machine specs:
Processor: Intel(R) Core(TM) i7-10870H CPU @ 2.20GHz (8 cores / 16 threads)
Installed RAM: 64 GB, DDR4-2666
GPU: NVIDIA GeForce RTX 3080 Laptop, 16GB GDDR6
SSD speed: 3400 MB/s (NVMe 4x 16 GT/s)
OS: Win11
Model: qwen3.8-flash-next-iq3_s
Engine: v0.1.18
Context: 65,536 tokens
KV cache: 8-bit, all in VRAM
Experts in VRAM: 3,219 (6.2 GB)
Speculation : MTP drafts up to 3 tokens, prompt lookup on
Images: on
View Raw Thread
Developer & User Discourse
FF-AnthonyKaul • Sep 28, 2026
What is your CPU usage when this happens?
shpati • Sep 28, 2026
Interestingly half the CPU threads (8 of 16) work at 95-99%, whereas the other half are used very little or not at all by the engine.
FF-AnthonyKaul • Sep 28, 2026
That is the desired outcome since using all the threads ends up with a spin lock and would actually make things slower. You should be seeing higher than 30tps on your machine.
Post your strata-q3_s.json args
Post your strata-q3_s.json args
shpati • Sep 28, 2026
```
{
"exe": "engine\\strata.exe",
"args": [
"--pack",
"..\\Strata-data\\packs\\iq3_s",
"--native",
"..\\Strata-data\\models\\IQ3_S\\Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf",
"--ple-gguf",
"..\\Strata-data\\models\\IQ3_S\\Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00002-of-00002.gguf",
"--expert-profile",
"data\\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--spec-min-p",
"0.5",
"--mtp",
"..\\Strata-data\\mtp\\rt",
"--max-context",
"65536",
"--kv",
"int8",
"--vision",
"--vram-reserve-mib",
"700",
"--control-vector-scaled",
"data\\experimental-speed-projection\\Qwen3.8-Flash-Next-experimental-speed-projection.gguf:1.0",
"--control-vector-layer-range",
"4",
"44",
"--cvec-mode",
"project",
"--cvec-dir",
"per-layer"
],
"cwd": ".",
"tokenizer": "..\\Strata-data\\packs\\iq3_s\\tokenizer",
"model_name": "qwen3.8-flash-next-iq3_s",
"log": "strata-iq3_s.log",
"lib_dirs": [
".ve...
{
"exe": "engine\\strata.exe",
"args": [
"--pack",
"..\\Strata-data\\packs\\iq3_s",
"--native",
"..\\Strata-data\\models\\IQ3_S\\Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf",
"--ple-gguf",
"..\\Strata-data\\models\\IQ3_S\\Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00002-of-00002.gguf",
"--expert-profile",
"data\\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--spec-min-p",
"0.5",
"--mtp",
"..\\Strata-data\\mtp\\rt",
"--max-context",
"65536",
"--kv",
"int8",
"--vision",
"--vram-reserve-mib",
"700",
"--control-vector-scaled",
"data\\experimental-speed-projection\\Qwen3.8-Flash-Next-experimental-speed-projection.gguf:1.0",
"--control-vector-layer-range",
"4",
"44",
"--cvec-mode",
"project",
"--cvec-dir",
"per-layer"
],
"cwd": ".",
"tokenizer": "..\\Strata-data\\packs\\iq3_s\\tokenizer",
"model_name": "qwen3.8-flash-next-iq3_s",
"log": "strata-iq3_s.log",
"lib_dirs": [
".ve...
FF-AnthonyKaul • Sep 28, 2026
can you add
"--pool-workers",
"4"
to your args and see what that does? Curious
"--pool-workers",
"4"
to your args and see what that does? Curious