Comment on: ZeroGPU
by [REDACTED]
Strong thesis, and it matches what I hit building in the wild. Most of my pipeline was never reasoning, it was "classify this comment into one of 7 stances" running hundreds of times per video. I prototyped it on an LLM (slow, per-call cost, single point of failure), then distilled it into a fine-tuned multilingual model exported to ONNX - cheap CPU box, deterministic, no API bill. Shipped it as PJQ (pjq.life). So your "80% of inference is routine, not reasoning" point isn't a forecast for me, it's already in prod. Curious where you draw the line between hosting a small model for the user vs someone bringing a task-specific fine-tune like mine - is the catalog fixed, or is BYO-model first-class?
About this Product
Other Comments / Reviews
SaaS Metrics