FastRouter.ai

Hey @ritprasad Congrats on the launch.How is it any different from OpenRouter? Any key differences you have implemented or plan to implement in the future?

Cost, latency and quality is three knobs where the third one does all the work and is the hardest to measure. We route across 30+ models in our own product and the thing that bit us wasn't a slow call or an expensive one, it was a small model returning something clean, confident and wrong, which passes every format check you have. So the number I'd want out of a control plane is cost per accepted output, not cost per call, because a model that's 4x cheaper and needs two regenerations just moved the bill somewhere nobody is measuring. If the quality signal behind routing is benchmarks rather than what your own users kept, it'll pick the cheap model at exactly the wrong moments.

Congrats Ritesh ! Keep going : )

congrats on the launch. my biggest headache is latency and failover at the same time, since we work with voice. once audio starts playing you can't quietly retry, so the fallback has to be decided before the first byte goes out. does your routing optimize for time to first token and p95 rather than the average? and if a provider dies mid stream, do you restart on another upstream or just surface the error?

Awesome product, congrats with a launch and good luck!

Really like the failover functionality. If a provider becomes unavailable or throttles during the request process, how quickly is it done and is the streaming capability and tool calls maintained for all the models?Btw, Congratulations @ritprasad , @andrej_gamser2 and team FastRouter 🚀✌️

grouping retries under the original request is useful. if a stream fails halfway through, can failover continue it or does the user get a new response from scratch?

Many congratulations, @ritprasad , @andrej_gamser2 and team! 😊

Routing across 200 plus models through one OpenAI compatible endpoint is the kind of thing teams only appreciate once they have written the same retry and fallback logic by hand. Optimizing for cost, latency and quality together is the interesting part, since those usually pull against each other. How transparent is the routing decision when you want to debug why a request went a certain way?

How quickly does it react when a model suddenly gets slower or starts giving weaker results?Congrats @ritprasad & team!Thanks for hunting another cool product @rohanrecommends :)

​am sending this straight to our engineering lead we desperately need something like this right now.

The first users usually seem to come from places where the problem is already being discusses. Curious if anyone has had better results from communities than from posting on their own profiles.

​love the idea of a single endpoint for all 200+ models. makes testing new releases so much cleaner.

Can devs prioritise which criteria they want to optimise for: cost, latency, quality, and reliability at the routing layer?

​really smooth execution. being able to set rate limits across multiple models in one place saves a lot of hassle.