Insight for: The inference speed is very slow

Slow inference speed (RTF > 2) on L40 GPU for dots.tts.

Analyzed: Jun 14, 2026

This issue directly addresses the slow inference speed of dots.tts, with a reported RTF exceeding 2 on an L40 GPU, significantly below competitive benchmarks (0.6 for Base/Soar, 0.4 for MF on H800 with `optimize`). This performance deficit is a critical barrier for real-time applications and high-throughput environments. While the `optimize` flag and `generate_stream` are suggested, the core problem remains the baseline performance. The request for a C++ implementation underscores the user's need for fundamental speed improvements. Slow RTF directly impacts operational costs and user experience, making dots.tts less attractive for demanding enterprise use cases.

inference speed RTF L40 GPU benchmark RTF optimize flag H800 voice clone mode generate_stream interface Base/Soar MF cache the reference audio pure C++ version

GitHub Issue

Parent Entity

The inference speed is very slow

State: Open