Slow inference speed (RTF > 2) on L40 GPU for dots.tts.

You can add the `--optimize` flag in current PyTorch version to boost inference speed. Our test results on H800 (voice clone mode, `generate_stream` interface, default inference setting): RTF is roughly 0.6 for Base/Soar and 0.4 for MF. For further speedup, you may cache the reference audio.

@xlians555 Is there any example of `generate_stream` ?

```python from dots_tts.runtime import DotsTtsRuntime import soundfile as sf import torch runtime = DotsTtsRuntime.from_pretrained( "/path/to/dots_tts_model", precision="bfloat16", optimize=True, ) # generate_stream yields audio chunks (torch.Tensor, shape (1, samples)) as they # are produced — useful for low-latency playback or piping to a client. stream = runtime.generate_stream( text="Hello, this is a streaming speech synthesis test.", prompt_audio_path="/path/to/reference.wav", prompt_text="The exact transcript of the reference audio.", num_steps=10, guidance_scale=1.0, ) # Option A: consume chunk-by-chunk (e.g. push to a player / websocket). chunks = [] for chunk in stream: chunks.append(chunk.detach().float().cpu()) # handle_chunk(chunk) # your real-time consumer here # Option B: concatenate and write the full waveform to disk. audio = torch.cat(chunks, dim=-1).squeeze().numpy() sf.writ...

Do you have plans to make this work in pure C++, like this: https://github.com/pwilkin/openmoss It would be great, and inference could be much faster. @xlians555

Thanks for the suggestion, but currently we don't have plans to implement a pure C++ version.