rednote-hilab/dots.tts

推荐200字以内，按句子/段落/语义切分均可，以你的实际体验为准

> 我测试了1000字中文VRAM占用为8.8G（实际上并不建议直接合成这么长的文本）。以下是一些tips供参考：
>
> * 对于长文本，最好在合适位置做一下切分，直接合成超长文本效果会差；
> * 参考音频10s左右即可，长参考音频不会带来更好的效果；
>
> optimize报错，可以先检查一下当前环境是否满足[recommended.txt](https://github.com/rednote-hilab/dots.tts/blob/main/constraints/recommended.txt)或[pyproject.toml](https://github.com/rednote-hilab/dots.tts/blob/main/pyproject.toml)的要求。

1.“对于长文本，最好在合适位置做一下切分”，请问具体如何切分？是基于句子或段落切分，还是基于 token 数量切分？
2.参考音频都在10s内

环境安装也是没问题的，都是按照readme安装的

我测试了1000字中文VRAM占用为8.8G（实际上并不建议直接合成这么长的文本，效果基本不可用）。以下是一些tips供参考：

- 对于长文本，最好在合适位置做一下切分，直接合成超长文本效果会差；
- 参考音频10s左右即可，长参考音频不会带来更好的效果；

optimize报错，可以先检查一下当前环境是否满足[recommended.txt](https://github.com/rednote-hilab/dots.tts/blob/main/constraints/recommended.txt)或[pyproject.toml](https://github.com/rednote-hilab/dots.tts/blob/main/pyproject.toml)的要求。

Thanks for the suggestion, but currently we don't have plans to implement a pure C++ version.

Do you have plans to make this work in pure C++, like this: https://github.com/pwilkin/openmoss
It would be great, and inference could be much faster.
@xlians555

Since it's a TTS base model, there is no built-in default voice. The recommended usage is to perform voice cloning by providing reference audio and the corresponding reference text. Alternatively, you can fine-tune the model with your own voice dataset. We do not recommend generating speech via random sampling directly with the base model, as it cannot guarantee consistent voice timbre and satisfactory audio quality.
由于这是一个 TTS 基座模型，本身并不自带默认音色。推荐的使用方式是通过提供参考音频和对应的参考文本来进行语音克隆。或者，你也可以用自己的语音数据集对模型进行微调。我们不推荐直接通过随机采样的方式用基座模型生成语音，因为这样无法保证音色的一致性以及令人满意的音频质量。

好像是这个--seed 参数， 不晓得 ，这里默认带了哪些音色， 有这些介绍吗？ 我目前想要一个正常一点，试用于法律问题回答的 男生音色， 不晓得，选多少，默认42 是个女生

```python
from dots_tts.runtime import DotsTtsRuntime
import soundfile as sf
import torch

runtime = DotsTtsRuntime.from_pretrained(
"/path/to/dots_tts_model",
precision="bfloat16",
optimize=True,
)

# generate_stream yields audio chunks (torch.Tensor, shape (1, samples)) as they
# are produced — useful for low-latency playback or piping to a client.
stream = runtime.generate_stream(
text="Hello, this is a streaming speech synthesis test.",
prompt_audio_path="/path/to/reference.wav",
prompt_text="The exact transcript of the reference audio.",
num_steps=10,
guidance_scale=1.0,
)

# Option A: consume chunk-by-chunk (e.g. push to a player / websocket).
chunks = []
for chunk in stream:
chunks.append(chunk.detach().float().cpu())
# handle_chunk(chunk) # your real-time consumer here

# Option B: concatenate and write the full waveform to disk.
audio = torch.cat(chunks, dim=-1).squeeze().numpy()
sf.writ...

@xlians555 Is there any example of `generate_stream` ?

You can add the `--optimize` flag in current PyTorch version to boost inference speed. Our test results on H800 (voice clone mode, `generate_stream` interface, default inference setting): RTF is roughly 0.6 for Base/Soar and 0.4 for MF. For further speedup, you may cache the reference audio.

Haha, we did the same thing，https://github.com/vanch007/mlx-dots-tts

Thanks for your contribution. We'll update the README to mention your work soon.