k2-fsa/OmniVoice

There seem to be many duplicate issues regarding stress control in Russian. I will close most of them and keep only the most recent one: https://github.com/k2-fsa/OmniVoice/issues/129

If you use letters in CAPS you will get bad audio outputs as if the speaker is drunk. Use small case or sentence case inputs to mitigate that as much as possible. Also I use audio references under 15 seconds for stable results. If you use audio over 30 seconds and inputs over 275 characters or about 48 words standard it may go bad too. Try to keep it under 45 words give or take while remaining under 275 characters total. (Including spaces)

I recommend using a text normalization tool to convert digits into words. The model can handle simple Chinese and English digits, as it has seen some of these patterns during training. However, Turkish training data is very limited, so digits are hard to process correctly.
You mentioned the English pronunciation sounds weird. I guess this is because the Turkish reference audio gives the generated English speech a Turkish accent. The model will handle your English text properly if you use voice design mode or an English reference audio instead.

For more robust digit handling, text normalization is standard practice for TTS models. For Chinese and English, you can use [WeTextProcessing](https://github.com/wenet-e2e/WeTextProcessing); for other languages, you’ll need to find a suitable tool yourself.

You could cache the reference audio encoded into the model's codebook representation but it's not much of a saving.

@Redtash1 no, since they are not providing a commercial product or a service the clause doesn't apply to them. But it might apply to users who want to build a commercial product and/or service built on top of OmniVoice

As of right now there is 104,915 downloads, so wouldn't they need the "special commercial license".

For everyone context: simplified by AI

This is a community license agreement for **Higgs Audio 2**, an audio model created by Boson AI that is built on top of Meta’s **Llama 3**.

Because it is derived from Llama 3, you must follow both this agreement **and** the [Meta Llama 3 Community License](https://llama.meta.com/llama3/license/).

Here is a simplified breakdown of what you can and cannot do:

### 1. The Basic Permission
You are granted a free, worldwide, non-transferable license to use, copy, modify, and distribute the Higgs Materials (the model, code, and weights).

### 2. The Big Condition: The 100,000 User Limit
If your product or service has more than **100,000 annual active users**, you are **not** allowed to use this model for free. You must contact Boson AI to request a special commercial license.

### 3. Requirements for Sharing (Redistribution)
If you share the model or build a product using it, you must:
* **Include Licenses:** Provide a copy of this agreement and t...

https://github.com/k2-fsa/OmniVoice/issues/37

https://github.com/k2-fsa/OmniVoice/issues/44

As far as I understand, the nature of the model is such that there exists no well defined internal artifact representing a voice. So all you can really do is use the same reference audio file over and over again

I haven't noticed this issue, unless my audio is already getting trimmed without me realizing? I can give it 5 minutes of audio and it still sounds fine, no instability.

Когда моделька училась в датасете скорее всего не было такого набора. Учитывая её размер, энтузиасты за вменяемый ценник могут дообучить её и получить, что-то типа "F5-TTS_RUSSIAN" от Misha24-10, тем более у него и датасет есть.

Как я понял многое зависит от референса, помимо всего, голос может быть один, вот в зависимости от отрывка (5-8 секунд) вывод меняется, я пробовал разные отрывки и на одном из пяти, вывод вполне приемлемый. Т.ч., пробуйте. У меня ударение ставит в 2 случаях из 3, если поставить спец символ, не знаю, меня результат устраивает, искажений нет.

Saving a used sample into a /samples folder, with a config, and a dropdown would be a good idea for the demo project.
If you are running this yourself outside of the UI, you would set up these configs/scripts yourself with the settings you need. But for many people, the Demo UI is still the most accessible option.

I could try implementing something like this if it would be of interest to the maintainer.

> in my opinion, you should implement a feature in the UI that automatically trims the reference audio to the recommended 6 seconds. This would ensure better results without requiring the user to edit their files manually, or you can add a auto trim or manual trim button.

It's a good idea, but implement with care I guess. If it's auto-cropped, the included sample text must also be editable by the user. So make sure that the cropped audio can be listened to. Also, I would see both of these best as being options, since maybe future training would make 60 second or more of data even better.

However, still keeping in mind that it's a demo, so some limitations can be there, as long as the result showcase the possibilities well.