par atul10
Open source · 232 downloads · 1 likes
Le modèle *Nepali Male v1* est un système de synthèse vocale (TTS) basé sur l'architecture VITS, spécialement conçu pour générer des voix masculines en népalais avec une grande expressivité. Il transforme du texte en parole naturelle en intégrant un encodeur de texte, un prédicteur de durée stochastique et un décodeur de spectrogrammes, le tout optimisé par des techniques d'apprentissage adversarial pour une qualité sonore réaliste. Grâce à son approche end-to-end, il capture les nuances de la prosodie et du rythme, permettant des variations naturelles pour un même texte d'entrée. Idéal pour des applications comme les assistants vocaux, les livres audio ou les outils d'accessibilité, il se distingue par sa capacité à produire des intonations variées tout en restant fidèle au contenu sémantique. Son entraînement multilingue (népalais et hindi) renforce sa polyvalence, tandis que son caractère non déterministe offre une flexibilité créative pour adapter la voix à différents contextes.
Nepali language
VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) is an end-to-end speech synthesis model that predicts a speech waveform conditional on an input text sequence. It is a conditional variational autoencoder (VAE) comprised of a posterior encoder, decoder, and conditional prior.
A set of spectrogram-based acoustic features are predicted by the flow-based module, which is formed of a Transformer-based text encoder and multiple coupling layers. The spectrogram is decoded using a stack of transposed convolutional layers, much in the same style as the HiFi-GAN vocoder. Motivated by the one-to-many nature of the TTS problem, where the same text input can be spoken in multiple ways, the model also includes a stochastic duration predictor, which allows the model to synthesise speech with different rhythms from the same input text.
The model is trained end-to-end with a combination of losses derived from variational lower bound and adversarial training. To improve the expressiveness of the model, normalizing flows are applied to the conditional prior distribution. During inference, the text encodings are up-sampled based on the duration prediction module, and then mapped into the waveform using a cascade of the flow module and HiFi-GAN decoder. Due to the stochastic nature of the duration predictor, the model is non-deterministic, and thus requires a fixed seed to generate the same speech waveform.
TTS is available in the 🤗 Transformers library from version 4.33 onwards. To use this checkpoint, first install the latest version of the library:
pip install --upgrade transformers accelerate
Then, run inference with the following code-snippet:
from transformers import VitsModel, AutoTokenizer
import torch
model = VitsModel.from_pretrained("procit001/nepali_male_v1")
tokenizer = AutoTokenizer.from_pretrained("procit001/nepali_male_v1")
text = "म पनि जान्छु है त अहिले लाई"
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
output = model(**inputs).waveform
The resulting waveform can be saved as a .wav file:
import scipy
scipy.io.wavfile.write("techno.wav", rate=model.config.sampling_rate, data=output)
Or displayed in a Jupyter Notebook / Google Colab:
from IPython.display import Audio
Audio(output, rate=model.config.sampling_rate)
The model is licensed as procitBV v1.
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.