huggingface /parler-tts
Inference and training library for high-quality TTS models.
★ 5,597⑂ 590👁 56⑃ 70● 158▤ 0
+0
日增长
5,597
stars
590
forks
56
watchers
70
prs
158
issues
0
releases
README
Parler-TTS
Parler-TTS is a lightweight text-to-speech (TTS) model that can generate high-quality, natural sounding speech in the style of a given speaker (gender, pitch, speaking style, etc). It is a reproduction of work from the paper Natural language guidance of high-fidelity text-to-speech with synthetic annotations by Dan Lyth and Simon King, from Stability AI and Edinburgh University respectively.
Contrarily to other TTS models, Parler-TTS is a fully open-source release. All of the datasets, pre-processing, training code and weights are released publicly under permissive license, enabling the community to build on our work and develop their own powerful TTS models.
This repository contains the inference and training code for Parler-TTS. It is designed to accompany the Data-Speech repository for dataset annotation.
[!IMPORTANT]
08/08/2024: We are proud to release two new Parler-TTS checkpoints:
1. Parler-TTS Mini, an 880M parameter model.
2. Parler-TTS Large, a 2.3B parameter model.
These checkpoints have been trained on 45k hours of audiobook data.
In addition, the code is optimized for much faster generation: we've added SDPA and Flash Attention 2 compatibility, as well as the ability to compile the model.
📖 Quick Index
Installation
Usage
- 🎲 Using a random voice
- 🎯 Using a specific speaker
Training
Demo
Model weights and datasets
Optimizing inference
Installation
Parler-TTS has light-weight dependencies and can be installed in one line:
sh
pip install git+https://github.com/huggingface/parler-tts.git
Apple Silicon users will need to run a follow-up command to make use the nightly PyTorch (2.4) build for bfloat16 support:
sh
pip3 install --pre torch torchaudio --index-url https://download.pytorch.org/whl/nightly/cpu
Usage
[!TIP]
You can directly try it out in an interactive demo here!
Using Parler-TTS is as simple as "bonjour". Simply install the library once:
sh
pip install git+https://github.com/huggingface/parler-tts.git
🎲 Random voice
Parler-TTS has been trained to generate speech with features that can be controlled with a simple text prompt, for example:
py
import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf
device = "cuda:0" if torch.cuda.is_available() else "cpu"
model = ParlerTTSForConditionalGeneration.from_pretrained("parler-tts/parler-tts-mini-v1").to(device)
tokenizer = AutoTokenizer.from_pretrained("parler-tts/parler-tts-mini-v1")
prompt = "Hey, how are you doing today?"
description = "A female speaker delivers a slightly expressive and animated speech with a moderate speed and pitch. The recording is of very high quality, with the speaker's voice sounding clear and very close up."
input_ids = tokenizer(description, return_tensors="pt").input_ids.to(device)
prompt_input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
generation = model.generate(input_ids=input_ids, prompt_input_idTop issues
#139Voice Consistency Working Pretty Well -- Plus Zero-Shot Cloning!37 comments · 2024-09-21#128adding a new language15 comments · 2024-09-01#67sharing results of korean(+english) bilingual training15 comments · 2024-06-10#11Is there a way to create consistent voices?12 comments · 2024-04-11#31Zero-Shot Voice Cloning10 comments · 2024-04-21#168Using sdpa and flash_attention_2 error8 comments · 2024-11-22#55Flash Attention Support7 comments · 2024-05-23#151Missing argument error6 comments · 2024-10-23