babysor /MockingBird
🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time
★ 36,865⑂ 5,148👁 298⑃ 102● 889▤ 1
-1
日增长
-6
周增长
-23
月增长
36,865
stars
5,148
forks
298
watchers
102
prs
889
issues
1
releases
README
🚧 While I no longer actively update this repo, you can find me continuously pushing this tech forward to good side and open-source. I'm also building an optimized and cloud hosted version: https://noiz.ai/ and we're hiring.
!mockingbird
English | 中文| 中文Linux
Features
🌍 Chinese supported mandarin and tested with multiple datasets: aidatatang_200zh, magicdata, aishell3, data_aishell, and etc.
🤩 PyTorch worked for pytorch, tested in version of 1.9.0(latest in August 2021), with GPU Tesla T4 and GTX 2060
🌍 Windows + Linux run in both Windows OS and linux OS (even in M1 MACOS)
🤩 Easy & Awesome effect with only newly-trained synthesizer, by reusing the pretrained encoder/vocoder
🌍 Webserver Ready to serve your result with remote calling
DEMO VIDEO
Quick Start
1. Install Requirements
1.1 General Setup
Follow the original repo to test if you got all environment ready.
Python 3.7 or higher is needed to run the toolbox.
Install PyTorch.
If you get an ERROR: Could not find a version that satisfies the requirement torch==1.9.0+cu102 (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2 ) This error is probably due to a low version of python, try using 3.9 and it will install successfully
Install ffmpeg.
Run pip install -r requirements.txt to install the remaining necessary packages.
The recommended environment here is Repo Tag 0.0.1 Pytorch1.9.0 with Torchvision0.10.0 and cudatoolkit10.2 requirements.txt webrtcvad-wheels because requirements. txt was exported a few months ago, so it doesn't work with newer versions
Install webrtcvad pip install webrtcvad-wheels(If you need)
or
- install dependencies with conda or mamba
conda env create -n env_name -f env.yml
mamba env create -n env_name -f env.yml
will create a virtual environment where necessary dependencies are installed. Switch to the new environment by conda activate env_name and enjoy it.
env.yml only includes the necessary dependencies to run the project,temporarily without monotonic-align. You can check the official website to install the GPU version of pytorch.
1.2 Setup with a M1 Mac
The following steps are a workaround to directly use the original demo_toolbox.pywithout the changing of codes.
Since the major issue comes with the PyQt5 packages used in demo_toolbox.py not compatible with M1 chips, were one to attempt on training models with the M1 chip, either that person can forgo demo_toolbox.py, or one can try the web.py in the project.
1.2.1 Install PyQt5, with ref here.
Create and open a Rosetta Terminal, with ref here.
Use system Python to create a virtual environment for the project
/usr/bin/python3 -m venv /PathToMockingBird/venv
source /PathToMockingBird/venv/bin/activate
Upgrade pip and install PyQt5
pip install --upgrade pip
pip install pyqt5
1.2.2 Install pyworld and ctc-segmentation
Both packages seem to be unique to this project and are not seen in the original Real-Time Voice Cloning project. Top issues
#20保姆级别教程(持续更新各类社区/非官方教程----145 comments · 2021-08-19#10想要支持更多数据集?在这里提建议40 comments · 2021-08-16#423新的ppg模式相关问题29 comments · 2022-03-05#165来贡献模型了,aishell3的24 comments · 2021-10-23#345模型声音问题23 comments · 2022-01-16#765分享合成器模型22 comments · 2022-10-11#314运行demo_toolbox后不能部分功能用不了报错8 comments · 2022-01-04#239分享600kstep模型20 comments · 2021-11-28
Discussions
#1000[NEW RELEASE] Introducing MARS5, open-source, insanely prosodic text-to-speech (TTS) model.0 comments · 2024-06-10#986PydanticImportError: `pydantic:parse_raw_as` has been removed in V2. For further information visit https://errors.pydantic.dev/2.6/u/import-error。运行web.py说Pydantic版本有问题,怎么解决0 comments · 2024-02-02#970请问日志中的Epoch: x/y 是什么意思?是否支持在命令行中指定一个训练步数,训练完成后停止训练?1 comments · 2023-11-21#959I found after added user library for postman API use, the results of voice synthesis are not at all like the user's voice0 comments · 2023-09-18#941请问语音合成中编码,声码,合成器的基本概念是什么?0 comments · 2023-07-28#935master程序运行错误1 comments · 2023-07-10#907ModuleNotFoundError: No module named 'streamlit'1 comments · 2023-05-31#905请问这个项目需要多高的硬件支持0 comments · 2023-05-29