旅行者，让我教你用VITS合成提瓦特语音吧！(原神角色版技术指南)
【可莉的快乐开场】哒哒哒~一起来玩语音魔法吧！嗨嗨~我是西风骑士团的可莉！今天琴团长说"可莉要好好学习"，所以可莉要教大家一个超级有趣的魔法——VITS语音合成！就像可莉能用蹦蹦炸弹炸鱼一样(嘘...不要告诉琴团长)，你们也能用这个魔法创造出提瓦特大陆上任何角色的声音哦！
【钟离的沉稳解说】何谓VITS？
轻抿一口茶
据古籍记载，VITS(Variational Inference with adversarial learning for end-to-end Text-to-Speech)是一种基于深度学习的端到端语音合成技术。相较于传统TTS系统，它无需复杂的声学模型和声码器，直接通过文本生成自然流畅的语音。
其核心在于：
[*]变分自编码器(VAE)结构
[*]对抗训练机制
[*]蒙特卡洛flow-based时长预测
【温迪的诗意比喻】VITS如风般自由
啊~让我用诗歌来解释吧！VITS就像蒙德城自由的风：
[*]文本编码器是"前奏"，将文字化作音符
[*]先验编码器是"间奏"，捕捉语音的韵律
[*]解码器是"高潮"，把音符变成动听歌声
[*]判别器则是"观众"，确保演出足够真实
【雷电将军的威严命令】安装必备工具
"此乃永恒之需，务必严格执行"：
# 基础环境
conda create -n vits python=3.8
conda activate vits
pip install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0 -f https://download.pytorch.org/whl/cu113/torch_stable.html
# 核心依赖
pip install numpy scipy matplotlib pandas
pip install tensorboard librosa unidecode inflect【神里绫华的优雅示范】获取代码仓库
"白鹭霜华，代码如雪般纯净"：
git clone https://github.com/Plachtaa/VITS-fast-fine-tuning.git
pip install -r requirements.txt原神角色模型(请遵守相关版权规定)
访问【VITS-genshin】下载相关的模型,密码是8243。将整个目录下载下来放在源码目录下。
【魈的实战演示】合成你的第一段语音
"听令！"
import os
import torch
import utils
import commons
import logging
import soundfile as sf
from text import text_to_sequence
from models import SynthesizerTrn
from torch import no_grad, LongTensor
logger = logging.getLogger("jieba")
logger.setLevel(logging.WARNING)
limitation = False
hps_ms = utils.get_hparams_from_file('./configs/uma_trilingual.json')
device = torch.device("cpu")
net_g_ms = SynthesizerTrn(
len(hps_ms.symbols),
hps_ms.data.filter_length // 2 + 1,
hps_ms.train.segment_size // hps_ms.data.hop_length,
n_speakers=hps_ms.data.n_speakers,
**hps_ms.model)
_ = net_g_ms.eval().to(device)
model, optimizer, learning_rate, epochs = utils.load_checkpoint("./pretrained_models/G_trilingual.pth", net_g_ms, None)
def get_text(text, hps):
text_norm = text_to_sequence(text, hps.symbols, hps.data.text_cleaners)
if hps.data.add_blank:
text_norm = commons.intersperse(text_norm, 0)
text_norm = LongTensor(text_norm)
return text_norm
def vits(text, language, speaker_id, noise_scale, noise_scale_w, length_scale):
if not len(text):
return "输入文本不能为空！", None, None
text = text.replace('\n', ' ').replace('\r', '').replace(" ", "")
length = len(text)
if length > 100:
return "输入文字过长!", None, None
if language == 0:
text = "{}".format(text)
elif language == 1:
text = "{}".format(text)
stn_tst = get_text(text, hps_ms)
with no_grad():
x_tst = stn_tst.unsqueeze(0).to(device)
x_tst_lengths = LongTensor().to(device)
speaker_id = LongTensor().to(device)
audio = net_g_ms.infer(x_tst, x_tst_lengths, sid=speaker_id, noise_scale=noise_scale, noise_scale_w=noise_scale_w,
length_scale=length_scale).data.cpu().float().numpy()
return 22050, audio
if __name__ == "__main__":
speakers = {
"特别周": 0, "无声铃鹿": 1, "东海帝王": 2, "丸善斯基": 3, "富士奇迹": 4, "小栗帽": 5, "黄金船": 6, "伏特加": 7, "大和赤骥": 8, "大树快车": 9,
"草上飞": 10, "菱亚马逊": 11, "目白麦昆": 12, "神鹰": 13, "好歌剧": 14, "成田白仁": 15, "鲁道夫象征": 16, "气槽": 17, "爱丽数码": 18, "青云天空": 19,
"玉藻十字": 20, "美妙姿势": 21, "琵琶晨光": 22, "重炮": 23, "曼城茶座": 24, "美普波旁": 25, "目白雷恩": 26, "雪之美人": 28, "米浴": 29, "艾尼斯风神": 30,
"爱丽速子": 31, "爱慕织姬": 32, "稻荷一": 33, "胜利奖券": 34, "空中神宫": 35, "荣进闪耀": 36, "真机伶": 37, "川上公主": 38, "黄金城市": 39, "樱花进王": 40,
"采珠": 41, "新光风": 42, "东商变革": 43, "超级小溪": 44, "醒目飞鹰": 45, "荒漠英雄": 46, "东瀛佐敦": 47, "中山庆典": 48, "成田大进": 49, "西野花": 50,
"春乌拉拉": 51, "青竹回忆": 52, "待兼福来": 55, "名将怒涛": 57, "目白多伯": 58, "优秀素质": 59, "帝王光环": 60, "待兼诗歌剧": 61, "生野狄杜斯": 62,
"目白善信": 63, "大拓太阳神": 64, "双涡轮": 65, "里见光钻": 66, "北部玄驹": 67, "樱花千代王": 68, "天狼星象征": 69, "目白阿尔丹": 70, "八重无敌": 71,
"鹤丸刚志": 72, "目白光明": 73, "樱花桂冠": 74, "成田路": 75, "也文摄辉": 76, "真弓快车": 80, "骏川手纲": 81, "小林历奇": 83, "奇锐骏": 85, "秋川理事长": 86,
"綾地": 87, "因幡": 88, "椎葉": 89, "仮屋": 90, "戸隠": 91, "九条裟罗": 92, "芭芭拉": 93, "派蒙": 94, "荒泷一斗": 96, "早柚": 97, "香菱": 98, "神里绫华": 99,
"重云": 100, "流浪者": 102, "优菈": 103, "凝光": 105, "钟离": 106, "雷电将军": 107, "枫原万叶": 108, "赛诺": 109, "诺艾尔": 112, "八重神子": 113, "凯亚": 114,
"魈": 115, "托马": 116, "可莉": 117, "迪卢克": 120, "夜兰": 121, "鹿野院平藏": 123, "辛焱": 124, "丽莎": 125, "云堇": 126, "坎蒂丝": 127, "罗莎莉亚": 128,
"北斗": 129, "珊瑚宫心海": 132, "烟绯": 133, "久岐忍": 136, "宵宫": 139, "安柏": 143, "迪奥娜": 144, "班尼特": 146, "雷泽": 147, "阿贝多": 151, "温迪": 152,
"空": 153, "神里绫人": 154, "琴": 155, "艾尔海森": 156, "莫娜": 157, "妮露": 159, "胡桃": 160, "甘雨": 161, "纳西妲": 162, "刻晴": 165, "荧": 169, "埃洛伊": 179,
"柯莱": 182, "多莉": 184, "提纳里": 186, "砂糖": 188, "行秋": 190, "奥兹": 193, "五郎": 198, "达达利亚": 202, "七七": 207, "申鹤": 217, "莱依拉": 228, "菲谢尔": 230
}
speed = 1
output_dir = "output"
if not os.path.exists(output_dir):
os.makedirs(output_dir)
for k inspeakers:
id = speakers
print("Speaker:",k)
sr, audio = vits('你好,我是玛丽', 0, torch.tensor(), 0.1, 0.668, 1.0/speed)
filename = os.path.join(output_dir, "{}.wav".format(k))
sf.write(filename, audio, samplerate=sr)将以上代码复制到Python脚本中。运行即可看到如下生成的效果:
https://img2024.cnblogs.com/blog/3082360/202505/3082360-20250516110122088-761160819.png
最后来听下效果:
八重神子 东海帝王【全体角色的谢幕】一起创造提瓦特之声吧！
"愿风神护佑你的代码没有bug！"(温迪) "此即，智慧之殿堂。"(纳西妲) "代码如契约，必须严格执行。"(钟离) "哒哒哒~可莉要去写更多语音啦！"(可莉)
来源：程序园用户自行投稿发布，如果侵权，请联系站长删除
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！
页:
[1]