Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
744ff26932 | ||
|
|
0d5f304eb0 | ||
|
|
533a49a0bd | ||
|
|
1cc06169f2 | ||
|
|
a377d89dcd | ||
|
|
d687121e99 | ||
|
|
badf6fe3d0 | ||
|
|
adb4a15f2d | ||
|
|
a5f9a782d6 |
No files matched your search
@@ -212,7 +212,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
|
||||
|
||||
| 工具名称 | 位置 | 使用方法 | 功能说明 |
|
||||
|:---:|:---|:---:|:---:|
|
||||
| 音频交互测试工具 | main》digital-human》index.html | 在 `main/digital-human` 执行 `python start.py` 后访问 `http://127.0.0.1:8006/index.html` | 测试音频播放和接收功能,验证Python端音频处理是否正常 |
|
||||
| 音频交互测试工具 | main》digital-human》index.html | 在 `main/digital-human` 按其 README 装好依赖后执行 `python start.py`,然后访问 `http://127.0.0.1:8006/index.html` | 测试音频播放和接收功能,验证Python端音频处理是否正常 |
|
||||
| 模型响应测试工具 | main》xiaozhi-server》performance_tester.py | 执行 `python performance_tester.py` | 测试ASR(语音识别)、LLM(大模型)、VLLM(视觉模型)、TTS(语音合成)等核心模块的响应速度 |
|
||||
|
||||
> 💡 提示:测试模型速度时,只会测试配置了密钥的模型。
|
||||
@@ -256,7 +256,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
|
||||
|
||||
| 使用方式 | 支持平台 | 免费平台 |
|
||||
|:---:|:---:|:---:|
|
||||
| openai 接口调用 | 阿里百炼、火山引擎、DeepSeek、智谱、Gemini、科大讯飞 | 智谱、Gemini |
|
||||
| openai 接口调用 | 阿里百炼、火山引擎、DeepSeek、智谱、Gemini、科大讯飞、LM Studio | 智谱、Gemini |
|
||||
| ollama 接口调用 | Ollama | - |
|
||||
| dify 接口调用 | Dify | - |
|
||||
| fastgpt 接口调用 | Fastgpt | - |
|
||||
@@ -300,7 +300,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
|
||||
| 使用方式 | 支持平台 | 免费平台 |
|
||||
|:---:|:---:|:---:|
|
||||
| 本地使用 | FunASR、SherpaASR、SherpaParaformerASR、VoskASR | FunASR、SherpaASR、SherpaParaformerASR、VoskASR |
|
||||
| 接口调用 | FunASRServer、火山引擎、科大讯飞、腾讯云、阿里云、百度云、OpenAI ASR、GroqASR、Qwen3ASRFlash | FunASRServer |
|
||||
| 接口调用 | FunASRServer、火山引擎、科大讯飞、腾讯云、阿里云及百炼、百度云、OpenAI ASR、GroqASR、Qwen3ASRFlash | FunASRServer |
|
||||
|
||||
---
|
||||
|
||||
|
||||
+20
-22
@@ -37,30 +37,28 @@ docker logs -f mcp-endpoint-server
|
||||
|
||||
此时,日志里会输出类似以下的日志
|
||||
```
|
||||
250705 INFO-=====下面的地址分别是智控台/单模块MCP接入点地址====
|
||||
250705 INFO-智控台MCP参数配置: http://172.22.0.2:8004/mcp_endpoint/health?key=abc
|
||||
250705 INFO-单模块部署MCP接入点: ws://172.22.0.2:8004/mcp_endpoint/mcp/?token=def
|
||||
250705 INFO-=====请根据具体部署选择使用,请勿泄露给任何人======
|
||||
250705 13:04:20[0.0.7][src.server]-INFO-=====下面的地址分别是智控台/多模块MCP接入点地址====
|
||||
250705 13:04:20[0.0.7][src.server]-INFO-智控台MCP参数配置: http://172.22.0.2:8004/mcp_endpoint/health?key=abc
|
||||
250705 13:04:20[0.0.7][src.server]-INFO-单模块部署MCP接入点: ws://172.22.0.2:8004/mcp_endpoint/mcp/?token=abc
|
||||
250705 13:04:20[0.0.7][src.server]-INFO-=====请根据具体部署选择使用,请勿泄露给任何人======
|
||||
```
|
||||
|
||||
上面第一行(`=====下面的地址分别是…====`)的措辞跟着 mcp-endpoint-server 的镜像版本走,不同版本可能不一样,以你实际看到的输出为准。
|
||||
|
||||
请你把两个接口地址复制出来:
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
由于你是docker部署,切不可直接使用上面的地址!
|
||||
|
||||
你先把地址复制出来,放在一个草稿里,你要知道你的电脑的局域网ip是什么,例如我的电脑局域网ip是`192.168.1.25`,那么
|
||||
原来我的接口地址
|
||||
```
|
||||
智控台MCP参数配置: http://172.22.0.2:8004/mcp_endpoint/health?key=abc
|
||||
单模块部署MCP接入点: ws://172.22.0.2:8004/mcp_endpoint/mcp/?token=def
|
||||
单模块部署MCP接入点: ws://172.22.0.2:8004/mcp_endpoint/mcp/?token=abc
|
||||
```
|
||||
就要改成
|
||||
```
|
||||
智控台MCP参数配置: http://192.168.1.25:8004/mcp_endpoint/health?key=abc
|
||||
单模块部署MCP接入点: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=def
|
||||
单模块部署MCP接入点: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
|
||||
```
|
||||
|
||||
改好后,请使用浏览器直接访问`智控台MCP参数配置`。当浏览器出现类似这样的代码,说明是成功了。
|
||||
@@ -87,7 +85,7 @@ docker logs -f mcp-endpoint-server
|
||||
```
|
||||
server:
|
||||
websocket: ws://你的ip或者域名:端口号/xiaozhi/v1/
|
||||
http_port: 8002
|
||||
http_port: 8003
|
||||
log:
|
||||
log_level: INFO
|
||||
|
||||
@@ -100,26 +98,26 @@ mcp_endpoint: 你的接入点websocket地址
|
||||
```
|
||||
server:
|
||||
websocket: ws://你的ip或者域名:端口号/xiaozhi/v1/
|
||||
http_port: 8002
|
||||
http_port: 8003
|
||||
log:
|
||||
log_level: INFO
|
||||
|
||||
# 此处可能还更多配置
|
||||
|
||||
mcp_endpoint: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=def
|
||||
mcp_endpoint: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
|
||||
```
|
||||
|
||||
配置好后,启动单模块会输出如下的日志。
|
||||
```
|
||||
250705[__main__]-INFO-初始化组件: vad成功 SileroVAD
|
||||
250705[__main__]-INFO-初始化组件: asr成功 FunASRServer
|
||||
250705[__main__]-INFO-OTA接口是 http://192.168.1.25:8002/xiaozhi/ota/
|
||||
250705[__main__]-INFO-视觉分析接口是 http://192.168.1.25:8002/mcp/vision/explain
|
||||
250705[__main__]-INFO-mcp接入点是 ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
|
||||
250705[__main__]-INFO-Websocket地址是 ws://192.168.1.25:8000/xiaozhi/v1/
|
||||
250705[__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
|
||||
250705[__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
|
||||
250705[__main__]-INFO-=============================================================
|
||||
250705 13:04:20[0.9.8_00000000000000][core.utils.modules_initialize]-INFO-初始化组件: vad成功 SileroVAD
|
||||
250705 13:04:20[0.9.8_00000000000000][core.utils.modules_initialize]-INFO-初始化组件: asr成功 FunASRServer
|
||||
250705 13:04:20[0.9.8_00000000000000][__main__]-INFO-OTA接口是 http://192.168.1.25:8003/xiaozhi/ota/
|
||||
250705 13:04:20[0.9.8_00000000000000][__main__]-INFO-视觉分析接口是 http://192.168.1.25:8003/mcp/vision/explain
|
||||
250705 13:04:20[0.9.8_00000000000000][__main__]-INFO-mcp接入点是 ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
|
||||
250705 13:04:20[0.9.8_00000000000000][__main__]-INFO-Websocket地址是 ws://192.168.1.25:8000/xiaozhi/v1/
|
||||
250705 13:04:20[0.9.8_00000000000000][__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
|
||||
250705 13:04:20[0.9.8_00000000000000][__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
|
||||
250705 13:04:20[0.9.8_00000000000000][__main__]-INFO-=============================================================
|
||||
```
|
||||
|
||||
如上,如果能输出类似的`mcp接入点是`中`ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc`说明配置成功了。
|
||||
|
||||
+26
-27
@@ -6,7 +6,7 @@
|
||||
Dieses Projekt basiert auf der Theorie und Technologie der Mensch-Maschine-symbiotischen Intelligenz zur Entwicklung intelligenter Terminal-Hardware- und Software-Systeme<br/>und bietet Backend-Dienste für das Open-Source-Hardware-Projekt
|
||||
<a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a><br/>
|
||||
Implementiert mit Python, Java und Vue gemäß dem <a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">Xiaozhi-Kommunikationsprotokoll</a><br/>
|
||||
Unterstützt MQTT+UDP-Protokoll, Websocket-Protokoll, MCP-Endpunkte und Stimmabdruckerkennung
|
||||
Unterstützt MQTT+UDP-Protokoll, Websocket-Protokoll, MCP-Endpunkte, Stimmabdruckerkennung und Wissensdatenbank
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
@@ -55,31 +55,31 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
|
||||
<picture>
|
||||
<img alt="响应速度感受" src="../images/demo9.png" /></picture>
|
||||
<img alt="Gefühl der Antwortgeschwindigkeit" src="../images/demo9.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
|
||||
<picture>
|
||||
<img alt="速度优化秘诀" src="../images/demo6.png" /></picture>
|
||||
<img alt="Das Geheimnis der Geschwindigkeitsoptimierung" src="../images/demo6.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
|
||||
<picture>
|
||||
<img alt="小智数字人 支持语音唤醒" src="../images/demo8.png" /></picture>
|
||||
<img alt="Xiaozhi-Digitalmensch mit Sprachaktivierung" src="../images/demo8.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
|
||||
<picture>
|
||||
<img alt="设备呼叫设备,打电话" src="../images/demo0.png" /></picture>
|
||||
<img alt="Anruf von Gerät zu Gerät" src="../images/demo0.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
|
||||
<picture>
|
||||
<img alt="复杂医疗场景" src="../images/demo1.png" /></picture>
|
||||
<img alt="Komplexes medizinisches Szenario" src="../images/demo1.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -87,31 +87,31 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
|
||||
<picture>
|
||||
<img alt="播放音乐查询天气播报新闻" src="../images/demo7.png" /></picture>
|
||||
<img alt="Musik abspielen, Wetter abfragen und Nachrichten vorlesen" src="../images/demo7.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
|
||||
<picture>
|
||||
<img alt="实时打断" src="../images/demo10.png" /></picture>
|
||||
<img alt="Unterbrechung in Echtzeit" src="../images/demo10.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
|
||||
<picture>
|
||||
<img alt="拍照识物品" src="../images/demo12.png" /></picture>
|
||||
<img alt="Objekte per Foto erkennen" src="../images/demo12.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
|
||||
<picture>
|
||||
<img alt="控制家电开关" src="../images/demo5.png" /></picture>
|
||||
<img alt="Haushaltsgeräte ein- und ausschalten" src="../images/demo5.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
|
||||
<picture>
|
||||
<img alt="多指令任务" src="../images/demo11.png" /></picture>
|
||||
<img alt="Aufgaben mit mehreren Befehlen" src="../images/demo11.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -119,31 +119,31 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
|
||||
<picture>
|
||||
<img alt="MCP接入点" src="../images/demo13.png" /></picture>
|
||||
<img alt="MCP-Endpunkt" src="../images/demo13.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
|
||||
<picture>
|
||||
<img alt="MQTT指令下发" src="../images/demo4.png" /></picture>
|
||||
<img alt="MQTT-Befehlsübermittlung" src="../images/demo4.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
|
||||
<picture>
|
||||
<img alt="声纹识别" src="../images/demo14.png" /></picture>
|
||||
<img alt="Stimmabdruckerkennung" src="../images/demo14.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
|
||||
<picture>
|
||||
<img alt="自定义音色" src="../images/demo2.png" /></picture>
|
||||
<img alt="Eigene Stimmfarbe" src="../images/demo2.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
|
||||
<picture>
|
||||
<img alt="使用粤语交流" src="../images/demo3.png" /></picture>
|
||||
<img alt="Gespräch auf Kantonesisch" src="../images/demo3.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -210,7 +210,7 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
|
||||
|
||||
| Werkzeugname | Standort | Verwendungsmethode | Funktionsbeschreibung |
|
||||
|:---:|:---|:---:|:---:|
|
||||
| Audio-Interaktionstesttool | main》digital-human》index.html | In `main/digital-human` `python start.py` ausführen und dann `http://127.0.0.1:8006/index.html` öffnen | Testet Audio-Wiedergabe- und Empfangsfunktionen, überprüft, ob die Python-seitige Audioverarbeitung normal ist |
|
||||
| Audio-Interaktionstesttool | main》digital-human》index.html | In `main/digital-human` zuerst die Abhängigkeiten wie in der README installieren, dann `python start.py` ausführen und `http://127.0.0.1:8006/index.html` öffnen | Testet Audio-Wiedergabe- und Empfangsfunktionen, überprüft, ob die Python-seitige Audioverarbeitung normal ist |
|
||||
| Modell-Reaktionstesttool | main》xiaozhi-server》performance_tester.py | Ausführen `python performance_tester.py` | Testet die Reaktionsgeschwindigkeit von Kernmodulen wie ASR (Spracherkennung), LLM (großes Modell), VLLM (Vision-Modell), TTS (Sprachsynthese) |
|
||||
|
||||
> 💡 Hinweis: Beim Testen der Modellgeschwindigkeit werden nur Modelle mit konfigurierten Schlüsseln getestet.
|
||||
@@ -218,7 +218,7 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
|
||||
---
|
||||
## Funktionsliste ✨
|
||||
### Implementiert ✅
|
||||

|
||||

|
||||
| Funktionsmodul | Beschreibung |
|
||||
|:---:|:---|
|
||||
| Kernarchitektur | Basierend auf [MQTT+UDP-Gateway](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket und HTTP-Servern, bietet vollständiges Konsolenverwaltungs- und Authentifizierungssystem |
|
||||
@@ -226,7 +226,7 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
|
||||
| Stimmabdruckerkennung | Unterstützt Mehrbenutzer-Stimmabdruckregistrierung, -verwaltung und -erkennung, verarbeitet parallel mit ASR, Echtzeit-Sprecheridentitätserkennung und Weitergabe an LLM für personalisierte Antworten |
|
||||
| Intelligenter Dialog | Unterstützt mehrere LLM (große Sprachmodelle), implementiert intelligenten Dialog |
|
||||
| Visuelle Wahrnehmung | Unterstützt mehrere VLLM (Vision Large Models), implementiert multimodale Interaktion |
|
||||
| Absichtserkennung | Unterstützt LLM-Absichtserkennung, Function Call-Funktionsaufruf, bietet plugin-basierten Absichtsverarbeitungsmechanismus |
|
||||
| Absichtserkennung | Unterstützt externe LLM-Absichtserkennung, autonome Funktionsaufrufe durch große Modelle, bietet plugin-basierten Absichtsverarbeitungsmechanismus |
|
||||
| Gedächtnissystem | Unterstützt lokales Kurzzeitgedächtnis, mem0ai-Schnittstellengedächtnis, PowerMem intelligentes Gedächtnis, mit Gedächtniszusammenfassungsfunktion |
|
||||
| Wissensdatenbank | Unterstützt RAGFlow-Wissensdatenbank, ermöglicht großem Modell die Bewertung, ob Wissensdatenbank benötigt wird, bevor geantwortet wird |
|
||||
| Werkzeugaufruf | Unterstützt Client-IOT-Protokoll, Client-MCP-Protokoll, Server-MCP-Protokoll, MCP-Endpunktprotokoll, benutzerdefinierte Werkzeugfunktionen |
|
||||
@@ -254,7 +254,7 @@ Xiaozhi ist ein Ökosystem. Wenn Sie dieses Produkt verwenden, können Sie sich
|
||||
|
||||
| Verwendungsmethode | Unterstützte Plattformen | Kostenlose Plattformen |
|
||||
|:---:|:---:|:---:|
|
||||
| OpenAI-Schnittstellenaufrufe | Alibaba Bailian, Volcano Engine Doubao, DeepSeek, Zhipu ChatGLM, Gemini, iFLYTEK | Zhipu ChatGLM, Gemini |
|
||||
| OpenAI-Schnittstellenaufrufe | Alibaba Bailian, Volcano Engine Doubao, Volcano Engine AI Gateway, DeepSeek, Zhipu ChatGLM, Gemini, iFLYTEK, LM Studio | Zhipu ChatGLM, Gemini |
|
||||
| Ollama-Schnittstellenaufrufe | Ollama | - |
|
||||
| Dify-Schnittstellenaufrufe | Dify | - |
|
||||
| FastGPT-Schnittstellenaufrufe | FastGPT | - |
|
||||
@@ -280,7 +280,7 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int
|
||||
|
||||
| Verwendungsmethode | Unterstützte Plattformen | Kostenlose Plattformen |
|
||||
|:---:|:---:|:---:|
|
||||
| Schnittstellenaufrufe | EdgeTTS, iFLYTEK, Volcano Engine Doubao TTS, Tencent Cloud, Alibaba Cloud TTS, AliYun Stream TTS, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS, Volcano Dual-Stream TTS | EdgeTTS, CosyVoiceSiliconflow (teilweise) |
|
||||
| Schnittstellenaufrufe | EdgeTTS, iFLYTEK, Volcano Engine Doubao TTS, Volcano Engine AI Gateway TTS, Tencent Cloud, Alibaba Cloud TTS, AliYun Stream TTS, Alibaba Bailian CosyVoice TTS, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS, Volcano Dual-Stream TTS | EdgeTTS, CosyVoiceSiliconflow (teilweise) |
|
||||
| Lokale Dienste | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, IndexStreamTTS, PaddleSpeechTTS | IndexStreamTTS, PaddleSpeechTTS, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
|
||||
|
||||
---
|
||||
@@ -298,7 +298,7 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int
|
||||
| Verwendungsmethode | Unterstützte Plattformen | Kostenlose Plattformen |
|
||||
|:---:|:---:|:---:|
|
||||
| Lokale Verwendung | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
|
||||
| Schnittstellenaufrufe | DoubaoASR, Doubao Streaming ASR, FunASRServer, iFLYTEK, TencentASR, AliyunASR, Aliyun Streaming ASR, Baidu ASR, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
| Schnittstellenaufrufe | DoubaoASR, Doubao Streaming ASR, Doubao Streaming ASR V2, FunASRServer, iFLYTEK, TencentASR, AliyunASR, Aliyun Streaming ASR, Alibaba Bailian Streaming ASR, Baidu ASR, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
|
||||
---
|
||||
|
||||
@@ -354,11 +354,10 @@ Die Stimmabdruckerkennung erfordert einen separat bereitgestellten [voiceprint-a
|
||||
| <img src="../images/logo_contributors.png" width="160"> | [Code-Mitwirkende](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Dank an [alle Code-Mitwirkenden](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), Ihre Bemühungen haben das Projekt robuster und leistungsfähiger gemacht. |
|
||||
|
||||
|
||||
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
|
||||
|
||||
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
</picture>
|
||||
</a>
|
||||
+24
-25
@@ -55,31 +55,31 @@ Want to see the usage effects? Click the videos below 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
|
||||
<picture>
|
||||
<img alt="响应速度感受" src="../images/demo9.png" /></picture>
|
||||
<img alt="The feel of the response speed" src="../images/demo9.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
|
||||
<picture>
|
||||
<img alt="速度优化秘诀" src="../images/demo6.png" /></picture>
|
||||
<img alt="The secret to speeding things up" src="../images/demo6.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
|
||||
<picture>
|
||||
<img alt="小智数字人 支持语音唤醒" src="../images/demo8.png" /></picture>
|
||||
<img alt="Xiaozhi digital human with voice wake-up" src="../images/demo8.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
|
||||
<picture>
|
||||
<img alt="设备呼叫设备,打电话" src="../images/demo0.png" /></picture>
|
||||
<img alt="Device-to-device calling" src="../images/demo0.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
|
||||
<picture>
|
||||
<img alt="复杂医疗场景" src="../images/demo1.png" /></picture>
|
||||
<img alt="A complex medical scenario" src="../images/demo1.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -87,31 +87,31 @@ Want to see the usage effects? Click the videos below 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
|
||||
<picture>
|
||||
<img alt="播放音乐查询天气播报新闻" src="../images/demo7.png" /></picture>
|
||||
<img alt="Play music, check the weather, and read the news" src="../images/demo7.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
|
||||
<picture>
|
||||
<img alt="实时打断" src="../images/demo10.png" /></picture>
|
||||
<img alt="Real-time interruption" src="../images/demo10.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
|
||||
<picture>
|
||||
<img alt="拍照识物品" src="../images/demo12.png" /></picture>
|
||||
<img alt="Take a photo to identify objects" src="../images/demo12.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
|
||||
<picture>
|
||||
<img alt="控制家电开关" src="../images/demo5.png" /></picture>
|
||||
<img alt="Turn home appliances on and off" src="../images/demo5.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
|
||||
<picture>
|
||||
<img alt="多指令任务" src="../images/demo11.png" /></picture>
|
||||
<img alt="Multi-command tasks" src="../images/demo11.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -119,31 +119,31 @@ Want to see the usage effects? Click the videos below 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
|
||||
<picture>
|
||||
<img alt="MCP接入点" src="../images/demo13.png" /></picture>
|
||||
<img alt="MCP access point" src="../images/demo13.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
|
||||
<picture>
|
||||
<img alt="MQTT指令下发" src="../images/demo4.png" /></picture>
|
||||
<img alt="MQTT command delivery" src="../images/demo4.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
|
||||
<picture>
|
||||
<img alt="声纹识别" src="../images/demo14.png" /></picture>
|
||||
<img alt="Voiceprint recognition" src="../images/demo14.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
|
||||
<picture>
|
||||
<img alt="自定义音色" src="../images/demo2.png" /></picture>
|
||||
<img alt="Custom voice timbre" src="../images/demo2.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
|
||||
<picture>
|
||||
<img alt="使用粤语交流" src="../images/demo3.png" /></picture>
|
||||
<img alt="Talking in Cantonese" src="../images/demo3.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -210,7 +210,7 @@ This project provides the following testing tools to help you verify the system
|
||||
|
||||
| Tool Name | Location | Usage Method | Function Description |
|
||||
|:---:|:---|:---:|:---:|
|
||||
| Audio Interaction Test Tool | main》digital-human》index.html | Run `python start.py` in `main/digital-human`, then open `http://127.0.0.1:8006/index.html` | Tests audio playback and reception functions, verifies if Python-side audio processing is normal |
|
||||
| Audio Interaction Test Tool | main》digital-human》index.html | In `main/digital-human`, install dependencies as its README says, then run `python start.py` and open `http://127.0.0.1:8006/index.html` | Tests audio playback and reception functions, verifies if Python-side audio processing is normal |
|
||||
| Model Response Test Tool | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Tests response speed of core modules such as ASR(speech recognition), LLM(large model), VLLM(vision model), TTS(speech synthesis) |
|
||||
|
||||
> 💡 Note: When testing model speed, only models with configured keys will be tested.
|
||||
@@ -218,7 +218,7 @@ This project provides the following testing tools to help you verify the system
|
||||
---
|
||||
## Feature List ✨
|
||||
### Implemented ✅
|
||||

|
||||

|
||||
| Feature Module | Description |
|
||||
|:---:|:---|
|
||||
| Core Architecture | Based on [MQTT+UDP gateway](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket and HTTP servers, provides complete console management and authentication system |
|
||||
@@ -226,7 +226,7 @@ This project provides the following testing tools to help you verify the system
|
||||
| Voiceprint Recognition | Supports multi-user voiceprint registration, management, and recognition, processes in parallel with ASR, real-time speaker identity recognition and passes to LLM for personalized responses |
|
||||
| Intelligent Dialogue | Supports multiple LLM(large language models), implements intelligent dialogue |
|
||||
| Visual Perception | Supports multiple VLLM(vision large models), implements multimodal interaction |
|
||||
| Intent Recognition | Supports LLM intent recognition, Function Call function calling, provides plugin-based intent processing mechanism |
|
||||
| Intent Recognition | Supports pluggable external LLM intent recognition and autonomous LLM function calling, provides plugin-based intent processing mechanism |
|
||||
| Memory System | Supports local short-term memory, mem0ai interface memory, PowerMem intelligent memory, with memory summarization functionality |
|
||||
| Knowledge Base | Supports RAGFlow knowledge base, enabling LLM to judge whether to schedule the knowledge base after receiving the user's question, and then answer the question |
|
||||
| Tool Calling | Supports client IOT protocol, client MCP protocol, server MCP protocol, MCP endpoint protocol, custom tool functions |
|
||||
@@ -254,7 +254,7 @@ Xiaozhi is an ecosystem. When using this product, you can also check out other [
|
||||
|
||||
| Usage Method | Supported Platforms | Free Platforms |
|
||||
|:---:|:---:|:---:|
|
||||
| OpenAI interface calls | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK | Zhipu, Gemini |
|
||||
| OpenAI interface calls | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
|
||||
| Ollama interface calls | Ollama | - |
|
||||
| Dify interface calls | Dify | - |
|
||||
| FastGPT interface calls | FastGPT | - |
|
||||
@@ -298,7 +298,7 @@ In fact, any VLLM that supports OpenAI interface calls can be integrated and use
|
||||
| Usage Method | Supported Platforms | Free Platforms |
|
||||
|:---:|:---:|:---:|
|
||||
| Local use | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
|
||||
| Interface calls | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
| Interface calls | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud and Bailian, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
|
||||
---
|
||||
|
||||
@@ -354,11 +354,10 @@ Voiceprint recognition requires a separately deployed [voiceprint-api](https://g
|
||||
| <img src="../images/logo_contributors.png" width="160"> | [Code Contributors](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Thanks to [all code contributors](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), your efforts have made the project more robust and powerful. |
|
||||
|
||||
|
||||
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
|
||||
|
||||
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
</picture>
|
||||
</a>
|
||||
+16
-14
@@ -168,8 +168,8 @@ Este proyecto ofrece dos formas de despliegue. Elija según lo que necesite:
|
||||
#### 🚀 Elección del método de despliegue
|
||||
| Método de despliegue | Características | Escenarios | Documentación de despliegue | Requisitos | Tutoriales en vídeo |
|
||||
|---------|------|---------|---------|---------|---------|
|
||||
| **Instalación simplificada** | Diálogo inteligente, gestión de un solo agente | Entornos con pocos recursos; los datos se guardan en archivos de configuración y no hace falta base de datos | [①Versión Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Despliegue desde el código fuente](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 núcleos y 4 GB si usa `FunASR`; 2 núcleos y 2 GB si todo va por API | - |
|
||||
| **Instalación de todos los módulos** | Diálogo inteligente, gestión multiusuario, gestión de varios agentes, uso de la interfaz de la consola inteligente | Experiencia con todas las funciones; los datos se guardan en una base de datos |[①Versión Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Despliegue desde el código fuente](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Tutorial de actualización automática desde el código fuente](../dev-ops-integration.md) | 4 núcleos y 8 GB si usa `FunASR`; 2 núcleos y 4 GB si todo va por API | [Tutorial en vídeo para arrancar el código fuente en local](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
|
||||
| **Instalación simplificada** | Diálogo inteligente, gestión de un solo agente | Entornos con pocos recursos; los datos se guardan en archivos de configuración y no hace falta base de datos | [①Versión Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Despliegue desde el código fuente](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| Ejecutar `FunASR` en local consume bastante memoria; si todo va por API, el consumo de recursos es bajo | - |
|
||||
| **Instalación de todos los módulos** | Diálogo inteligente, gestión multiusuario, gestión de varios agentes, uso de la interfaz de la consola inteligente | Experiencia con todas las funciones; los datos se guardan en una base de datos |[①Versión Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Despliegue desde el código fuente](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Tutorial de actualización automática desde el código fuente](../dev-ops-integration.md) | Ejecutar `FunASR` en local consume bastante memoria, y además requiere recursos de base de datos; si todo va por API, el consumo de recursos es bajo | [Tutorial en vídeo para arrancar el código fuente en local](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
|
||||
|
||||
Las preguntas frecuentes y los tutoriales relacionados están en [este enlace](../FAQ.md)
|
||||
|
||||
@@ -210,8 +210,8 @@ Este proyecto incluye las siguientes herramientas para comprobar el sistema y el
|
||||
|
||||
| Herramienta | Ubicación | Uso | Qué hace |
|
||||
|:---:|:---|:---:|:---:|
|
||||
| Prueba de interacción de audio | main》digital-human》index.html | En `main/digital-human`, ejecute `python start.py` y abra `http://127.0.0.1:8006/index.html` | Prueba la reproducción y la recepción de audio, y comprueba que el procesamiento de audio en Python funciona |
|
||||
| Prueba de respuesta de los modelos | main》xiaozhi-server》performance_tester.py | Ejecute `python performance_tester.py` | Mide la velocidad de respuesta de los cuatro módulos principales: ASR (reconocimiento de voz), LLM (modelo de lenguaje), VLLM (modelo de visión) y TTS (síntesis de voz) |
|
||||
| Prueba de interacción de audio | main》digital-human》index.html | En `main/digital-human`, instale las dependencias como indica su README, después ejecute `python start.py` y abra `http://127.0.0.1:8006/index.html` | Prueba la reproducción y la recepción de audio, y comprueba que el procesamiento de audio en Python funciona |
|
||||
| Prueba de respuesta de los modelos | main》xiaozhi-server》performance_tester.py | Ejecute `python performance_tester.py` | Mide la velocidad de respuesta de los principales módulos: ASR (reconocimiento de voz), LLM (modelo de lenguaje), VLLM (modelo de visión) y TTS (síntesis de voz) |
|
||||
|
||||
> 💡 Nota: al medir la velocidad de los modelos, solo se prueban los modelos que tienen una clave configurada.
|
||||
|
||||
@@ -226,15 +226,15 @@ Este proyecto incluye las siguientes herramientas para comprobar el sistema y el
|
||||
| Huella de voz | Admite el registro, la gestión y el reconocimiento de la huella de voz de varios usuarios; se procesa en paralelo con el ASR, identifica en tiempo real quién habla y se lo pasa al LLM para una respuesta personalizada |
|
||||
| Diálogo inteligente | Admite varios LLM (modelos de lenguaje) para mantener un diálogo inteligente |
|
||||
| Percepción visual | Admite varios VLLM (modelos de visión) para una interacción multimodal |
|
||||
| Reconocimiento de intención | Admite el reconocimiento de intención con el modelo de lenguaje (LLM) y la llamada autónoma a funciones del modelo, y ofrece un mecanismo de procesamiento de intenciones mediante plugins |
|
||||
| Reconocimiento de intención | Admite el reconocimiento de intención con un modelo de lenguaje (LLM) externo y conectable, y la llamada autónoma a funciones del modelo, y ofrece un mecanismo de procesamiento de intenciones mediante plugins |
|
||||
| Sistema de memoria | Admite memoria local a corto plazo, memoria por la interfaz mem0ai y memoria inteligente PowerMem, con resumen de la memoria |
|
||||
| Base de conocimiento | Admite la base de conocimiento RAGFlow, para que el modelo decida si debe consultarla antes de responder |
|
||||
| Llamada a herramientas | Admite el protocolo IOT del cliente, el protocolo MCP del cliente, el protocolo MCP del servidor, el protocolo de punto de acceso MCP y funciones de herramientas personalizadas |
|
||||
| Envío de instrucciones | Apoyado en el protocolo MQTT, permite enviar instrucciones MCP desde la consola inteligente a los dispositivos ESP32 |
|
||||
| Backend de administración | Ofrece una interfaz web de administración, con gestión de usuarios, configuración del sistema y gestión de dispositivos. La interfaz se muestra en chino simplificado, chino tradicional e inglés |
|
||||
| Herramientas de prueba | Incluye herramientas de prueba de rendimiento, de modelos de visión y de interacción de audio |
|
||||
| Backend de administración | Ofrece una interfaz web de administración, con gestión de usuarios, configuración del sistema y gestión de dispositivos. La interfaz se muestra en chino simplificado, chino tradicional, inglés, alemán, vietnamita y portugués de Brasil |
|
||||
| Herramientas de prueba | Incluye herramientas de prueba de rendimiento y de interacción de audio |
|
||||
| Soporte de despliegue | Admite el despliegue con Docker y el despliegue local, y ofrece una gestión completa de los archivos de configuración |
|
||||
| Sistema de plugins | Admite extensiones de plugins funcionales, el desarrollo de plugins propios y la carga en caliente de plugins |
|
||||
| Sistema de plugins | Admite extensiones de plugins funcionales y el desarrollo de plugins propios; los nuevos archivos de plugin se cargan automáticamente en la siguiente conexión del dispositivo, y los cambios en el código de un plugin ya cargado requieren reiniciar el servicio |
|
||||
|
||||
### En desarrollo 🚧
|
||||
|
||||
@@ -254,7 +254,7 @@ Xiaozhi es un ecosistema. Al usar este producto, también puede ver otros [proye
|
||||
|
||||
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|
||||
|:---:|:---:|:---:|
|
||||
| Llamadas por la interfaz OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK | Zhipu, Gemini |
|
||||
| Llamadas por la interfaz OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
|
||||
| Llamadas por la interfaz Ollama | Ollama | - |
|
||||
| Llamadas por la interfaz Dify | Dify | - |
|
||||
| Llamadas por la interfaz FastGPT | FastGPT | - |
|
||||
@@ -280,8 +280,8 @@ En la práctica, se puede integrar cualquier VLLM que admita llamadas por la int
|
||||
|
||||
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|
||||
|:---:|:---:|:---:|
|
||||
| Llamadas por interfaz | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud y Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, GizwitsTTS, ACGNTTS, OpenAITTS, Lingxi Streaming TTS, MinimaxTTS | Lingxi Streaming TTS, EdgeTTS, CosyVoiceSiliconflow (parcial) |
|
||||
| Servicio local | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, Index-TTS, PaddleSpeech | Index-TTS, PaddleSpeech, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
|
||||
| Llamadas por interfaz | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud y Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS | EdgeTTS, CosyVoiceSiliconflow (parcial) |
|
||||
| Servicio local | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, IndexStreamTTS, PaddleSpeechTTS | IndexStreamTTS, PaddleSpeechTTS, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
|
||||
|
||||
---
|
||||
|
||||
@@ -297,8 +297,8 @@ En la práctica, se puede integrar cualquier VLLM que admita llamadas por la int
|
||||
|
||||
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|
||||
|:---:|:---:|:---:|
|
||||
| Uso local | FunASR, SherpaASR | FunASR, SherpaASR |
|
||||
| Llamadas por interfaz | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud, Baidu Cloud, OpenAI ASR | FunASRServer |
|
||||
| Uso local | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
|
||||
| Llamadas por interfaz | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud y Bailian, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
|
||||
---
|
||||
|
||||
@@ -306,7 +306,9 @@ En la práctica, se puede integrar cualquier VLLM que admita llamadas por la int
|
||||
|
||||
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|
||||
|:---:|:---:|:---:|
|
||||
| Uso local | 3D-Speaker | 3D-Speaker |
|
||||
| Llamadas por interfaz | Servicio de reconocimiento de huella de voz (voiceprint-api) | - |
|
||||
|
||||
El reconocimiento de huella de voz requiere desplegar por separado el servicio [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) (basado en 3D-Speaker), que depende de MySQL; consulte la [guía de integración de la huella de voz](../voiceprint-integration.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
+25
-26
@@ -55,31 +55,31 @@ Quer ver os efeitos de uso? Clique nos vídeos abaixo 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
|
||||
<picture>
|
||||
<img alt="响应速度感受" src="../images/demo9.png" /></picture>
|
||||
<img alt="A sensação da velocidade de resposta" src="../images/demo9.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
|
||||
<picture>
|
||||
<img alt="速度优化秘诀" src="../images/demo6.png" /></picture>
|
||||
<img alt="O segredo para otimizar a velocidade" src="../images/demo6.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
|
||||
<picture>
|
||||
<img alt="小智数字人 支持语音唤醒" src="../images/demo8.png" /></picture>
|
||||
<img alt="Humano digital Xiaozhi com ativação por voz" src="../images/demo8.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
|
||||
<picture>
|
||||
<img alt="设备呼叫设备,打电话" src="../images/demo0.png" /></picture>
|
||||
<img alt="Chamadas de dispositivo para dispositivo" src="../images/demo0.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
|
||||
<picture>
|
||||
<img alt="复杂医疗场景" src="../images/demo1.png" /></picture>
|
||||
<img alt="Cenário médico complexo" src="../images/demo1.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -87,31 +87,31 @@ Quer ver os efeitos de uso? Clique nos vídeos abaixo 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
|
||||
<picture>
|
||||
<img alt="播放音乐查询天气播报新闻" src="../images/demo7.png" /></picture>
|
||||
<img alt="Reproduzir música, consultar o tempo e ler as notícias" src="../images/demo7.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
|
||||
<picture>
|
||||
<img alt="实时打断" src="../images/demo10.png" /></picture>
|
||||
<img alt="Interrupção em tempo real" src="../images/demo10.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
|
||||
<picture>
|
||||
<img alt="拍照识物品" src="../images/demo12.png" /></picture>
|
||||
<img alt="Reconhecer objetos com uma foto" src="../images/demo12.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
|
||||
<picture>
|
||||
<img alt="控制家电开关" src="../images/demo5.png" /></picture>
|
||||
<img alt="Controlar o acionamento de eletrodomésticos" src="../images/demo5.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
|
||||
<picture>
|
||||
<img alt="多指令任务" src="../images/demo11.png" /></picture>
|
||||
<img alt="Tarefas com vários comandos" src="../images/demo11.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -119,31 +119,31 @@ Quer ver os efeitos de uso? Clique nos vídeos abaixo 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
|
||||
<picture>
|
||||
<img alt="MCP接入点" src="../images/demo13.png" /></picture>
|
||||
<img alt="Ponto de acesso MCP" src="../images/demo13.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
|
||||
<picture>
|
||||
<img alt="MQTT指令下发" src="../images/demo4.png" /></picture>
|
||||
<img alt="Envio de comandos MQTT" src="../images/demo4.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
|
||||
<picture>
|
||||
<img alt="声纹识别" src="../images/demo14.png" /></picture>
|
||||
<img alt="Reconhecimento de impressão vocal" src="../images/demo14.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
|
||||
<picture>
|
||||
<img alt="自定义音色" src="../images/demo2.png" /></picture>
|
||||
<img alt="Tom de voz personalizado" src="../images/demo2.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
|
||||
<picture>
|
||||
<img alt="使用粤语交流" src="../images/demo3.png" /></picture>
|
||||
<img alt="Conversa em cantonês" src="../images/demo3.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -210,7 +210,7 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
|
||||
|
||||
| Nome da Ferramenta | Localização | Método de Uso | Descrição da Função |
|
||||
|:---:|:---|:---:|:---:|
|
||||
| Ferramenta de Teste de Interação por Áudio | main》digital-human》index.html | Execute `python start.py` em `main/digital-human` e depois abra `http://127.0.0.1:8006/index.html` | Testa as funções de reprodução e recepção de áudio, verifica se o processamento de áudio no lado Python está normal |
|
||||
| Ferramenta de Teste de Interação por Áudio | main》digital-human》index.html | Em `main/digital-human`, instale as dependências como indica o README, depois execute `python start.py` e abra `http://127.0.0.1:8006/index.html` | Testa as funções de reprodução e recepção de áudio, verifica se o processamento de áudio no lado Python está normal |
|
||||
| Ferramenta de Teste de Resposta de Modelo | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Testa a velocidade de resposta de módulos principais como ASR(reconhecimento de fala), LLM(modelo de linguagem), VLLM(modelo de visão), TTS(síntese de voz) |
|
||||
|
||||
> 💡 Nota: Ao testar a velocidade dos modelos, apenas os modelos com chaves configuradas serão testados.
|
||||
@@ -218,7 +218,7 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
|
||||
---
|
||||
## Lista de Funcionalidades ✨
|
||||
### Implementado ✅
|
||||

|
||||

|
||||
| Módulo de Funcionalidade | Descrição |
|
||||
|:---:|:---|
|
||||
| Arquitetura Principal | Baseado em [gateway MQTT+UDP](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), servidores WebSocket e HTTP, fornece sistema completo de gerenciamento de console e autenticação |
|
||||
@@ -226,7 +226,7 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
|
||||
| Reconhecimento de Impressão Vocal | Suporta registro, gerenciamento e reconhecimento de impressão vocal de múltiplos usuários, processa em paralelo com o ASR, reconhecimento de identidade do falante em tempo real e repassa ao LLM para respostas personalizadas |
|
||||
| Diálogo Inteligente | Suporta múltiplos LLM (modelos de linguagem de grande porte), implementa diálogo inteligente |
|
||||
| Percepção Visual | Suporta múltiplos VLLM (modelos de visão de grande porte), implementa interação multimodal |
|
||||
| Reconhecimento de Intenção | Suporta reconhecimento de intenção por LLM, Function Call (chamada de função), fornece mecanismo de processamento de intenção baseado em plugins |
|
||||
| Reconhecimento de Intenção | Suporta reconhecimento de intenção por LLM externo e conectável, chamada de função autônoma do modelo de linguagem, fornece mecanismo de processamento de intenção baseado em plugins |
|
||||
| Sistema de Memória | Suporta memória local de curto prazo, memória via interface mem0ai, memória inteligente PowerMem, com funcionalidade de resumo de memória |
|
||||
| Base de Conhecimento | Suporta base de conhecimento RAGFlow, permitindo que o LLM julgue se deve acionar a base de conhecimento após receber a pergunta do usuário, e então responda à pergunta |
|
||||
| Chamada de Ferramentas | Suporta protocolo IOT do cliente, protocolo MCP do cliente, protocolo MCP do servidor, protocolo de endpoint MCP, funções de ferramentas personalizadas |
|
||||
@@ -254,7 +254,7 @@ Xiaozhi é um ecossistema. Ao utilizar este produto, você também pode conferir
|
||||
|
||||
| Método de Uso | Plataformas Suportadas | Plataformas Gratuitas |
|
||||
|:---:|:---:|:---:|
|
||||
| Chamadas via interface OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK | Zhipu, Gemini |
|
||||
| Chamadas via interface OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
|
||||
| Chamadas via interface Ollama | Ollama | - |
|
||||
| Chamadas via interface Dify | Dify | - |
|
||||
| Chamadas via interface FastGPT | FastGPT | - |
|
||||
@@ -298,7 +298,7 @@ Na verdade, qualquer VLLM que suporte chamadas via interface OpenAI pode ser int
|
||||
| Método de Uso | Plataformas Suportadas | Plataformas Gratuitas |
|
||||
|:---:|:---:|:---:|
|
||||
| Uso local | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
|
||||
| Chamadas via interface | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
| Chamadas via interface | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud e Bailian, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
|
||||
---
|
||||
|
||||
@@ -308,7 +308,7 @@ Na verdade, qualquer VLLM que suporte chamadas via interface OpenAI pode ser int
|
||||
|:---:|:---:|:---:|
|
||||
| Chamadas via interface | Serviço de voiceprint (voiceprint-api) | - |
|
||||
|
||||
O reconhecimento de voz por impressão digital exige um serviço [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) implantado separadamente (baseado em 3D-Speaker) e MySQL. Veja o [guia de integração de voiceprint](../voiceprint-integration.md).
|
||||
O reconhecimento de impressão vocal exige um serviço [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) implantado separadamente (baseado em 3D-Speaker) e MySQL. Veja o [guia de integração de voiceprint](../voiceprint-integration.md).
|
||||
|
||||
---
|
||||
|
||||
@@ -354,11 +354,10 @@ O reconhecimento de voz por impressão digital exige um serviço [voiceprint-api
|
||||
| <img src="../images/logo_contributors.png" width="160"> | [Contribuidores de Código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Agradecimentos a [todos os contribuidores de código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), seus esforços tornaram o projeto mais robusto e poderoso. |
|
||||
|
||||
|
||||
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
|
||||
|
||||
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
</picture>
|
||||
</a>
|
||||
+30
-31
@@ -6,7 +6,7 @@
|
||||
Dự án này dựa trên lý thuyết và công nghệ trí tuệ cộng sinh người-máy để phát triển hệ thống phần mềm và phần cứng thiết bị đầu cuối thông minh<br/>Cung cấp dịch vụ backend cho dự án phần cứng thông minh mã nguồn mở
|
||||
<a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a><br/>
|
||||
Được triển khai bằng Python, Java, Vue theo <a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">giao thức truyền thông Xiaozhi</a><br/>
|
||||
Hỗ trợ giao thức MQTT+UDP, giao thức Websocket, điểm truy cập MCP, nhận dạng giọng nói và kho tri thức
|
||||
Hỗ trợ giao thức MQTT+UDP, giao thức Websocket, điểm truy cập MCP, nhận dạng vân giọng và kho tri thức
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
@@ -39,7 +39,7 @@ Spearheaded by Professor Siyuan Liu's Team (South China University of Technology
|
||||
</br>
|
||||
Được dẫn dắt bởi nhóm Giáo sư Lưu Tư Nguyên (Đại học Bách khoa Nam Trung Quốc)
|
||||
</br>
|
||||
<img src="../images/hnlg.jpg" alt="华南理工大学" width="50%">
|
||||
<img src="../images/hnlg.jpg" alt="Đại học Bách khoa Nam Trung Quốc" width="50%">
|
||||
</p>
|
||||
|
||||
---
|
||||
@@ -55,31 +55,31 @@ Muốn xem hiệu quả sử dụng? Hãy xem video 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
|
||||
<picture>
|
||||
<img alt="响应速度感受" src="../images/demo9.png" /></picture>
|
||||
<img alt="Cảm giác tốc độ phản hồi" src="../images/demo9.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
|
||||
<picture>
|
||||
<img alt="速度优化秘诀" src="../images/demo6.png" /></picture>
|
||||
<img alt="Bí quyết tối ưu tốc độ" src="../images/demo6.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
|
||||
<picture>
|
||||
<img alt="小智数字人 支持语音唤醒" src="../images/demo8.png" /></picture>
|
||||
<img alt="Người ảo số Xiaozhi có đánh thức bằng giọng nói" src="../images/demo8.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
|
||||
<picture>
|
||||
<img alt="设备呼叫设备,打电话" src="../images/demo0.png" /></picture>
|
||||
<img alt="Gọi giữa các thiết bị" src="../images/demo0.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
|
||||
<picture>
|
||||
<img alt="复杂医疗场景" src="../images/demo1.png" /></picture>
|
||||
<img alt="Tình huống y tế phức tạp" src="../images/demo1.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -87,31 +87,31 @@ Muốn xem hiệu quả sử dụng? Hãy xem video 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
|
||||
<picture>
|
||||
<img alt="播放音乐查询天气播报新闻" src="../images/demo7.png" /></picture>
|
||||
<img alt="Phát nhạc, tra cứu thời tiết và đọc tin tức" src="../images/demo7.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
|
||||
<picture>
|
||||
<img alt="实时打断" src="../images/demo10.png" /></picture>
|
||||
<img alt="Ngắt giữa chừng theo thời gian thực" src="../images/demo10.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
|
||||
<picture>
|
||||
<img alt="拍照识物品" src="../images/demo12.png" /></picture>
|
||||
<img alt="Nhận dạng vật thể qua ảnh chụp" src="../images/demo12.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
|
||||
<picture>
|
||||
<img alt="控制家电开关" src="../images/demo5.png" /></picture>
|
||||
<img alt="Bật tắt thiết bị gia dụng" src="../images/demo5.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
|
||||
<picture>
|
||||
<img alt="多指令任务" src="../images/demo11.png" /></picture>
|
||||
<img alt="Nhiệm vụ nhiều lệnh" src="../images/demo11.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -119,31 +119,31 @@ Muốn xem hiệu quả sử dụng? Hãy xem video 🎥
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
|
||||
<picture>
|
||||
<img alt="MCP接入点" src="../images/demo13.png" /></picture>
|
||||
<img alt="Điểm truy cập MCP" src="../images/demo13.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
|
||||
<picture>
|
||||
<img alt="MQTT指令下发" src="../images/demo4.png" /></picture>
|
||||
<img alt="Gửi lệnh MQTT" src="../images/demo4.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
|
||||
<picture>
|
||||
<img alt="声纹识别" src="../images/demo14.png" /></picture>
|
||||
<img alt="Nhận dạng vân giọng" src="../images/demo14.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
|
||||
<picture>
|
||||
<img alt="自定义音色" src="../images/demo2.png" /></picture>
|
||||
<img alt="Giọng nói tùy chỉnh" src="../images/demo2.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
|
||||
<picture>
|
||||
<img alt="使用粤语交流" src="../images/demo3.png" /></picture>
|
||||
<img alt="Trò chuyện bằng tiếng Quảng Đông" src="../images/demo3.png" /></picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
@@ -170,7 +170,7 @@ Dự án này cung cấp hai phương pháp triển khai, vui lòng chọn theo
|
||||
| Phương pháp triển khai | Đặc điểm | Tình huống áp dụng | Tài liệu triển khai | Yêu cầu cấu hình | Video hướng dẫn |
|
||||
|---------|------|---------|---------|---------|---------|
|
||||
| **Cài đặt tối giản** | Đối thoại thông minh, quản lý đơn tác nhân | Môi trường cấu hình thấp, dữ liệu lưu trong tệp cấu hình, không cần cơ sở dữ liệu | [①Phiên bản Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Triển khai mã nguồn](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| `FunASR` cục bộ cần nhiều bộ nhớ; dùng toàn API thì mức dùng tài nguyên thấp | - |
|
||||
| **Cài đặt toàn bộ module** | Đối thoại thông minh, quản lý đa người dùng, quản lý đa tác nhân, bảng điều khiển thông minh | Trải nghiệm đầy đủ tính năng, dữ liệu lưu trong cơ sở dữ liệu |[①Phiên bản Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Triển khai mã nguồn](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Hướng dẫn tự động cập nhật triển khai mã nguồn](../dev-ops-integration.md) | `FunASR` cục bộ cần nhiều bộ nhớ, thêm tài nguyên cơ sở dữ liệu; dùng toàn API thì mức dùng tài nguyên thấp| [Video hướng dẫn khởi động mã nguồn cục bộ](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
|
||||
| **Cài đặt toàn bộ module** | Đối thoại thông minh, quản lý đa người dùng, quản lý đa tác nhân, thao tác qua bảng điều khiển thông minh | Trải nghiệm đầy đủ tính năng, dữ liệu lưu trong cơ sở dữ liệu |[①Phiên bản Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Triển khai mã nguồn](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Hướng dẫn tự động cập nhật triển khai mã nguồn](../dev-ops-integration.md) | `FunASR` cục bộ cần nhiều bộ nhớ, thêm tài nguyên cơ sở dữ liệu; dùng toàn API thì mức dùng tài nguyên thấp| [Video hướng dẫn khởi động mã nguồn cục bộ](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
|
||||
|
||||
Câu hỏi thường gặp và hướng dẫn liên quan, vui lòng tham khảo [liên kết này](../FAQ.md)
|
||||
|
||||
@@ -193,7 +193,7 @@ Công cụ kiểm tra dịch vụ: https://2662r3426b.vicp.fun/test/
|
||||
>
|
||||
> 2. `Cấu hình streaming`: Phù hợp với demo, đào tạo, hơn 2 đồng thời, v.v., sử dụng công nghệ xử lý streaming, tốc độ phản hồi nhanh hơn, trải nghiệm tốt hơn.
|
||||
>
|
||||
> Từ phiên bản `0.5.2`, dự án hỗ trợ cấu hình streaming, so với phiên bản đầu, tốc độ phản hồi cải thiện khoảng `2.5 giây`, cải thiện đáng kể trải nghiệm người dùng.
|
||||
> Từ phiên bản `0.5.2`, dự án hỗ trợ cấu hình streaming, so với các phiên bản trước đó, tốc độ phản hồi cải thiện khoảng `2.5 giây`, cải thiện đáng kể trải nghiệm người dùng.
|
||||
|
||||
| Tên module | Cài đặt miễn phí cho người mới | Cấu hình streaming |
|
||||
|:---:|:---:|:---:|
|
||||
@@ -211,7 +211,7 @@ Dự án này cung cấp các công cụ kiểm tra sau để giúp bạn xác m
|
||||
|
||||
| Tên công cụ | Vị trí | Phương pháp sử dụng | Mô tả chức năng |
|
||||
|:---:|:---|:---:|:---:|
|
||||
| Công cụ kiểm tra tương tác âm thanh | main》digital-human》index.html | Chạy `python start.py` trong `main/digital-human`, sau đó mở `http://127.0.0.1:8006/index.html` | Kiểm tra chức năng phát và nhận âm thanh, xác minh xử lý âm thanh phía Python có bình thường không |
|
||||
| Công cụ kiểm tra tương tác âm thanh | main》digital-human》index.html | Trong `main/digital-human`, cài đặt các phụ thuộc theo README, rồi chạy `python start.py` và mở `http://127.0.0.1:8006/index.html` | Kiểm tra chức năng phát và nhận âm thanh, xác minh xử lý âm thanh phía Python có bình thường không |
|
||||
| Công cụ kiểm tra phản hồi mô hình | main》xiaozhi-server》performance_tester.py | Thực hiện `python performance_tester.py` | Kiểm tra tốc độ phản hồi của các module cốt lõi như ASR(Nhận dạng giọng nói), LLM(Mô hình lớn), VLLM(Mô hình thị giác), TTS(Tổng hợp giọng nói) |
|
||||
|
||||
> 💡 Gợi ý: Khi kiểm tra tốc độ mô hình, chỉ kiểm tra các mô hình đã cấu hình khóa.
|
||||
@@ -219,7 +219,7 @@ Dự án này cung cấp các công cụ kiểm tra sau để giúp bạn xác m
|
||||
---
|
||||
## Danh sách tính năng ✨
|
||||
### Đã thực hiện ✅
|
||||

|
||||

|
||||
| Module chức năng | Mô tả |
|
||||
|:---:|:---|
|
||||
| Kiến trúc cốt lõi | Dựa trên [cổng MQTT+UDP](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket, máy chủ HTTP, cung cấp hệ thống quản lý bảng điều khiển và xác thực hoàn chỉnh |
|
||||
@@ -255,7 +255,7 @@ Xiaozhi là một hệ sinh thái, khi bạn sử dụng sản phẩm này, bạ
|
||||
|
||||
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|
||||
|:---:|:---:|:---:|
|
||||
| Gọi giao diện openai | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFlytek | Zhipu, Gemini |
|
||||
| Gọi giao diện openai | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
|
||||
| Gọi giao diện ollama | Ollama | - |
|
||||
| Gọi giao diện dify | Dify | - |
|
||||
| Gọi giao diện fastgpt | Fastgpt | - |
|
||||
@@ -281,7 +281,7 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
|
||||
|
||||
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|
||||
|:---:|:---:|:---:|
|
||||
| Gọi giao diện | EdgeTTS, iFlytek, Volcano Engine, Tencent Cloud, Alibaba Cloud và Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS | EdgeTTS, CosyVoiceSiliconflow(một phần) |
|
||||
| Gọi giao diện | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud và Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS | EdgeTTS, CosyVoiceSiliconflow(một phần) |
|
||||
| Dịch vụ cục bộ | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, IndexStreamTTS, PaddleSpeechTTS | IndexStreamTTS, PaddleSpeechTTS, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
|
||||
|
||||
---
|
||||
@@ -299,7 +299,7 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
|
||||
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|
||||
|:---:|:---:|:---:|
|
||||
| Sử dụng cục bộ | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
|
||||
| Gọi giao diện | FunASRServer, Volcano Engine, iFlytek, Tencent Cloud, Alibaba Cloud, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
| Gọi giao diện | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud và Bailian, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
|
||||
|
||||
---
|
||||
|
||||
@@ -307,9 +307,9 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
|
||||
|
||||
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|
||||
|:---:|:---:|:---:|
|
||||
| Gọi giao diện | Dịch vụ nhận dạng giọng nói (voiceprint-api) | - |
|
||||
| Gọi giao diện | Dịch vụ nhận dạng vân giọng (voiceprint-api) | - |
|
||||
|
||||
Nhận dạng giọng nói theo dấu vân tay cần dịch vụ [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) được triển khai riêng (dựa trên 3D-Speaker) và MySQL. Xem [hướng dẫn tích hợp voiceprint](../voiceprint-integration.md).
|
||||
Nhận dạng vân giọng cần dịch vụ [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) được triển khai riêng (dựa trên 3D-Speaker) và MySQL. Xem [hướng dẫn tích hợp voiceprint](../voiceprint-integration.md).
|
||||
|
||||
---
|
||||
|
||||
@@ -355,11 +355,10 @@ Nhận dạng giọng nói theo dấu vân tay cần dịch vụ [voiceprint-api
|
||||
| <img src="../images/logo_contributors.png" width="160"> | [Người đóng góp mã](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Cảm ơn [tất cả người đóng góp mã](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), sự cống hiến của bạn khiến dự án mạnh mẽ và vững chắc hơn. |
|
||||
|
||||
|
||||
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
|
||||
|
||||
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
|
||||
</picture>
|
||||
</a>
|
||||
@@ -2,7 +2,7 @@ import os
|
||||
import importlib.util
|
||||
import asyncio
|
||||
|
||||
print("使用前请根据docs/performance_tester.md的说明准备配置。")
|
||||
print("使用前请根据仓库根目录下 docs/performance_tester.md 的说明准备配置。")
|
||||
|
||||
|
||||
def list_performance_tester_modules():
|
||||
|
||||
Reference in new issue
Block a user