Compare commits

..
Author SHA1 Message Date
Sakura-RanChen 398f0bd097 调整相关逻辑, 2026-11-03 15:56:07 +08:00
Sakura-RanChen 7e450fca0d fix(manager-mobile): 修复Node 随机源误判、垫片健壮性及补丁哈希更新 2026-11-02 10:33:33 +08:00
Sakura-RanChen 42f7e55c72 fix(manager): 升级 sm-crypto 至 0.5.8 修复 SM2 弱随机数漏洞 (CVE-2026-73567) 2026-10-30 15:18:18 +08:00
wengzh 1a095d5bed Merge pull request #3523 from xinnan-tech/py-test-fix
Py test fix
2026-10-29 14:20:59 +08:00
Sakura-RanChen 01289b8fd4 仅保留mcp2.x行为 2026-10-29 14:17:38 +08:00
shinjiyuandshinjiyu ef2fcdbcb1 fix(server_mcp): read Tool.input_schema for mcp 2.3.0 (#3522)
mcp==2.3.0 Tool stores the JSON Schema on input_schema (alias
inputSchema). Accessing tool.inputSchema raises AttributeError and
aborts server_mcp init after a successful tools/list.

Use getattr fallback so function-calling conversion works.

Closes #3521

Co-authored-by: shinjiyu <shinjiyu@users.noreply.github.com>
2026-10-29 14:08:44 +08:00
hrz 942ac2fe12 通讯录没起名时不再取对方设备的名字 (#3473)
* 别把对方给设备起的名字抄进自己的通讯录

通讯录里没起名的时候,以前会跑去设备表查目标设备的名字。那是设备主人在智控台上自己起的,
直接抄进调用方的通讯录,等于把别人的昵称漏了出去。现在改成一律用 MAC 尾号称呼,
只有目标是自己那台设备时,才用自己设备表里的名字。补了单测,覆盖对方已起名、目标是自己、
显式起名、目标没登记这几种情况。

* 取自己的设备名改用不会撞行的查法

设备表里同一个 MAC 可能有两行只差大小写,一次只准取一条的查法撞上就抛异常,保存通讯录整个接口带崩。改成按归属查的那一种,按 id 取第一条。

补齐自己那台设备在设备表里没起名时退回尾号称呼的用例,另有几条用例的名字跟实际测的东西对不上,一并改掉。
2026-10-29 11:56:48 +08:00
wengzh 4dbed0ab1d Merge pull request #3520 from xinnan-tech/py-test-weather
fix: harden weather and news plugins (#3465)
2026-10-29 10:10:50 +08:00
Jacobandhrz 15d9b0fba1 fix: harden weather and news plugins (#3465)
* fix: harden weather and news plugins

* 修掉天气新闻插件的三处鲁棒性缺陷

城市查询不再写死国内范围,境外城市也能查到。实时天气和七天预报要都拿到才走 JSON,缺一个就整体退回网页路径,不产出残缺报告。网页抓取加上跟随跳转,旧链接的 302 也能拿到内容。城市查不到时不再让用户去核对地名,多数情况是密钥失效。报告补上气压和降水量。新闻 URL 拼接抽成纯函数,配了 23 项单测。

---------

Co-authored-by: hrz <1710360675@qq.com>
2026-10-29 09:46:01 +08:00
wengzh 8c5330f742 Merge pull request #3516 from xinnan-tech/py-test-think
Py test think
2026-10-28 16:38:12 +08:00
Sakura-RanChen 549261fdff PR 提供的keep 逃生门要求用户自行发现问题、了解该魔法值并手改 configJson,对普通用户不可达。
调整LLM thinking 域名禁用规则支持按模型名豁免(避免 MaaS glm-5.3 被注入参数后 400)
2026-10-28 16:18:58 +08:00
2cb0562ade feat(llm): 华为云 MaaS 思考强度实例级配置 + 管理台 select 字段类型 (#3446)
- openai provider: THINKING_DISABLED_DOMAINS 补 modelarts-maas.com
- openai provider: 实例配置支持可选 thinking 字段(off/low/medium/high/keep),
  实例值优先于域名默认表; keep=完全直通(适配强制思考且拒绝该参数的模型, 如 MaaS glm-5.3)
- manager-web: 模型配置表单渲染器支持 select 字段类型(el-select + field.options)
  (ModelEditDialog / AddModelDialog 两处渲染点与类型映射)

Co-authored-by: SecInsight <secinsight@users.noreply.github.com>
Co-authored-by: Sakura-RanChen <1908198662@qq.com>
2026-10-28 15:20:28 +08:00
hrz d9c2e8c5d9 德语版README把MCP接入点改用Zugriffspunkt (#3511)
Endpunkt说的是终端点,这里是访问点,术语换过来跟英文版一致。
2026-10-27 21:18:24 +08:00
hrz 1a675e4d6b 部署文档改准:示例配置、镜像架构和日志样例 (#3507)
* 部署文档改准:示例配置、镜像架构和日志样例

最简配置示例里选中的模型改成仓库默认那个;镜像架构那句改成同时提供两种架构;升级步骤去掉单模块没有的 web 容器;compose 文件名、章节编号、配置路径和日志版本号一并改对。

* 部署文档的启动日志样例改成实际模块串

全局日志器拿不到各模块的名字,样例里那串缩写换成实际输出的样子。

* 部署文档的升级步骤编号改对并补全日志样例

升级那几步改成4开头,删掉单模块用不到的web镜像,样例补上视觉分析接口那行,分隔符也对齐实际输出。

* 部署文档里配置说明的路径改成仓库链接

按 docker 方式部署的人,xiaozhi-server 目录下并没有 config.yaml,这句路径换成能点开的仓库链接,两种方式都用得上。
2026-10-27 21:13:05 +08:00
hrz 40a7ed0391 三份译文里的术语跟本文件其它地方对齐 (#3506)
德语版把回答速度统一成反应速度,英语版和葡语版把 MCP 接入点统一成本文件其它地方用的说法。
2026-10-27 21:10:55 +08:00
hrz 62e316f298 把 README 的错字修正和品牌名统一带到 main (#3504)
* 修中文 README 的四处错字并统一品牌名写法

中文 README 自己有四处错:括号不配对、该写客户端的地方写成了客户、提示语用逗号收尾、OpenAI 写成了小写。葡语版和越南语版跟着源文一起小写的几处也改了。

* 把中文源的括号和越南语版的句号改对

中文源文 Memory 那行第 3 格还是全角括号,改成半角,
跟另外五版一样。越南语版提示语那行还是逗号收尾,改成句号。
2026-10-27 21:09:57 +08:00
hrz 76cef92cb2 修正性能测试和声纹识别两份文档里跟代码对不上的十二处 (#3502)
* 修正两份操作文档里跟代码对不上的地方

性能测试那份把配置路径写错、没提要先装依赖,步骤里少了非流式语音合成,流式语音合成要配哪些供应器也没写;声纹识别那份的重启命令指向两个不同的编排文件,有两句话连写三遍,编号也乱了,示例漏了相似度阈值。逐条改到跟代码一致。

* 把性能测试文档的说明改得跟代码一致

供应器表里两家的写法与代码相反,改成如实描述;步骤清单补回流式语音识别;
示例里的视觉模型段改成能跑通的形态;声纹文档两个重复的标题改掉一个。

* 性能测试文档的说法跟Linkerai跳过行为对齐

它没配就整项跳过,不再算进报错后跳过那一类。与#3510的行为对齐。

* 性能测试文档改对缺供应器时的记法

不是跳过,是报错后记成失败,结果表里多一行。带守卫的那几家才是整项跳过。

* 强调易错处
2026-10-27 21:09:09 +08:00
hrz 826c25d3e5 修正 PaddleSpeech 部署文档里对不上代码的四处 (#3501)
* 修正 PaddleSpeech 部署文档里对不上的地方

示例配置里写了 provider 根本不读的那项采样率,还说 http 协议能自动选,其实只实现了 websocket;代码那一节是空的;启动命令没说在哪个目录跑,配置文件的路径只给了 Windows 写法。逐条改到跟代码一致。

* 补上 PaddleSpeech 部署文档的配置目录和环境说明

配置文件那节改成新建,写明 data 目录被 gitignore 忽略,要先建出来。
数字人页面的启动命令改成从仓库根目录算,另开一个终端。
代码块围栏换成 bash,另补一段说明小智服务用哪个 Python 环境,
以及数字人页面那套依赖在哪。

* PaddleSpeech 部署文档修四处会照抄失败的地方

启动命令的围栏改成 bash;建 data 目录那条标明在仓库根目录执行;启动服务那段不再重复 cd;数字人那节补一句 numpy 的依赖来自哪两个文件。

* 修正conda环境名
2026-10-27 21:05:07 +08:00
hrz f1965167cc 性能测试跳过未配置的Linkerai语音合成 (#3510)
配置里没有这个供应器时整项跳过,不再报一条测试失败。
2026-10-27 15:35:38 +08:00
wengzh 6c352e5978 Merge pull request #3436 from xinnan-tech/perf/hello-handshake-fast
hello 握手首屏响应从 4s 降到 200ms
2026-10-27 14:34:13 +08:00
Sakura-RanChen 032f6f1582 fix: 修复唤醒流程错误阻塞事件接收循环 2026-10-27 10:36:35 +08:00
Sakura-RanChen 82d2acc78c - audio_to_data 缓存键加入文件 mtime:regen 覆写 wav 后旧缓存自动失效,
并在 regen 后预编码新文件,唤醒回复不再播旧音频且延迟收益保留
- enhanced_prompt 缓存键补上 emoji_enabled 与 prompt 哈希:
  同设备不同客户端(ESP32/digital-human)不再互相污染,改配置即时生效
- 删除 initialize 响应后的 tools/list 兜底补发:id=2 同连接复用
  违反 JSON-RPC 与设备端严格递增约定(#3461/#3485),且兜不住任何场景
2026-10-26 17:09:09 +08:00
Sakura-RanChen 77be288c8e Merge branch 'perf/hello-handshake-fast' of https://github.com/xinnan-tech/xiaozhi-esp32-server into perf/hello-handshake-fast
# Conflicts:
#	main/xiaozhi-server/requirements.txt
2026-10-26 17:04:54 +08:00
hrz 0492dc5924 del:移除测试项 2026-10-26 16:36:46 +08:00
Sakura-RanChen ee726f49e6 Merge remote-tracking branch 'origin/main' into perf/hello-handshake-fast
# Conflicts:
#	main/xiaozhi-server/requirements.txt
2026-10-26 15:05:17 +08:00
hrz 4ed3197a6f pr02 P2 注释预热和百分位
首段缓冲帧数注释改成说人话版本;唤醒词预热按常见音色列表走并把吞掉的失败改成日志提示;基准脚本百分位改成线性插值与 numpy 一致;requirements 补上基准测试用到的 websocket-client。
2026-10-14 23:19:37 +08:00
hrz 4dd194ef8b pr02 P1 三条全部兜底
device_prompt 缓存跨日不失效导致今天日期/天气被烤死,改成独立 key 并带 5 分钟 ttl;mcp 工具列表在收到初始化响应后补回并发发送,helloHandle 预发万一丢帧时也能兜住;启服脚本去硬编码路径与全机 pkill,改用相对路径与 PID 文件。
2026-10-14 23:19:33 +08:00
hrz 8c41faac8b 握手基准脚本后台取帧贴近真实客户端
原版 recv 在二进制语音帧期间必须按顺序 drain 才能拿到 mcp 文本帧,测出来偏慢。改成后台线程持续把帧按到达时间入队、主循环按队列处理,模拟 digital-human 的异步 onmessage 行为。附 handshake_summary_real.txt 是方案 C 后 10 次跑出的数据。
2026-10-14 23:05:20 +08:00
hrz 597ab5f415 每连接冗余计算与日志降级
提示词模板结果按设备 id 缓存复用、声纹未配置和 mcp 配置文件缺失两条提示从警告降到调试,让每条新连接少跑几百毫秒的模板渲染、少两次同步日志。
2026-10-14 23:05:15 +08:00
hrz 4f1cb98503 hello 握手并发与缓存
唤醒词音频改为带缓存且进程启动时预热、首段流控缓冲帧数从 5 调到 2、hello 阶段并发发出 mcp 工具列表帧让客户端在听到声音前就准备好、删掉 mcp 初始化后的固定 sleep。砍幅 t_hello_to_tools_ready p50 从 4019ms 降到 219ms。
2026-10-14 23:05:12 +08:00
43 changed files with 1392 additions and 169 deletions

No files matched your search

+7 -7
View File
@@ -175,7 +175,7 @@ Spearheaded by Professor Siyuan Liu's Team (South China University of Technology
常见问题及相关教程,可参考[这个链接](./docs/FAQ.md)
> 💡 提示:以下是按最新代码部署后的测试平台,有需要可烧录测试,并发为6个,每天会清空数据,
> 💡 提示:以下是按最新代码部署后的测试平台,有需要可烧录测试,并发为6个,每天会清空数据。
```
智控台地址: https://2662r3426b.vicp.fun
@@ -203,7 +203,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| VLLM(视觉大模型) | glm-4v-flash(智谱) | 👍qwen3.5-flash(阿里百炼) |
| TTS(语音合成) | EdgeTTS(微软) | 👍HuoshanDoubleStreamTTS(火山流式) |
| Intent(意图识别) | function_call(函数调用) | function_call(函数调用) |
| Memory(记忆功能) | mem_local_short(本地短期记忆) | mem_local_short(本地短期记忆) |
| Memory(记忆功能) | mem_local_short(本地短期记忆) | mem_local_short(本地短期记忆) |
如果您关心各组件的耗时,请查阅[小智各组件性能测试报告](https://github.com/xinnan-tech/xiaozhi-performance-research),可按报告中的测试方法在您的环境中实际测试。
@@ -231,7 +231,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 意图识别 | 支持外挂的大模型意图识别、大模型自主函数调用,提供插件化意图处理机制 |
| 记忆系统 | 支持本地短期记忆、mem0ai接口记忆、PowerMem智能记忆,具备记忆总结功能 |
| 知识库 | 支持RAGFlow知识库,让大模型判断需要调度知识库后再回答 |
| 工具调用 | 支持客户端IOT协议、客户MCP协议、服务端MCP协议、MCP接入点协议、自定义工具函数 |
| 工具调用 | 支持客户端IOT协议、客户端MCP协议、服务端MCP协议、MCP接入点协议、自定义工具函数 |
| 指令下发 | 依托MQTT协议,支持从智控台将MCP指令下发到ESP32设备 |
| 管理后台 | 提供Web管理界面,支持用户管理、系统配置和设备管理;界面支持中文简体、中文繁体、英文、德语、越南语、巴西葡萄牙语显示 |
| 测试工具 | 提供模型响应性能测试工具和音频交互测试工具 |
@@ -256,7 +256,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 使用方式 | 支持平台 | 免费平台 |
|:---:|:---:|:---:|
| openai 接口调用 | 阿里百炼、火山引擎、DeepSeek、智谱、Gemini、科大讯飞、LM Studio | 智谱、Gemini |
| OpenAI 接口调用 | 阿里百炼、火山引擎、DeepSeek、智谱、Gemini、科大讯飞、LM Studio | 智谱、Gemini |
| ollama 接口调用 | Ollama | - |
| dify 接口调用 | Dify | - |
| fastgpt 接口调用 | Fastgpt | - |
@@ -264,7 +264,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| xinference 接口调用 | Xinference | - |
| homeassistant 接口调用 | HomeAssistant | - |
实际上,任何支持 openai 接口调用的 LLM 均可接入使用。
实际上,任何支持 OpenAI 接口调用的 LLM 均可接入使用。
---
@@ -272,9 +272,9 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 使用方式 | 支持平台 | 免费平台 |
|:---:|:---:|:---:|
| openai 接口调用 | 阿里百炼、智谱ChatGLMVLLM | 智谱ChatGLMVLLM |
| OpenAI 接口调用 | 阿里百炼、智谱ChatGLMVLLM | 智谱ChatGLMVLLM |
实际上,任何支持 openai 接口调用的 VLLM 均可接入使用。
实际上,任何支持 OpenAI 接口调用的 VLLM 均可接入使用。
---
+15 -17
View File
@@ -2,7 +2,7 @@
![请参考-最简化架构图](../docs/images/deploy1.png)
# 方式一:Docker只运行Server
`0.8.2`版本开始,本项目发行的docker镜像只支持`x86架构`,如果需要在`arm64架构`的CPU上部署,可按照[这个教程](docker-build.md)在本机编译`arm64的镜像`。
本项目发行的docker镜像同时提供`amd64`和`arm64`两种架构,可以直接拉取使用;如需自定义镜像,可按照[这个教程](docker-build.md)在本机编译。
## 1. 安装docker
@@ -36,9 +36,9 @@ xiaozhi-server
#### 1.1.3 下载配置文件
你需要下载两个配置文件:`docker-compose.yaml` 和 `config.yaml`。需要从项目仓库下载这两个文件。
你需要下载两个配置文件:`docker-compose.yml` 和 `config.yaml`。需要从项目仓库下载这两个文件。
##### 1.1.3.1 下载 docker-compose.yaml
##### 1.1.3.1 下载 docker-compose.yml
用浏览器打开[这个链接](../main/xiaozhi-server/docker-compose.yml)。
@@ -91,25 +91,22 @@ docker logs -f xiaozhi-esp32-server
这时,你就要留意日志信息,可以根据这个教程,判断是否成功了。[跳转到运行状态确认](#运行状态确认)
## 5. 版本升级操作
## 4. 版本升级操作
如果后期想升级版本,可以这么操作
5.1、备份好`data`文件夹中的`.config.yaml`文件,一些关键的配置到时复制到新的`.config.yaml`文件里。
4.1、备份好`data`文件夹中的`.config.yaml`文件,一些关键的配置到时复制到新的`.config.yaml`文件里。
请注意是对关键密钥逐个复制,不要直接覆盖。因为新的`.config.yaml`文件可能有一些新的配置项,旧的`.config.yaml`文件不一定有。
5.2、执行以下命令
4.2、执行以下命令
```
docker stop xiaozhi-esp32-server
docker rm xiaozhi-esp32-server
docker stop xiaozhi-esp32-server-web
docker rm xiaozhi-esp32-server-web
docker rmi ghcr.nju.edu.cn/xinnan-tech/xiaozhi-esp32-server:server_latest
docker rmi ghcr.nju.edu.cn/xinnan-tech/xiaozhi-esp32-server:web_latest
```
5.3、重新按docker方式部署
4.3、重新按docker方式部署
# 方式二:本地源码只运行Server
@@ -213,14 +210,14 @@ prompt: |
请你像一个人一样说话,请勿返回配置xml及其他特殊字符。
selected_module:
LLM: DoubaoLLM
LLM: ChatGLMLLM
LLM:
ChatGLMLLM:
api_key: xxxxxxxxxxxxxxx.xxxxxx
```
建议先将最简单的配置运行起来,然后再去`xiaozhi/config.yaml`阅读配置的使用说明。
建议先将最简单的配置运行起来,然后再去[`../main/xiaozhi-server/config.yaml`](../main/xiaozhi-server/config.yaml)阅读配置的使用说明。
比如你要换更换模型,修改`selected_module`下的配置就行。
## 模型文件
@@ -238,11 +235,12 @@ LLM:
如果你能看到,类似以下日志,则是本项目服务启动成功的标志。
```
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-OTA接口是 http://192.168.4.123:8003/xiaozhi/ota/
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-Websocket地址是 ws://192.168.4.123:8000/xiaozhi/v1/
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-=======================================================
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-OTA接口是 http://192.168.4.123:8003/xiaozhi/ota/
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-视觉分析接口是 http://192.168.4.123:8003/mcp/vision/explain
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-Websocket地址是 ws://192.168.4.123:8000/xiaozhi/v1/
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-=============================================================
```
正常来说,如果您是通过源码运行本项目,日志会有你的接口地址信息。
+40 -8
View File
@@ -50,10 +50,11 @@ paddlespeech tts --input "你好,这是一次测试"
### 7.修改tts_online_application.yaml配置
参考目录 ```"PaddleSpeech\demos\streaming_tts_server\conf\tts_online_application.yaml"```
Linux / WSL 2 用正斜杠写法:```"PaddleSpeech/demos/streaming_tts_server/conf/tts_online_application.yaml"```
选择```tts_online_application.yaml```文件用编辑器打开,设置```protocol```为```websocket```
### 8.启动服务
```yaml
```bash
paddlespeech_server start --config_file ./demos/streaming_tts_server/conf/tts_online_application.yaml
#官方默认启动命令:
paddlespeech_server start --config_file ./conf/tts_online_application.yaml
@@ -68,10 +69,19 @@ INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8092 (Press CTRL+C to quit)
```
## 三、修改小智的配置文件
### 1.```main/xiaozhi-server/core/providers/tts/paddle_speech.py```
## 三、配置小智服务
### 1. 无需修改代码(provider 已内置于 main/xiaozhi-server/core/providers/tts/paddle_speech.py)
### 2.新建 `main/xiaozhi-server/data/.config.yaml`
`main/xiaozhi-server/data/` 目录不在仓库里,是被 `.gitignore` 忽略掉的,所以第一次用要先把它建出来,再在里面新建 `.config.yaml` 文件(在仓库根目录执行,上面的 `cd PaddleSpeech` 之后当前目录还停在 PaddleSpeech 源码目录里):
```bash
mkdir -p main/xiaozhi-server/data
```
这个文件是必需的,少了它小智服务起不来,会直接报错退出。
### 2.```main/xiaozhi-server/data/.config.yaml```
使用单模块部署
```yaml
selected_module:
@@ -79,19 +89,41 @@ selected_module:
TTS:
PaddleSpeechTTS:
type: paddle_speech
protocol: websocket
protocol: websocket # 当前仅支持 websocket
url: ws://127.0.0.1:8092/paddlespeech/tts/streaming # TTS 服务的 URL 地址,指向本地服务器 [websocket默认ws://127.0.0.1:8092/paddlespeech/tts/streaming]
spk_id: 0 # 发音人 ID,0 通常表示默认的发音人
sample_rate: 24000 # 采样率 [websocket默认24000,http默认0 自动选择]
speed: 1.0 # 语速,1.0 表示正常语速,>1 表示加快,<1 表示减慢
volume: 1.0 # 音量,1.0 表示正常音量,>1 表示增大,<1 表示减小
save_path: # 保存路径
```
采样率固定 24000,provider 内部写死,配置里不用写这一项。
### 3.启动xiaozhi服务
```py
小智服务用**另一个** Python 环境,不要用前面给 PaddleSpeech 建的 `paddle_env` —— 那个环境里装的是 PaddleSpeech 自己那套依赖,版本由它决定,跟本项目锁定的版本不一定一致(`paddlespeech` 对 numpy 没写版本上限,装进去的 numpy 未必是本项目要的那个)。本项目推荐的 Python 版本和依赖版本写在 `main/xiaozhi-server/requirements.txt` 里(文件开头写着推荐 python3.10,另外锁了 torch 2.2.2、numpy 1.26.4)。
先建环境、装依赖(在仓库根目录执行):
```bash
conda create -n xiaozhi-esp32-server python=3.10 -y
conda activate xiaozhi-esp32-server
cd main/xiaozhi-server
pip install -r requirements.txt
```
然后在 `main/xiaozhi-server` 目录下启动(目录不能换:provider 是按相对路径 `core/providers/tts/` 找的,见 `core/utils/tts.py:35`):
```bash
# 上一段已经 cd 到 main/xiaozhi-server 了,这段不用再 cd
# 如果是另开的新终端,先 cd main/xiaozhi-server 再执行
python app.py
```
启动`main/digital-human`下的`python start.py`后,打开`http://127.0.0.1:8006/index.html`,测试连接和发送消息时paddlespeech端是否有输出日志
要验证语音合成的实际输出,另开一个终端,在仓库根目录下跑数字人页面(这套代码另有一份依赖清单,在 `main/digital-human/wakeword_runtime/requirements.txt` 里,缺了会报缺包;唤醒词那部分还要 numpy —— `core/detector.py:7` 和 `core/microphone.py:4` 在模块顶层就 import numpy,用上面建的 `xiaozhi-esp32-server` 跑的话 numpy 已经在里面,不会有这个问题):
```bash
cd main/digital-human
python start.py
```
打开`http://127.0.0.1:8006/index.html`,测试连接和发送消息时paddlespeech端是否有输出日志
输出日志参考:
```
+47 -2
View File
@@ -1,8 +1,15 @@
# 语音识别、流式语音识别、大语言模型、非流式语音合成、流式语音合成、视觉模型的性能测试工具使用指南
0.先装依赖(在仓库根目录执行)
```
cd main/xiaozhi-server
pip install -r requirements.txt
```
测试脚本要 import `yaml`、`aiohttp`、`tabulate`,少一个都起不来。`tabulate` 不在 requirements.txt 里,是 `edge-tts` 间接带进来的。
1.在main/xiaozhi-server目录下创建data目录
2.在data目录下创建.config.yaml文件
3.在.data/config.yaml中,写入你的语音识别、大语言模型、流式语音合成、视觉模型的参数
3.在data/.config.yaml中,写入你的语音识别、流式语音识别、大语言模型、非流式语音合成、流式语音合成、视觉模型的参数
例如:
```
LLM:
@@ -16,12 +23,50 @@ LLM:
api_key: 你的chat-glm web key
TTS:
EdgeTTS:
# 定义TTS API类型
type: edge
voice: zh-CN-XiaoxiaoNeural
VLLM:
ChatGLMVLLM:
# 定义视觉模型API类型
type: openai
# glm-4v-flash 是智谱的视觉模型,需要在智谱平台申请api_key
model_name: glm-4v-flash
url: https://open.bigmodel.cn/api/paas/v4/
api_key: 你的api_key
ASR:
FunASR:
type: fun_local
model_dir: models/SenseVoiceSmall
language: auto
```
4.在main/xiaozhi-server目录下运行performance_tester.py:
```
python performance_tester.py
```
```
## 几点说明
- 脚本第一行提示的 `docs/performance_tester.md` 就是本文档,它在仓库根目录的 `docs/` 下;脚本本身在 `main/xiaozhi-server/` 目录里跑,两处不是一个目录。
- LLM、VLLM、非流式语音合成这三项会把 `config.yaml` 和 `data/.config.yaml` 两份配置合并起来再遍历,所以实际会测**所有配好 api_key 的供应器**,不只是你在 `.config.yaml` 里写的那一个;api_key 还是占位符的会被跳过。
- 非流式的语音识别这一项读配置的方式和别的不一样:它自己扫 `data/` 目录下的 `*.config.yaml`,只取其中的 `ASR` 段,**不会**用 `config.yaml` 里的默认值。想测语音识别,就得把供应器写进 `data/.config.yaml`。
- 流式语音识别读的是合并后的配置,固定测 `DoubaoStreamASR`、`Qwen3ASRFlash`、`XunfeiStreamASR` 这三家。`config.yaml` 里这三项带的 key 是占位符,要在 `data/.config.yaml` 的 `ASR` 段里填上自己的 key 才会测出延迟。
- 视觉模型的配置写在 `data/.config.yaml` 的 `VLLM:` 段里,这段**不能只写一个空键**:写空的话合并后整段是空的,这项测试会直接跳过、什么都不测。
- 流式语音合成一次会跑这些供应器:
| 供应器 | 说明 |
|---|---|
| `AliyunStreamTTS` | 阿里云流式 TTS,无条件调用 |
| `HuoshanDoubleStreamTTS` | 火山引擎流式 TTS,无条件调用 |
| `PaddleSpeechTTS` | PaddleSpeech 流式 TTS,无条件调用 |
| `IndexStreamTTS` | IndexStream 流式 TTS,无条件调用 |
| `LinkeraiTTS` | `config.yaml` 的 TTS 段里没有这个供应器,默认不测、整项跳过;要测得自己在 `.config.yaml` 里补上 |
| `AliBLTTS` | 阿里百炼流式 TTS,默认就会测:判断读的是合并后的配置,`config.yaml` 里本来就带着这一项 |
| `XunFeiTTS` | 讯飞流式 TTS,同上,默认就会测 |
`AliBLTTS` 和 `XunFeiTTS` 这两项默认带的 key 是占位符,不在 `.config.yaml` 里换成自己的 key 就会报失败;想直接跳过,把这两项在 `.config.yaml` 里写成空的(键名下面不填内容)就行。
标了「无条件调用」的这几个(`AliyunStreamTTS`、`HuoshanDoubleStreamTTS`、`PaddleSpeechTTS`、`IndexStreamTTS`),供应器缺失或被清空时,那一项会报错,错误被测试方法内部的异常处理吃掉,这一项被记成失败 —— 结果表里会多出一行失败记录,但不影响其它项。带守卫的那几家(`AliBLTTS`、`XunFeiTTS`、`LinkeraiTTS`)是整项跳过:没配就不报错、结果表里也不会有这一行。
+4 -4
View File
@@ -6,7 +6,7 @@
Dieses Projekt basiert auf der Theorie und Technologie der Mensch-Maschine-symbiotischen Intelligenz zur Entwicklung intelligenter Terminal-Hardware- und Software-Systeme<br/>und bietet Backend-Dienste für das Open-Source-Hardware-Projekt
<a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a><br/>
Implementiert mit Python, Java und Vue gemäß dem <a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">Xiaozhi-Kommunikationsprotokoll</a><br/>
Unterstützt MQTT+UDP-Protokoll, Websocket-Protokoll, MCP-Endpunkte, Stimmabdruckerkennung und Wissensdatenbank
Unterstützt MQTT+UDP-Protokoll, Websocket-Protokoll, MCP-Zugriffspunkte, Stimmabdruckerkennung und Wissensdatenbank
</p>
<p align="center">
@@ -55,7 +55,7 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="Gefühl der Antwortgeschwindigkeit" src="../images/demo9.png" /></picture>
<img alt="Das Gefühl der Reaktionsgeschwindigkeit" src="../images/demo9.png" /></picture>
</a>
</td>
<td>
@@ -119,7 +119,7 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP-Endpunkt" src="../images/demo13.png" /></picture>
<img alt="MCP-Zugriffspunkt" src="../images/demo13.png" /></picture>
</a>
</td>
<td>
@@ -229,7 +229,7 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
| Absichtserkennung | Unterstützt externe LLM-Absichtserkennung, autonome Funktionsaufrufe durch große Modelle, bietet plugin-basierten Absichtsverarbeitungsmechanismus |
| Gedächtnissystem | Unterstützt lokales Kurzzeitgedächtnis, mem0ai-Schnittstellengedächtnis, PowerMem intelligentes Gedächtnis, mit Gedächtniszusammenfassungsfunktion |
| Wissensdatenbank | Unterstützt RAGFlow-Wissensdatenbank, ermöglicht großem Modell die Bewertung, ob Wissensdatenbank benötigt wird, bevor geantwortet wird |
| Werkzeugaufruf | Unterstützt Client-IOT-Protokoll, Client-MCP-Protokoll, Server-MCP-Protokoll, MCP-Endpunktprotokoll, benutzerdefinierte Werkzeugfunktionen |
| Werkzeugaufruf | Unterstützt Client-IOT-Protokoll, Client-MCP-Protokoll, Server-MCP-Protokoll, MCP-Zugriffspunktprotokoll, benutzerdefinierte Werkzeugfunktionen |
| Befehlsübermittlung | Basierend auf MQTT-Protokoll, unterstützt die Übermittlung von MCP-Befehlen von der intelligenten Steuerkonsole an ESP32-Geräte |
| Verwaltungs-Backend | Bietet Web-Verwaltungsoberfläche, unterstützt Benutzerverwaltung, Systemkonfiguration und Geräteverwaltung; Oberfläche unterstützt vereinfachtes Chinesisch, traditionelles Chinesisch, Englisch, Deutsch, Vietnamesisch und brasilianisches Portugiesisch an |
| Testwerkzeuge | Bietet Leistungstestwerkzeuge für Modellantworten und Audio-Interaktionstestwerkzeuge |
+1 -1
View File
@@ -229,7 +229,7 @@ This project provides the following testing tools to help you verify the system
| Intent Recognition | Supports pluggable external LLM intent recognition and autonomous LLM function calling, provides plugin-based intent processing mechanism |
| Memory System | Supports local short-term memory, mem0ai interface memory, PowerMem intelligent memory, with memory summarization functionality |
| Knowledge Base | Supports RAGFlow knowledge base, enabling LLM to judge whether to schedule the knowledge base after receiving the user's question, and then answer the question |
| Tool Calling | Supports client IOT protocol, client MCP protocol, server MCP protocol, MCP endpoint protocol, custom tool functions |
| Tool Calling | Supports client IOT protocol, client MCP protocol, server MCP protocol, MCP access point protocol, custom tool functions |
| Command Delivery | Supports MCP command delivery to ESP32 devices via MQTT protocol from Smart Console |
| Management Backend | Provides Web management interface, supports user management, system configuration and device management; Supports Simplified Chinese, Traditional Chinese, English, German, Vietnamese and Brazilian Portuguese display |
| Testing Tools | Provides model response performance testing tools and audio interaction testing tools |
+2 -2
View File
@@ -229,7 +229,7 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
| Reconhecimento de Intenção | Suporta reconhecimento de intenção por LLM externo e conectável, chamada de função autônoma do modelo de linguagem, fornece mecanismo de processamento de intenção baseado em plugins |
| Sistema de Memória | Suporta memória local de curto prazo, memória via interface mem0ai, memória inteligente PowerMem, com funcionalidade de resumo de memória |
| Base de Conhecimento | Suporta base de conhecimento RAGFlow, permitindo que o LLM julgue se deve acionar a base de conhecimento após receber a pergunta do usuário, e então responda à pergunta |
| Chamada de Ferramentas | Suporta protocolo IOT do cliente, protocolo MCP do cliente, protocolo MCP do servidor, protocolo de endpoint MCP, funções de ferramentas personalizadas |
| Chamada de Ferramentas | Suporta protocolo IOT do cliente, protocolo MCP do cliente, protocolo MCP do servidor, protocolo de ponto de acesso MCP, funções de ferramentas personalizadas |
| Envio de Comandos | Suporta envio de comandos MCP para dispositivos ESP32 via protocolo MQTT a partir do Console Inteligente |
| Backend de Gerenciamento | Fornece interface de gerenciamento Web, suporta gerenciamento de usuários, configuração do sistema e gerenciamento de dispositivos; Suporta exibição em Chinês Simplificado, Chinês Tradicional, Inglês, Alemão, Vietnamita e Português do Brasil |
| Ferramentas de Teste | Fornece ferramentas de teste de desempenho de resposta de modelo e ferramentas de teste de interação por áudio |
@@ -262,7 +262,7 @@ Xiaozhi é um ecossistema. Ao utilizar este produto, você também pode conferir
| Chamadas via interface Xinference | Xinference | - |
| Chamadas via interface HomeAssistant | HomeAssistant | - |
Na verdade, qualquer LLM que suporte chamadas via interface openai pode ser integrado e utilizado.
Na verdade, qualquer LLM que suporte chamadas via interface OpenAI pode ser integrado e utilizado.
---
+5 -5
View File
@@ -174,7 +174,7 @@ Dự án này cung cấp hai phương pháp triển khai, vui lòng chọn theo
Câu hỏi thường gặp và hướng dẫn liên quan, vui lòng tham khảo [liên kết này](../FAQ.md)
> 💡 Gợi ý: Dưới đây là nền tảng thử nghiệm được triển khai theo mã mới nhất, có thể flash để thử nghiệm nếu cần, đồng thời là 6, dữ liệu sẽ được xóa mỗi ngày,
> 💡 Gợi ý: Dưới đây là nền tảng thử nghiệm được triển khai theo mã mới nhất, có thể flash để thử nghiệm nếu cần, đồng thời là 6, dữ liệu sẽ được xóa mỗi ngày.
```
Địa chỉ bảng điều khiển thông minh: https://2662r3426b.vicp.fun
@@ -255,7 +255,7 @@ Xiaozhi là một hệ sinh thái, khi bạn sử dụng sản phẩm này, bạ
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|:---:|:---:|:---:|
| Gọi giao diện openai | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
| Gọi giao diện OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
| Gọi giao diện ollama | Ollama | - |
| Gọi giao diện dify | Dify | - |
| Gọi giao diện fastgpt | Fastgpt | - |
@@ -263,7 +263,7 @@ Xiaozhi là một hệ sinh thái, khi bạn sử dụng sản phẩm này, bạ
| Gọi giao diện xinference | Xinference | - |
| Gọi giao diện homeassistant | HomeAssistant | - |
Trên thực tế, bất kỳ LLM nào hỗ trợ gọi giao diện openai đều có thể truy cập sử dụng.
Trên thực tế, bất kỳ LLM nào hỗ trợ gọi giao diện OpenAI đều có thể truy cập sử dụng.
---
@@ -271,9 +271,9 @@ Trên thực tế, bất kỳ LLM nào hỗ trợ gọi giao diện openai đề
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|:---:|:---:|:---:|
| Gọi giao diện openai | Alibaba Bailian, Zhipu ChatGLMVLLM | Zhipu ChatGLMVLLM |
| Gọi giao diện OpenAI | Alibaba Bailian, Zhipu ChatGLMVLLM | Zhipu ChatGLMVLLM |
Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đều có thể truy cập sử dụng.
Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện OpenAI đều có thể truy cập sử dụng.
---
+10 -4
View File
@@ -16,7 +16,7 @@
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`voiceprint-api-main`
你需要把它重命名成`voiceprint-api`。
## 第二步, 创建数据库和表
## 第二步, 确认 MySQL 能访问
声纹识别需要依赖`mysql`数据库。如果你之前已经部署`智控台`,说明你已经安装了`mysql`。你可以共用它。
@@ -39,7 +39,7 @@ telnet 127.0.0.1 3306
networks:
- default
expose:
- "3306:3306"
- 3306
```
修改后
@@ -58,8 +58,9 @@ telnet 127.0.0.1 3306
# 进入你docker-compose_all.yml所在的文件夹,例如我的是xiaozhi-server
cd xiaozhi-server
docker compose -f docker-compose_all.yml down
docker compose -f docker-compose.yml up -d
docker compose -f docker-compose_all.yml up -d
```
数据库服务`xiaozhi-esp32-server-db`在`docker-compose_all.yml`里,最小化的`docker-compose.yml`只有服务本身,所以两条命令都用`docker-compose_all.yml`。
启动完后,在宿主机再使用`telnet`命令,看看能不能正常访问`mysql`的`3306`端口。
```
@@ -171,6 +172,8 @@ http://192.168.1.25:8005/voiceprint/health?key=abcd
然后搜索参数`server.voice_print`,此时,它的值应该是`null`值。
点击修改按钮,把上一步得来的`声纹接口地址`粘贴到`参数值`里。然后保存。
声纹识别的相似度阈值是另一个系统参数`server.voiceprint_similarity_threshold`(默认0.4,范围0.0-1.0,数值越高越严格),要调整就在同一个`参数管理`页面里改。
如果能保存成功,说明一切顺利,你可以去智能体查看效果了。如果不成功,说明智控台无法访问声纹识别,很大概率是网络防火墙,或者没有填写正确的局域网ip。
## 第二步 设置智能体记忆模式
@@ -186,7 +189,7 @@ http://192.168.1.25:8005/voiceprint/health?key=abcd
在智控台,`智能体管理`页面,在智能体的面板里,有一个`声纹识别`按钮,点击它。在底部有一个`新增按钮`。就可以对某个人说的话进行声纹注册。
在弹出的框里,`描述`这个属性建议填写上,可以是这个人的职业、性格、爱好。方便智能体对说话人进行分析和了解。
## 第三步 和你的智能体聊天
## 第五步 验证效果
将你的设备通电,问它,你知道我是谁吗?如果他能回答得出,说明声纹识别功能正常。
@@ -205,6 +208,9 @@ voiceprint:
- "test1,张三,张三是一个程序员"
- "test2,李四,李四是一个产品经理"
- "test3,王五,王五是一个设计师"
# 声纹识别相似度阈值,范围0.0-1.0,默认0.4
# 数值越高越严格,减少误识别但可能增加拒识率
similarity_threshold: 0.4
```
把上一步得来的 `声纹接口地址` 粘贴到 `url` 里。然后保存。
@@ -31,8 +31,6 @@ class AudioPlugin(Plugin):
self.source.stop()
def shutdown(self) -> None:
if self.source is not None:
self.source.stop()
if self.app is not None:
self.app.audio_source = None
self.source = None
@@ -51,14 +51,13 @@ class TestRuntimeApplication:
logger.info("test runtime application started")
def stop(self) -> None:
was_running = self._is_running
self._is_running = False
if was_running:
self.event_bridge.publish_service_stopping()
logger.info("test runtime application stopped")
# 启动中途失败时 _is_running 还是 False,但插件可能已经开过采集流,
# 这里不提前返回,保证采集流一定被关掉
if not self._is_running:
return
self.event_bridge.publish_service_stopping()
self.plugins.stop_all()
self._is_running = False
logger.info("test runtime application stopped")
def shutdown(self) -> None:
self.stop()
@@ -95,6 +95,10 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
callerNickname = targetBook.get(bookKey(callerMac));
}
if (StringUtils.isBlank(callerNickname)) {
// 主叫显示名取的是主叫方自己那台设备在设备表里的名字,不是对方的。
// 对方那边给主叫方起的称呼在上面的 targetBook 里已经查过。
// 产品上按「来电显示」的口径处理,所以这里不套通讯录那套「不抄对方私产」的限制
// 这个接口没有登录用户上下文,归属校验那个查法在这里会返回空,只能用不校验归属的查法
DeviceEntity callerDevice = deviceService.getDeviceByMacAddress(callerMac);
if (callerDevice != null) {
callerNickname = callerDevice.getAlias();
@@ -212,11 +216,18 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
DeviceAddressBookEntity entity = new DeviceAddressBookEntity();
entity.setMacAddress(macAddress);
entity.setTargetMac(targetMac);
// 如果 alias 为空,就默认使用设备名称;对方设备尚未登记时退回尾号称呼
// 调用方没起名就退回尾号称呼。对方设备在智控台上起的名字是对方的私产,
// 抄进调用方的通讯录等于把别人的昵称漏出去,所以别人的设备一律不去设备表取名
if (StringUtils.isBlank(alias)) {
DeviceEntity targetDevice = deviceService.getDeviceByMacAddress(targetMac);
if (targetDevice != null) {
alias = targetDevice.getAlias();
// 只有给自己那台设备起名时才用设备表里的名字,那本来就是调用方自己的
if (isOwnDevice(macAddress, targetMac)) {
// 用带归属校验的那个查法。设备表里同一个 MAC 可能有两行只差大小写,
// 一次只准取一条的那种查法撞上这种数据会抛 TooManyResultsException,
// 整个保存接口带崩;按 id 取第一条,撞行也不报错
DeviceEntity ownDevice = deviceService.getOwnedDeviceByMacAddress(macAddress);
if (ownDevice != null && StringUtils.isNotBlank(ownDevice.getAlias())) {
alias = ownDevice.getAlias();
}
}
// 设备登记了但用户没起过名,alias 建表就是 null,这是默认状态
// 不是异常。这时候落个 null 进去,缓存刷新会把整条跳过,
@@ -304,6 +315,16 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
return macAddress == null ? null : macAddress.toLowerCase();
}
/**
* 目标设备是不是调用方自己那台
*
* <p>
* MAC 不区分大小写,比较前先按通讯录的键归一化,跟别处的比较口径保持一致
*/
private static boolean isOwnDevice(String macAddress, String targetMac) {
return bookKey(macAddress) != null && bookKey(macAddress).equals(bookKey(targetMac));
}
private String formatMacAsDeviceName(String mac) {
if (StringUtils.isBlank(mac) || mac.length() < 2) {
return mac;
@@ -1,5 +1,6 @@
package xiaozhi.modules.device.service.impl;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
@@ -21,6 +22,7 @@ import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;
import org.apache.ibatis.exceptions.TooManyResultsException;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;
import org.mockito.MockedStatic;
@@ -75,7 +77,7 @@ class DeviceAddressBookServiceImplTest {
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
when(deviceService.getDeviceByMacAddress(OTHER_MAC)).thenReturn(null);
when(deviceService.getOwnedDeviceByMacAddress(anyString())).thenReturn(null);
service.saveOrUpdate(MAC, OTHER_MAC, null, true);
@@ -85,10 +87,11 @@ class DeviceAddressBookServiceImplTest {
assertNotNull(entity.getAlias());
assertEquals("尾号为66的设备", entity.getAlias());
assertTrue(entity.getHasPermission());
verify(deviceService, never()).getOwnedDeviceByMacAddress(anyString());
}
@Test
void blankAliasOnRegisteredTargetWithoutNicknameFallsBackToMacSuffix() {
void blankAliasOnAnotherDeviceFallsBackToMacSuffixWithoutAskingTheDeviceTable() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
@@ -101,6 +104,7 @@ class DeviceAddressBookServiceImplTest {
registered.setMacAddress(OTHER_MAC);
registered.setAlias(null);
when(deviceService.getDeviceByMacAddress(OTHER_MAC)).thenReturn(registered);
when(deviceService.getOwnedDeviceByMacAddress(OTHER_MAC)).thenReturn(registered);
service.saveOrUpdate(MAC, OTHER_MAC, null, true);
@@ -108,6 +112,171 @@ class DeviceAddressBookServiceImplTest {
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为66的设备", entityCaptor.getValue().getAlias(),
"登记了但没起名的设备落个空名,缓存刷新会跳过这条,对方永远呼不过来");
verify(deviceService, never()).getDeviceByMacAddress(anyString());
verify(deviceService, never()).getOwnedDeviceByMacAddress(anyString());
}
@Test
void blankAliasOnRegisteredTargetDoesNotTakeOverTheirNickname() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity theirs = new DeviceEntity();
theirs.setMacAddress(OTHER_MAC);
theirs.setAlias("老王的音响");
when(deviceService.getDeviceByMacAddress(OTHER_MAC)).thenReturn(theirs);
when(deviceService.getOwnedDeviceByMacAddress(OTHER_MAC)).thenReturn(theirs);
service.saveOrUpdate(MAC, OTHER_MAC, null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为66的设备", entityCaptor.getValue().getAlias(),
"对方在智控台上给这台设备起的名字被抄进了调用方的通讯录");
verify(deviceService, never()).getDeviceByMacAddress(anyString());
verify(deviceService, never()).getOwnedDeviceByMacAddress(anyString());
}
@Test
void blankAliasOnOwnDeviceUsesTheNameFromItsOwnDeviceRow() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity mine = new DeviceEntity();
mine.setMacAddress(MAC);
mine.setAlias("我的书房");
when(deviceService.getOwnedDeviceByMacAddress(MAC)).thenReturn(mine);
service.saveOrUpdate(MAC, MAC, null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("我的书房", entityCaptor.getValue().getAlias(),
"给自己那台设备起名,用的就是自己设备表里的名字");
}
@Test
void ownDeviceWithoutAliasInDeviceTableFallsBackToMacSuffix() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity mine = new DeviceEntity();
mine.setMacAddress(MAC);
mine.setAlias(null);
when(deviceService.getOwnedDeviceByMacAddress(MAC)).thenReturn(mine);
service.saveOrUpdate(MAC, MAC, null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为55的设备", entityCaptor.getValue().getAlias(),
"自己的设备在设备表里也没起名,落个空名的话缓存刷新会跳过这条");
}
@Test
void duplicateDeviceRowsDifferingOnlyInCaseDoNotBreakNaming() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
// 设备表里同一个 MAC 有两行只差大小写,一次只准取一条的那种查法会直接抛异常
when(deviceService.getDeviceByMacAddress(anyString()))
.thenThrow(new TooManyResultsException("selectOne 撞上两行只差大小写的数据"));
DeviceEntity first = new DeviceEntity();
first.setMacAddress(MAC);
first.setAlias("我的书房");
when(deviceService.getOwnedDeviceByMacAddress(MAC)).thenReturn(first);
assertDoesNotThrow(() -> service.saveOrUpdate(MAC, MAC, null, true),
"同 MAC 有两行只差大小写时,取名不能把整个接口带崩");
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("我的书房", entityCaptor.getValue().getAlias(),
"撞行时按 id 取第一条那条的名字");
}
@Test
void ownDeviceIsRecognizedWhateverCaseTheAddressIsWrittenIn() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity mine = new DeviceEntity();
mine.setMacAddress(HEX_MAC);
mine.setAlias("我的书房");
when(deviceService.getOwnedDeviceByMacAddress(HEX_MAC)).thenReturn(mine);
service.saveOrUpdate(HEX_MAC, HEX_MAC.toUpperCase(), null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("我的书房", entityCaptor.getValue().getAlias(),
"同一个地址写成大小写两种样子,就被当成别人的设备了");
}
@Test
void explicitAliasIsKeptAsIs() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
service.saveOrUpdate(MAC, OTHER_MAC, "客厅", true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("客厅", entityCaptor.getValue().getAlias(),
"自己给别人的设备起名是通讯录的本意,不能被兜底逻辑改掉");
verify(deviceService, never()).getDeviceByMacAddress(anyString());
verify(deviceService, never()).getOwnedDeviceByMacAddress(anyString());
}
@Test
void blankAliasOnUnregisteredTargetDoesNotBlowUp() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
when(deviceService.getOwnedDeviceByMacAddress(anyString())).thenReturn(null);
assertDoesNotThrow(() -> service.saveOrUpdate(MAC, OTHER_MAC, null, true),
"目标设备没登记时接口得正常返回,不能抛异常");
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为66的设备", entityCaptor.getValue().getAlias());
}
@Test
@@ -325,7 +494,7 @@ class DeviceAddressBookServiceImplTest {
DeviceService deviceService = mock(DeviceService.class);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
when(deviceService.getDeviceByMacAddress(anyString())).thenReturn(null);
when(deviceService.getOwnedDeviceByMacAddress(anyString())).thenReturn(null);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, mock(SysParamsService.class));
+5
View File
@@ -120,6 +120,11 @@ export default defineManifestConfig({
/* 小程序特有相关 */
'mp-weixin': {
appid: VITE_WX_APPID,
// 仅供开发者工具选择调试基础库版本;线上客户端的最低基础库门槛需在
// 微信公众平台「开发管理 → 基础库最低版本设置」中配置(代码里改不动)。
// 功能硬性要求:wx.getRandomValues 需基础库 >= 2.17.3,低于该版本的
// 线上客户端登录时将收到 fail-closed 报错
libVersion: '2.19.4',
setting: {
urlCheck: false,
// 是否启用 ES6 转 ES5
+1 -1
View File
@@ -101,7 +101,7 @@
"js-cookie": "^3.0.5",
"pinia": "2.0.36",
"pinia-plugin-persistedstate": "3.2.1",
"sm-crypto": "^0.3.13",
"sm-crypto": "0.5.8",
"vue": "^3.4.21",
"wot-design-uni": "^1.9.1",
"z-paging": "2.8.7"
@@ -0,0 +1,29 @@
diff --git a/src/sm2/rng.js b/src/sm2/rng.js
index c6572c1756579fb100c0d7bdf0cec323971ec211..cdb1f1fcc7d58c52138f6f9fd76240bae5dde5d4 100644
--- a/src/sm2/rng.js
+++ b/src/sm2/rng.js
@@ -54,8 +54,14 @@ function rngSeedTime() {
rngSeedInt(new Date().getTime())
}
-// Initialize the pool with junk if needed.
-if (rngPool == null) {
+// [xiaozhi patch] 熵池初始化从模块加载时推迟到首次取随机字节时(见 rngGetByte)。
+// 原因:微信小程序等环境没有同步的 crypto.getRandomValues,模块加载即取熵会直接抛异常;
+// 推迟后,调用方可在首次使用前异步准备好真随机源(见 manager-mobile/src/utils/secure-random.ts)。
+// fail-closed 语义保持不变:首次取熵时仍要求存在可用的 crypto.getRandomValues,否则抛出异常。
+function ensureRngPool() {
+ if (rngPool != null)
+ return
+
rngPool = []
rngPptr = 0
let t
@@ -82,6 +88,7 @@ if (rngPool == null) {
function rngGetByte() {
if (rngState == null) {
+ ensureRngPool()
rngSeedTime()
rngState = new Arcfour()
rngState.init(rngPool)
+8 -5
View File
@@ -11,6 +11,9 @@ patchedDependencies:
'@dcloudio/uni-h5':
hash: 5763725268e9a493075be3c82d91b590b9ae30e997a04a418494dd7d6d327b9d
path: patches/@dcloudio__uni-h5.patch
sm-crypto@0.5.8:
hash: 62930665cf9612107916df2f12c9deab4ad907f6ac9080520b84bffe016c3352
path: patches/sm-crypto@0.5.8.patch
importers:
@@ -92,8 +95,8 @@ importers:
specifier: 3.2.1
version: 3.2.1(pinia@2.0.36(typescript@5.7.2)(vue@3.5.15(typescript@5.7.2)))
sm-crypto:
specifier: ^0.3.13
version: 0.3.14
specifier: 0.5.8
version: 0.5.8(patch_hash=62930665cf9612107916df2f12c9deab4ad907f6ac9080520b84bffe016c3352)
vue:
specifier: ^3.4.21
version: 3.5.15(typescript@5.7.2)
@@ -6026,8 +6029,8 @@ packages:
resolution: {integrity: sha512-bSiSngZ/jWeX93BqeIAbImyTbEihizcwNjFoRUIY/T1wWQsfsm2Vw1agPKylXvQTU7iASGdHhyqRlqQzfz+Htg==}
engines: {node: '>=18'}
sm-crypto@0.3.14:
resolution: {integrity: sha512-sR7NuGAJH93Om8keAF2/rj1EqFzEiUey4aKGyDo3nQ2BiVqhgq8UkiIoIp5qhV9w4pG7qArhfqyewcG8AbhXbg==}
sm-crypto@0.5.8:
resolution: {integrity: sha512-bpz1/PthgZHcLHyIl28mbYZHmeuTDrdWpwHZ7BGzFjQuB7OOhjlbcPEMt44aqUfZ2S5YJXM8v20xYO1j9CC45Q==}
source-map-js@1.2.1:
resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==}
@@ -14156,7 +14159,7 @@ snapshots:
ansi-styles: 6.2.1
is-fullwidth-code-point: 5.0.0
sm-crypto@0.3.14:
sm-crypto@0.5.8(patch_hash=62930665cf9612107916df2f12c9deab4ad907f6ac9080520b84bffe016c3352):
dependencies:
jsbn: 1.1.0
+1 -1
View File
@@ -1,6 +1,6 @@
packages:
- '**'
- '!node_modules'
patchedDependencies:
'@dcloudio/uni-h5': patches/@dcloudio__uni-h5.patch
sm-crypto@0.5.8: patches/sm-crypto@0.5.8.patch
@@ -0,0 +1,126 @@
<script lang="ts">
import { defineComponent } from 'vue'
// #ifdef APP-PLUS
import { ensureSecureRandomReady, registerRandomBridge } from '@/utils/secure-random'
// #endif
interface BridgeItem {
length: number
resolve: (bytes: number[]) => void
reject: (err: Error) => void
}
/**
* App 端专用(页面中 #ifdef APP-PLUS 条件挂载):
* 逻辑层无 Web Crypto,经 renderjs 在视图层 WebView 取
* crypto.getRandomValues 后 callMethod 回传,注册为 secure-random 的随机源。
*/
export default defineComponent({
data() {
return {
/** 变化后触发 renderjs 取随机(每次新对象引用确保 prop 变更被感知) */
bridgeRequest: null as { id: number, length: number } | null,
// 仅类型检查用:renderjs 模块绑定由 uni-app 编译期处理,运行时不读取
secureRandomRenderer: { onPropChange: () => {} } as { onPropChange: (...args: any[]) => void },
seq: 0,
queue: [] as BridgeItem[],
sending: false,
currentId: 0,
timeoutTimer: undefined as ReturnType<typeof setTimeout> | undefined,
}
},
mounted() {
// #ifdef APP-PLUS
registerRandomBridge((length: number) =>
this.requestBytes(length).then(bytes => new Uint8Array(bytes)),
)
ensureSecureRandomReady().catch((e: Error) => {
console.warn('[secure-random] App 端随机源预热失败', e)
})
// #endif
},
beforeUnmount() {
clearTimeout(this.timeoutTimer)
},
methods: {
requestBytes(length: number): Promise<number[]> {
return new Promise((resolve, reject) => {
this.queue.push({ length, resolve, reject })
this.pump()
})
},
/** 串行发送:prop 变更过快时视图层可能丢失中间状态,故一次只发一个请求 */
pump() {
if (this.sending || this.queue.length === 0)
return
this.sending = true
const item = this.queue[0]
this.currentId = ++this.seq
this.bridgeRequest = { id: this.currentId, length: item.length }
// 超时保护:视图层无响应时按 fail-closed 报错,避免登录流程永久挂起
clearTimeout(this.timeoutTimer)
this.timeoutTimer = setTimeout(() => {
this.onRandomResult({ id: this.currentId, error: 'renderjs 随机数请求超时' })
}, 5000)
},
/** renderjs 通过 callMethod 回传结果(或错误) */
onRandomResult(res: { id: number, bytes?: number[], error?: string }) {
if (res.id !== this.currentId)
return
clearTimeout(this.timeoutTimer)
const item = this.queue.shift()
this.sending = false
if (item) {
if (res.error)
item.reject(new Error(res.error))
else
item.resolve(res.bytes || [])
}
this.pump()
},
},
})
</script>
<script module="secureRandomRenderer" lang="renderjs">
// 视图层(WebView):可访问 crypto.getRandomValues
export default {
methods: {
onPropChange(newVal, oldVal, ownerInstance, instance) {
if (!newVal || !newVal.id || !newVal.length)
return
try {
const cryptoObj = (typeof crypto !== 'undefined' && crypto)
|| (typeof window !== 'undefined' && window.crypto)
if (!cryptoObj || typeof cryptoObj.getRandomValues !== 'function')
throw new Error('视图层缺少 crypto.getRandomValues')
const bytes = new Uint8Array(newVal.length)
cryptoObj.getRandomValues(bytes)
ownerInstance.callMethod('onRandomResult', { id: newVal.id, bytes: Array.from(bytes) })
}
catch (e) {
ownerInstance.callMethod('onRandomResult', {
id: newVal.id,
error: String((e && e.message) || e),
})
}
},
},
}
</script>
<template>
<view
class="secure-random-bridge"
:prop="bridgeRequest"
:change:prop="secureRandomRenderer.onPropChange"
/>
</template>
<style>
.secure-random-bridge {
width: 0;
height: 0;
overflow: hidden;
}
</style>
+1
View File
@@ -92,6 +92,7 @@
"minified": true
},
"usingComponents": true,
"libVersion": "2.19.4",
"optimization": {
"subPackages": true
},
@@ -13,6 +13,10 @@ import { onLoad } from '@dcloudio/uni-app'
import { computed, onMounted, ref } from 'vue'
// 导入API接口
import { retrievePassword, sendSmsCode } from '@/api/auth'
// #ifdef APP-PLUS
// App 端随机源桥接:为 SM2 加密提供安全随机数(无视觉展示)
import SecureRandomBridge from '@/components/secure-random-bridge/index.vue'
// #endif
// 导入国际化相关功能
import { initI18n, t } from '@/i18n'
import { useConfigStore } from '@/store'
@@ -235,7 +239,7 @@ async function handleResetPassword() {
try {
// 拼接图形验证码和新密码进行加密
const captchaAndPassword = formData.value.captcha + formData.value.newPassword
encryptedPassword = sm2Encrypt(sm2PublicKey.value, captchaAndPassword)
encryptedPassword = await sm2Encrypt(sm2PublicKey.value, captchaAndPassword)
}
catch (error) {
console.error('密码加密失败:', error)
@@ -305,6 +309,9 @@ onMounted(async () => {
class="app-container box-border h-screen w-full"
:style="{ paddingTop: `${safeAreaInsets?.top}px` }"
>
<!-- #ifdef APP-PLUS -->
<SecureRandomBridge />
<!-- #endif -->
<view class="header">
<view class="back-button" @click="goBack">
<wd-icon name="arrow-left" custom-class="back-icon" />
@@ -13,6 +13,10 @@ import type { LoginData } from '@/api/auth'
import type { Language } from '@/store/lang'
import { computed, onMounted, ref } from 'vue'
import { login } from '@/api/auth'
// #ifdef APP-PLUS
// App 端随机源桥接:为 SM2 加密提供安全随机数(无视觉展示)
import SecureRandomBridge from '@/components/secure-random-bridge/index.vue'
// #endif
// 导入国际化相关功能
import { changeLanguage, getCurrentLanguage, getSupportedLanguages, initI18n, t } from '@/i18n'
import { useConfigStore, useUserStore } from '@/store'
@@ -202,7 +206,7 @@ async function handleLogin() {
try {
// 拼接验证码和密码
const captchaAndPassword = formData.value.captcha + formData.value.password
encryptedPassword = sm2Encrypt(sm2PublicKey, captchaAndPassword)
encryptedPassword = await sm2Encrypt(sm2PublicKey, captchaAndPassword)
}
catch (error) {
console.error('密码加密失败:', error)
@@ -281,6 +285,9 @@ onMounted(async () => {
<template>
<view class="app-container box-border h-screen w-full" :style="{ paddingTop: `${safeAreaInsets?.top}px` }">
<!-- #ifdef APP-PLUS -->
<SecureRandomBridge />
<!-- #endif -->
<view class="header">
<view class="logo-section">
<wd-img :width="80" :height="80" round src="/static/logo.png" class="logo" />
@@ -11,6 +11,10 @@
<script lang="ts" setup>
import { computed, onMounted, ref } from 'vue'
import { register, sendSmsCode } from '@/api/auth'
// #ifdef APP-PLUS
// App 端随机源桥接:为 SM2 加密提供安全随机数(无视觉展示)
import SecureRandomBridge from '@/components/secure-random-bridge/index.vue'
// #endif
// 导入国际化相关功能
import { initI18n, t } from '@/i18n'
import { useConfigStore } from '@/store'
@@ -251,7 +255,7 @@ async function handleRegister() {
try {
// 拼接验证码和密码
const captchaAndPassword = formData.value.captcha + formData.value.password
encryptedPassword = sm2Encrypt(sm2PublicKey.value, captchaAndPassword)
encryptedPassword = await sm2Encrypt(sm2PublicKey.value, captchaAndPassword)
}
catch (error) {
console.error('密码加密失败:', error)
@@ -325,6 +329,9 @@ onMounted(async () => {
<template>
<view class="app-container box-border h-screen w-full" :style="{ paddingTop: `${safeAreaInsets?.top}px` }">
<!-- #ifdef APP-PLUS -->
<SecureRandomBridge />
<!-- #endif -->
<view class="header">
<view class="back-button" @click="goBack">
<wd-icon name="arrow-left" custom-class="back-icon" />
+24 -9
View File
@@ -2,6 +2,21 @@ import smCrypto from 'sm-crypto'
import { pages, subPackages } from '@/pages.json'
import { isMpWeixin } from './platform'
import { ensureSecureRandomReady } from './secure-random'
/**
* 获取 sm2 模块(确保密码学安全随机源已就绪)。
*
* sm-crypto >= 0.5.0(CVE-2026-73567 修复)强制要求 crypto.getRandomValues。
* 小程序等环境没有该 API,项目中通过 pnpm patch(patches/sm-crypto@0.5.8.patch)
* 将 sm-crypto 的熵池初始化推迟到首次取随机字节时,并由 secure-random.ts 提供
* 垫片。因此小程序端首次调用 sm2 前必须 await 本方法,用 wx.getRandomValues
* 预热真随机缓冲池;浏览器 / Node 等原生支持 globalThis.crypto 的环境立即就绪。
*/
async function getSm2(): Promise<typeof smCrypto.sm2> {
await ensureSecureRandomReady()
return smCrypto.sm2
}
/**
* 运行时服务端地址覆盖存储键
@@ -201,11 +216,11 @@ export function getEnvBaseUploadUrl() {
/**
* 生成SM2密钥对(十六进制格式)
* @returns {object} 包含公钥和私钥的对象
* @returns {Promise<object>} 包含公钥和私钥的对象
*/
export function generateSm2KeyPairHex() {
export async function generateSm2KeyPairHex() {
// 使用sm-crypto库生成SM2密钥对
const sm2 = smCrypto.sm2
const sm2 = await getSm2()
const keypair = sm2.generateKeyPairHex()
return {
@@ -220,9 +235,9 @@ export function generateSm2KeyPairHex() {
* SM2公钥加密
* @param {string} publicKey 公钥(十六进制格式)
* @param {string} plainText 明文
* @returns {string} 加密后的密文(十六进制格式)
* @returns {Promise<string>} 加密后的密文(十六进制格式)
*/
export function sm2Encrypt(publicKey: string, plainText: string): string {
export async function sm2Encrypt(publicKey: string, plainText: string): Promise<string> {
if (!publicKey) {
throw new Error('公钥不能为null或undefined')
}
@@ -231,7 +246,7 @@ export function sm2Encrypt(publicKey: string, plainText: string): string {
throw new Error('明文不能为空')
}
const sm2 = smCrypto.sm2
const sm2 = await getSm2()
// SM2加密,添加04前缀表示未压缩公钥
const encrypted = sm2.doEncrypt(plainText, publicKey, 1)
// 转换为十六进制格式(与后端保持一致,添加04前缀)
@@ -244,10 +259,10 @@ export function sm2Encrypt(publicKey: string, plainText: string): string {
* SM2私钥解密
* @param {string} privateKey 私钥(十六进制格式)
* @param {string} cipherText 密文(十六进制格式)
* @returns {string} 解密后的明文
* @returns {Promise<string>} 解密后的明文(当前项目内暂无调用点,保留备用)
*/
export function sm2Decrypt(privateKey: string, cipherText: string): string {
const sm2 = smCrypto.sm2
export async function sm2Decrypt(privateKey: string, cipherText: string): Promise<string> {
const sm2 = await getSm2()
// 移除04前缀(与后端保持一致)
const dataWithoutPrefix = cipherText.startsWith('04') ? cipherText.substring(2) : cipherText
// SM2解密
@@ -0,0 +1,194 @@
/**
* 为 sm-crypto >= 0.5.0 提供 crypto.getRandomValues 兼容层
* (0.5.0 起强制要求 CSPRNG,不支持时模块加载即抛异常,见 CVE-2026-73567)。
*
* 策略:
* 1. 经平台异步 CSPRNG API 分块并行取真随机字节,填入同步缓冲池。
* 探测顺序 wx → uni → my → tt → swan → ks → qq → jd → lark → xhs;
* App 端逻辑层无 CSPRNG,
* 由 secure-random-bridge 组件经 renderjs 回传(registerRandomBridge 兜底)。
* 2. 垫片同步从缓冲池取字节,池不足即抛错,不回退弱随机(fail-closed)。
* 3. sm-crypto 熵池初始化已通过 pnpm patch 推迟到首次取随机
* (patches/sm-crypto@0.5.8.patch),首次 sm2 调用前 await
* ensureSecureRandomReady() 即可。
*
* 无任何 CSPRNG 的平台(如支付宝/百度小程序)会 reject,属预期行为。
*/
const CHUNK_SIZE = 64
const CHUNK_COUNT = 8
const LOW_WATER_MARK = 128
// 标记垫片自身,探测时与原生实现区分,避免失败重试被短路成假成功
const SHIM_MARKER = Symbol('secureRandomShim')
// App 端桥接通道,由 secure-random-bridge 组件挂载时注册
let bridgeProvider: ((length: number) => Promise<Uint8Array>) | null = null
export function registerRandomBridge(provider: (length: number) => Promise<Uint8Array>) {
bridgeProvider = provider
}
let pool: number[] = []
let readyPromise: Promise<void> | null = null
let refillInFlight = false
function getGlobalObj(): Record<string, any> | null {
// sm-crypto 读取的是 globalThis.crypto,垫片必须装在同一对象上
if (typeof globalThis !== 'undefined')
return globalThis as unknown as Record<string, any>
return null
}
function hasNativeGetRandomValues(): boolean {
try {
// 探测 globalThis.crypto:Node、h5:ssr 等环境有原生 CSPRNG 但无 window
const fn = getGlobalObj()?.crypto?.getRandomValues
// 排除垫片自身,否则失败重试会被短路成假成功
return typeof fn === 'function' && !(fn as any)[SHIM_MARKER]
}
catch {
return false
}
}
/**
* 按优先级探测提供异步 CSPRNG 的平台全局对象。
* uni.getRandomValues 仅有声明无实现,故排在 wx 之后兜底;
* 其余平台对象未在 TS 类型中声明,经 globalThis 探测,
* 对应平台未来实现同名 API 时自动生效。
*/
function findRandomApi(): { name: string, api: any } | null {
const globalObj = getGlobalObj()
const candidates: Array<[string, any]> = [
['wx', typeof wx !== 'undefined' ? wx : null],
['uni', typeof uni !== 'undefined' ? uni : null],
['my', globalObj ? globalObj.my : null],
['tt', globalObj ? globalObj.tt : null],
['swan', globalObj ? globalObj.swan : null],
['ks', globalObj ? globalObj.ks : null],
['qq', globalObj ? globalObj.qq : null],
['jd', globalObj ? globalObj.jd : null],
['lark', globalObj ? globalObj.lark : null],
['xhs', globalObj ? globalObj.xhs : null],
]
for (const [name, api] of candidates) {
if (api && typeof api.getRandomValues === 'function')
return { name, api }
}
return null
}
function fetchRandomChunk(length: number): Promise<Uint8Array> {
return new Promise((resolve, reject) => {
const found = findRandomApi()
if (!found && !bridgeProvider) {
reject(new Error(
'当前平台不支持获取密码学安全随机数(getRandomValues),无法安全执行 SM2 加密。'
+ '当前已适配:微信小程序(wx.getRandomValues)、浏览器、Node 环境、'
+ 'App 端(renderjs 桥接,需页面挂载 secure-random-bridge 组件);'
+ '其余小程序平台因缺少 CSPRNG API 被按 fail-closed 原则拒绝,'
+ '如需支持请扩展 secure-random.ts 的 findRandomApi 适配链。',
))
return
}
if (found) {
found.api.getRandomValues({
length,
success: (res: { randomValues: ArrayBuffer | Uint8Array | number[] }) => {
resolve(new Uint8Array(res.randomValues))
},
fail: (err: unknown) => {
reject(err instanceof Error ? err : new Error(`getRandomValues 调用失败: ${JSON.stringify(err)}`))
},
})
return
}
bridgeProvider!(length).then(resolve, (err: unknown) => {
reject(err instanceof Error ? err : new Error(`renderjs 桥接取随机失败: ${String(err)}`))
})
})
}
async function refillPool(totalLength: number): Promise<void> {
const chunks = await Promise.all(
Array.from({ length: Math.ceil(totalLength / CHUNK_SIZE) }, () => fetchRandomChunk(CHUNK_SIZE)),
)
for (const chunk of chunks) {
for (let i = 0; i < chunk.length; i++)
pool.push(chunk[i])
}
}
function scheduleRefill() {
if (refillInFlight)
return
refillInFlight = true
refillPool(256)
.catch((err) => {
// 补充失败仅告警:池仍够用则无感,耗尽时由 getRandomValues 抛错兜底
console.warn('[secure-random] 补充随机数缓冲池失败', err)
})
.finally(() => {
refillInFlight = false
})
}
function createGetRandomValues<T extends ArrayBufferView>(array: T): T {
// 按字节拷贝,兼容 Uint16Array/Uint32Array 等视图
const view = array as unknown as ArrayBufferView
const bytes = new Uint8Array(view.buffer, view.byteOffset, view.byteLength)
if (pool.length < bytes.length) {
scheduleRefill()
throw new Error('安全随机数缓冲池未就绪或已耗尽,请先等待 ensureSecureRandomReady() 完成')
}
for (let i = 0; i < bytes.length; i++)
bytes[i] = pool[i]
pool = pool.slice(bytes.length)
if (pool.length < LOW_WATER_MARK)
scheduleRefill()
return array
}
;(createGetRandomValues as any)[SHIM_MARKER] = true
function installShim() {
const globalObj = getGlobalObj()
if (!globalObj)
throw new Error('无法获取全局对象,无法安装 crypto.getRandomValues 垫片')
if (globalObj.crypto && typeof globalObj.crypto.getRandomValues === 'function')
return
if (globalObj.crypto) {
// 只补缺的方法,保留宿主 crypto 其他成员;defineProperty 规避只读访问器赋值失败
Object.defineProperty(globalObj.crypto, 'getRandomValues', {
value: createGetRandomValues,
configurable: true,
writable: true,
})
}
else {
globalObj.crypto = { getRandomValues: createGetRandomValues }
}
}
/**
* 确保密码学安全随机源可用(首次 sm2 调用前必须 await)。
* 原生支持的环境直接就绪;缺失环境安装垫片并预热缓冲池;
* 无 CSPRNG 的环境 reject,由调用方 fail-closed 处理。
*/
export function ensureSecureRandomReady(): Promise<void> {
if (hasNativeGetRandomValues())
return Promise.resolve()
if (!readyPromise) {
readyPromise = (async () => {
installShim()
await refillPool(CHUNK_SIZE * CHUNK_COUNT)
})().catch((err) => {
// 失败后重置,允许后续重试
readyPromise = null
throw err
})
}
return readyPromise
}
+4 -4
View File
@@ -17,7 +17,7 @@
"normalize.css": "^8.0.1",
"opus-decoder": "^0.7.7",
"opus-recorder": "^8.0.5",
"sm-crypto": "^0.3.13",
"sm-crypto": "^0.5.8",
"v-code-diff": "^1.16.0",
"vue": "^2.6.14",
"vue-axios": "^3.5.2",
@@ -12485,9 +12485,9 @@
}
},
"node_modules/sm-crypto": {
"version": "0.3.13",
"resolved": "https://registry.npmmirror.com/sm-crypto/-/sm-crypto-0.3.13.tgz",
"integrity": "sha512-ztNF+pZq6viCPMA1A6KKu3bgpkmYti5avykRHbcFIdSipFdkVmfUw2CnpM2kBJyppIalqvczLNM3wR8OQ0pT5w==",
"version": "0.5.8",
"resolved": "https://registry.npmmirror.com/sm-crypto/-/sm-crypto-0.5.8.tgz",
"integrity": "sha512-bpz1/PthgZHcLHyIl28mbYZHmeuTDrdWpwHZ7BGzFjQuB7OOhjlbcPEMt44aqUfZ2S5YJXM8v20xYO1j9CC45Q==",
"license": "MIT",
"dependencies": {
"jsbn": "^1.1.0"
+1 -1
View File
@@ -22,7 +22,7 @@
"normalize.css": "^8.0.1",
"opus-decoder": "^0.7.7",
"opus-recorder": "^8.0.5",
"sm-crypto": "^0.3.13",
"sm-crypto": "^0.5.8",
"v-code-diff": "^1.16.0",
"vue": "^2.6.14",
"vue-axios": "^3.5.2",
@@ -77,7 +77,11 @@
<el-form :model="formData.configJson" label-width="auto" label-position="left" class="custom-form">
<div v-for="(row, rowIndex) in chunkedCallInfoFields" :key="rowIndex" class="form-row">
<el-form-item v-for="field in row" :key="field.prop" :label="field.label" :prop="field.prop" style="flex: 1;">
<el-input v-if="field.type === 'json-textarea'" v-model="formData.configJson[field.prop]" type="textarea"
<el-select v-if="field.type === 'select'" v-model="formData.configJson[field.prop]"
:placeholder="field.placeholder" class="custom-input-bg" style="width: 100%">
<el-option v-for="opt in field.options" :key="opt" :label="opt" :value="opt"></el-option>
</el-select>
<el-input v-else-if="field.type === 'json-textarea'" v-model="formData.configJson[field.prop]" type="textarea"
:rows="3" :placeholder="field.placeholder" class="custom-input-bg">
</el-input>
<el-input v-else v-model="formData.configJson[field.prop]" :placeholder="field.placeholder"
@@ -164,7 +168,8 @@ export default {
fields: JSON.parse(item.fields || '[]').map(f => ({
label: f.label,
prop: f.key,
type: f.type === 'dict' ? 'json-textarea' : f.type === 'password' ? 'password' : 'text',
type: f.type === 'dict' ? 'json-textarea' : f.type === 'password' ? 'password' : (f.type === 'select' ? 'select' : 'text'),
options: f.options || [],
placeholder: `请输入${f.key}`
}))
}))
@@ -81,6 +81,11 @@
"></el-input>
</template>
<el-select v-else-if="field.type === 'select'" v-model="form.configJson[field.prop]"
:placeholder="field.placeholder" style="width: 100%">
<el-option v-for="opt in field.options" :key="opt" :label="opt" :value="opt"></el-option>
</el-select>
<el-input v-else v-model="form.configJson[field.prop]" :placeholder="field.placeholder" :type="field.type"
:show-password="field.type === 'password'" @focus="
isSensitiveField(field.prop)
@@ -312,7 +317,10 @@ export default {
? "json-textarea"
: f.type === "password"
? "password"
: "text",
: f.type === "select"
? "select"
: "text",
options: f.options || [],
placeholder: `请输入${f.key}`,
}));
+4
View File
@@ -10,6 +10,7 @@ from core.http_server import SimpleHttpServer
from core.websocket_server import WebSocketServer
from core.utils.util import check_ffmpeg_installed
from core.utils.gc_manager import get_gc_manager
from core.handle.helloHandle import preheat_wakeup_audio_cache
TAG = __name__
logger = setup_logging()
@@ -64,6 +65,9 @@ async def main():
# 添加 stdin 监控任务
stdin_task = asyncio.create_task(monitor_stdin())
# 预热唤醒词音频编码缓存,让首条连接就命中
asyncio.create_task(preheat_wakeup_audio_cache())
# 启动全局GC管理器(5分钟清理一次)
gc_manager = get_gc_manager(interval_seconds=300)
await gc_manager.start()
+1 -1
View File
@@ -781,7 +781,7 @@ class ConnectionHandler:
self.voiceprint_provider = voiceprint_provider
self.logger.bind(tag=TAG).info("声纹识别功能已在连接时动态启用")
else:
self.logger.bind(tag=TAG).warning("声纹识别功能启用但配置不完整")
self.logger.bind(tag=TAG).debug("声纹识别功能启用但配置不完整")
else:
self.logger.bind(tag=TAG).info("声纹识别功能未启用")
except Exception as e:
+49 -6
View File
@@ -4,6 +4,7 @@ import uuid
import random
import asyncio
from typing import TYPE_CHECKING
from config.logger import setup_logging
if TYPE_CHECKING:
from core.connection import ConnectionHandler
@@ -13,9 +14,14 @@ from core.providers.tts.dto.dto import SentenceType
from core.utils.wakeup_word import WakeupWordsConfig
from core.handle.sendAudioHandle import sendAudioMessage, send_tts_message
from core.utils.util import remove_punctuation_and_length, opus_datas_to_wav_bytes
from core.providers.tools.device_mcp import MCPClient, send_mcp_initialize_message
from core.providers.tools.device_mcp import (
MCPClient,
send_mcp_initialize_message,
send_mcp_tools_list_request,
)
TAG = __name__
_wakeup_logger = setup_logging()
WAKEUP_CONFIG = {
"refresh_time": 10,
@@ -63,6 +69,8 @@ async def handleHelloMessage(conn: "ConnectionHandler", msg_json):
# The device waits for the server hello before processing MCP messages.
if features and features.get("mcp"):
asyncio.create_task(send_mcp_initialize_message(conn))
# 并发预发 tools/list,让帧排在 audio 流之前到达 client
asyncio.create_task(send_mcp_tools_list_request(conn))
async def checkWakeupWords(conn: "ConnectionHandler", text):
@@ -105,19 +113,19 @@ async def checkWakeupWords(conn: "ConnectionHandler", text):
}
# 获取音频数据
opus_packets = await audio_to_data(response.get("file_path"), use_cache=False)
opus_packets = await audio_to_data(response.get("file_path"), use_cache=True)
# 播放唤醒词回复
conn.client_abort = False
# 将唤醒词回复视为新会话,生成新的 sentence_id,确保流控器重置
conn.sentence_id = str(uuid.uuid4().hex)
conn.logger.bind(tag=TAG).info(f"播放唤醒词回复: {response.get('text')}")
await sendAudioMessage(conn, SentenceType.FIRST, opus_packets, response.get("text"))
await sendAudioMessage(conn, SentenceType.LAST, [], None)
# 补充对话
conn.dialogue.put(Message(role="assistant", content=response.get("text")))
conn.logger.bind(tag=TAG).info(f"播放唤醒词回复: {response.get('text')}")
await sendAudioMessage(conn, SentenceType.FIRST, opus_packets, response.get("text"))
asyncio.create_task(_finish_wakeup_reply(conn))
# 检查是否需要更新唤醒词回复
if time.time() - response.get("time", 0) > WAKEUP_CONFIG["refresh_time"]:
@@ -126,6 +134,16 @@ async def checkWakeupWords(conn: "ConnectionHandler", text):
return True
async def _finish_wakeup_reply(conn: "ConnectionHandler"):
"""等音频队列清空后再发 stop(时序:全部音频帧→stop),期间不阻塞接收循环"""
try:
await sendAudioMessage(conn, SentenceType.LAST, [], None)
except asyncio.CancelledError:
raise
except Exception as e:
conn.logger.bind(tag=TAG).error(f"唤醒词回复收尾失败: {e}")
async def wakeupWordsResponse(conn: "ConnectionHandler"):
if not conn.tts:
return
@@ -160,7 +178,32 @@ async def wakeupWordsResponse(conn: "ConnectionHandler"):
f.write(wav_bytes)
# 更新配置
wakeup_words_config.update_wakeup_response(voice, file_path, result)
# 覆写文件后 mtime 变化使旧缓存失效,此处立刻预编码新文件,
# 下次唤醒直接命中缓存,编码成本不占用唤醒响应路径
await audio_to_data(file_path, use_cache=True)
finally:
# 确保在任何情况下都释放锁
if _wakeup_response_lock.locked():
_wakeup_response_lock.release()
async def preheat_wakeup_audio_cache():
"""进程启动时预热唤醒词音频编码缓存
把 wakeup_words/{voice}.wav 在 Opus 编码结果一次性算好塞进全局 cache,
后续每条连接上来就能直接命中,省掉 wav→PCM→Opus 的数百毫秒开销。
"""
# 进程启动时 tts 还未初始化,无法拿到 conn.tts.voice,按常见 fallback 列表预热
preheat_voices = ["default", "female-shaonv", "male-qn-jingying"]
fallback_path = "config/assets/wakeup_words_short.wav"
for voice in preheat_voices:
file_path = fallback_path
try:
response = wakeup_words_config.get_wakeup_response(voice)
if response and response.get("file_path"):
file_path = response["file_path"]
await audio_to_data(file_path, use_cache=True)
except Exception as e:
_wakeup_logger.bind(tag=TAG).warning(
f"预热 voice={voice} 失败: {e}"
)
@@ -12,10 +12,8 @@ from core.providers.tts.dto.dto import SentenceType
from core.utils.audioRateController import AudioRateController
TAG = __name__
# 音频帧时长(毫秒)
AUDIO_FRAME_DURATION = 60
# 预缓冲包数量,直接发送以减少延迟
PRE_BUFFER_COUNT = 5
AUDIO_FRAME_DURATION = 60 # 音频帧时长(毫秒),与 Opus 编码 60ms/帧对齐
PRE_BUFFER_COUNT = 2 # 首段发送时不入队的帧数,避免流控延迟拉高首帧到达时间
async def sendAudioMessage(conn: "ConnectionHandler", sentenceType, audios, text, sentence_id=None):
@@ -16,12 +16,20 @@ THINKING_DISABLED_DOMAINS = {
"bigmodel.cn": {"thinking": {"type": "disabled"}},
"moonshot.cn": {"thinking": {"type": "disabled"}},
"volces.com": {"thinking": {"type": "disabled"}},
"modelarts-maas.com": {"thinking": {"type": "disabled"}},
}
# 域名规则的模型名豁免表: 强制思考模型不接受该参数注入(如 MaaS glm-5.3 传 thinking 必 400)
THINKING_DISABLED_MODEL_EXCLUSIONS = {
"modelarts-maas.com": ("glm-5.3",),
}
class LLMProvider(LLMProviderBase):
def __init__(self, config):
self.model_name = config.get("model_name")
# 可选实例级思考强度: off/disabled/none/disable | low/medium/high | 空/其他=按域名默认
self.thinking_mode = str(config.get("thinking") or "").strip().lower()
self.api_key = config.get("api_key")
if "base_url" in config:
self.base_url = config.get("base_url")
@@ -80,14 +88,34 @@ class LLMProvider(LLMProviderBase):
return dialogue
def _apply_thinking_disabled(self, request_params: dict):
"""根据域名自动禁用思考模式"""
"""思考模式控制: 实例配置(thinking)优先, 否则按域名默认表"""
mode = getattr(self, "thinking_mode", "")
if mode in ("keep", "passthrough", "default"):
# 不干预: 既不发 thinking 参数也不查域名表(适用于强制思考/拒绝该参数的模型, 如 MaaS glm-5.3)
return
if mode in ("off", "disabled", "none", "disable"):
request_params.setdefault("extra_body", {}).update({"thinking": {"type": "disabled"}})
logger.bind(tag=TAG).info("思考强度=off: 已禁用思考模式(实例配置)")
return
if mode in ("low", "medium", "high"):
request_params.setdefault("extra_body", {}).update(
{"reasoning_effort": mode, "thinking": {"type": "enabled"}})
logger.bind(tag=TAG).info(f"思考强度={mode}: reasoning_effort 已设置(实例配置)")
return
parsed_url = urlparse(self.base_url)
domain = parsed_url.netloc
model_name = (self.model_name or "").lower()
for disabled_domain, params in THINKING_DISABLED_DOMAINS.items():
if disabled_domain in domain:
request_params.setdefault("extra_body", {}).update(params)
logger.bind(tag=TAG).info(f"为域名 {domain} 禁用思考模式,参数: {params}")
if disabled_domain not in domain:
continue
exclusions = THINKING_DISABLED_MODEL_EXCLUSIONS.get(disabled_domain, ())
if any(ex in model_name for ex in exclusions):
logger.bind(tag=TAG).info(
f"域名 {domain} 命中禁思考规则, 但模型 {self.model_name} 强制思考且拒绝该参数, 跳过注入")
break
request_params.setdefault("extra_body", {}).update(params)
logger.bind(tag=TAG).info(f"为域名 {domain} 禁用思考模式,参数: {params}")
break
def response(self, session_id, dialogue, **kwargs):
dialogue = self.normalize_dialogue(dialogue)
@@ -62,11 +62,6 @@ async def handle_mcp_message(
logger.bind(tag=TAG).debug(
f"客户端MCP服务器信息: name={name}, version={version}"
)
await asyncio.sleep(1)
logger.bind(tag=TAG).debug("初始化完成,开始请求MCP工具列表")
await send_mcp_tools_list_request(conn)
return
elif msg_id == 2: # mcpToolsListID
@@ -107,7 +107,7 @@ class ServerMCPClient:
"function": {
"name": name,
"description": tool.description,
"parameters": tool.inputSchema,
"parameters": tool.input_schema,
},
}
for name, tool in self.tools_dict.items()
@@ -24,7 +24,7 @@ class ServerMCPManager:
self.config_path = get_project_dir() + "data/.mcp_server_settings.json"
if not os.path.exists(self.config_path):
self.config_path = ""
logger.bind(tag=TAG).warning(
logger.bind(tag=TAG).debug(
f"请检查mcp服务配置文件:data/.mcp_server_settings.json"
)
self.clients: Dict[str, ServerMCPClient] = {}
@@ -5,6 +5,7 @@
import os
import asyncio
import hashlib
import threading
from typing import Dict, Any, TYPE_CHECKING
@@ -120,8 +121,9 @@ class PromptManager:
# 使用传入的提示词并缓存(如果有设备ID)
if device_id:
device_cache_key = f"device_prompt:{device_id}"
self.cache_manager.set(self.CacheType.DEVICE_PROMPT, device_cache_key, user_prompt)
self.cache_manager.set(
self.CacheType.DEVICE_PROMPT, device_cache_key, user_prompt
)
self.logger.bind(tag=TAG).debug(f"设备 {device_id} 的提示词已缓存")
self.logger.bind(tag=TAG).info(f"使用快速提示词: {user_prompt[:50]}...")
@@ -253,6 +255,18 @@ class PromptManager:
if not self.base_prompt_template:
return user_prompt
# 缓存键须覆盖所有渲染输入:emoji 开关、设备 prompt 变更否则会命中过期提示词
emoji_enabled = kwargs.get("emoji_enabled", True)
prompt_hash = hashlib.md5(user_prompt.encode("utf-8")).hexdigest()[:8]
enhanced_cache_key = (
f"enhanced_prompt:{device_id}:{emoji_enabled}:{prompt_hash}"
)
cached_prompt = self.cache_manager.get(
self.CacheType.DEVICE_PROMPT, enhanced_cache_key
)
if cached_prompt is not None:
return cached_prompt
try:
# 获取最新的时间信息(不缓存)
today_date, today_weekday, lunar_date = self._get_current_time_info()
@@ -301,11 +315,13 @@ class PromptManager:
*args,
**kwargs,
)
device_cache_key = f"device_prompt:{device_id}"
self.cache_manager.set(
self.CacheType.DEVICE_PROMPT, device_cache_key, enhanced_prompt
self.CacheType.DEVICE_PROMPT,
enhanced_cache_key,
enhanced_prompt,
ttl=300,
)
self.logger.bind(tag=TAG).info(
self.logger.bind(tag=TAG).debug(
f"构建增强提示词成功,长度: {len(enhanced_prompt)}"
)
return enhanced_prompt
+6 -2
View File
@@ -310,8 +310,12 @@ async def audio_to_data(
from core.utils.cache.manager import cache_manager
from core.utils.cache.config import CacheType
# 生成缓存键,包含文件路径和编码类型
cache_key = f"{audio_file_path}:{is_opus}"
# 缓存键含 mtime:文件被原地覆写时旧缓存自动失效(如唤醒词回复重新 TTS),避免播到过期音频
try:
mtime = os.path.getmtime(audio_file_path)
cache_key = f"{audio_file_path}:{mtime}:{is_opus}"
except OSError:
cache_key = f"{audio_file_path}:{is_opus}"
# 尝试从缓存获取结果
if use_cache:
@@ -647,8 +647,9 @@ class StreamTTSPerformanceTester:
self.results.append(result)
# 测试Linkerai TTS
result = await self.test_linkerai_tts(test_text, test_count)
self.results.append(result)
if self.config.get("TTS", {}).get("LinkeraiTTS"):
result = await self.test_linkerai_tts(test_text, test_count)
self.results.append(result)
# 测试IndexStreamTTS
result = await self.test_indexstream_tts(test_text, test_count)
@@ -1,6 +1,7 @@
import random
import httpx
from io import BytesIO
from urllib.parse import quote, urlsplit, urlunsplit, parse_qsl
from markitdown import MarkItDown, StreamInfo
from config.logger import setup_logging
from plugins_func.register import register_function, ToolType, ActionResponse, Action
@@ -54,6 +55,27 @@ CHANNEL_MAP = {
# 默认新闻来源字典,当配置中没有指定时使用
DEFAULT_NEWS_SOURCES = "澎湃新闻;百度热搜;财联社"
DEFAULT_NEWS_API_URL = "https://newsnow.busiyi.world/api/s?id="
def _build_news_api_url(source, configured_url=None):
"""拼出新闻接口的请求地址和查询参数。
配置里带 {source} 占位符就替换占位符;否则把 id 设成新闻源 id,
这样配置里已经写了 id 时也不会拼出脏串。
"""
api_url = str(configured_url).strip() if configured_url else DEFAULT_NEWS_API_URL
if "{source}" in api_url:
return api_url.replace("{source}", quote(source, safe="")), None
parts = urlsplit(api_url)
query = dict(parse_qsl(parts.query, keep_blank_values=True))
query["id"] = source
return (
urlunsplit((parts.scheme, parts.netloc, parts.path, "", parts.fragment)),
query,
)
def _get_newsnow_config(conn):
# 从连接配置获取
plugins = conn.config.get("plugins", {})
@@ -61,6 +83,8 @@ def _get_newsnow_config(conn):
sources = newsnow.get("news_sources", "")
if isinstance(sources, str) and sources.strip():
return sources
if isinstance(sources, (list, tuple)):
return ";".join(str(item).strip() for item in sources if str(item).strip())
return ""
@@ -113,23 +137,26 @@ GET_NEWS_FROM_NEWSNOW_FUNCTION_DESC = {
async def fetch_news_from_api(conn: "ConnectionHandler", source="thepaper"):
"""从API获取新闻列表"""
try:
api_url = f"https://newsnow.busiyi.world/api/s?id={source}"
news_config = conn.config.get("plugins", {}).get("get_news_from_newsnow", {})
if news_config.get("url"):
api_url = news_config["url"] + source
configured_url = news_config.get("url") if isinstance(news_config, dict) else None
api_url, request_params = _build_news_api_url(source, configured_url)
headers = {"User-Agent": "Mozilla/5.0"}
async with httpx.AsyncClient(timeout=httpx.Timeout(10.0, connect=3.0)) as client:
response = await client.get(api_url, headers=headers)
response = await client.get(api_url, params=request_params, headers=headers)
if response.status_code != 200:
logger.bind(tag=TAG).error(
f"获取新闻API失败:HTTP {response.status_code},source={source}"
)
return []
data = response.json()
if "items" in data:
return data["items"]
else:
items = data.get("items") if isinstance(data, dict) else None
if not isinstance(items, list):
logger.bind(tag=TAG).error(f"获取新闻API响应格式错误: {data}")
return []
return [item for item in items if isinstance(item, dict) and item.get("title")]
except Exception as e:
logger.bind(tag=TAG).error(f"获取新闻API失败: {e}")
@@ -239,18 +266,24 @@ async def get_news_from_newsnow(
english_source_id = None
# 检查输入的中文名称是否在配置的新闻源中
news_sources_list = [
name.strip() for name in news_sources.split(";") if name.strip()
]
news_sources_list = [name.strip() for name in news_sources.split(";") if name.strip()]
if source in news_sources_list:
# 如果输入的中文名称在配置的新闻源中,在 CHANNEL_MAP 中查找对应的英文ID
english_source_id = CHANNEL_MAP.get(source)
elif not source and news_sources_list:
source = news_sources_list[0]
english_source_id = CHANNEL_MAP.get(source)
# 如果找不到对应的英文ID,使用默认源
if not english_source_id:
logger.bind(tag=TAG).warning(f"无效的新闻源: {source},使用默认源澎湃新闻")
english_source_id = "thepaper"
source = "澎湃新闻"
fallback_source = next(
(name for name in news_sources_list if CHANNEL_MAP.get(name)),
"澎湃新闻",
)
logger.bind(tag=TAG).warning(
f"无效的新闻源: {source},使用配置中的默认源{fallback_source}"
)
english_source_id = CHANNEL_MAP[fallback_source]
source = fallback_source
logger.bind(tag=TAG).info(f"获取新闻: 新闻源={source}({english_source_id})")
@@ -266,20 +299,21 @@ async def get_news_from_newsnow(
# 随机选择一条新闻
selected_news = random.choice(news_items)
title = selected_news.get("title", "未知标题")
# 保存当前新闻链接到连接对象,以便后续查询详情
if not hasattr(conn, "last_newsnow_link"):
conn.last_newsnow_link = {}
conn.last_newsnow_link = {
"url": selected_news.get("url", "#"),
"title": selected_news.get("title", "未知标题"),
"title": title,
"source_id": english_source_id,
}
# 构建新闻报告
news_report = (
f"根据下列数据,用{lang}回应用户的新闻查询请求:\n\n"
f"新闻标题: {selected_news['title']}\n"
f"新闻标题: {title}\n"
# f"新闻来源: {source}\n"
f"(请以自然、流畅的方式向用户播报这条新闻标题,"
f"提示用户可以要求获取详细内容,此时会获取新闻的详细内容。)"
@@ -1,3 +1,4 @@
import asyncio
import httpx
from bs4 import BeautifulSoup
from config.logger import setup_logging
@@ -112,20 +113,106 @@ WEATHER_CODE_MAP = {
async def fetch_city_info(location, api_key, api_host):
url = f"https://{api_host}/geo/v2/city/lookup?key={api_key}&location={location}&lang=zh"
url = f"https://{api_host}/geo/v2/city/lookup"
async with httpx.AsyncClient(timeout=httpx.Timeout(5.0, connect=3.0)) as client:
response = await client.get(url, headers=HEADERS)
data = response.json()
if data.get("error") is not None:
response = await client.get(
url,
params={
"key": api_key,
"location": location,
"lang": "zh",
"number": 1,
},
headers=HEADERS,
)
try:
data = response.json()
except ValueError:
logger.bind(tag=TAG).error(
f"获取天气失败,原因:{data.get('error', {}).get('detail')}"
f"获取天气城市失败:HTTP {response.status_code} 返回了非 JSON 内容"
)
return None
if not isinstance(data, dict):
logger.bind(tag=TAG).error("获取天气城市失败:响应不是 JSON 对象")
return None
if response.status_code != 200 or data.get("code") != "200":
logger.bind(tag=TAG).error(
f"获取天气城市失败:HTTP {response.status_code},代码 {data.get('code')}"
)
return None
return data.get("location", [])[0] if data.get("location") else None
async def fetch_weather_data(city_info, api_key, api_host):
location_id = city_info.get("id")
if not location_id:
return None
base_url = f"https://{api_host}/v7/weather"
params = {"key": api_key, "location": location_id, "lang": "zh"}
async def request(client, endpoint):
response = await client.get(f"{base_url}/{endpoint}", params=params, headers=HEADERS)
try:
data = response.json()
except ValueError:
return None
if response.status_code != 200 or data.get("code") != "200":
logger.bind(tag=TAG).error(
f"获取天气数据失败:接口={endpoint} HTTP {response.status_code},代码 {data.get('code')}"
)
return None
return data
try:
async with httpx.AsyncClient(timeout=httpx.Timeout(10.0, connect=3.0)) as client:
now, daily = await asyncio.gather(
request(client, "now"), request(client, "7d")
)
except Exception as e:
logger.bind(tag=TAG).warning(f"获取天气 JSON 接口失败,将尝试网页接口: {e}")
return None
if now is None or daily is None:
return None
return now, daily
def build_weather_report(city_info, now_data, daily_data):
city_name = city_info.get("name") or city_info.get("adm2") or "未知位置"
now = now_data.get("now", {})
weather_report = f"您查询的位置是:{city_name}\n\n"
weather_report += f"当前天气: {now.get('text', '未知')}\n"
current_basic = {
"温度": f"{now['temp']}℃" if now.get("temp") is not None else None,
"体感温度": f"{now['feelsLike']}℃" if now.get("feelsLike") is not None else None,
"湿度": f"{now['humidity']}%" if now.get("humidity") is not None else None,
"风向": now.get("windDir"),
"风力": now.get("windScale"),
"能见度": f"{now['vis']}公里" if now.get("vis") is not None else None,
"气压": f"{now['pressure']}百帕" if now.get("pressure") is not None else None,
"降水量": f"{now['precip']}毫米" if now.get("precip") is not None else None,
}
current_basic = {key: value for key, value in current_basic.items() if value}
if current_basic:
weather_report += "详细参数:\n"
weather_report += "".join(f" · {key}: {value}\n" for key, value in current_basic.items())
weather_report += "\n未来7天预报:\n"
for item in daily_data.get("daily", [])[:7]:
date = item.get("fxDate", "未知日期")
weather = item.get("textDay") or WEATHER_CODE_MAP.get(item.get("iconDay"), "未知")
high = item.get("tempMax", "未知")
low = item.get("tempMin", "未知")
weather_report += f"{date}: {weather},气温 {low}~{high}℃\n"
weather_report += "\n(如需某一天的具体天气,请告诉我日期)"
return weather_report
async def fetch_weather_page(url):
async with httpx.AsyncClient(timeout=httpx.Timeout(10.0, connect=3.0)) as client:
async with httpx.AsyncClient(
timeout=httpx.Timeout(10.0, connect=3.0), follow_redirects=True
) as client:
response = await client.get(url, headers=HEADERS)
return BeautifulSoup(response.text, "html.parser") if response.status_code == 200 else None
@@ -201,29 +288,31 @@ async def get_weather(conn: "ConnectionHandler", location: str = None, lang: str
city_info = await fetch_city_info(location, api_key, api_host)
if not city_info:
return ActionResponse(
Action.REQLLM, f"未找到相关的城市: {location},请确认地点是否正确", None
Action.REQLLM,
f"暂时查不到 {location} 的天气(城市未找到或天气服务不可用),请稍后再试",
None,
)
soup = await fetch_weather_page(city_info["fxLink"])
if not soup:
return ActionResponse(Action.REQLLM, None, "请求失败")
city_name, current_abstract, current_basic, temps_list = parse_weather_info(soup)
weather_report = f"您查询的位置是:{city_name}\n\n当前天气: {current_abstract}\n"
# 添加有效的当前天气参数
if current_basic:
weather_report += "详细参数:\n"
for key, value in current_basic.items():
if value != "0": # 过滤无效值
weather_report += f" · {key}: {value}\n"
# 添加7天预报
weather_report += "\n未来7天预报:\n"
for date, weather, high, low in temps_list:
weather_report += f"{date}: {weather},气温 {low}~{high}\n"
# 提示语
weather_report += "\n(如需某一天的具体天气,请告诉我日期)"
weather_data = await fetch_weather_data(city_info, api_key, api_host)
if weather_data:
weather_report = build_weather_report(city_info, *weather_data)
else:
fx_link = city_info.get("fxLink")
soup = await fetch_weather_page(fx_link) if fx_link else None
if not soup:
return ActionResponse(Action.REQLLM, None, "请求天气数据失败")
city_name, current_abstract, current_basic, temps_list = parse_weather_info(soup)
weather_report = f"您查询的位置是:{city_name}\n\n当前天气: {current_abstract}\n"
if current_basic:
weather_report += "详细参数:\n"
weather_report += "".join(
f" · {key}: {value}\n" for key, value in current_basic.items() if value != "0"
)
weather_report += "\n未来7天预报:\n"
weather_report += "".join(
f"{date}: {weather},气温 {low}~{high}\n"
for date, weather, high, low in temps_list
)
weather_report += "\n(如需某一天的具体天气,请告诉我日期)"
# 缓存完整的天气报告
cache_manager.set(CacheType.WEATHER, weather_cache_key, weather_report)
@@ -0,0 +1,336 @@
"""Tests for the weather and newsnow plugin helpers.
The two plugins talk to external APIs, so the HTTP layer is replaced with a
fake client and every request is recorded. Nothing here touches the network.
"""
import httpx
import pytest
from plugins_func.functions import get_news_from_newsnow as news_module
from plugins_func.functions import get_weather as weather_module
class FakeResponse:
def __init__(self, status_code=200, payload=None, text="", json_error=None):
self.status_code = status_code
self.payload = payload
self.text = text
self.json_error = json_error
def json(self):
if self.json_error is not None:
raise self.json_error
return self.payload
class FakeAsyncClient:
"""Stands in for httpx.AsyncClient and records what was requested."""
requests = []
responder = None
init_kwargs = {}
def __init__(self, *args, **kwargs):
FakeAsyncClient.init_kwargs = kwargs
async def __aenter__(self):
return self
async def __aexit__(self, *exc_info):
return False
async def get(self, url, params=None, headers=None):
FakeAsyncClient.requests.append({"url": url, "params": params})
return FakeAsyncClient.responder(url, params)
@pytest.fixture
def fake_http(monkeypatch):
"""Replace httpx.AsyncClient with the recording fake."""
FakeAsyncClient.requests = []
FakeAsyncClient.responder = lambda url, params: FakeResponse(payload={"code": "200"})
FakeAsyncClient.init_kwargs = {}
monkeypatch.setattr(httpx, "AsyncClient", FakeAsyncClient)
return FakeAsyncClient
class FakeConn:
def __init__(self, plugins=None):
self.config = {"plugins": plugins or {}}
self.client_ip = None
# ---------------------------------------------------------------------------
# 城市查询
# ---------------------------------------------------------------------------
async def test_city_lookup_returns_none_on_error_code(fake_http):
fake_http.responder = lambda url, params: FakeResponse(payload={"code": "402"})
result = await weather_module.fetch_city_info("杭州", "key", "host")
assert result is None
async def test_city_lookup_returns_none_on_non_json_body(fake_http):
fake_http.responder = lambda url, params: FakeResponse(
text="<html>oops</html>", json_error=ValueError("not json")
)
result = await weather_module.fetch_city_info("杭州", "key", "host")
assert result is None
async def test_city_lookup_returns_none_on_non_dict_body(fake_http):
fake_http.responder = lambda url, params: FakeResponse(payload=["not", "an", "object"])
result = await weather_module.fetch_city_info("杭州", "key", "host")
assert result is None
async def test_city_lookup_returns_none_on_empty_location(fake_http):
fake_http.responder = lambda url, params: FakeResponse(
payload={"code": "200", "location": []}
)
result = await weather_module.fetch_city_info("杭州", "key", "host")
assert result is None
async def test_city_lookup_searches_all_cities(fake_http):
"""不带 range 参数就是全球检索,写死 cn 会查不到境外城市。"""
fake_http.responder = lambda url, params: FakeResponse(
payload={"code": "200", "location": [{"id": "101210101", "name": "杭州"}]}
)
result = await weather_module.fetch_city_info("杭州", "key", "host")
assert result == {"id": "101210101", "name": "杭州"}
sent = fake_http.requests[0]["params"]
assert "range" not in sent
assert sent["location"] == "杭州"
assert sent["number"] == 1
# ---------------------------------------------------------------------------
# 天气数据
# ---------------------------------------------------------------------------
def _weather_responder(now_payload, daily_payload):
def responder(url, params):
if url.endswith("/now"):
return now_payload if isinstance(now_payload, FakeResponse) else FakeResponse(
payload=now_payload
)
return daily_payload if isinstance(daily_payload, FakeResponse) else FakeResponse(
payload=daily_payload
)
return responder
async def test_weather_data_needs_both_now_and_daily(fake_http):
fake_http.responder = _weather_responder(
{"code": "200", "now": {"text": "晴"}},
FakeResponse(status_code=401, payload={"code": "401"}),
)
assert await weather_module.fetch_weather_data({"id": "1"}, "key", "host") is None
async def test_weather_data_needs_daily_when_now_failed(fake_http):
fake_http.responder = _weather_responder(
FakeResponse(status_code=401, payload={"code": "401"}),
{"code": "200", "daily": [{"fxDate": "2026-10-24"}]},
)
assert await weather_module.fetch_weather_data({"id": "1"}, "key", "host") is None
async def test_weather_data_returns_both_when_all_succeed(fake_http):
fake_http.responder = _weather_responder(
{"code": "200", "now": {"text": "晴"}},
{"code": "200", "daily": [{"fxDate": "2026-10-24"}]},
)
now, daily = await weather_module.fetch_weather_data({"id": "1"}, "key", "host")
assert now["now"]["text"] == "晴"
assert daily["daily"][0]["fxDate"] == "2026-10-24"
async def test_weather_page_follows_redirects(fake_http):
"""旧形态 fxLink 会 302,不跟随重定向就拿不到内容。"""
fake_http.responder = lambda url, params: FakeResponse(text="<html></html>")
await weather_module.fetch_weather_page("https://www.qweather.com/weather/101210101.html")
assert fake_http.init_kwargs.get("follow_redirects") is True
def test_weather_report_keeps_pressure_and_precipitation():
report = weather_module.build_weather_report(
{"name": "杭州"},
{"now": {"text": "多云", "temp": "24", "pressure": "1003", "precip": "0.0"}},
{"daily": []},
)
assert "气压: 1003百帕" in report
assert "降水量: 0.0毫米" in report
async def test_weather_report_keeps_pressure_and_precipitation_in_flow(monkeypatch):
"""走完整条链路时报告里也要有气压和降水量。"""
monkeypatch.setattr(
weather_module,
"fetch_city_info",
_async_value({"id": "1", "name": "杭州"}),
)
monkeypatch.setattr(
weather_module,
"fetch_weather_data",
_async_value(
({"now": {"text": "多云", "pressure": "1003", "precip": "0.0"}}, {"daily": []})
),
)
response = await weather_module.get_weather(
FakeConn({"get_weather": {"api_key": "k", "api_host": "h"}}), location="气压测试市"
)
assert "气压: 1003百帕" in response.result
assert "降水量: 0.0毫米" in response.result
async def test_missing_fxlink_does_not_call_web_fallback(monkeypatch):
"""城市信息里没有 fxLink 时不该去抓网页,直接报失败。"""
called = []
async def fake_city_info(*args, **kwargs):
return {"id": "1", "name": "杭州"}
async def fake_weather_data(*args, **kwargs):
return None
async def fake_page(url):
called.append(url)
return None
monkeypatch.setattr(weather_module, "fetch_city_info", fake_city_info)
monkeypatch.setattr(weather_module, "fetch_weather_data", fake_weather_data)
monkeypatch.setattr(weather_module, "fetch_weather_page", fake_page)
response = await weather_module.get_weather(
FakeConn({"get_weather": {"api_key": "k", "api_host": "h"}}), location="无链接市"
)
assert called == []
assert response.response == "请求天气数据失败"
async def test_city_not_found_message_does_not_blame_the_place_name(monkeypatch):
"""key 失效和城市找不到都落到这句话,不能让用户去核对地名。"""
async def fake_city_info(*args, **kwargs):
return None
monkeypatch.setattr(weather_module, "fetch_city_info", fake_city_info)
response = await weather_module.get_weather(
FakeConn({"get_weather": {"api_key": "k", "api_host": "h"}}), location="随便什么市"
)
assert "请确认地点是否正确" not in response.result
assert "天气服务不可用" in response.result
# ---------------------------------------------------------------------------
# 新闻接口
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"configured_url, expected_url, expected_params",
[
(None, "https://newsnow.busiyi.world/api/s", {"id": "thepaper"}),
(
"https://newsnow.busiyi.world/api/s?id=thepaper",
"https://newsnow.busiyi.world/api/s",
{"id": "thepaper"},
),
(
"https://newsnow.busiyi.world/api/s?id={source}",
"https://newsnow.busiyi.world/api/s?id=thepaper",
None,
),
(
"https://newsnow.busiyi.world/api/s",
"https://newsnow.busiyi.world/api/s",
{"id": "thepaper"},
),
(
"https://newsnow.busiyi.world/api/s?id=",
"https://newsnow.busiyi.world/api/s",
{"id": "thepaper"},
),
(
"https://newsnow.busiyi.world/api/s?id=thepaper&limit=5",
"https://newsnow.busiyi.world/api/s",
{"id": "thepaper", "limit": "5"},
),
],
)
def test_news_api_url_forms(configured_url, expected_url, expected_params):
url, params = news_module._build_news_api_url("thepaper", configured_url)
assert url == expected_url
assert params == expected_params
async def test_news_api_url_forms_are_actually_requested(fake_http):
"""配置里写了 id 也不能拼出脏串。"""
fake_http.responder = lambda url, params: FakeResponse(
payload={"items": [{"id": "1", "title": "标题"}]}
)
conn = FakeConn({"get_news_from_newsnow": {"url": "https://newsnow.busiyi.world/api/s?id=thepaper"}})
await news_module.fetch_news_from_api(conn, "thepaper")
assert fake_http.requests[0]["url"] == "https://newsnow.busiyi.world/api/s"
assert fake_http.requests[0]["params"] == {"id": "thepaper"}
async def test_news_items_must_be_a_list(fake_http):
fake_http.responder = lambda url, params: FakeResponse(payload={"items": "oops"})
assert await news_module.fetch_news_from_api(FakeConn(), "thepaper") == []
async def test_news_items_without_title_are_filtered_out(fake_http):
fake_http.responder = lambda url, params: FakeResponse(
payload={"items": [{"id": "1"}, {"id": "2", "title": ""}]}
)
assert await news_module.fetch_news_from_api(FakeConn(), "thepaper") == []
async def test_news_items_keep_valid_entries(fake_http):
fake_http.responder = lambda url, params: FakeResponse(
payload={"items": [{"id": "1", "title": ""}, {"id": "2", "title": "有效标题"}]}
)
items = await news_module.fetch_news_from_api(FakeConn(), "thepaper")
assert items == [{"id": "2", "title": "有效标题"}]
def _async_value(value):
async def _inner(*args, **kwargs):
return value
return _inner