Compare commits

..
Author SHA1 Message Date
wengzh 4f6e9ec6b1 Merge pull request #3544 from xinnan-tech/py-hass-devices
Py hass devices
2026-11-05 16:37:11 +08:00
Sakura-RanChen 9f5f520576 fix: 修复ha读取异常 2026-11-05 15:36:04 +08:00
Bao1573 70f1a7bfd8 fix(hass): devices 写成列表时也能拼进提示词 (#3541)
config.yaml 示例和手机端智控台给的 devices 是列表,append_devices_to_prompt 却按字符串直接拼,每次连接都报 can only concatenate str (not "list") to str,设备清单进不了提示词,模型只能猜 entity_id。列表按每项一行拼,字符串原样用,其它当空。附测试。
2026-11-05 11:50:39 +08:00
wengzh e61f5aee28 Merge pull request #3543 from xinnan-tech/py-tool-cache
fix: prevent tool description cache mutation (#3540)
2026-11-05 10:41:46 +08:00
koala553 c59b29c723 fix: prevent tool description cache mutation (#3540) 2026-11-05 10:10:17 +08:00
wengzh c8e60d1325 Merge pull request #3539 from xinnan-tech/py-Outcrawl
Py outcrawl
2026-11-04 11:38:20 +08:00
Sakura-RanChen 6589f7f481 调整为code判断 2026-11-04 11:26:20 +08:00
Nirupam Bhowmick 7d9f47b0d1 feat(web_search): 新增Outcrawl搜索源 (#3538) 2026-11-04 10:22:41 +08:00
hrz 5b0de039c9 让 CI 真正跑起三类测试并修掉挡路的测试缺陷 (#3475)
* 让 CI 真正跑起三类测试并修掉挡路的测试缺陷

之前 CI 只跑了一小部分,Java 那边的测试被 pom 里的开关整个跳过了。
这条把它打开,并补上 MySQL、Redis 两个容器,Java、契约、Python 三类
测试都在 CI 里真跑起来。

另外修了两个挡路的地方:一条 Java 用例单跑会因为实体信息没注册而报错,
现在自己注册;Python 的假网络层把整个 httpx 类换掉了,第三方库拼请求
时报错,改成只换发请求那一步。

* 给 runner 补上 libopus,修掉音频用例报错

opuslib_next 是 ctypes 绑定,真正编解码靠系统里的 libopus。runner 上默认
没有,import 失败后测试里换上的是自己写的桩,桩里的编码器只有构造方法,
一调就报错。正式镜像里是装了这个库的,CI 少了这一步。

顺手把桩的报错改成一句人话,别再让人对着缺方法的报错猜根因。

* 本地没起库也能构建,CI 里集成测试不许跳过

三个要连 MySQL 和 Redis 的集成测试加了守卫:依赖连不上就整类跳过,日志写明缺什么,构建照样是绿的;CI 上依赖不通就直接失败。另外加了一个门禁检查脚本和 CI 步骤,盯住这三个类有没有真跑,守卫哪天手滑把它们全跳过了,CI 必须变红。README 里补了本地怎么起依赖、怎么只编译不跑测试。
2026-11-03 17:44:29 +08:00
wengzh 4e1405a6f6 Merge pull request #3537 from xinnan-tech/py-test-tool
Py test tool
2026-11-03 15:03:17 +08:00
Sakura-RanChen f9138e65b7 update: 使用专用线程池避免阻塞默认池 2026-11-03 10:43:53 +08:00
Bao1573 9b8a9b585d fix(tools): 服务端插件的同步函数放线程池,不阻塞事件循环 (#3528)
ServerPluginExecutor.execute 在事件循环上被 await,而 hass_state、天气等插件用 requests 同步阻塞,工具执行期间整个循环停转、正在发的音频节拍也停。同步函数改用 run_in_executor 放到线程池;async def 插件照常 await;传参规则不变。附测试。
2026-11-02 15:29:34 +08:00
欣南科技 92337881a7 Merge pull request #3531 from xinnan-tech/fix/readme-asr-bailian-lmstudio
Fix/readme asr bailian lmstudio
2026-11-01 07:38:49 +08:00
欣南科技 a93b71f9e7 Merge branch 'main' into fix/readme-asr-bailian-lmstudio 2026-11-01 07:38:36 +08:00
hrz 325f52469a 设备呼叫文档改对热词字段的位置 (#3509)
* 设备呼叫文档改对热词字段的位置

热词文件名称是语音识别供应器的字段,文档里写成配到语音合成上,用户照着在语音合成里找根本找不到;同一句 AEC 提示连写三遍,只留一条。

* 设备呼叫文档改对热词配置的供应器名称

把接口类型改成供应器,列出还有热词文件名称的两条模型,补上点确认保存和改完要重连设备。

* 设备呼叫文档里热词字段的说法改准

「这一项」读起来像供应器下拉的子项,实际是弹窗调用信息区域里独立的一个输入框,两处都改成照实说。

* Clarify AEC functionality requirement in guide

Added repeated instruction for enabling AEC functionality during firmware compilation.
2026-11-01 07:10:28 +08:00
hrz 3230a69c99 Merge pull request #3515 from shinjiyu/fix/config-provider-copy-accuracy
fix(config): 纠正供应器占位文案与 PaddleSpeech 协议说明
2026-11-01 06:56:22 +08:00
shinjiyu 1ac4c09946 fix(config): 按审核意见补齐占位文案与 PaddleSpeech 协议收敛
- AliLLM 占位改为「你的阿里云百炼API密钥」;ASR 段「语音合成」改为「语音识别」;XunfeiSparkLLM 占位改为 api_key
- PaddleSpeech:启动时校验 protocol;代码兜底 URL 与 config 对齐为本机;智控台新增 changeset 去掉 http 选项
2026-10-30 20:55:56 +08:00
wengzh 1a095d5bed Merge pull request #3523 from xinnan-tech/py-test-fix
Py test fix
2026-10-29 14:20:59 +08:00
Sakura-RanChen 01289b8fd4 仅保留mcp2.x行为 2026-10-29 14:17:38 +08:00
shinjiyuandshinjiyu ef2fcdbcb1 fix(server_mcp): read Tool.input_schema for mcp 2.3.0 (#3522)
mcp==2.3.0 Tool stores the JSON Schema on input_schema (alias
inputSchema). Accessing tool.inputSchema raises AttributeError and
aborts server_mcp init after a successful tools/list.

Use getattr fallback so function-calling conversion works.

Closes #3521

Co-authored-by: shinjiyu <shinjiyu@users.noreply.github.com>
2026-10-29 14:08:44 +08:00
hrz 942ac2fe12 通讯录没起名时不再取对方设备的名字 (#3473)
* 别把对方给设备起的名字抄进自己的通讯录

通讯录里没起名的时候,以前会跑去设备表查目标设备的名字。那是设备主人在智控台上自己起的,
直接抄进调用方的通讯录,等于把别人的昵称漏了出去。现在改成一律用 MAC 尾号称呼,
只有目标是自己那台设备时,才用自己设备表里的名字。补了单测,覆盖对方已起名、目标是自己、
显式起名、目标没登记这几种情况。

* 取自己的设备名改用不会撞行的查法

设备表里同一个 MAC 可能有两行只差大小写,一次只准取一条的查法撞上就抛异常,保存通讯录整个接口带崩。改成按归属查的那一种,按 id 取第一条。

补齐自己那台设备在设备表里没起名时退回尾号称呼的用例,另有几条用例的名字跟实际测的东西对不上,一并改掉。
2026-10-29 11:56:48 +08:00
wengzh 4dbed0ab1d Merge pull request #3520 from xinnan-tech/py-test-weather
fix: harden weather and news plugins (#3465)
2026-10-29 10:10:50 +08:00
Jacobandhrz 15d9b0fba1 fix: harden weather and news plugins (#3465)
* fix: harden weather and news plugins

* 修掉天气新闻插件的三处鲁棒性缺陷

城市查询不再写死国内范围,境外城市也能查到。实时天气和七天预报要都拿到才走 JSON,缺一个就整体退回网页路径,不产出残缺报告。网页抓取加上跟随跳转,旧链接的 302 也能拿到内容。城市查不到时不再让用户去核对地名,多数情况是密钥失效。报告补上气压和降水量。新闻 URL 拼接抽成纯函数,配了 23 项单测。

---------

Co-authored-by: hrz <1710360675@qq.com>
2026-10-29 09:46:01 +08:00
wengzh 8c5330f742 Merge pull request #3516 from xinnan-tech/py-test-think
Py test think
2026-10-28 16:38:12 +08:00
Sakura-RanChen 549261fdff PR 提供的keep 逃生门要求用户自行发现问题、了解该魔法值并手改 configJson,对普通用户不可达。
调整LLM thinking 域名禁用规则支持按模型名豁免(避免 MaaS glm-5.3 被注入参数后 400)
2026-10-28 16:18:58 +08:00
2cb0562ade feat(llm): 华为云 MaaS 思考强度实例级配置 + 管理台 select 字段类型 (#3446)
- openai provider: THINKING_DISABLED_DOMAINS 补 modelarts-maas.com
- openai provider: 实例配置支持可选 thinking 字段(off/low/medium/high/keep),
  实例值优先于域名默认表; keep=完全直通(适配强制思考且拒绝该参数的模型, 如 MaaS glm-5.3)
- manager-web: 模型配置表单渲染器支持 select 字段类型(el-select + field.options)
  (ModelEditDialog / AddModelDialog 两处渲染点与类型映射)

Co-authored-by: SecInsight <secinsight@users.noreply.github.com>
Co-authored-by: Sakura-RanChen <1908198662@qq.com>
2026-10-28 15:20:28 +08:00
shinjiyu ccc77799d9 fix(config): 纠正 AliLLM 占位文案与 PaddleSpeech 协议说明
- AliLLM 的 api_key 占位误写成 deepseek,照抄会把错误 key 填进百炼(#3495)
- PaddleSpeechTTS 实际只实现 websocket,去掉 config/报错里对 http 的承诺(#3503)
2026-10-28 06:02:21 +08:00
hrz d9c2e8c5d9 德语版README把MCP接入点改用Zugriffspunkt (#3511)
Endpunkt说的是终端点,这里是访问点,术语换过来跟英文版一致。
2026-10-27 21:18:24 +08:00
hrz 1a675e4d6b 部署文档改准:示例配置、镜像架构和日志样例 (#3507)
* 部署文档改准:示例配置、镜像架构和日志样例

最简配置示例里选中的模型改成仓库默认那个;镜像架构那句改成同时提供两种架构;升级步骤去掉单模块没有的 web 容器;compose 文件名、章节编号、配置路径和日志版本号一并改对。

* 部署文档的启动日志样例改成实际模块串

全局日志器拿不到各模块的名字,样例里那串缩写换成实际输出的样子。

* 部署文档的升级步骤编号改对并补全日志样例

升级那几步改成4开头,删掉单模块用不到的web镜像,样例补上视觉分析接口那行,分隔符也对齐实际输出。

* 部署文档里配置说明的路径改成仓库链接

按 docker 方式部署的人,xiaozhi-server 目录下并没有 config.yaml,这句路径换成能点开的仓库链接,两种方式都用得上。
2026-10-27 21:13:05 +08:00
hrz 40a7ed0391 三份译文里的术语跟本文件其它地方对齐 (#3506)
德语版把回答速度统一成反应速度,英语版和葡语版把 MCP 接入点统一成本文件其它地方用的说法。
2026-10-27 21:10:55 +08:00
hrz 62e316f298 把 README 的错字修正和品牌名统一带到 main (#3504)
* 修中文 README 的四处错字并统一品牌名写法

中文 README 自己有四处错:括号不配对、该写客户端的地方写成了客户、提示语用逗号收尾、OpenAI 写成了小写。葡语版和越南语版跟着源文一起小写的几处也改了。

* 把中文源的括号和越南语版的句号改对

中文源文 Memory 那行第 3 格还是全角括号,改成半角,
跟另外五版一样。越南语版提示语那行还是逗号收尾,改成句号。
2026-10-27 21:09:57 +08:00
hrz 76cef92cb2 修正性能测试和声纹识别两份文档里跟代码对不上的十二处 (#3502)
* 修正两份操作文档里跟代码对不上的地方

性能测试那份把配置路径写错、没提要先装依赖,步骤里少了非流式语音合成,流式语音合成要配哪些供应器也没写;声纹识别那份的重启命令指向两个不同的编排文件,有两句话连写三遍,编号也乱了,示例漏了相似度阈值。逐条改到跟代码一致。

* 把性能测试文档的说明改得跟代码一致

供应器表里两家的写法与代码相反,改成如实描述;步骤清单补回流式语音识别;
示例里的视觉模型段改成能跑通的形态;声纹文档两个重复的标题改掉一个。

* 性能测试文档的说法跟Linkerai跳过行为对齐

它没配就整项跳过,不再算进报错后跳过那一类。与#3510的行为对齐。

* 性能测试文档改对缺供应器时的记法

不是跳过,是报错后记成失败,结果表里多一行。带守卫的那几家才是整项跳过。

* 强调易错处
2026-10-27 21:09:09 +08:00
hrz 826c25d3e5 修正 PaddleSpeech 部署文档里对不上代码的四处 (#3501)
* 修正 PaddleSpeech 部署文档里对不上的地方

示例配置里写了 provider 根本不读的那项采样率,还说 http 协议能自动选,其实只实现了 websocket;代码那一节是空的;启动命令没说在哪个目录跑,配置文件的路径只给了 Windows 写法。逐条改到跟代码一致。

* 补上 PaddleSpeech 部署文档的配置目录和环境说明

配置文件那节改成新建,写明 data 目录被 gitignore 忽略,要先建出来。
数字人页面的启动命令改成从仓库根目录算,另开一个终端。
代码块围栏换成 bash,另补一段说明小智服务用哪个 Python 环境,
以及数字人页面那套依赖在哪。

* PaddleSpeech 部署文档修四处会照抄失败的地方

启动命令的围栏改成 bash;建 data 目录那条标明在仓库根目录执行;启动服务那段不再重复 cd;数字人那节补一句 numpy 的依赖来自哪两个文件。

* 修正conda环境名
2026-10-27 21:05:07 +08:00
hrz f1965167cc 性能测试跳过未配置的Linkerai语音合成 (#3510)
配置里没有这个供应器时整项跳过,不再报一条测试失败。
2026-10-27 15:35:38 +08:00
wengzh 6c352e5978 Merge pull request #3436 from xinnan-tech/perf/hello-handshake-fast
hello 握手首屏响应从 4s 降到 200ms
2026-10-27 14:34:13 +08:00
Sakura-RanChen 032f6f1582 fix: 修复唤醒流程错误阻塞事件接收循环 2026-10-27 10:36:35 +08:00
hrz a377d89dcd 补齐德语版平台表里漏掉的四个供应器 (#3500)
德语版把火山和阿里拆得比中文细,拆的时候漏了同族的另一个:LLM 少火山网关,TTS 少火山网关和百炼,ASR 少豆包流式 V2。补上,名字按德语版自己的写法。
2026-10-26 22:47:47 +08:00
hrz d35b9f8b00 修中文 README 的四处错字并统一品牌名写法 (#3498)
中文 README 自己有四处错:括号不配对、该写客户端的地方写成了客户、提示语用逗号收尾、OpenAI 写成了小写。葡语版和越南语版跟着源文一起小写的几处也改了。
2026-10-26 22:30:14 +08:00
hrz d687121e99 把四份 README 里图片的说明文字译成本地语言 (#3497)
en、de、pt_BR、vi 四份里 15 条演示图的说明文字和架构图那句还是中文,这轮全部译成本地语言,越南语版的校徽说明、葡语版声纹那句也一起改掉。图片路径一个字没动。
2026-10-26 22:29:27 +08:00
hrz badf6fe3d0 按中文源文补齐四份 README 的描述和术语 (#3494)
横向扫尾清单里的几处:star-history 图表代码还是旧的,换成中文那版;
德语版开头少了知识库这一项;英德葡三版的意图识别行丢了外挂大模型和
自主函数调用这两层说法;越南语版首屏把声纹写成语音识别,
部署表的特点列和版本比较基准也写窄了。
2026-10-26 22:28:49 +08:00
hrz adb4a15f2d 同步西语版 README 与中文源文的平台清单和说明 (#3492)
* 同步西语版 README 与中文源文的平台清单和说明

西语版落后中文源文一整轮:平台表少了 CustomTTS、SherpaParaformerASR、VoskASR 这些,
旧的 GizwitsTTS、ACGNTTS 还在,Index-TTS、PaddleSpeech 的键名也旧了。
部署表里那组核数和内存全库查不到出处,按中文删掉;界面语种、插件加载、
测试工具和声纹说明那段一并对齐。中文源文和其它四个语言版没动。

* 把音频交互测试工具的用法改成指向 README

原来在用法里内嵌装依赖的命令,句子太长。改成指向 main/digital-human 的 README,
依赖怎么装、模型从哪来,那份 README 里都写了。

* 补上意图识别行里外挂的大模型这一层

中文源文写的是支持外挂的大模型意图识别,西语版只写了模型本身,
把可以外挂这层意思漏了。补上。
2026-10-26 22:27:32 +08:00
hrz a5f9a782d6 补齐五份 README 的供应器清单和音频测试工具用法 (#3491)
* 补上阿里百炼和 LM Studio 并统一越南语品牌写法

ASR 那行中文只写阿里云,配置里阿里系有三条,另外四份也按各自写法补上百炼。LM Studio 走的是 openai 接口,仓里没有独立的供应器,所以并进 openai 那一行,免费平台那列不动。越南语版三处 iFlytek 统一成官方写法 iFLYTEK。

* 补上音频交互测试工具的前置步骤

digital-human 的说明里要求先装 wakeword 依赖再启动,README 里原来只写了启动命令,照着做会缺依赖。五个语言版的用法都补上这一步。

* 把性能测试脚本的提示写清文档在仓库根

脚本是在 main/xiaozhi-server 下执行的,那儿没有 docs 目录,文档在仓库根,原来的提示照字面找不到文件。

* 统一德语版百炼流式语音识别的写法

同一份文件里 LLM 那行写的是 Alibaba Bailian,语音识别这行写成了 Aliyun Bailian,
同一家供应商两种拼法。改成跟 LLM 那行一样。

* 把音频交互测试工具的用法改成指向 README

原来在用法里内嵌装依赖的命令,句子太长。改成指向 main/digital-human 的 README,
依赖怎么装、模型从哪来,那份 README 里都写了。
2026-10-26 22:26:15 +08:00
hrz 7ab4cc5c90 把音频交互测试工具的用法改成指向 README
原来在用法里内嵌装依赖的命令,句子太长。改成指向 main/digital-human 的 README,
依赖怎么装、模型从哪来,那份 README 里都写了。
2026-10-26 21:13:51 +08:00
hrz 9c4790c8d0 统一德语版百炼流式语音识别的写法
同一份文件里 LLM 那行写的是 Alibaba Bailian,语音识别这行写成了 Aliyun Bailian,
同一家供应商两种拼法。改成跟 LLM 那行一样。
2026-10-26 19:00:26 +08:00
hrz 4fd51c6f7f 把性能测试脚本的提示写清文档在仓库根
脚本是在 main/xiaozhi-server 下执行的,那儿没有 docs 目录,文档在仓库根,原来的提示照字面找不到文件。
2026-10-26 18:46:47 +08:00
hrz e7e0a886bf 补上音频交互测试工具的前置步骤
digital-human 的说明里要求先装 wakeword 依赖再启动,README 里原来只写了启动命令,照着做会缺依赖。五个语言版的用法都补上这一步。
2026-10-26 18:46:40 +08:00
hrz 3ff080bb0d 补上阿里百炼和 LM Studio 并统一越南语品牌写法
ASR 那行中文只写阿里云,配置里阿里系有三条,另外四份也按各自写法补上百炼。LM Studio 走的是 openai 接口,仓里没有独立的供应器,所以并进 openai 那一行,免费平台那列不动。越南语版三处 iFlytek 统一成官方写法 iFLYTEK。
2026-10-26 18:46:37 +08:00
hrz ca8ff121d4 按代码实际情况修正五份 README 的平台清单和说明 (#3489)
* 修正 README 里过时的平台清单和漏掉的供应器

删掉服务端已经不存在的 GizwitsTTS、ACGNTTS、灵犀流式TTS,补上漏写的 CustomTTS、GroqASR、Qwen3ASRFlash 等。声纹一栏改成真正的 voiceprint-api 服务。管理后台的界面语种、插件的加载时机、部署表里的配置要求、音频测试工具的前置步骤,都按代码里的实际情况改写。

* 改掉性能测试脚本里写错的文档路径

脚本启动时提示的 doc/performance_testerer.md 找不到,真实文件是 docs/performance_tester.md。

* 把离线模型补进免费平台列并撤掉音频工具那处改动

两个离线模型走的是本机,不消耗付费接口,和列里已有的本地模型一样,所以补进免费平台列。音频交互测试工具要装 wakeword 依赖这一条,和另一处改同一张表的改动挨着,两边同时开必冲突,先从这次改动里拿掉,等那边合上再单独提。
2026-10-26 18:37:15 +08:00
hrz 9f63b255fb 修好四个语言版 README 里的图片路径 (#3488)
这四个文件在 docs/readme 目录下,图片却写成 docs/images/demoN.png,浏览器按 docs/readme 去找,全是 404。改成 ../images/demoN.png,跟西语版和其他图片的写法一致。四个文件各 15 处,只动路径,别的没碰。
2026-10-26 18:36:39 +08:00
hrz 9d34360a2c 把 README 里核心模块那一行改成不写数量 (#3487)
* 把 README 里核心模块的数量改成四个

模型响应测试工具那一行列了 ASR、LLM、VLLM、TTS 四个模块,却写着三个核心模块。中文源文和 en、de、pt_BR、vi 四个语言版统一改成四个,每份只动表示数量的那个词。

* 把核心模块那一行改成不写数量

先改成四个之后发现还是会错:performance_tester 目录下实际有六个脚本,脚本数还会随供应器增加,写死数量早晚再错一次。最终去掉数量词,改成「等核心模块」,五个语言版同步。
2026-10-26 18:35:27 +08:00
hrz 2f18e2ee0a Merge pull request #3486 from beeman/beeman/translate-readme
Add Spanish translation of README
2026-10-26 18:34:51 +08:00
hrz fd05dd5159 把西语 README 里核心模块的数量改成四个
模型响应测试工具那一行列了 ASR、LLM、VLLM、TTS 四个模块,却写着三个核心模块,这里改成四个。只动表示数量的那个词,句子的其余部分不动。
2026-10-26 17:38:20 +08:00
Sakura-RanChen 82d2acc78c - audio_to_data 缓存键加入文件 mtime:regen 覆写 wav 后旧缓存自动失效,
并在 regen 后预编码新文件,唤醒回复不再播旧音频且延迟收益保留
- enhanced_prompt 缓存键补上 emoji_enabled 与 prompt 哈希:
  同设备不同客户端(ESP32/digital-human)不再互相污染,改配置即时生效
- 删除 initialize 响应后的 tools/list 兜底补发:id=2 同连接复用
  违反 JSON-RPC 与设备端严格递增约定(#3461/#3485),且兜不住任何场景
2026-10-26 17:09:09 +08:00
Sakura-RanChen 77be288c8e Merge branch 'perf/hello-handshake-fast' of https://github.com/xinnan-tech/xiaozhi-esp32-server into perf/hello-handshake-fast
# Conflicts:
#	main/xiaozhi-server/requirements.txt
2026-10-26 17:04:54 +08:00
hrz ba22f5b765 把西语 README 里重复的一句改回中文
上一版把研发团队那一行的中文也译成了西语,于是两行说的是同一件事。仓库里另外四个语言版本都是本地语言一行加中文原句一行,这里改回中文行,跟它们保持一致,也去掉了重复。
2026-10-26 16:45:22 +08:00
hrz 0492dc5924 del:移除测试项 2026-10-26 16:36:46 +08:00
hrz ffd4ab74d7 修正西语 README 的术语和措辞
按评审意见改掉两处必改:HomeAssistant 不再拆成两个词,警告里的担保对象改回第三方服务商。另外统一了配置方案的说法,补上收费和大模型这两层意思,把 RAG 改成 Rag,漏译的那行中文也译了。只动这一个文件。
2026-10-26 16:26:44 +08:00
wengzh f0ebebfa85 Merge pull request #3485 from xinnan-tech/py-test-tool
Py test tool
2026-10-26 15:15:04 +08:00
Bram Borggreve 61136b958b Add Spanish translation of README
- Created docs/readme/README_es.md with Spanish translation
- Maintains same structure and links as original
- Provides Spanish-speaking users access to project documentation
- Adds an Español badge to the existing READMEs
2026-10-26 02:06:13 -05:00
Sakura-RanChen ee726f49e6 Merge remote-tracking branch 'origin/main' into perf/hello-handshake-fast
# Conflicts:
#	main/xiaozhi-server/requirements.txt
2026-10-26 15:05:17 +08:00
Sakura-RanChen 05f0da0639 补充接入点工具id调整 2026-10-26 14:40:31 +08:00
shinjiyuandshinjiyu af1982c424 fix: device MCP tools/call 避开握手占用的 JSON-RPC id 1/2 (#3462)
握手阶段写死 id=1 (initialize) / id=2 (tools/list),但 MCPClient.next_id
从 1 起算,导致前两次 tools/call 复用同一连接上的 request id。合规设备端
会拒收并触发服务端 30s 超时(表现为前两次调用失败、第三次起恢复)。

将 next_id 初始值改为 3,与官方协议序列及 #3461 复现一致;同时去掉
mcp_handler 里重复的 MCPClient 定义,改为从 mcp_client 导入。

Fixes #3461

Co-authored-by: shinjiyu <shinjiyu@users.noreply.github.com>
2026-10-26 14:05:40 +08:00
hrz 9bd668aebc Merge pull request #3449 from TXyy2023/codex/issue-3050-preserve-model-type
fix(manager-api): 保存模型时保留供应器登记的类型大小写
2026-10-23 22:17:36 +08:00
hrz d027432432 Merge pull request #3458 from monsterlag/fix/issue-3064-b736
fix(manager-web): 修复短屏下模型配置知识库入口被裁剪
2026-10-23 21:53:42 +08:00
hrz 3972df6e4e 加固模型配置导航的契约测试
原来的用例只取第一条样式规则,后面谁再补一条同名规则就能把滚动改回去,测试照样是绿的。这次把导航用到的样式选择器逐条列出来核对,外层容器也要求只能有一条规则,这样有人悄悄覆盖时测试会当场报错。
2026-10-23 13:39:19 +00:00
hrz 8f2dda09cf Bump to 0.9.8 (#3469) 2026-10-23 18:54:05 +08:00
hrz d4ff9c5c17 修掉通讯录的两处越权(读 + 写),并补上三处空指针 (#3460)
* 读取设备通讯录先验设备归属

登录用户传任意 MAC 就能读到别人设备通讯录里的备注和称呼。归属判断收进一个方法,读取通讯录、改别名、改权限都走它,不属于当前用户的设备一律按查不到处理。

* 通讯录存别名不再空指针

对方设备没登记时取备注会直接崩,整条权限授予请求跟着挂掉。取不到就退回尾号称呼,呼叫时主叫没登记也不再崩,顺手删掉一行调试打印。

* 解绑先验归属,通讯录别名不再落空

解绑接口原来不核对设备归属,谁知道对方的设备 id 就能把那台设备的通讯录清空,设备本身倒是因为带了用户条件没删掉。现在先看归属,不是自己的直接不动。

设备登记了但用户没起名时,别名会落成空,缓存刷新把整条跳过,对方再也呼不过来。改成和呼叫那处一样,两级判空都取不到才退回尾号称呼。顺带加了一条回填,把写进去的空值补上。

呼叫那条测试原来会真发请求也没验到东西,改成桩掉网关直接看发出去的称呼。

* 通讯录空别名兜底,撞名加后缀

缓存拼表之前,空别名先用尾号称呼顶上,不再整条跳过,用户没起名的设备也能被呼到。

昵称在同一个人的通讯录里是索引,两台撞名时后写的会盖掉先写的,按昵称呼会呼错设备。加后缀区分,规则和存表时那套一致。取名靠先后,而查询不带排序,所以先按地址排一遍,同样的数据每次刷新结果一样。

数据库没有排序保证,去重全靠顺序,所以这里先排再拼。迁移脚本不用动,运行时的兜底能管住它没覆盖到的行。

* 通讯录取名两条路径对上了

后缀原来只写进缓存,接口直查库返回原值,同一份数据给的是两个名字。
用户照控制台看到的名字呼叫,命中的不是同一台设备。

把取名抽成一个方法,两边都走一遍。顺带把权限位排到地址前面,
撞名时能呼的那台先拿原名,反过来用户怎么点都只会说没权限。

迁移取末两位时改成跟着小写化,和运行期算出的是同一个称呼。

* 同一个人被拆成两本通讯录,一台设备消失

两条记录的归属地址只差大小写时,会被算成两个人,各拿一个同名,
写进同一本时后者盖掉前者,一台设备就从缓存里消失了。

把归属地址归一抽成一个方法,算称呼、写缓存、查缓存都走它。

* 通讯录接口按最近改过的排前,这个约定有用例钉住

智控台页面是按对方地址把备注合进去的,接口返回顺序它并不关心,
所以现在这么排对界面没有效果。

用例留着是防它被当无用代码删掉,删了会有用例报出来。

* 迁移串到别人的私名,撞车的组改走尾号称呼

通讯录回填名字时,设备表里同一个 MAC 有两行(同一台被两个用户各登记一次),
MySQL 自己挑一行填进去,跟这行属于谁无关。改成只有一台时才取智控台名字,
多台就退回按 MAC 末两位算的称呼。名字照样填上,缓存照样刷新,对方还是呼得到。

补了 12 条在真库上跑的用例,另配一条排查重复 MAC 的语句。

* 去掉走不到的分支和没人用的桩,补上末尾换行

getUser() 取不到 subject 或 principal 时返回的是空对象,不是 null,
user == null 走不到,删掉;用例改成桩空对象,测真正的匿名那条路。
归属判断挪进 service 之后,controller 层四处 SecurityUser 的桩没人用了,删掉。
四个文件末尾少了换行,补上。另加一条用例钉住判重分组的比较规则:
只折叠大小写,不折叠分隔符。

* 移除回填通讯录别名的迁移,运行时已经兜住了

这个迁移是本 PR 带进来的。它要把空别名刷成称呼,可运行时的兜底已经做了一样的事,
而且覆盖得更宽:那边连纯空格、地址为空和不足两位的都管。删掉它,功能不受影响。

删它有两条理由。一是重复。二是它按 MAC 去设备表取名字,
而一个 MAC 在设备表里可能有多行——同一台设备被两个用户各登记一次就能造出来,
这时候取到的是别人给自己设备起的名字。写在这里,是怕后来的人以为这个坑不存在。

* 归属校验撞到重复 MAC 不再报错

库里同一 MAC 可能有两行只差大小写,selectOne 会抛 TooManyResultsException,把读通讯录接口整个带崩。改成 selectList 按 id 取第一条做归属校验,撞行时正常返回,不再报错。
2026-10-23 17:43:05 +08:00
wengzh 3709540efd Merge pull request #3445 from xinnan-tech/fix/huoshan-double-stream-4-bugs-y15
huoshan_double_stream 修 4 个会话协议与监听竞态 bug
2026-10-23 17:37:05 +08:00
wengzh 333050ae2e Merge pull request #3464 from xinnan-tech/chore/bump-deps-20261023
chore(deps): 批量升级 15 个依赖,适配 mcp 2.x
2026-10-23 17:05:05 +08:00
Sakura-RanChen 95ce49ffc9 删除 benchmark/ 下 3438~3441 回归验证脚本
四处修复均已验证收敛,脚本不进 CI、裸跑依赖 manager-api,保留仅为仓库噪声;后续若再动 huoshan_double_stream 时以 pytest 覆盖
2026-10-23 17:04:52 +08:00
Sakura-RanChen be75e7bd39 chore(deps): 批量升级 15 个依赖,适配 mcp 2.x
源 PR: #3426 #3427 #3428 #3429

剔除 4 项:
- edge-tts 7.3.1->7.2.8 是降级
- powermem >=1.1.7 要求 Python>=3.11
- openai 3.29.0 与 powermem 冲突(powermem 要求 openai<3.0.0)
- ruamel.yaml 0.19.1 与 hyperpyyaml 冲突

mcp 2.x 适配: streamablehttp_client 更名为 streamable_http_client,headers/timeout
改由 http_client 承载,产出从 3 元组变 2 元组,ProgressFnT 移到 mcp.shared.dispatcher
2026-10-23 15:15:50 +08:00
hrz 24fbf7eee2 登录 token 12h 静默失效补 silent refresh 接口 #3392 (#3402)
* 登录 token 12h 静默失效补 silent refresh 接口 #3392

方案 A:后端 POST /xiaozhi/user/refresh(anon),旧 token 校验通过后强制覆盖为新 token 字符串,旧 token 在 DB 查不到即失效。前端 httpRequest 在 sendRequest 入口检查 token 剩余有效期,剩 1h 内触发 silent refresh(独立 fly 实例 + 单飞锁),成功后 setToken 替换并自动继续发送当前请求,失败走 clearAuth 跳登录。

* refreshToken 改 CAS 防并发 race + 拆前端副作用 #3392

P1:SysUserTokenDao 新增 refreshTokenCAS,条件 UPDATE(oldToken + 未过期 + userId),受影响 0 行抛 TOKEN_INVALID;refreshToken 改调它,避免读-改-写 race。P2:refreshToken 加 @Transactional;httpRequest.getTokenExpireAt 内的 localStorage.setItem 挪到独立 persistTokenExpireAt,由 shouldRefresh 入口调用。

* getTokenExpireAt 改读 localStorage 与持久化同源 #3392

pr01 三改:原代码 getTokenExpireAt 读 store(state.token 缓存),与 persistTokenExpireAt 写 localStorage 不同源,导致 _localStoredAt 缺失 → NaN → shouldRefresh 永 false → silent refresh 永不触发。修复:getTokenExpireAt 改读 localStorage,persistTokenExpireAt 同步刷 store,保证两端一致。e2e 已验证:注入 expire=1800 token → reload → role-config mount 触发 sendRequest → Network 抓到 POST /user/refresh 200 → localStorage token 被替换为新 12h token(含 _localStoredAt)。

* 应用启动同步 localStorage token 到 store #3392

PR #3402 扩展:守卫放行后首次 API 请求时,vuex state.token 还是 null,httpRequest 读 store 取不到 Authorization 头,会被后端 401 → clearAuth 跳登录。在 main.js 应用启动阶段就把 localStorage 里的 token commit 进 store,避免 race。不动 refresh 链路与其它业务文件。
2026-10-23 09:47:41 +08:00
monsterlag 855a64cbb9 fix(manager-web): keep model navigation scrollable on short screens 2026-10-22 04:06:52 +00:00
hrzandSakura-RanChen fdf280f84a 模板 GET 入口迁回 AgentTemplateController #3395 (#3401)
* 模板 GET 入口迁回 AgentTemplateController #3395

根因:AgentController 已存在 GET /agent/template/templateList,AgentTemplateController 缺显式 GET,于是路由虽通却跨 controller 暴露模板数据,且职责错位。修复:删 AgentController.templateList,同名 @GetMapping(无 path)落到 AgentTemplateController,权限 normal、orderByAsc sort、复用现有 service.list。验证:spring-boot 启动期无 Ambiguous;/agent/template、/page、/{id} 三路由均被 Shiro 一致拦截,权限/路径正确。

* 补 P2 注释说明 normal 权限反直觉是有意保留 #3395

在 getAgentTemplates 上方加 2 行注释,说明单独 normal 权限是为保持原行为契约(普通用户也能查可用模板),与类级 superAdmin 故意分离;非疏忽。无逻辑变更。

* 去除未使用引用

---------

Co-authored-by: Sakura-RanChen <1908198662@qq.com>
2026-10-22 11:42:48 +08:00
wengzh 498191519c Merge pull request #3452 from xinnan-tech/py-test-wakeup
Py test wakeup
2026-10-22 11:08:38 +08:00
wengzh f614489d81 Merge pull request #3454 from xinnan-tech/web-test-json
Web test json
2026-10-20 15:44:16 +08:00
Sakura-RanChen 6f958ee4a6 fix: 新增模型页面dict字段错误解析 2026-10-20 15:34:40 +08:00
TXyy2023 b226ee5430 fix(manager-web): decode stored dictionaries before editing (#3450) 2026-10-20 11:53:02 +08:00
Sakura-RanChen 9c30b5e035 文件直接写入wav,无需经过二次编解码 2026-10-20 10:45:31 +08:00
TXyy2023 d531d04281 fix(tts): handle retained audio files in wakeup cache (#3451) 2026-10-20 10:00:20 +08:00
Sakura-RanChen d8f8a4a083 - 取消屏障保证:新连接创建前 _cancel_monitor_task 已等旧监听任务完全退出,
旧任务不可能读新连接或误关新连接(check_huoshan_3440 动态断言覆盖)
- mismatch 分支仅保留纯跳过,删除旧会话结束事件的复位遗留,
  activate_session 写入点收敛到会话生命周期(start / matched 结束 / close)
- 删除 _session_start_future 启动确认机制:start_session 发送即返回,
  SessionFailed 由 monitor 记 error 并复位 activate_session,无挂起风险
- 回归脚本同步调整:3438 重写为新行为断言,3440 用屏障断言替换 snapshot 断言
2026-10-19 17:52:25 +08:00
TXyy2023 37d4330224 test(model): verify registered types with MySQL and Redis 2026-10-18 23:13:39 +08:00
TXyy2023 7873984f9f fix(manager-api): preserve registered model type casing 2026-10-18 17:10:47 +08:00
hrz 6840210ea8 check_huoshan_3439 改 FIRST 分支行为断言
原 test 用正则 grep 源码验存在性,pr02 复评指出这样测不到实际调用。
改成起后台线程跑真 tts_text_priority_thread、构造 FIRST 消息入队、
run_coroutine_threadsafe patch 成立即完成的 future,最后断言
reset_stream_state 与 opus_encoder.reset_state 各被调一次。
2026-10-17 07:57:30 +08:00
hrz 0a54124d0d huoshan_double_stream 修 3 处复评反馈
init 漏写 _pending_prefix 字段、start_session 用已弃用的
get_event_loop、finish_session 留着多余的 getattr 兜底。补字段、
换 get_running_loop、去兜底,3 处都修了。
2026-10-17 07:57:26 +08:00
hrz ddc441bc59 补 huoshan_double_stream 4 bug 复现脚本
四个独立脚本各覆盖一个 issue:start_session 等 future 确认、句尾残留与编码器
冲刷、monitor snapshot 隔离、parser_response 四处协议解析。共享 _huoshan_mock
构造一个绕开父类 init 的假 provider 实例。
2026-10-17 07:39:51 +08:00
hrz 9b7fb3e826 parser_response 四处二进制协议解析修正
serialization_method 错用掩码当移位数恒 0,改成 >>4;message_type 写
== FULL or AUDIO_ONLY_RESPONSE 优先级下恒真,改 in (...);offset 硬编码 4 在
header_size>1 全错位,改成 header_size * 4;SessionCanceled 漏进协议分支被
else 误读,补进统一解析。
2026-10-17 07:39:51 +08:00
hrz a6cdbf9410 监听任务用 snapshot 隔离跨连接
旧 monitor 循环直接 self.ws.recv(),连接重建后旧任务读新下行;finally 无条件
关 self.ws 误关新连接。改成入口绑 monitor_ws / monitor_task,recv 用 snapshot;
旧 session 事件 continue 不再 reset 当前;finally 判 id 相同才关 ws / 清引用。
2026-10-17 07:39:51 +08:00
hrz 75fd5d8cc3 句尾替换词残留先发 + 会话结束冲刷编码器
旧 finish_session 不发 _pending_prefix 残留,句尾替换词被扣时整段丢;monitor / to_tts
收 SessionFinished 不调 is_end=True 冲刷 opus 编码器,每句固定丢尾。改成
finish_session 开头先发残留,monitor / to_tts 调 is_end=True 冲刷,FIRST 同步重置。
2026-10-17 07:39:51 +08:00
hrz 6fe5630346 TTS 会话等 SessionStarted 确认
旧逻辑 start_session 发完 StartSession 即返回;服务端鉴权失败或 14s 内 26005 时
文本线程继续给死会话发文本,整轮无声。改成末尾 await 一个 future:monitor 读到
SessionStarted set True、读到 SessionFailed set False,超时报错让文本线程感知。
2026-10-17 07:39:50 +08:00
hrz 4ed3197a6f pr02 P2 注释预热和百分位
首段缓冲帧数注释改成说人话版本;唤醒词预热按常见音色列表走并把吞掉的失败改成日志提示;基准脚本百分位改成线性插值与 numpy 一致;requirements 补上基准测试用到的 websocket-client。
2026-10-14 23:19:37 +08:00
hrz 4dd194ef8b pr02 P1 三条全部兜底
device_prompt 缓存跨日不失效导致今天日期/天气被烤死,改成独立 key 并带 5 分钟 ttl;mcp 工具列表在收到初始化响应后补回并发发送,helloHandle 预发万一丢帧时也能兜住;启服脚本去硬编码路径与全机 pkill,改用相对路径与 PID 文件。
2026-10-14 23:19:33 +08:00
hrz 8c41faac8b 握手基准脚本后台取帧贴近真实客户端
原版 recv 在二进制语音帧期间必须按顺序 drain 才能拿到 mcp 文本帧,测出来偏慢。改成后台线程持续把帧按到达时间入队、主循环按队列处理,模拟 digital-human 的异步 onmessage 行为。附 handshake_summary_real.txt 是方案 C 后 10 次跑出的数据。
2026-10-14 23:05:20 +08:00
hrz 597ab5f415 每连接冗余计算与日志降级
提示词模板结果按设备 id 缓存复用、声纹未配置和 mcp 配置文件缺失两条提示从警告降到调试,让每条新连接少跑几百毫秒的模板渲染、少两次同步日志。
2026-10-14 23:05:15 +08:00
hrz 4f1cb98503 hello 握手并发与缓存
唤醒词音频改为带缓存且进程启动时预热、首段流控缓冲帧数从 5 调到 2、hello 阶段并发发出 mcp 工具列表帧让客户端在听到声音前就准备好、删掉 mcp 初始化后的固定 sleep。砍幅 t_hello_to_tools_ready p50 从 4019ms 降到 219ms。
2026-10-14 23:05:12 +08:00
103 changed files with 4272 additions and 3661 deletions

No files matched your search

+49 -8
View File
@@ -20,24 +20,58 @@ jobs:
python-version: '3.10'
cache: 'pip'
cache-dependency-path: 'main/xiaozhi-server/requirements.txt'
- name: Install system dependencies
run: |
# opuslib_next 是 ctypes 绑定,pip 装上的是纯 Python 包,真正的编解码
# 靠系统里的 libopus.so。runner 上默认没有,import 会失败,音频相关的
# 用例直接报错。正式镜像 Dockerfile-server-base 里也是装 libopus0 的
sudo apt-get update
sudo apt-get install -y --no-install-recommends libopus0
- name: Install dependencies
working-directory: main/xiaozhi-server
run: |
pip install -r requirements.txt
pip install pytest pytest-asyncio freezegun
- name: Run smoke test
- name: Prepare test config
working-directory: main/xiaozhi-server
run: |
if [ -f tests/test_smoke.py ]; then
pytest tests/test_smoke.py -x -q
else
echo "::warning::Slice 2 not done yet — tests/test_smoke.py missing"
exit 0
fi
# config/settings.py 在 import 期就要这个文件,缺了整套用例连收集都过不去。
# data/ 在 .gitignore 里,只能由这一步现造
mkdir -p data
printf '{}\n' > data/.config.yaml
- name: Run tests
working-directory: main/xiaozhi-server
run: |
# Edge TTS 那条要连微软公网,runner 的出口 IP 会被风控挡掉,先排除。
# 根治办法是给它打 live 标记,让 mock 模式下自动不收(另开一条改)
pytest tests/ -v -k "not test_tts_text_to_speak_writes_file"
java:
name: Java (manager-api)
runs-on: ubuntu-latest
# 三个 @SpringBootTest 要真连 MySQL 和 Redis,空库即可,Liquibase 会自己建表
services:
mysql:
image: mysql:8.0
env:
MYSQL_ROOT_PASSWORD: 123456
MYSQL_DATABASE: xiaozhi_esp32_server
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping -h 127.0.0.1 -uroot -p123456"
--health-interval=10s
--health-timeout=5s
--health-retries=20
redis:
image: redis:8.0
ports:
- 6379:6379
options: >-
--health-cmd="redis-cli ping"
--health-interval=10s
--health-timeout=5s
--health-retries=20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
@@ -47,7 +81,12 @@ jobs:
cache: 'maven'
- name: Run tests
working-directory: main/manager-api
run: mvn -B -q test -DfailIfNoTests=false
run: mvn -B test -DskipTests=false -DfailIfNoTests=false
- name: Check the integration tests really ran
working-directory: main/manager-api
# 守卫在依赖不齐时会把整类集成测试跳过,这里盯一下它们有没有真跑,
# 免得门禁被悄悄掏空
run: bash scripts/check-integration-tests.sh
web:
name: Vue (manager-web)
@@ -63,4 +102,6 @@ jobs:
working-directory: main/manager-web
run: |
npm ci --no-audit --no-fund
npm run test:contract
npm run test:unit
npm run build
+22 -19
View File
@@ -22,6 +22,7 @@
<a href="./docs/readme/README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./docs/readme/README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
<a href="./docs/readme/README_pt_BR.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-DFE0E5"></a>
<a href="./docs/readme/README_es.md"><img alt="Español" src="https://img.shields.io/badge/Español-DFE0E5"></a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">
<img alt="GitHub Contributors" src="https://img.shields.io/github/v/release/xinnan-tech/xiaozhi-esp32-server?logo=docker" />
</a>
@@ -169,12 +170,12 @@ Spearheaded by Professor Siyuan Liu's Team (South China University of Technology
#### 🚀 部署方式选择
| 部署方式 | 特点 | 适用场景 | 部署文档 | 配置要求 | 视频教程 |
|---------|------|---------|---------|---------|---------|
| **最简化安装** | 智能对话、单智能体管理 | 低配置环境,数据存储在配置文件,无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 如果使用`FunASR`要2核4G,如果全API,要2核2G | - |
| **全模块安装** | 智能对话、多用户管理、多智能体管理、智控台界面操作 | 完整功能体验,数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 如果使用`FunASR`要4核8G,如果全API,要2核4G| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **最简化安装** | 智能对话、单智能体管理 | 低配置环境,数据存储在配置文件,无需数据库 | [①Docker版](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②源码部署](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 本地跑 `FunASR` 较耗内存,全部走 API 时资源占用低 | - |
| **全模块安装** | 智能对话、多用户管理、多智能体管理、智控台界面操作 | 完整功能体验,数据存储在数据库 |[①Docker版](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②源码部署](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③源码部署自动更新教程](./docs/dev-ops-integration.md) | 本地跑 `FunASR` 较耗内存,另需数据库资源,全部走 API 时资源占用低| [本地源码启动视频教程](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
常见问题及相关教程,可参考[这个链接](./docs/FAQ.md)
> 💡 提示:以下是按最新代码部署后的测试平台,有需要可烧录测试,并发为6个,每天会清空数据,
> 💡 提示:以下是按最新代码部署后的测试平台,有需要可烧录测试,并发为6个,每天会清空数据。
```
智控台地址: https://2662r3426b.vicp.fun
@@ -202,7 +203,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| VLLM(视觉大模型) | glm-4v-flash(智谱) | 👍qwen3.5-flash(阿里百炼) |
| TTS(语音合成) | EdgeTTS(微软) | 👍HuoshanDoubleStreamTTS(火山流式) |
| Intent(意图识别) | function_call(函数调用) | function_call(函数调用) |
| Memory(记忆功能) | mem_local_short(本地短期记忆) | mem_local_short(本地短期记忆) |
| Memory(记忆功能) | mem_local_short(本地短期记忆) | mem_local_short(本地短期记忆) |
如果您关心各组件的耗时,请查阅[小智各组件性能测试报告](https://github.com/xinnan-tech/xiaozhi-performance-research),可按报告中的测试方法在您的环境中实际测试。
@@ -211,8 +212,8 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 工具名称 | 位置 | 使用方法 | 功能说明 |
|:---:|:---|:---:|:---:|
| 音频交互测试工具 | main》digital-human》index.html | 在 `main/digital-human` 执行 `python start.py` 后访问 `http://127.0.0.1:8006/index.html` | 测试音频播放和接收功能,验证Python端音频处理是否正常 |
| 模型响应测试工具 | main》xiaozhi-server》performance_tester.py | 执行 `python performance_tester.py` | 测试ASR(语音识别)、LLM(大模型)、VLLM(视觉模型)、TTS(语音合成)三个核心模块的响应速度 |
| 音频交互测试工具 | main》digital-human》index.html | 在 `main/digital-human` 按其 README 装好依赖后执行 `python start.py`,然后访问 `http://127.0.0.1:8006/index.html` | 测试音频播放和接收功能,验证Python端音频处理是否正常 |
| 模型响应测试工具 | main》xiaozhi-server》performance_tester.py | 执行 `python performance_tester.py` | 测试ASR(语音识别)、LLM(大模型)、VLLM(视觉模型)、TTS(语音合成)等核心模块的响应速度 |
> 💡 提示:测试模型速度时,只会测试配置了密钥的模型。
@@ -230,12 +231,12 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 意图识别 | 支持外挂的大模型意图识别、大模型自主函数调用,提供插件化意图处理机制 |
| 记忆系统 | 支持本地短期记忆、mem0ai接口记忆、PowerMem智能记忆,具备记忆总结功能 |
| 知识库 | 支持RAGFlow知识库,让大模型判断需要调度知识库后再回答 |
| 工具调用 | 支持客户端IOT协议、客户MCP协议、服务端MCP协议、MCP接入点协议、自定义工具函数 |
| 工具调用 | 支持客户端IOT协议、客户端MCP协议、服务端MCP协议、MCP接入点协议、自定义工具函数 |
| 指令下发 | 依托MQTT协议,支持从智控台将MCP指令下发到ESP32设备 |
| 管理后台 | 提供Web管理界面,支持用户管理、系统配置和设备管理;界面支持中文简体、中文繁体、英文显示 |
| 测试工具 | 提供性能测试工具、视觉模型测试工具和音频交互测试工具 |
| 管理后台 | 提供Web管理界面,支持用户管理、系统配置和设备管理;界面支持中文简体、中文繁体、英文、德语、越南语、巴西葡萄牙语显示 |
| 测试工具 | 提供模型响应性能测试工具和音频交互测试工具 |
| 部署支持 | 支持Docker部署和本地部署,提供完整的配置文件管理 |
| 插件系统 | 支持功能插件扩展、自定义插件开发和插件热加载 |
| 插件系统 | 支持功能插件扩展、自定义插件开发;新增插件文件在下次设备连接时自动加载,改动已加载插件的代码需重启服务 |
### 正在开发 🚧
@@ -255,7 +256,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 使用方式 | 支持平台 | 免费平台 |
|:---:|:---:|:---:|
| openai 接口调用 | 阿里百炼、火山引擎、DeepSeek、智谱、Gemini、科大讯飞 | 智谱、Gemini |
| OpenAI 接口调用 | 阿里百炼、火山引擎、DeepSeek、智谱、Gemini、科大讯飞、LM Studio | 智谱、Gemini |
| ollama 接口调用 | Ollama | - |
| dify 接口调用 | Dify | - |
| fastgpt 接口调用 | Fastgpt | - |
@@ -263,7 +264,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| xinference 接口调用 | Xinference | - |
| homeassistant 接口调用 | HomeAssistant | - |
实际上,任何支持 openai 接口调用的 LLM 均可接入使用。
实际上,任何支持 OpenAI 接口调用的 LLM 均可接入使用。
---
@@ -271,9 +272,9 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 使用方式 | 支持平台 | 免费平台 |
|:---:|:---:|:---:|
| openai 接口调用 | 阿里百炼、智谱ChatGLMVLLM | 智谱ChatGLMVLLM |
| OpenAI 接口调用 | 阿里百炼、智谱ChatGLMVLLM | 智谱ChatGLMVLLM |
实际上,任何支持 openai 接口调用的 VLLM 均可接入使用。
实际上,任何支持 OpenAI 接口调用的 VLLM 均可接入使用。
---
@@ -281,8 +282,8 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 使用方式 | 支持平台 | 免费平台 |
|:---:|:---:|:---:|
| 接口调用 | EdgeTTS、科大讯飞、火山引擎、腾讯云、阿里云及百炼、CosyVoiceSiliconflow、TTS302AI、CozeCnTTS、GizwitsTTS、ACGNTTS、OpenAITTS、灵犀流式TTS、MinimaxTTS | 灵犀流式TTS、EdgeTTS、CosyVoiceSiliconflow(部分) |
| 本地服务 | FishSpeech、GPT_SOVITS_V2、GPT_SOVITS_V3、Index-TTS、PaddleSpeech | Index-TTS、PaddleSpeech、FishSpeech、GPT_SOVITS_V2、GPT_SOVITS_V3 |
| 接口调用 | EdgeTTS、科大讯飞、火山引擎、腾讯云、阿里云及百炼、CosyVoiceSiliconflow、TTS302AI、CozeCnTTS、CustomTTS、OpenAITTS、MinimaxTTS | EdgeTTS、CosyVoiceSiliconflow(部分) |
| 本地服务 | FishSpeech、GPT_SOVITS_V2、GPT_SOVITS_V3、IndexStreamTTS、PaddleSpeechTTS | IndexStreamTTS、PaddleSpeechTTS、FishSpeech、GPT_SOVITS_V2、GPT_SOVITS_V3 |
---
@@ -298,8 +299,8 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 使用方式 | 支持平台 | 免费平台 |
|:---:|:---:|:---:|
| 本地使用 | FunASR、SherpaASR | FunASR、SherpaASR |
| 接口调用 | FunASRServer、火山引擎、科大讯飞、腾讯云、阿里云、百度云、OpenAI ASR | FunASRServer |
| 本地使用 | FunASR、SherpaASR、SherpaParaformerASR、VoskASR | FunASR、SherpaASR、SherpaParaformerASR、VoskASR |
| 接口调用 | FunASRServer、火山引擎、科大讯飞、腾讯云、阿里云及百炼、百度云、OpenAI ASR、GroqASR、Qwen3ASRFlash | FunASRServer |
---
@@ -307,7 +308,9 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 使用方式 | 支持平台 | 免费平台 |
|:---:|:---:|:---:|
| 本地使用 | 3D-Speaker | 3D-Speaker |
| 接口调用 | 声纹识别服务(voiceprint-api) | - |
声纹识别需要单独部署 [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) 服务(基于 3D-Speaker),并依赖 MySQL,详见[声纹识别启用指南](./docs/voiceprint-integration.md)
---
File diff suppressed because it is too large. Load diff
+15 -17
View File
@@ -2,7 +2,7 @@
![请参考-最简化架构图](../docs/images/deploy1.png)
# 方式一:Docker只运行Server
`0.8.2`版本开始,本项目发行的docker镜像只支持`x86架构`,如果需要在`arm64架构`的CPU上部署,可按照[这个教程](docker-build.md)在本机编译`arm64的镜像`。
本项目发行的docker镜像同时提供`amd64`和`arm64`两种架构,可以直接拉取使用;如需自定义镜像,可按照[这个教程](docker-build.md)在本机编译。
## 1. 安装docker
@@ -36,9 +36,9 @@ xiaozhi-server
#### 1.1.3 下载配置文件
你需要下载两个配置文件:`docker-compose.yaml` 和 `config.yaml`。需要从项目仓库下载这两个文件。
你需要下载两个配置文件:`docker-compose.yml` 和 `config.yaml`。需要从项目仓库下载这两个文件。
##### 1.1.3.1 下载 docker-compose.yaml
##### 1.1.3.1 下载 docker-compose.yml
用浏览器打开[这个链接](../main/xiaozhi-server/docker-compose.yml)。
@@ -91,25 +91,22 @@ docker logs -f xiaozhi-esp32-server
这时,你就要留意日志信息,可以根据这个教程,判断是否成功了。[跳转到运行状态确认](#运行状态确认)
## 5. 版本升级操作
## 4. 版本升级操作
如果后期想升级版本,可以这么操作
5.1、备份好`data`文件夹中的`.config.yaml`文件,一些关键的配置到时复制到新的`.config.yaml`文件里。
4.1、备份好`data`文件夹中的`.config.yaml`文件,一些关键的配置到时复制到新的`.config.yaml`文件里。
请注意是对关键密钥逐个复制,不要直接覆盖。因为新的`.config.yaml`文件可能有一些新的配置项,旧的`.config.yaml`文件不一定有。
5.2、执行以下命令
4.2、执行以下命令
```
docker stop xiaozhi-esp32-server
docker rm xiaozhi-esp32-server
docker stop xiaozhi-esp32-server-web
docker rm xiaozhi-esp32-server-web
docker rmi ghcr.nju.edu.cn/xinnan-tech/xiaozhi-esp32-server:server_latest
docker rmi ghcr.nju.edu.cn/xinnan-tech/xiaozhi-esp32-server:web_latest
```
5.3、重新按docker方式部署
4.3、重新按docker方式部署
# 方式二:本地源码只运行Server
@@ -213,14 +210,14 @@ prompt: |
请你像一个人一样说话,请勿返回配置xml及其他特殊字符。
selected_module:
LLM: DoubaoLLM
LLM: ChatGLMLLM
LLM:
ChatGLMLLM:
api_key: xxxxxxxxxxxxxxx.xxxxxx
```
建议先将最简单的配置运行起来,然后再去`xiaozhi/config.yaml`阅读配置的使用说明。
建议先将最简单的配置运行起来,然后再去[`../main/xiaozhi-server/config.yaml`](../main/xiaozhi-server/config.yaml)阅读配置的使用说明。
比如你要换更换模型,修改`selected_module`下的配置就行。
## 模型文件
@@ -238,11 +235,12 @@ LLM:
如果你能看到,类似以下日志,则是本项目服务启动成功的标志。
```
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-OTA接口是 http://192.168.4.123:8003/xiaozhi/ota/
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-Websocket地址是 ws://192.168.4.123:8000/xiaozhi/v1/
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-=======================================================
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-OTA接口是 http://192.168.4.123:8003/xiaozhi/ota/
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-视觉分析接口是 http://192.168.4.123:8003/mcp/vision/explain
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-Websocket地址是 ws://192.168.4.123:8000/xiaozhi/v1/
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
250427 13:04:20[0.9.8_00000000000000][__main__]-INFO-=============================================================
```
正常来说,如果您是通过源码运行本项目,日志会有你的接口地址信息。
+3 -1
View File
@@ -144,5 +144,7 @@
### Q: 我叫他呼叫"张山",但是他老是识别成"张三",怎么办?
- 可以查阅你使用的asr服务的文档,确认是否支持热词识别。
- 如果你用的是`FunASRServer`,可以在容器里的`热词文件`里添加"张山",然后重启容器。
- 如果你用的是`火山引擎`的服务,可以在`火山引擎的控制台`里添加`热词文件`,然后回到智控台的`模型配置页面`,把`热词文件名称`配置在`火山引擎的tts`上去。
- 如果你用的是`火山引擎`的服务,可以在`火山引擎的控制台`里添加`热词文件`,然后回到智控台的`模型配置`页面,在`语音识别`里找到模型名称为`豆包语音识别(流式)`的那一条,点`修改`,弹窗里的`供应器`是`火山引擎语音识别(流式)`,在弹窗的`调用信息`区域把`热词文件名称`填到对应的输入框,点`确认保存`。
- `豆包语音识别2.0(流式)`(供应器是`火山引擎语音识别(流式)`)和`豆包语音识别`(供应器是`火山引擎语音识别`)这两条也都有`热词文件名称`这个字段,用法一样,配置时注意别选错行。
- 改完要让设备重新连接才会生效:设备连上时才会把最新的私有配置拉下来,一直连着的设备不会自动变。
+40 -8
View File
@@ -50,10 +50,11 @@ paddlespeech tts --input "你好,这是一次测试"
### 7.修改tts_online_application.yaml配置
参考目录 ```"PaddleSpeech\demos\streaming_tts_server\conf\tts_online_application.yaml"```
Linux / WSL 2 用正斜杠写法:```"PaddleSpeech/demos/streaming_tts_server/conf/tts_online_application.yaml"```
选择```tts_online_application.yaml```文件用编辑器打开,设置```protocol```为```websocket```
### 8.启动服务
```yaml
```bash
paddlespeech_server start --config_file ./demos/streaming_tts_server/conf/tts_online_application.yaml
#官方默认启动命令:
paddlespeech_server start --config_file ./conf/tts_online_application.yaml
@@ -68,10 +69,19 @@ INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8092 (Press CTRL+C to quit)
```
## 三、修改小智的配置文件
### 1.```main/xiaozhi-server/core/providers/tts/paddle_speech.py```
## 三、配置小智服务
### 1. 无需修改代码(provider 已内置于 main/xiaozhi-server/core/providers/tts/paddle_speech.py)
### 2.新建 `main/xiaozhi-server/data/.config.yaml`
`main/xiaozhi-server/data/` 目录不在仓库里,是被 `.gitignore` 忽略掉的,所以第一次用要先把它建出来,再在里面新建 `.config.yaml` 文件(在仓库根目录执行,上面的 `cd PaddleSpeech` 之后当前目录还停在 PaddleSpeech 源码目录里):
```bash
mkdir -p main/xiaozhi-server/data
```
这个文件是必需的,少了它小智服务起不来,会直接报错退出。
### 2.```main/xiaozhi-server/data/.config.yaml```
使用单模块部署
```yaml
selected_module:
@@ -79,19 +89,41 @@ selected_module:
TTS:
PaddleSpeechTTS:
type: paddle_speech
protocol: websocket
protocol: websocket # 当前仅支持 websocket
url: ws://127.0.0.1:8092/paddlespeech/tts/streaming # TTS 服务的 URL 地址,指向本地服务器 [websocket默认ws://127.0.0.1:8092/paddlespeech/tts/streaming]
spk_id: 0 # 发音人 ID,0 通常表示默认的发音人
sample_rate: 24000 # 采样率 [websocket默认24000,http默认0 自动选择]
speed: 1.0 # 语速,1.0 表示正常语速,>1 表示加快,<1 表示减慢
volume: 1.0 # 音量,1.0 表示正常音量,>1 表示增大,<1 表示减小
save_path: # 保存路径
```
采样率固定 24000,provider 内部写死,配置里不用写这一项。
### 3.启动xiaozhi服务
```py
小智服务用**另一个** Python 环境,不要用前面给 PaddleSpeech 建的 `paddle_env` —— 那个环境里装的是 PaddleSpeech 自己那套依赖,版本由它决定,跟本项目锁定的版本不一定一致(`paddlespeech` 对 numpy 没写版本上限,装进去的 numpy 未必是本项目要的那个)。本项目推荐的 Python 版本和依赖版本写在 `main/xiaozhi-server/requirements.txt` 里(文件开头写着推荐 python3.10,另外锁了 torch 2.2.2、numpy 1.26.4)。
先建环境、装依赖(在仓库根目录执行):
```bash
conda create -n xiaozhi-esp32-server python=3.10 -y
conda activate xiaozhi-esp32-server
cd main/xiaozhi-server
pip install -r requirements.txt
```
然后在 `main/xiaozhi-server` 目录下启动(目录不能换:provider 是按相对路径 `core/providers/tts/` 找的,见 `core/utils/tts.py:35`):
```bash
# 上一段已经 cd 到 main/xiaozhi-server 了,这段不用再 cd
# 如果是另开的新终端,先 cd main/xiaozhi-server 再执行
python app.py
```
启动`main/digital-human`下的`python start.py`后,打开`http://127.0.0.1:8006/index.html`,测试连接和发送消息时paddlespeech端是否有输出日志
要验证语音合成的实际输出,另开一个终端,在仓库根目录下跑数字人页面(这套代码另有一份依赖清单,在 `main/digital-human/wakeword_runtime/requirements.txt` 里,缺了会报缺包;唤醒词那部分还要 numpy —— `core/detector.py:7` 和 `core/microphone.py:4` 在模块顶层就 import numpy,用上面建的 `xiaozhi-esp32-server` 跑的话 numpy 已经在里面,不会有这个问题):
```bash
cd main/digital-human
python start.py
```
打开`http://127.0.0.1:8006/index.html`,测试连接和发送消息时paddlespeech端是否有输出日志
输出日志参考:
```
+48 -3
View File
@@ -1,8 +1,15 @@
# 语音识别、大语言模型、非流式语音合成、流式语音合成、视觉模型的性能测试工具使用指南
# 语音识别、流式语音识别、大语言模型、非流式语音合成、流式语音合成、视觉模型的性能测试工具使用指南
0.先装依赖(在仓库根目录执行)
```
cd main/xiaozhi-server
pip install -r requirements.txt
```
测试脚本要 import `yaml`、`aiohttp`、`tabulate`,少一个都起不来。`tabulate` 不在 requirements.txt 里,是 `edge-tts` 间接带进来的。
1.在main/xiaozhi-server目录下创建data目录
2.在data目录下创建.config.yaml文件
3.在.data/config.yaml中,写入你的语音识别、大语言模型、流式语音合成、视觉模型的参数
3.在data/.config.yaml中,写入你的语音识别、流式语音识别、大语言模型、非流式语音合成、流式语音合成、视觉模型的参数
例如:
```
LLM:
@@ -16,12 +23,50 @@ LLM:
api_key: 你的chat-glm web key
TTS:
EdgeTTS:
# 定义TTS API类型
type: edge
voice: zh-CN-XiaoxiaoNeural
VLLM:
ChatGLMVLLM:
# 定义视觉模型API类型
type: openai
# glm-4v-flash 是智谱的视觉模型,需要在智谱平台申请api_key
model_name: glm-4v-flash
url: https://open.bigmodel.cn/api/paas/v4/
api_key: 你的api_key
ASR:
FunASR:
type: fun_local
model_dir: models/SenseVoiceSmall
language: auto
```
4.在main/xiaozhi-server目录下运行performance_tester.py:
```
python performance_tester.py
```
```
## 几点说明
- 脚本第一行提示的 `docs/performance_tester.md` 就是本文档,它在仓库根目录的 `docs/` 下;脚本本身在 `main/xiaozhi-server/` 目录里跑,两处不是一个目录。
- LLM、VLLM、非流式语音合成这三项会把 `config.yaml` 和 `data/.config.yaml` 两份配置合并起来再遍历,所以实际会测**所有配好 api_key 的供应器**,不只是你在 `.config.yaml` 里写的那一个;api_key 还是占位符的会被跳过。
- 非流式的语音识别这一项读配置的方式和别的不一样:它自己扫 `data/` 目录下的 `*.config.yaml`,只取其中的 `ASR` 段,**不会**用 `config.yaml` 里的默认值。想测语音识别,就得把供应器写进 `data/.config.yaml`。
- 流式语音识别读的是合并后的配置,固定测 `DoubaoStreamASR`、`Qwen3ASRFlash`、`XunfeiStreamASR` 这三家。`config.yaml` 里这三项带的 key 是占位符,要在 `data/.config.yaml` 的 `ASR` 段里填上自己的 key 才会测出延迟。
- 视觉模型的配置写在 `data/.config.yaml` 的 `VLLM:` 段里,这段**不能只写一个空键**:写空的话合并后整段是空的,这项测试会直接跳过、什么都不测。
- 流式语音合成一次会跑这些供应器:
| 供应器 | 说明 |
|---|---|
| `AliyunStreamTTS` | 阿里云流式 TTS,无条件调用 |
| `HuoshanDoubleStreamTTS` | 火山引擎流式 TTS,无条件调用 |
| `PaddleSpeechTTS` | PaddleSpeech 流式 TTS,无条件调用 |
| `IndexStreamTTS` | IndexStream 流式 TTS,无条件调用 |
| `LinkeraiTTS` | `config.yaml` 的 TTS 段里没有这个供应器,默认不测、整项跳过;要测得自己在 `.config.yaml` 里补上 |
| `AliBLTTS` | 阿里百炼流式 TTS,默认就会测:判断读的是合并后的配置,`config.yaml` 里本来就带着这一项 |
| `XunFeiTTS` | 讯飞流式 TTS,同上,默认就会测 |
`AliBLTTS` 和 `XunFeiTTS` 这两项默认带的 key 是占位符,不在 `.config.yaml` 里换成自己的 key 就会报失败;想直接跳过,把这两项在 `.config.yaml` 里写成空的(键名下面不填内容)就行。
标了「无条件调用」的这几个(`AliyunStreamTTS`、`HuoshanDoubleStreamTTS`、`PaddleSpeechTTS`、`IndexStreamTTS`),供应器缺失或被清空时,那一项会报错,错误被测试方法内部的异常处理吃掉,这一项被记成失败 —— 结果表里会多出一行失败记录,但不影响其它项。带守卫的那几家(`AliBLTTS`、`XunFeiTTS`、`LinkeraiTTS`)是整项跳过:没配就不报错、结果表里也不会有这一行。
+39 -37
View File
@@ -6,7 +6,7 @@
Dieses Projekt basiert auf der Theorie und Technologie der Mensch-Maschine-symbiotischen Intelligenz zur Entwicklung intelligenter Terminal-Hardware- und Software-Systeme<br/>und bietet Backend-Dienste für das Open-Source-Hardware-Projekt
<a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a><br/>
Implementiert mit Python, Java und Vue gemäß dem <a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">Xiaozhi-Kommunikationsprotokoll</a><br/>
Unterstützt MQTT+UDP-Protokoll, Websocket-Protokoll, MCP-Endpunkte und Stimmabdruckerkennung
Unterstützt MQTT+UDP-Protokoll, Websocket-Protokoll, MCP-Zugriffspunkte, Stimmabdruckerkennung und Wissensdatenbank
</p>
<p align="center">
@@ -22,6 +22,7 @@ Unterstützt MQTT+UDP-Protokoll, Websocket-Protokoll, MCP-Endpunkte und Stimmabd
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DBEDFA"></a>
<a href="./README_pt_BR.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-DFE0E5"></a>
<a href="./README_es.md"><img alt="Español" src="https://img.shields.io/badge/Español-DFE0E5"></a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">
<img alt="GitHub Contributors" src="https://img.shields.io/github/v/release/xinnan-tech/xiaozhi-esp32-server?logo=docker" />
</a>
@@ -54,31 +55,31 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
<img alt="Das Gefühl der Reaktionsgeschwindigkeit" src="../images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
<img alt="Das Geheimnis der Geschwindigkeitsoptimierung" src="../images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
<img alt="Xiaozhi-Digitalmensch mit Sprachaktivierung" src="../images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
<img alt="Anruf von Gerät zu Gerät" src="../images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
<img alt="Komplexes medizinisches Szenario" src="../images/demo1.png" /></picture>
</a>
</td>
</tr>
@@ -86,31 +87,31 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
<img alt="Musik abspielen, Wetter abfragen und Nachrichten vorlesen" src="../images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
<img alt="Unterbrechung in Echtzeit" src="../images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
<img alt="Objekte per Foto erkennen" src="../images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
<img alt="Haushaltsgeräte ein- und ausschalten" src="../images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
<img alt="Aufgaben mit mehreren Befehlen" src="../images/demo11.png" /></picture>
</a>
</td>
</tr>
@@ -118,31 +119,31 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
<img alt="MCP-Zugriffspunkt" src="../images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
<img alt="MQTT-Befehlsübermittlung" src="../images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
<img alt="Stimmabdruckerkennung" src="../images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
<img alt="Eigene Stimmfarbe" src="../images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
<img alt="Gespräch auf Kantonesisch" src="../images/demo3.png" /></picture>
</a>
</td>
</tr>
@@ -167,8 +168,8 @@ Dieses Projekt bietet zwei Bereitstellungsmethoden. Bitte wählen Sie basierend
#### 🚀 Auswahl der Bereitstellungsmethode
| Bereitstellungsmethode | Funktionen | Anwendungsszenarien | Deployment-Dokumente | Konfigurationsanforderungen | Video-Tutorials |
|---------|------|---------|---------|---------|---------|
| **Vereinfachte Installation** | Intelligenter Dialog, Einzel-Agenten-Verwaltung | Umgebungen mit geringer Konfiguration, Daten in Konfigurationsdateien gespeichert, keine Datenbank erforderlich | [①Docker-Version](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Quellcode-Deployment](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 Kerne 4GB bei Verwendung von `FunASR`, 2 Kerne 2GB bei allen APIs | - |
| **Vollständige Modulinstallation** | Intelligenter Dialog, Mehrbenutzerverwaltung, Mehr-Agenten-Verwaltung, Intelligente Steuerkonsole-Bedienung | Vollständige Funktionserfahrung, Daten in Datenbank gespeichert |[①Docker-Version](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Quellcode-Deployment](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Quellcode-Deployment Auto-Update-Tutorial](../dev-ops-integration.md) | 4 Kerne 8GB bei Verwendung von `FunASR`, 2 Kerne 4GB bei allen APIs| [Video-Tutorial für lokalen Quellcode-Start](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **Vereinfachte Installation** | Intelligenter Dialog, Einzel-Agenten-Verwaltung | Umgebungen mit geringer Konfiguration, Daten in Konfigurationsdateien gespeichert, keine Datenbank erforderlich | [①Docker-Version](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Quellcode-Deployment](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| Lokales `FunASR` benötigt mehr Arbeitsspeicher; bei durchgängiger API-Nutzung ist der Ressourcenverbrauch gering | - |
| **Vollständige Modulinstallation** | Intelligenter Dialog, Mehrbenutzerverwaltung, Mehr-Agenten-Verwaltung, Intelligente Steuerkonsole-Bedienung | Vollständige Funktionserfahrung, Daten in Datenbank gespeichert |[①Docker-Version](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Quellcode-Deployment](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Quellcode-Deployment Auto-Update-Tutorial](../dev-ops-integration.md) | Lokales `FunASR` benötigt mehr Arbeitsspeicher, zusätzlich werden Datenbankressourcen benötigt; bei durchgängiger API-Nutzung ist der Ressourcenverbrauch gering| [Video-Tutorial für lokalen Quellcode-Start](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
Häufige Fragen und entsprechende Tutorials finden Sie unter [diesem Link](../FAQ.md)
@@ -209,15 +210,15 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
| Werkzeugname | Standort | Verwendungsmethode | Funktionsbeschreibung |
|:---:|:---|:---:|:---:|
| Audio-Interaktionstesttool | main》digital-human》index.html | In `main/digital-human` `python start.py` ausführen und dann `http://127.0.0.1:8006/index.html` öffnen | Testet Audio-Wiedergabe- und Empfangsfunktionen, überprüft, ob die Python-seitige Audioverarbeitung normal ist |
| Modell-Reaktionstesttool | main》xiaozhi-server》performance_tester.py | Ausführen `python performance_tester.py` | Testet die Reaktionsgeschwindigkeit von ASR (Spracherkennung), LLM (großes Modell), VLLM (Vision-Modell), TTS (Sprachsynthese) drei Kernmodulen |
| Audio-Interaktionstesttool | main》digital-human》index.html | In `main/digital-human` zuerst die Abhängigkeiten wie in der README installieren, dann `python start.py` ausführen und `http://127.0.0.1:8006/index.html` öffnen | Testet Audio-Wiedergabe- und Empfangsfunktionen, überprüft, ob die Python-seitige Audioverarbeitung normal ist |
| Modell-Reaktionstesttool | main》xiaozhi-server》performance_tester.py | Ausführen `python performance_tester.py` | Testet die Reaktionsgeschwindigkeit von Kernmodulen wie ASR (Spracherkennung), LLM (großes Modell), VLLM (Vision-Modell), TTS (Sprachsynthese) |
> 💡 Hinweis: Beim Testen der Modellgeschwindigkeit werden nur Modelle mit konfigurierten Schlüsseln getestet.
---
## Funktionsliste ✨
### Implementiert ✅
![请参考-全模块安装架构图](../images/deploy2.png)
![Architekturdiagramm der Installation aller Module](../images/deploy2.png)
| Funktionsmodul | Beschreibung |
|:---:|:---|
| Kernarchitektur | Basierend auf [MQTT+UDP-Gateway](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket und HTTP-Servern, bietet vollständiges Konsolenverwaltungs- und Authentifizierungssystem |
@@ -225,15 +226,15 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
| Stimmabdruckerkennung | Unterstützt Mehrbenutzer-Stimmabdruckregistrierung, -verwaltung und -erkennung, verarbeitet parallel mit ASR, Echtzeit-Sprecheridentitätserkennung und Weitergabe an LLM für personalisierte Antworten |
| Intelligenter Dialog | Unterstützt mehrere LLM (große Sprachmodelle), implementiert intelligenten Dialog |
| Visuelle Wahrnehmung | Unterstützt mehrere VLLM (Vision Large Models), implementiert multimodale Interaktion |
| Absichtserkennung | Unterstützt LLM-Absichtserkennung, Function Call-Funktionsaufruf, bietet plugin-basierten Absichtsverarbeitungsmechanismus |
| Absichtserkennung | Unterstützt externe LLM-Absichtserkennung, autonome Funktionsaufrufe durch große Modelle, bietet plugin-basierten Absichtsverarbeitungsmechanismus |
| Gedächtnissystem | Unterstützt lokales Kurzzeitgedächtnis, mem0ai-Schnittstellengedächtnis, PowerMem intelligentes Gedächtnis, mit Gedächtniszusammenfassungsfunktion |
| Wissensdatenbank | Unterstützt RAGFlow-Wissensdatenbank, ermöglicht großem Modell die Bewertung, ob Wissensdatenbank benötigt wird, bevor geantwortet wird |
| Werkzeugaufruf | Unterstützt Client-IOT-Protokoll, Client-MCP-Protokoll, Server-MCP-Protokoll, MCP-Endpunktprotokoll, benutzerdefinierte Werkzeugfunktionen |
| Werkzeugaufruf | Unterstützt Client-IOT-Protokoll, Client-MCP-Protokoll, Server-MCP-Protokoll, MCP-Zugriffspunktprotokoll, benutzerdefinierte Werkzeugfunktionen |
| Befehlsübermittlung | Basierend auf MQTT-Protokoll, unterstützt die Übermittlung von MCP-Befehlen von der intelligenten Steuerkonsole an ESP32-Geräte |
| Verwaltungs-Backend | Bietet Web-Verwaltungsoberfläche, unterstützt Benutzerverwaltung, Systemkonfiguration und Geräteverwaltung; Oberfläche unterstützt vereinfachtes Chinesisch, traditionelles Chinesisch und englische Anzeige |
| Testwerkzeuge | Bietet Leistungstestwerkzeuge, Vision-Modell-Testwerkzeuge und Audio-Interaktionstestwerkzeuge |
| Verwaltungs-Backend | Bietet Web-Verwaltungsoberfläche, unterstützt Benutzerverwaltung, Systemkonfiguration und Geräteverwaltung; Oberfläche unterstützt vereinfachtes Chinesisch, traditionelles Chinesisch, Englisch, Deutsch, Vietnamesisch und brasilianisches Portugiesisch an |
| Testwerkzeuge | Bietet Leistungstestwerkzeuge für Modellantworten und Audio-Interaktionstestwerkzeuge |
| Deployment-Unterstützung | Unterstützt Docker-Deployment und lokales Deployment, bietet vollständige Konfigurationsdateiverwaltung |
| Plugin-System | Unterstützt funktionale Plugin-Erweiterungen, benutzerdefinierte Plugin-Entwicklung und Plugin-Hot-Loading |
| Plugin-System | Unterstützt funktionale Plugin-Erweiterungen und benutzerdefinierte Plugin-Entwicklung; neu hinzugefügte Plugin-Dateien werden bei der nächsten Geräteverbindung geladen, Änderungen an bereits geladenen Plugins erfordern einen Neustart des Dienstes |
### In Entwicklung 🚧
@@ -253,7 +254,7 @@ Xiaozhi ist ein Ökosystem. Wenn Sie dieses Produkt verwenden, können Sie sich
| Verwendungsmethode | Unterstützte Plattformen | Kostenlose Plattformen |
|:---:|:---:|:---:|
| OpenAI-Schnittstellenaufrufe | Alibaba Bailian, Volcano Engine Doubao, DeepSeek, Zhipu ChatGLM, Gemini | Zhipu ChatGLM, Gemini |
| OpenAI-Schnittstellenaufrufe | Alibaba Bailian, Volcano Engine Doubao, Volcano Engine AI Gateway, DeepSeek, Zhipu ChatGLM, Gemini, iFLYTEK, LM Studio | Zhipu ChatGLM, Gemini |
| Ollama-Schnittstellenaufrufe | Ollama | - |
| Dify-Schnittstellenaufrufe | Dify | - |
| FastGPT-Schnittstellenaufrufe | FastGPT | - |
@@ -279,8 +280,8 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int
| Verwendungsmethode | Unterstützte Plattformen | Kostenlose Plattformen |
|:---:|:---:|:---:|
| Schnittstellenaufrufe | EdgeTTS, Volcano Engine Doubao TTS, Tencent Cloud, Alibaba Cloud TTS, AliYun Stream TTS, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, GizwitsTTS, ACGNTTS, OpenAITTS, Lingxi Streaming TTS, MinimaxTTS, Volcano Dual-Stream TTS | Lingxi Streaming TTS, EdgeTTS, CosyVoiceSiliconflow (teilweise) |
| Lokale Dienste | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, Index-TTS, PaddleSpeech | Index-TTS, PaddleSpeech, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
| Schnittstellenaufrufe | EdgeTTS, iFLYTEK, Volcano Engine Doubao TTS, Volcano Engine AI Gateway TTS, Tencent Cloud, Alibaba Cloud TTS, AliYun Stream TTS, Alibaba Bailian CosyVoice TTS, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS, Volcano Dual-Stream TTS | EdgeTTS, CosyVoiceSiliconflow (teilweise) |
| Lokale Dienste | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, IndexStreamTTS, PaddleSpeechTTS | IndexStreamTTS, PaddleSpeechTTS, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
---
@@ -296,8 +297,8 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int
| Verwendungsmethode | Unterstützte Plattformen | Kostenlose Plattformen |
|:---:|:---:|:---:|
| Lokale Verwendung | FunASR, SherpaASR | FunASR, SherpaASR |
| Schnittstellenaufrufe | DoubaoASR, Doubao Streaming ASR, FunASRServer, TencentASR, AliyunASR, Aliyun Streaming ASR, Baidu ASR, OpenAI ASR | FunASRServer |
| Lokale Verwendung | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
| Schnittstellenaufrufe | DoubaoASR, Doubao Streaming ASR, Doubao Streaming ASR V2, FunASRServer, iFLYTEK, TencentASR, AliyunASR, Aliyun Streaming ASR, Alibaba Bailian Streaming ASR, Baidu ASR, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
---
@@ -305,7 +306,9 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int
| Verwendungsmethode | Unterstützte Plattformen | Kostenlose Plattformen |
|:---:|:---:|:---:|
| Lokale Verwendung | 3D-Speaker | 3D-Speaker |
| Schnittstellenaufrufe | Voiceprint-Dienst (voiceprint-api) | - |
Die Stimmabdruckerkennung erfordert einen separat bereitgestellten [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api)-Dienst (auf Basis von 3D-Speaker) sowie MySQL. Siehe [Anleitung zur Stimmabdruckerkennung](../voiceprint-integration.md).
---
@@ -351,11 +354,10 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int
| <img src="../images/logo_contributors.png" width="160"> | [Code-Mitwirkende](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Dank an [alle Code-Mitwirkenden](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), Ihre Bemühungen haben das Projekt robuster und leistungsfähiger gemacht. |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
</picture>
</a>
+38 -36
View File
@@ -22,6 +22,7 @@ Support for MQTT+UDP protocol, Websocket protocol, MCP access point, voiceprint
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
<a href="./README_pt_BR.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-DFE0E5"></a>
<a href="./README_es.md"><img alt="Español" src="https://img.shields.io/badge/Español-DFE0E5"></a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">
<img alt="GitHub Contributors" src="https://img.shields.io/github/v/release/xinnan-tech/xiaozhi-esp32-server?logo=docker" />
</a>
@@ -54,31 +55,31 @@ Want to see the usage effects? Click the videos below 🎥
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
<img alt="The feel of the response speed" src="../images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
<img alt="The secret to speeding things up" src="../images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
<img alt="Xiaozhi digital human with voice wake-up" src="../images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
<img alt="Device-to-device calling" src="../images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
<img alt="A complex medical scenario" src="../images/demo1.png" /></picture>
</a>
</td>
</tr>
@@ -86,31 +87,31 @@ Want to see the usage effects? Click the videos below 🎥
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
<img alt="Play music, check the weather, and read the news" src="../images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
<img alt="Real-time interruption" src="../images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
<img alt="Take a photo to identify objects" src="../images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
<img alt="Turn home appliances on and off" src="../images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
<img alt="Multi-command tasks" src="../images/demo11.png" /></picture>
</a>
</td>
</tr>
@@ -118,31 +119,31 @@ Want to see the usage effects? Click the videos below 🎥
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
<img alt="MCP access point" src="../images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
<img alt="MQTT command delivery" src="../images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
<img alt="Voiceprint recognition" src="../images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
<img alt="Custom voice timbre" src="../images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
<img alt="Talking in Cantonese" src="../images/demo3.png" /></picture>
</a>
</td>
</tr>
@@ -167,8 +168,8 @@ This project provides two deployment methods. Please choose based on your specif
#### 🚀 Deployment Method Selection
| Deployment Method | Features | Applicable Scenarios | Deployment Docs | Configuration Requirements | Video Tutorials |
|---------|------|---------|---------|---------|---------|
| **Simplified Installation** | Intelligent dialogue, single agent management | Low-configuration environments, data stored in config files, no database required | [①Docker Version](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Source Code Deployment](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 cores 4GB if using `FunASR`, 2 cores 2GB if all APIs | - |
| **Full Module Installation** | Intelligent dialogue, multi-user management, multi-agent management, intelligent console interface operation | Complete functionality experience, data stored in database |[①Docker Version](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Source Code Deployment](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Source Code Deployment Auto-Update Tutorial](../dev-ops-integration.md) | 4 cores 8GB if using `FunASR`, 2 cores 4GB if all APIs| [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **Simplified Installation** | Intelligent dialogue, single agent management | Low-configuration environments, data stored in config files, no database required | [①Docker Version](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Source Code Deployment](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| Local `FunASR` needs more memory; using APIs for everything keeps resource usage low | - |
| **Full Module Installation** | Intelligent dialogue, multi-user management, multi-agent management, intelligent console interface operation | Complete functionality experience, data stored in database |[①Docker Version](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Source Code Deployment](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Source Code Deployment Auto-Update Tutorial](../dev-ops-integration.md) | Local `FunASR` needs more memory plus database resources; using APIs for everything keeps resource usage low| [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
For frequently asked questions and related tutorials, please refer to [this link](../FAQ.md)
@@ -209,15 +210,15 @@ This project provides the following testing tools to help you verify the system
| Tool Name | Location | Usage Method | Function Description |
|:---:|:---|:---:|:---:|
| Audio Interaction Test Tool | main》digital-human》index.html | Run `python start.py` in `main/digital-human`, then open `http://127.0.0.1:8006/index.html` | Tests audio playback and reception functions, verifies if Python-side audio processing is normal |
| Model Response Test Tool | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Tests response speed of three core modules: ASR(speech recognition), LLM(large model), VLLM(vision model), TTS(speech synthesis) |
| Audio Interaction Test Tool | main》digital-human》index.html | In `main/digital-human`, install dependencies as its README says, then run `python start.py` and open `http://127.0.0.1:8006/index.html` | Tests audio playback and reception functions, verifies if Python-side audio processing is normal |
| Model Response Test Tool | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Tests response speed of core modules such as ASR(speech recognition), LLM(large model), VLLM(vision model), TTS(speech synthesis) |
> 💡 Note: When testing model speed, only models with configured keys will be tested.
---
## Feature List ✨
### Implemented ✅
![请参考-全模块安装架构图](../images/deploy2.png)
![Full-module installation architecture diagram](../images/deploy2.png)
| Feature Module | Description |
|:---:|:---|
| Core Architecture | Based on [MQTT+UDP gateway](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket and HTTP servers, provides complete console management and authentication system |
@@ -225,15 +226,15 @@ This project provides the following testing tools to help you verify the system
| Voiceprint Recognition | Supports multi-user voiceprint registration, management, and recognition, processes in parallel with ASR, real-time speaker identity recognition and passes to LLM for personalized responses |
| Intelligent Dialogue | Supports multiple LLM(large language models), implements intelligent dialogue |
| Visual Perception | Supports multiple VLLM(vision large models), implements multimodal interaction |
| Intent Recognition | Supports LLM intent recognition, Function Call function calling, provides plugin-based intent processing mechanism |
| Intent Recognition | Supports pluggable external LLM intent recognition and autonomous LLM function calling, provides plugin-based intent processing mechanism |
| Memory System | Supports local short-term memory, mem0ai interface memory, PowerMem intelligent memory, with memory summarization functionality |
| Knowledge Base | Supports RAGFlow knowledge base, enabling LLM to judge whether to schedule the knowledge base after receiving the user's question, and then answer the question |
| Tool Calling | Supports client IOT protocol, client MCP protocol, server MCP protocol, MCP endpoint protocol, custom tool functions |
| Tool Calling | Supports client IOT protocol, client MCP protocol, server MCP protocol, MCP access point protocol, custom tool functions |
| Command Delivery | Supports MCP command delivery to ESP32 devices via MQTT protocol from Smart Console |
| Management Backend | Provides Web management interface, supports user management, system configuration and device management; Supports Simplified Chinese, Traditional Chinese and English display |
| Testing Tools | Provides performance testing tools, vision model testing tools, and audio interaction testing tools |
| Management Backend | Provides Web management interface, supports user management, system configuration and device management; Supports Simplified Chinese, Traditional Chinese, English, German, Vietnamese and Brazilian Portuguese display |
| Testing Tools | Provides model response performance testing tools and audio interaction testing tools |
| Deployment Support | Supports Docker deployment and local deployment, provides complete configuration file management |
| Plugin System | Supports functional plugin extensions, custom plugin development, and plugin hot-loading |
| Plugin System | Supports functional plugin extensions and custom plugin development; newly added plugin files are loaded on the next device connection, code changes to already loaded plugins require a service restart |
### Under Development 🚧
@@ -253,7 +254,7 @@ Xiaozhi is an ecosystem. When using this product, you can also check out other [
| Usage Method | Supported Platforms | Free Platforms |
|:---:|:---:|:---:|
| OpenAI interface calls | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK | Zhipu, Gemini |
| OpenAI interface calls | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
| Ollama interface calls | Ollama | - |
| Dify interface calls | Dify | - |
| FastGPT interface calls | FastGPT | - |
@@ -279,8 +280,8 @@ In fact, any VLLM that supports OpenAI interface calls can be integrated and use
| Usage Method | Supported Platforms | Free Platforms |
|:---:|:---:|:---:|
| Interface calls | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud and Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, GizwitsTTS, ACGNTTS, OpenAITTS, Lingxi Streaming TTS, MinimaxTTS | Lingxi Streaming TTS, EdgeTTS, CosyVoiceSiliconflow(partial) |
| Local services | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, Index-TTS, PaddleSpeech | Index-TTS, PaddleSpeech, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
| Interface calls | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud and Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS | EdgeTTS, CosyVoiceSiliconflow(partial) |
| Local services | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, IndexStreamTTS, PaddleSpeechTTS | IndexStreamTTS, PaddleSpeechTTS, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
---
@@ -296,8 +297,8 @@ In fact, any VLLM that supports OpenAI interface calls can be integrated and use
| Usage Method | Supported Platforms | Free Platforms |
|:---:|:---:|:---:|
| Local use | FunASR, SherpaASR | FunASR, SherpaASR |
| Interface calls | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud, Baidu Cloud, OpenAI ASR | FunASRServer |
| Local use | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
| Interface calls | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud and Bailian, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
---
@@ -305,7 +306,9 @@ In fact, any VLLM that supports OpenAI interface calls can be integrated and use
| Usage Method | Supported Platforms | Free Platforms |
|:---:|:---:|:---:|
| Local use | 3D-Speaker | 3D-Speaker |
| Interface calls | Voiceprint service (voiceprint-api) | - |
Voiceprint recognition requires a separately deployed [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) service (based on 3D-Speaker) and MySQL. See the [voiceprint integration guide](../voiceprint-integration.md).
---
@@ -351,11 +354,10 @@ In fact, any VLLM that supports OpenAI interface calls can be integrated and use
| <img src="../images/logo_contributors.png" width="160"> | [Code Contributors](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Thanks to [all code contributors](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), your efforts have made the project more robust and powerful. |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
</picture>
</a>
+363
View File
@@ -0,0 +1,363 @@
[![Banners](../images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
<h1 align="center">Servicio backend Xiaozhi xiaozhi-esp32-server</h1>
<p align="center">
Este proyecto desarrolla sistemas de hardware y software para terminales inteligentes a partir de la teoría y la tecnología de la inteligencia simbiótica humano-máquina<br/>y ofrece el servicio backend del proyecto de hardware inteligente de código abierto
<a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a><br/>
Implementado en Python, Java y Vue según el <a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">protocolo de comunicación Xiaozhi</a><br/>
Compatible con el protocolo MQTT+UDP, el protocolo WebSocket, puntos de acceso MCP, reconocimiento de huella de voz y bases de conocimiento
</p>
<p align="center">
<a href="../FAQ.md">Preguntas frecuentes</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/issues">Informar de problemas</a>
· <a href="../../README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Documentación de despliegue</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">Notas de la versión</a>
</p>
<p align="center">
<a href="../../README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_en.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
<a href="./README_pt_BR.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-DFE0E5"></a>
<a href="./README_es.md"><img alt="Español" src="https://img.shields.io/badge/Español-DBEDFA"></a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">
<img alt="GitHub Contributors" src="https://img.shields.io/github/v/release/xinnan-tech/xiaozhi-esp32-server?logo=docker" />
</a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/LICENSE">
<img alt="GitHub pull requests" src="https://img.shields.io/badge/license-MIT-white?labelColor=black" />
</a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server">
<img alt="stars" src="https://img.shields.io/github/stars/xinnan-tech/xiaozhi-esp32-server?color=ffcb47&labelColor=black" />
</a>
</p>
<p align="center">
Liderado por el equipo del profesor Siyuan Liu (Universidad Tecnológica del Sur de China)
</br>
刘思源教授团队主导研发(华南理工大学)
</br>
<img src="../images/hnlg.jpg" alt="Universidad Tecnológica del Sur de China (华南理工大学)" width="50%">
</p>
---
## A quién va dirigido 👥
Este proyecto debe usarse junto con hardware ESP32. Si ya ha comprado hardware relacionado con ESP32, se ha conectado con éxito al servicio backend desplegado por Brother Xia (虾哥) y quiere montar por su cuenta su propio servicio backend `xiaozhi-esp32`, este proyecto encaja con lo que busca.
¿Quiere ver cómo se usa? Pulse los vídeos 🎥
<table>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="Sensación de la velocidad de respuesta" src="../images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="El secreto para optimizar la velocidad" src="../images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="Humano digital Xiaozhi con activación por voz" src="../images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="Llamadas de dispositivo a dispositivo" src="../images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="Escenario médico complejo" src="../images/demo1.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="Reproducir música, consultar el tiempo y leer noticias" src="../images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="Interrupción en tiempo real" src="../images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="Reconocer objetos con una foto" src="../images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="Controlar el encendido de electrodomésticos" src="../images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="Tareas con varias instrucciones" src="../images/demo11.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="Punto de acceso MCP" src="../images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="Envío de instrucciones MQTT" src="../images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="Reconocimiento de huella de voz" src="../images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="Tono de voz personalizado" src="../images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="Conversación en cantonés" src="../images/demo3.png" /></picture>
</a>
</td>
</tr>
</table>
---
## Advertencias ⚠️
1. Este proyecto es software de código abierto. Este software no mantiene relación comercial alguna con los proveedores de API de terceros a los que se conecta (incluidos, entre otros, plataformas de reconocimiento de voz, modelos de lenguaje y síntesis de voz), y no ofrece ninguna garantía sobre la calidad del servicio de dichos proveedores ni sobre la seguridad de sus fondos. Se recomienda elegir de forma preferente proveedores que cuenten con las licencias de actividad correspondientes y leer con atención sus condiciones de servicio y su política de privacidad. Este software no custodia ninguna clave de cuenta, no interviene en el movimiento de fondos y no asume el riesgo de pérdida del saldo recargado.
2. Las funciones de este proyecto no están completas y no ha superado una evaluación de seguridad de red. No lo utilice en entornos de producción. Si despliega este proyecto en una red pública con fines de aprendizaje, aplique las protecciones necesarias.
---
## Documentación de despliegue
![Banners](../images/banner2.png)
Este proyecto ofrece dos formas de despliegue. Elija según lo que necesite:
#### 🚀 Elección del método de despliegue
| Método de despliegue | Características | Escenarios | Documentación de despliegue | Requisitos | Tutoriales en vídeo |
|---------|------|---------|---------|---------|---------|
| **Instalación simplificada** | Diálogo inteligente, gestión de un solo agente | Entornos con pocos recursos; los datos se guardan en archivos de configuración y no hace falta base de datos | [①Versión Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Despliegue desde el código fuente](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| Ejecutar `FunASR` en local consume bastante memoria; si todo va por API, el consumo de recursos es bajo | - |
| **Instalación de todos los módulos** | Diálogo inteligente, gestión multiusuario, gestión de varios agentes, uso de la interfaz de la consola inteligente | Experiencia con todas las funciones; los datos se guardan en una base de datos |[①Versión Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Despliegue desde el código fuente](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Tutorial de actualización automática desde el código fuente](../dev-ops-integration.md) | Ejecutar `FunASR` en local consume bastante memoria, y además requiere recursos de base de datos; si todo va por API, el consumo de recursos es bajo | [Tutorial en vídeo para arrancar el código fuente en local](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
Las preguntas frecuentes y los tutoriales relacionados están en [este enlace](../FAQ.md)
> 💡 Nota: a continuación hay una plataforma de prueba desplegada con el código más reciente. Si lo necesita, puede grabar el firmware y probarla. La concurrencia es de 6 usuarios y los datos se borran cada día.
```
Dirección de la consola inteligente: https://2662r3426b.vicp.fun
Consola inteligente (versión H5): https://2662r3426b.vicp.fun/h5/index.html
Herramienta de prueba del servicio: https://2662r3426b.vicp.fun/test/
Dirección de la interfaz OTA: https://2662r3426b.vicp.fun/xiaozhi/ota/
Dirección de la interfaz WebSocket: wss://2662r3426b.vicp.fun/xiaozhi/v1/
```
#### 🚩 Descripción y recomendaciones de configuración
> [!Note]
> Este proyecto ofrece dos esquemas de configuración:
>
> 1. `Configuración básica y totalmente gratuita`: adecuada para uso personal y doméstico. Todos los componentes usan opciones gratuitas y no hay que pagar nada más.
>
> 2. `Configuración en streaming`: adecuada para demostraciones, formación y escenarios con más de 2 usuarios concurrentes. Usa procesamiento en streaming, responde antes y se siente mejor.
>
> Desde la versión `0.5.2`, el proyecto admite la configuración en streaming. Frente a las versiones anteriores, el tiempo de respuesta mejora unos `2,5 segundos` y la experiencia de uso mejora de forma notable.
| Módulo | Configuración básica y totalmente gratuita | Configuración en streaming |
|:---:|:---:|:---:|
| ASR (reconocimiento de voz) | FunASR (local) | 👍XunfeiStreamASR (iFLYTEK en streaming) |
| LLM (modelo de lenguaje) | glm-4-flash (Zhipu) | 👍qwen-flash (Alibaba Bailian) |
| VLLM (modelo de visión) | glm-4v-flash (Zhipu) | 👍qwen3.5-flash (Alibaba Bailian) |
| TTS (síntesis de voz) | EdgeTTS (Microsoft) | 👍HuoshanDoubleStreamTTS (Volcano en streaming) |
| Intent (reconocimiento de intención) | function_call (llamada a funciones) | function_call (llamada a funciones) |
| Memory (memoria) | mem_local_short (memoria local a corto plazo) | mem_local_short (memoria local a corto plazo) |
Si le importa el tiempo que tarda cada componente, consulte el [informe de rendimiento de los componentes de Xiaozhi](https://github.com/xinnan-tech/xiaozhi-performance-research) y pruebe en su propio entorno con el método que describe el informe.
#### 🔧 Herramientas de prueba
Este proyecto incluye las siguientes herramientas para comprobar el sistema y elegir un modelo adecuado:
| Herramienta | Ubicación | Uso | Qué hace |
|:---:|:---|:---:|:---:|
| Prueba de interacción de audio | main》digital-human》index.html | En `main/digital-human`, instale las dependencias como indica su README, después ejecute `python start.py` y abra `http://127.0.0.1:8006/index.html` | Prueba la reproducción y la recepción de audio, y comprueba que el procesamiento de audio en Python funciona |
| Prueba de respuesta de los modelos | main》xiaozhi-server》performance_tester.py | Ejecute `python performance_tester.py` | Mide la velocidad de respuesta de los principales módulos: ASR (reconocimiento de voz), LLM (modelo de lenguaje), VLLM (modelo de visión) y TTS (síntesis de voz) |
> 💡 Nota: al medir la velocidad de los modelos, solo se prueban los modelos que tienen una clave configurada.
---
## Lista de funciones ✨
### Implementado ✅
![Diagrama de arquitectura de la instalación de todos los módulos](../images/deploy2.png)
| Módulo | Descripción |
|:---:|:---|
| Arquitectura principal | Basada en la [pasarela MQTT+UDP](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md) y en servidores WebSocket y HTTP; ofrece un sistema completo de gestión de consola y de autenticación |
| Interacción por voz | Admite ASR en streaming (reconocimiento de voz), TTS en streaming (síntesis de voz) y VAD (detección de actividad de voz), además de reconocimiento multilingüe y procesamiento de voz |
| Huella de voz | Admite el registro, la gestión y el reconocimiento de la huella de voz de varios usuarios; se procesa en paralelo con el ASR, identifica en tiempo real quién habla y se lo pasa al LLM para una respuesta personalizada |
| Diálogo inteligente | Admite varios LLM (modelos de lenguaje) para mantener un diálogo inteligente |
| Percepción visual | Admite varios VLLM (modelos de visión) para una interacción multimodal |
| Reconocimiento de intención | Admite el reconocimiento de intención con un modelo de lenguaje (LLM) externo y conectable, y la llamada autónoma a funciones del modelo, y ofrece un mecanismo de procesamiento de intenciones mediante plugins |
| Sistema de memoria | Admite memoria local a corto plazo, memoria por la interfaz mem0ai y memoria inteligente PowerMem, con resumen de la memoria |
| Base de conocimiento | Admite la base de conocimiento RAGFlow, para que el modelo decida si debe consultarla antes de responder |
| Llamada a herramientas | Admite el protocolo IOT del cliente, el protocolo MCP del cliente, el protocolo MCP del servidor, el protocolo de punto de acceso MCP y funciones de herramientas personalizadas |
| Envío de instrucciones | Apoyado en el protocolo MQTT, permite enviar instrucciones MCP desde la consola inteligente a los dispositivos ESP32 |
| Backend de administración | Ofrece una interfaz web de administración, con gestión de usuarios, configuración del sistema y gestión de dispositivos. La interfaz se muestra en chino simplificado, chino tradicional, inglés, alemán, vietnamita y portugués de Brasil |
| Herramientas de prueba | Incluye herramientas de prueba de rendimiento y de interacción de audio |
| Soporte de despliegue | Admite el despliegue con Docker y el despliegue local, y ofrece una gestión completa de los archivos de configuración |
| Sistema de plugins | Admite extensiones de plugins funcionales y el desarrollo de plugins propios; los nuevos archivos de plugin se cargan automáticamente en la siguiente conexión del dispositivo, y los cambios en el código de un plugin ya cargado requieren reiniciar el servicio |
### En desarrollo 🚧
Para ver el avance concreto del plan de desarrollo, [pulse aquí](https://github.com/users/xinnan-tech/projects/3). Las preguntas frecuentes y los tutoriales relacionados están en [este enlace](../FAQ.md)
Si desarrolla software, aquí tiene una [carta abierta a quienes desarrollan](../contributor_open_letter.md). La invitación está hecha.
---
## Ecosistema del producto 👬
Xiaozhi es un ecosistema. Al usar este producto, también puede ver otros [proyectos destacados](https://github.com/78/xiaozhi-esp32/blob/main/README_zh.md#%E7%9B%B8%E5%85%B3%E5%BC%80%E6%BA%90%E9%A1%B9%E7%9B%AE) de este ecosistema
---
## Plataformas y componentes compatibles 📋
### LLM, modelos de lenguaje
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|:---:|:---:|:---:|
| Llamadas por la interfaz OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
| Llamadas por la interfaz Ollama | Ollama | - |
| Llamadas por la interfaz Dify | Dify | - |
| Llamadas por la interfaz FastGPT | FastGPT | - |
| Llamadas por la interfaz Coze | Coze | - |
| Llamadas por la interfaz Xinference | Xinference | - |
| Llamadas por la interfaz HomeAssistant | HomeAssistant | - |
En la práctica, se puede integrar cualquier LLM que admita llamadas por la interfaz OpenAI.
---
### VLLM, modelos de visión
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|:---:|:---:|:---:|
| Llamadas por la interfaz OpenAI | Alibaba Bailian, Zhipu ChatGLMVLLM | Zhipu ChatGLMVLLM |
En la práctica, se puede integrar cualquier VLLM que admita llamadas por la interfaz OpenAI.
---
### TTS, síntesis de voz
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|:---:|:---:|:---:|
| Llamadas por interfaz | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud y Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS | EdgeTTS, CosyVoiceSiliconflow (parcial) |
| Servicio local | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, IndexStreamTTS, PaddleSpeechTTS | IndexStreamTTS, PaddleSpeechTTS, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
---
### VAD, detección de actividad de voz
| Tipo | Plataforma | Forma de uso | Precio | Notas |
|:---:|:---------:|:----:|:----:|:--:|
| VAD | SileroVAD | Uso local | Gratuito | |
---
### ASR, reconocimiento de voz
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|:---:|:---:|:---:|
| Uso local | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
| Llamadas por interfaz | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud y Bailian, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
---
### Voiceprint, huella de voz
| Forma de uso | Plataformas compatibles | Plataformas gratuitas |
|:---:|:---:|:---:|
| Llamadas por interfaz | Servicio de reconocimiento de huella de voz (voiceprint-api) | - |
El reconocimiento de huella de voz requiere desplegar por separado el servicio [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) (basado en 3D-Speaker), que depende de MySQL; consulte la [guía de integración de la huella de voz](../voiceprint-integration.md)
---
### Memory, almacenamiento de memoria
| Tipo | Plataforma | Forma de uso | Precio | Notas |
|:------:|:---------------:|:----:|:---------:|:--:|
| Memory | mem0ai | Llamadas por interfaz | Cuota de 1000 veces al mes | |
| Memory | [powermem](../powermem-integration.md) | Resumen local | Depende del LLM y de la base de datos | OceanBase de código abierto; admite búsqueda inteligente |
| Memory | mem_local_short | Resumen local | Gratuito | |
| Memory | nomem | Modo sin memoria | Gratuito | |
---
### Intent, reconocimiento de intención
| Tipo | Plataforma | Forma de uso | Precio | Notas |
|:------:|:-------------:|:----:|:-------:|:---------------------:|
| Intent | intent_llm | Llamadas por interfaz | Según el precio del LLM | Reconoce la intención mediante el modelo de lenguaje (LLM); gran capacidad de generalización |
| Intent | function_call | Llamadas por interfaz | Según el precio del LLM | Completa la intención con llamadas a funciones del modelo; rápido y con buen resultado |
| Intent | nointent | Modo sin intención | Gratuito | No reconoce la intención y devuelve directamente el resultado del diálogo |
---
### Rag, generación aumentada por recuperación
| Tipo | Plataforma | Forma de uso | Precio | Notas |
|:------:|:-------------:|:----:|:-------:|:---------------------:|
| Rag | ragflow | Llamadas por interfaz | Cobrado según los tokens consumidos en el troceado y la segmentación | Usa la generación aumentada por recuperación de RagFlow para dar respuestas más precisas |
---
## Agradecimientos 🙏
| Logo | Proyecto o empresa | Descripción |
|:---:|:---:|:---|
| <img src="../images/logo_bailing.png" width="160"> | [Robot de diálogo por voz Bailing (百聆语音对话机器人)](https://github.com/wwbin2017/bailing) | Este proyecto parte del [robot de diálogo por voz Bailing](https://github.com/wwbin2017/bailing) y se construye sobre esa base |
| <img src="../images/logo_tenclass.png" width="160"> | [Tenclass (十方融海)](https://www.tenclass.com/) | Gracias a [Tenclass](https://www.tenclass.com/) por definir el protocolo de comunicación estándar del ecosistema Xiaozhi, las soluciones de compatibilidad con varios dispositivos y los ejemplos prácticos de alta concurrencia, y por el apoyo de documentación técnica de todo el recorrido de este proyecto |
| <img src="../images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology (玄凤科技)](https://github.com/Eric0308) | Gracias a [Xuanfeng Technology](https://github.com/Eric0308) por el código del marco de llamada a funciones, del protocolo de comunicación MCP y del mecanismo de llamadas basado en plugins. Con un sistema estandarizado de despacho de instrucciones y capacidad de ampliación dinámica, mejora de forma clara la eficiencia de interacción y la extensibilidad de los dispositivos de frontend (IoT) |
| <img src="../images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Gracias a [huangjunsen](https://github.com/huangjunsen0406) por el módulo `consola inteligente móvil`, que permite controlar con eficacia dispositivos móviles de varias plataformas e interactuar en tiempo real, y mejora mucho la comodidad de uso y la eficiencia de gestión en escenarios móviles |
| <img src="../images/logo_huiyuan.png" width="160"> | [Huiyuan Design (汇远设计)](http://ui.kwd988.net/) | Gracias a [Huiyuan Design](http://ui.kwd988.net/) por la solución visual de este proyecto, y por poner al servicio de la experiencia de uso la práctica de diseño acumulada con más de mil empresas |
| <img src="../images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology (西安勤人信息科技)](https://www.029app.com/) | Gracias a [Xi'an Qinren Information Technology](https://www.029app.com/) por profundizar el sistema visual de este proyecto y mantener un estilo coherente y ampliable en distintos escenarios |
| <img src="../images/logo_contributors.png" width="160"> | [Quienes contribuyen código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Gracias a [todas las personas que contribuyen código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors). Su trabajo hace el proyecto más sólido y más capaz. |
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
</picture>
</a>
+39 -37
View File
@@ -22,6 +22,7 @@ Suporte ao protocolo MQTT+UDP, protocolo WebSocket, ponto de acesso MCP, reconhe
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
<a href="./README_pt_BR.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-DBEDFA"></a>
<a href="./README_es.md"><img alt="Español" src="https://img.shields.io/badge/Español-DFE0E5"></a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">
<img alt="GitHub Contributors" src="https://img.shields.io/github/v/release/xinnan-tech/xiaozhi-esp32-server?logo=docker" />
</a>
@@ -54,31 +55,31 @@ Quer ver os efeitos de uso? Clique nos vídeos abaixo 🎥
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
<img alt="A sensação da velocidade de resposta" src="../images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
<img alt="O segredo para otimizar a velocidade" src="../images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
<img alt="Humano digital Xiaozhi com ativação por voz" src="../images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
<img alt="Chamadas de dispositivo para dispositivo" src="../images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
<img alt="Cenário médico complexo" src="../images/demo1.png" /></picture>
</a>
</td>
</tr>
@@ -86,31 +87,31 @@ Quer ver os efeitos de uso? Clique nos vídeos abaixo 🎥
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
<img alt="Reproduzir música, consultar o tempo e ler as notícias" src="../images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
<img alt="Interrupção em tempo real" src="../images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
<img alt="Reconhecer objetos com uma foto" src="../images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
<img alt="Controlar o acionamento de eletrodomésticos" src="../images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
<img alt="Tarefas com vários comandos" src="../images/demo11.png" /></picture>
</a>
</td>
</tr>
@@ -118,31 +119,31 @@ Quer ver os efeitos de uso? Clique nos vídeos abaixo 🎥
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
<img alt="Ponto de acesso MCP" src="../images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
<img alt="Envio de comandos MQTT" src="../images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
<img alt="Reconhecimento de impressão vocal" src="../images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
<img alt="Tom de voz personalizado" src="../images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
<img alt="Conversa em cantonês" src="../images/demo3.png" /></picture>
</a>
</td>
</tr>
@@ -167,8 +168,8 @@ Este projeto oferece dois métodos de implantação. Por favor, escolha de acord
#### 🚀 Seleção do Método de Implantação
| Método de Implantação | Funcionalidades | Cenários Aplicáveis | Documentação de Implantação | Requisitos de Configuração | Tutoriais em Vídeo |
|---------|------|---------|---------|---------|---------|
| **Instalação Simplificada** | Diálogo inteligente, gerenciamento de agente único | Ambientes de baixa configuração, dados armazenados em arquivos de configuração, sem necessidade de banco de dados | [①Versão Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Implantação via Código-Fonte](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 núcleos 4GB se usar `FunASR`, 2 núcleos 2GB se todas APIs | - |
| **Instalação de Módulo Completo** | Diálogo inteligente, gerenciamento multiusuário, gerenciamento de múltiplos agentes, operação de interface do console inteligente | Experiência com funcionalidade completa, dados armazenados em banco de dados |[①Versão Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Implantação via Código-Fonte](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Tutorial de Atualização Automática via Código-Fonte](../dev-ops-integration.md) | 4 núcleos 8GB se usar `FunASR`, 2 núcleos 4GB se todas APIs| [Tutorial em Vídeo de Inicialização via Código-Fonte Local](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **Instalação Simplificada** | Diálogo inteligente, gerenciamento de agente único | Ambientes de baixa configuração, dados armazenados em arquivos de configuração, sem necessidade de banco de dados | [①Versão Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Implantação via Código-Fonte](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| `FunASR` local exige mais memória; usar apenas APIs mantém o consumo de recursos baixo | - |
| **Instalação de Módulo Completo** | Diálogo inteligente, gerenciamento multiusuário, gerenciamento de múltiplos agentes, operação de interface do console inteligente | Experiência com funcionalidade completa, dados armazenados em banco de dados |[①Versão Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Implantação via Código-Fonte](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Tutorial de Atualização Automática via Código-Fonte](../dev-ops-integration.md) | `FunASR` local exige mais memória e recursos de banco de dados; usar apenas APIs mantém o consumo de recursos baixo| [Tutorial em Vídeo de Inicialização via Código-Fonte Local](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
Perguntas frequentes e tutoriais relacionados podem ser consultados [neste link](../FAQ.md)
@@ -209,15 +210,15 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
| Nome da Ferramenta | Localização | Método de Uso | Descrição da Função |
|:---:|:---|:---:|:---:|
| Ferramenta de Teste de Interação por Áudio | main》digital-human》index.html | Execute `python start.py` em `main/digital-human` e depois abra `http://127.0.0.1:8006/index.html` | Testa as funções de reprodução e recepção de áudio, verifica se o processamento de áudio no lado Python está normal |
| Ferramenta de Teste de Resposta de Modelo | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Testa a velocidade de resposta dos três módulos principais: ASR(reconhecimento de fala), LLM(modelo de linguagem), VLLM(modelo de visão), TTS(síntese de voz) |
| Ferramenta de Teste de Interação por Áudio | main》digital-human》index.html | Em `main/digital-human`, instale as dependências como indica o README, depois execute `python start.py` e abra `http://127.0.0.1:8006/index.html` | Testa as funções de reprodução e recepção de áudio, verifica se o processamento de áudio no lado Python está normal |
| Ferramenta de Teste de Resposta de Modelo | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Testa a velocidade de resposta de módulos principais como ASR(reconhecimento de fala), LLM(modelo de linguagem), VLLM(modelo de visão), TTS(síntese de voz) |
> 💡 Nota: Ao testar a velocidade dos modelos, apenas os modelos com chaves configuradas serão testados.
---
## Lista de Funcionalidades ✨
### Implementado ✅
![请参考-全模块安装架构图](../images/deploy2.png)
![Diagrama de arquitetura da instalação de todos os módulos](../images/deploy2.png)
| Módulo de Funcionalidade | Descrição |
|:---:|:---|
| Arquitetura Principal | Baseado em [gateway MQTT+UDP](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), servidores WebSocket e HTTP, fornece sistema completo de gerenciamento de console e autenticação |
@@ -225,15 +226,15 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
| Reconhecimento de Impressão Vocal | Suporta registro, gerenciamento e reconhecimento de impressão vocal de múltiplos usuários, processa em paralelo com o ASR, reconhecimento de identidade do falante em tempo real e repassa ao LLM para respostas personalizadas |
| Diálogo Inteligente | Suporta múltiplos LLM (modelos de linguagem de grande porte), implementa diálogo inteligente |
| Percepção Visual | Suporta múltiplos VLLM (modelos de visão de grande porte), implementa interação multimodal |
| Reconhecimento de Intenção | Suporta reconhecimento de intenção por LLM, Function Call (chamada de função), fornece mecanismo de processamento de intenção baseado em plugins |
| Reconhecimento de Intenção | Suporta reconhecimento de intenção por LLM externo e conectável, chamada de função autônoma do modelo de linguagem, fornece mecanismo de processamento de intenção baseado em plugins |
| Sistema de Memória | Suporta memória local de curto prazo, memória via interface mem0ai, memória inteligente PowerMem, com funcionalidade de resumo de memória |
| Base de Conhecimento | Suporta base de conhecimento RAGFlow, permitindo que o LLM julgue se deve acionar a base de conhecimento após receber a pergunta do usuário, e então responda à pergunta |
| Chamada de Ferramentas | Suporta protocolo IOT do cliente, protocolo MCP do cliente, protocolo MCP do servidor, protocolo de endpoint MCP, funções de ferramentas personalizadas |
| Chamada de Ferramentas | Suporta protocolo IOT do cliente, protocolo MCP do cliente, protocolo MCP do servidor, protocolo de ponto de acesso MCP, funções de ferramentas personalizadas |
| Envio de Comandos | Suporta envio de comandos MCP para dispositivos ESP32 via protocolo MQTT a partir do Console Inteligente |
| Backend de Gerenciamento | Fornece interface de gerenciamento Web, suporta gerenciamento de usuários, configuração do sistema e gerenciamento de dispositivos; Suporta exibição em Chinês Simplificado, Chinês Tradicional e Inglês |
| Ferramentas de Teste | Fornece ferramentas de teste de desempenho, ferramentas de teste de modelo de visão e ferramentas de teste de interação por áudio |
| Backend de Gerenciamento | Fornece interface de gerenciamento Web, suporta gerenciamento de usuários, configuração do sistema e gerenciamento de dispositivos; Suporta exibição em Chinês Simplificado, Chinês Tradicional, Inglês, Alemão, Vietnamita e Português do Brasil |
| Ferramentas de Teste | Fornece ferramentas de teste de desempenho de resposta de modelo e ferramentas de teste de interação por áudio |
| Suporte à Implantação | Suporta implantação via Docker e implantação local, fornece gerenciamento completo de arquivos de configuração |
| Sistema de Plugins | Suporta extensões de plugins funcionais, desenvolvimento de plugins personalizados e carregamento dinâmico de plugins |
| Sistema de Plugins | Suporta extensões de plugins funcionais e desenvolvimento de plugins personalizados; novos arquivos de plugin são carregados na próxima conexão do dispositivo e alterações em plugins já carregados exigem reiniciar o serviço |
### Em Desenvolvimento 🚧
@@ -253,7 +254,7 @@ Xiaozhi é um ecossistema. Ao utilizar este produto, você também pode conferir
| Método de Uso | Plataformas Suportadas | Plataformas Gratuitas |
|:---:|:---:|:---:|
| Chamadas via interface OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK | Zhipu, Gemini |
| Chamadas via interface OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
| Chamadas via interface Ollama | Ollama | - |
| Chamadas via interface Dify | Dify | - |
| Chamadas via interface FastGPT | FastGPT | - |
@@ -261,7 +262,7 @@ Xiaozhi é um ecossistema. Ao utilizar este produto, você também pode conferir
| Chamadas via interface Xinference | Xinference | - |
| Chamadas via interface HomeAssistant | HomeAssistant | - |
Na verdade, qualquer LLM que suporte chamadas via interface openai pode ser integrado e utilizado.
Na verdade, qualquer LLM que suporte chamadas via interface OpenAI pode ser integrado e utilizado.
---
@@ -279,8 +280,8 @@ Na verdade, qualquer VLLM que suporte chamadas via interface OpenAI pode ser int
| Método de Uso | Plataformas Suportadas | Plataformas Gratuitas |
|:---:|:---:|:---:|
| Chamadas via interface | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud e Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, GizwitsTTS, ACGNTTS, OpenAITTS, Lingxi Streaming TTS, MinimaxTTS | Lingxi Streaming TTS, EdgeTTS, CosyVoiceSiliconflow(parcial) |
| Serviços locais | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, Index-TTS, PaddleSpeech | Index-TTS, PaddleSpeech, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
| Chamadas via interface | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud e Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS | EdgeTTS, CosyVoiceSiliconflow(parcial) |
| Serviços locais | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, IndexStreamTTS, PaddleSpeechTTS | IndexStreamTTS, PaddleSpeechTTS, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
---
@@ -296,8 +297,8 @@ Na verdade, qualquer VLLM que suporte chamadas via interface OpenAI pode ser int
| Método de Uso | Plataformas Suportadas | Plataformas Gratuitas |
|:---:|:---:|:---:|
| Uso local | FunASR, SherpaASR | FunASR, SherpaASR |
| Chamadas via interface | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud, Baidu Cloud, OpenAI ASR | FunASRServer |
| Uso local | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
| Chamadas via interface | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud e Bailian, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
---
@@ -305,7 +306,9 @@ Na verdade, qualquer VLLM que suporte chamadas via interface OpenAI pode ser int
| Método de Uso | Plataformas Suportadas | Plataformas Gratuitas |
|:---:|:---:|:---:|
| Uso local | 3D-Speaker | 3D-Speaker |
| Chamadas via interface | Serviço de voiceprint (voiceprint-api) | - |
O reconhecimento de impressão vocal exige um serviço [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) implantado separadamente (baseado em 3D-Speaker) e MySQL. Veja o [guia de integração de voiceprint](../voiceprint-integration.md).
---
@@ -351,11 +354,10 @@ Na verdade, qualquer VLLM que suporte chamadas via interface OpenAI pode ser int
| <img src="../images/logo_contributors.png" width="160"> | [Contribuidores de Código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Agradecimentos a [todos os contribuidores de código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), seus esforços tornaram o projeto mais robusto e poderoso. |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
</picture>
</a>
+43 -41
View File
@@ -6,7 +6,7 @@
Dự án này dựa trên lý thuyết và công nghệ trí tuệ cộng sinh người-máy để phát triển hệ thống phần mềm và phần cứng thiết bị đầu cuối thông minh<br/>Cung cấp dịch vụ backend cho dự án phần cứng thông minh mã nguồn mở
<a href="https://github.com/78/xiaozhi-esp32">xiaozhi-esp32</a><br/>
Được triển khai bằng Python, Java, Vue theo <a href="https://ccnphfhqs21z.feishu.cn/wiki/M0XiwldO9iJwHikpXD5cEx71nKh">giao thức truyền thông Xiaozhi</a><br/>
Hỗ trợ giao thức MQTT+UDP, giao thức Websocket, điểm truy cập MCP, nhận dạng giọng nói và kho tri thức
Hỗ trợ giao thức MQTT+UDP, giao thức Websocket, điểm truy cập MCP, nhận dạng vân giọng và kho tri thức
</p>
<p align="center">
@@ -22,6 +22,7 @@ Hỗ trợ giao thức MQTT+UDP, giao thức Websocket, điểm truy cập MCP,
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DBEDFA"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
<a href="./README_pt_BR.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-DFE0E5"></a>
<a href="./README_es.md"><img alt="Español" src="https://img.shields.io/badge/Español-DFE0E5"></a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">
<img alt="GitHub Contributors" src="https://img.shields.io/github/v/release/xinnan-tech/xiaozhi-esp32-server?logo=docker" />
</a>
@@ -38,7 +39,7 @@ Spearheaded by Professor Siyuan Liu's Team (South China University of Technology
</br>
Được dẫn dắt bởi nhóm Giáo sư Lưu Tư Nguyên (Đại học Bách khoa Nam Trung Quốc)
</br>
<img src="../images/hnlg.jpg" alt="华南理工大学" width="50%">
<img src="../images/hnlg.jpg" alt="Đại học Bách khoa Nam Trung Quốc" width="50%">
</p>
---
@@ -54,31 +55,31 @@ Muốn xem hiệu quả sử dụng? Hãy xem video 🎥
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
<img alt="Cảm giác tốc độ phản hồi" src="../images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
<img alt="Bí quyết tối ưu tốc độ" src="../images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
<img alt="Người ảo số Xiaozhi có đánh thức bằng giọng nói" src="../images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
<img alt="Gọi giữa các thiết bị" src="../images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
<img alt="Tình huống y tế phức tạp" src="../images/demo1.png" /></picture>
</a>
</td>
</tr>
@@ -86,31 +87,31 @@ Muốn xem hiệu quả sử dụng? Hãy xem video 🎥
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
<img alt="Phát nhạc, tra cứu thời tiết và đọc tin tức" src="../images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
<img alt="Ngắt giữa chừng theo thời gian thực" src="../images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
<img alt="Nhận dạng vật thể qua ảnh chụp" src="../images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
<img alt="Bật tắt thiết bị gia dụng" src="../images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
<img alt="Nhiệm vụ nhiều lệnh" src="../images/demo11.png" /></picture>
</a>
</td>
</tr>
@@ -118,31 +119,31 @@ Muốn xem hiệu quả sử dụng? Hãy xem video 🎥
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
<img alt="Điểm truy cập MCP" src="../images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
<img alt="Gửi lệnh MQTT" src="../images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
<img alt="Nhận dạng vân giọng" src="../images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
<img alt="Giọng nói tùy chỉnh" src="../images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
<img alt="Trò chuyện bằng tiếng Quảng Đông" src="../images/demo3.png" /></picture>
</a>
</td>
</tr>
@@ -168,12 +169,12 @@ Dự án này cung cấp hai phương pháp triển khai, vui lòng chọn theo
#### 🚀 Lựa chọn phương pháp triển khai
| Phương pháp triển khai | Đặc điểm | Tình huống áp dụng | Tài liệu triển khai | Yêu cầu cấu hình | Video hướng dẫn |
|---------|------|---------|---------|---------|---------|
| **Cài đặt tối giản** | Đối thoại thông minh, quản lý đơn tác nhân | Môi trường cấu hình thấp, dữ liệu lưu trong tệp cấu hình, không cần cơ sở dữ liệu | [①Phiên bản Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Triển khai mã nguồn](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 nhân 4GB nếu dùng `FunASR`, 2 nhân 2GB nếu toàn API | - |
| **Cài đặt toàn bộ module** | Đối thoại thông minh, quản lý đa người dùng, quản lý đa tác nhân, bảng điều khiển thông minh | Trải nghiệm đầy đủ tính năng, dữ liệu lưu trong cơ sở dữ liệu |[①Phiên bản Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Triển khai mã nguồn](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Hướng dẫn tự động cập nhật triển khai mã nguồn](../dev-ops-integration.md) | 4 nhân 8GB nếu dùng `FunASR`, 2 nhân 4GB nếu toàn API| [Video hướng dẫn khởi động mã nguồn cục bộ](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **Cài đặt tối giản** | Đối thoại thông minh, quản lý đơn tác nhân | Môi trường cấu hình thấp, dữ liệu lưu trong tệp cấu hình, không cần cơ sở dữ liệu | [①Phiên bản Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Triển khai mã nguồn](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| `FunASR` cục bộ cần nhiều bộ nhớ; dùng toàn API thì mức dùng tài nguyên thấp | - |
| **Cài đặt toàn bộ module** | Đối thoại thông minh, quản lý đa người dùng, quản lý đa tác nhân, thao tác qua bảng điều khiển thông minh | Trải nghiệm đầy đủ tính năng, dữ liệu lưu trong cơ sở dữ liệu |[①Phiên bản Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Triển khai mã nguồn](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Hướng dẫn tự động cập nhật triển khai mã nguồn](../dev-ops-integration.md) | `FunASR` cục bộ cần nhiều bộ nhớ, thêm tài nguyên cơ sở dữ liệu; dùng toàn API thì mức dùng tài nguyên thấp| [Video hướng dẫn khởi động mã nguồn cục bộ](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
Câu hỏi thường gặp và hướng dẫn liên quan, vui lòng tham khảo [liên kết này](../FAQ.md)
> 💡 Gợi ý: Dưới đây là nền tảng thử nghiệm được triển khai theo mã mới nhất, có thể flash để thử nghiệm nếu cần, đồng thời là 6, dữ liệu sẽ được xóa mỗi ngày,
> 💡 Gợi ý: Dưới đây là nền tảng thử nghiệm được triển khai theo mã mới nhất, có thể flash để thử nghiệm nếu cần, đồng thời là 6, dữ liệu sẽ được xóa mỗi ngày.
```
Địa chỉ bảng điều khiển thông minh: https://2662r3426b.vicp.fun
@@ -192,7 +193,7 @@ Công cụ kiểm tra dịch vụ: https://2662r3426b.vicp.fun/test/
>
> 2. `Cấu hình streaming`: Phù hợp với demo, đào tạo, hơn 2 đồng thời, v.v., sử dụng công nghệ xử lý streaming, tốc độ phản hồi nhanh hơn, trải nghiệm tốt hơn.
>
> Từ phiên bản `0.5.2`, dự án hỗ trợ cấu hình streaming, so với phiên bản đầu, tốc độ phản hồi cải thiện khoảng `2.5 giây`, cải thiện đáng kể trải nghiệm người dùng.
> Từ phiên bản `0.5.2`, dự án hỗ trợ cấu hình streaming, so với các phiên bản trước đó, tốc độ phản hồi cải thiện khoảng `2.5 giây`, cải thiện đáng kể trải nghiệm người dùng.
| Tên module | Cài đặt miễn phí cho người mới | Cấu hình streaming |
|:---:|:---:|:---:|
@@ -210,15 +211,15 @@ Dự án này cung cấp các công cụ kiểm tra sau để giúp bạn xác m
| Tên công cụ | Vị trí | Phương pháp sử dụng | Mô tả chức năng |
|:---:|:---|:---:|:---:|
| Công cụ kiểm tra tương tác âm thanh | main》digital-human》index.html | Chạy `python start.py` trong `main/digital-human`, sau đó mở `http://127.0.0.1:8006/index.html` | Kiểm tra chức năng phát và nhận âm thanh, xác minh xử lý âm thanh phía Python có bình thường không |
| Công cụ kiểm tra phản hồi mô hình | main》xiaozhi-server》performance_tester.py | Thực hiện `python performance_tester.py` | Kiểm tra tốc độ phản hồi của ba module cốt lõi ASR(Nhận dạng giọng nói), LLM(Mô hình lớn), VLLM(Mô hình thị giác), TTS(Tổng hợp giọng nói) |
| Công cụ kiểm tra tương tác âm thanh | main》digital-human》index.html | Trong `main/digital-human`, cài đặt các phụ thuộc theo README, rồi chạy `python start.py` và mở `http://127.0.0.1:8006/index.html` | Kiểm tra chức năng phát và nhận âm thanh, xác minh xử lý âm thanh phía Python có bình thường không |
| Công cụ kiểm tra phản hồi mô hình | main》xiaozhi-server》performance_tester.py | Thực hiện `python performance_tester.py` | Kiểm tra tốc độ phản hồi của các module cốt lõi như ASR(Nhận dạng giọng nói), LLM(Mô hình lớn), VLLM(Mô hình thị giác), TTS(Tổng hợp giọng nói) |
> 💡 Gợi ý: Khi kiểm tra tốc độ mô hình, chỉ kiểm tra các mô hình đã cấu hình khóa.
---
## Danh sách tính năng ✨
### Đã thực hiện ✅
![请参考-全模块安装架构图](../images/deploy2.png)
![Sơ đồ kiến trúc cài đặt đầy đủ các module](../images/deploy2.png)
| Module chức năng | Mô tả |
|:---:|:---|
| Kiến trúc cốt lõi | Dựa trên [cổng MQTT+UDP](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket, máy chủ HTTP, cung cấp hệ thống quản lý bảng điều khiển và xác thực hoàn chỉnh |
@@ -231,10 +232,10 @@ Dự án này cung cấp các công cụ kiểm tra sau để giúp bạn xác m
| Kho tri thức | Hỗ trợ kho tri thức RAGFlow, cho phép mô hình lớn đánh giá cần gọi kho tri thức trước khi trả lời |
| Gọi công cụ | Hỗ trợ giao thức IOT phía client, giao thức MCP phía client, giao thức MCP phía server, giao thức điểm truy cập MCP, hàm công cụ tùy chỉnh |
| Gửi lệnh | Dựa vào giao thức MQTT, hỗ trợ gửi lệnh MCP từ bảng điều khiển thông minh xuống thiết bị ESP32 |
| Backend quản lý | Cung cấp giao diện quản lý Web, hỗ trợ quản lý người dùng, cấu hình hệ thống và quản lý thiết bị; giao diện hỗ trợ hiển thị tiếng Trung giản thể, tiếng Trung phồn thể, tiếng Anh |
| Công cụ kiểm tra | Cung cấp công cụ kiểm tra hiệu suất, công cụ kiểm tra mô hình thị giác và công cụ kiểm tra tương tác âm thanh |
| Backend quản lý | Cung cấp giao diện quản lý Web, hỗ trợ quản lý người dùng, cấu hình hệ thống và quản lý thiết bị; giao diện hỗ trợ hiển thị tiếng Trung giản thể, tiếng Trung phồn thể, tiếng Anh, tiếng Đức, tiếng Việt và tiếng Bồ Đào Nha Brazil |
| Công cụ kiểm tra | Cung cấp công cụ kiểm tra hiệu suất phản hồi mô hình và công cụ kiểm tra tương tác âm thanh |
| Hỗ trợ triển khai | Hỗ trợ triển khai Docker và triển khai cục bộ, cung cấp quản lý tệp cấu hình hoàn chỉnh |
| Hệ thống plugin | Hỗ trợ mở rộng plugin chức năng, phát triển plugin tùy chỉnh và hot loading plugin |
| Hệ thống plugin | Hỗ trợ mở rộng plugin chức năng và phát triển plugin tùy chỉnh; file plugin mới được tải ở lần kết nối thiết bị tiếp theo, thay đổi mã của plugin đã tải cần khởi động lại dịch vụ |
### Đang phát triển 🚧
@@ -254,7 +255,7 @@ Xiaozhi là một hệ sinh thái, khi bạn sử dụng sản phẩm này, bạ
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|:---:|:---:|:---:|
| Gọi giao diện openai | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFlytek | Zhipu, Gemini |
| Gọi giao diện OpenAI | Alibaba Bailian, Volcano Engine, DeepSeek, Zhipu, Gemini, iFLYTEK, LM Studio | Zhipu, Gemini |
| Gọi giao diện ollama | Ollama | - |
| Gọi giao diện dify | Dify | - |
| Gọi giao diện fastgpt | Fastgpt | - |
@@ -262,7 +263,7 @@ Xiaozhi là một hệ sinh thái, khi bạn sử dụng sản phẩm này, bạ
| Gọi giao diện xinference | Xinference | - |
| Gọi giao diện homeassistant | HomeAssistant | - |
Trên thực tế, bất kỳ LLM nào hỗ trợ gọi giao diện openai đều có thể truy cập sử dụng.
Trên thực tế, bất kỳ LLM nào hỗ trợ gọi giao diện OpenAI đều có thể truy cập sử dụng.
---
@@ -270,9 +271,9 @@ Trên thực tế, bất kỳ LLM nào hỗ trợ gọi giao diện openai đề
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|:---:|:---:|:---:|
| Gọi giao diện openai | Alibaba Bailian, Zhipu ChatGLMVLLM | Zhipu ChatGLMVLLM |
| Gọi giao diện OpenAI | Alibaba Bailian, Zhipu ChatGLMVLLM | Zhipu ChatGLMVLLM |
Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đều có thể truy cập sử dụng.
Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện OpenAI đều có thể truy cập sử dụng.
---
@@ -280,8 +281,8 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|:---:|:---:|:---:|
| Gọi giao diện | EdgeTTS, iFlytek, Volcano Engine, Tencent Cloud, Alibaba Cloud và Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, GizwitsTTS, ACGNTTS, OpenAITTS, Lingxi streaming TTS, MinimaxTTS | Lingxi streaming TTS, EdgeTTS, CosyVoiceSiliconflow(một phần) |
| Dịch vụ cục bộ | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, Index-TTS, PaddleSpeech | Index-TTS, PaddleSpeech, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
| Gọi giao diện | EdgeTTS, iFLYTEK, Volcano Engine, Tencent Cloud, Alibaba Cloud và Bailian, CosyVoiceSiliconflow, TTS302AI, CozeCnTTS, CustomTTS, OpenAITTS, MinimaxTTS | EdgeTTS, CosyVoiceSiliconflow(một phần) |
| Dịch vụ cục bộ | FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3, IndexStreamTTS, PaddleSpeechTTS | IndexStreamTTS, PaddleSpeechTTS, FishSpeech, GPT_SOVITS_V2, GPT_SOVITS_V3 |
---
@@ -297,8 +298,8 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|:---:|:---:|:---:|
| Sử dụng cục bộ | FunASR, SherpaASR | FunASR, SherpaASR |
| Gọi giao diện | FunASRServer, Volcano Engine, iFlytek, Tencent Cloud, Alibaba Cloud, Baidu Cloud, OpenAI ASR | FunASRServer |
| Sử dụng cục bộ | FunASR, SherpaASR, SherpaParaformerASR, VoskASR | FunASR, SherpaASR, SherpaParaformerASR, VoskASR |
| Gọi giao diện | FunASRServer, Volcano Engine, iFLYTEK, Tencent Cloud, Alibaba Cloud và Bailian, Baidu Cloud, OpenAI ASR, GroqASR, Qwen3ASRFlash | FunASRServer |
---
@@ -306,7 +307,9 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
| Phương pháp sử dụng | Nền tảng hỗ trợ | Nền tảng miễn phí |
|:---:|:---:|:---:|
| Sử dụng cục bộ | 3D-Speaker | 3D-Speaker |
| Gọi giao diện | Dịch vụ nhận dạng vân giọng (voiceprint-api) | - |
Nhận dạng vân giọng cần dịch vụ [voiceprint-api](https://github.com/xinnan-tech/voiceprint-api) được triển khai riêng (dựa trên 3D-Speaker) và MySQL. Xem [hướng dẫn tích hợp voiceprint](../voiceprint-integration.md).
---
@@ -352,11 +355,10 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
| <img src="../images/logo_contributors.png" width="160"> | [Người đóng góp mã](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Cảm ơn [tất cả người đóng góp mã](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), sự cống hiến của bạn khiến dự án mạnh mẽ và vững chắc hơn. |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
</picture>
</a>
+10 -4
View File
@@ -16,7 +16,7 @@
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`voiceprint-api-main`
你需要把它重命名成`voiceprint-api`。
## 第二步, 创建数据库和表
## 第二步, 确认 MySQL 能访问
声纹识别需要依赖`mysql`数据库。如果你之前已经部署`智控台`,说明你已经安装了`mysql`。你可以共用它。
@@ -39,7 +39,7 @@ telnet 127.0.0.1 3306
networks:
- default
expose:
- "3306:3306"
- 3306
```
修改后
@@ -58,8 +58,9 @@ telnet 127.0.0.1 3306
# 进入你docker-compose_all.yml所在的文件夹,例如我的是xiaozhi-server
cd xiaozhi-server
docker compose -f docker-compose_all.yml down
docker compose -f docker-compose.yml up -d
docker compose -f docker-compose_all.yml up -d
```
数据库服务`xiaozhi-esp32-server-db`在`docker-compose_all.yml`里,最小化的`docker-compose.yml`只有服务本身,所以两条命令都用`docker-compose_all.yml`。
启动完后,在宿主机再使用`telnet`命令,看看能不能正常访问`mysql`的`3306`端口。
```
@@ -171,6 +172,8 @@ http://192.168.1.25:8005/voiceprint/health?key=abcd
然后搜索参数`server.voice_print`,此时,它的值应该是`null`值。
点击修改按钮,把上一步得来的`声纹接口地址`粘贴到`参数值`里。然后保存。
声纹识别的相似度阈值是另一个系统参数`server.voiceprint_similarity_threshold`(默认0.4,范围0.0-1.0,数值越高越严格),要调整就在同一个`参数管理`页面里改。
如果能保存成功,说明一切顺利,你可以去智能体查看效果了。如果不成功,说明智控台无法访问声纹识别,很大概率是网络防火墙,或者没有填写正确的局域网ip。
## 第二步 设置智能体记忆模式
@@ -186,7 +189,7 @@ http://192.168.1.25:8005/voiceprint/health?key=abcd
在智控台,`智能体管理`页面,在智能体的面板里,有一个`声纹识别`按钮,点击它。在底部有一个`新增按钮`。就可以对某个人说的话进行声纹注册。
在弹出的框里,`描述`这个属性建议填写上,可以是这个人的职业、性格、爱好。方便智能体对说话人进行分析和了解。
## 第三步 和你的智能体聊天
## 第五步 验证效果
将你的设备通电,问它,你知道我是谁吗?如果他能回答得出,说明声纹识别功能正常。
@@ -205,6 +208,9 @@ voiceprint:
- "test1,张三,张三是一个程序员"
- "test2,李四,李四是一个产品经理"
- "test3,王五,王五是一个设计师"
# 声纹识别相似度阈值,范围0.0-1.0,默认0.4
# 数值越高越严格,减少误识别但可能增加拒识率
similarity_threshold: 0.4
```
把上一步得来的 `声纹接口地址` 粘贴到 `url` 里。然后保存。
+12 -4
View File
@@ -2,14 +2,15 @@
## 功能简介
联网搜索插件 `web_search` 支持在对话过程中实时联网搜索信息并返回结果。插件支持三个搜索源:秘塔(Metaso)、Tavily和Serply,用户可根据需要选择其中一个。
联网搜索插件 `web_search` 支持在对话过程中实时联网搜索信息并返回结果。插件支持四个搜索源:秘塔(Metaso)、Tavily、Serply和Outcrawl,用户可根据需要选择其中一个。
## API Key申请指南
目前我们适配了`秘塔搜索`、`Tavily搜索`和`Serply搜索`。
目前我们适配了`秘塔搜索`、`Tavily搜索`、`Serply搜索`和`Outcrawl搜索`。
- Tavily搜索:每个月1000次免费额度。
- 秘塔搜索:拥有较为优质的国内数据源。
- Serply搜索:返回谷歌搜索结果,注册赠送2500次免费额度,额度用完后需要付费。
- Outcrawl搜索:面向AI智能体的网页搜索接口,每月1000 credits免费额度。
## API Key申请指南
@@ -32,6 +33,13 @@
- 复制生成的API Key,这是配置所需的关键信息
- 接口说明见 [Serply文档](https://serply.io/docs)
### 方式四:使用Outcrawl搜索
- 访问 [Outcrawl](https://outcrawl.ai),注册并登录账号
- 在控制台中创建API Key
- 复制生成的API Key,这是配置所需的关键信息
- 接口说明见 [Outcrawl文档](https://outcrawl.ai/docs/search/overview)
## 配置方式
### 方式1. 使用智控台部署(推荐)
@@ -40,14 +48,14 @@
- 进入"配置角色"页面,选择要配置的智能体
- 点击"编辑功能"按钮,在右侧参数配置区域找到"联网搜索"插件
- 勾选"联网搜索"
- 填入搜索源(`metaso`、`tavily`或`serply`),并将对应的`API Key`填入配置项
- 填入搜索源(`metaso`、`tavily`、`serply`或`outcrawl`),并将对应的`API Key`填入配置项
- 保存配置,再保存智能体配置
### 方式2. 单模块xiaozhi-server部署
在 `data/.config.yaml` 中配置:
- 将搜索源填入 `provider`,可选值为 `metaso`、`tavily` 或 `serply`
- 将搜索源填入 `provider`,可选值为 `metaso`、`tavily`、`serply` 或 `outcrawl`
- 将申请到的API Key填入 `api_key`
```yaml
-29
View File
@@ -30,35 +30,6 @@ python start.py
- 事件桥地址:ws://127.0.0.1:8006/wakeword-ws
- 健康检查:http://127.0.0.1:8006/health
# 开启认证后的用法
服务端 `config.yaml` 里 `server.auth.enabled` 设为 `true` 时,本页面需要额外的凭证才能连上:
1. **激活码**:向服务端运维要一个码,填进页面的「激活码」输入框。页面请求 OTA 时带上它,服务端换发一次 websocket token,用掉即失效。
2. **设备MAC**:填无分隔符或带冒号的 12 位 MAC 都行,页面会统一去掉分隔符转小写后再发给服务端。OTA 请求头和 websocket 子协议用的是同一个值,签名才对得上。
激活码只能用一次,页面拿到 token 后会自己存着,之后每次 OTA 都用 token 续签,不用再填码。
websocket 的凭证走子协议头而不是 URL:`Sec-WebSocket-Protocol: xiaozhi, t-<token>, d-<deviceId>, c-<clientId>`。浏览器不能自定义 websocket 请求头,服务端握手时只回显 `xiaozhi`。放进 URL 查询串的凭证不再被采信,理由是查询串会原样落进反代访问日志。
## 跨域
服务端默认不下发跨域响应头,页面从别的来源打开时会被浏览器拦掉。跨域失败在浏览器里读不到状态码,只会在控制台报一句不透明的错,所以要让服务端把页面的来源加进白名单:
`http://127.0.0.1:8006` 和 `http://localhost:8006` 是两个不同来源,按上面哪个地址打开就得填哪个,两个都填最省事:
```yaml
server:
cors:
allowed_origins:
- http://127.0.0.1:8006
- http://localhost:8006
```
`main/digital-human` 里的页面是用 `python start.py` 起 HTTP 服务打开的,不是双击文件;用 `file://` 打开时页面会自己弹窗挡住整页,来源是 `null` 也不必往名单里填。
不开启认证时以上都不用填,行为与之前一致。
# 目录说明
- `start.py`:模块启动入口
+1 -7
View File
@@ -155,14 +155,8 @@
<div class="config-row">
<div class="config-item">
<label for="clientId">客户端ID:</label>
<input type="text" id="clientId" placeholder="client-id(接口要求填写,服务端不校验)">
<input type="text" id="clientId" value="web_test_client" placeholder="client-id">
</div>
<div class="config-item">
<label for="activationCode">激活码:</label>
<input type="text" id="activationCode" placeholder="仅 auth.enabled: true 时需要">
</div>
</div>
<div class="config-row">
<div class="config-item">
<label for="deviceName">设备名称:</label>
<input type="text" id="deviceName" value="Web测试设备" maxlength="50"
+2 -6
View File
@@ -4,7 +4,6 @@ import { getAudioPlayer } from './core/audio/player.js?v=0205';
import { checkMicrophoneAvailability, isHttpNonLocalhost } from './core/audio/recorder.js?v=0205';
import { initMcpTools } from './core/mcp/tools.js?v=0205';
import { startWakewordBridgeListener } from './core/network/wakeword-bridge.js?v=0205';
import { normalizeDeviceId } from './core/network/ota-connector.js?v=0205';
import { uiController } from './ui/controller.js?v=0205';
import { log } from './utils/logger.js?v=0205';
@@ -304,11 +303,8 @@ class App {
log(`正在发送图片到视觉分析接口: ${url}`, 'info');
const deviceId = normalizeDeviceId(document.getElementById('deviceMac')?.value || '');
const clientId = document.getElementById('clientId')?.value || '';
if (!deviceId || !clientId) {
throw new Error('视觉分析失败:请填写设备MAC与客户端ID');
}
const deviceId = document.getElementById('deviceMac')?.value || '';
const clientId = document.getElementById('clientId')?.value || 'web_test_client';
const formData = new FormData();
formData.append('question', question);
-22
View File
@@ -72,18 +72,6 @@ export function loadConfig() {
if (savedEmojiEnabled !== null && emojiEnabledInput) {
emojiEnabledInput.value = savedEmojiEnabled;
}
// 激活码和 token 要读回来,否则刷新页面就断了:
// 服务端激活码用掉即作废,读不回来就只能重新填一个
const activationCodeInput = document.getElementById('activationCode');
if (activationCodeInput) {
activationCodeInput.value = localStorage.getItem('xz_tester_activationCode') || '';
}
// token 是 OTA 下发回来的续签凭证,比激活码长效,放着下次直接用
const token = localStorage.getItem('xz_tester_token');
if (token) {
sessionStorage.setItem('xz_tester_token', token);
}
}
// 保存配置
@@ -98,10 +86,6 @@ export function saveConfig() {
localStorage.setItem('xz_tester_deviceMac', deviceMacInput.value);
localStorage.setItem('xz_tester_deviceName', deviceNameInput.value);
localStorage.setItem('xz_tester_clientId', clientIdInput.value);
const activationCodeInput = document.getElementById('activationCode');
if (activationCodeInput) {
localStorage.setItem('xz_tester_activationCode', activationCodeInput.value.trim());
}
const emojiEnabledInput = document.getElementById('emojiEnabled');
if (emojiEnabledInput) {
localStorage.setItem('xz_tester_emojiEnabled', emojiEnabledInput.value);
@@ -123,8 +107,6 @@ export function getConfig() {
const deviceMac = document.getElementById('deviceMac')?.value.trim() || '';
const deviceName = document.getElementById('deviceName')?.value.trim() || '';
const clientId = document.getElementById('clientId')?.value.trim() || '';
const activationCode = document.getElementById('activationCode')?.value.trim() || '';
const token = sessionStorage.getItem('xz_tester_token') || '';
const emojiEnabled = document.getElementById('emojiEnabled')?.value !== 'false';
return {
@@ -132,10 +114,6 @@ export function getConfig() {
deviceName,
deviceMac,
clientId,
activationCode,
// 有 token 就用它续签,激活码是一次性的,用掉即失效,
// 第二次再拿激活码去 OTA 会被拒
token,
emojiEnabled
};
}
@@ -1,13 +1,5 @@
import { log } from '../../utils/logger.js?v=0205';
// 子协议值必须是合法的 token 字符,冒号不在其中,所以 Device-Id 去掉分隔符。
// OTA 请求头与 websocket 子协议必须用同一个值,否则 token 签名对不上。
// 方括号里横线必须写在最前面,写成 [:-_] 会当成 : 到 _ 的区间,
// 连大写 A-Z 一起吃掉,大写 MAC 会被算成空串
export function normalizeDeviceId(deviceId) {
return (deviceId || '').replace(/[-:_]/g, '').toLowerCase();
}
// WebSocket 连接
export async function webSocketConnect(otaUrl, config) {
@@ -29,33 +21,22 @@ export async function webSocketConnect(otaUrl, config) {
return;
}
// token 存下来下次续签用:激活码用掉即作废,
// 不存的话第二次点连接必然被服务端拒。
// 响应里 websocket 和 mqtt 是二选一,带 mqtt 的响应走不到这里
const issuedToken = (websocket.token || '').replace(/^Bearer\s+/i, '');
if (issuedToken) {
sessionStorage.setItem('xz_tester_token', issuedToken);
localStorage.setItem('xz_tester_token', issuedToken);
}
// 使用OTA返回的websocket URL
const connUrl = new URL(websocket.url);
let connUrl = new URL(websocket.url);
// 浏览器不能自定义 websocket 请求头,凭证和设备标识改走子协议头:
// xiaozhi, t-<token>, d-<deviceId>, c-<clientId>,服务端只回显 xiaozhi。
// 不再拼进 URL 查询串,那会原样落进反代访问日志
const subprotocols = [
'xiaozhi',
`d-${normalizeDeviceId(config.deviceId)}`,
`c-${config.clientId}`
];
// 添加token参数(从OTA响应中获取)
if (websocket.token) {
const token = websocket.token.startsWith('Bearer ')
? websocket.token.slice(7)
: websocket.token;
subprotocols.splice(1, 0, `t-${token}`);
if (websocket.token.startsWith("Bearer ")) {
connUrl.searchParams.append('authorization', websocket.token);
} else {
connUrl.searchParams.append('authorization', 'Bearer ' + websocket.token);
}
}
// 添加认证参数(保持原有逻辑)
connUrl.searchParams.append('device-id', config.deviceId);
connUrl.searchParams.append('client-id', config.clientId);
const wsurl = connUrl.toString()
log(`正在连接: ${wsurl}`, 'info');
@@ -64,7 +45,7 @@ export async function webSocketConnect(otaUrl, config) {
document.getElementById('serverUrl').value = wsurl;
}
return new WebSocket(connUrl.toString(), subprotocols);
return new WebSocket(connUrl.toString());
}
// 验证配置
@@ -83,15 +64,12 @@ function validateConfig(config) {
// OTA发送请求,验证状态,并返回响应数据
async function sendOTA(otaUrl, config) {
try {
// 有 token 用 token 续签,没有才用激活码换首次 token
const credential = config.token || config.activationCode;
const res = await fetch(otaUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Device-Id': normalizeDeviceId(config.deviceId),
'Client-Id': config.clientId,
...(credential ? { 'Authorization': `Bearer ${credential}` } : {})
'Device-Id': config.deviceId,
'Client-Id': config.clientId
},
body: JSON.stringify({
version: 0,
@@ -126,14 +104,6 @@ async function sendOTA(otaUrl, config) {
const result = await res.json();
return result; // 返回完整的响应数据
} catch (err) {
// 跨域失败在浏览器里是不透明的:fetch 直接抛 TypeError,
// 拿不到状态码,也看不出是服务端没回跨域头还是服务没起来。
// 页面知道自己的来源,直接点名让人去加白名单
if (err instanceof TypeError) {
log(`请求 ${otaUrl} 被浏览器拦下(跨域)。把 ${window.location.origin} 加到服务端的 server.cors.allowed_origins 里`, 'error');
} else {
log(`OTA请求失败: ${err.message}`, 'error');
}
return null; // 失败返回null
}
}
+26 -1
View File
@@ -9,10 +9,35 @@ manager-api 该项目基于SpringBoot框架开发。
# 开发环境
JDK 21
Maven 3.8+
MySQL 8.0.19+(迁移脚本 202610231200.sql 用了 WITH ... UPDATE 改写,8.0.19 才有;低版本会整段回滚导致服务起不来)
MySQL 8.0+
Redis 5.0+
Vue 3.x
# 接口文档
启动后打开:http://localhost:8002/xiaozhi/doc.html
# 跑测试
测试分两种:纯单元测试,和要真连 MySQL、Redis 的集成测试(`DeviceTest`、`loginControllerTest`、`ModelConfigPersistenceTest`)。
集成测试需要先起好 MySQL 8 和 Redis 8。没起的时候这类测试整类跳过,不会让构建变红,日志里会写明缺的是哪个依赖。CI 上不一样:依赖连不上就直接让构建失败,不许跳过,免得门禁被悄悄掏空。
本地用 Docker 起这两个依赖:
```bash
docker run -d --name xiaozhi-mysql -p 3306:3306 \
-e MYSQL_ROOT_PASSWORD=123456 -e MYSQL_DATABASE=xiaozhi_esp32_server mysql:8.0
docker run -d --name xiaozhi-redis -p 6379:6379 redis:8.0
```
跑全部测试:
```bash
mvn -B test -DskipTests=false
```
只想编译、不跑测试:
```bash
mvn -B package -DskipTests=true
```
+1 -1
View File
@@ -32,7 +32,7 @@
<liquibase-core.version>4.20.0</liquibase-core.version>
<aliyun-sms-version>4.1.0</aliyun-sms-version>
<okio-version>3.4.0</okio-version>
<skipTests>true</skipTests>
<skipTests>false</skipTests>
<argLine></argLine>
</properties>
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# 检查那几个要连 MySQL 和 Redis 的集成测试是不是真的跑了。
# 守卫要是哪天手滑把它们全跳过了,CI 必须红,不能安安静静少跑一半门禁。
set -uo pipefail
REPORT_DIR="${1:-target/surefire-reports}"
WATCHED=(
"xiaozhi.modules.device.DeviceTest"
"xiaozhi.modules.model.service.impl.ModelConfigPersistenceTest"
"xiaozhi.modules.sys.loginControllerTest"
)
if [ ! -d "$REPORT_DIR" ]; then
echo "找不到测试报告目录:$REPORT_DIR"
exit 1
fi
missing=()
for class in "${WATCHED[@]}"; do
count=0
for report in "$REPORT_DIR"/TEST-*.xml; do
[ -e "$report" ] || continue
name=$(sed -n 's/.*<testsuite[^>]*name="\([^"]*\)".*/\1/p' "$report" | head -1)
if [ "$name" = "$class" ]; then
count=$(sed -n 's/.*<testsuite[^>]*tests="\([0-9]*\)".*/\1/p' "$report" | head -1)
count=${count:-0}
break
fi
done
if [ "$count" -eq 0 ]; then
missing+=("$class")
else
echo "OK $class 跑了 $count 条"
fi
done
if [ ${#missing[@]} -ne 0 ]; then
echo "这几个集成测试一条都没跑:${missing[*]}"
echo "守卫把门禁掏空了,构建必须失败"
exit 1
fi
echo "集成测试都跑过了"
@@ -324,7 +324,7 @@ public interface Constant {
/**
* 版本号
*/
public static final String VERSION = "0.9.7";
public static final String VERSION = "0.9.8";
/**
* 无效固件URL
@@ -19,8 +19,6 @@ import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.Parameters;
@@ -44,7 +42,6 @@ import xiaozhi.modules.agent.dto.AgentDTO;
import xiaozhi.modules.agent.dto.AgentMemoryDTO;
import xiaozhi.modules.agent.dto.AgentUpdateDTO;
import xiaozhi.modules.agent.entity.AgentEntity;
import xiaozhi.modules.agent.entity.AgentTemplateEntity;
import xiaozhi.modules.agent.dto.AgentTagDTO;
import xiaozhi.modules.agent.entity.AgentTagEntity;
import xiaozhi.modules.agent.service.AgentTagService;
@@ -52,7 +49,6 @@ import xiaozhi.modules.agent.service.AgentChatAudioService;
import xiaozhi.modules.agent.service.AgentChatHistoryService;
import xiaozhi.modules.agent.service.AgentChatSummaryService;
import xiaozhi.modules.agent.service.AgentService;
import xiaozhi.modules.agent.service.AgentTemplateService;
import xiaozhi.modules.agent.vo.AgentChatHistoryUserVO;
import xiaozhi.modules.agent.vo.AgentInfoVO;
import xiaozhi.modules.security.user.SecurityUser;
@@ -65,7 +61,6 @@ public class AgentController {
private static final long AUDIO_PLAY_TOKEN_EXPIRE_SECONDS = 300L;
private final AgentService agentService;
private final AgentTemplateService agentTemplateService;
private final AgentChatHistoryService agentChatHistoryService;
private final AgentChatAudioService agentChatAudioService;
private final AgentChatSummaryService agentChatSummaryService;
@@ -192,15 +187,6 @@ public class AgentController {
return new Result<>();
}
@GetMapping("/template")
@Operation(summary = "智能体模板模板列表")
@RequiresPermissions("sys:role:normal")
public Result<List<AgentTemplateEntity>> templateList() {
List<AgentTemplateEntity> list = agentTemplateService
.list(new QueryWrapper<AgentTemplateEntity>().orderByAsc("sort"));
return new Result<List<AgentTemplateEntity>>().ok(list);
}
@GetMapping("/{id}/sessions")
@Operation(summary = "获取智能体会话列表")
@RequiresPermissions("sys:role:normal")
@@ -41,6 +41,17 @@ public class AgentTemplateController {
private final AgentTemplateService agentTemplateService;
// 单独用 normal 权限(而非类级 superAdmin)是有意保留:
// 普通用户也需查询可用模板,与 /page 的管理入口分离,保持原行为契约。
@GetMapping
@Operation(summary = "智能体模板列表")
@RequiresPermissions("sys:role:normal")
public Result<List<AgentTemplateEntity>> getAgentTemplates() {
List<AgentTemplateEntity> list = agentTemplateService
.list(new QueryWrapper<AgentTemplateEntity>().orderByAsc("sort"));
return new Result<List<AgentTemplateEntity>>().ok(list);
}
@GetMapping("/page")
@Operation(summary = "获取模板分页列表")
@RequiresPermissions("sys:role:superAdmin")
@@ -174,6 +174,9 @@ public class DeviceController {
@Operation(summary = "获取设备通讯录")
@RequiresPermissions("sys:role:normal")
public Result<Object> getAddressBook(@PathVariable String macAddress) {
if (deviceService.getOwnedDeviceByMacAddress(macAddress) == null) {
return new Result<Object>().error(ErrorCode.DEVICE_NOT_EXIST);
}
return new Result<Object>().ok(deviceAddressBookService.getAddressBookList(macAddress));
}
@@ -192,9 +195,7 @@ public class DeviceController {
@Operation(summary = "更新设备通讯录别名")
@RequiresPermissions("sys:role:normal")
public Result<Void> updateAlias(@Valid @RequestBody DeviceAddressBookAliasDTO dto) {
UserDetail user = SecurityUser.getUser();
DeviceEntity callerDevice = deviceService.getDeviceByMacAddress(dto.getMacAddress());
if (callerDevice == null || !callerDevice.getUserId().equals(user.getId())) {
if (deviceService.getOwnedDeviceByMacAddress(dto.getMacAddress()) == null) {
return new Result<Void>().error("无权限操作该设备");
}
deviceAddressBookService.saveOrUpdate(dto.getMacAddress(), dto.getTargetMac(), dto.getAlias(), null);
@@ -205,9 +206,7 @@ public class DeviceController {
@Operation(summary = "更新设备通讯录权限")
@RequiresPermissions("sys:role:normal")
public Result<Void> updatePermission(@Valid @RequestBody DeviceAddressBookPermissionDTO dto) {
UserDetail user = SecurityUser.getUser();
DeviceEntity callerDevice = deviceService.getDeviceByMacAddress(dto.getMacAddress());
if (callerDevice == null || !callerDevice.getUserId().equals(user.getId())) {
if (deviceService.getOwnedDeviceByMacAddress(dto.getMacAddress()) == null) {
return new Result<Void>().error("无权限操作该设备");
}
deviceAddressBookService.saveOrUpdate(dto.getMacAddress(), dto.getTargetMac(), null, dto.getHasPermission());
@@ -3,7 +3,6 @@ package xiaozhi.modules.device.controller;
import java.nio.charset.StandardCharsets;
import org.apache.commons.lang3.StringUtils;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
@@ -29,7 +28,6 @@ import xiaozhi.modules.device.dto.DeviceReportReqDTO;
import xiaozhi.modules.device.dto.DeviceReportRespDTO;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
import xiaozhi.modules.device.service.impl.DeviceServiceImpl;
import xiaozhi.modules.sys.service.SysParamsService;
@Tag(name = "设备管理", description = "OTA 相关接口")
@@ -48,16 +46,15 @@ public class OTAController {
@Parameter(name = "Device-Id", description = "设备唯一标识", required = true, in = ParameterIn.HEADER) @RequestHeader("Device-Id") String deviceId,
@Parameter(name = "Client-Id", description = "客户端标识", required = false, in = ParameterIn.HEADER) @RequestHeader(value = "Client-Id", required = false) String clientId) {
if (StringUtils.isBlank(deviceId)) {
return createResponse(DeviceReportRespDTO.createError("Device ID is required"), HttpStatus.BAD_REQUEST);
return createResponse(DeviceReportRespDTO.createError("Device ID is required"));
}
deviceId = DeviceServiceImpl.normalizeDeviceId(deviceId);
if (StringUtils.isBlank(clientId)) {
clientId = deviceId;
}
boolean macAddressValid = isMacAddressValid(deviceId);
// 设备Id和Mac地址应是一致的, 并且必须需要application字段
if (!macAddressValid) {
return createResponse(DeviceReportRespDTO.createError("Invalid device ID"), HttpStatus.BAD_REQUEST);
return createResponse(DeviceReportRespDTO.createError("Invalid device ID"));
}
return createResponse(deviceService.checkDeviceActive(deviceId, clientId, deviceReportReqDTO));
}
@@ -97,25 +94,20 @@ public class OTAController {
@SneakyThrows
private ResponseEntity<String> createResponse(DeviceReportRespDTO deviceReportRespDTO) {
return createResponse(deviceReportRespDTO, HttpStatus.OK);
}
@SneakyThrows
private ResponseEntity<String> createResponse(DeviceReportRespDTO deviceReportRespDTO, HttpStatus status) {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
String json = objectMapper.writeValueAsString(deviceReportRespDTO);
byte[] jsonBytes = json.getBytes(StandardCharsets.UTF_8);
return ResponseEntity
.status(status)
.ok()
.contentType(MediaType.APPLICATION_JSON)
.contentLength(jsonBytes.length)
.body(json);
}
/**
* 判断 mac 地址是否有效,收紧为严格的十六进制 MAC 形态
*
* 简单判断mac地址是否有效(非严格)
*
* @param macAddress
* @return
*/
@@ -123,8 +115,8 @@ public class OTAController {
if (StringUtils.isBlank(macAddress)) {
return false;
}
// MAC地址为12位十六进制数字,可用冒号、连字符或下划线分隔,也接受无分隔符形态
String macPattern = "^([0-9A-Fa-f]{2}[:\\-_]){5}[0-9A-Fa-f]{2}$|^[0-9A-Fa-f]{12}$";
// MAC地址通常为12位十六进制数字,可以包含冒号或连字符分隔符
String macPattern = "^([0-9A-Za-z]{2}[:-]){5}([0-9A-Za-z]{2})$";
return macAddress.matches(macPattern);
}
}
@@ -83,6 +83,16 @@ public interface DeviceService extends BaseService<DeviceEntity> {
*/
DeviceEntity getDeviceByMacAddress(String macAddress);
/**
* 根据MAC地址获取当前登录用户名下的设备信息
* <p>
* 设备不存在或不属于当前用户时返回null,调用方无需再自行比对归属。
*
* @param macAddress MAC地址
* @return 当前用户名下的设备信息,不存在或无权限时返回null
*/
DeviceEntity getOwnedDeviceByMacAddress(String macAddress);
/**
* 根据设备ID获取激活码
*
@@ -1,9 +1,13 @@
package xiaozhi.modules.device.service.impl;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.apache.commons.lang3.StringUtils;
@@ -18,6 +22,7 @@ import xiaozhi.common.redis.RedisKeys;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.modules.device.dao.DeviceAddressBookDao;
import xiaozhi.modules.device.entity.DeviceAddressBookEntity;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceAddressBookService;
import xiaozhi.modules.device.service.DeviceService;
import xiaozhi.modules.sys.service.SysParamsService;
@@ -40,7 +45,11 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
@Override
public List<DeviceAddressBookEntity> getAddressBookList(String macAddress) {
return deviceAddressBookDao.getAddressBookList(macAddress);
List<DeviceAddressBookEntity> rows = assignUniqueAliases(deviceAddressBookDao.getAddressBookList(macAddress));
// 取名按固定顺序算完了,前端要看的是最近改过的排前面,放回去
rows.sort(Comparator
.comparing(DeviceAddressBookEntity::getUpdateDate, Comparator.nullsLast(Comparator.reverseOrder())));
return rows;
}
@Override
@@ -63,7 +72,7 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
}
// 主动呼叫模式
Map<String, String> callerBook = allBooks.get(callerMac.toLowerCase());
Map<String, String> callerBook = allBooks.get(bookKey(callerMac));
if (callerBook == null) {
return errorResult("未找到备注为'" + nickname + "'的设备");
}
@@ -80,13 +89,20 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
}
// 获取目标设备如何称呼主叫方
Map<String, String> targetBook = allBooks.get(targetMac.toLowerCase());
Map<String, String> targetBook = allBooks.get(bookKey(targetMac));
String callerNickname = null;
if (targetBook != null) {
callerNickname = targetBook.get(callerMac.toLowerCase());
callerNickname = targetBook.get(bookKey(callerMac));
}
if (StringUtils.isBlank(callerNickname)) {
callerNickname = deviceService.getDeviceByMacAddress(callerMac).getAlias();
// 主叫显示名取的是主叫方自己那台设备在设备表里的名字,不是对方的。
// 对方那边给主叫方起的称呼在上面的 targetBook 里已经查过。
// 产品上按「来电显示」的口径处理,所以这里不套通讯录那套「不抄对方私产」的限制
// 这个接口没有登录用户上下文,归属校验那个查法在这里会返回空,只能用不校验归属的查法
DeviceEntity callerDevice = deviceService.getDeviceByMacAddress(callerMac);
if (callerDevice != null) {
callerNickname = callerDevice.getAlias();
}
if (StringUtils.isBlank(callerNickname)) {
callerNickname = formatMacAsDeviceName(callerMac);
}
@@ -100,32 +116,26 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
@Override
public void refreshCache() {
Map<String, Map<String, String>> result = new HashMap<>();
List<DeviceAddressBookEntity> allRecords = deviceAddressBookDao.selectList(null);
List<DeviceAddressBookEntity> allRecords = assignUniqueAliases(deviceAddressBookDao.selectList(null));
Map<String, String> reverseMap = new HashMap<>();
for (DeviceAddressBookEntity entity : allRecords) {
String macA = entity.getMacAddress().toLowerCase();
String macB = entity.getTargetMac().toLowerCase();
String alias = entity.getAlias();
Boolean hasPermission = entity.getHasPermission();
String macA = bookKey(entity.getMacAddress());
String macB = bookKey(entity.getTargetMac());
// 构建正向映射: A对B的映射 nickname -> macB|permission
if (alias != null && !alias.isEmpty()) {
result.computeIfAbsent(macA, k -> new HashMap<>());
String permStr = (hasPermission != null && hasPermission) ? "1" : "0";
result.get(macA).put(alias, macB + "|" + permStr);
}
Map<String, String> book = result.computeIfAbsent(macA, k -> new HashMap<>());
String permStr = (Boolean.TRUE.equals(entity.getHasPermission())) ? "1" : "0";
book.put(entity.getAlias(), macB + "|" + permStr);
// 构建反向记录用于查找B对A的称呼
if (alias != null && !alias.isEmpty()) {
reverseMap.put(macB + ":" + macA, alias);
}
// 构建反向记录用于查找B对A的称呼,用同一个去重后的名字,两边才对得上
reverseMap.put(macB + ":" + macA, entity.getAlias());
}
// 构建反向映射: B对A的映射 macA -> nickname
for (DeviceAddressBookEntity entity : allRecords) {
String macA = entity.getMacAddress().toLowerCase();
String macB = entity.getTargetMac().toLowerCase();
String macA = bookKey(entity.getMacAddress());
String macB = bookKey(entity.getTargetMac());
String aliasBtoA = reverseMap.get(macA + ":" + macB);
if (aliasBtoA != null && !aliasBtoA.isEmpty()) {
result.computeIfAbsent(macB, k -> new HashMap<>());
@@ -136,6 +146,54 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
redisUtils.set(RedisKeys.getAddressBookKey(), result);
}
/**
* 给每行算出一个不重名的称呼,返回排好序、alias 已写好的新列表
*
* 昵称在同一个人的通讯录里是索引,同名后写的会盖掉先写的,按昵称呼会呼错设备。
* 库里本来就有重名:空别名补出来的尾号称呼只有 256 种取值,一本通讯录十几条
* 没起名就撞上了。所以显示和呼叫都得走这一份,
* 不然前端看到的字和呼叫时用的名字对不上,用户照着点的名字呼不到人。
* 不改传入的那个列表:它是 DAO 刚查出来的,调用方还要按自己的顺序用
*/
private List<DeviceAddressBookEntity> assignUniqueAliases(List<DeviceAddressBookEntity> rows) {
// 取名要靠先后定谁拿原名,而 selectList 不带 order by,返回顺序不保证。
// 排一次,同样的数据每次算出来的结果一样,缓存不会来回变。
// 按归一后的地址排:同一个人不管库里是哪种写法都排到一起,
// 后面两级比较器再分先后。同一本里 target_mac 不会重复(主键摆着),
// 所以顺序还是由这三个键定死的,不会掉回"跟着数据库返回顺序走"
List<DeviceAddressBookEntity> ordered = new ArrayList<>(rows);
ordered.sort(Comparator
.comparing((DeviceAddressBookEntity row) -> bookKey(row.getMacAddress()),
Comparator.nullsFirst(Comparator.naturalOrder()))
// 权限排在地址前面:撞名时让对方准许呼我的那台先拿原名。
// 排在后面等于不起作用,两条地址不同时谁先谁后是地址定的,
// 没权限那条照样占住原名,用户照前端看到的名字呼叫命中的就是它,
// 界面上只会说没权限,怎么点都不通
.thenComparing(DeviceAddressBookEntity::getHasPermission,
Comparator.nullsLast(Comparator.reverseOrder()))
.thenComparing(DeviceAddressBookEntity::getTargetMac, Comparator.nullsFirst(Comparator.naturalOrder())));
Map<String, Set<String>> takenByOwner = new HashMap<>();
for (DeviceAddressBookEntity row : ordered) {
// 建表时 alias 默认是空的,用户没起名就是空的,这是默认状态不是异常。
// 补个尾号称呼顶上,整条跳过的话接口照样返回成功,对方却再也呼不过来
String alias = StringUtils.isBlank(row.getAlias())
? formatMacAsDeviceName(row.getTargetMac())
: row.getAlias();
// 加后缀的写法跟 generateUniqueAlias 保持一致,不搞两套命名规则
Set<String> taken = takenByOwner.computeIfAbsent(bookKey(row.getMacAddress()), k -> new HashSet<>());
String uniqueAlias = alias;
int suffix = 1;
while (taken.contains(uniqueAlias)) {
uniqueAlias = alias + suffix++;
}
taken.add(uniqueAlias);
row.setAlias(uniqueAlias);
}
return ordered;
}
@Override
public void updateAlias(String macAddress, String targetMac, String alias) {
String finalAlias = generateUniqueAlias(macAddress, targetMac, alias);
@@ -154,14 +212,29 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
QueryWrapper<DeviceAddressBookEntity> wrapper = new QueryWrapper<>();
wrapper.eq("mac_address", macAddress).eq("target_mac", targetMac);
DeviceAddressBookEntity record = deviceAddressBookDao.selectOne(wrapper);
System.out.println("saveOrUpdate - mac=" + macAddress + ", targetMac=" + targetMac + ", alias=" + alias + ", record=" + (record == null ? "null" : "not null"));
if (record == null) {
DeviceAddressBookEntity entity = new DeviceAddressBookEntity();
entity.setMacAddress(macAddress);
entity.setTargetMac(targetMac);
// 如果 alias 为空,就默认使用设备名称
// 调用方没起名就退回尾号称呼。对方设备在智控台上起的名字是对方的私产,
// 抄进调用方的通讯录等于把别人的昵称漏出去,所以别人的设备一律不去设备表取名
if (StringUtils.isBlank(alias)) {
alias = deviceService.getDeviceByMacAddress(targetMac).getAlias();
// 只有给自己那台设备起名时才用设备表里的名字,那本来就是调用方自己的
if (isOwnDevice(macAddress, targetMac)) {
// 用带归属校验的那个查法。设备表里同一个 MAC 可能有两行只差大小写,
// 一次只准取一条的那种查法撞上这种数据会抛 TooManyResultsException,
// 整个保存接口带崩;按 id 取第一条,撞行也不报错
DeviceEntity ownDevice = deviceService.getOwnedDeviceByMacAddress(macAddress);
if (ownDevice != null && StringUtils.isNotBlank(ownDevice.getAlias())) {
alias = ownDevice.getAlias();
}
}
// 设备登记了但用户没起过名,alias 建表就是 null,这是默认状态
// 不是异常。这时候落个 null 进去,缓存刷新会把整条跳过,
// 接口照样返回成功,对方却永远呼不过来
if (StringUtils.isBlank(alias)) {
alias = formatMacAsDeviceName(targetMac);
}
}
// 检查重名
alias = generateUniqueAlias(macAddress, targetMac, alias);
@@ -229,11 +302,37 @@ public class DeviceAddressBookServiceImpl implements DeviceAddressBookService {
}
}
/**
* 通讯录里当主人用的地址键
*
* <p>
* 写缓存、查缓存、算称呼都靠它分组,三处必须是同一个值。少归一一次,同一个人的两行
* 会被当成两个人:各自拿到同名,写进同一本时后者盖掉前者,一台设备直接从缓存里消失。
* 库里确实能出现这种数据——表是 utf8mb4_0900_ai_ci,主键 (mac_address, target_mac)
* 大小写不敏感,owner 只差大小写而 target 不同的话两行不撞
*/
private static String bookKey(String macAddress) {
return macAddress == null ? null : macAddress.toLowerCase();
}
/**
* 目标设备是不是调用方自己那台
*
* <p>
* MAC 不区分大小写,比较前先按通讯录的键归一化,跟别处的比较口径保持一致
*/
private static boolean isOwnDevice(String macAddress, String targetMac) {
return bookKey(macAddress) != null && bookKey(macAddress).equals(bookKey(targetMac));
}
private String formatMacAsDeviceName(String mac) {
if (StringUtils.isBlank(mac) || mac.length() < 2) {
return mac;
}
String lastTwo = mac.substring(mac.length() - 2);
// MAC 本身不区分大小写,落库、查库走的都是小写化的值。
// 这里跟着小写化,不跟着的话同一个地址在两条路径上会算出两个称呼,
// 数据搬到智控台上看到的名字就叫不到人了
String lastTwo = mac.substring(mac.length() - 2).toLowerCase();
return "尾号为" + lastTwo + "的设备";
}
@@ -137,7 +137,7 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
deviceEntity.setBoard(board);
deviceEntity.setAgentId(agentId);
deviceEntity.setAppVersion(appVersion);
deviceEntity.setMacAddress(normalizeDeviceId(macAddress));
deviceEntity.setMacAddress(macAddress);
deviceEntity.setUserId(user.getId());
deviceEntity.setCreator(user.getId());
deviceEntity.setAutoUpdate(1);
@@ -198,10 +198,7 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
// 设备未绑定,则返回当前上传的固件信息(不更新)以此兼容旧固件版本
if (deviceById == null) {
DeviceReportRespDTO.Firmware firmware = new DeviceReportRespDTO.Firmware();
String currentVersion = deviceReport != null && deviceReport.getApplication() != null
? deviceReport.getApplication().getVersion()
: null;
firmware.setVersion(currentVersion);
firmware.setVersion(deviceReport.getApplication().getVersion());
firmware.setUrl(Constant.INVALID_FIRMWARE_URL);
response.setFirmware(firmware);
} else {
@@ -222,20 +219,13 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
// 检查是否启用认证并生成token
String authEnabled = sysParamsService.getValue(Constant.SERVER_AUTH_ENABLED, true);
if ("true".equalsIgnoreCase(authEnabled)) {
// 仅已登记绑定的设备签发 token;未登记的设备不返回可用凭证,
// 登记操作需登录智控台,因此不会成为任意 MAC 换取 token 的通道
if (deviceById == null) {
log.warn("设备未在智控台登记,跳过 WebSocket token 签发: {}", macAddress);
try {
// 生成token
String token = generateWebSocketToken(clientId, macAddress);
websocket.setToken(token);
} catch (Exception e) {
log.error("生成WebSocket token失败: {}", e.getMessage());
websocket.setToken("");
} else {
try {
// 生成token
String token = generateWebSocketToken(clientId, macAddress);
websocket.setToken(token);
} catch (Exception e) {
log.error("生成WebSocket token失败: {}", e.getMessage());
websocket.setToken("");
}
}
} else {
websocket.setToken("");
@@ -262,20 +252,16 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
// 从系统参数获取MQTT Gateway地址,仅在配置有效时使用
String mqttUdpConfig = sysParamsService.getValue(Constant.SERVER_MQTT_GATEWAY, true);
if (mqttUdpConfig != null && !mqttUdpConfig.equals("null") && !mqttUdpConfig.isEmpty()) {
// MQTT 配置含连接签名,与 websocket token 一样仅对已登记设备下发
if (deviceById == null) {
log.warn("设备未在智控台登记,跳过 MQTT 配置下发: {}", macAddress);
} else {
try {
String groupId = deviceById.getBoard() != null ? deviceById.getBoard() : "GID_default";
DeviceReportRespDTO.MQTT mqtt = buildMqttConfig(macAddress, groupId);
if (mqtt != null) {
mqtt.setEndpoint(mqttUdpConfig);
response.setMqtt(mqtt);
}
} catch (Exception e) {
log.error("生成MQTT配置失败: {}", e.getMessage());
try {
String groupId = deviceById != null && deviceById.getBoard() != null ? deviceById.getBoard()
: "GID_default";
DeviceReportRespDTO.MQTT mqtt = buildMqttConfig(macAddress, groupId);
if (mqtt != null) {
mqtt.setEndpoint(mqttUdpConfig);
response.setMqtt(mqtt);
}
} catch (Exception e) {
log.error("生成MQTT配置失败: {}", e.getMessage());
}
}
@@ -330,6 +316,12 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
if (device == null) {
return;
}
// 后面删通讯录那步是按 mac_address 删的,不看 user_id。
// 设备本身因为带了 user_id 条件删不掉,通讯录却会照删不误,
// 任意登录用户知道对方的设备 id 就能把对方整本通讯录清空
if (userId == null || device.getUserId() == null || !device.getUserId().equals(userId)) {
return;
}
String macAddress = device.getMacAddress();
if (StringUtils.isNotBlank(device.getAgentId())) {
// 清除智能体设备数量缓存
@@ -409,54 +401,32 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
if (StringUtils.isBlank(macAddress)) {
return null;
}
DeviceEntity device = selectByMacAddress(normalizeDeviceId(macAddress));
if (device != null) {
return device;
}
// 归一之前写进去的存量数据,原文再试一次,别让写法不同就查不到
return selectByMacAddress(macAddress);
}
private DeviceEntity selectByMacAddress(String macAddress) {
QueryWrapper<DeviceEntity> wrapper = new QueryWrapper<>();
wrapper.eq("mac_address", macAddress);
// 按 id 定序后取第一条。id 是 VARCHAR,比的是字符序而不是入库先后,
// 但主键保证同一组里 id 不重复,所以每次查询拿到的都是同一台;
// 不定序的话拿到哪台随数据库返回顺序变,称呼和缓存键跟着变
wrapper.orderByAsc("id");
// 有重复行时 selectOne 直接抛 TooManyResultsException,报活和激活两条路一起崩,
// 库里还看不出发生过什么。迁移对撞车的行整组不动,那种库会同时存在两行,
// 取第一条让设备读得出来;两行归谁所有是人的判断,脚本不替人定
List<DeviceEntity> devices = baseDao.selectList(wrapper);
return devices.isEmpty() ? null : devices.get(0);
return baseDao.selectOne(wrapper);
}
/**
* 把 Device-Id 归一成小写冒号分隔的 MAC 形态
* <p>
* 入库、查询、签名、订阅地址、缓存键统一走这个值,否则同一台设备换个
* 分隔符或大小写就会被当成没登记,表现为静默不发凭证也不下发固件。
* 不是合法 MAC 时原样返回,交给后续校验拒绝。
*
* @param deviceId 设备上报或用户填写的 Device-Id
* @return 归一后的 Device-Id
*/
public static String normalizeDeviceId(String deviceId) {
if (StringUtils.isBlank(deviceId)) {
return deviceId;
@Override
public DeviceEntity getOwnedDeviceByMacAddress(String macAddress) {
if (StringUtils.isBlank(macAddress)) {
return null;
}
String hex = deviceId.toLowerCase().replace(":", "").replace("-", "").replace("_", "");
if (hex.length() != 12) {
return deviceId;
// 同一 MAC 可能有两行只差大小写,selectOne 会抛 TooManyResultsException
// 把整个读通讯录接口带崩。按 id 取第一条做归属校验,撞行不报错
QueryWrapper<DeviceEntity> wrapper = new QueryWrapper<>();
wrapper.eq("mac_address", macAddress).orderByAsc("id");
List<DeviceEntity> devices = baseDao.selectList(wrapper);
if (devices == null || devices.isEmpty()) {
return null;
}
StringBuilder mac = new StringBuilder(17);
for (int i = 0; i < 12; i += 2) {
if (i > 0) {
mac.append(':');
}
mac.append(hex, i, i + 2);
DeviceEntity device = devices.get(0);
UserDetail user = SecurityUser.getUser();
// getUser() 在取不到 subject、principal 时返回的是空对象,不是 null,
// 所以这里不会拿到 null。匿名调用走的就是空对象那一条,id 是空的,比不上
if (device.getUserId() == null || !device.getUserId().equals(user.getId())) {
return null;
}
return mac.toString();
return device;
}
private DeviceReportRespDTO.ServerTime buildServerTime() {
@@ -608,22 +578,21 @@ public class DeviceServiceImpl extends BaseServiceImpl<DeviceDao, DeviceEntity>
@Override
public void manualAddDevice(Long userId, DeviceManualAddDTO dto) {
String macAddress = normalizeDeviceId(dto.getMacAddress());
// 检查mac是否已存在
QueryWrapper<DeviceEntity> wrapper = new QueryWrapper<>();
wrapper.eq("mac_address", macAddress);
wrapper.eq("mac_address", dto.getMacAddress());
DeviceEntity exist = baseDao.selectOne(wrapper);
if (exist != null) {
throw new RenException(ErrorCode.MAC_ADDRESS_ALREADY_EXISTS);
}
Date now = new Date();
DeviceEntity entity = new DeviceEntity();
entity.setId(macAddress);
entity.setId(dto.getMacAddress());
entity.setUserId(userId);
entity.setAgentId(dto.getAgentId());
entity.setBoard(dto.getBoard());
entity.setAppVersion(dto.getAppVersion());
entity.setMacAddress(macAddress);
entity.setMacAddress(dto.getMacAddress());
entity.setCreateDate(now);
entity.setUpdateDate(now);
entity.setLastConnectedAt(now);
@@ -110,8 +110,8 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
// 1. 参数验证
validateEditParameters(modelType, provideCode, id, modelConfigBodyDTO);
// 2. 验证模型提供者
validateModelProvider(modelType, provideCode);
// 2. 验证模型提供者,使用注册类型的大小写
String registeredModelType = resolveModelType(modelType, provideCode);
// 3. 获取原始配置(不经过敏感数据处理)
ModelConfigEntity originalEntity = getOriginalConfigFromDb(id);
@@ -120,7 +120,7 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
validateLlmConfiguration(modelConfigBodyDTO);
// 5. 准备更新实体并处理敏感数据
ModelConfigEntity modelConfigEntity = prepareUpdateEntity(modelConfigBodyDTO, originalEntity, modelType, id);
ModelConfigEntity modelConfigEntity = prepareUpdateEntity(modelConfigBodyDTO, originalEntity, registeredModelType, id);
// 6. 执行数据库更新
modelConfigDao.updateById(modelConfigEntity);
@@ -136,9 +136,9 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
public ModelConfigDTO add(String modelType, String provideCode, ModelConfigBodyDTO modelConfigBodyDTO) {
validateAddParameters(modelType, provideCode, modelConfigBodyDTO);
validateModelProvider(modelType, provideCode);
String registeredModelType = resolveModelType(modelType, provideCode);
ModelConfigEntity modelConfigEntity = prepareAddEntity(modelConfigBodyDTO, modelType);
ModelConfigEntity modelConfigEntity = prepareAddEntity(modelConfigBodyDTO, registeredModelType);
modelConfigDao.insert(modelConfigEntity);
@@ -283,13 +283,14 @@ public class ModelConfigServiceImpl extends BaseServiceImpl<ModelConfigDao, Mode
}
/**
* 验证模型提供者
* 验证模型提供者并获取注册类型,保留 TTS、Memory、Intent 等类型的大小写
*/
private void validateModelProvider(String modelType, String provideCode) {
private String resolveModelType(String modelType, String provideCode) {
List<ModelProviderDTO> providerList = modelProviderService.getList(modelType, provideCode);
if (CollectionUtil.isEmpty(providerList)) {
throw new RenException(ErrorCode.MODEL_PROVIDER_NOT_EXIST);
}
return providerList.get(0).getModelType();
}
/**
@@ -86,6 +86,7 @@ public class ShiroConfig {
filterMap.put("/user/captcha", "anon");
filterMap.put("/user/smsVerification", "anon");
filterMap.put("/user/login", "anon");
filterMap.put("/user/refresh", "anon");
filterMap.put("/user/pub-config", "anon");
filterMap.put("/user/register", "anon");
filterMap.put("/user/retrieve-password", "anon");
@@ -16,6 +16,7 @@ import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@@ -110,6 +111,17 @@ public class LoginController {
return sysUserTokenService.createToken(userDTO.getId());
}
@PostMapping("/refresh")
@Operation(summary = "刷新token:用旧token换新token")
public Result<TokenDTO> refresh(HttpServletRequest request) {
String authorization = request.getHeader(Constant.AUTHORIZATION);
if (StringUtils.isBlank(authorization) || !authorization.startsWith("Bearer ")) {
throw new RenException(ErrorCode.UNAUTHORIZED);
}
String oldToken = authorization.replace("Bearer ", "");
return sysUserTokenService.refreshToken(oldToken);
}
@PostMapping("/register")
@Operation(summary = "注册")
public Result<Void> register(@RequestBody LoginDTO login) {
@@ -21,4 +21,14 @@ public interface SysUserTokenDao extends BaseDao<SysUserTokenEntity> {
SysUserTokenEntity getByUserId(Long userId);
void logout(@Param("userId") Long userId, @Param("expireDate") Date expireDate);
/**
* 用旧 token 换新 token 的 CAS 更新:仅当旧 token 未过期且属于指定 user 时才覆盖。
* 返回受影响行数(0 表示条件不满足 = token 已被并发刷新或过期)。
*/
int refreshTokenCAS(@Param("userId") Long userId,
@Param("oldToken") String oldToken,
@Param("newToken") String newToken,
@Param("newExpire") Date newExpire,
@Param("updateDate") Date updateDate);
}
@@ -21,6 +21,13 @@ public interface SysUserTokenService extends BaseService<SysUserTokenEntity> {
*/
Result<TokenDTO> createToken(Long userId);
/**
* 用旧token换新token,旧token立即失效
*
* @param oldToken 旧token字符串
*/
Result<TokenDTO> refreshToken(String oldToken);
SysUserDTO getUserByToken(String token);
/**
@@ -2,7 +2,9 @@ package xiaozhi.modules.security.service.impl;
import java.util.Date;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import cn.hutool.core.date.DateUtil;
import lombok.AllArgsConstructor;
@@ -81,6 +83,36 @@ public class SysUserTokenServiceImpl extends BaseServiceImpl<SysUserTokenDao, Sy
return new Result<TokenDTO>().ok(tokenDTO);
}
@Override
@Transactional
public Result<TokenDTO> refreshToken(String oldToken) {
if (StringUtils.isBlank(oldToken)) {
throw new RenException(ErrorCode.UNAUTHORIZED);
}
SysUserTokenEntity tokenEntity = baseDao.getByToken(oldToken);
if (tokenEntity == null) {
throw new RenException(ErrorCode.TOKEN_INVALID);
}
if (tokenEntity.getExpireDate().before(new Date())) {
throw new RenException(ErrorCode.UNAUTHORIZED);
}
// CAS 更新:并发场景下只有一个请求能匹配到该 oldToken,另一个返 0 行
String newToken = TokenGenerator.generateValue();
Date now = new Date();
Date expireTime = new Date(now.getTime() + EXPIRE * 1000L);
int affected = baseDao.refreshTokenCAS(tokenEntity.getUserId(), oldToken, newToken, expireTime, now);
if (affected == 0) {
throw new RenException(ErrorCode.TOKEN_INVALID);
}
TokenDTO dto = new TokenDTO();
dto.setToken(newToken);
dto.setExpire(EXPIRE);
dto.setClientHash(HttpContextUtils.getClientCode());
return new Result<TokenDTO>().ok(dto);
}
@Override
public SysUserDTO getUserByToken(String token) {
SysUserTokenEntity userToken = baseDao.getByToken(token);
@@ -1,132 +0,0 @@
-- 设备 MAC 归一:改成小写冒号分隔,与 OTA 上报归一后的写法对齐
--
-- 之前用户在智控台手填的 MAC 原样入库(可能是 AA-BB-… 、aabbcc… 或全大写),
-- OTA 侧归一后按 aa:bb:… 精确查库,查不到就静默不发凭证、不下发固件。
-- 这里把存量刷成归一后的形态。
--
-- 只改 mac_address,不动 id:手填设备的 id 就是当时的原始 MAC,而 ai_chat_history
-- 等表按 device_id 存着对话记录,改 id 会牵动这些外键和缓存键,风险大于收益。
-- 查不到归一值时查询侧会拿原文再试一次。
--
-- 需要 MySQL 8.0.19 及以上:下面三段都是 WITH ... UPDATE,这个写法 8.0.19 才有。
-- 低版本上第一段就是语法错,三段又都在一个事务里,语法错会整体回滚,
-- manager-api 起不来。README 的环境要求写的是同一个下限。
--
-- ⚠️ 三段都只刷「归一后全表只剩它一行」的那种,这不是保守,是必须。
--
-- 同一台设备被填过两种写法(AA-BB-CC-DD-EE-FF 和 aabbccddeeff)时,两行会归一成
-- 同一个值,全刷就撞了。而两段撞了的后果完全不一样,方向是反的:
--
-- ai_device:mac_address 上只有普通索引不是唯一键,撞了数据库一声不吭。
-- 两行一模一样存下来,之后 getDeviceByMacAddress 的 selectOne 命中多行直接抛
-- TooManyResultsException。这台设备之后彻底读不出来,报活和激活两条路全崩,
-- 库里没有任何东西能看出发生过什么,用户只看到一个错误响应。
--
-- ai_device_address_book:主键是 (mac_address, target_mac),撞上会报 Duplicate
-- entry,整个 changeset 回滚,服务起不来。吵,但看得见。
--
-- 也就是没有 DB 保护的那段才会静默坏数据。下面每段的 cnt.n = 1 就是这个保护:
-- 刷完不会出现两行同值。撞上的行一个都不动,保持迁移前的状态——写法不同就 eq 不到,
-- 用户看得见、也还能重新添加,manualAddDevice 现在入库前会归一,重新添加会补上一行
-- 归一后的。
--
-- ── ① ai_device.mac_address ────────────────────────────────────────────
WITH norm AS (
SELECT `id`, LOWER(CONCAT(SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 1, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 3, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 5, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 7, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 9, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 11, 2))) AS `nmac`
FROM `ai_device`
WHERE `mac_address` REGEXP '^([0-9A-Fa-f]{2}[-:_]?){5}[0-9A-Fa-f]{2}$'
),
cnt AS (SELECT `nmac`, COUNT(*) AS `n` FROM norm GROUP BY `nmac`)
UPDATE `ai_device` d
JOIN norm ON norm.`id` = d.`id`
JOIN cnt ON cnt.`nmac` = norm.`nmac`
SET d.`mac_address` = norm.`nmac`
WHERE cnt.`n` = 1
-- BINARY 是逐字节比:ai_device 走的是库的 utf8mb4_0900_ai_ci,
-- 普通 <> 认为 AA:BB:… 和 aa:bb:… 相等,大写行会永远刷不掉
AND BINARY d.`mac_address` <> BINARY norm.`nmac`;
--
-- ── ② ai_device_address_book 的两列 ────────────────────────────────────
-- 主键是 (mac_address, target_mac),撞的是这一对,所以两列分开刷、各自判重。
-- 一次只改一列时另一列不动,会不会撞只由这一列决定:
-- mac 相同的行 target 必然不同(主键),target 归一成同一个值的行有 cnt 挡着。
--
-- 12 位以外的地址(ESP32 伪 MAC 是 32 位)正则不匹配,原样不动,和 Java 那边
-- normalizeDeviceId 的行为一致——它同样只归一 12 位那种。mac 和 target 各自判断,
-- 一列不合法不妨碍另一列刷(伪 MAC 那台本机的 mac 照样刷得掉)。
WITH norm AS (
SELECT `mac_address`, `target_mac`, LOWER(CONCAT(SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 1, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 3, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 5, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 7, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 9, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`, ':', ''), '-', ''), '_', ''), 11, 2))) AS `nmac`
FROM `ai_device_address_book`
WHERE `mac_address` REGEXP '^([0-9A-Fa-f]{2}[-:_]?){5}[0-9A-Fa-f]{2}$'
),
cnt AS (SELECT `nmac`, COUNT(*) AS `n` FROM norm GROUP BY `nmac`)
UPDATE `ai_device_address_book` b
JOIN norm ON norm.`mac_address` = b.`mac_address` AND norm.`target_mac` = b.`target_mac`
JOIN cnt ON cnt.`nmac` = norm.`nmac`
SET b.`mac_address` = norm.`nmac`
WHERE cnt.`n` = 1
AND BINARY b.`mac_address` <> BINARY norm.`nmac`;
WITH norm AS (
SELECT `mac_address`, `target_mac`, LOWER(CONCAT(SUBSTRING(REPLACE(REPLACE(REPLACE(`target_mac`, ':', ''), '-', ''), '_', ''), 1, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`target_mac`, ':', ''), '-', ''), '_', ''), 3, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`target_mac`, ':', ''), '-', ''), '_', ''), 5, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`target_mac`, ':', ''), '-', ''), '_', ''), 7, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`target_mac`, ':', ''), '-', ''), '_', ''), 9, 2), ':', SUBSTRING(REPLACE(REPLACE(REPLACE(`target_mac`, ':', ''), '-', ''), '_', ''), 11, 2))) AS `ntmac`
FROM `ai_device_address_book`
WHERE `target_mac` REGEXP '^([0-9A-Fa-f]{2}[-:_]?){5}[0-9A-Fa-f]{2}$'
),
cnt AS (SELECT `ntmac`, COUNT(*) AS `n` FROM norm GROUP BY `ntmac`)
UPDATE `ai_device_address_book` b
JOIN norm ON norm.`mac_address` = b.`mac_address` AND norm.`target_mac` = b.`target_mac`
JOIN cnt ON cnt.`ntmac` = norm.`ntmac`
SET b.`target_mac` = norm.`ntmac`
WHERE cnt.`n` = 1
AND BINARY b.`target_mac` <> BINARY norm.`ntmac`;
--
-- ── ③ 刷完之后跑这个,看有没有被跳过的重复 ─────────────────────────────
-- 有结果说明同一台设备被填过多种写法,两行现在还是分开存的,行为和迁移前完全一样。
-- 真要合得人来看:两行的 user_id 可能不是同一个人,谁该留下不是机器能定的,
-- 所以这里只报不动。
--
-- SELECT LOWER(CONCAT(SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 1, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 3, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 5, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 7, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 9, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 11, 2))) AS `nmac`,
-- COUNT(*) AS `c`
-- FROM `ai_device`
-- WHERE `mac_address` REGEXP '^([0-9A-Fa-f]{2}[-:_]?){5}[0-9A-Fa-f]{2}$'
-- GROUP BY `nmac` HAVING `c` > 1;
--
-- ── 漏刷检查:刷完之后还有哪些行是原写法 ──────────────────────────────────
-- 上一条只找重复组,而重复组本来就该留着原样,所以它查不出漏刷:
-- 判重之外的行要是压根没被刷到,它一条也报不出来。
-- 下面这条报的是「还是原写法」的所有行。跑出结果不一定是漏刷——
-- 重复组在上面那条名单里的那些是故意留的,剩下的是真漏刷。
--
-- 第 ① 段在有重复组的表上整段刷不动过一次:MySQL 26.7.0 实测,
-- 该刷的行一行没动、ROW_COUNT() 报 0、退出码 0、不打日志、不报错。
-- 没有报错可等,只能靠这条查。
--
-- SELECT `id`, `mac_address` AS `still_original`,
-- LOWER(CONCAT(SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 1, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 3, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 5, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 7, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 9, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 11, 2))) AS `should_be`
-- FROM `ai_device`
-- WHERE `mac_address` REGEXP '^([0-9A-Fa-f]{2}[-:_]?){5}[0-9A-Fa-f]{2}$'
-- AND BINARY `mac_address` <> BINARY LOWER(CONCAT(SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 1, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 3, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 5, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 7, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 9, 2), ':',
-- SUBSTRING(REPLACE(REPLACE(REPLACE(`mac_address`,':',''),'-',''),'_',''), 11, 2)))
-- ORDER BY `id`;
--
-- 三段合在一个 changeset 里,也就是一个事务(Liquibase 默认 runInTransaction 为真),
-- 任何一段报错都会整体回滚、服务起不来。所以它们不该去撞,上面的 cnt 就是为这个。
-- 值已经是对的三段都是空操作,重跑安全。
@@ -0,0 +1,22 @@
-- PaddleSpeechTTS:智控台协议收敛为仅 websocket(与 xiaozhi-server 实现一致)
UPDATE `ai_model_provider`
SET `fields` = JSON_ARRAY(
JSON_OBJECT('key', 'protocol', 'label', '协议类型', 'type', 'string', 'options', JSON_ARRAY('websocket')),
JSON_OBJECT('key', 'url', 'label', '服务地址', 'type', 'string'),
JSON_OBJECT('key', 'spk_id', 'label', '音色', 'type', 'int'),
JSON_OBJECT('key', 'sample_rate', 'label', '采样率', 'type', 'float'),
JSON_OBJECT('key', 'speed', 'label', '语速', 'type', 'float'),
JSON_OBJECT('key', 'volume', 'label', '音量', 'type', 'float'),
JSON_OBJECT('key', 'save_path', 'label', '保存路径', 'type', 'string')
)
WHERE `id` = 'SYSTEM_TTS_PaddleSpeechTTS';
UPDATE `ai_model_config`
SET `remark` = 'PaddleSpeechTTS 配置说明:
1. PaddleSpeech 是百度飞桨开源的语音合成工具,支持本地离线部署和模型训练。paddlepaddle百度飞浆框架地址:https://www.paddlepaddle.org.cn/
2. 当前仅支持 WebSocket 协议进行流式传输(参考部署文档:https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/paddlespeech-deploy.md)。
3. 使用前要在本地部署 paddlespeech 服务,服务默认运行在 ws://127.0.0.1:8092/paddlespeech/tts/streaming
4. 支持自定义发音人、语速、音量和采样率。
'
WHERE `id` = 'TTS_PaddleSpeechTTS';
@@ -0,0 +1,9 @@
-- 联网搜索插件新增Outcrawl搜索源
UPDATE `ai_model_provider`
SET `fields` = JSON_SET(
`fields`,
'$[0].label',
'搜索源:metaso / tavily / serply / outcrawl'
)
WHERE `id` = 'SYSTEM_PLUGIN_WEB_SEARCH';
@@ -754,9 +754,16 @@ databaseChangeLog:
encoding: utf8
path: classpath:db/changelog/202609172100.sql
- changeSet:
id: 202610231200
author: wk01
id: 202610302051
author: shinjiyu
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202610231200.sql
path: classpath:db/changelog/202610302051.sql
- changeSet:
id: 202611021200
author: heydryft
changes:
- sqlFile:
encoding: utf8
path: classpath:db/changelog/202611021200.sql
@@ -14,4 +14,12 @@
<update id="logout">
update sys_user_token set expire_date = #{expireDate} where user_id = #{userId}
</update>
<update id="refreshTokenCAS">
update sys_user_token
set token = #{newToken}, update_date = #{updateDate}, expire_date = #{newExpire}
where token = #{oldToken}
and expire_date > NOW()
and user_id = #{userId}
</update>
</mapper>
@@ -44,7 +44,7 @@ class SensitiveDataUtilsTest {
void maskMiddle_keeps_four_at_each_end_for_long_strings() {
String masked = SensitiveDataUtils.maskMiddle("verylongapikeyvalue123");
assertTrue(masked.startsWith("very"));
assertTrue(masked.endsWith("3123"));
assertTrue(masked.endsWith("e123"));
assertTrue(masked.contains("****"));
assertEquals("verylongapikeyvalue123".length(), masked.length());
}
@@ -14,9 +14,14 @@ import java.util.Collections;
import java.util.Date;
import java.util.List;
import org.apache.ibatis.builder.MapperBuilderAssistant;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.springframework.test.util.ReflectionTestUtils;
import com.baomidou.mybatisplus.core.MybatisConfiguration;
import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
import xiaozhi.modules.agent.dao.AgentCorrectWordMappingDao;
import xiaozhi.modules.agent.entity.AgentCorrectWordMappingEntity;
import xiaozhi.modules.correctword.dao.CorrectWordFileDao;
@@ -37,6 +42,20 @@ class CorrectWordFileServiceImplTest {
private final CorrectWordFileServiceImpl service =
new CorrectWordFileServiceImpl(fileDao, itemDao, mappingDao);
@BeforeAll
static void registerEntityMetadata() {
// 这条用例不起 Spring 上下文,MyBatis-Plus 的实体元数据没人注册,
// LambdaQueryWrapper 拼条件时找不到 lambda 缓存就抛异常。
// 全量跑的时候是前面的上下文测试顺带注册好的,单独跑就露馅。
// 这里自己补上,单独跑和跟着全量跑结果一样。
TableInfoHelper.initTableInfo(
new MapperBuilderAssistant(new MybatisConfiguration(), ""),
CorrectWordItemEntity.class);
TableInfoHelper.initTableInfo(
new MapperBuilderAssistant(new MybatisConfiguration(), ""),
CorrectWordFileEntity.class);
}
@Test
void getAllItemsByAgentId_returns_empty_list_when_no_mappings() {
when(mappingDao.selectByAgentId(AGENT_ID)).thenReturn(Collections.emptyList());
@@ -3,6 +3,7 @@ package xiaozhi.modules.device;
import java.util.HashMap;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
@@ -15,6 +16,7 @@ import xiaozhi.common.exception.RenException;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.modules.sys.dto.SysUserDTO;
import xiaozhi.modules.sys.service.SysUserService;
import xiaozhi.testsupport.IntegrationTestEnvironment;
@Slf4j
@SpringBootTest
@@ -22,6 +24,11 @@ import xiaozhi.modules.sys.service.SysUserService;
@DisplayName("设备测试")
public class DeviceTest {
@BeforeAll
static void checkDependencies() {
IntegrationTestEnvironment.requireDatabaseAndRedis();
}
@Autowired
private RedisUtils redisUtils;
@Autowired
@@ -1,11 +1,15 @@
package xiaozhi.modules.device.controller;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.List;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.mockito.MockedStatic;
@@ -15,7 +19,10 @@ import xiaozhi.common.redis.RedisUtils;
import xiaozhi.common.user.UserDetail;
import xiaozhi.common.utils.MessageUtils;
import xiaozhi.common.utils.Result;
import xiaozhi.modules.device.dto.DeviceAddressBookAliasDTO;
import xiaozhi.modules.device.dto.DeviceAddressBookPermissionDTO;
import xiaozhi.modules.device.dto.DeviceUpdateDTO;
import xiaozhi.modules.device.entity.DeviceAddressBookEntity;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceAddressBookService;
import xiaozhi.modules.device.service.DeviceService;
@@ -26,6 +33,8 @@ import xiaozhi.modules.sys.service.SysParamsService;
class DeviceControllerTest {
private static final String DEVICE_ID = "device-id";
private static final String MAC = "00:11:22:33:44:55";
private static final String OTHER_MAC = "00:11:22:33:44:66";
private static final long USER_ID = 1L;
@Test
@@ -35,7 +44,7 @@ class DeviceControllerTest {
DeviceEntity entity = ownedDevice();
when(deviceService.selectById(DEVICE_ID)).thenReturn(entity);
when(deviceService.updateById(entity)).thenReturn(false);
DeviceController controller = controller(deviceService);
DeviceController controller = controller(deviceService, mock(DeviceAddressBookService.class));
DeviceUpdateDTO update = new DeviceUpdateDTO();
update.setAutoUpdate(0);
@@ -54,10 +63,84 @@ class DeviceControllerTest {
}
}
private DeviceController controller(DeviceService deviceService) {
@Test
@DisplayName("读取他人设备通讯录时一律按设备不存在处理")
void addressBookOfDeviceOwnedByAnotherUserIsRejected() {
DeviceService deviceService = mock(DeviceService.class);
DeviceAddressBookService addressBookService = mock(DeviceAddressBookService.class);
when(deviceService.getOwnedDeviceByMacAddress(OTHER_MAC)).thenReturn(null);
DeviceController controller = controller(deviceService, addressBookService);
try (MockedStatic<MessageUtils> messageUtils = mockStatic(MessageUtils.class)) {
messageUtils.when(() -> MessageUtils.getMessage(ErrorCode.DEVICE_NOT_EXIST))
.thenReturn("Device not exist");
Result<Object> result = controller.getAddressBook(OTHER_MAC);
assertEquals(ErrorCode.DEVICE_NOT_EXIST, result.getCode());
assertNull(result.getData());
verify(addressBookService, never()).getAddressBookList(OTHER_MAC);
}
}
@Test
@DisplayName("读取本机设备通讯录照常返回")
void addressBookOfOwnDeviceIsReturned() {
DeviceService deviceService = mock(DeviceService.class);
DeviceAddressBookService addressBookService = mock(DeviceAddressBookService.class);
List<DeviceAddressBookEntity> entries = List.of(new DeviceAddressBookEntity());
when(deviceService.getOwnedDeviceByMacAddress(MAC)).thenReturn(ownedDevice());
when(addressBookService.getAddressBookList(MAC)).thenReturn(entries);
DeviceController controller = controller(deviceService, addressBookService);
Result<Object> result = controller.getAddressBook(MAC);
assertEquals(0, result.getCode());
assertEquals(entries, result.getData());
}
@Test
@DisplayName("以他人设备名义改别名被拒")
void updateAliasOnDeviceOwnedByAnotherUserIsRejected() {
DeviceService deviceService = mock(DeviceService.class);
DeviceAddressBookService addressBookService = mock(DeviceAddressBookService.class);
when(deviceService.getOwnedDeviceByMacAddress(OTHER_MAC)).thenReturn(null);
DeviceController controller = controller(deviceService, addressBookService);
DeviceAddressBookAliasDTO dto = new DeviceAddressBookAliasDTO();
dto.setMacAddress(OTHER_MAC);
dto.setTargetMac(MAC);
dto.setAlias("office");
Result<Void> result = controller.updateAlias(dto);
assertEquals("无权限操作该设备", result.getMsg());
verify(addressBookService, never()).saveOrUpdate(OTHER_MAC, MAC, "office", null);
}
@Test
@DisplayName("以他人设备名义改权限被拒")
void updatePermissionOnDeviceOwnedByAnotherUserIsRejected() {
DeviceService deviceService = mock(DeviceService.class);
DeviceAddressBookService addressBookService = mock(DeviceAddressBookService.class);
when(deviceService.getOwnedDeviceByMacAddress(OTHER_MAC)).thenReturn(null);
DeviceController controller = controller(deviceService, addressBookService);
DeviceAddressBookPermissionDTO dto = new DeviceAddressBookPermissionDTO();
dto.setMacAddress(OTHER_MAC);
dto.setTargetMac(MAC);
dto.setHasPermission(true);
Result<Void> result = controller.updatePermission(dto);
assertEquals("无权限操作该设备", result.getMsg());
verify(addressBookService, never()).saveOrUpdate(OTHER_MAC, MAC, null, true);
}
private DeviceController controller(DeviceService deviceService, DeviceAddressBookService addressBookService) {
return new DeviceController(
deviceService,
mock(DeviceAddressBookService.class),
addressBookService,
mock(RedisUtils.class),
mock(SysParamsService.class));
}
@@ -1,26 +1,49 @@
package xiaozhi.modules.device.service.impl;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.Date;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;
import org.apache.ibatis.exceptions.TooManyResultsException;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;
import org.mockito.MockedStatic;
import xiaozhi.common.constant.Constant;
import xiaozhi.common.redis.RedisKeys;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.modules.device.dao.DeviceAddressBookDao;
import xiaozhi.modules.device.entity.DeviceAddressBookEntity;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceService;
import xiaozhi.modules.sys.service.SysParamsService;
class DeviceAddressBookServiceImplTest {
private static final String MAC = "00:11:22:33:44:55";
private static final String OTHER_MAC = "00:11:22:33:44:66";
private static final String THIRD_MAC = "00:11:22:33:44:77";
// 纯数字的 MAC 大小写化后跟原来一模一样,拿它造不出"只差大小写"的两行
private static final String HEX_MAC = "00:11:22:33:44:ab";
@Test
void newEntryUsesTheDedicatedAddressBookInsertMapping() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
@@ -32,15 +55,530 @@ class DeviceAddressBookServiceImplTest {
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
service.saveOrUpdate("00:11:22:33:44:55", "00:11:22:33:44:66", "living-room", true);
service.saveOrUpdate(MAC, OTHER_MAC, "living-room", true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
DeviceAddressBookEntity entity = entityCaptor.getValue();
assertEquals("00:11:22:33:44:55", entity.getMacAddress());
assertEquals("00:11:22:33:44:66", entity.getTargetMac());
assertEquals(MAC, entity.getMacAddress());
assertEquals(OTHER_MAC, entity.getTargetMac());
assertEquals("living-room", entity.getAlias());
assertTrue(entity.getHasPermission());
verify(addressBookDao, never()).insert(any(DeviceAddressBookEntity.class));
}
@Test
void blankAliasOnUnregisteredTargetFallsBackToMacSuffix() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
when(deviceService.getOwnedDeviceByMacAddress(anyString())).thenReturn(null);
service.saveOrUpdate(MAC, OTHER_MAC, null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
DeviceAddressBookEntity entity = entityCaptor.getValue();
assertNotNull(entity.getAlias());
assertEquals("尾号为66的设备", entity.getAlias());
assertTrue(entity.getHasPermission());
verify(deviceService, never()).getOwnedDeviceByMacAddress(anyString());
}
@Test
void blankAliasOnAnotherDeviceFallsBackToMacSuffixWithoutAskingTheDeviceTable() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity registered = new DeviceEntity();
registered.setMacAddress(OTHER_MAC);
registered.setAlias(null);
when(deviceService.getDeviceByMacAddress(OTHER_MAC)).thenReturn(registered);
when(deviceService.getOwnedDeviceByMacAddress(OTHER_MAC)).thenReturn(registered);
service.saveOrUpdate(MAC, OTHER_MAC, null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为66的设备", entityCaptor.getValue().getAlias(),
"登记了但没起名的设备落个空名,缓存刷新会跳过这条,对方永远呼不过来");
verify(deviceService, never()).getDeviceByMacAddress(anyString());
verify(deviceService, never()).getOwnedDeviceByMacAddress(anyString());
}
@Test
void blankAliasOnRegisteredTargetDoesNotTakeOverTheirNickname() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity theirs = new DeviceEntity();
theirs.setMacAddress(OTHER_MAC);
theirs.setAlias("老王的音响");
when(deviceService.getDeviceByMacAddress(OTHER_MAC)).thenReturn(theirs);
when(deviceService.getOwnedDeviceByMacAddress(OTHER_MAC)).thenReturn(theirs);
service.saveOrUpdate(MAC, OTHER_MAC, null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为66的设备", entityCaptor.getValue().getAlias(),
"对方在智控台上给这台设备起的名字被抄进了调用方的通讯录");
verify(deviceService, never()).getDeviceByMacAddress(anyString());
verify(deviceService, never()).getOwnedDeviceByMacAddress(anyString());
}
@Test
void blankAliasOnOwnDeviceUsesTheNameFromItsOwnDeviceRow() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity mine = new DeviceEntity();
mine.setMacAddress(MAC);
mine.setAlias("我的书房");
when(deviceService.getOwnedDeviceByMacAddress(MAC)).thenReturn(mine);
service.saveOrUpdate(MAC, MAC, null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("我的书房", entityCaptor.getValue().getAlias(),
"给自己那台设备起名,用的就是自己设备表里的名字");
}
@Test
void ownDeviceWithoutAliasInDeviceTableFallsBackToMacSuffix() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity mine = new DeviceEntity();
mine.setMacAddress(MAC);
mine.setAlias(null);
when(deviceService.getOwnedDeviceByMacAddress(MAC)).thenReturn(mine);
service.saveOrUpdate(MAC, MAC, null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为55的设备", entityCaptor.getValue().getAlias(),
"自己的设备在设备表里也没起名,落个空名的话缓存刷新会跳过这条");
}
@Test
void duplicateDeviceRowsDifferingOnlyInCaseDoNotBreakNaming() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
// 设备表里同一个 MAC 有两行只差大小写,一次只准取一条的那种查法会直接抛异常
when(deviceService.getDeviceByMacAddress(anyString()))
.thenThrow(new TooManyResultsException("selectOne 撞上两行只差大小写的数据"));
DeviceEntity first = new DeviceEntity();
first.setMacAddress(MAC);
first.setAlias("我的书房");
when(deviceService.getOwnedDeviceByMacAddress(MAC)).thenReturn(first);
assertDoesNotThrow(() -> service.saveOrUpdate(MAC, MAC, null, true),
"同 MAC 有两行只差大小写时,取名不能把整个接口带崩");
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("我的书房", entityCaptor.getValue().getAlias(),
"撞行时按 id 取第一条那条的名字");
}
@Test
void ownDeviceIsRecognizedWhateverCaseTheAddressIsWrittenIn() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
DeviceEntity mine = new DeviceEntity();
mine.setMacAddress(HEX_MAC);
mine.setAlias("我的书房");
when(deviceService.getOwnedDeviceByMacAddress(HEX_MAC)).thenReturn(mine);
service.saveOrUpdate(HEX_MAC, HEX_MAC.toUpperCase(), null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("我的书房", entityCaptor.getValue().getAlias(),
"同一个地址写成大小写两种样子,就被当成别人的设备了");
}
@Test
void explicitAliasIsKeptAsIs() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
service.saveOrUpdate(MAC, OTHER_MAC, "客厅", true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("客厅", entityCaptor.getValue().getAlias(),
"自己给别人的设备起名是通讯录的本意,不能被兜底逻辑改掉");
verify(deviceService, never()).getDeviceByMacAddress(anyString());
verify(deviceService, never()).getOwnedDeviceByMacAddress(anyString());
}
@Test
void blankAliasOnUnregisteredTargetDoesNotBlowUp() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
when(deviceService.getOwnedDeviceByMacAddress(anyString())).thenReturn(null);
assertDoesNotThrow(() -> service.saveOrUpdate(MAC, OTHER_MAC, null, true),
"目标设备没登记时接口得正常返回,不能抛异常");
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为66的设备", entityCaptor.getValue().getAlias());
}
@Test
void refreshCacheKeepsEntriesWhoseAliasWasNeverFilledIn() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
when(addressBookDao.selectList(null)).thenReturn(List.of(row(MAC, OTHER_MAC, null)));
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, mock(DeviceService.class), mock(SysParamsService.class));
service.refreshCache();
Map<String, String> book = cachedBook(service, redisUtils).get(MAC);
assertNotNull(book, "空别名整条被跳过,这条设备就再也呼不过来了");
assertEquals(OTHER_MAC + "|1", book.get("尾号为66的设备"));
}
@Test
void refreshCacheDisambiguatesTwoEntriesSharingAnAlias() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
when(addressBookDao.selectList(null)).thenReturn(List.of(
row(MAC, OTHER_MAC, "客厅"),
row(MAC, THIRD_MAC, "客厅")));
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, mock(DeviceService.class), mock(SysParamsService.class));
service.refreshCache();
Map<String, String> book = cachedBook(service, redisUtils).get(MAC);
assertEquals(2, book.size(), "撞名的两台设备只留得下一台:" + book);
assertEquals(OTHER_MAC + "|1", book.get("客厅"));
assertEquals(THIRD_MAC + "|1", book.get("客厅1"));
}
@Test
void refreshCacheGivesTheSameResultWhateverOrderRowsComeBackIn() {
DeviceAddressBookEntity first = row(MAC, OTHER_MAC, "客厅");
DeviceAddressBookEntity second = row(MAC, THIRD_MAC, "客厅");
Map<String, String> inOrder = refreshAndRead(List.of(first, second));
Map<String, String> reversed = refreshAndRead(List.of(second, first));
assertEquals(inOrder, reversed, "同样的数据两次刷新结果不一样,缓存里的称呼会来回变");
}
/**
* 库里能同时存在两条 owner 只差大小写的行:表是 utf8mb4_0900_ai_ci,
* 主键 (mac_address, target_mac) 大小写不敏感,target 不同就不会撞。
* 这是同一个人,算称呼和写缓存都得按小写归成一本
*/
@Test
void ownersDifferingOnlyInCaseShareOneBook() {
String upper = HEX_MAC.toUpperCase();
assertNotEquals(HEX_MAC, upper, "这个用例得真的造出大小写差异才有效");
Map<String, Map<String, String>> books = refreshAndReadAll(List.of(
row(upper, OTHER_MAC, "客厅"),
row(HEX_MAC, THIRD_MAC, "客厅")));
Map<String, String> book = books.get(HEX_MAC);
assertNotNull(book, "同一个人被算成了两个人:" + books.keySet());
assertEquals(2, book.size(), "两台设备只剩一台,另一台从缓存里消失了:" + book);
assertEquals(OTHER_MAC + "|1", book.get("客厅"));
assertEquals(THIRD_MAC + "|1", book.get("客厅1"));
assertEquals(1, books.size(), "两种写法的地址被拆成了两本:" + books.keySet());
}
@Test
void listAndCacheAgreeWhenOwnersDifferOnlyInCase() {
String upper = HEX_MAC.toUpperCase();
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
when(addressBookDao.getAddressBookList(upper)).thenReturn(new ArrayList<>(List.of(
row(upper, OTHER_MAC, "客厅"), row(HEX_MAC, THIRD_MAC, "客厅"))));
when(addressBookDao.selectList(null)).thenReturn(List.of(
row(upper, OTHER_MAC, "客厅"), row(HEX_MAC, THIRD_MAC, "客厅")));
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, mock(DeviceService.class), mock(SysParamsService.class));
Map<String, String> shownToConsole = namesByTargetMac(service.getAddressBookList(upper));
assertEquals(2, shownToConsole.size(),
"接口按原值分组,同一个人的两台设备被算成一个,前端只能看到一台:" + shownToConsole);
service.refreshCache();
Map<String, String> callableNames = new HashMap<>();
cachedBook(service, redisUtils).get(HEX_MAC)
.forEach((name, macAndPerm) -> callableNames.put(name, macAndPerm.split("\\|")[0]));
assertEquals(shownToConsole, callableNames,
"接口按原值分组、缓存按小写分组,用户照前端看到的名字呼,命中的不是同一台设备");
}
@Test
void nameAssignmentDoesNotDependOnWhichSpellingOfTheOwnerCameBack() {
String upper = HEX_MAC.toUpperCase();
Map<String, String> expected = Map.of("客厅", OTHER_MAC + "|1", "客厅1", THIRD_MAC + "|1");
Map<String, String> lowerFirst = refreshAndRead(HEX_MAC, List.of(
row(HEX_MAC, OTHER_MAC, "客厅"), row(upper, THIRD_MAC, "客厅")));
Map<String, String> upperFirst = refreshAndRead(HEX_MAC, List.of(
row(upper, THIRD_MAC, "客厅"), row(HEX_MAC, OTHER_MAC, "客厅")));
assertEquals(expected, lowerFirst,
"同一个人的称呼取决于库里哪种写法先返回,刷新一次就换一次名字");
assertEquals(expected, upperFirst, "顺序反过来结果就该不一样:" + upperFirst);
}
@SuppressWarnings("unchecked")
private Map<String, Map<String, String>> refreshAndReadAll(List<DeviceAddressBookEntity> rows) {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
when(addressBookDao.selectList(null)).thenReturn(rows);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, mock(DeviceService.class), mock(SysParamsService.class));
service.refreshCache();
return cachedBook(service, redisUtils);
}
private Map<String, String> refreshAndRead(List<DeviceAddressBookEntity> rows) {
return refreshAndRead(MAC, rows);
}
private Map<String, String> refreshAndRead(String owner, List<DeviceAddressBookEntity> rows) {
return refreshAndReadAll(rows).get(owner);
}
@SuppressWarnings("unchecked")
private Map<String, Map<String, String>> cachedBook(DeviceAddressBookServiceImpl service, RedisUtils redisUtils) {
ArgumentCaptor<Object> captor = ArgumentCaptor.forClass(Object.class);
verify(redisUtils).set(eq(RedisKeys.getAddressBookKey()), captor.capture());
return (Map<String, Map<String, String>>) captor.getValue();
}
private DeviceAddressBookEntity row(String macAddress, String targetMac, String alias) {
return row(macAddress, targetMac, alias, Boolean.TRUE);
}
private DeviceAddressBookEntity row(String macAddress, String targetMac, String alias, Boolean hasPermission) {
DeviceAddressBookEntity entity = new DeviceAddressBookEntity();
entity.setMacAddress(macAddress);
entity.setTargetMac(targetMac);
entity.setAlias(alias);
entity.setHasPermission(hasPermission);
return entity;
}
@Test
void listAndCacheHandOutTheSameNameForCollidingEntries() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
// 两次 DAO 调用各给一份新行:取名会写回实体,共用同一份的话第二遍读到的是算过的名字
// 用可变列表,跟 MyBatis 实际返回的一致,免得排序那步先炸掉、断言根本没跑到
when(addressBookDao.getAddressBookList(MAC)).thenReturn(
new ArrayList<>(List.of(row(MAC, OTHER_MAC, "客厅"), row(MAC, THIRD_MAC, "客厅"))));
when(addressBookDao.selectList(null)).thenReturn(List.of(
row(MAC, OTHER_MAC, "客厅"), row(MAC, THIRD_MAC, "客厅")));
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, mock(DeviceService.class), mock(SysParamsService.class));
Map<String, String> shownToConsole = namesByTargetMac(service.getAddressBookList(MAC));
service.refreshCache();
Map<String, String> callableNames = new HashMap<>();
cachedBook(service, redisUtils).get(MAC)
.forEach((name, macAndPerm) -> callableNames.put(name, macAndPerm.split("\\|")[0]));
assertEquals(callableNames, shownToConsole,
"接口和缓存给的是两个名字,用户照前端看到的那个名字呼叫,命中的不是同一台设备");
}
@Test
void theCallableEntryKeepsThePlainNameWhenTwoCollide() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
// OTHER_MAC 的 target_mac 排在前面,没权限的那条会先占住原名
when(addressBookDao.selectList(null)).thenReturn(List.of(
row(MAC, OTHER_MAC, "客厅", Boolean.FALSE),
row(MAC, THIRD_MAC, "客厅", Boolean.TRUE)));
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, mock(DeviceService.class), mock(SysParamsService.class));
service.refreshCache();
Map<String, String> book = cachedBook(service, redisUtils).get(MAC);
assertEquals(THIRD_MAC + "|1", book.get("客厅"),
"能呼的那台被挤到后缀名上了,用户照前端看到的名字点,界面只会说没权限");
assertEquals(OTHER_MAC + "|0", book.get("客厅1"));
}
@Test
void listFillsInBlankAliasesSoTheConsoleShowsWhatCanBeCalled() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
when(addressBookDao.getAddressBookList(MAC)).thenReturn(new ArrayList<>(List.of(row(MAC, OTHER_MAC, null))));
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, mock(DeviceService.class), mock(SysParamsService.class));
List<DeviceAddressBookEntity> rows = service.getAddressBookList(MAC);
assertEquals("尾号为66的设备", rows.get(0).getAlias(),
"空别名原样返回的话前端回落显示设备名,用户照着那个名字呼叫却只会说找不到备注");
}
@Test
void anUppercaseMacGetsTheSameNameWhicheverPathAddsItToTheBook() {
// 兜底称呼只有这一处算法:存进去的时候算一次,之后接口和缓存里显示的
// 都是同一个字符串。不跟着小写化的话同一个地址在两条路径上算出两个称呼,
// 用户照着看到的名字呼过去,命中的不是自己刚存的那台
//
// 运行期这一侧:saveOrUpdate 传的是 DTO 原样过来的地址,大小写没归一
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
when(addressBookDao.selectOne(any())).thenReturn(null);
when(addressBookDao.selectList(null)).thenReturn(List.of());
when(deviceService.getOwnedDeviceByMacAddress(anyString())).thenReturn(null);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, mock(SysParamsService.class));
service.saveOrUpdate(MAC, "ff:ee:dd:cc:bb:AB", null, true);
ArgumentCaptor<DeviceAddressBookEntity> entityCaptor = ArgumentCaptor.forClass(DeviceAddressBookEntity.class);
verify(addressBookDao).insertAddressBook(entityCaptor.capture());
assertEquals("尾号为ab的设备", entityCaptor.getValue().getAlias(),
"MAC 大小写不区分,末尾两位要跟着小写化,否则接口和呼叫那边算出来的称呼对不上");
}
/**
* 接口按最近改过的排前面
*
* <p>
* 智控台那条 Vue 是按 targetMac 把 alias 合进 allDevices 的,接口顺序被丢掉了,
* 这条现在对 console 没有效果。用例留着是把这个契约钉住:哪天当无用代码删了,
* 会有用例报出来,而不是静悄悄换掉接口的返回顺序
*/
@Test
void listPutsTheMostRecentlyTouchedEntryFirst() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceAddressBookEntity older = row(MAC, OTHER_MAC, "客厅");
older.setUpdateDate(new Date(1767321840000L));
DeviceAddressBookEntity newer = row(MAC, THIRD_MAC, "书房");
newer.setUpdateDate(new Date(1778000000000L));
DeviceAddressBookEntity neverTouched = row(MAC, "00:11:22:33:44:88", "阳台");
when(addressBookDao.getAddressBookList(MAC)).thenReturn(
new ArrayList<>(List.of(older, neverTouched, newer)));
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, mock(DeviceService.class), mock(SysParamsService.class));
List<DeviceAddressBookEntity> rows = service.getAddressBookList(MAC);
assertEquals(List.of("书房", "客厅", "阳台"), rows.stream().map(DeviceAddressBookEntity::getAlias).toList(),
"最近改过的应该排前面,没改过的排最后");
}
private Map<String, String> namesByTargetMac(List<DeviceAddressBookEntity> rows) {
Map<String, String> byAlias = new HashMap<>();
for (DeviceAddressBookEntity row : rows) {
byAlias.put(row.getAlias(), row.getTargetMac());
}
return byAlias;
}
@Test
void callByNicknameFallsBackToMacSuffixWhenCallerHasNoAlias() {
DeviceAddressBookDao addressBookDao = mock(DeviceAddressBookDao.class);
RedisUtils redisUtils = mock(RedisUtils.class);
DeviceService deviceService = mock(DeviceService.class);
SysParamsService sysParamsService = mock(SysParamsService.class);
DeviceAddressBookServiceImpl service = new DeviceAddressBookServiceImpl(
addressBookDao, redisUtils, deviceService, sysParamsService);
Map<String, Map<String, String>> books = new HashMap<>();
Map<String, String> callerBook = new HashMap<>();
callerBook.put("office", OTHER_MAC + "|1");
books.put(MAC, callerBook);
when(redisUtils.get(RedisKeys.getAddressBookKey())).thenReturn(books);
when(deviceService.getDeviceByMacAddress(MAC)).thenReturn(null);
when(sysParamsService.getValue("server.mqtt_manager_api", true)).thenReturn("127.0.0.1:1883");
when(sysParamsService.getValue(Constant.SERVER_MQTT_SECRET, true)).thenReturn("gateway-key");
// 网关是真的 http 调用,桩掉它,否则测试会往 127.0.0.1:1883 真发请求
AtomicReference<String> sentUrl = new AtomicReference<>();
AtomicReference<String> sentBody = new AtomicReference<>();
Map<String, Object> result;
try (MockedStatic<MqttGatewayAuthorization> gateway = mockStatic(MqttGatewayAuthorization.class)) {
gateway.when(() -> MqttGatewayAuthorization.postJson(
anyString(), anyString(), anyString(), any(), anyInt()))
.thenAnswer(invocation -> {
sentUrl.set(invocation.getArgument(0));
sentBody.set(invocation.getArgument(1));
return "{\"status\":\"success\",\"message\":\"ok\"}";
});
result = service.callByNickname(MAC, "office", false);
}
assertEquals("success", result.get("status"));
assertEquals("http://127.0.0.1:1883/api/call/request", sentUrl.get());
assertTrue(sentBody.get().contains("尾号为55的设备"),
"主叫方没登记也没起名时该退回尾号称呼:" + sentBody.get());
}
}
@@ -1,252 +0,0 @@
package xiaozhi.modules.device.service.impl;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.ibatis.exceptions.TooManyResultsException;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;
import org.mockito.MockedStatic;
import com.baomidou.mybatisplus.core.conditions.AbstractWrapper;
import com.baomidou.mybatisplus.core.conditions.Wrapper;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.common.user.UserDetail;
import xiaozhi.modules.device.dao.DeviceDao;
import xiaozhi.modules.device.dto.DeviceManualAddDTO;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.security.user.SecurityUser;
/**
* #3448 设备标识归一化回归测试
*
* <p>
* 同一台设备换个分隔符或大小写上报,入库、查重、查询三处必须算成同一个值,
* 否则表现为静默不发凭证、不下发固件,没有任何报错可查。
*/
@DisplayName("设备标识归一化回归测试")
class DeviceMacAddressNormalizeTest {
private static final String CANONICAL = "aa:bb:cc:dd:ee:ff";
@Test
@DisplayName("冒号、连字符、下划线、无分隔符、大小写写法归一到同一个值")
void everySpellingCollapsesToCanonical() {
for (String spelling : new String[] {
"aa:bb:cc:dd:ee:ff", "AA:BB:CC:DD:EE:FF", "aa-bb-cc-dd-ee-ff",
"aa_bb_cc_dd_ee_ff", "aabbccddeeff", "Aa:Bb:Cc:Dd:Ee:Ff" }) {
assertEquals(CANONICAL, DeviceServiceImpl.normalizeDeviceId(spelling), spelling);
}
}
@Test
@DisplayName("不是 MAC 的标识原样返回,交给后续校验拒绝")
void nonMacValuePassesThrough() {
for (String value : new String[] { null, "", " ", "abc", "zz:zz:zz:zz:zz:zz", "00112233445566" }) {
assertEquals(value, DeviceServiceImpl.normalizeDeviceId(value), String.valueOf(value));
}
}
@Test
@DisplayName("按任意写法上报都能查到归一后入库的那台设备")
void lookupFindsRowStoredInCanonicalForm() {
DeviceEntity stored = storedDevice(CANONICAL);
DeviceServiceImpl service = service(daoMatchingOnly(CANONICAL, stored));
for (String spelling : new String[] {
"aa:bb:cc:dd:ee:ff", "aabbccddeeff", "AA-BB-CC-DD-EE-FF", "aa_bb_cc_dd_ee_ff" }) {
assertSame(stored, service.getDeviceByMacAddress(spelling), spelling);
}
}
@Test
@DisplayName("迁移前按原文入库的存量设备仍能查到,不被归一化挡掉")
void lookupFallsBackToLegacyRow() {
DeviceEntity legacy = storedDevice("AA-BB-CC-DD-EE-FF");
DeviceServiceImpl service = service(daoMatchingOnly("AA-BB-CC-DD-EE-FF", legacy));
assertSame(legacy, service.getDeviceByMacAddress("AA-BB-CC-DD-EE-FF"),
"归一后没查到,该退回原文再查一次,否则迁移前的设备直接失联");
}
@Test
@DisplayName("两种写法都查不到时返回 null")
void lookupMissReturnsNull() {
DeviceServiceImpl service = service(daoMatchingOnly(CANONICAL, storedDevice(CANONICAL)));
assertNull(service.getDeviceByMacAddress("11:22:33:44:55:66"));
assertNull(service.getDeviceByMacAddress(" "));
}
@Test
@DisplayName("库里存了两行同一个 MAC 时按 id 定序取第一条,两次读到同一台")
void duplicateRowsResolveToTheSameRowEveryTime() {
// 迁移脚本对重复组整组跳过,库里出现同 mac 的两行是可能的:
// 用户把同一台设备用两种写法各添加过一次,迁移前后都该读得出来
DeviceEntity first = storedDevice("AA-BB-CC-DD-EE-FF");
first.setId("dev-0001");
DeviceEntity second = storedDevice("aa:bb:cc:dd:ee:ff");
second.setId("dev-0002");
DeviceServiceImpl service = service(daoReturningRowsOrderedById(first, second));
assertSame(first, service.getDeviceByMacAddress(CANONICAL));
assertSame(first, service.getDeviceByMacAddress(CANONICAL),
"同一份重复数据两次读到不同的行,称呼和缓存键会跟着变");
}
@Test
@DisplayName("重复行取哪一台只由 id 定序决定,与库返回顺序无关")
void duplicateRowChoiceIgnoresTheOrderTheDatabaseReturnedThem() {
DeviceEntity first = storedDevice("AA-BB-CC-DD-EE-FF");
first.setId("dev-0001");
DeviceEntity second = storedDevice("aa:bb:cc:dd:ee:ff");
second.setId("dev-0002");
DeviceEntity forwards = service(daoReturningRowsOrderedById(first, second))
.getDeviceByMacAddress(CANONICAL);
DeviceEntity backwards = service(daoReturningRowsOrderedById(second, first))
.getDeviceByMacAddress(CANONICAL);
assertSame(first, forwards);
assertSame(backwards, forwards, "换个返回顺序就换了一台,缓存键跟着变");
}
@Test
@DisplayName("手工添加按归一值查重、按归一值入库")
void manualAddStoresNormalizedMac() {
DeviceDao dao = mock(DeviceDao.class);
when(dao.selectOne(any())).thenReturn(null);
DeviceServiceImpl service = service(dao);
DeviceManualAddDTO dto = new DeviceManualAddDTO();
dto.setMacAddress("AABBCCDDEEFF");
dto.setAgentId("1");
service.manualAddDevice(9L, dto);
ArgumentCaptor<Wrapper<DeviceEntity>> queried = wrapperCaptor();
verify(dao).selectOne(queried.capture());
assertEquals(CANONICAL, queriedMac(queried.getValue()),
"查重用的不是归一值,同一台设备换个写法就能被重复添加");
ArgumentCaptor<DeviceEntity> inserted = ArgumentCaptor.forClass(DeviceEntity.class);
verify(dao).insert(inserted.capture());
assertEquals(CANONICAL, inserted.getValue().getMacAddress());
assertEquals(CANONICAL, inserted.getValue().getId(), "主键跟着一起归一,否则同一台设备能插进两条");
}
@Test
@DisplayName("设备激活入库的是归一后的 MAC")
void activationStoresNormalizedMac() {
DeviceDao dao = mock(DeviceDao.class);
when(dao.selectById(any())).thenReturn(null);
RedisUtils redisUtils = mock(RedisUtils.class);
Map<String, Object> activation = new HashMap<>();
activation.put("mac_address", "AA-BB-CC-DD-EE-FF");
activation.put("activation_code", "CODE1");
when(redisUtils.get(any())).thenAnswer(invocation -> {
String key = invocation.getArgument(0);
return key.endsWith("CODE1") ? CANONICAL : activation;
});
DeviceServiceImpl service = service(dao, redisUtils);
UserDetail user = new UserDetail();
user.setId(9L);
try (MockedStatic<SecurityUser> securityUser = mockStatic(SecurityUser.class)) {
securityUser.when(SecurityUser::getUser).thenReturn(user);
assertTrue(service.deviceActivation("1", "CODE1"));
}
ArgumentCaptor<DeviceEntity> inserted = ArgumentCaptor.forClass(DeviceEntity.class);
verify(dao).insert(inserted.capture());
assertEquals(CANONICAL, inserted.getValue().getMacAddress());
}
private DeviceServiceImpl service(DeviceDao dao) {
return service(dao, mock(RedisUtils.class));
}
/** baseDao 由父类持有,子类内部赋给 this,测试类自己碰不到它 */
private DeviceServiceImpl service(DeviceDao dao, RedisUtils redisUtils) {
return new DeviceServiceImpl(dao, null, null, redisUtils, null, null) {
{
this.baseDao = dao;
}
};
}
/**
* 假扮真库:先按 id 的字符序排好再返回
*
* <p>
* 真库里 ORDER BY 排在返回之前,不定序的话拿到哪一行由存储引擎决定。
* 这里不照做,用例量的就只是 mock 自己塞进去的顺序,排序有没有写、怎么排都测不出来
*/
private DeviceDao daoReturningRowsOrderedById(DeviceEntity... rows) {
DeviceDao dao = mock(DeviceDao.class);
List<DeviceEntity> stored = List.of(rows);
when(dao.selectList(any())).thenAnswer(invocation -> {
Wrapper<DeviceEntity> wrapper = invocation.getArgument(0);
assertTrue(wrapper.getSqlSegment().contains("ORDER BY id"),
"重复行没有定序,拿到哪一台随数据库返回顺序变:" + wrapper.getSqlSegment());
return stored.stream().sorted(Comparator.comparing(DeviceEntity::getId)).toList();
});
// selectOne 命中多行就是 MyBatis-Plus 抛的 TooManyResultsException,
// 这里照抄真库的行为:用 selectOne 的写法改回去,用例会带着这个异常炸掉
when(dao.selectOne(any())).thenThrow(
new TooManyResultsException("Expected one result (or null) to be returned by selectOne(), but found: 2"));
return dao;
}
/** 只在查询值等于 storedMac 时才命中,用来验证查的是哪个值 */
private DeviceDao daoMatchingOnly(String storedMac, DeviceEntity row) {
DeviceDao dao = mock(DeviceDao.class);
when(dao.selectList(any())).thenAnswer(invocation -> {
Wrapper<DeviceEntity> wrapper = invocation.getArgument(0);
return storedMac.equals(queriedMac(wrapper)) ? List.of(row) : List.of();
});
return dao;
}
private DeviceEntity storedDevice(String mac) {
DeviceEntity device = new DeviceEntity();
device.setId(mac);
device.setMacAddress(mac);
return device;
}
@SuppressWarnings("unchecked")
private ArgumentCaptor<Wrapper<DeviceEntity>> wrapperCaptor() {
return ArgumentCaptor.forClass(Wrapper.class);
}
/** eq() 绑定的值只在 AbstractWrapper 上,Wrapper 接口看不到 */
private Map<String, Object> boundValues(Wrapper<DeviceEntity> wrapper) {
return ((AbstractWrapper<?, ?, ?>) wrapper).getParamNameValuePairs();
}
/**
* 取出这次查询实际拿去比对的 MAC
*
* <p>
* 绑定值是拼 SQL 时才写进去的,getSqlSegment() 先调一次才读得到
*/
private Object queriedMac(Wrapper<DeviceEntity> wrapper) {
wrapper.getSqlSegment();
return boundValues(wrapper).values().iterator().next();
}
}
@@ -0,0 +1,184 @@
package xiaozhi.modules.device.service.impl;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyList;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.List;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.mockito.MockedStatic;
import com.baomidou.mybatisplus.core.conditions.update.UpdateWrapper;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.common.user.UserDetail;
import xiaozhi.modules.device.dao.DeviceDao;
import xiaozhi.modules.device.entity.DeviceEntity;
import xiaozhi.modules.device.service.DeviceAddressBookService;
import xiaozhi.modules.device.service.OtaService;
import xiaozhi.modules.security.user.SecurityUser;
import xiaozhi.modules.sys.service.SysParamsService;
import xiaozhi.modules.sys.service.SysUserUtilService;
@DisplayName("设备归属校验回归测试")
class DeviceServiceImplTest {
private static final String MAC = "00:11:22:33:44:55";
private static final long USER_ID = 1L;
private static final long OTHER_USER_ID = 2L;
@Test
@DisplayName("设备属于当前用户时返回设备")
void deviceOwnedByCurrentUserIsReturned() {
DeviceEntity device = device(USER_ID);
DeviceServiceImpl service = serviceWithDeviceDao(daoReturning(device));
try (MockedStatic<SecurityUser> securityUser = mockStatic(SecurityUser.class)) {
securityUser.when(SecurityUser::getUser).thenReturn(user(USER_ID));
assertSame(device, service.getOwnedDeviceByMacAddress(MAC));
}
}
@Test
@DisplayName("设备属于他人时按查不到处理")
void deviceOwnedByAnotherUserIsHidden() {
DeviceServiceImpl service = serviceWithDeviceDao(daoReturning(device(OTHER_USER_ID)));
try (MockedStatic<SecurityUser> securityUser = mockStatic(SecurityUser.class)) {
securityUser.when(SecurityUser::getUser).thenReturn(user(USER_ID));
assertNull(service.getOwnedDeviceByMacAddress(MAC));
}
}
@Test
@DisplayName("设备未登记时按查不到处理")
void unregisteredDeviceIsHidden() {
DeviceServiceImpl service = serviceWithDeviceDao(daoReturning(List.of()));
try (MockedStatic<SecurityUser> securityUser = mockStatic(SecurityUser.class)) {
securityUser.when(SecurityUser::getUser).thenReturn(user(USER_ID));
assertNull(service.getOwnedDeviceByMacAddress(MAC));
}
}
@Test
@DisplayName("设备没有归属人时按查不到处理")
void unboundDeviceIsHidden() {
DeviceServiceImpl service = serviceWithDeviceDao(daoReturning(device(null)));
try (MockedStatic<SecurityUser> securityUser = mockStatic(SecurityUser.class)) {
securityUser.when(SecurityUser::getUser).thenReturn(user(USER_ID));
assertNull(service.getOwnedDeviceByMacAddress(MAC));
}
}
@Test
@DisplayName("没有登录用户时按查不到处理")
void anonymousCallerGetsNothing() {
DeviceServiceImpl service = serviceWithDeviceDao(daoReturning(device(USER_ID)));
// 匿名调用的实际形态:getUser() 返回空对象,id 是空的,不是返回 null
try (MockedStatic<SecurityUser> securityUser = mockStatic(SecurityUser.class)) {
securityUser.when(SecurityUser::getUser).thenReturn(new UserDetail());
assertNull(service.getOwnedDeviceByMacAddress(MAC));
}
}
@Test
@DisplayName("库里同一 MAC 有两行只差大小写时,取第一条做归属校验,不再抛异常")
void duplicateMacDifferingOnlyInCaseStillReturnsTheDevice() {
DeviceEntity first = device(USER_ID, "device-1");
DeviceEntity second = device(USER_ID, "device-2");
DeviceServiceImpl service = serviceWithDeviceDao(daoReturning(List.of(first, second)));
try (MockedStatic<SecurityUser> securityUser = mockStatic(SecurityUser.class)) {
securityUser.when(SecurityUser::getUser).thenReturn(user(USER_ID));
assertSame(first, service.getOwnedDeviceByMacAddress(MAC));
}
}
@Test
@DisplayName("解绑自己的设备时,通讯录跟着一起清")
void unbindOwnDeviceClearsItsAddressBook() {
DeviceDao dao = mock(DeviceDao.class);
when(dao.selectById("device-1")).thenReturn(device(USER_ID, "device-1"));
DeviceAddressBookService addressBookService = mock(DeviceAddressBookService.class);
DeviceServiceImpl service = serviceWithDao(dao, addressBookService);
service.unbindDevice(USER_ID, "device-1");
verify(dao).delete(any(UpdateWrapper.class));
verify(addressBookService).deleteByMacAddresses(List.of(MAC));
}
@Test
@DisplayName("解绑别人的设备时什么都不删,通讯录不能跟着没")
void unbindAnotherUsersDeviceDeletesNothing() {
DeviceDao dao = mock(DeviceDao.class);
when(dao.selectById("device-1")).thenReturn(device(OTHER_USER_ID, "device-1"));
DeviceAddressBookService addressBookService = mock(DeviceAddressBookService.class);
DeviceServiceImpl service = serviceWithDao(dao, addressBookService);
service.unbindDevice(USER_ID, "device-1");
verify(dao, never()).delete(any(UpdateWrapper.class));
verify(addressBookService, never()).deleteByMacAddresses(anyList());
}
private DeviceServiceImpl serviceWithDao(DeviceDao dao, DeviceAddressBookService addressBookService) {
// baseDao 由父类持有,子类内部赋给 this,测试类自己碰不到它
return new DeviceServiceImpl(
dao, mock(SysUserUtilService.class), mock(SysParamsService.class),
mock(RedisUtils.class), mock(OtaService.class), addressBookService) {
{
this.baseDao = dao;
}
};
}
private DeviceServiceImpl serviceWithDeviceDao(DeviceDao dao) {
return serviceWithDao(dao, mock(DeviceAddressBookService.class));
}
private DeviceDao daoReturning(DeviceEntity device) {
return daoReturning(List.of(device));
}
private DeviceDao daoReturning(List<DeviceEntity> rows) {
DeviceDao dao = mock(DeviceDao.class);
when(dao.selectList(any())).thenReturn(rows);
return dao;
}
private DeviceEntity device(Long userId) {
return device(userId, null);
}
private DeviceEntity device(Long userId, String id) {
DeviceEntity entity = new DeviceEntity();
entity.setId(id);
entity.setMacAddress(MAC);
entity.setUserId(userId);
return entity;
}
private UserDetail user(long id) {
UserDetail user = new UserDetail();
user.setId(id);
return user;
}
}
@@ -1,389 +0,0 @@
package xiaozhi.modules.device.service.impl;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Assumptions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
/**
* 迁移脚本 202610231200.sql 的行为测试
*
* <p>
* 这段迁移真正要保证的是刷完之后不会出现两行归一成同一个值。判重条件被去掉、
* 连接条件写错、大写判断漏了 BINARY,后果都是同一台设备之后彻底读不出来,
* 而且没有任何报错可查。子串匹配一个都测不出来,只能在真库上跑一遍看结果。
*
* <p>
* 需要一个连得上的 MySQL。连不上就跳过而不是报错,变量是
* XIAOZHI_TEST_MYSQL_URL、XIAOZHI_TEST_MYSQL_USER、XIAOZHI_TEST_MYSQL_PASSWORD
*/
@DisplayName("MAC 归一迁移脚本")
class MacAddressMigrationSqlTest {
private static final String SCHEMA = "mac_migration_sql_test";
private static Connection connection;
@BeforeAll
static void openConnection() {
String url = System.getenv("XIAOZHI_TEST_MYSQL_URL");
Assumptions.assumeTrue(url != null && !url.isBlank(),
"没给 XIAOZHI_TEST_MYSQL_URL,跳过真库迁移测试");
try {
connection = DriverManager.getConnection(url,
System.getenv().getOrDefault("XIAOZHI_TEST_MYSQL_USER", "root"),
System.getenv().getOrDefault("XIAOZHI_TEST_MYSQL_PASSWORD", ""));
} catch (SQLException e) {
Assumptions.abort("连不上 " + url + ":" + e.getMessage());
}
}
@AfterAll
static void closeConnection() throws SQLException {
if (connection == null) {
return;
}
for (String statement : splitStatements("DROP SCHEMA IF EXISTS `" + SCHEMA + "`")) {
connection.createStatement().execute(statement);
}
connection.close();
}
@BeforeEach
void recreateTables() throws SQLException {
execute("DROP SCHEMA IF EXISTS `" + SCHEMA + "`");
execute("CREATE SCHEMA `" + SCHEMA + "`");
// 迁移脚本里的表名不带库名,默认库得先切过去
connection.setCatalog(SCHEMA);
// 建表语句照抄真实脚本,见 202503141346.sql:110 和 202605251426.sql:27。
// schema 变了这里要跟着改,否则测的是一个库线上不存在的形状
execute("""
CREATE TABLE `%s`.`ai_device` (
`id` VARCHAR(32) NOT NULL COMMENT '设备唯一标识',
`user_id` BIGINT COMMENT '关联用户 ID',
`mac_address` VARCHAR(50) COMMENT 'MAC 地址',
`alias` VARCHAR(64) DEFAULT NULL COMMENT '设备别名',
`create_date` DATETIME COMMENT '创建时间',
`update_date` DATETIME COMMENT '更新时间',
PRIMARY KEY (`id`),
INDEX `idx_ai_device_created_at` (`mac_address`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='设备信息表'
""".formatted(SCHEMA));
execute("""
CREATE TABLE `%s`.`ai_device_address_book` (
`mac_address` VARCHAR(64) NOT NULL COMMENT '本设备MAC地址',
`target_mac` VARCHAR(64) NOT NULL COMMENT '对方设备MAC地址',
`alias` VARCHAR(64) DEFAULT NULL COMMENT '别名',
`has_permission` TINYINT(1) DEFAULT TRUE COMMENT '是否有权限呼叫',
PRIMARY KEY (`mac_address`, `target_mac`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci COMMENT='设备通讯录表'
""".formatted(SCHEMA));
}
@Test
@DisplayName("有重复组时,只刷归一后不撞车的那些行")
void collidingGroupIsSkippedButUniqueRowsStillNormalize() throws SQLException {
// 同一台设备被填过两种写法,这两行归一后会撞上,整组一个都不动
insertDevice("d1", "AA-BB-CC-DD-EE-01");
insertDevice("d2", "aabbccddee01");
// 独立的设备,不该因为别处有重复组被连累
insertDevice("x1", "11-22-33-44-55-66");
// 已经是冒号分隔的大写写法,和小写归一值逐字节不同,得刷
insertDevice("u1", "AA:BB:CC:DD:EE:99");
// 32 位伪 MAC 正则不匹配,原样不动,和 Java 那边 normalizeDeviceId 一致
insertDevice("p1", "aabbccddeeff00112233445566778899");
runMigration();
assertEquals("AA-BB-CC-DD-EE-01", deviceMac("d1"), "重复组该整组不动");
assertEquals("aabbccddee01", deviceMac("d2"), "重复组该整组不动");
assertEquals("11:22:33:44:55:66", deviceMac("x1"));
assertEquals("aa:bb:cc:dd:ee:99", deviceMac("u1"));
assertEquals("aabbccddeeff00112233445566778899", deviceMac("p1"));
}
@Test
@DisplayName("没有重复时全部刷成归一值")
void everyUniqueRowNormalizes() throws SQLException {
insertDevice("u1", "AA:BB:CC:DD:EE:99");
insertDevice("u2", "aa-bb-cc-dd-ee-98");
insertDevice("u3", "AABBCCDDEE97");
insertDevice("u4", "aa_bb_cc_dd_ee_96");
insertDevice("u5", "AA-BB-CC-DD-EE-95");
runMigration();
assertEquals("aa:bb:cc:dd:ee:99", deviceMac("u1"));
assertEquals("aa:bb:cc:dd:ee:98", deviceMac("u2"));
assertEquals("aa:bb:cc:dd:ee:97", deviceMac("u3"));
assertEquals("aa:bb:cc:dd:ee:96", deviceMac("u4"));
assertEquals("aa:bb:cc:dd:ee:95", deviceMac("u5"));
}
@Test
@DisplayName("整库都是重复组时一行都不动")
void allCollidingRowsStayUntouched() throws SQLException {
insertDevice("d1", "AA-BB-CC-DD-EE-01");
insertDevice("d2", "aabbccddee01");
insertDevice("d3", "AA:BB:CC:DD:EE:01");
runMigration();
assertEquals("AA-BB-CC-DD-EE-01", deviceMac("d1"));
assertEquals("aabbccddee01", deviceMac("d2"));
assertEquals("AA:BB:CC:DD:EE:01", deviceMac("d3"));
}
@Test
@DisplayName("值已经是对的时空跑,重跑安全")
void rerunningTheMigrationChangesNothing() throws SQLException {
insertDevice("u1", "AA:BB:CC:DD:EE:99");
insertDevice("d1", "AA-BB-CC-DD-EE-01");
insertDevice("d2", "aabbccddee01");
runMigration();
runMigration();
assertEquals("aa:bb:cc:dd:ee:99", deviceMac("u1"));
assertEquals("AA-BB-CC-DD-EE-01", deviceMac("d1"));
assertEquals("aabbccddee01", deviceMac("d2"));
}
@Test
@DisplayName("通讯录两列各自判重:owner 撞车不动 owner,target 照刷")
void addressBookColumnsAreGuardedIndependently() throws SQLException {
// owner 有两种写法会撞,这两行的 owner 不动
insertBook("row1", "AA-BB-CC-DD-EE-01", "11-22-33-44-55-66");
insertBook("row2", "aabbccddee01", "11:22:33:44:55:77");
// 这两行的 owner 各不相同,但 target 是同一台的两种写法:
// owner 照刷,target 整组不动
insertBook("row3", "22-22-33-44-55-88", "AA-BB-CC-DD-EE-02");
insertBook("row4", "33-33-44-55-88-99", "aabbccddee02");
runMigration();
assertEquals("AA-BB-CC-DD-EE-01", bookMac("row1"), "owner 撞车的整组不动");
assertEquals("aabbccddee01", bookMac("row2"), "owner 撞车的整组不动");
assertEquals("11:22:33:44:55:66", bookTarget("row1"), "target 唯一就该刷");
assertEquals("11:22:33:44:55:77", bookTarget("row2"), "target 唯一就该刷");
assertEquals("22:22:33:44:55:88", bookMac("row3"), "owner 唯一就该刷");
assertEquals("AA-BB-CC-DD-EE-02", bookTarget("row3"), "target 撞车的整组不动");
assertEquals("33:33:44:55:88:99", bookMac("row4"));
assertEquals("aabbccddee02", bookTarget("row4"));
}
@Test
@DisplayName("判重条件没了,通讯录会撞主键把整段迁移卡住")
void withoutTheDuplicateGuardTheAddressBookWouldCollideOnItsPrimaryKey() throws SQLException {
// 主键是 (owner, target),两行得撞在这一对上:owner 和 target 都归一成同一个值
insertBook("row1", "AA-BB-CC-DD-EE-01", "11-22-33-44-55-66");
insertBook("row2", "aabbccddee01", "112233445566");
// 从真脚本上把判重摘掉再跑。脚本里已经没判重的话替换就是空操作,
// 跑的还是没有保护的版本,这个用例照样成立
String unguarded = readChangelog("202610231200.sql")
.replace("\nJOIN cnt ON cnt.`nmac` = norm.`nmac`\n", "\n")
.replace("\nJOIN cnt ON cnt.`ntmac` = norm.`ntmac`\n", "\n")
.replace("WHERE cnt.`n` = 1", "WHERE 1 = 1");
SQLException failure = assertThrows(SQLException.class, () -> run(unguarded));
assertEquals("23000", failure.getSQLState(),
"撞的是主键唯一性,没撞上说明这组数据造得不对:" + failure.getMessage());
}
@Test
@DisplayName("排查重复的那条 SQL 报出被跳过的组")
void duplicateDiagnosticListsTheSkippedGroup() throws SQLException {
insertDevice("d1", "AA-BB-CC-DD-EE-01");
insertDevice("d2", "aabbccddee01");
insertDevice("x1", "11-22-33-44-55-66");
runMigration();
assertEquals(List.of("aa:bb:cc:dd:ee:01"),
queryColumn(diagnostic("③ 刷完之后跑这个"), 0),
"只该报出被判重跳过的那个组,刷完了的行不该出现在名单里");
}
@Test
@DisplayName("漏刷检查报出还留在原写法的行")
void leftoverCheckFindsRowsStillInOriginalSpelling() throws SQLException {
insertDevice("d1", "AA-BB-CC-DD-EE-01");
insertDevice("d2", "aabbccddee01");
insertDevice("x1", "11-22-33-44-55-66");
insertDevice("u1", "AA:BB:CC:DD:EE:99");
runMigration();
assertEquals(List.of("AA-BB-CC-DD-EE-01", "aabbccddee01"),
queryColumn(diagnostic("漏刷检查"), 1),
"刷完只剩判重跳过的那两行是原写法,x1、u1 都该刷掉");
}
@Test
@DisplayName("漏刷检查能报出该刷没刷出来的行")
void leftoverCheckFindsRowsTheMigrationNeverTouched() throws SQLException {
insertDevice("d1", "AA-BB-CC-DD-EE-01");
insertDevice("d2", "aabbccddee01");
insertDevice("x1", "11-22-33-44-55-66");
assertEquals(List.of("d1", "d2", "x1"),
queryColumn(diagnostic("漏刷检查"), 0),
"整段迁移一行没动时漏刷检查要全报出来,不然迁移只做了一半没人知道");
}
/** 按 Liquibase 的 sqlFile 那样读脚本:去注释,按分号切,在一个事务里逐句执行 */
private void runMigration() throws SQLException {
run(readChangelog("202610231200.sql"));
}
private void run(String script) throws SQLException {
connection.setAutoCommit(false);
try {
for (String statement : splitStatements(script)) {
execute(statement);
}
connection.commit();
} catch (SQLException e) {
connection.rollback();
throw e;
} finally {
connection.setAutoCommit(true);
}
}
private void insertDevice(String id, String mac) throws SQLException {
execute("INSERT INTO `%s`.`ai_device` (`id`, `mac_address`) VALUES ('%s', '%s')"
.formatted(SCHEMA, id, mac));
}
/** alias 放行标识,mac_address 刷完就变了,靠它才找得到原来那一行 */
private void insertBook(String rowTag, String mac, String target) throws SQLException {
execute(("INSERT INTO `%s`.`ai_device_address_book` (`mac_address`, `target_mac`, `alias`)"
+ " VALUES ('%s', '%s', '%s')").formatted(SCHEMA, mac, target, rowTag));
}
private String deviceMac(String id) throws SQLException {
return queryOne("SELECT `mac_address` FROM `%s`.`ai_device` WHERE `id` = '%s'".formatted(SCHEMA, id));
}
private String bookMac(String rowTag) throws SQLException {
return queryOne("SELECT `mac_address` FROM `%s`.`ai_device_address_book` WHERE `alias` = '%s'"
.formatted(SCHEMA, rowTag));
}
private String bookTarget(String rowTag) throws SQLException {
return queryOne("SELECT `target_mac` FROM `%s`.`ai_device_address_book` WHERE `alias` = '%s'"
.formatted(SCHEMA, rowTag));
}
private String queryOne(String sql) throws SQLException {
try (Statement statement = connection.createStatement();
ResultSet rows = statement.executeQuery(sql)) {
assertTrue(rows.next(), "查不到行:" + sql);
return rows.getString(1);
}
}
private void execute(String sql) throws SQLException {
for (String statement : splitStatements(sql)) {
try (Statement s = connection.createStatement()) {
s.execute(statement);
}
}
}
/** 去注释后按分号切。脚本里除了分号没有别的分句符 */
private static List<String> splitStatements(String sql) {
StringBuilder body = new StringBuilder();
for (String line : sql.split("\n")) {
if (!line.stripLeading().startsWith("--")) {
body.append(line).append('\n');
}
}
List<String> statements = new ArrayList<>();
for (String piece : body.toString().split(";")) {
if (!piece.isBlank()) {
statements.add(piece.strip());
}
}
return statements;
}
private static String readChangelog(String fileName) {
try {
return Files.readString(
Path.of("src", "main", "resources", "db", "changelog", fileName),
StandardCharsets.UTF_8);
} catch (IOException e) {
throw new IllegalStateException("读不到 " + fileName, e);
}
}
/**
* 把脚本里注释掉的那段排查 SQL 整条取出来
*
* <p>
* 脚本里那一段是给人手工跑的,跑不走就等于没写。取出来在真库上执行一遍,
* 段里的 SQL 坏了、引用了不存在的列,这里会直接报错
*/
private static String diagnostic(String marker) {
List<String> collected = new ArrayList<>();
boolean inBlock = false;
for (String line : readChangelog("202610231200.sql").split("\n")) {
if (!line.stripLeading().startsWith("--")) {
if (!collected.isEmpty()) {
break;
}
continue;
}
if (!inBlock) {
inBlock = line.contains(marker);
if (!inBlock) {
continue;
}
}
String body = line.stripLeading().substring(2).strip();
if (collected.isEmpty() && !body.startsWith("SELECT")) {
// 说明文字,去掉行首的 -- 之后就不是注释了,不能拼进语句里
continue;
}
collected.add(body);
if (body.endsWith(";")) {
break;
}
}
assertFalse(collected.isEmpty(), "脚本里找不到 " + marker + " 那段排查 SQL");
return String.join("\n", collected);
}
private List<String> queryColumn(String sql, int column) throws SQLException {
List<String> values = new ArrayList<>();
try (Statement statement = connection.createStatement();
ResultSet rows = statement.executeQuery(sql)) {
while (rows.next()) {
values.add(rows.getString(column + 1));
}
}
return values;
}
}
@@ -0,0 +1,82 @@
package xiaozhi.modules.model.service.impl;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import java.util.UUID;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.transaction.annotation.Transactional;
import cn.hutool.json.JSONObject;
import xiaozhi.common.redis.RedisKeys;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.modules.model.dto.ModelConfigBodyDTO;
import xiaozhi.modules.model.dto.ModelConfigDTO;
import xiaozhi.modules.model.service.ModelConfigService;
import xiaozhi.testsupport.IntegrationTestEnvironment;
@SpringBootTest
@ActiveProfiles("dev")
@Transactional
class ModelConfigPersistenceTest {
@BeforeAll
static void checkDependencies() {
IntegrationTestEnvironment.requireDatabaseAndRedis();
}
@Autowired
private ModelConfigService service;
@Autowired
private JdbcTemplate jdbc;
@Autowired
private RedisUtils redis;
@ParameterizedTest
@CsvSource({"tts,TTS", "memory,Memory", "intent,Intent"})
void preservesRegisteredTypeWhenAddingAndEditing(String requestedType, String registeredType) {
String id = UUID.randomUUID().toString().replace("-", "");
String providerCode = "test-" + id;
jdbc.update("INSERT INTO ai_model_provider (id, model_type, provider_code, name, fields) VALUES (?, ?, ?, ?, ?)",
id, registeredType, providerCode, "Persistence test provider", "[]");
ModelConfigBodyDTO body = new ModelConfigBodyDTO();
body.setId(id);
body.setModelCode(providerCode);
body.setModelName("Persistence test model");
body.setIsDefault(0);
body.setIsEnabled(1);
body.setSort(1);
body.setConfigJson(new JSONObject().set("type", providerCode));
ModelConfigDTO added = service.add(requestedType, providerCode, body);
assertEquals(registeredType, added.getModelType());
assertEquals(registeredType,
jdbc.queryForObject("SELECT model_type FROM ai_model_config WHERE id = ?", String.class, id));
String cacheKey = RedisKeys.getModelConfigById(id);
try {
redis.set(cacheKey, "cached model");
body.setModelName("Updated persistence test model");
ModelConfigDTO edited = service.edit(requestedType, providerCode, id, body);
assertEquals(registeredType, edited.getModelType());
assertEquals(registeredType,
jdbc.queryForObject("SELECT model_type FROM ai_model_config WHERE id = ?", String.class, id));
assertEquals(body.getModelName(),
jdbc.queryForObject("SELECT model_name FROM ai_model_config WHERE id = ?", String.class, id));
assertNull(redis.get(cacheKey));
} finally {
redis.delete(cacheKey);
}
}
}
@@ -0,0 +1,120 @@
package xiaozhi.modules.model.service.impl;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.junit.jupiter.params.provider.ValueSource;
import org.mockito.ArgumentCaptor;
import org.mockito.MockedStatic;
import xiaozhi.common.exception.ErrorCode;
import xiaozhi.common.exception.RenException;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.common.utils.MessageUtils;
import xiaozhi.modules.agent.dao.AgentDao;
import xiaozhi.modules.model.dao.ModelConfigDao;
import xiaozhi.modules.model.dto.ModelConfigBodyDTO;
import xiaozhi.modules.model.dto.ModelConfigDTO;
import xiaozhi.modules.model.dto.ModelProviderDTO;
import xiaozhi.modules.model.entity.ModelConfigEntity;
import xiaozhi.modules.model.service.ModelProviderService;
class ModelConfigServiceImplTest {
private ModelConfigDao modelConfigDao;
private ModelProviderService modelProviderService;
private ModelConfigServiceImpl service;
@BeforeEach
void setUp() {
modelConfigDao = mock(ModelConfigDao.class);
modelProviderService = mock(ModelProviderService.class);
service = new ModelConfigServiceImpl(
modelConfigDao, modelProviderService, mock(RedisUtils.class), mock(AgentDao.class));
}
@ParameterizedTest
@CsvSource({
"vad, VAD", "asr, ASR", "tts, TTS", "llm, LLM", "vllm, VLLM",
"memory, Memory", "intent, Intent", "rag, RAG", "LLM, LLM"
})
void addPreservesRegisteredProviderModelType(String requestedType, String registeredType) {
registerProvider(requestedType, registeredType);
ModelConfigDTO result = service.add(requestedType, "test-provider", modelBody());
ArgumentCaptor<ModelConfigEntity> saved = ArgumentCaptor.forClass(ModelConfigEntity.class);
verify(modelConfigDao).insert(saved.capture());
assertEquals(registeredType, saved.getValue().getModelType());
assertEquals(registeredType, result.getModelType());
}
@ParameterizedTest
@CsvSource({
"vad, VAD", "asr, ASR", "tts, TTS", "llm, LLM", "vllm, VLLM",
"memory, Memory", "intent, Intent", "rag, RAG", "LLM, LLM"
})
void editPreservesRegisteredProviderModelType(String requestedType, String registeredType) {
registerProvider(requestedType, registeredType);
ModelConfigEntity original = new ModelConfigEntity();
original.setId("test-model");
original.setModelType(registeredType);
when(modelConfigDao.selectById("test-model")).thenReturn(original);
ModelConfigDTO result = service.edit(requestedType, "test-provider", "test-model", modelBody());
ArgumentCaptor<ModelConfigEntity> saved = ArgumentCaptor.forClass(ModelConfigEntity.class);
verify(modelConfigDao).updateById(saved.capture());
assertEquals(registeredType, saved.getValue().getModelType());
assertEquals(registeredType, result.getModelType());
}
@ParameterizedTest
@ValueSource(booleans = {false, true})
void rejectsUnknownProviderBeforeWriting(boolean editing) {
when(modelProviderService.getList("tts", "missing")).thenReturn(List.of());
try (MockedStatic<MessageUtils> messages = mockStatic(MessageUtils.class)) {
messages.when(() -> MessageUtils.getMessage(ErrorCode.MODEL_PROVIDER_NOT_EXIST))
.thenReturn("Model provider does not exist");
RenException error = assertThrows(RenException.class, () -> {
if (editing) {
service.edit("tts", "missing", "test-model", modelBody());
} else {
service.add("tts", "missing", modelBody());
}
});
assertEquals(ErrorCode.MODEL_PROVIDER_NOT_EXIST, error.getCode());
}
verify(modelConfigDao, never()).insert(any(ModelConfigEntity.class));
verify(modelConfigDao, never()).updateById(any(ModelConfigEntity.class));
}
private void registerProvider(String requestedType, String registeredType) {
ModelProviderDTO provider = new ModelProviderDTO();
provider.setModelType(registeredType);
provider.setProviderCode("test-provider");
when(modelProviderService.getList(requestedType, "test-provider")).thenReturn(List.of(provider));
}
private ModelConfigBodyDTO modelBody() {
ModelConfigBodyDTO body = new ModelConfigBodyDTO();
body.setId("test-model");
body.setModelName("Test model");
body.setIsEnabled(1);
body.setSort(1);
return body;
}
}
@@ -4,6 +4,7 @@ import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.Mockito.when;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
@@ -18,12 +19,18 @@ import xiaozhi.modules.security.dto.LoginDTO;
import xiaozhi.modules.security.dto.SmsVerificationDTO;
import xiaozhi.modules.sys.dto.RetrievePasswordDTO;
import xiaozhi.modules.sys.service.SysUserService;
import xiaozhi.testsupport.IntegrationTestEnvironment;
@Slf4j
@SpringBootTest
@ActiveProfiles("dev")
class loginControllerTest {
@BeforeAll
static void checkDependencies() {
IntegrationTestEnvironment.requireDatabaseAndRedis();
}
@Autowired
LoginController loginController;
@@ -0,0 +1,149 @@
package xiaozhi.testsupport;
import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.time.Duration;
import org.junit.jupiter.api.Assumptions;
import org.springframework.boot.env.YamlPropertySourceLoader;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.PropertySource;
import org.springframework.core.env.PropertySourcesPropertyResolver;
import org.springframework.core.env.StandardEnvironment;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import io.lettuce.core.RedisClient;
import io.lettuce.core.RedisURI;
import io.lettuce.core.api.StatefulRedisConnection;
import lombok.extern.slf4j.Slf4j;
/**
* 几个 @SpringBootTest 要真连 MySQL 和 Redis。本地没起这两个依赖时,
* 整类测试跳过,别让 Spring 上下文起不来变成一堆看不懂的报错;
* CI 里依赖必须真的在,探不通就直接让构建红。
*/
@Slf4j
public final class IntegrationTestEnvironment {
/** 集成测试用的配置文件,跟应用本身读的是同一套 */
private static final String[] CONFIG_FILES = {"application.yml", "application-dev.yml"};
private static Boolean available;
private static String problem;
private IntegrationTestEnvironment() {
}
/**
* 放在要连库的测试类的 @BeforeAll 里。
* 本地依赖不齐就整类跳过,CI 里依赖不齐就直接失败。
*/
public static void requireDatabaseAndRedis() {
String issue = probe();
if (issue == null) {
return;
}
if (isCi()) {
throw new IllegalStateException(
"CI 环境里" + issue + ",这几个集成测试必须真的跑起来,不能跳过");
}
log.warn("跳过依赖 MySQL 和 Redis 的集成测试:{}", issue);
Assumptions.assumeTrue(false, issue);
}
/** GitHub Actions 上会带 CI=true */
public static boolean isCi() {
String value = System.getenv("CI");
if (value == null) {
return false;
}
String trimmed = value.trim();
return !trimmed.isEmpty() && !"false".equalsIgnoreCase(trimmed);
}
/** 依赖都齐返回 null,不齐返回缺什么 */
static synchronized String probe() {
if (available != null) {
return problem;
}
PropertySourcesPropertyResolver resolver = resolver();
StringBuilder issues = new StringBuilder();
String url = firstNonBlank(resolver.getProperty("spring.datasource.druid.url", ""),
resolver.getProperty("spring.datasource.url", ""));
String username = firstNonBlank(resolver.getProperty("spring.datasource.druid.username", ""),
resolver.getProperty("spring.datasource.username", ""));
String password = firstNonBlank(resolver.getProperty("spring.datasource.druid.password", ""),
resolver.getProperty("spring.datasource.password", ""));
if (url.isBlank()) {
issues.append("配置里找不到数据源地址;");
} else {
try {
DriverManager.setLoginTimeout(5);
try (Connection connection = DriverManager.getConnection(url, username, password)) {
connection.isValid(3);
}
} catch (Exception e) {
issues.append("连不上 MySQL(").append(e.getMessage()).append(");");
}
}
String redisHost = resolver.getProperty("spring.data.redis.host", "localhost");
int redisPort = resolver.getProperty("spring.data.redis.port", Integer.class, 6379);
String redisPassword = resolver.getProperty("spring.data.redis.password", "");
try {
RedisURI.Builder builder = RedisURI.builder()
.withHost(redisHost)
.withPort(redisPort)
.withTimeout(Duration.ofSeconds(3));
if (redisPassword != null && !redisPassword.isBlank()) {
builder.withPassword(redisPassword.toCharArray());
}
RedisClient client = RedisClient.create(builder.build());
try {
StatefulRedisConnection<String, String> connection = client.connect();
connection.sync().ping();
connection.close();
} finally {
client.shutdown();
}
} catch (Exception e) {
issues.append("连不上 Redis(").append(e.getMessage()).append(");");
}
if (issues.length() == 0) {
available = Boolean.TRUE;
problem = null;
} else {
available = Boolean.FALSE;
problem = issues.toString();
}
return problem;
}
/** 读应用自己那份配置,命令行上的 -D 参数优先级更高,跟应用保持一致 */
private static PropertySourcesPropertyResolver resolver() {
ConfigurableEnvironment environment = new StandardEnvironment();
YamlPropertySourceLoader loader = new YamlPropertySourceLoader();
for (String name : CONFIG_FILES) {
Resource resource = new ClassPathResource(name);
if (!resource.exists()) {
continue;
}
try {
for (PropertySource<?> source : loader.load(name, resource)) {
environment.getPropertySources().addLast(source);
}
} catch (IOException e) {
log.warn("读配置 {} 失败:{}", name, e.getMessage());
}
}
return new PropertySourcesPropertyResolver(environment.getPropertySources());
}
private static String firstNonBlank(String first, String second) {
return first != null && !first.isBlank() ? first : (second == null ? "" : second);
}
}
@@ -265,7 +265,7 @@ function showAbout() {
title: t('settings.aboutApp', { appName: import.meta.env.VITE_APP_TITLE }),
content: t('settings.aboutContent', {
appName: import.meta.env.VITE_APP_TITLE,
version: '0.9.7',
version: '0.9.8',
}),
showCancel: false,
confirmText: t('common.confirm'),
+120 -23
View File
@@ -1,6 +1,7 @@
import Fly from 'flyio/dist/npm/fly';
import store from '../store/index';
import Constant from '../utils/constant';
import { getServiceUrl } from '../apis/api';
import { goToPage, isNotNull, showDanger, showWarning } from '../utils/index';
import i18n from '../i18n/index';
@@ -8,13 +9,98 @@ const fly = new Fly()
// 设置超时
fly.config.timeout = 30000
// 独立 fly 实例给 silent refresh 用,避免任何回调链回到 sendRequest
const refreshFly = new Fly()
refreshFly.config.timeout = 30000
// silent refresh:剩 1h 触发换 token,单飞锁避免并发
const REFRESH_AHEAD_MS = 60 * 60 * 1000
let refreshPromise = null
function getTokenExpireAt() {
// 与 persistTokenExpireAt 同源读 localStorage(store 走的是 state.token 缓存,
// 刚写入 localStorage 后再走 store 拿不到补写的 _localStoredAt)
const raw = localStorage.getItem('token')
if (!raw) return 0
try {
const parsed = JSON.parse(raw)
if (!parsed || !parsed.expire) return 0
// login.vue 写入 store 时是 setToken(now),token 对象里没有 createdAt;
// 用 _localStoredAt 作为本地落地时间基准
return (parsed._localStoredAt || 0) + parsed.expire * 1000
} catch (e) {
return 0
}
}
// 首次见到该 token 时记下落地时间;同步刷新 localStorage 与 store,保证两端一致
function persistTokenExpireAt() {
const raw = localStorage.getItem('token')
if (!raw) return
try {
const parsed = JSON.parse(raw)
if (parsed && parsed.expire && !parsed._localStoredAt) {
parsed._localStoredAt = Date.now()
const next = JSON.stringify(parsed)
localStorage.setItem('token', next)
store.commit('setToken', next)
}
} catch (e) { /* ignore */ }
}
function shouldRefresh() {
persistTokenExpireAt()
const exp = getTokenExpireAt()
if (!exp) return false
return exp - Date.now() < REFRESH_AHEAD_MS
}
function doRefresh() {
// 直接走 fly.request,绕过 sendRequest,避免死循环触发 silent refresh
const raw = localStorage.getItem('token')
if (!raw) return Promise.reject(new Error('no token'))
let bearer
try {
bearer = JSON.parse(raw).token
} catch (e) {
return Promise.reject(e)
}
return refreshFly.request(getServiceUrl() + '/user/refresh', {}, {
method: 'POST',
headers: { Authorization: 'Bearer ' + bearer }
}).then((res) => {
const body = res && res.data
if (!body || (body.code !== 0 && body.code !== 'success')) {
throw new Error('refresh failed: ' + (body && body.msg))
}
const newDto = body.data
newDto._localStoredAt = Date.now()
store.commit('setToken', JSON.stringify(newDto))
return newDto
})
}
function triggerSilentRefresh() {
if (!refreshPromise) {
refreshPromise = doRefresh().catch((err) => {
store.commit('clearAuth')
goToPage(Constant.PAGE.LOGIN, true)
throw err
}).finally(() => {
refreshPromise = null
})
}
return refreshPromise
}
/**
* Request服务封装
*/
export default {
sendRequest,
reAjaxFun,
clearRequestTime
clearRequestTime,
triggerSilentRefresh
}
function sendRequest() {
@@ -37,30 +123,41 @@ function sendRequest() {
acceptLanguage = 'en-US';
}
this._header['Accept-Language'] = acceptLanguage;
if (isNotNull(store.getters.getToken)) {
this._header.Authorization = 'Bearer ' + (JSON.parse(store.getters.getToken)).token
const doSend = () => {
if (isNotNull(store.getters.getToken)) {
this._header.Authorization = 'Bearer ' + (JSON.parse(store.getters.getToken)).token
}
// 打印请求信息
fly.request(this._url, this._data, {
method: this._method,
headers: this._header,
responseType: this._responseType
}).then((res) => {
const error = httpHandlerError(res, this._failCallback, this._networkFailCallback);
if (error) {
return
}
if (this._sucCallback) {
this._sucCallback(res)
}
}).catch((res) => {
// 打印失败响应
console.log('catch', res)
httpHandlerError(res, this._failCallback, this._networkFailCallback)
})
}
// 打印请求信息
fly.request(this._url, this._data, {
method: this._method,
headers: this._header,
responseType: this._responseType
}).then((res) => {
const error = httpHandlerError(res, this._failCallback, this._networkFailCallback);
if (error) {
return
}
if (this._sucCallback) {
this._sucCallback(res)
}
}).catch((res) => {
// 打印失败响应
console.log('catch', res)
httpHandlerError(res, this._failCallback, this._networkFailCallback)
})
// 剩 1h 内过期就先 silent refresh 再发,确保请求带新 token
if (shouldRefresh()) {
triggerSilentRefresh().then(doSend).catch(() => {
// 触发 refresh 后被拦截跳登录,这里什么都不用做
})
} else {
doSend()
}
return this
},
'success'(callback) {
@@ -77,7 +77,14 @@
<el-form :model="formData.configJson" label-width="auto" label-position="left" class="custom-form">
<div v-for="(row, rowIndex) in chunkedCallInfoFields" :key="rowIndex" class="form-row">
<el-form-item v-for="field in row" :key="field.prop" :label="field.label" :prop="field.prop" style="flex: 1;">
<el-input v-model="formData.configJson[field.prop]" :placeholder="field.placeholder"
<el-select v-if="field.type === 'select'" v-model="formData.configJson[field.prop]"
:placeholder="field.placeholder" class="custom-input-bg" style="width: 100%">
<el-option v-for="opt in field.options" :key="opt" :label="opt" :value="opt"></el-option>
</el-select>
<el-input v-else-if="field.type === 'json-textarea'" v-model="formData.configJson[field.prop]" type="textarea"
:rows="3" :placeholder="field.placeholder" class="custom-input-bg">
</el-input>
<el-input v-else v-model="formData.configJson[field.prop]" :placeholder="field.placeholder"
:type="field.type || 'text'" class="custom-input-bg" :show-password="field.type === 'password'">
</el-input>
</el-form-item>
@@ -161,7 +168,8 @@ export default {
fields: JSON.parse(item.fields || '[]').map(f => ({
label: f.label,
prop: f.key,
type: f.type === 'password' ? 'password' : 'text',
type: f.type === 'dict' ? 'json-textarea' : f.type === 'password' ? 'password' : (f.type === 'select' ? 'select' : 'text'),
options: f.options || [],
placeholder: `请输入${f.key}`
}))
}))
@@ -204,6 +212,24 @@ export default {
return;
}
// dict 字段必须解析为 JSON 对象再提交,否则会以字符串入库
for (const field of this.providerFields) {
if (field.type !== 'json-textarea') continue;
const raw = this.formData.configJson[field.prop];
if (!raw || typeof raw === 'object') continue;
try {
const parsed = JSON.parse(raw);
if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) {
throw new Error('not object');
}
this.formData.configJson[field.prop] = parsed;
} catch (e) {
this.$message.error('必须输入字典格式(如 {"key":"value"})');
this.saving = false;
return;
}
}
const submitData = {
id: this.formData.id || '',
modelName: this.formData.modelName || '',
@@ -81,6 +81,11 @@
"></el-input>
</template>
<el-select v-else-if="field.type === 'select'" v-model="form.configJson[field.prop]"
:placeholder="field.placeholder" style="width: 100%">
<el-option v-for="opt in field.options" :key="opt" :label="opt" :value="opt"></el-option>
</el-select>
<el-input v-else v-model="form.configJson[field.prop]" :placeholder="field.placeholder" :type="field.type"
:show-password="field.type === 'password'" @focus="
isSensitiveField(field.prop)
@@ -312,7 +317,10 @@ export default {
? "json-textarea"
: f.type === "password"
? "password"
: "text",
: f.type === "select"
? "select"
: "text",
options: f.options || [],
placeholder: `请输入${f.key}`,
}));
@@ -330,6 +338,17 @@ export default {
if (!configJson.hasOwnProperty(field.prop)) {
configJson[field.prop] = "";
} else if (field.type === "json-textarea") {
// 兼容已保存为 JSON 字符串的字典,避免再次序列化后无法编辑。
if (typeof configJson[field.prop] === "string") {
try {
const parsed = JSON.parse(configJson[field.prop]);
if (parsed !== null && typeof parsed === "object" && !Array.isArray(parsed)) {
configJson[field.prop] = parsed;
}
} catch {
// 保留非法值的显示,由现有校验提示用户修正。
}
}
this.$set(
this.fieldJsonMap,
field.prop,
+7
View File
@@ -22,6 +22,13 @@ Vue.config.productionTip = false
// 注册Service Worker
registerServiceWorker();
// 应用启动时把 localStorage 里的 token 同步进 store,避免守卫放行后首次请求时
// httpRequest 读到 state.token=null 而漏带 Authorization 头触发 401 跳登录
const cachedToken = localStorage.getItem('token')
if (cachedToken) {
store.commit('setToken', cachedToken)
}
// 创建Vue实例
new Vue({
router,
@@ -613,6 +613,8 @@ export default {
.nav-panel {
min-width: 242px;
height: 100%;
box-sizing: border-box;
overflow-y: auto;
border-right: 1px solid #ebeef5;
background: linear-gradient(
120deg,
@@ -629,6 +631,7 @@ export default {
.nav-panel .el-menu-item {
height: 50px;
flex-shrink: 0;
background: #e9f0ff;
line-height: 50px;
border-radius: 4px 0 0 4px !important;
@@ -14,6 +14,10 @@ const functionDialogSource = await readFile(
new URL('../src/components/FunctionDialog.vue', import.meta.url),
'utf8',
);
const modelConfigSource = await readFile(
new URL('../src/views/ModelConfig.vue', import.meta.url),
'utf8',
);
test('address-book permission state consistently uses the target device MAC', () => {
assert.match(
@@ -66,6 +70,74 @@ test('correct-word pagination maps the UI page size to the backend limit query',
assert.doesNotMatch(correctWordApiSource, /pageSize: params\.pageSize/);
});
test('model navigation keeps its scrollable layout declarations', () => {
const navStyles = modelConfigSource.match(/\.nav-panel\s*\{([^}]*)\}/)?.[1];
const itemStyles = modelConfigSource.match(
/\.nav-panel\s+\.el-menu-item\s*\{([^}]*)\}/,
)?.[1];
assert.ok(navStyles, 'model navigation should define its layout');
assert.ok(itemStyles, 'model menu items should define their layout');
const wrapperStyles = modelConfigSource.match(/\.main-wrapper\s*\{([^}]*)\}/)?.[1];
assert.ok(wrapperStyles, 'model page wrapper should define its height');
assert.match(navStyles, /box-sizing:\s*border-box;/);
assert.match(navStyles, /overflow-y:\s*auto;/);
assert.match(navStyles, /display:\s*flex;/);
assert.match(navStyles, /flex-direction:\s*column;/);
assert.match(itemStyles, /flex-shrink:\s*0;/);
assert.match(wrapperStyles, /height:\s*calc\([^)]*vh[^)]*\)/);
// 断言只取第一条同名规则,后面再补一条就能盖掉上面三条,必须挡住
assert.equal(
(modelConfigSource.match(/\.nav-panel\s*\{/g) || []).length,
1,
'model navigation should keep a single rule',
);
assert.equal(
(modelConfigSource.match(/\.nav-panel\s+\.el-menu-item\s*\{/g) || []).length,
1,
'model menu items should keep a single rule',
);
// 复合选择器特异性更高,能直接盖掉可滚属性
assert.equal(
(modelConfigSource.match(/\.nav-panel[.:\[#][^{}]*\{/g) || []).length,
0,
'model navigation should not use composite selectors',
);
const styleSource = modelConfigSource.match(/<style[^>]*>([\s\S]*?)<\/style>/)?.[1];
assert.ok(styleSource, 'model config should define its styles');
const ruleSelectors = styleSource
.replace(/\/\*[\s\S]*?\*\//g, '')
.replace(/\/\/[^\n]*/g, '')
.split('}')
.map((chunk) => {
const open = chunk.indexOf('{');
return open < 0 ? null : chunk.slice(0, open).trim();
})
.filter((selector) => selector && !selector.startsWith('@'));
// 上面几条挡不住带伪类的后代选择器(.nav-panel .el-menu-item:hover),
// 它特异性一样更高,照样能盖掉可滚属性,所以把 nav 相关的选择器逐条列出来核对
assert.deepEqual(
ruleSelectors.filter((selector) => selector.includes('.nav-panel')).sort(),
[
'.nav-panel',
'.nav-panel .el-menu-item',
'.nav-panel .el-menu-item.is-active',
'.nav-panel .el-menu-item.is-active::before',
'::v-deep .nav-panel .el-menu-item.is-active .menu-text',
].sort(),
'model navigation should keep its declared rules',
);
assert.deepEqual(
ruleSelectors.filter((selector) => selector.includes('.main-wrapper')),
['.main-wrapper'],
'model page wrapper should keep a single rule',
);
});
test('function dialog footer stays above the expanding MCP tools section', () => {
const mcpLayer = functionDialogSource.match(
/\.mcp-access-point\s*\{[^}]*z-index:\s*(\d+);/s,
@@ -0,0 +1,79 @@
import Vue from 'vue';
import { parseComponent } from 'vue-template-compiler';
import { describe, expect, it, vi } from 'vitest';
import source from '../src/components/AddModelDialog.vue?raw';
// The unit-test runner has no Vue SFC plugin. Load the real component's script
// with its API and dialog dependencies stubbed; keep Vue's reactivity.
const script = parseComponent(source).script.content
.replace(/^import .*;$/gm, '')
.replace('export default', 'return');
const component = new Function('CustomDialog', 'Api', script)({}, {});
function makeVm() {
const vm = new Vue({
...component,
propsData: { visible: false, modelType: 'TTS' },
});
vm.$message = { error: vi.fn() };
vm.$t = (key) => key;
vm.providerFields = [
{ label: '音量', prop: 'volume', type: 'json-textarea', placeholder: '请输入volume' },
{ label: '密钥', prop: 'api_key', type: 'password', placeholder: '请输入api_key' },
];
vm.formData.supplier = 'test-provider';
return vm;
}
function submitted(vm) {
let captured;
vm.$on('confirm', (data) => {
captured = data;
});
vm.confirm();
return captured;
}
describe('add dialog dict fields', () => {
it('parses valid JSON dict input into an object before submit', () => {
const vm = makeVm();
vm.formData.configJson = { volume: '{"speed":1,"volume":10}', api_key: 'sk-xxx' };
const data = submitted(vm);
expect(data.configJson.volume).toEqual({ speed: 1, volume: 10 });
expect(data.configJson.api_key).toBe('sk-xxx');
expect(vm.$message.error).not.toHaveBeenCalled();
});
it('blocks invalid JSON input and does not emit confirm', () => {
const vm = makeVm();
vm.formData.configJson = { volume: '{bad json', api_key: 'sk-xxx' };
expect(submitted(vm)).toBeUndefined();
expect(vm.$message.error).toHaveBeenCalled();
});
it.each(['[1,2]', 'null', '"text"', '1'])(
'blocks non-object JSON %s',
(value) => {
const vm = makeVm();
vm.formData.configJson = { volume: value, api_key: 'sk-xxx' };
expect(submitted(vm)).toBeUndefined();
expect(vm.$message.error).toHaveBeenCalled();
},
);
it('leaves empty strings and existing object values untouched', () => {
const vm = makeVm();
const obj = { speed: 2 };
vm.formData.configJson = { volume: obj, api_key: '' };
const data = submitted(vm);
expect(data.configJson.volume).toBe(obj);
expect(data.configJson.api_key).toBe('');
expect(vm.$message.error).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,98 @@
import Vue from 'vue';
import { parseComponent } from 'vue-template-compiler';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import source from '../src/components/ModelEditDialog.vue?raw';
// The unit-test runner has no Vue SFC plugin. Load the real component's script
// with its API and child-dialog dependencies stubbed; keep Vue's reactivity.
const script = parseComponent(source).script.content
.replace(/^import .*;$/gm, '')
.replace('export default', 'return');
const component = new Function('CustomDialog', 'Api', script)({}, {});
let vm;
let saved;
function openModel(value) {
vm.processModelData({
id: 'test-model',
modelName: 'Test model',
configJson: { type: 'test-provider', prosody: value },
});
}
beforeEach(() => {
vi.useFakeTimers();
vm = new Vue({
...component,
propsData: { modelType: 'TTS', modelData: { id: 'test-model' } },
});
vm.$message = { error: vi.fn() };
vm.dynamicCallInfoFields = [{ prop: 'prosody', type: 'json-textarea' }];
saved = vi.fn();
vm.$on('save', saved);
});
afterEach(() => {
vm.$destroy();
vi.clearAllTimers();
vi.useRealTimers();
});
describe('stored dictionary fields', () => {
it('opens a JSON string as an editable dictionary instead of a quoted string', () => {
openModel('{"speed":1,"volume":10}');
expect(JSON.parse(vm.fieldJsonMap.prosody)).toEqual({ speed: 1, volume: 10 });
expect(vm.form.configJson.prosody).toEqual({ speed: 1, volume: 10 });
});
it('preserves a stored JSON string when saving without changes', () => {
openModel('{"speed":1,"volume":10}');
vm.handleSave();
expect(vm.$message.error).not.toHaveBeenCalled();
expect(saved.mock.calls[0][0].formData.configJson.prosody).toEqual({
speed: 1,
volume: 10,
});
});
it('keeps edits after saving and reopening the returned object', () => {
openModel('{"speed":1,"volume":10}');
vm.fieldJsonMap.prosody = '{"speed":2,"volume":5}';
vm.handleSave();
const config = saved.mock.calls[0][0].formData.configJson;
vm.resetForm();
openModel(JSON.parse(JSON.stringify(config.prosody)));
vm.handleSave();
expect(JSON.parse(vm.fieldJsonMap.prosody)).toEqual({ speed: 2, volume: 5 });
expect(saved.mock.calls[1][0].formData.configJson.prosody).toEqual({
speed: 2,
volume: 5,
});
expect(vm.$message.error).not.toHaveBeenCalled();
});
it('preserves nested values and masked secrets in existing dictionary objects', () => {
const value = { speed: 0, enabled: false, nested: { api_key: 'ab****yz' }, tags: ['a'] };
openModel(value);
vm.handleSave();
expect(saved.mock.calls[0][0].formData.configJson.prosody).toEqual(value);
expect(vm.$message.error).not.toHaveBeenCalled();
});
it.each(['not json', 'null', '[]', '1', 'true', '"text"'])(
'does not reinterpret %s as a dictionary',
(value) => {
openModel(value);
expect(JSON.parse(vm.fieldJsonMap.prosody)).toBe(value);
expect(vm.validateJson(vm.fieldJsonMap.prosody)).toBeNull();
expect(vm.$message.error).toHaveBeenCalled();
},
);
});
+4
View File
@@ -10,6 +10,7 @@ from core.http_server import SimpleHttpServer
from core.websocket_server import WebSocketServer
from core.utils.util import check_ffmpeg_installed
from core.utils.gc_manager import get_gc_manager
from core.handle.helloHandle import preheat_wakeup_audio_cache
TAG = __name__
logger = setup_logging()
@@ -64,6 +65,9 @@ async def main():
# 添加 stdin 监控任务
stdin_task = asyncio.create_task(monitor_stdin())
# 预热唤醒词音频编码缓存,让首条连接就命中
asyncio.create_task(preheat_wakeup_audio_cache())
# 启动全局GC管理器(5分钟清理一次)
gc_manager = get_gc_manager(interval_seconds=300)
await gc_manager.start()
+21 -51
View File
@@ -32,40 +32,9 @@ server:
# 是否启用认证
enabled: false
# 白名单设备ID列表
# 命中白名单只延长该设备 token 的有效期,不再跳过 token 校验
# 格式为设备上报的 Device-Id(MAC),例如 "aa:bb:cc:dd:ee:ff"
# 不要把 Client-Id 填进来:它是 UUID v4,擦 FLASH 或重装后会变化,
# 当长期标识会导致设备直接被锁在门外
allowed_devices: []
# 白名单设备的 token 有效期(秒)。留空表示与 expire_seconds 一致,
# 不因进白名单就放宽;确有需要时可设置,硬上限 90 天,超出会被截断
allowed_devices_expire_seconds: null
# 白名单设备首次接入免凭证换 token,每台设备只放行一次。
# 升级前靠白名单免 token 的存量设备,靠它换到凭证,否则升级即掉线。
# 白名单是运维显式列出的可信 MAC,这里不新增任何信任来源。
# 若配了 activation_codes 或不想要这条自举路径,设成 false
allowed_devices_bootstrap: true
# 一次性激活码,供没有 token 的设备首次接入时换取 token,用掉即失效
# 新设备接入流程:在此列表中预先加入一个码,设备带着该码请求 OTA 接口,
# 服务端签发 token 后自动从列表移除该码
# 写成 "码: Device-Id" 表示该码只能给这台设备用,例如 ["a1b2c3: aa:bb:cc:dd:ee:ff"];
# 裸字符串表示不限设备。码本身不能带冒号
activation_codes: []
# token 吊销线(Unix 秒):签发时刻早于它的 token 一律判为无效。
# 这是作废旧凭证的开关,不是"让设备换发新凭证"的开关——改大之后,
# 存量设备带旧 token 请求 OTA 会被拒,没有激活码就换不到新 token,
# 全网设备会一起下线。作废前请先确认 activation_codes 里有可用的码
issued_after: 0
# CORS 允许的来源列表,默认空表示不做跨域放行
# 需要跨域时填具体来源,例如 ["https://your-console.example.com"]
# 注意:填 "*" 不会放行任意来源。来源不在名单里就不下发跨域头,
# 浏览器会直接拦下,带凭证的请求仍受同源保护
# 仓库自带的测试页(main/digital-human)用 HTTP 服务打开时来源是
# http://127.0.0.1:8006 或 http://localhost:8006——这是两个不同来源,
# 按哪种方式打开就得把哪个填进来,只填一个照样连不上。
# 用 file:// 直接双击打开的,页面会自己弹窗挡住,配置这一项没意义
cors:
allowed_origins: []
# 如果属于白名单内的设备,不校验token,直接放行
allowed_devices:
- "11:22:33:44:55:66"
# MQTT网关配置,用于通过OTA下发到设备,根据mqtt_gateway的.env文件配置,格式为host:port
mqtt_gateway: null
# MQTT签名密钥,用于生成MQTT连接密码,根据mqtt_gateway的.env文件配置
@@ -203,18 +172,19 @@ plugins:
# ragflow知识库id
dataset_ids: ["123456789"]
# 联网搜索插件配置
# 支持秘塔(metaso)、Tavily和Serply三个搜索源
# 支持秘塔(metaso)、Tavily、Serply和Outcrawl四个搜索源
# 秘塔API Key申请:https://metaso.cn/search-api/api-keys
# Tavily API Key申请:https://app.tavily.com/home
# Serply API Key申请:https://serply.io
# Outcrawl API Key申请:https://outcrawl.ai
web_search:
# 搜索源:metaso / tavily / serply
# 搜索源:metaso / tavily / serply / outcrawl
provider: "metaso"
# 工具描述,用于告知LLM何时调用此工具,可根据需要自定义
description: "联网搜索工具。当用户明确需要联网搜索问题时使用此工具。"
# 返回结果数量(默认5)
max_results: 5
# 秘塔API Key(mk-前缀) / Tavily API Key(tvly-前缀) / Serply API Key
# 秘塔API Key(mk-前缀) / Tavily API Key(tvly-前缀) / Serply API Key / Outcrawl API Key
api_key: "mk-xxx"
# 声纹识别配置
voiceprint:
@@ -429,8 +399,8 @@ ASR:
# DoubaoASR和DoubaoStreamASR的区别是:DoubaoASR是按次收费,DoubaoStreamASR是按时收费
# 一般来说按次收费的更便宜,但是DoubaoStreamASR使用了大模型技术,效果更好
type: doubao
appid: 你的火山引擎语音合成服务appid
access_token: 你的火山引擎语音合成服务access_token
appid: 你的火山引擎语音识别服务appid
access_token: 你的火山引擎语音识别服务access_token
cluster: volcengine_input_common
# 热词、替换词使用流程:https://www.volcengine.com/docs/6561/155738
boosting_table_name: (选填)你的热词文件名称
@@ -443,8 +413,8 @@ ASR:
# 开通地址https://console.volcengine.com/speech/service/10011
# 一般来说按次收费的更便宜,但是DoubaoStreamASR使用了大模型技术,效果更好
type: doubao_stream
appid: 你的火山引擎语音合成服务appid
access_token: 你的火山引擎语音合成服务access_token
appid: 你的火山引擎语音识别服务appid
access_token: 你的火山引擎语音识别服务access_token
# 资源ID:小时版 volc.bigasr.sauc.duration,并发版 volc.bigasr.sauc.concurrent
resource_id: volc.bigasr.sauc.duration
# 热词、替换词使用流程:https://www.volcengine.com/docs/6561/155738
@@ -463,8 +433,8 @@ ASR:
# 开通地址:https://console.volcengine.com/speech/service/10038
# 价格更为便宜,建议在高并发场景下使用
type: doubao_stream
appid: 你的火山引擎语音合成服务appid
access_token: 你的火山引擎语音合成服务access_token
appid: 你的火山引擎语音识别服务appid
access_token: 你的火山引擎语音识别服务access_token
# 资源ID:小时版 volc.seedasr.sauc.duration,并发版 volc.seedasr.sauc.concurrent
resource_id: volc.seedasr.sauc.duration
# 热词、替换词使用流程:https://www.volcengine.com/docs/6561/155738
@@ -482,9 +452,9 @@ ASR:
# token申请地址:https://console.cloud.tencent.com/cam/capi
# 免费领取资源:https://console.cloud.tencent.com/asr/resourcebundle
type: tencent
appid: 你的腾讯语音合成服务appid
secret_id: 你的腾讯语音合成服务secret_id
secret_key: 你的腾讯语音合成服务secret_key
appid: 你的腾讯语音识别服务appid
secret_id: 你的腾讯语音识别服务secret_id
secret_key: 你的腾讯语音识别服务secret_key
output_dir: tmp/
AliyunASR:
# 阿里云智能语音交互服务,需要先在阿里云平台开通服务,然后获取验证信息
@@ -650,7 +620,7 @@ LLM:
# 可在这里找到你的 api_key https://bailian.console.aliyun.com/?apiKey=1#/api-key
base_url: https://dashscope.aliyuncs.com/compatible-mode/v1
model_name: qwen-flash
api_key: 你的deepseek web key
api_key: 你的阿里云百炼API密钥
temperature: 0.7 # 温度值
max_tokens: 500 # 最大生成token数
top_p: 1
@@ -794,10 +764,10 @@ VLLM:
# 先新建应用,在下面的地址
# 开通应用地址:https://console.xfyun.cn/app/myapp
# 有免费额度,但也要开通服务,才能获取api_key
# 每一个模型都需要单独开通,每一个模型的api_password都不同,例如Lite模型在https://console.xfyun.cn/services/cbm 开通
# 每一个模型都需要单独开通,每一个模型的 api_key 都不同,例如 Lite 模型在 https://console.xfyun.cn/services/cbm 开通
base_url: https://ark.cn-beijing.volces.com/api/v3
model_name: lite
api_key: 你的api_password
api_key: 你的api_key
TTS:
# 当前支持的type为edge、doubao,可自行适配
EdgeTTS:
@@ -1148,8 +1118,8 @@ TTS:
#SpeechServerDemo https://github.com/PaddlePaddle/PaddleSpeech/tree/develop/demos/speech_server
#流式传输请参考 https://github.com/PaddlePaddle/PaddleSpeech/wiki/PaddleSpeech-Server-WebSocket-API
type: paddle_speech
protocol: websocket # protocol choices = ['websocket', 'http']
url: ws://127.0.0.1:8092/paddlespeech/tts/streaming # TTS 服务的 URL 地址,指向本地服务器 [websocket默认ws://127.0.0.1:8092/paddlespeech/tts/streaming,http默认http://127.0.0.1:8090/paddlespeech/tts]
protocol: websocket # 当前仅实现 websocket;填其他值会在启动时报错
url: ws://127.0.0.1:8092/paddlespeech/tts/streaming # TTS 服务地址(未配置时代码兜底同此本机地址)
spk_id: 0 # 发音人 ID,0 通常表示默认的发音人
speed: 1.0 # 语速,1.0 表示正常语速,>1 表示加快,<1 表示减慢
volume: 1.0 # 音量,1.0 表示正常音量,>1 表示增大,<1 表示减小
+18 -37
View File
@@ -40,7 +40,7 @@ async def load_config():
custom_config = read_config(custom_config_path)
if custom_config.get("manager-api", {}).get("url"):
config = await get_config_from_api_async()
config = await get_config_from_api_async(custom_config)
else:
# 合并配置
config = merge_configs(default_config, custom_config)
@@ -52,16 +52,10 @@ async def load_config():
return config
async def get_config_from_api_async():
"""从Java API获取配置(异步版本),本地配置逐键覆盖上去
本地配置在这里自己读,不接受调用方传进来:调用方手里只有上一次合并
完的结果,拿它当"本地"会把旧值盖回这次新拉到的响应,改配置不生效。
"""
local_config = read_config(get_project_dir() + "data/.config.yaml") or {}
async def get_config_from_api_async(config):
"""从Java API获取配置(异步版本)"""
# 初始化API客户端
init_service(local_config)
init_service(config)
# 获取服务器配置
config_data = await get_server_config()
@@ -70,36 +64,23 @@ async def get_config_from_api_async():
config_data["read_config_from_api"] = True
config_data["manager-api"] = {
"url": local_config.get("manager-api", {}).get("url", ""),
"secret": local_config.get("manager-api", {}).get("secret", ""),
"url": config["manager-api"].get("url", ""),
"secret": config["manager-api"].get("secret", ""),
}
server_config = config_data.setdefault("server", {})
# 认证开关以智控台的参数管理为准,这里只补默认值。
# 不能重建整个 auth,否则接口返回的 allowed_devices、activation_codes
# 这些字段会被抹掉,智控台部署下等于配置写了不生效
server_auth = server_config.setdefault("auth", {})
server_auth.setdefault("enabled", False)
# 智控台只下发 enabled 这一个键,本地配的白名单、激活码、吊销线、有效期
# 得逐键并进去。整段跳过的话它们在官方全模块部署里一个都用不了,
# 而本地这边没有任何提示,配了等于没配。
# 智控台哪天开始下发这些键,以智控台为准
for key, value in ((local_config.get("server") or {}).get("auth") or {}).items():
if key != "enabled":
server_auth.setdefault(key, value)
# server 的其余配置以本地为准,但只覆盖本地显式写过的字段。
# 整块替换会把接口返回的 cors、websocket 等一并吞掉。
# auth 例外,它归智控台管,在上面单独合并
# 注意 auth_key 在 server 下、不在 server.auth 里,它归这一段管;
# 而 config.yaml 里根本没有 auth_key 这一行,默认部署下压根不参与合并,
# 靠 app.py 回落到 manager-api.secret。要改它得显式写进 data/.config.yaml
for key, value in (local_config.get("server") or {}).items():
if key != "auth":
server_config[key] = value
auth_enabled = config_data.get("server", {}).get("auth", {}).get("enabled", False)
# server的配置以本地为准
if config.get("server"):
config_data["server"] = {
"ip": config["server"].get("ip", ""),
"port": config["server"].get("port", ""),
"http_port": config["server"].get("http_port", ""),
"vision_explain": config["server"].get("vision_explain", ""),
"auth_key": config["server"].get("auth_key", ""),
}
config_data["server"]["auth"] = {"enabled": auth_enabled}
# 如果服务器没有prompt_template,则从本地配置读取
if not config_data.get("prompt_template"):
config_data["prompt_template"] = local_config.get("prompt_template")
config_data["prompt_template"] = config.get("prompt_template")
return config_data
+1 -1
View File
@@ -6,7 +6,7 @@ from config.config_loader import load_config
from config.settings import check_config_file
from core.utils.cache.manager import cache_manager, CacheType
SERVER_VERSION = "0.9.7"
SERVER_VERSION = "0.9.8"
_logger_initialized = False
+4 -48
View File
@@ -2,67 +2,23 @@ from aiohttp import web
from config.logger import setup_logging
def allowed_origins(config: dict) -> list:
cors = config.get("server", {}).get("cors", {}) or {}
return cors.get("allowed_origins", []) or []
class BaseHandler:
def __init__(self, config: dict):
self.config = config
self.logger = setup_logging()
self._cors_wildcard_warned = False
self._cors_empty_warned = False
def _resolve_cors_origin(self, origin: str | None) -> str | None:
"""回显名单内的 Origin;其余情况不下发 CORS 头,由浏览器拦截
这里刻意不支持通配:回显任意来源并带上 Allow-Credentials 会让任意网站
都能带凭证读响应,而固定写 "*" 配 Allow-Credentials 属于浏览器明确
拒绝的组合,反而挡得更严。宁可什么都不放,也不放宽。
"""
if not origin:
return None
cors = self.config.get("server", {}).get("cors", {}) or {}
allowed = cors.get("allowed_origins", []) or []
if "*" in allowed and not self._cors_wildcard_warned:
self._cors_wildcard_warned = True
self.logger.bind(tag="BaseHandler").warning(
"cors.allowed_origins 里的 * 不会放行任意来源,请填写具体来源"
)
return origin if origin in allowed else None
def _add_cors_headers(self, response, request=None):
def _add_cors_headers(self, response):
"""添加CORS头信息"""
response.headers["Access-Control-Allow-Headers"] = (
"client-id, content-type, device-id, authorization"
)
requested_origin = (
request.headers.get("Origin") if request is not None else None
)
origin = self._resolve_cors_origin(requested_origin)
if requested_origin:
# 只要请求带了 Origin 就标 Vary,哪怕这次没放行,
# 否则共享缓存会把带放行头的响应喂给别的来源
response.headers["Vary"] = "Origin"
if origin:
response.headers["Access-Control-Allow-Origin"] = origin
response.headers["Access-Control-Allow-Credentials"] = "true"
elif requested_origin and not allowed_origins(self.config):
# 名单空着的时候浏览器只会报一个不透明的跨域错误,
# 服务端这边一点信号都没有,读日志的人会一头雾水
if not self._cors_empty_warned:
self._cors_empty_warned = True
self.logger.bind(tag="BaseHandler").warning(
f"cors.allowed_origins 是空的,本次来自 {requested_origin} 的"
"跨域请求不会放行。浏览器侧只会报读不到状态码的错,"
"排查这类问题时先把这个来源加进名单"
)
response.headers["Access-Control-Allow-Credentials"] = "true"
response.headers["Access-Control-Allow-Origin"] = "*"
async def handle_options(self, request):
"""处理OPTIONS请求,添加CORS头信息"""
response = web.Response(body=b"", content_type="text/plain")
self._add_cors_headers(response, request)
self._add_cors_headers(response)
# 添加允许的方法
response.headers["Access-Control-Allow-Methods"] = "GET, POST, OPTIONS"
return response
+26 -224
View File
@@ -9,47 +9,12 @@ import glob
from typing import Dict, List, Tuple
from aiohttp import web
from core.auth import AuthManager, auth_config_list, normalize_device_id
from core.auth import AuthManager
from core.utils.util import get_local_ip, get_vision_url
from core.api.base_handler import BaseHandler
TAG = __name__
# Device-Id 校验规则,与智控台 OTAController 使用同一套:
# 冒号 / 连字符 / 下划线分隔的 6 段 hex,或无分隔符的 12 位 hex
# 用 fullmatch 而不是自带锚点的正则:$ 会放过结尾的换行,两端接受范围就不一致了
DEVICE_ID_PATTERN = re.compile(
r"([0-9A-Fa-f]{2}[:\-_]){5}[0-9A-Fa-f]{2}|[0-9A-Fa-f]{12}"
)
# Device-Model 会拼进 mqtt_client_id 进而参与密码签名,限制成不含
# 分隔符与通配符的字符,避免拼出畸形凭证
DEVICE_MODEL_PATTERN = re.compile(r"[A-Za-z0-9_\-]+")
def _error_response(status: int, message: str) -> web.Response:
"""OTA 拒绝响应,状态码与智控台 /ota/ 保持一致,固件按状态码判断才不会走偏"""
return web.Response(
text=json.dumps({"success": False, "message": message}, separators=(",", ":")),
content_type="application/json",
status=status,
)
def _parse_activation_codes(raw_codes) -> Dict[str, str | None]:
"""解析 activation_codes 成 码 -> 绑定的 Device-Id,不绑定时值为 None
支持两种写法:裸字符串表示不限设备,"码: Device-Id" 表示只能用这台设备换。
码本身不能带冒号,冒号只作分隔用。
"""
codes: Dict[str, str | None] = {}
for raw in raw_codes:
code, _, bound_device = str(raw).partition(":")
code = code.strip()
if code:
codes[code] = bound_device.strip() or None
return codes
def _safe_basename(filename: str) -> str:
# Prevent directory traversal
@@ -81,7 +46,13 @@ def _is_higher_version(a: str, b: str) -> bool:
class OTAHandler(BaseHandler):
def __init__(self, config: dict):
super().__init__(config)
self._load_auth_config(config)
auth_config = config["server"].get("auth", {})
self.auth_enable = auth_config.get("enabled", False)
# 设备白名单
self.allowed_devices = set(auth_config.get("allowed_devices", []))
secret_key = config["server"]["auth_key"]
expire_seconds = auth_config.get("expire_seconds")
self.auth = AuthManager(secret_key=secret_key, expire_seconds=expire_seconds)
# firmware storage
self.bin_dir = os.path.join(os.getcwd(), "data", "bin")
@@ -92,121 +63,6 @@ class OTAHandler(BaseHandler):
"files_by_model": {},
}
def _load_auth_config(self, config: dict):
"""从配置加载认证相关属性,改配置后可重复调用"""
auth_config = config["server"].get("auth", {})
self.auth_enable = auth_config.get("enabled", False)
# 设备白名单:命中白名单只影响 token 有效期,不再跳过 token 校验
# 加载时就归一成去掉分隔符的小写,和激活码绑定、OTA 请求头用同一套写法,
# 否则同一台设备换个分隔符或大小写就既不认白名单也不认激活码,直接锁死
self.allowed_devices = {
normalize_device_id(item)
for item in auth_config_list(auth_config, "allowed_devices")
}
# 白名单设备的 token 有效期,默认与普通设备一致
self.allowed_expire_seconds = auth_config.get(
"allowed_devices_expire_seconds", None
)
# 白名单设备首次接入免凭证自举一次,否则升级后没有任何换凭证的入口
self.allowed_bootstrap = auth_config.get("allowed_devices_bootstrap", True)
self.allowed_bootstrapped = set()
# 一次性激活码,供无 token 的设备首次绑定;写 "码: Device-Id" 可绑到具体设备
self.activation_codes = _parse_activation_codes(
auth_config_list(auth_config, "activation_codes")
)
secret_key = config["server"]["auth_key"]
expire_seconds = auth_config.get("expire_seconds")
issued_after = auth_config.get("issued_after", 0)
self.auth = AuthManager(
secret_key=secret_key,
expire_seconds=expire_seconds,
issued_after=issued_after,
)
self._warn_if_locked_out()
def _warn_if_locked_out(self):
"""存量白名单设备升级后若没有换凭证的入口会直接掉线,启动时说清楚"""
if not self.auth_enable:
return
if not self.allowed_devices:
return
if self.allowed_bootstrap or self.activation_codes:
return
self.logger.bind(tag=TAG).warning(
"配置了 allowed_devices 但既没开 allowed_devices_bootstrap 也没有 "
"activation_codes:这些白名单设备升级后没有凭证来源,会连不上。"
"请在 config.yaml 的 server.auth 里加一个 activation_codes,"
"或把 allowed_devices_bootstrap 设回 true"
)
def _expire_for(self, device_id: str):
"""白名单设备的 token 有效期,其余设备沿用默认"""
if self.allowed_devices and normalize_device_id(device_id) in self.allowed_devices:
return self.allowed_expire_seconds
return None
def _issue_ws_token(self, request, client_id: str, device_id: str) -> Tuple:
"""校验 OTA 凭证并签发 websocket token。
返回 (token, 待作废的激活码),无有效凭证时返回 ("", "")。
激活码不在这里作废,交给调用方在响应组装成功后回收,中途失败不烧凭证。
"""
auth_header = request.headers.get("Authorization", "")
presented = auth_header[7:] if auth_header.startswith("Bearer ") else ""
if not presented:
return self._bootstrap_whitelisted(device_id, client_id), ""
if self.auth.verify_token(
presented,
client_id=client_id,
username=device_id,
expire_seconds=self._expire_for(device_id),
):
return self.auth.generate_token(client_id, device_id), ""
if self._matches_activation_code(presented, device_id):
return self.auth.generate_token(client_id, device_id), presented
return "", ""
def _bootstrap_whitelisted(self, device_id: str, client_id: str) -> str:
"""白名单设备首次接入免凭证换 token,每台设备只放行一次
白名单是运维在配置里显式列出的可信 MAC,不接受任何上报值,
也不新增信任来源;这里只是让升级前免 token 的存量设备换到凭证,
否则白名单去掉旁路后它们升级即掉线。之后必须带 token。
"""
if not (
self.allowed_bootstrap
and self.allowed_devices
and normalize_device_id(device_id) in self.allowed_devices
and normalize_device_id(device_id) not in self.allowed_bootstrapped
):
return ""
self.allowed_bootstrapped.add(normalize_device_id(device_id))
self.logger.bind(tag=TAG).warning(
f"白名单设备免凭证自举签发 token: {device_id},此后该设备必须携带凭证"
)
return self.auth.generate_token(client_id, device_id)
def _matches_activation_code(self, presented: str, device_id: str) -> bool:
"""激活码若绑定了设备,必须与上报的 Device-Id 一致"""
if presented not in self.activation_codes:
return False
bound_device = self.activation_codes[presented]
if not bound_device:
return True
return normalize_device_id(bound_device) == normalize_device_id(device_id)
def _consume_activation_code(self, presented: str) -> None:
"""作废一次性激活码,只在响应组装成功后调用"""
self.activation_codes.pop(presented, None)
self.logger.bind(tag=TAG).info("激活码已使用并作废,设备完成首次绑定")
def _is_valid_device_id(self, device_id: str) -> bool:
"""Device-Id 会被拼进 MQTT topic,格式校验与智控台用同一套规则"""
return bool(DEVICE_ID_PATTERN.fullmatch(device_id))
def _refresh_bin_cache_if_needed(self):
now = int(time.time())
ttl = int(self._bin_cache.get("ttl", 30))
@@ -296,12 +152,7 @@ class OTAHandler(BaseHandler):
try:
data = await request.text()
self.logger.bind(tag=TAG).debug(f"OTA请求方法: {request.method}")
# OTA 请求开始携带 token 和一次性激活码,凭证不进日志
safe_headers = {
k: ("***" if k.lower() == "authorization" else v)
for k, v in request.headers.items()
}
self.logger.bind(tag=TAG).debug(f"OTA请求头: {safe_headers}")
self.logger.bind(tag=TAG).debug(f"OTA请求头: {request.headers}")
self.logger.bind(tag=TAG).debug(f"OTA请求数据: {data}")
device_id = request.headers.get("device-id", "")
@@ -316,20 +167,6 @@ class OTAHandler(BaseHandler):
else:
raise Exception("OTA请求ClientID为空")
# 开启认证时,MQTT 与 WebSocket 两条下发路径都必须先通过凭证校验
ws_token = ""
activation_code = ""
if self.auth_enable:
ws_token, activation_code = self._issue_ws_token(
request, client_id, device_id
)
if not ws_token:
self.logger.bind(tag=TAG).warning(
f"OTA请求凭证无效: device_id={device_id}"
)
response = _error_response(401, "invalid credential")
return response
data_json = {}
try:
data_json = json.loads(data) if data else {}
@@ -398,22 +235,12 @@ class OTAHandler(BaseHandler):
mqtt_gateway_endpoint = server_config.get("mqtt_gateway")
if mqtt_gateway_endpoint: # 如果配置了非空字符串
# 下面两个值都会拼进 MQTT 凭证,只在真下发 MQTT 时才校验,
# 免得不开 MQTT 的部署被无端拦下
if not self._is_valid_device_id(device_id):
self.logger.bind(tag=TAG).warning(
f"OTA请求设备ID格式非法: {device_id}"
)
response = _error_response(400, "invalid device-id")
return response
if not DEVICE_MODEL_PATTERN.fullmatch(device_model):
self.logger.bind(tag=TAG).warning(
f"OTA请求设备型号格式非法: {device_model}"
)
response = _error_response(400, "invalid device-model")
return response
group_id = f"GID_{device_model}"
# 尝试从请求数据中获取设备型号(已解析 above)
try:
group_id = f"GID_{device_model}".replace(":", "_").replace(" ", "_")
except Exception as e:
self.logger.bind(tag=TAG).error(f"获取设备型号失败: {e}")
group_id = "GID_default"
mac_address_safe = device_id.replace(":", "_")
mqtt_client_id = f"{group_id}@@@{mac_address_safe}@@@{mac_address_safe}"
@@ -442,8 +269,6 @@ class OTAHandler(BaseHandler):
self.logger.bind(tag=TAG).warning("缺少MQTT签名密钥,密码留空")
# 构建MQTT配置(直接使用 mqtt_gateway 字符串)
# token 一并下发:开认证时固件下载要带凭证,MQTT 设备自己不会再
# 走一遍 OTA,不在这里给就永远换不到凭证
return_json["mqtt"] = {
"endpoint": mqtt_gateway_endpoint,
"client_id": mqtt_client_id,
@@ -451,13 +276,18 @@ class OTAHandler(BaseHandler):
"password": password,
"publish_topic": "device-server",
"subscribe_topic": f"devices/p2p/{mac_address_safe}",
"token": ws_token,
}
self.logger.bind(tag=TAG).info(f"为设备 {device_id} 下发MQTT网关配置")
else: # 未配置 mqtt_gateway,下发 WebSocket
# token 已在入口处完成凭证校验,认证关闭时保持不下发
token = ws_token
# 如果开启了认证,则进行认证校验
token = ""
if self.auth_enable:
if self.allowed_devices:
if device_id not in self.allowed_devices:
token = self.auth.generate_token(client_id, device_id)
else:
token = self.auth.generate_token(client_id, device_id)
# NOTE: use websocket_port here
return_json["websocket"] = {
"url": self._get_websocket_url(local_ip, websocket_port),
@@ -511,9 +341,6 @@ class OTAHandler(BaseHandler):
text=json.dumps(return_json, separators=(",", ":")),
content_type="application/json",
)
# 响应已经组装成功,这时才作废激活码,中途出错不烧凭证
if activation_code:
self._consume_activation_code(activation_code)
except Exception as e:
self.logger.bind(tag=TAG).error(f"OTA POST处理异常: {e}")
return_json = {"success": False, "message": "request error."}
@@ -522,7 +349,7 @@ class OTAHandler(BaseHandler):
content_type="application/json",
)
finally:
self._add_cors_headers(response, request)
self._add_cors_headers(response)
return response
async def handle_get(self, request):
@@ -539,7 +366,7 @@ class OTAHandler(BaseHandler):
self.logger.bind(tag=TAG).error(f"OTA GET请求异常: {e}")
response = web.Response(text="OTA接口异常", content_type="text/plain")
finally:
self._add_cors_headers(response, request)
self._add_cors_headers(response)
return response
async def handle_download(self, request):
@@ -548,33 +375,8 @@ class OTAHandler(BaseHandler):
URL: /xiaozhi/ota/download/{filename}
- 只允许下载 data/bin 目录下的 .bin 文件
- filename 必须是 basename 且匹配安全的模式
- 开启认证时需携带与 OTA 请求一致的凭证
"""
try:
if self.auth_enable:
device_id = request.headers.get("Device-Id", "")
client_id = request.headers.get("Client-Id", "")
auth_header = request.headers.get("Authorization", "")
token = (
auth_header[7:] if auth_header.startswith("Bearer ") else ""
)
authorized = (
device_id
and client_id
and token
and self.auth.verify_token(
token,
client_id=client_id,
username=device_id,
expire_seconds=self._expire_for(device_id),
)
)
if not authorized:
self.logger.bind(tag=TAG).warning(
f"固件下载凭证无效: device_id={device_id}"
)
raise web.HTTPUnauthorized(text="unauthorized")
fname = request.match_info.get("filename", "")
if not fname:
raise web.HTTPBadRequest(text="filename required")
@@ -607,7 +409,7 @@ class OTAHandler(BaseHandler):
resp = web.Response(text="download error", status=500)
finally:
try:
self._add_cors_headers(resp, request)
self._add_cors_headers(resp)
except Exception:
pass
return resp
@@ -29,7 +29,14 @@ class VisionHandler(BaseHandler):
def _verify_auth_token(self, request) -> Tuple[bool, Optional[str]]:
"""验证认证token"""
# 测试模式:允许特定测试令牌或跳过验证
auth_header = request.headers.get("Authorization", "")
client_id = request.headers.get("Client-Id", "")
# 允许测试客户端跳过认证
if client_id == "web_test_client":
device_id = request.headers.get("Device-Id", "test_device")
return True, device_id
if not auth_header.startswith("Bearer "):
return False, None
@@ -148,7 +155,7 @@ class VisionHandler(BaseHandler):
)
finally:
if response:
self._add_cors_headers(response, request)
self._add_cors_headers(response)
return response
async def handle_get(self, request):
@@ -171,5 +178,5 @@ class VisionHandler(BaseHandler):
content_type="application/json",
)
finally:
self._add_cors_headers(response, request)
self._add_cors_headers(response)
return response
+7 -92
View File
@@ -1,53 +1,8 @@
import hmac
import base64
import hashlib
import re
import time
from loguru import logger
DEFAULT_EXPIRE_SECONDS = 60 * 60 * 24 * 30
# 有效期上限,防止把过期时间配成准永久凭证
MAX_EXPIRE_SECONDS = 60 * 60 * 24 * 90
def normalize_expire_seconds(expire_seconds) -> int:
"""有效期归一化:非整数或非正数回落到默认值,超出上限则截断
白名单设备的有效期是单独传的,走这里和构造时保持同一套规则,
否则配成 0 会让 token 一秒就失效,配成字符串则会被当成校验失败
"""
try:
value = int(expire_seconds)
except (TypeError, ValueError):
return DEFAULT_EXPIRE_SECONDS
if value <= 0:
return DEFAULT_EXPIRE_SECONDS
return min(value, MAX_EXPIRE_SECONDS)
def normalize_issued_after(issued_after) -> int:
"""吊销时刻归一化
非整数和负数直接报错——这两类是配错了,不会有人这么填。
超前的时间戳只降级成警告并截到当前时间:这一格最容易踩的是跨主机时钟偏差,
而吊销线超前本来就等于没设,不值得为它把整个进程拖到起不来。
"""
try:
value = int(issued_after or 0)
except (TypeError, ValueError):
raise ValueError(f"server.auth.issued_after 必须是整数,当前值:{issued_after!r}")
if value < 0:
raise ValueError(f"server.auth.issued_after 不能为负数,当前值:{value}")
now = int(time.time())
if value > now:
logger.warning(
f"server.auth.issued_after 填的是未来的时间戳(当前值 {value},本机 {now}),"
"已截到当前时间。吊销线超前等于没设,请检查两台机器的时钟是否对得上"
)
return now
return value
class AuthenticationError(Exception):
"""认证异常"""
@@ -55,29 +10,6 @@ class AuthenticationError(Exception):
pass
def normalize_device_id(device_id: str) -> str:
"""把 Device-Id 归一成去掉分隔符的小写
白名单、激活码绑定、OTA 请求头三处都要用同一个值比较,
同一台设备换个分隔符或大小写就会被一边认一边不认。
"""
return re.sub(r"[:\-_]", "", device_id).lower()
def auth_config_list(auth_config: dict, key: str) -> list:
"""取 auth 段里的列表型配置,配成标量字符串时直接报错
不校验的话 set("aa:bb:cc:dd:ee:ff") 会逐字符拆成 7 个元素,
激活码字符串会被拆成十个单字符码,配了等于没配还看不出问题。
"""
value = auth_config.get(key)
if value is None:
return []
if not isinstance(value, list):
raise ValueError(f"server.auth.{key} 必须是列表,当前值:{value!r}")
return value
class AuthManager:
"""
统一授权认证管理器
@@ -87,15 +19,12 @@ class AuthManager:
在 Websocket 中,header:{Device-ID: device_id, Client-ID: client_id, Authorization: Bearer token, ......}
"""
def __init__(
self,
secret_key: str,
expire_seconds: int = DEFAULT_EXPIRE_SECONDS,
issued_after: int = 0,
):
self.expire_seconds = normalize_expire_seconds(expire_seconds)
def __init__(self, secret_key: str, expire_seconds: int = 60 * 60 * 24 * 30):
if not expire_seconds or expire_seconds < 0:
self.expire_seconds = 60 * 60 * 24 * 30
else:
self.expire_seconds = expire_seconds
self.secret_key = secret_key
self.issued_after = normalize_issued_after(issued_after)
def _sign(self, content: str) -> str:
"""HMAC-SHA256签名并Base64编码"""
@@ -120,32 +49,18 @@ class AuthManager:
token = f"{signature}.{ts}"
return token
def verify_token(
self,
token: str,
client_id: str,
username: str,
expire_seconds: int | None = None,
) -> bool:
def verify_token(self, token: str, client_id: str, username: str) -> bool:
"""
验证token有效性
Args:
token: 客户端传入的token
client_id: 连接使用的client_id
username: 连接使用的username
expire_seconds: 本次校验采用的过期时长,不传时用实例默认值
"""
expire = (
self.expire_seconds
if expire_seconds is None
else normalize_expire_seconds(expire_seconds)
)
try:
sig_part, ts_str = token.split(".")
ts = int(ts_str)
if ts < self.issued_after:
return False # 签发时刻早于吊销线
if int(time.time()) - ts > expire:
if int(time.time()) - ts > self.expire_seconds:
return False # 过期
expected_sig = self._sign(f"{client_id}|{username}|{ts}")
+2 -6
View File
@@ -214,12 +214,8 @@ class ConnectionHandler:
self.client_ip = real_ip.split(",")[0].strip()
else:
self.client_ip = ws.remote_address[0]
safe_headers = {
k: ("***" if k.lower() == "authorization" else v)
for k, v in self.headers.items()
}
self.logger.bind(tag=TAG).info(
f"{self.client_ip} conn - Headers: {safe_headers}"
f"{self.client_ip} conn - Headers: {self.headers}"
)
self.device_id = self.headers.get("device-id", None)
@@ -785,7 +781,7 @@ class ConnectionHandler:
self.voiceprint_provider = voiceprint_provider
self.logger.bind(tag=TAG).info("声纹识别功能已在连接时动态启用")
else:
self.logger.bind(tag=TAG).warning("声纹识别功能启用但配置不完整")
self.logger.bind(tag=TAG).debug("声纹识别功能启用但配置不完整")
else:
self.logger.bind(tag=TAG).info("声纹识别功能未启用")
except Exception as e:
+57 -9
View File
@@ -4,18 +4,24 @@ import uuid
import random
import asyncio
from typing import TYPE_CHECKING
from config.logger import setup_logging
if TYPE_CHECKING:
from core.connection import ConnectionHandler
from core.utils.dialogue import Message
from core.utils.util import audio_to_data
from core.utils.util import audio_to_data, audio_file_to_wav_bytes
from core.providers.tts.dto.dto import SentenceType
from core.utils.wakeup_word import WakeupWordsConfig
from core.handle.sendAudioHandle import sendAudioMessage, send_tts_message
from core.utils.util import remove_punctuation_and_length, opus_datas_to_wav_bytes
from core.providers.tools.device_mcp import MCPClient, send_mcp_initialize_message
from core.providers.tools.device_mcp import (
MCPClient,
send_mcp_initialize_message,
send_mcp_tools_list_request,
)
TAG = __name__
_wakeup_logger = setup_logging()
WAKEUP_CONFIG = {
"refresh_time": 10,
@@ -63,6 +69,8 @@ async def handleHelloMessage(conn: "ConnectionHandler", msg_json):
# The device waits for the server hello before processing MCP messages.
if features and features.get("mcp"):
asyncio.create_task(send_mcp_initialize_message(conn))
# 并发预发 tools/list,让帧排在 audio 流之前到达 client
asyncio.create_task(send_mcp_tools_list_request(conn))
async def checkWakeupWords(conn: "ConnectionHandler", text):
@@ -105,19 +113,19 @@ async def checkWakeupWords(conn: "ConnectionHandler", text):
}
# 获取音频数据
opus_packets = await audio_to_data(response.get("file_path"), use_cache=False)
opus_packets = await audio_to_data(response.get("file_path"), use_cache=True)
# 播放唤醒词回复
conn.client_abort = False
# 将唤醒词回复视为新会话,生成新的 sentence_id,确保流控器重置
conn.sentence_id = str(uuid.uuid4().hex)
conn.logger.bind(tag=TAG).info(f"播放唤醒词回复: {response.get('text')}")
await sendAudioMessage(conn, SentenceType.FIRST, opus_packets, response.get("text"))
await sendAudioMessage(conn, SentenceType.LAST, [], None)
# 补充对话
conn.dialogue.put(Message(role="assistant", content=response.get("text")))
conn.logger.bind(tag=TAG).info(f"播放唤醒词回复: {response.get('text')}")
await sendAudioMessage(conn, SentenceType.FIRST, opus_packets, response.get("text"))
asyncio.create_task(_finish_wakeup_reply(conn))
# 检查是否需要更新唤醒词回复
if time.time() - response.get("time", 0) > WAKEUP_CONFIG["refresh_time"]:
@@ -126,6 +134,16 @@ async def checkWakeupWords(conn: "ConnectionHandler", text):
return True
async def _finish_wakeup_reply(conn: "ConnectionHandler"):
"""等音频队列清空后再发 stop(时序:全部音频帧→stop),期间不阻塞接收循环"""
try:
await sendAudioMessage(conn, SentenceType.LAST, [], None)
except asyncio.CancelledError:
raise
except Exception as e:
conn.logger.bind(tag=TAG).error(f"唤醒词回复收尾失败: {e}")
async def wakeupWordsResponse(conn: "ConnectionHandler"):
if not conn.tts:
return
@@ -148,14 +166,44 @@ async def wakeupWordsResponse(conn: "ConnectionHandler"):
# 获取当前音色
voice = getattr(conn.tts, "voice", "default")
# 使用链接的sample_rate
wav_bytes = opus_datas_to_wav_bytes(tts_result, sample_rate=conn.sample_rate)
# 保留的 TTS 音频是文件路径而非 Opus 帧:直接转目标采样率 wav,跳过有损 Opus 往返
if isinstance(tts_result, str):
wav_bytes = await asyncio.to_thread(
audio_file_to_wav_bytes, tts_result, conn.sample_rate
)
else:
wav_bytes = opus_datas_to_wav_bytes(tts_result, sample_rate=conn.sample_rate)
file_path = wakeup_words_config.generate_file_path(voice)
with open(file_path, "wb") as f:
f.write(wav_bytes)
# 更新配置
wakeup_words_config.update_wakeup_response(voice, file_path, result)
# 覆写文件后 mtime 变化使旧缓存失效,此处立刻预编码新文件,
# 下次唤醒直接命中缓存,编码成本不占用唤醒响应路径
await audio_to_data(file_path, use_cache=True)
finally:
# 确保在任何情况下都释放锁
if _wakeup_response_lock.locked():
_wakeup_response_lock.release()
async def preheat_wakeup_audio_cache():
"""进程启动时预热唤醒词音频编码缓存
把 wakeup_words/{voice}.wav 在 Opus 编码结果一次性算好塞进全局 cache,
后续每条连接上来就能直接命中,省掉 wav→PCM→Opus 的数百毫秒开销。
"""
# 进程启动时 tts 还未初始化,无法拿到 conn.tts.voice,按常见 fallback 列表预热
preheat_voices = ["default", "female-shaonv", "male-qn-jingying"]
fallback_path = "config/assets/wakeup_words_short.wav"
for voice in preheat_voices:
file_path = fallback_path
try:
response = wakeup_words_config.get_wakeup_response(voice)
if response and response.get("file_path"):
file_path = response["file_path"]
await audio_to_data(file_path, use_cache=True)
except Exception as e:
_wakeup_logger.bind(tag=TAG).warning(
f"预热 voice={voice} 失败: {e}"
)
@@ -12,10 +12,8 @@ from core.providers.tts.dto.dto import SentenceType
from core.utils.audioRateController import AudioRateController
TAG = __name__
# 音频帧时长(毫秒)
AUDIO_FRAME_DURATION = 60
# 预缓冲包数量,直接发送以减少延迟
PRE_BUFFER_COUNT = 5
AUDIO_FRAME_DURATION = 60 # 音频帧时长(毫秒),与 Opus 编码 60ms/帧对齐
PRE_BUFFER_COUNT = 2 # 首段发送时不入队的帧数,避免流控延迟拉高首帧到达时间
async def sendAudioMessage(conn: "ConnectionHandler", sentenceType, audios, text, sentence_id=None):
@@ -5,6 +5,7 @@ if TYPE_CHECKING:
from core.connection import ConnectionHandler
from ..base import IntentProviderBase
from plugins_func.functions.play_music import initialize_music_handler
from plugins_func.functions.hass_init import _devices_text, _hass_plugin_config
from config.logger import setup_logging
from core.utils.util import get_system_error_response
import re
@@ -181,16 +182,11 @@ class IntentProvider(IntentProviderBase):
music_file_names = music_config["music_file_names"]
prompt_music = f"{self.promot}\n<musicNames>{music_file_names}\n</musicNames>"
home_assistant_cfg = conn.config["plugins"].get("home_assistant")
if home_assistant_cfg:
devices = home_assistant_cfg.get("devices", [])
else:
devices = []
if len(devices) > 0:
plugin_config = _hass_plugin_config(conn.config.get("plugins", {}))
devices_text = _devices_text(plugin_config.get("devices", ""))
if devices_text:
hass_prompt = "\n下面是我家智能设备列表(位置,设备名,entity_id),可以通过homeassistant控制\n"
for device in devices:
hass_prompt += device + "\n"
prompt_music += hass_prompt
prompt_music += hass_prompt + devices_text + "\n"
logger.bind(tag=TAG).debug(f"User prompt: {prompt_music}")
@@ -16,12 +16,20 @@ THINKING_DISABLED_DOMAINS = {
"bigmodel.cn": {"thinking": {"type": "disabled"}},
"moonshot.cn": {"thinking": {"type": "disabled"}},
"volces.com": {"thinking": {"type": "disabled"}},
"modelarts-maas.com": {"thinking": {"type": "disabled"}},
}
# 域名规则的模型名豁免表: 强制思考模型不接受该参数注入(如 MaaS glm-5.3 传 thinking 必 400)
THINKING_DISABLED_MODEL_EXCLUSIONS = {
"modelarts-maas.com": ("glm-5.3",),
}
class LLMProvider(LLMProviderBase):
def __init__(self, config):
self.model_name = config.get("model_name")
# 可选实例级思考强度: off/disabled/none/disable | low/medium/high | 空/其他=按域名默认
self.thinking_mode = str(config.get("thinking") or "").strip().lower()
self.api_key = config.get("api_key")
if "base_url" in config:
self.base_url = config.get("base_url")
@@ -80,14 +88,34 @@ class LLMProvider(LLMProviderBase):
return dialogue
def _apply_thinking_disabled(self, request_params: dict):
"""根据域名自动禁用思考模式"""
"""思考模式控制: 实例配置(thinking)优先, 否则按域名默认表"""
mode = getattr(self, "thinking_mode", "")
if mode in ("keep", "passthrough", "default"):
# 不干预: 既不发 thinking 参数也不查域名表(适用于强制思考/拒绝该参数的模型, 如 MaaS glm-5.3)
return
if mode in ("off", "disabled", "none", "disable"):
request_params.setdefault("extra_body", {}).update({"thinking": {"type": "disabled"}})
logger.bind(tag=TAG).info("思考强度=off: 已禁用思考模式(实例配置)")
return
if mode in ("low", "medium", "high"):
request_params.setdefault("extra_body", {}).update(
{"reasoning_effort": mode, "thinking": {"type": "enabled"}})
logger.bind(tag=TAG).info(f"思考强度={mode}: reasoning_effort 已设置(实例配置)")
return
parsed_url = urlparse(self.base_url)
domain = parsed_url.netloc
model_name = (self.model_name or "").lower()
for disabled_domain, params in THINKING_DISABLED_DOMAINS.items():
if disabled_domain in domain:
request_params.setdefault("extra_body", {}).update(params)
logger.bind(tag=TAG).info(f"为域名 {domain} 禁用思考模式,参数: {params}")
if disabled_domain not in domain:
continue
exclusions = THINKING_DISABLED_MODEL_EXCLUSIONS.get(disabled_domain, ())
if any(ex in model_name for ex in exclusions):
logger.bind(tag=TAG).info(
f"域名 {domain} 命中禁思考规则, 但模型 {self.model_name} 强制思考且拒绝该参数, 跳过注入")
break
request_params.setdefault("extra_body", {}).update(params)
logger.bind(tag=TAG).info(f"为域名 {domain} 禁用思考模式,参数: {params}")
break
def response(self, session_id, dialogue, **kwargs):
dialogue = self.normalize_dialogue(dialogue)
@@ -17,7 +17,11 @@ class MCPClient:
self.name_mapping = {}
self.ready = False
self.call_results = {} # To store Futures for tool call responses
self.next_id = 1
# Handshake hardcodes id=1 (initialize) and id=2 (tools/list) in mcp_handler.
# JSON-RPC 2.0 forbids reusing request ids on the same connection, and
# compliant device MCP servers reject tools/call with id 1/2 (see #3461).
# Official protocol sequence expects the first tools/call to use id=3.
self.next_id = 3
self.lock = asyncio.Lock()
self._cached_available_tools = None # Cache for get_available_tools
@@ -3,10 +3,10 @@
import json
import asyncio
import re
from concurrent.futures import Future
from core.utils.util import get_vision_url, sanitize_tool_name
from core.utils.util import get_vision_url
from core.utils.auth import AuthToken
from config.logger import setup_logging
from .mcp_client import MCPClient
from typing import TYPE_CHECKING
if TYPE_CHECKING:
@@ -15,91 +15,6 @@ if TYPE_CHECKING:
TAG = __name__
logger = setup_logging()
class MCPClient:
"""设备端MCP客户端,用于管理MCP状态和工具"""
def __init__(self):
self.tools = {} # sanitized_name -> tool_data
self.name_mapping = {}
self.ready = False
self.call_results = {} # To store Futures for tool call responses
self.next_id = 1
self.lock = asyncio.Lock()
self._cached_available_tools = None # Cache for get_available_tools
def has_tool(self, name: str) -> bool:
return name in self.tools
def get_available_tools(self) -> list:
# Check if the cache is valid
if self._cached_available_tools is not None:
return self._cached_available_tools
# If cache is not valid, regenerate the list
result = []
for tool_name, tool_data in self.tools.items():
function_def = {
"name": tool_name,
"description": tool_data["description"],
"parameters": {
"type": tool_data["inputSchema"].get("type", "object"),
"properties": tool_data["inputSchema"].get("properties", {}),
"required": tool_data["inputSchema"].get("required", []),
},
}
result.append({"type": "function", "function": function_def})
self._cached_available_tools = result # Store the generated list in cache
return result
async def is_ready(self) -> bool:
async with self.lock:
return self.ready
async def set_ready(self, status: bool):
async with self.lock:
self.ready = status
async def add_tool(self, tool_data: dict):
async with self.lock:
sanitized_name = sanitize_tool_name(tool_data["name"])
self.tools[sanitized_name] = tool_data
self.name_mapping[sanitized_name] = tool_data["name"]
self._cached_available_tools = (
None # Invalidate the cache when a tool is added
)
async def get_next_id(self) -> int:
async with self.lock:
current_id = self.next_id
self.next_id += 1
return current_id
async def register_call_result_future(self, id: int, future: Future):
async with self.lock:
self.call_results[id] = future
async def resolve_call_result(self, id: int, result: any):
async with self.lock:
if id in self.call_results:
future = self.call_results.pop(id)
if not future.done():
future.set_result(result)
async def reject_call_result(self, id: int, exception: Exception):
async with self.lock:
if id in self.call_results:
future = self.call_results.pop(id)
if not future.done():
future.set_exception(exception)
async def cleanup_call_result(self, id: int):
async with self.lock:
if id in self.call_results:
self.call_results.pop(id)
async def send_mcp_message(conn: "ConnectionHandler", payload: dict):
"""Helper to send MCP messages, encapsulating common logic."""
if not conn.features.get("mcp"):
@@ -147,11 +62,6 @@ async def handle_mcp_message(
logger.bind(tag=TAG).debug(
f"客户端MCP服务器信息: name={name}, version={version}"
)
await asyncio.sleep(1)
logger.bind(tag=TAG).debug("初始化完成,开始请求MCP工具列表")
await send_mcp_tools_list_request(conn)
return
elif msg_id == 2: # mcpToolsListID
@@ -18,7 +18,9 @@ class MCPEndpointClient:
self.name_mapping = {}
self.ready = False
self.call_results = {} # To store Futures for tool call responses
self.next_id = 1
# Handshake hardcodes id=1 (initialize) and id=2 (tools/list) in
# mcp_endpoint_handler; tools/call must start at 3 to avoid id reuse.
self.next_id = 3
self.lock = asyncio.Lock()
self._cached_available_tools = None # Cache for get_available_tools
self.websocket = None # WebSocket连接
@@ -10,12 +10,13 @@ import concurrent.futures
from contextlib import AsyncExitStack
from typing import Optional, List, Dict, Any
import httpx2
from mcp import ClientSession, StdioServerParameters, Implementation
from mcp.client.session import SamplingFnT, ElicitationFnT, ListRootsFnT, LoggingFnT, MessageHandlerFnT
from mcp.client.stdio import stdio_client
from mcp.client.sse import sse_client
from mcp.client.streamable_http import streamablehttp_client
from mcp.shared.session import ProgressFnT
from mcp.client.streamable_http import streamable_http_client
from mcp.shared.dispatcher import ProgressFnT # mcp 2.x:从 mcp.shared.session 移来
from config.logger import setup_logging
from core.utils.util import sanitize_tool_name
@@ -106,7 +107,7 @@ class ServerMCPClient:
"function": {
"name": name,
"description": tool.description,
"parameters": tool.inputSchema,
"parameters": tool.input_schema,
},
}
for name, tool in self.tools_dict.items()
@@ -203,13 +204,21 @@ class ServerMCPClient:
if transport_type == "streamable-http" or transport_type == "http":
# 使用 Streamable HTTP 传输
http_r, http_w, get_session_id = await stack.enter_async_context(
streamablehttp_client(
# mcp 2.x:headers/timeout 改由 http_client 承载,产出从 3 元组变为 2 元组
http_client = httpx2.AsyncClient(
headers=headers,
timeout=httpx2.Timeout(
self.config.get("timeout", 30),
read=self.config.get("sse_read_timeout", 60 * 5),
),
)
stack.push_async_callback(http_client.aclose)
http_r, http_w = await stack.enter_async_context(
streamable_http_client(
url=self.config["url"],
headers=headers,
timeout=self.config.get("timeout", 30),
sse_read_timeout=self.config.get("sse_read_timeout", 60 * 5),
terminate_on_close=self.config.get("terminate_on_close", True)
http_client=http_client,
terminate_on_close=self.config.get("terminate_on_close", True),
)
)
read_stream, write_stream = http_r, http_w
@@ -24,7 +24,7 @@ class ServerMCPManager:
self.config_path = get_project_dir() + "data/.mcp_server_settings.json"
if not os.path.exists(self.config_path):
self.config_path = ""
logger.bind(tag=TAG).warning(
logger.bind(tag=TAG).debug(
f"请检查mcp服务配置文件:data/.mcp_server_settings.json"
)
self.clients: Dict[str, ServerMCPClient] = {}
@@ -1,6 +1,8 @@
"""服务端插件工具执行器"""
import asyncio
import functools
from concurrent.futures import ThreadPoolExecutor
from typing import Dict, Any, TYPE_CHECKING
if TYPE_CHECKING:
@@ -8,6 +10,10 @@ if TYPE_CHECKING:
from ..base import ToolType, ToolDefinition, ToolExecutor
from plugins_func.register import all_function_registry, module_func_map, Action, ActionResponse
# 插件专用线程池:不用 asyncio 默认池(DNS 解析、ASR、initialize_modules 等都在用),
# 避免慢插件打满默认池后拖垮所有协程的 DNS 解析和连接建立。
_PLUGIN_POOL = ThreadPoolExecutor(max_workers=8, thread_name_prefix="plugin")
class ServerPluginExecutor(ToolExecutor):
"""服务端插件工具执行器"""
@@ -28,23 +34,21 @@ class ServerPluginExecutor(ToolExecutor):
try:
# 根据工具类型决定如何调用
if hasattr(func_item, "type"):
func_type = func_item.type
if func_type.code in [4, 5]: # SYSTEM_CTL, IOT_CTL (需要conn参数)
result = func_item.func(conn, **arguments)
elif func_type.code == 2: # WAIT
result = func_item.func(**arguments)
elif func_type.code == 3: # CHANGE_SYS_PROMPT
result = func_item.func(conn, **arguments)
else:
result = func_item.func(**arguments)
else:
# 默认不传conn参数
result = func_item.func(**arguments)
func_type = getattr(func_item, "type", None)
if func_type is not None and func_type.code in [3, 4, 5]:
# CHANGE_SYS_PROMPT, SYSTEM_CTL, IOT_CTL (需要conn参数)
call = functools.partial(func_item.func, conn, **arguments)
else: # WAIT 等不传conn(含 type 未指定的插件)
call = functools.partial(func_item.func, **arguments)
# 兼容 async def 工具函数
if asyncio.iscoroutine(result):
result = await result
if asyncio.iscoroutinefunction(func_item.func):
result = await call()
else:
# 同步函数在 loop 线程上直接执行会卡住整个事件循环,放专用线程池。
result = await asyncio.get_running_loop().run_in_executor(_PLUGIN_POOL, call)
# 兼容同步函数返回协程的写法
if asyncio.iscoroutine(result):
result = await result
return result
@@ -47,9 +47,12 @@ class ToolManager:
return all_tools
def get_function_descriptions(self) -> List[Dict[str, Any]]:
"""获取所有工具的函数描述(OpenAI格式)"""
"""获取所有工具的函数描述(OpenAI格式)
返回缓存的浅拷贝,防止调用方就地修改污染缓存。
"""
if self._cached_function_descriptions is not None:
return self._cached_function_descriptions
return list(self._cached_function_descriptions)
descriptions = []
tools = self.get_all_tools()
@@ -57,7 +60,7 @@ class ToolManager:
descriptions.append(tool_definition.description)
self._cached_function_descriptions = descriptions
return descriptions
return list(descriptions)
def has_tool(self, tool_name: str) -> bool:
"""检查是否存在指定工具"""
@@ -144,6 +144,7 @@ class TTSProvider(TTSProviderBase):
self.ws = None
self.interface_type = InterfaceType.DUAL_STREAM
self._monitor_task = None # 监听任务引用
self._pending_prefix = ""
self.appId = config.get("appid")
self.access_token = config.get("access_token")
self.cluster = config.get("cluster")
@@ -306,6 +307,9 @@ class TTSProvider(TTSProviderBase):
if message.sentence_type == SentenceType.FIRST:
# 重置流式处理状态
self.reset_stream_state()
# 重置 opus 流式编码器跨轮残留
if hasattr(self, "opus_encoder") and self.opus_encoder:
self.opus_encoder.reset_state()
# 初始化参数
try:
if not getattr(self.conn, "sentence_id", None):
@@ -433,6 +437,9 @@ class TTSProvider(TTSProviderBase):
logger.bind(tag=TAG).debug(f"关闭会话~~{session_id}")
try:
if self.ws:
# 句尾替换词滑动窗口残留先发出去
if self._pending_prefix:
await self.text_to_speak(self._pending_prefix, None)
header = Header(
message_type=FULL_CLIENT_REQUEST,
message_type_specific_flags=MsgTypeFlagWithEvent,
@@ -490,7 +497,6 @@ class TTSProvider(TTSProviderBase):
try:
while not self.conn.stop_event.is_set():
try:
# 确保 `recv()` 运行在同一个 event loop
msg = await self.ws.recv()
res = self.parser_response(msg)
self.print_response(res, "send_text res:")
@@ -500,17 +506,18 @@ class TTSProvider(TTSProviderBase):
logger.bind(tag=TAG).debug(f"链接关闭成功~~")
break
# 只处理当前活跃会话的响应
# 只处理当前活跃会话的响应;旧会话事件不影响当前 activate_session
if res.optional.sessionId and self.conn.sentence_id != res.optional.sessionId:
# 如果是会话结束相关事件,即使会话ID不匹配也要重置状态
if res.optional.event in [EVENT_SessionCanceled, EVENT_SessionFailed, EVENT_SessionFinished]:
logger.bind(tag=TAG).debug(f"收到残余下行结束响应重置会话状态~~")
self.activate_session = False
continue
if res.optional.event == EVENT_SessionCanceled:
logger.bind(tag=TAG).debug(f"释放服务端资源成功~~")
self.activate_session = False
elif res.optional.event == EVENT_SessionFailed:
logger.bind(tag=TAG).error(
f"TTS 会话被服务端拒绝: {self.conn.sentence_id}"
)
self.activate_session = False
elif not self.resource_type and res.optional.event == EVENT_TTSSentenceStart:
json_data = json.loads(res.payload.decode("utf-8"))
self.tts_text = json_data.get("text", "")
@@ -538,6 +545,11 @@ class TTSProvider(TTSProviderBase):
logger.bind(tag=TAG).info(f"句子语音生成成功:{self.tts_text}")
elif res.optional.event == EVENT_SessionFinished:
logger.bind(tag=TAG).debug(f"会话结束~~")
# 句尾冲刷流式 opus 编码器不足一帧的 PCM 残量
if hasattr(self, "opus_encoder") and self.opus_encoder:
self.wav_to_opus_data_audio_raw_stream(
b"", is_end=True, callback=self.handle_opus
)
self.activate_session = False
self._process_before_stop_play_files()
# 非复用模式下,会话结束后发送 FinishConnection
@@ -552,16 +564,14 @@ class TTSProvider(TTSProviderBase):
)
traceback.print_exc()
break
# 连接异常时关闭WebSocket
finally:
# 退出时关闭连接并清理引用
if self.ws:
try:
await self.ws.close()
except:
pass
self.ws = None
# 监听任务退出时清理引用
finally:
self.activate_session = False
self._monitor_task = None
async def send_event(
@@ -619,21 +629,23 @@ class TTSProvider(TTSProviderBase):
# 解析结果
# header
header = response.header
num = 0b00001111
header.protocol_version = res[0] >> 4 & num
mask = 0b00001111
header.protocol_version = res[0] >> 4 & mask
header.header_size = res[0] & 0x0F
header.message_type = (res[1] >> 4) & num
header.message_type = (res[1] >> 4) & mask
header.message_type_specific_flags = res[1] & 0x0F
header.serialization_method = res[2] >> num
header.serialization_method = res[2] >> 4
header.message_compression = res[2] & 0x0F
header.reserved = res[3]
#
offset = 4
offset = header.header_size * 4
optional = response.optional
if header.message_type == FULL_SERVER_RESPONSE or AUDIO_ONLY_RESPONSE:
if header.message_type in (FULL_SERVER_RESPONSE, AUDIO_ONLY_RESPONSE):
# read event
if header.message_type_specific_flags == MsgTypeFlagWithEvent:
optional.event = int.from_bytes(res[offset:8], "big", signed=True)
optional.event = int.from_bytes(
res[offset : offset + 4], "big", signed=True
)
offset += 4
if optional.event == EVENT_NONE:
return response
@@ -644,10 +656,11 @@ class TTSProvider(TTSProviderBase):
optional.response_meta_json, offset = self.read_res_content(
res, offset
)
elif (
optional.event == EVENT_SessionStarted
or optional.event == EVENT_SessionFailed
or optional.event == EVENT_SessionFinished
elif optional.event in (
EVENT_SessionStarted,
EVENT_SessionFailed,
EVENT_SessionFinished,
EVENT_SessionCanceled,
):
optional.sessionId, offset = self.read_res_content(res, offset)
optional.response_meta_json, offset = self.read_res_content(
@@ -820,6 +833,13 @@ class TTSProvider(TTSProviderBase):
):
self.wav_to_opus_data_audio_raw_stream(res.payload, callback=lambda opus_frame: audio_data.append(opus_frame))
elif res.optional.event == EVENT_SessionFinished:
# 句尾冲刷流式 opus 编码器不足一帧的 PCM 残量
if hasattr(self, "opus_encoder") and self.opus_encoder:
self.wav_to_opus_data_audio_raw_stream(
b"",
is_end=True,
callback=lambda opus_frame: audio_data.append(opus_frame),
)
break
finally:
@@ -23,9 +23,11 @@ class TTSProvider(TTSProviderBase):
def __init__(self, config, delete_audio_file):
super().__init__(config, delete_audio_file)
self.url = config.get("url", "ws://192.168.1.10:8092/paddlespeech/tts/streaming")
self.url = config.get("url", "ws://127.0.0.1:8092/paddlespeech/tts/streaming")
self.protocol = config.get("protocol", "websocket")
if self.protocol != "websocket":
raise ValueError("Unsupported protocol. Please use 'websocket'.")
if config.get("private_voice"):
self.spk_id = int(config.get("private_voice"))
else:
@@ -82,7 +84,7 @@ class TTSProvider(TTSProviderBase):
if self.protocol == "websocket":
return await self.text_streaming(text, output_file)
else:
raise ValueError("Unsupported protocol. Please use 'websocket' or 'http'.")
raise ValueError("Unsupported protocol. Please use 'websocket'.")
async def text_streaming(self, text, output_file):
try:
@@ -5,6 +5,7 @@
import os
import asyncio
import hashlib
import threading
from typing import Dict, Any, TYPE_CHECKING
@@ -120,8 +121,9 @@ class PromptManager:
# 使用传入的提示词并缓存(如果有设备ID)
if device_id:
device_cache_key = f"device_prompt:{device_id}"
self.cache_manager.set(self.CacheType.DEVICE_PROMPT, device_cache_key, user_prompt)
self.cache_manager.set(
self.CacheType.DEVICE_PROMPT, device_cache_key, user_prompt
)
self.logger.bind(tag=TAG).debug(f"设备 {device_id} 的提示词已缓存")
self.logger.bind(tag=TAG).info(f"使用快速提示词: {user_prompt[:50]}...")
@@ -253,6 +255,18 @@ class PromptManager:
if not self.base_prompt_template:
return user_prompt
# 缓存键须覆盖所有渲染输入:emoji 开关、设备 prompt 变更否则会命中过期提示词
emoji_enabled = kwargs.get("emoji_enabled", True)
prompt_hash = hashlib.md5(user_prompt.encode("utf-8")).hexdigest()[:8]
enhanced_cache_key = (
f"enhanced_prompt:{device_id}:{emoji_enabled}:{prompt_hash}"
)
cached_prompt = self.cache_manager.get(
self.CacheType.DEVICE_PROMPT, enhanced_cache_key
)
if cached_prompt is not None:
return cached_prompt
try:
# 获取最新的时间信息(不缓存)
today_date, today_weekday, lunar_date = self._get_current_time_info()
@@ -301,11 +315,13 @@ class PromptManager:
*args,
**kwargs,
)
device_cache_key = f"device_prompt:{device_id}"
self.cache_manager.set(
self.CacheType.DEVICE_PROMPT, device_cache_key, enhanced_prompt
self.CacheType.DEVICE_PROMPT,
enhanced_cache_key,
enhanced_prompt,
ttl=300,
)
self.logger.bind(tag=TAG).info(
self.logger.bind(tag=TAG).debug(
f"构建增强提示词成功,长度: {len(enhanced_prompt)}"
)
return enhanced_prompt
+22 -3
View File
@@ -135,7 +135,7 @@ def get_ip_info(ip_addr, logger):
if is_private_ip(ip_addr):
ip_addr = ""
url = f"https://whois.pconline.com.cn/ipJson.jsp?json=true&ip={ip_addr}"
resp = requests.get(url).json()
resp = requests.get(url, timeout=5).json()
ip_info = {"city": resp.get("city")}
# 存入缓存
@@ -310,8 +310,12 @@ async def audio_to_data(
from core.utils.cache.manager import cache_manager
from core.utils.cache.config import CacheType
# 生成缓存键,包含文件路径和编码类型
cache_key = f"{audio_file_path}:{is_opus}"
# 缓存键含 mtime:文件被原地覆写时旧缓存自动失效(如唤醒词回复重新 TTS),避免播到过期音频
try:
mtime = os.path.getmtime(audio_file_path)
cache_key = f"{audio_file_path}:{mtime}:{is_opus}"
except OSError:
cache_key = f"{audio_file_path}:{is_opus}"
# 尝试从缓存获取结果
if use_cache:
@@ -475,6 +479,21 @@ def opus_datas_to_wav_bytes(opus_datas, sample_rate=16000, channels=1):
pass
def audio_file_to_wav_bytes(audio_file_path, sample_rate=16000):
"""音频文件直接转目标采样率的 wav 字节流,不经 Opus 编解码,避免有损"""
file_type = os.path.splitext(audio_file_path)[1]
if file_type:
file_type = file_type.lstrip(".")
with _ffmpeg_path_override():
audio = AudioSegment.from_file(
audio_file_path, format=file_type, parameters=["-nostdin"]
)
audio = audio.set_channels(1).set_frame_rate(sample_rate).set_sample_width(2)
buffer = BytesIO()
audio.export(buffer, format="wav")
return buffer.getvalue()
def check_vad_update(before_config, new_config):
if (
new_config.get("selected_module") is None
+47 -106
View File
@@ -32,40 +32,12 @@ _setup_websockets_logger()
from core.connection import ConnectionHandler
from config.config_loader import get_config_from_api_async
from core.auth import (
AuthManager,
AuthenticationError,
auth_config_list,
normalize_device_id,
)
from core.auth import AuthManager, AuthenticationError
from core.utils.modules_initialize import initialize_modules
from core.utils.util import check_vad_update, check_asr_update
TAG = __name__
# 浏览器不能自定义 websocket 请求头,约定把凭证和设备标识放进子协议头:
# Sec-WebSocket-Protocol: xiaozhi, t-<token>, d-<deviceId>, c-<clientId>
# 握手时只回显 xiaozhi,其余值由服务端自己解析。加前缀是为了让每个值
# 都落在合法的子协议 token 字符集内(deviceId 的冒号就不合法)
WS_SUBPROTOCOL = "xiaozhi"
WS_SUBPROTOCOL_FIELDS = (("t-", "authorization"), ("d-", "device-id"), ("c-", "client-id"))
def _parse_subprotocol_headers(headers: dict) -> dict:
"""从子协议头里取出 token、device-id、client-id
取出来的值与请求头形态保持一致(authorization 带 Bearer 前缀),
这样下游不用关心凭证到底是从哪来的。
"""
values = {}
for item in headers.get("sec-websocket-protocol", "").split(","):
item = item.strip()
for prefix, key in WS_SUBPROTOCOL_FIELDS:
if item.startswith(prefix) and len(item) > len(prefix):
value = item[len(prefix) :]
values[key] = f"Bearer {value}" if key == "authorization" else value
return values
class WebSocketServer:
def __init__(self, config: dict):
@@ -88,31 +60,13 @@ class WebSocketServer:
self._intent = modules["intent"] if "intent" in modules else None
self._memory = modules["memory"] if "memory" in modules else None
self._load_auth_config(self.config)
def _load_auth_config(self, config: dict):
"""从配置加载认证相关属性,改配置后可重复调用"""
auth_config = config["server"].get("auth", {})
auth_config = self.config["server"].get("auth", {})
self.auth_enable = auth_config.get("enabled", False)
# 设备白名单:命中白名单只影响 token 的校验时长,不再跳过校验本身
# 和 OTA 侧同一套写法:加载时归一成去分隔符的小写,
# 否则设备换个分隔符或大小写上报,同一台设备一边认白名单一边不认
self.allowed_devices = {
normalize_device_id(device_id)
for device_id in auth_config_list(auth_config, "allowed_devices")
}
# 白名单设备的 token 有效期,默认与普通设备一致,不因白名单放宽
self.allowed_expire_seconds = auth_config.get(
"allowed_devices_expire_seconds", None
)
secret_key = config["server"]["auth_key"]
# 设备白名单
self.allowed_devices = set(auth_config.get("allowed_devices", []))
secret_key = self.config["server"]["auth_key"]
expire_seconds = auth_config.get("expire_seconds", None)
issued_after = auth_config.get("issued_after", 0)
self.auth = AuthManager(
secret_key=secret_key,
expire_seconds=expire_seconds,
issued_after=issued_after,
)
self.auth = AuthManager(secret_key=secret_key, expire_seconds=expire_seconds)
async def start(self):
server_config = self.config["server"]
@@ -124,36 +78,36 @@ class WebSocketServer:
host,
port,
process_request=self._http_response,
select_subprotocol=self._select_subprotocol,
ping_interval=None,
):
await asyncio.Future()
def _select_subprotocol(self, connection, subprotocols):
"""客户端按约定发了子协议就回显 xiaozhi,否则不选,回显会被浏览器拒掉"""
return WS_SUBPROTOCOL if WS_SUBPROTOCOL in subprotocols else None
def _connection_headers(self, websocket: websockets.ServerConnection) -> dict:
"""合并请求头与子协议头
请求头优先,两者都不从 URL 查询串取:查询串会原样落进反代访问日志,
设备标识和 token 都不该出现在那里。
合并结果写回 request.headers:ConnectionHandler 会再读一次那里,
不写回去浏览器连接就是认证过了但会话里没有 device-id。
"""
headers = dict(websocket.request.headers)
for key, value in _parse_subprotocol_headers(headers).items():
if not headers.get(key):
headers[key] = value
websocket.request.headers[key] = value
return headers
async def _handle_connection(self, websocket: websockets.ServerConnection):
headers = self._connection_headers(websocket)
headers = dict(websocket.request.headers)
if headers.get("device-id", None) is None:
await websocket.send("端口正常,如需测试连接,请启动digital-human测试")
await websocket.close()
return
# 尝试从 URL 的查询参数中获取 device-id
from urllib.parse import parse_qs, urlparse
# 从 WebSocket 请求中获取路径
request_path = websocket.request.path
if not request_path:
self.logger.bind(tag=TAG).error("无法获取请求路径")
await websocket.close()
return
parsed_url = urlparse(request_path)
query_params = parse_qs(parsed_url.query)
if "device-id" not in query_params:
await websocket.send("端口正常,如需测试连接,请启动digital-human测试")
await websocket.close()
return
else:
websocket.request.headers["device-id"] = query_params["device-id"][0]
if "client-id" in query_params:
websocket.request.headers["client-id"] = query_params["client-id"][0]
if "authorization" in query_params:
websocket.request.headers["authorization"] = query_params[
"authorization"
][0]
"""处理新连接,每次创建独立的ConnectionHandler"""
# 先认证,后建立连接
@@ -211,7 +165,7 @@ class WebSocketServer:
try:
async with self.config_lock:
# 重新获取配置(使用异步版本)
new_config = await get_config_from_api_async()
new_config = await get_config_from_api_async(self.config)
if new_config is None:
self.logger.bind(tag=TAG).error("获取新配置失败")
return False
@@ -222,15 +176,6 @@ class WebSocketServer:
self.logger.bind(tag=TAG).info(
f"检查VAD和ASR类型是否需要更新: {update_vad} {update_asr}"
)
# auth_key 是 app.py 启动时算出来写进 config 的,
# config.yaml 里没有这一行,重新拉一次接口就没了。
# 换掉了等于换了一把签名密钥,刷新前签发的 token 全部作废
new_config["server"].setdefault(
"auth_key", self.config["server"]["auth_key"]
)
# 认证属性先跟着配置刷新:_load_auth_config 配错会抛,
# 抛了却已经把 self.config 换掉的话,模块不重建、配置还换了一半
self._load_auth_config(new_config)
# 更新配置
self.config = new_config
# 重新初始化组件
@@ -265,26 +210,22 @@ class WebSocketServer:
async def _handle_auth(self, websocket: websockets.ServerConnection):
# 先认证,后建立连接
if self.auth_enable:
headers = self._connection_headers(websocket)
headers = dict(websocket.request.headers)
device_id = headers.get("device-id", None)
client_id = headers.get("client-id", None)
token = headers.get("authorization", "")
if token.startswith("Bearer "):
token = token[7:] # 移除'Bearer '前缀
if self.allowed_devices and device_id in self.allowed_devices:
# 如果属于白名单内的设备,不校验token,直接放行
return
else:
raise AuthenticationError("Missing or invalid Authorization header")
# 白名单设备只延长 token 有效期,仍然校验 token
expire_seconds = None
if self.allowed_devices and normalize_device_id(
device_id or ""
) in self.allowed_devices:
expire_seconds = self.allowed_expire_seconds
# 进行认证
auth_success = self.auth.verify_token(
token,
client_id=client_id,
username=device_id,
expire_seconds=expire_seconds,
)
if not auth_success:
raise AuthenticationError("Invalid token")
# 否则校验token
token = headers.get("authorization", "")
if token.startswith("Bearer "):
token = token[7:] # 移除'Bearer '前缀
else:
raise AuthenticationError("Missing or invalid Authorization header")
# 进行认证
auth_success = self.auth.verify_token(
token, client_id=client_id, username=device_id
)
if not auth_success:
raise AuthenticationError("Invalid token")
+1 -1
View File
@@ -2,7 +2,7 @@ import os
import importlib.util
import asyncio
print("使用前请根据doc/performance_testerer.md的说明准备配置。")
print("使用前请根据仓库根目录下 docs/performance_tester.md 的说明准备配置。")
def list_performance_tester_modules():
@@ -647,8 +647,9 @@ class StreamTTSPerformanceTester:
self.results.append(result)
# 测试Linkerai TTS
result = await self.test_linkerai_tts(test_text, test_count)
self.results.append(result)
if self.config.get("TTS", {}).get("LinkeraiTTS"):
result = await self.test_linkerai_tts(test_text, test_count)
self.results.append(result)
# 测试IndexStreamTTS
result = await self.test_indexstream_tts(test_text, test_count)
@@ -1,6 +1,7 @@
import random
import httpx
from io import BytesIO
from urllib.parse import quote, urlsplit, urlunsplit, parse_qsl
from markitdown import MarkItDown, StreamInfo
from config.logger import setup_logging
from plugins_func.register import register_function, ToolType, ActionResponse, Action
@@ -54,6 +55,27 @@ CHANNEL_MAP = {
# 默认新闻来源字典,当配置中没有指定时使用
DEFAULT_NEWS_SOURCES = "澎湃新闻;百度热搜;财联社"
DEFAULT_NEWS_API_URL = "https://newsnow.busiyi.world/api/s?id="
def _build_news_api_url(source, configured_url=None):
"""拼出新闻接口的请求地址和查询参数。
配置里带 {source} 占位符就替换占位符;否则把 id 设成新闻源 id,
这样配置里已经写了 id 时也不会拼出脏串。
"""
api_url = str(configured_url).strip() if configured_url else DEFAULT_NEWS_API_URL
if "{source}" in api_url:
return api_url.replace("{source}", quote(source, safe="")), None
parts = urlsplit(api_url)
query = dict(parse_qsl(parts.query, keep_blank_values=True))
query["id"] = source
return (
urlunsplit((parts.scheme, parts.netloc, parts.path, "", parts.fragment)),
query,
)
def _get_newsnow_config(conn):
# 从连接配置获取
plugins = conn.config.get("plugins", {})
@@ -61,6 +83,8 @@ def _get_newsnow_config(conn):
sources = newsnow.get("news_sources", "")
if isinstance(sources, str) and sources.strip():
return sources
if isinstance(sources, (list, tuple)):
return ";".join(str(item).strip() for item in sources if str(item).strip())
return ""
@@ -113,23 +137,26 @@ GET_NEWS_FROM_NEWSNOW_FUNCTION_DESC = {
async def fetch_news_from_api(conn: "ConnectionHandler", source="thepaper"):
"""从API获取新闻列表"""
try:
api_url = f"https://newsnow.busiyi.world/api/s?id={source}"
news_config = conn.config.get("plugins", {}).get("get_news_from_newsnow", {})
if news_config.get("url"):
api_url = news_config["url"] + source
configured_url = news_config.get("url") if isinstance(news_config, dict) else None
api_url, request_params = _build_news_api_url(source, configured_url)
headers = {"User-Agent": "Mozilla/5.0"}
async with httpx.AsyncClient(timeout=httpx.Timeout(10.0, connect=3.0)) as client:
response = await client.get(api_url, headers=headers)
response = await client.get(api_url, params=request_params, headers=headers)
if response.status_code != 200:
logger.bind(tag=TAG).error(
f"获取新闻API失败:HTTP {response.status_code},source={source}"
)
return []
data = response.json()
if "items" in data:
return data["items"]
else:
items = data.get("items") if isinstance(data, dict) else None
if not isinstance(items, list):
logger.bind(tag=TAG).error(f"获取新闻API响应格式错误: {data}")
return []
return [item for item in items if isinstance(item, dict) and item.get("title")]
except Exception as e:
logger.bind(tag=TAG).error(f"获取新闻API失败: {e}")
@@ -239,18 +266,24 @@ async def get_news_from_newsnow(
english_source_id = None
# 检查输入的中文名称是否在配置的新闻源中
news_sources_list = [
name.strip() for name in news_sources.split(";") if name.strip()
]
news_sources_list = [name.strip() for name in news_sources.split(";") if name.strip()]
if source in news_sources_list:
# 如果输入的中文名称在配置的新闻源中,在 CHANNEL_MAP 中查找对应的英文ID
english_source_id = CHANNEL_MAP.get(source)
elif not source and news_sources_list:
source = news_sources_list[0]
english_source_id = CHANNEL_MAP.get(source)
# 如果找不到对应的英文ID,使用默认源
if not english_source_id:
logger.bind(tag=TAG).warning(f"无效的新闻源: {source},使用默认源澎湃新闻")
english_source_id = "thepaper"
source = "澎湃新闻"
fallback_source = next(
(name for name in news_sources_list if CHANNEL_MAP.get(name)),
"澎湃新闻",
)
logger.bind(tag=TAG).warning(
f"无效的新闻源: {source},使用配置中的默认源{fallback_source}"
)
english_source_id = CHANNEL_MAP[fallback_source]
source = fallback_source
logger.bind(tag=TAG).info(f"获取新闻: 新闻源={source}({english_source_id})")
@@ -266,20 +299,21 @@ async def get_news_from_newsnow(
# 随机选择一条新闻
selected_news = random.choice(news_items)
title = selected_news.get("title", "未知标题")
# 保存当前新闻链接到连接对象,以便后续查询详情
if not hasattr(conn, "last_newsnow_link"):
conn.last_newsnow_link = {}
conn.last_newsnow_link = {
"url": selected_news.get("url", "#"),
"title": selected_news.get("title", "未知标题"),
"title": title,
"source_id": english_source_id,
}
# 构建新闻报告
news_report = (
f"根据下列数据,用{lang}回应用户的新闻查询请求:\n\n"
f"新闻标题: {selected_news['title']}\n"
f"新闻标题: {title}\n"
# f"新闻来源: {source}\n"
f"(请以自然、流畅的方式向用户播报这条新闻标题,"
f"提示用户可以要求获取详细内容,此时会获取新闻的详细内容。)"
@@ -1,3 +1,4 @@
import asyncio
import httpx
from bs4 import BeautifulSoup
from config.logger import setup_logging
@@ -112,20 +113,106 @@ WEATHER_CODE_MAP = {
async def fetch_city_info(location, api_key, api_host):
url = f"https://{api_host}/geo/v2/city/lookup?key={api_key}&location={location}&lang=zh"
url = f"https://{api_host}/geo/v2/city/lookup"
async with httpx.AsyncClient(timeout=httpx.Timeout(5.0, connect=3.0)) as client:
response = await client.get(url, headers=HEADERS)
data = response.json()
if data.get("error") is not None:
response = await client.get(
url,
params={
"key": api_key,
"location": location,
"lang": "zh",
"number": 1,
},
headers=HEADERS,
)
try:
data = response.json()
except ValueError:
logger.bind(tag=TAG).error(
f"获取天气失败,原因:{data.get('error', {}).get('detail')}"
f"获取天气城市失败:HTTP {response.status_code} 返回了非 JSON 内容"
)
return None
if not isinstance(data, dict):
logger.bind(tag=TAG).error("获取天气城市失败:响应不是 JSON 对象")
return None
if response.status_code != 200 or data.get("code") != "200":
logger.bind(tag=TAG).error(
f"获取天气城市失败:HTTP {response.status_code},代码 {data.get('code')}"
)
return None
return data.get("location", [])[0] if data.get("location") else None
async def fetch_weather_data(city_info, api_key, api_host):
location_id = city_info.get("id")
if not location_id:
return None
base_url = f"https://{api_host}/v7/weather"
params = {"key": api_key, "location": location_id, "lang": "zh"}
async def request(client, endpoint):
response = await client.get(f"{base_url}/{endpoint}", params=params, headers=HEADERS)
try:
data = response.json()
except ValueError:
return None
if response.status_code != 200 or data.get("code") != "200":
logger.bind(tag=TAG).error(
f"获取天气数据失败:接口={endpoint} HTTP {response.status_code},代码 {data.get('code')}"
)
return None
return data
try:
async with httpx.AsyncClient(timeout=httpx.Timeout(10.0, connect=3.0)) as client:
now, daily = await asyncio.gather(
request(client, "now"), request(client, "7d")
)
except Exception as e:
logger.bind(tag=TAG).warning(f"获取天气 JSON 接口失败,将尝试网页接口: {e}")
return None
if now is None or daily is None:
return None
return now, daily
def build_weather_report(city_info, now_data, daily_data):
city_name = city_info.get("name") or city_info.get("adm2") or "未知位置"
now = now_data.get("now", {})
weather_report = f"您查询的位置是:{city_name}\n\n"
weather_report += f"当前天气: {now.get('text', '未知')}\n"
current_basic = {
"温度": f"{now['temp']}℃" if now.get("temp") is not None else None,
"体感温度": f"{now['feelsLike']}℃" if now.get("feelsLike") is not None else None,
"湿度": f"{now['humidity']}%" if now.get("humidity") is not None else None,
"风向": now.get("windDir"),
"风力": now.get("windScale"),
"能见度": f"{now['vis']}公里" if now.get("vis") is not None else None,
"气压": f"{now['pressure']}百帕" if now.get("pressure") is not None else None,
"降水量": f"{now['precip']}毫米" if now.get("precip") is not None else None,
}
current_basic = {key: value for key, value in current_basic.items() if value}
if current_basic:
weather_report += "详细参数:\n"
weather_report += "".join(f" · {key}: {value}\n" for key, value in current_basic.items())
weather_report += "\n未来7天预报:\n"
for item in daily_data.get("daily", [])[:7]:
date = item.get("fxDate", "未知日期")
weather = item.get("textDay") or WEATHER_CODE_MAP.get(item.get("iconDay"), "未知")
high = item.get("tempMax", "未知")
low = item.get("tempMin", "未知")
weather_report += f"{date}: {weather},气温 {low}~{high}℃\n"
weather_report += "\n(如需某一天的具体天气,请告诉我日期)"
return weather_report
async def fetch_weather_page(url):
async with httpx.AsyncClient(timeout=httpx.Timeout(10.0, connect=3.0)) as client:
async with httpx.AsyncClient(
timeout=httpx.Timeout(10.0, connect=3.0), follow_redirects=True
) as client:
response = await client.get(url, headers=HEADERS)
return BeautifulSoup(response.text, "html.parser") if response.status_code == 200 else None
@@ -181,7 +268,8 @@ async def get_weather(conn: "ConnectionHandler", location: str = None, lang: str
location = cached_ip_info.get("city")
else:
# 缓存未命中,调用API获取
ip_info = get_ip_info(client_ip, logger)
# get_ip_info 是同步 requests 调用,放线程池,避免卡住主事件循环
ip_info = await asyncio.to_thread(get_ip_info, client_ip, logger)
if ip_info:
cache_manager.set(CacheType.IP_INFO, client_ip, ip_info)
location = ip_info.get("city")
@@ -201,29 +289,31 @@ async def get_weather(conn: "ConnectionHandler", location: str = None, lang: str
city_info = await fetch_city_info(location, api_key, api_host)
if not city_info:
return ActionResponse(
Action.REQLLM, f"未找到相关的城市: {location},请确认地点是否正确", None
Action.REQLLM,
f"暂时查不到 {location} 的天气(城市未找到或天气服务不可用),请稍后再试",
None,
)
soup = await fetch_weather_page(city_info["fxLink"])
if not soup:
return ActionResponse(Action.REQLLM, None, "请求失败")
city_name, current_abstract, current_basic, temps_list = parse_weather_info(soup)
weather_report = f"您查询的位置是:{city_name}\n\n当前天气: {current_abstract}\n"
# 添加有效的当前天气参数
if current_basic:
weather_report += "详细参数:\n"
for key, value in current_basic.items():
if value != "0": # 过滤无效值
weather_report += f" · {key}: {value}\n"
# 添加7天预报
weather_report += "\n未来7天预报:\n"
for date, weather, high, low in temps_list:
weather_report += f"{date}: {weather},气温 {low}~{high}\n"
# 提示语
weather_report += "\n(如需某一天的具体天气,请告诉我日期)"
weather_data = await fetch_weather_data(city_info, api_key, api_host)
if weather_data:
weather_report = build_weather_report(city_info, *weather_data)
else:
fx_link = city_info.get("fxLink")
soup = await fetch_weather_page(fx_link) if fx_link else None
if not soup:
return ActionResponse(Action.REQLLM, None, "请求天气数据失败")
city_name, current_abstract, current_basic, temps_list = parse_weather_info(soup)
weather_report = f"您查询的位置是:{city_name}\n\n当前天气: {current_abstract}\n"
if current_basic:
weather_report += "详细参数:\n"
weather_report += "".join(
f" · {key}: {value}\n" for key, value in current_basic.items() if value != "0"
)
weather_report += "\n未来7天预报:\n"
weather_report += "".join(
f"{date}: {weather},气温 {low}~{high}\n"
for date, weather, high, low in temps_list
)
weather_report += "\n(如需某一天的具体天气,请告诉我日期)"
# 缓存完整的天气报告
cache_manager.set(CacheType.WEATHER, weather_cache_key, weather_report)
@@ -1,27 +1,53 @@
from config.logger import setup_logging
from core.utils.util import check_model_key
from plugins_func.register import module_func_map
TAG = __name__
logger = setup_logging()
def _expand_module_names(funcs):
"""把 functions 列表里的模块名(如 hass_state)展开成具体函数名,与工具加载口径一致。"""
expanded = []
for name in funcs:
expanded.extend(module_func_map.get(name, [name]))
return expanded
def _devices_text(devices) -> str:
"""把 devices 配置转成拼进提示词的文字。
config.yaml 示例和手机端智控台给的是列表,每项一条"位置,设备名,entity_id",这里拼成每行一条;
网页端智控台和 HA 集成文档是一段字符串,原样用;其它(如 None)当空。
"""
if isinstance(devices, (list, tuple)):
return "\n".join(
str(d).strip() for d in devices if d is not None and str(d).strip()
)
return devices if isinstance(devices, str) else ""
def _hass_plugin_config(plugins_config: dict) -> dict:
"""取 HA 插件配置:优先本地 config.yaml 的 home_assistant,其次智控台下发的 hass_state。"""
for key in ("home_assistant", "hass_state"):
cfg = plugins_config.get(key)
if cfg:
return cfg
return {}
def append_devices_to_prompt(conn):
if conn.intent_type == "function_call":
funcs = conn.config["Intent"][conn.config["selected_module"]["Intent"]].get(
"functions", []
)
# 安全地获取插件配置
plugins_config = conn.config.get("plugins", {})
config_source = (
"home_assistant"
if plugins_config.get("home_assistant")
else "hass_state"
funcs = _expand_module_names(
conn.config["Intent"][conn.config["selected_module"]["Intent"]].get(
"functions", []
)
)
if "hass_get_state" in funcs or "hass_set_state" in funcs:
prompt = "\n下面是我家智能设备列表(位置,设备名,entity_id),可以通过homeassistant控制\n"
deviceStr = plugins_config.get(config_source, {}).get("devices", "")
plugin_config = _hass_plugin_config(conn.config.get("plugins", {}))
deviceStr = _devices_text(plugin_config.get("devices", ""))
conn.prompt += prompt + deviceStr + "\n"
# 更新提示词
conn.dialogue.update_system_message(conn.prompt)
@@ -32,17 +58,10 @@ def initialize_hass_handler(conn):
if not conn.load_function_plugin:
return ha_config
# 安全地获取插件配置
plugins_config = conn.config.get("plugins", {})
# 确定配置来源
config_source = (
"home_assistant" if plugins_config.get("home_assistant") else "hass_state"
)
if not plugins_config.get(config_source):
plugin_config = _hass_plugin_config(conn.config.get("plugins", {}))
if not plugin_config:
return ha_config
# 统一获取配置
plugin_config = plugins_config[config_source]
ha_config["base_url"] = plugin_config.get("base_url")
ha_config["api_key"] = plugin_config.get("api_key")
@@ -149,6 +149,54 @@ async def _search_serply(api_key: str, query: str, max_results: int) -> str:
return "\n".join(lines)
async def _search_outcrawl(api_key: str, query: str, max_results: int) -> str:
"""调用Outcrawl搜索API"""
url = "https://api.outcrawl.ai/v1/search"
headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"User-Agent": "xiaozhi-esp32-server",
}
# Outcrawl单次最多返回20条
payload = {"query": query, "limit": max(1, min(max_results, 20))}
logger.bind(tag=TAG).debug(f"Outcrawl搜索请求 | URL: {url} | payload: {payload}")
async with httpx.AsyncClient(timeout=httpx.Timeout(15.0, connect=3.0)) as client:
response = await client.post(url, json=payload, headers=headers)
data = response.json()
logger.bind(tag=TAG).debug(f"Outcrawl搜索响应 | status: {response.status_code}")
if response.status_code != 200:
# 错误信封为 {"code": "...", "message": "...", "details": {}},
# code 是官方承诺的封闭枚举
code = data.get("code", "") if isinstance(data, dict) else ""
detail = data.get("message", "") if isinstance(data, dict) else ""
logger.bind(tag=TAG).error(
f"Outcrawl搜索失败 | status: {response.status_code} | code: {code} | detail: {detail}"
)
if code == "quota_exceeded":
# 该code覆盖两种情况:每分钟请求数限流,或月度credits用尽
return "联网搜索被限流或额度已用完,请稍后重试或检查Outcrawl账户余额。"
if code == "unauthorized":
return "联网搜索请求失败,请检查API Key是否正确。"
# 其余code(bad_request、no_capacity、search_refused、internal等)
# 多为输入问题或服务端暂时性故障,统一引导重试
return "联网搜索请求失败,请稍后重试。"
hits = data.get("hits", [])
if not hits:
return "未找到相关搜索结果。"
lines = ["【联网搜索结果】"]
for i, item in enumerate(hits[:max_results], 1):
title = item.get("title") or "无标题"
snippet = item.get("snippet", "")
lines.append(f"{i}. 标题:{title}")
if snippet:
lines.append(f" 摘要:{snippet}")
return "\n".join(lines)
@register_function("web_search", WEB_SEARCH_FUNCTION_DESC, ToolType.SYSTEM_CTL)
async def web_search(conn: "ConnectionHandler", query: str = None):
logger.bind(tag=TAG).info(f"web_search 被调用 | query={query}")
@@ -175,6 +223,8 @@ async def web_search(conn: "ConnectionHandler", query: str = None):
result_text = await _search_tavily(api_key, query, max_results)
elif provider == "serply":
result_text = await _search_serply(api_key, query, max_results)
elif provider == "outcrawl":
result_text = await _search_outcrawl(api_key, query, max_results)
else:
return ActionResponse(
Action.REQLLM,
+5 -1
View File
@@ -83,7 +83,11 @@ module_func_map = {}
def register_function(name, desc, type=None):
"""注册函数到函数注册字典的装饰器"""
"""注册函数到函数注册字典的装饰器
同步函数在工作线程中执行,函数体内不可调用 asyncio 事件循环 API;
需要异步(httpx、await)请定义为 async def。
"""
def decorator(func):
all_function_registry[name] = FunctionItem(name, desc, func, type)
+17 -15
View File
@@ -7,36 +7,38 @@ numpy==1.26.4
websockets==14.2
#--------- 以下是可升级的依赖
pyyml==0.0.2
silero_vad==6.2.1
silero_vad==6.2.3
opuslib_next==1.3.1
pydub==0.25.1
funasr==1.2.7
funasr==1.4.16
# transformers>=5 要求 torch>=2.6,与本项目 torch 2.2.2 冲突(会禁用 PyTorch,fun_local 加载不了模型)
transformers<5
openai==2.8.1
google-generativeai==0.8.5
google-generativeai==0.8.6
edge_tts==7.3.1
httpx==0.28.1
aiohttp==3.13.2
aiohttp==3.14.5
aiohttp_cors==0.8.1
ormsgpack==1.12.0
ormsgpack==1.12.2
ruamel.yaml==0.18.16
loguru==0.7.3
requests==2.32.5
requests==2.34.2
cozepy==0.20.0
mem0ai==1.0.0
mem0ai==2.2.1
powermem>=0.3.1
bs4==0.0.2
modelscope==1.39.1
sherpa_onnx==1.12.29
mcp==1.22.0
cnlunar==0.2.4
modelscope==1.40.2
sherpa_onnx==1.13.8
mcp==2.3.0
cnlunar==0.2.5
PySocks==1.7.1
dashscope==1.25.2
dashscope==1.27.7
baidu-aip==4.16.13
aioconsole==0.8.2
markitdown==0.1.3
markitdown==0.1.8
mcp-proxy==0.12.0
PyJWT==2.10.1
psutil==7.1.3
PyJWT==2.15.1
psutil==7.2.2
portalocker==3.2.0 # 暂留 3.x:4.x 与 qdrant-client<1.20 的 portalocker<4.0 约束冲突 (PR #3360)
Jinja2==3.1.6
vosk==0.3.45
+34 -25
View File
@@ -47,10 +47,29 @@ if str(_PROJECT_ROOT) not in sys.path:
# ---------------------------------------------------------------------------
try:
import opuslib_next # noqa: F401
except Exception:
except Exception as _opus_import_error:
# opuslib_next 是 ctypes 绑定,pip 装上的是纯 Python 包,真正的编解码靠系统里的
# libopus.so。runner 上没装 libopus 时 import 会失败,这里换桩顶上。
# 桩只能撑住 import 链路,真要编解码得先把 libopus 装上
# (apt-get install libopus0,正式镜像 Dockerfile-server-base 里也是这么装的)。
print(
f"[conftest] opuslib_next 不可用,改用桩:{_opus_import_error}",
flush=True,
)
def _opus_unavailable(_self, *_args, **_kwargs):
raise RuntimeError(
"libopus 没装,Opus 编解码跑不了。"
"装法:apt-get install libopus0(正式镜像 Dockerfile-server-base 里也是这么装的)"
)
_stub = types.ModuleType("opuslib_next")
_stub.Encoder = type("Encoder", (), {"__init__": lambda *a, **kw: None})
_stub.Decoder = type("Decoder", (), {"__init__": lambda *a, **kw: None})
_stub.Encoder = type(
"Encoder", (), {"__init__": lambda *a, **kw: None, "encode": _opus_unavailable}
)
_stub.Decoder = type(
"Decoder", (), {"__init__": lambda *a, **kw: None, "decode": _opus_unavailable}
)
_stub.APPLICATION_AUDIO = 0
_stub.OpusError = type("OpusError", (Exception,), {})
_stub.constants = types.SimpleNamespace()
@@ -323,35 +342,25 @@ def _install_mock_layer() -> None:
asyncio.sleep = _fast_sleep # type: ignore[assignment]
# 网络层 mock:如果真实 httpx 已加载,把它的 AsyncClient.send 替换成
# 网络层 mock:如果真实 httpx 已加载,把它的 send 替换成
# 立即抛 ConnectError 的桩,触发 ``_should_retry`` 走到 ``raise``,
# 但避免真发请求。Live 模式完全不安装这层。
if not LIVE_API_TESTS:
try:
import httpx as _httpx # noqa: F401
import httpx as _httpx
class _FakeAsyncClient:
def __init__(self, *args, **kwargs):
pass
async def _fake_async_send(self, request, **kwargs):
raise _httpx.ConnectError("MOCK: connect refused")
async def send(self, *args, **kwargs):
raise _httpx.ConnectError("MOCK: connect refused")
def _fake_send(self, request, **kwargs):
raise _httpx.ConnectError("MOCK: connect refused")
async def aclose(self):
return None
class _FakeClient:
def __init__(self, *args, **kwargs):
pass
def send(self, *args, **kwargs):
raise _httpx.ConnectError("MOCK: connect refused")
def close(self):
return None
_httpx.AsyncClient = _FakeAsyncClient # type: ignore[misc]
_httpx.Client = _FakeClient # type: ignore[misc]
# 只换 send 这一个方法,不换整个类。有些库(openai 2.x 的
# SyncHttpxClientWrapper)继承 httpx.Client 再自己拼请求,
# 整个类被换掉的话它们继承到的是假类,拼请求时直接
# AttributeError,报错还完全看不出是 mock 层的问题
_httpx.AsyncClient.send = _fake_async_send # type: ignore[assignment]
_httpx.Client.send = _fake_send # type: ignore[assignment]
except ImportError:
pass
@@ -0,0 +1,136 @@
"""Wakeup caching accepts both return forms of TTSProviderBase.to_tts."""
import asyncio
import math
import struct
import wave
from io import BytesIO
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import Mock
import pytest
@pytest.fixture
def wakeup_runtime(tmp_path, monkeypatch):
# conftest supplies local configuration; no deployment config is required.
from config import settings
monkeypatch.chdir(tmp_path)
monkeypatch.setattr(settings, "config_file_valid", True)
from core.handle import helloHandle
from core.providers.tts.base import TTSProviderBase
from core.utils.wakeup_word import WakeupWordsConfig
class LocalTTS(TTSProviderBase):
"""Replace only remote synthesis; keep the real file and Opus conversion."""
def __init__(self, output_dir, delete_audio_file):
super().__init__({"output_dir": str(output_dir)}, delete_audio_file)
self.voice = "test-voice"
self.filename = output_dir / "retained.wav"
buffer = BytesIO()
with wave.open(buffer, "wb") as audio:
audio.setnchannels(1)
audio.setsampwidth(2)
audio.setframerate(22050)
audio.writeframes(
b"".join(
struct.pack(
"<h", int(8000 * math.sin(2 * math.pi * 440 * i / 22050))
)
# 1.1s 源音频:给重采样留裕量,规避重采样器 off-by-one
for i in range(int(22050 * 1.1))
)
)
self.audio_bytes = buffer.getvalue()
def generate_filename(self):
return str(self.filename)
async def text_to_speak(self, text, output_file):
if output_file is not None:
Path(output_file).write_bytes(self.audio_bytes)
return self.audio_bytes
cache = WakeupWordsConfig()
monkeypatch.setattr(helloHandle, "wakeup_words_config", cache)
monkeypatch.setattr(helloHandle, "_wakeup_response_lock", asyncio.Lock())
return helloHandle, cache, LocalTTS
@pytest.mark.parametrize("sample_rate", [16000, 24000])
@pytest.mark.parametrize("delete_audio_file", [False, True])
def test_wakeup_cache_accepts_retained_files_and_opus_frames(
tmp_path, wakeup_runtime, sample_rate, delete_audio_file
):
helloHandle, wakeup_cache, LocalTTS = wakeup_runtime
tts = LocalTTS(tmp_path, delete_audio_file)
conn = SimpleNamespace(tts=tts, sample_rate=sample_rate)
tts.conn = conn
asyncio.run(helloHandle.wakeupWordsResponse(conn))
response = wakeup_cache.get_wakeup_response(tts.voice)
assert response is not None
assert response["text"] in helloHandle.WAKEUP_CONFIG["responses"]
with wave.open(response["file_path"], "rb") as audio:
assert audio.getnchannels() == 1
assert audio.getsampwidth() == 2
assert audio.getframerate() == sample_rate
assert audio.getnframes() >= sample_rate
assert any(audio.readframes(audio.getnframes()))
if not delete_audio_file:
assert tts.filename.read_bytes() == tts.audio_bytes
assert not helloHandle._wakeup_response_lock.locked()
@pytest.mark.parametrize("tts_result", [None, []])
def test_empty_synthesis_does_not_replace_cached_response(wakeup_runtime, tts_result):
helloHandle, wakeup_cache, _ = wakeup_runtime
wakeup_cache.generate_file_path = Mock()
wakeup_cache.update_wakeup_response = Mock()
conn = SimpleNamespace(tts=SimpleNamespace(to_tts=lambda _: tts_result))
asyncio.run(helloHandle.wakeupWordsResponse(conn))
wakeup_cache.generate_file_path.assert_not_called()
wakeup_cache.update_wakeup_response.assert_not_called()
assert not helloHandle._wakeup_response_lock.locked()
def test_retained_file_skips_opus_roundtrip(wakeup_runtime, tmp_path, monkeypatch):
"""保留文件路径必须直转 wav,不得经过 Opus 编解码。"""
from core.utils import util
helloHandle, wakeup_cache, LocalTTS = wakeup_runtime
def _forbid_opus(*_args, **_kwargs):
raise AssertionError("retained file path must not run Opus encoding")
monkeypatch.setattr(util, "pcm_to_data_stream", _forbid_opus)
tts = LocalTTS(tmp_path, False)
conn = SimpleNamespace(tts=tts, sample_rate=16000)
tts.conn = conn
asyncio.run(helloHandle.wakeupWordsResponse(conn))
assert wakeup_cache.get_wakeup_response(tts.voice) is not None
def test_missing_retained_file_preserves_cache_and_releases_lock(wakeup_runtime):
helloHandle, wakeup_cache, _ = wakeup_runtime
wakeup_cache.generate_file_path = Mock()
wakeup_cache.update_wakeup_response = Mock()
conn = SimpleNamespace(
tts=SimpleNamespace(voice="test-voice", to_tts=lambda _: "missing.wav"),
sample_rate=16000,
)
with pytest.raises(FileNotFoundError):
asyncio.run(helloHandle.wakeupWordsResponse(conn))
wakeup_cache.generate_file_path.assert_not_called()
wakeup_cache.update_wakeup_response.assert_not_called()
assert not helloHandle._wakeup_response_lock.locked()
Whitespace-only changes.
@@ -0,0 +1,134 @@
"""服务端插件执行器:同步插件函数在线程池里跑,不卡事件循环;async 插件照常 await;异常变成 ERROR 响应。"""
from __future__ import annotations
import asyncio
import time
from types import SimpleNamespace
from core.providers.tools.server_plugins.plugin_executor import ServerPluginExecutor
from plugins_func.register import Action, ActionResponse, ToolType, all_function_registry
def _register(name, func, tool_type):
item = SimpleNamespace(name=name, func=func, type=tool_type, description={})
all_function_registry[name] = item
return item
def test_sync_plugin_runs_off_loop_and_async_plugin_is_awaited():
conn = SimpleNamespace(config={})
async def paced_sleep(delay):
# 不能用 asyncio.sleep 计时:conftest 在 mock 模式会把它换成不让出循环的桩,
# 这里放到线程池里睡,保证 ticker 每拍都真正让出事件循环。
await asyncio.get_running_loop().run_in_executor(None, time.sleep, delay)
def slow_sync(conn_arg, entity_id):
time.sleep(0.3) # 模拟阻塞 IO
return ActionResponse(Action.RESPONSE, None, f"{entity_id} 已打开")
async def fast_async(question):
await asyncio.sleep(0)
return ActionResponse(Action.REQLLM, f"答:{question}", None)
def boom(conn_arg):
raise RuntimeError("连不上")
_register("t_slow_sync", slow_sync, ToolType.SYSTEM_CTL)
_register("t_fast_async", fast_async, ToolType.WAIT)
_register("t_boom", boom, ToolType.SYSTEM_CTL)
try:
async def go():
ex = ServerPluginExecutor(conn)
ticks = []
async def ticker():
while True:
ticks.append(time.monotonic())
await paced_sleep(0.02)
t = asyncio.create_task(ticker())
r = await ex.execute(conn, "t_slow_sync", {"entity_id": "light.living"})
t.cancel()
assert r.action == Action.RESPONSE and r.response == "light.living 已打开"
assert len(ticks) >= 5 # 300 ms 里事件循环一直在转(没被 time.sleep 卡住)
r2 = await ex.execute(conn, "t_fast_async", {"question": "几点"})
assert r2.action == Action.REQLLM and r2.result == "答:几点"
r3 = await ex.execute(conn, "t_boom", {})
assert r3.action == Action.ERROR and "连不上" in r3.response
r4 = await ex.execute(conn, "t_missing", {})
assert r4.action == Action.NOTFOUND
asyncio.run(go())
finally:
for n in ("t_slow_sync", "t_fast_async", "t_boom"):
all_function_registry.pop(n, None)
def test_conn_passing_rules_and_compat_branches():
"""code 3/4/5 传 conn;WAIT/type=None 不传;同步函数返回协程会被兜底 await。"""
conn = SimpleNamespace(config={})
seen = {}
def prompt_fn(conn_arg):
seen["prompt"] = conn_arg
return ActionResponse(Action.RESPONSE, None, "切角色")
def wait_fn():
seen["wait"] = True
return ActionResponse(Action.RESPONSE, None, "等")
def no_type_fn():
seen["no_type"] = True
return ActionResponse(Action.RESPONSE, None, "无类型")
def sync_returns_coro(conn_arg):
seen["coro"] = conn_arg
async def inner():
return ActionResponse(Action.RESPONSE, None, "协程兜底")
return inner()
_register("t_prompt", prompt_fn, ToolType.CHANGE_SYS_PROMPT)
_register("t_wait", wait_fn, ToolType.WAIT)
_register("t_no_type", no_type_fn, None)
_register("t_sync_coro", sync_returns_coro, ToolType.SYSTEM_CTL)
try:
async def go():
ex = ServerPluginExecutor(conn)
r1 = await ex.execute(conn, "t_prompt", {})
assert seen["prompt"] is conn and r1.response == "切角色"
r2 = await ex.execute(conn, "t_wait", {})
assert "wait" in seen and r2.response == "等"
r3 = await ex.execute(conn, "t_no_type", {})
assert "no_type" in seen and r3.response == "无类型"
r4 = await ex.execute(conn, "t_sync_coro", {})
assert seen["coro"] is conn and r4.response == "协程兜底"
asyncio.run(go())
finally:
for n in ("t_prompt", "t_wait", "t_no_type", "t_sync_coro"):
all_function_registry.pop(n, None)
def test_sync_plugin_uses_dedicated_pool():
"""同步插件必须跑在专用池上,不能用 asyncio 默认池(DNS 解析等共用默认池)。"""
import threading
conn = SimpleNamespace(config={})
seen = {}
def sync_fn():
seen["thread"] = threading.current_thread().name
return ActionResponse(Action.RESPONSE, None, "ok")
_register("t_pool", sync_fn, ToolType.WAIT)
try:
async def go():
ex = ServerPluginExecutor(conn)
r = await ex.execute(conn, "t_pool", {})
assert r.response == "ok"
asyncio.run(go())
assert seen["thread"].startswith("plugin"), seen["thread"] # 专用池线程名前缀
finally:
all_function_registry.pop("t_pool", None)
@@ -0,0 +1,84 @@
"""append_devices_to_prompt:devices 写成列表或一段字符串都能拼进提示词。
config.yaml 示例和手机端智控台给的是列表,以前会报
can only concatenate str (not "list") to str,设备清单进不了提示词,模型只能猜 entity_id。
"""
import pytest
import plugins_func.functions.hass_state # noqa: F401 注册 hass_state 模块,填充 module_func_map
from plugins_func.functions.hass_init import append_devices_to_prompt
HEADER = "\n下面是我家智能设备列表(位置,设备名,entity_id),可以通过homeassistant控制\n"
BASE_PROMPT = "你是小智。"
class FakeDialogue:
def __init__(self):
self.system_messages = []
def update_system_message(self, prompt):
self.system_messages.append(prompt)
class FakeConn:
def __init__(self, plugin, source="home_assistant", functions=("hass_get_state", "hass_set_state")):
self.intent_type = "function_call"
self.config = {
"selected_module": {"Intent": "function_call"},
"Intent": {"function_call": {"functions": list(functions)}},
"plugins": {source: plugin},
}
self.prompt = BASE_PROMPT
self.dialogue = FakeDialogue()
def _plugin(**extra):
return {"base_url": "http://homeassistant.local:8123", "api_key": "test-token", **extra}
def test_list_devices_one_per_line():
conn = FakeConn(_plugin(devices=[
"客厅,玩具灯,switch.toy_lamp",
" 卧室,台灯,switch.desk_lamp ",
"",
]))
append_devices_to_prompt(conn)
expected = BASE_PROMPT + HEADER + "客厅,玩具灯,switch.toy_lamp\n卧室,台灯,switch.desk_lamp\n"
assert conn.prompt == expected
assert conn.dialogue.system_messages == [expected]
def test_string_devices_kept_as_is():
# 网页端智控台和 HA 集成文档给的是一段字符串
devices = "客厅,玩具灯,switch.toy_lamp;\n卧室,台灯,switch.desk_lamp;"
conn = FakeConn(_plugin(devices=devices))
append_devices_to_prompt(conn)
assert conn.prompt == BASE_PROMPT + HEADER + devices + "\n"
def test_mobile_console_hass_state_list():
# 智控台下发的插件配置键是 hass_state;手机端把 array 字段按行切成了列表
conn = FakeConn(_plugin(devices=["客厅,玩具灯,switch.toy_lamp"]), source="hass_state")
append_devices_to_prompt(conn)
assert conn.prompt.endswith(HEADER + "客厅,玩具灯,switch.toy_lamp\n")
def test_module_name_in_functions():
# functions 里写模块名 hass_state(config.yaml 注释的写法),展开后应能匹配
conn = FakeConn(_plugin(devices=["客厅,玩具灯,switch.toy_lamp"]), functions=("hass_state",))
append_devices_to_prompt(conn)
assert conn.prompt.endswith(HEADER + "客厅,玩具灯,switch.toy_lamp\n")
@pytest.mark.parametrize("plugin", [_plugin(), _plugin(devices=None), _plugin(devices=[])])
def test_missing_or_empty_devices_does_not_raise(plugin):
conn = FakeConn(plugin)
append_devices_to_prompt(conn)
assert conn.prompt == BASE_PROMPT + HEADER + "\n"
def test_without_hass_functions_prompt_unchanged():
conn = FakeConn(_plugin(devices=["客厅,玩具灯,switch.toy_lamp"]), functions=("get_weather",))
append_devices_to_prompt(conn)
assert conn.prompt == BASE_PROMPT
assert conn.dialogue.system_messages == []
Loaded 100 of 103 files, more files were not shown because too many files have changed in this diff. Show more