Compare commits

...
607 Commits
Author SHA1 Message Date
hrz 8f2dda09cf Bump to 0.9.8 (#3469) 2026-10-23 18:54:05 +08:00
hrz d4ff9c5c17 修掉通讯录的两处越权(读 + 写),并补上三处空指针 (#3460)
* 读取设备通讯录先验设备归属

登录用户传任意 MAC 就能读到别人设备通讯录里的备注和称呼。归属判断收进一个方法,读取通讯录、改别名、改权限都走它,不属于当前用户的设备一律按查不到处理。

* 通讯录存别名不再空指针

对方设备没登记时取备注会直接崩,整条权限授予请求跟着挂掉。取不到就退回尾号称呼,呼叫时主叫没登记也不再崩,顺手删掉一行调试打印。

* 解绑先验归属,通讯录别名不再落空

解绑接口原来不核对设备归属,谁知道对方的设备 id 就能把那台设备的通讯录清空,设备本身倒是因为带了用户条件没删掉。现在先看归属,不是自己的直接不动。

设备登记了但用户没起名时,别名会落成空,缓存刷新把整条跳过,对方再也呼不过来。改成和呼叫那处一样,两级判空都取不到才退回尾号称呼。顺带加了一条回填,把写进去的空值补上。

呼叫那条测试原来会真发请求也没验到东西,改成桩掉网关直接看发出去的称呼。

* 通讯录空别名兜底,撞名加后缀

缓存拼表之前,空别名先用尾号称呼顶上,不再整条跳过,用户没起名的设备也能被呼到。

昵称在同一个人的通讯录里是索引,两台撞名时后写的会盖掉先写的,按昵称呼会呼错设备。加后缀区分,规则和存表时那套一致。取名靠先后,而查询不带排序,所以先按地址排一遍,同样的数据每次刷新结果一样。

数据库没有排序保证,去重全靠顺序,所以这里先排再拼。迁移脚本不用动,运行时的兜底能管住它没覆盖到的行。

* 通讯录取名两条路径对上了

后缀原来只写进缓存,接口直查库返回原值,同一份数据给的是两个名字。
用户照控制台看到的名字呼叫,命中的不是同一台设备。

把取名抽成一个方法,两边都走一遍。顺带把权限位排到地址前面,
撞名时能呼的那台先拿原名,反过来用户怎么点都只会说没权限。

迁移取末两位时改成跟着小写化,和运行期算出的是同一个称呼。

* 同一个人被拆成两本通讯录,一台设备消失

两条记录的归属地址只差大小写时,会被算成两个人,各拿一个同名,
写进同一本时后者盖掉前者,一台设备就从缓存里消失了。

把归属地址归一抽成一个方法,算称呼、写缓存、查缓存都走它。

* 通讯录接口按最近改过的排前,这个约定有用例钉住

智控台页面是按对方地址把备注合进去的,接口返回顺序它并不关心,
所以现在这么排对界面没有效果。

用例留着是防它被当无用代码删掉,删了会有用例报出来。

* 迁移串到别人的私名,撞车的组改走尾号称呼

通讯录回填名字时,设备表里同一个 MAC 有两行(同一台被两个用户各登记一次),
MySQL 自己挑一行填进去,跟这行属于谁无关。改成只有一台时才取智控台名字,
多台就退回按 MAC 末两位算的称呼。名字照样填上,缓存照样刷新,对方还是呼得到。

补了 12 条在真库上跑的用例,另配一条排查重复 MAC 的语句。

* 去掉走不到的分支和没人用的桩,补上末尾换行

getUser() 取不到 subject 或 principal 时返回的是空对象,不是 null,
user == null 走不到,删掉;用例改成桩空对象,测真正的匿名那条路。
归属判断挪进 service 之后,controller 层四处 SecurityUser 的桩没人用了,删掉。
四个文件末尾少了换行,补上。另加一条用例钉住判重分组的比较规则:
只折叠大小写,不折叠分隔符。

* 移除回填通讯录别名的迁移,运行时已经兜住了

这个迁移是本 PR 带进来的。它要把空别名刷成称呼,可运行时的兜底已经做了一样的事,
而且覆盖得更宽:那边连纯空格、地址为空和不足两位的都管。删掉它,功能不受影响。

删它有两条理由。一是重复。二是它按 MAC 去设备表取名字,
而一个 MAC 在设备表里可能有多行——同一台设备被两个用户各登记一次就能造出来,
这时候取到的是别人给自己设备起的名字。写在这里,是怕后来的人以为这个坑不存在。

* 归属校验撞到重复 MAC 不再报错

库里同一 MAC 可能有两行只差大小写,selectOne 会抛 TooManyResultsException,把读通讯录接口整个带崩。改成 selectList 按 id 取第一条做归属校验,撞行时正常返回,不再报错。
2026-10-23 17:43:05 +08:00
wengzh 3709540efd Merge pull request #3445 from xinnan-tech/fix/huoshan-double-stream-4-bugs-y15
huoshan_double_stream 修 4 个会话协议与监听竞态 bug
2026-10-23 17:37:05 +08:00
wengzh 333050ae2e Merge pull request #3464 from xinnan-tech/chore/bump-deps-20261023
chore(deps): 批量升级 15 个依赖,适配 mcp 2.x
2026-10-23 17:05:05 +08:00
Sakura-RanChen 95ce49ffc9 删除 benchmark/ 下 3438~3441 回归验证脚本
四处修复均已验证收敛,脚本不进 CI、裸跑依赖 manager-api,保留仅为仓库噪声;后续若再动 huoshan_double_stream 时以 pytest 覆盖
2026-10-23 17:04:52 +08:00
Sakura-RanChen be75e7bd39 chore(deps): 批量升级 15 个依赖,适配 mcp 2.x
源 PR: #3426 #3427 #3428 #3429

剔除 4 项:
- edge-tts 7.3.1->7.2.8 是降级
- powermem >=1.1.7 要求 Python>=3.11
- openai 3.29.0 与 powermem 冲突(powermem 要求 openai<3.0.0)
- ruamel.yaml 0.19.1 与 hyperpyyaml 冲突

mcp 2.x 适配: streamablehttp_client 更名为 streamable_http_client,headers/timeout
改由 http_client 承载,产出从 3 元组变 2 元组,ProgressFnT 移到 mcp.shared.dispatcher
2026-10-23 15:15:50 +08:00
hrz 24fbf7eee2 登录 token 12h 静默失效补 silent refresh 接口 #3392 (#3402)
* 登录 token 12h 静默失效补 silent refresh 接口 #3392

方案 A:后端 POST /xiaozhi/user/refresh(anon),旧 token 校验通过后强制覆盖为新 token 字符串,旧 token 在 DB 查不到即失效。前端 httpRequest 在 sendRequest 入口检查 token 剩余有效期,剩 1h 内触发 silent refresh(独立 fly 实例 + 单飞锁),成功后 setToken 替换并自动继续发送当前请求,失败走 clearAuth 跳登录。

* refreshToken 改 CAS 防并发 race + 拆前端副作用 #3392

P1:SysUserTokenDao 新增 refreshTokenCAS,条件 UPDATE(oldToken + 未过期 + userId),受影响 0 行抛 TOKEN_INVALID;refreshToken 改调它,避免读-改-写 race。P2:refreshToken 加 @Transactional;httpRequest.getTokenExpireAt 内的 localStorage.setItem 挪到独立 persistTokenExpireAt,由 shouldRefresh 入口调用。

* getTokenExpireAt 改读 localStorage 与持久化同源 #3392

pr01 三改:原代码 getTokenExpireAt 读 store(state.token 缓存),与 persistTokenExpireAt 写 localStorage 不同源,导致 _localStoredAt 缺失 → NaN → shouldRefresh 永 false → silent refresh 永不触发。修复:getTokenExpireAt 改读 localStorage,persistTokenExpireAt 同步刷 store,保证两端一致。e2e 已验证:注入 expire=1800 token → reload → role-config mount 触发 sendRequest → Network 抓到 POST /user/refresh 200 → localStorage token 被替换为新 12h token(含 _localStoredAt)。

* 应用启动同步 localStorage token 到 store #3392

PR #3402 扩展:守卫放行后首次 API 请求时,vuex state.token 还是 null,httpRequest 读 store 取不到 Authorization 头,会被后端 401 → clearAuth 跳登录。在 main.js 应用启动阶段就把 localStorage 里的 token commit 进 store,避免 race。不动 refresh 链路与其它业务文件。
2026-10-23 09:47:41 +08:00
hrzandSakura-RanChen fdf280f84a 模板 GET 入口迁回 AgentTemplateController #3395 (#3401)
* 模板 GET 入口迁回 AgentTemplateController #3395

根因:AgentController 已存在 GET /agent/template/templateList,AgentTemplateController 缺显式 GET,于是路由虽通却跨 controller 暴露模板数据,且职责错位。修复:删 AgentController.templateList,同名 @GetMapping(无 path)落到 AgentTemplateController,权限 normal、orderByAsc sort、复用现有 service.list。验证:spring-boot 启动期无 Ambiguous;/agent/template、/page、/{id} 三路由均被 Shiro 一致拦截,权限/路径正确。

* 补 P2 注释说明 normal 权限反直觉是有意保留 #3395

在 getAgentTemplates 上方加 2 行注释,说明单独 normal 权限是为保持原行为契约(普通用户也能查可用模板),与类级 superAdmin 故意分离;非疏忽。无逻辑变更。

* 去除未使用引用

---------

Co-authored-by: Sakura-RanChen <1908198662@qq.com>
2026-10-22 11:42:48 +08:00
wengzh 498191519c Merge pull request #3452 from xinnan-tech/py-test-wakeup
Py test wakeup
2026-10-22 11:08:38 +08:00
wengzh f614489d81 Merge pull request #3454 from xinnan-tech/web-test-json
Web test json
2026-10-20 15:44:16 +08:00
Sakura-RanChen 6f958ee4a6 fix: 新增模型页面dict字段错误解析 2026-10-20 15:34:40 +08:00
TXyy2023 b226ee5430 fix(manager-web): decode stored dictionaries before editing (#3450) 2026-10-20 11:53:02 +08:00
Sakura-RanChen 9c30b5e035 文件直接写入wav,无需经过二次编解码 2026-10-20 10:45:31 +08:00
TXyy2023 d531d04281 fix(tts): handle retained audio files in wakeup cache (#3451) 2026-10-20 10:00:20 +08:00
Sakura-RanChen d8f8a4a083 - 取消屏障保证:新连接创建前 _cancel_monitor_task 已等旧监听任务完全退出,
旧任务不可能读新连接或误关新连接(check_huoshan_3440 动态断言覆盖)
- mismatch 分支仅保留纯跳过,删除旧会话结束事件的复位遗留,
  activate_session 写入点收敛到会话生命周期(start / matched 结束 / close)
- 删除 _session_start_future 启动确认机制:start_session 发送即返回,
  SessionFailed 由 monitor 记 error 并复位 activate_session,无挂起风险
- 回归脚本同步调整:3438 重写为新行为断言,3440 用屏障断言替换 snapshot 断言
2026-10-19 17:52:25 +08:00
hrz 6840210ea8 check_huoshan_3439 改 FIRST 分支行为断言
原 test 用正则 grep 源码验存在性,pr02 复评指出这样测不到实际调用。
改成起后台线程跑真 tts_text_priority_thread、构造 FIRST 消息入队、
run_coroutine_threadsafe patch 成立即完成的 future,最后断言
reset_stream_state 与 opus_encoder.reset_state 各被调一次。
2026-10-17 07:57:30 +08:00
hrz 0a54124d0d huoshan_double_stream 修 3 处复评反馈
init 漏写 _pending_prefix 字段、start_session 用已弃用的
get_event_loop、finish_session 留着多余的 getattr 兜底。补字段、
换 get_running_loop、去兜底,3 处都修了。
2026-10-17 07:57:26 +08:00
hrz ddc441bc59 补 huoshan_double_stream 4 bug 复现脚本
四个独立脚本各覆盖一个 issue:start_session 等 future 确认、句尾残留与编码器
冲刷、monitor snapshot 隔离、parser_response 四处协议解析。共享 _huoshan_mock
构造一个绕开父类 init 的假 provider 实例。
2026-10-17 07:39:51 +08:00
hrz 9b7fb3e826 parser_response 四处二进制协议解析修正
serialization_method 错用掩码当移位数恒 0,改成 >>4;message_type 写
== FULL or AUDIO_ONLY_RESPONSE 优先级下恒真,改 in (...);offset 硬编码 4 在
header_size>1 全错位,改成 header_size * 4;SessionCanceled 漏进协议分支被
else 误读,补进统一解析。
2026-10-17 07:39:51 +08:00
hrz a6cdbf9410 监听任务用 snapshot 隔离跨连接
旧 monitor 循环直接 self.ws.recv(),连接重建后旧任务读新下行;finally 无条件
关 self.ws 误关新连接。改成入口绑 monitor_ws / monitor_task,recv 用 snapshot;
旧 session 事件 continue 不再 reset 当前;finally 判 id 相同才关 ws / 清引用。
2026-10-17 07:39:51 +08:00
hrz 75fd5d8cc3 句尾替换词残留先发 + 会话结束冲刷编码器
旧 finish_session 不发 _pending_prefix 残留,句尾替换词被扣时整段丢;monitor / to_tts
收 SessionFinished 不调 is_end=True 冲刷 opus 编码器,每句固定丢尾。改成
finish_session 开头先发残留,monitor / to_tts 调 is_end=True 冲刷,FIRST 同步重置。
2026-10-17 07:39:51 +08:00
hrz 6fe5630346 TTS 会话等 SessionStarted 确认
旧逻辑 start_session 发完 StartSession 即返回;服务端鉴权失败或 14s 内 26005 时
文本线程继续给死会话发文本,整轮无声。改成末尾 await 一个 future:monitor 读到
SessionStarted set True、读到 SessionFailed set False,超时报错让文本线程感知。
2026-10-17 07:39:50 +08:00
hrz eb9dc8ca0f 把 pydub 探针绑到当前解释器同目录的完整版 ffmpeg (#3444)
edge_tts 升 7.3.1;edge.py 撤 WSS_URL patch 改默认
(4.36 跑默认 URL 稳定)。util.py 加 PATH 临时注入,
绕开 Mac 上 Trae CN 注入的精简版 ffmpeg/ffprobe,
让 pydub 拿 conda env 完整版。引用计数加锁防多线
程串。
2026-10-15 15:43:39 +08:00
wengzh 27399a2fb6 Merge pull request #3430 from xinnan-tech/py-text-test
Py text test
2026-10-14 14:11:53 +08:00
wengzh 7c43ecfcb5 Merge pull request #3434 from xinnan-tech/py-test-asr
fix: SenseVoice 日语识别结果按词带空格 (#3412)
2026-10-14 11:54:39 +08:00
korexil 0be634c627 fix: SenseVoice 日语识别结果按词带空格 (#3412)
SenseVoice 识别日语时偶尔按词输出空格(如「今日 の こと を 少し 長め に」),
这些空格会原样进入对话历史并交给 LLM。lang_tag_filter 现在去掉两个
中日文字符之间的空格,韩文、英文的词间空格保持不变。
2026-10-14 10:32:56 +08:00
Sakura-RanChen 932962e8d3 补充注释说明,调整测试样例 2026-10-14 10:10:28 +08:00
wengzh 60c2a7ee17 Merge pull request #3431 from xinnan-tech/py-content-text
Py content text
2026-10-13 17:13:42 +08:00
Sakura-RanChen a01f0aa29b 去除没必要的测试,上游为str不需要防御性增加 2026-10-13 16:54:42 +08:00
korexil 4445aa7510 fix: FunASR 识别结果以 JSON 原样进入对话历史 (#3411)
FunASR 返回 {"content","language","emotion"} 时,startToChat 只在含 speaker
字段时才解包,否则整段 JSON 字符串会作为用户消息交给 LLM 并写入对话历史。
现在没有 speaker 时只取 content。
2026-10-13 16:44:13 +08:00
wengzh dc86a780a2 Merge pull request #3418 from xinnan-tech/py-test
Py test
2026-10-13 15:47:30 +08:00
Sakura-RanChen a5e47f2c4e 补充字符,调整字符判断 2026-10-13 15:29:28 +08:00
Sakura-RanChen 2057b99aeb 迁移测试用例,优化相关注释 2026-10-13 14:31:21 +08:00
wengzh 7957817a72 Merge pull request #3425 from xinnan-tech/chore/dependabot-monthly-groups
Dependabot 改月度更新并按用途分组
2026-10-13 11:41:15 +08:00
hrz a6eeb107f4 fix(deps): 补 dependabot ignore portalocker
requirements.txt 把 portalocker 锁在 3.x,4.x 与 qdrant-client<1.20 的
portalocker<4.0 冲突 (PR #3360)。Dependabot 默认推 4.x 升级会破约束。
依 openrz 评审 P1 在 ignore 列表追加 portalocker 一项。
2026-10-13 11:28:37 +08:00
wengzh 847337cabb Merge pull request #3419 from xinnan-tech/fix/3391-route-guard-whiteList
fix(router): 路由守卫改 anon 白名单反向匹配
2026-10-13 11:15:15 +08:00
Sakura-RanChen b8f23ffa66 调整相关代码 2026-10-13 10:40:47 +08:00
lww9029 dc581d7185 fix: 修复登录状态下还能访问白名单路由问题 2026-10-13 10:36:25 +08:00
hrz 741ec299c0 fix(tts): edge-tts 失声改走 msedgeservices (#3420)
* fix(tts): edge-tts 失声改走 msedgeservices

微软 2026-10 起对 legacy 消费者接口返回 403,
edge-tts 7.2.8 仍走 speech.platform.bing.com 走不通。
上游 PR rany2/edge-tts#491 切到 api.msedgeservices.com 修好,
但 7.2.8 仍是 PyPI 最新版本。

仓库在 edge.py 顶部临时把 WSS_URL 顶住,等 7.2.9 出了删掉。

* fix(tts): patch 还要写 communicate.WSS_URL

edge-tts 7.2.8 仍指 legacy bing 接口,10 月起微软 403。
上游 PR #491 已切到 msedgeservices.com,7.2.9 才进 PyPI,先顶住。
上次只改 constants.WSS_URL 不够,communicate 用 from import 是模块局部名,
LEGB 找不到新值。这次同时改 communicate.WSS_URL 才生效。
2026-10-13 09:58:30 +08:00
hrz 505925fb21 Dependabot 改月度更新并按用途分组
原本每周跑一次会刷很多零碎 PR,看着烦。改成每月一次,并把第三方 AI 服务、本地模型、网络栈拆成几组聚一起,每组出一个 PR 就行。被锁住的 torch/numpy/websockets 几个直接忽略掉,免得它推升级 PR 进来。
2026-10-13 09:34:02 +08:00
hrz 4aa63baf75 fix(router): 路由守卫改 anon 白名单反向匹配
业务路由越加越多,原来硬编码的 8 项守卫名单跟不上了,
未登录直接进十来个业务页看到骨架再被踢,体感抖。

守卫改成只对登录、注册、找回密码、根路径这四页放行,其余一律查 token。
顺手把 9 处没人读的 meta.requiresAuth 和 meta.title 死字段删了。
2026-10-12 18:02:36 +08:00
korexil 70ab19d23f fix: TTS 分句在引号内切开句子 (#3413)
_get_segment_text 会在引号内的标点处切分(如「わぁ!」って、“好!”然后),
把引号拆到两段里,或留下只有「。」的片段。新增 _quote_safe_cut:切分点后的
右引号与句末标点一并带走;位于未闭合引号内或右引号后句子仍在继续时,顺延到
下一个标点。不含引号的文本切分结果不变。
2026-10-12 16:26:40 +08:00
Sakura-RanChen e5f326285e update: 优化client 生命周期对齐业务请求——重试内连接复用、锁外构造、safe_close 可打断重试 2026-10-12 15:42:51 +08:00
hrz 4576e391bb fix(ci): 阿里云 PyPI 镜像 metadata 缺失回退官方源 (#3417)
基镜像构建挂了——日志显示 pip 在装 requirements.txt 阶段退出码 1,
罪魁是阿里云 pypi 镜像对 mcp==1.22.0 的元数据文件 404。

装包那行加 --extra-index-url https://pypi.org/simple,
主源没货时自动去官方源捞。
2026-10-12 10:28:35 +08:00
Yanhan Ruan ef1d24e40d fix(server): 修复临时事件循环结束后 HTTP 客户端未释放的问题 (#3416) 2026-10-12 10:00:28 +08:00
wengzh c30c40bf56 Merge pull request #3415 from xinnan-tech/chore/bump-deps-20261010065106-1659
chore(deps): 批量升级 4 个允许的依赖
2026-10-10 15:24:17 +08:00
Sakura-RanChen 199c4ce365 chore(deps): batch bump 允许升级的依赖 2026-10-10 14:53:30 +08:00
hrz a6cd755cf2 聊天记录弹框底部不再留大量空白 (#3410)
弹框本身和 body 用 !important 撑到视口(body 顶开 size="large" 默认多扣的 56px footer 高度)。body 的高度由 flex: 1 自动撑满,max-height 只用来顶开原来的 950px 上限。
2026-10-09 17:24:39 +08:00
hrz 23d0e64d9d 召回测试弹框与切片弹框样式对齐 (#3409)
召回测试弹框宽度从 900px 调到 1200px 并启用大弹框样式,高度、上下间距与切片弹框保持一致。结果区域改用 flex 填充而不是写死的 400px 高度。
2026-10-09 17:18:06 +08:00
hrz 96c224c4e4 切片和聊天记录弹框统一为大弹框样式 (#3407)
知识库查看切片弹框加 size="large",聊天记录弹框去掉旧样式覆盖并把 body 改成 flex 列布局,底部下载按钮不再绝对定位,两个弹框跟历史版本对齐。
2026-10-09 17:02:49 +08:00
hrz 2b6483662a 召回测试弹框回车不再刷新页面 (#3408)
给 el-form 加阻止默认提交的处理,按回车时不再触发浏览器刷新。
2026-10-09 17:01:52 +08:00
hrz aef973e6b9 7 处 el-select 加 filterable 支持输入筛选 #3405 (#3406)
主诉:手动添加设备下拉无搜索(FIRMWARE_TYPE 字典 54 项)。统一修同类位置:替换词多选加 collapse-tags;登录/注册/找回密码的区号下拉加 filterable(海外 200+ 项)。无新依赖、纯 element-ui 原生属性。
2026-10-08 15:39:46 +08:00
hrz 7a260ee143 弹框关掉后再点不开以及宽度高度统一收口 (#3388)
* 统一管理后台弹框更舒展

把 CustomDialog 默认宽度从 600 加到 720,正文给到至少 360 的高度,
默认弹框就不再显得局促。参数编辑弹框也跟着加宽。智能体历史版
本那条列表、对比和恢复预览都给到至少 420 的最小高度,避免内容
少时框太矮看起来小气。其他像添加智能体、绑定设备这些原本就合适
的弹框没动。

* 弹框外观统一收口

原本几个弹框长得参差不齐:标题字号、按钮文案、留白高度各管各的。这次把新建智能体、新建设备、改模型、改参数这几个弹框接到同一个底座上,按钮统一成「确定」「取消」,输入框字号高度对齐,遮罩背景加了模糊,弹框太高就内部滚动,四个弹框看起来一个模子

* 修复弹框关掉后再点不开

之前几个弹框改用新底座的时候,关闭那一路少了一段转发:父页面用双向绑定来收弹框关闭的信号,子页面应该再把关闭事件抛上去告诉父页面。这一段被删了,下次再点添加按钮时父页面那边状态没复位,弹框就拉不开来。现在把这段补回去。

* 弹框宽度和高度统一收口

同一批弹框的尺寸还是各管各的:模型配置里改参数那个是跟着窗口大小按比例伸缩,历史提示词的源编辑弹框是 900 像素,参数管理里又是 720 像素,智能体变更详情那个上下还留了一大块空白。这次收到同一个共用底子上:宽度都对齐到 720 像素,弹框默认上下贴齐屏幕边、内容多时中间滚动,变更详情那个高度撑到接近屏幕顶到底。

* 管理后台弹框按大小分类统一外观

管理后台里的弹框按大小分成两类来收口。占整页的大弹框(历史版本、变更详情、模型修改、模型参数修正等)撑满整个浏览器高度,下面不再留白;小弹框(添加智能体、添加设备、知识库、字典、角色模板等)改成对齐屏幕正中,不再顶在屏幕上沿。修改模型那条弹框也加宽到 960,左右留白加大到 40,跟历史版本对齐,看起来更舒展。

* 大弹框上下留白收口到十像素并按三档统一宽度

大弹框上下各留十像素让边框和圆角清晰可见。宽度按用途分三档:表单 960、含代码 1080、表格 80% 加宽度上限。供应器、上下文供应器、音色克隆都改 960;TTS 从 90% 改 80%;角色模板从 1200 改 1080;缓存查看从 70% 改 80%。新增模板的大文本框改成自适应高度。

* 弹框样式与文案收口处理

AddDeviceDialog 确认后关闭弹框;CustomDialog 去掉 :global 与紧邻选择器,合并 el-dialog__body 样式块;删除未使用的 --dialog-form/--dialog-code;AddWisdomBodyDialog 的 cancel/handleClose 委托到同一个实现;多语言文案同步去冒号与双点。

* 弹框遮罩样式移到全局作用域

scoped 块里的 ::v-deep .v-modal 打不到 body 层的遮罩,改在 global.scss 用全局选择器。

* 弹框 wrapper 居中样式移到全局作用域

scoped 块里的 ::v-deep .el-dialog__wrapper 同样打不到 element-ui 自渲染的 wrapper,跟 .v-modal 一起挪到 global.scss。

* 弹框提交状态异常收口

设备激活码错误路径不再卡 loading;Provider 与固件弹框补 loading 兜底。

* 弹框视觉异常收口

首页删除确认弹框 wrapper 居中;历史版本弹框去掉内层多余滚动条。

* 设备 API 链式返回修复

bindDevice 等方法补上 return,让 .fail() 兜底回调能正常注册到请求对象上。

* 设备 API 错误回调参数补全

device.js 六个方法加 failCallback 参数;user.js login 网络重试补 failCallback;AddDeviceDialog 改四参数签名。

* 设备状态 API 失败回调引用修复

getDeviceStatus 的 .fail() 块把函数体内 callback(res) 改成与签名一致的 if (failCallback) failCallback(err),避免业务错误时抛 ReferenceError。
2026-10-07 08:03:25 +08:00
欣南科技 87c6df77b0 Merge pull request #3386 from xinnan-tech/0.9.7
Bump to 0.9.7
2026-09-29 10:03:47 +08:00
hrz eb51f420a2 Bump to 0.9.7 2026-09-29 09:13:14 +08:00
DaGou12138 1eefb4ea86 Merge pull request #3385 from xinnan-tech/serpapi
Serpapi
2026-09-28 10:25:33 +08:00
hrz 1211f8202b Merge pull request #3376 from googio/feat/serply-search
feat(web_search): 新增Serply搜索源
2026-09-28 09:36:42 +08:00
wengzh 788f5301fd Merge pull request #3378 from xinnan-tech/fix-deps
fix(deps): 移除 chardet 直接依赖
2026-09-21 09:38:29 +08:00
Sakura-RanChen edad16aedf fix(deps): 移除 chardet 直接依赖 2026-09-20 15:55:21 +08:00
serply 65a3fae9fc feat(web_search): 新增Serply搜索源
联网搜索插件在秘塔和Tavily之外增加Serply,返回谷歌搜索结果。
通过 provider: "serply" 选用,未配置时行为不变。
2026-09-17 16:45:28 -04:00
wengzh f4ba65f224 Merge pull request #3375 from xinnan-tech/fix-snapshot-dialog-diff-style
fix: 修复 v-code-diff 斜体样式干扰 diff 展示
2026-09-17 17:54:02 +08:00
lww9029 87c2bae55a fix: 修复 v-code-diff 斜体样式干扰 diff 展示 2026-09-17 17:50:27 +08:00
wengzh 628d4cedb7 Merge pull request #3373 from xinnan-tech/feat-snapshot-dialog-code-diff
feat: 历史版本快照增加代码对比视图
2026-09-17 17:06:59 +08:00
Sakura-RanChenandSakura-RanChen 6c3f89cae1 chore(deps): 批量升级 4 个允许的依赖 (#3374)
* chore(deps): batch bump 允许升级的依赖

* test: 添加 5 条核心链路测试 + 默认 mock 模式

参照 PR #3370 的整体结构,新增 STT/LLM/TTS/RAG/Memory 五个 test_*.py
覆盖关键链路。conftest.py 默认 mock 模式:

- PYTEST_CONFIG_FILE 指向单模块配置文件(不入库)
- RUN_LIVE_API_TESTS=1 走真请求(必须配齐密钥)
- 真实存在的第三方包(openai/httpx)保留,仅 patch httpx 网络层
- 装了一半会炸的本地模型(torch/funasr/modelscope)强制 shim
- mock load_config / init_service,避免触发远程 manager-api

mock 模式目标:dep-upgrade orchestrator / CI 不需要任何密钥即可跑通。

---------

Co-authored-by: Sakura-RanChen <Sakura-RanChen@local>
2026-09-17 15:57:43 +08:00
lww9029 4745186cea style: 调整窄屏样式 2026-09-17 10:43:26 +08:00
lww9029 1ddc97b626 feat: 快照对话框增加代码对比视图 2026-09-17 10:04:15 +08:00
wengzh 6afc54a17d Merge pull request #3366 from xinnan-tech/feat-tts-alibl
Feat tts alibl
2026-09-11 17:35:08 +08:00
wengzh daccfeb9dd Merge pull request #3364 from xinnan-tech/fix-memory-title
fix: 记忆模式为nomem时错误生成标题
2026-09-11 17:34:43 +08:00
Sakura-RanChen 1846d3e935 调整相关逻辑 2026-09-09 16:17:13 +08:00
Sakura-RanChen b1a7607dd5 Merge pull request #3345 from apexlsai/feature/alibl-workspace-streaming
feat(tts): 完善阿里百炼 Workspace 流式语音合成
2026-09-09 14:22:59 +08:00
Sakura-RanChen d6d2da3d76 fix: 记忆模式为nomem时错误生成标题 2026-09-09 10:33:37 +08:00
CGD 5aa46538d5 Merge pull request #3358 from xinnan-tech/fix-audio-delay
fix: 修复固定延迟不生效,调整相关逻辑
2026-09-08 10:53:25 +08:00
Sakura-RanChen ed98be5347 Merge pull request #3357 from xinnan-tech/py-test-client-toctou
根治ManageApiClient单例TOCTOU竞态,锁保护+统一防护
2026-09-08 10:00:20 +08:00
Sakura-RanChen 79a95af399 fix: 修复固定延迟不生效,调整相关逻辑 2026-09-07 11:38:06 +08:00
3030332422 32cba36237 fix: 修复 ManageApiClient 关闭期间的连接池创建竞态 2026-09-07 11:14:50 +08:00
CGD 82e3c1874f Merge pull request #3353 from muxinshijie/feature/manage-api-client-toctou
fix: 根治ManageApiClient单例TOCTOU竞态,锁保护+统一防护 #3339
2026-09-07 10:31:58 +08:00
欣南科技 c478257517 Merge pull request #3356 from xinnan-tech/fix/docs-typos
docs: 批量修正文档里的错别字和拼写错误
2026-09-06 08:44:01 +08:00
欣南科技 9fdb72cfd7 Merge pull request #3355 from xinnan-tech/fix/3354-建议修改docsmcpvisionintegrationmd
docs: 修正视觉模型配置文档里的错字
2026-09-06 08:43:38 +08:00
hrz 3c31315ecc docs: 批量修正文档里的错别字和拼写错误
扫了仓库里的文档,挑出明显写错的字和拼错的英文,统一改了一遍。包括「Anaconda Prompt」漏字、「火上」→「火山」、「设别」→「设备」、「tababr」→「tabbar」、「aotodl」→「autodl」、「mananger-web」→「manager-web」等十几处。仅文档修改,不影响代码。
2026-09-06 08:37:51 +08:00
hrz 07c6cdf16a docs: 修正视觉模型配置文档里的错字
按 issue #3354,把"视觉打语言模型"改成"视觉大语言模型",跟上下文一致。
2026-09-06 08:34:57 +08:00
hrz 8a3c2c51a0 Merge pull request #3351 from taiwan-dog/pm/disable-default-websocket-keepalive
fix: disable library WebSocket keepalive for device server
2026-09-05 00:37:23 +08:00
hrz 046fdc8aa4 Merge pull request #3350 from taiwan-dog/pm/hello-before-mcp-initialize
fix: send server hello before MCP initialize
2026-09-05 00:26:15 +08:00
muxinshijie 61e4879826 fix: 根治ManageApiClient单例TOCTOU竞态,锁保护+统一防护 #3339 2026-09-05 00:14:09 +08:00
yillkid eda3c23d51 fix: disable library WebSocket keepalive for device server
websockets.serve() defaults to ping_interval=20 and ping_timeout=20.
ESP32 clients that do not answer those pings are dropped with
1011 keepalive ping timeout roughly 40 seconds into a healthy session.
The application-level ping is configured separately via
enable_websocket_ping.

Refs #3348
2026-09-04 19:00:24 +08:00
yillkid a03a0a73db fix: send server hello before MCP initialize
The MCP initialize task was scheduled before the server hello was sent,
so the MCP frame could reach the device first. ESP32 firmware waits for
the server hello before it treats the channel as ready, so the device
never answered the initialize request.

Refs #3347
2026-09-04 19:00:24 +08:00
欣南科技 27e780e4c1 Merge pull request #3342 from xinnan-tech/test-infrastructure
test: 新增三语测试基建(Python/Java/Vue)+ CI
2026-09-04 13:54:31 +08:00
hrz 4b1ccce575 Merge pull request #3346 from taiwan-dog/pm/fix-funasr-result-type
fix: normalize FunASR result type
2026-09-04 13:40:00 +08:00
yillkid 323c2248b4 [pm] fix: normalize FunASR result type 2026-09-04 13:10:58 +08:00
Jacob a31ba7ef6d fix(tts): synchronize Bailian streaming subtitles 2026-09-04 12:05:34 +08:00
Jacob 935184b1d0 feat(tts): support Bailian workspace endpoints 2026-09-04 12:05:09 +08:00
hrz c7b126ca2f Makefile test-web 同时跑契约与 vitest 2026-09-02 22:24:36 +08:00
hrz 95954c8dae 加日期格式化工具测试,24 例 2026-09-02 22:24:36 +08:00
hrz 17c0da7f87 加 featureManager 单元测试,7 例 2026-09-02 22:24:36 +08:00
hrz dcb3d3e0b0 加 deviceTime 工具 vitest 测试,22 例 2026-09-02 22:24:36 +08:00
hrz e58de03be2 配 vitest:jsdom 环境与覆盖率 2026-09-02 22:24:36 +08:00
hrz 975bbe2cfd 前端引入 vitest 测试工具链 2026-09-02 22:24:36 +08:00
hrz 88d6d13a62 加敏感数据工具测试,9 例 2026-09-02 22:24:36 +08:00
hrz c88149a5d4 加知识库适配器工厂测试,6 例 2026-09-02 22:24:36 +08:00
hrz 6a3aaf475e 加纠错词文件服务单元测试,9 例 2026-09-02 22:24:35 +08:00
hrz caab6a145d 加插件包导入冒烟测试,1 例 2026-09-02 22:24:35 +08:00
hrz 7ac454a1b9 加意图/模型/记忆工厂测试,6 例 2026-09-02 22:24:35 +08:00
hrz ba20bebaeb 加对话消息测试 11 例,含异步 2026-09-02 22:24:35 +08:00
hrz 922adbb290 加配置合并函数测试,7 例 2026-09-02 22:24:35 +08:00
hrz 6eca417534 加 current_time 测试 7 例,冻结时间 2026-09-02 22:24:35 +08:00
hrz 93d54f175a 加 output_counter 测试,7 例 2026-09-02 22:24:35 +08:00
hrz 34ab39ba69 加 textUtils 行为测试,11 例 2026-09-02 22:24:35 +08:00
hrz a0dd138e93 配 pytest 与 conftest,修模块路径 2026-09-02 22:24:35 +08:00
hrz b90dd01826 加 Python 冒烟测试,验证 pytest 流水线 2026-09-02 22:24:35 +08:00
hrz 22b1703557 新增 GitHub Actions,并行跑三语测试 2026-09-02 22:24:35 +08:00
hrz 3363965f63 新增顶层 Makefile,统一测试入口 2026-09-02 22:24:35 +08:00
hrz eb4b28c59e Merge pull request #3336 from muxinshijie/feature/fix-noneType-bug
fix: 修复safe_close多线程竞态导致NoneType空指针
2026-09-02 18:38:10 +08:00
muxinshijie 74cc31d1cc 服务关闭时主线程调用safe_close()将instance置None,与守护线程存在竞态条件,因此必须先判空 2026-08-27 00:13:21 +08:00
hrz c02becdc80 Merge pull request #3332 from MsThink/main
refactor(manager-web): 将备注重命名为 设备别名
2026-08-21 21:39:45 +08:00
colorful b16ed2ed36 style(manager-web): 调整通讯录中的设备状态在线/离线状态图标间距,居中 2026-08-20 10:54:15 +08:00
colorful af9abdd771 refactor(manager-web): 将备注重命名为 设备别名
- 更新所有语言包中的相关文案
- 同步更新组件及变量命名
2026-08-19 15:23:55 +08:00
hrz e1876f1ce1 Merge pull request #3315 from xinnan-tech/fix-model-name
更换豆包、Gemini的LLM模型、修复 performance_tester 异步配置加载错误
2026-08-18 16:31:48 +08:00
hrz ced7103639 Merge branch 'main' into fix-model-name 2026-08-18 16:31:21 +08:00
hrz cb0b075304 Merge pull request #3326 from MsThink/main
fix(知识库管理): 上传弹窗禁止点击遮罩关闭并添加提交加载状态
2026-08-18 16:26:46 +08:00
欣南科技 cdfb2a1187 Merge pull request #3331 from xinnan-tech/openrz-patch-1
Delete docs/images/console-params.png
2026-08-18 16:24:32 +08:00
hrz 8b64205522 Delete docs/images/console-params.png 2026-08-18 16:23:50 +08:00
hrz 6529b80c64 Merge pull request #3329 from Jehuty-ML/feat/params-management-group-ui
feat(manager-web): 参数管理按命名空间聚合展示
2026-08-18 16:10:52 +08:00
weirui d76825fae5 fix(manager-web): 参数管理聚合支持分页拉全量并重置空态全选
按 total 循环拉取全部页,避免超过 500 条时缺数据;displaySections 为空时同步重置 isAllSelected。
2026-08-18 10:19:39 +08:00
weirui c55dc8282f docs: 补充参数管理聚合界面截图,便于 PR 审阅对照 2026-08-17 18:56:27 +08:00
weirui 4870b5a0a2 feat(manager-web): 参数管理按命名空间聚合展示
一次拉取全量参数并按 paramCode 前缀分组折叠,避免默认分页难找相关配置。
2026-08-17 17:56:40 +08:00
wengzh ae3f970c92 Merge pull request #3322 from chentyke/fix/tts-timeout-config-propagation
fix(tts): 统一 TTS 请求超时配置
2026-08-17 17:41:45 +08:00
msgpo 8367f9b607 fix(知识库管理): 上传弹窗禁止点击遮罩关闭并添加提交加载状态 2026-08-15 15:02:04 +08:00
3030332422 59b51e021b update:暂时撤回对gemini模型的更替 2026-08-14 20:08:51 +08:00
3030332422 2c89b284b2 fix:修复Liquibase changeset ID误改的问题 2026-08-14 10:37:40 +08:00
Tyke Chen 04b3ad5aae fix(tts): validate timeout range 2026-08-13 15:59:41 +08:00
Tyke Chen 2c43a13d6e fix(tts): unify configurable request timeouts 2026-08-13 15:02:17 +08:00
Tyke Chen 17560a7d29 Merge pull request #3320 from henson957/fix/siliconflow-http-timeout
fix(tts): SiliconFlow provider 增加请求超时与 HTTP 错误处理
2026-08-13 14:25:06 +08:00
henson957 db7db3de67 fix(tts): reuse base tts_timeout config instead of hardcoded 15 2026-08-13 13:28:17 +08:00
henson957 d03d0d0982 fix(tts): add HTTP timeout and error handling to SiliconFlow provider
- requests.request() had no timeout: when the SiliconFlow API slows down,
  the request hangs forever and the device produces no audio intermittently
- non-200 responses (rate limit, error page) were written to disk as audio;
  now the status code is checked and an exception is raised with the body
2026-08-12 17:37:04 +08:00
Tyke Chen 545979873b Merge pull request #3318 from JansenChang/codex/fix-function-dialog-mcp-overlap
fix(manager-web): keep function dialog footer accessible with many MCP tools
2026-08-11 17:27:44 +08:00
ZhengYS 16b6841710 fix: keep function dialog footer accessible 2026-08-10 12:00:30 +08:00
3030332422 2a465f0fb5 fix: 修复 performance_tester 异步配置加载错误 2026-08-07 10:23:25 +08:00
3030332422 ea6b0c3746 update:豆包模型更换为doubao-seed-2-0-lite-260215,Gemini模型更换为gemini-3.6-flash 2026-08-07 09:09:53 +08:00
hrz 7475ea2b2d Merge pull request #3313 from xinnan-tech/fix-deepseek-model
deepseek更改模型为v4-flash并且关闭思考模式
2026-08-06 18:10:01 +08:00
欣南科技 1fa178cd36 Merge pull request #3314 from xinnan-tech/fix-star-img
fixed:Star History Chart
2026-08-06 18:08:59 +08:00
hrz 93885058e9 fixed:Star History Chart 2026-08-06 18:08:18 +08:00
3030332422 88eed86dc2 update:关闭deepseek的思考模式 2026-08-06 13:07:04 +08:00
3030332422 a269953787 fix:deepseek模型名称更改为deepseek-v4-flash 2026-08-06 11:10:31 +08:00
hrz 50895d6e33 Merge pull request #3310 from xinnan-tech/py-fix-pr_3304
展开模块级别插件名为具体函数名,合并 HA 设备控制插件为 hass_state 模块
2026-08-04 09:25:44 +08:00
3030332422 a21a2e34d0 refactor: 合并 HA 设备控制插件为 hass_state 模块 2026-07-29 09:58:26 +08:00
3030332422 7a22b61969 fix:修复与函数同名时遗漏其他函数的问题 2026-07-28 17:05:18 +08:00
CGD 477f85b97c Merge pull request #3304 from MsThink/main
fix(plugins): 展开模块级别插件名为具体函数名
2026-07-28 15:22:58 +08:00
CGD de45f73efd Merge pull request #3303 from chentyke/fix/frontend-contract-consistency
fix(manager): align frontend and backend field contracts
2026-07-28 09:37:39 +08:00
CGD 1dc46e30d7 Merge pull request #3302 from xinnan-tech/refactor/manager-api-tool-dependencies
refactor(manager-api): 升级常用工具依赖并收敛冗余实现
2026-07-28 09:28:00 +08:00
Sakura-RanChen 5e0d853256 Merge pull request #3309 from xinnan-tech/py-fix-reportHandle
refactor(reportHandle): 重构上报处理逻辑,适配多格式音频转换
2026-07-27 15:24:15 +08:00
wengzh 2a9f809700 refactor(reportHandle): 重构上报处理逻辑,适配多格式音频转换
1.  重命名参数与变量,明确上报类型与音频格式
2.  拆分PCM和Opus转WAV逻辑,分别处理不同输入格式
3.  移动校验逻辑到上报队列函数开头,提前拦截无效上报
4.  完善Opus转WAV的解码与资源释放流程
2026-07-27 09:59:32 +08:00
CGD f5ed1aaec8 Merge pull request #3305 from xinnan-tech/update_version
Bump to 0.9.6
2026-07-24 11:32:33 +08:00
colorful 523915f9fe fix(plugins): 展开模块级别插件名为具体函数名
在一个 function 文件中可能注册多个 @register_function,当配置下发或
本地配置使用模块名(文件名)时,需要自动展开为具体的函数名列表。

改动内容:
- register.py: 新增 module_func_map 全局映射,在 register_function
  装饰器中自动记录模块名 → 函数名列表的映射关系
- connection.py: 处理服务端下发的插件配置时,将模块级的 plugins 配置
  复制到各具体函数名 key 下,并将模块名展开写入 Intent functions 列表
- plugin_executor.py: 新增 _expand_plugin_names() 兜底展开逻辑和
  _get_plugin_description() 双层查找方法,确保本地配置场景也能正确展开
2026-07-24 10:47:00 +08:00
Tyke Chen 18e5c47a06 fix(manager): align frontend and backend field contracts 2026-07-24 10:21:05 +08:00
hrz 9da58e254c Bump to 0.9.6 2026-07-24 09:53:03 +08:00
Tyke Chen 4a2c90ec87 refactor(manager-api): 升级常用工具依赖并收敛冗余实现 2026-07-24 09:06:15 +08:00
CGD 27e57631a7 Merge pull request #3300 from xinnan-tech/test/manager-api-runtime-warnings
test(manager-api): 清理测试运行告警
2026-07-23 17:46:04 +08:00
CGD b1e39da9b8 Merge pull request #3301 from chentyke/fix/issue-3299-auto-update-state
fix: 修复设备自动升级开关状态不生效
2026-07-22 17:38:13 +08:00
Tyke Chen e87fc766d2 fix: restore device auto-update switch behavior 2026-07-22 17:10:21 +08:00
Tyke Chen 942d55118b test(manager-api): clean test runtime warnings 2026-07-22 16:41:44 +08:00
CGD f47f3b050b Merge pull request #3297 from xinnan-tech/fix/manager-api-compiler-warnings
fix(manager-api): 清理编译器告警
2026-07-22 16:22:46 +08:00
wengzh e3453b1a87 Merge pull request #3298 from xinnan-tech/py-fix-intent
Py fix intent
2026-07-21 15:16:19 +08:00
Tyke Chen d760465408 fix(manager-api): make JSON collections type-safe 2026-07-21 14:43:25 +08:00
Tyke Chen bdb2538239 fix(manager-api): clean deprecated and generic warnings 2026-07-21 14:43:25 +08:00
CGD ea6c144e43 Merge pull request #3294 from xinnan-tech/fix/manager-api-startup-warnings
fix(manager-api): 消除启动阶段告警
2026-07-21 14:37:22 +08:00
CGD a79aa455d6 Merge pull request #3292 from xinnan-tech/fix/manager-api-test-i18n
test: 修复 manager-api 国际化测试配置
2026-07-21 14:29:30 +08:00
wengzh 6e92d169ec refactor(get_news_from_newsnow): 改用流式处理解析网页内容
将原有的直接转换响应内容改为流式读取字节流并传入参数,适配MarkItDown的convert_stream接口,优化大内容处理时的内存占用
2026-07-20 15:51:23 +08:00
Sakura-RanChen ed89c05245 fix: intent阻塞主线程 2026-07-20 14:48:26 +08:00
Tyke Chen b500d1c6bd fix(manager-api): avoid duplicate address book insert mapping 2026-07-20 14:46:14 +08:00
Tyke Chen 0598b09629 fix(manager-api): remove invalid logback startup directives 2026-07-20 14:46:14 +08:00
Tyke Chen e4907121f4 fix(manager-api): eliminate premature bean initialization warnings 2026-07-20 14:46:14 +08:00
CGD 7c58fa37b2 Merge pull request #3293 from xinnan-tech/refactor/mybatis-plus-single-source
refactor(manager-api): consolidate MyBatis-Plus 3.5.17 integration
2026-07-20 14:42:31 +08:00
Tyke Chen 2a618d2f8f test: 修复 manager-api 国际化测试配置 2026-07-20 10:34:33 +08:00
Tyke Chen 10a67c772f refactor(manager-api): consolidate mybatis-plus 3.5.17 integration 2026-07-20 10:31:08 +08:00
CGD 7e8e5d34b8 Merge pull request #3279 from xinnan-tech/fix/issue-3276-context-provider
fix: 修复 ContextProviderListTypeHandler 多版本兼容问题
2026-07-17 09:28:14 +08:00
CGD c6bc20c05e Merge pull request #3283 from xinnan-tech/fix/device-timezone-consistency
fix: 修复跨时区场景下设备时间与在线状态异常
2026-07-16 16:58:49 +08:00
wengzh 1998807b69 Merge pull request #3285 from xinnan-tech/py-aec
perf: 优化相关逻辑减少重复计算
2026-07-14 16:36:16 +08:00
Sakura-RanChen 55c0495661 perf: 优化相关逻辑减少重复计算 2026-07-14 15:52:30 +08:00
Tyke Chen dbeda01697 fix: 修复设备跨时区时间与在线状态 2026-07-14 10:08:23 +08:00
Tyke Chen 023dea2441 fix: 修复上下文源类型处理器兼容问题 2026-07-13 16:03:39 +08:00
CGD ad3fb4d8c8 Merge pull request #3278 from xinnan-tech/agent-snapshot-history-fixes
fix: 完善智能体配置历史与恢复逻辑
2026-07-13 15:47:08 +08:00
3030332422 176785830a update:修改部分样式 2026-07-13 15:44:06 +08:00
Tyke Chen d78ecfce6f Merge remote-tracking branch 'origin/main' into agent-snapshot-history-fixes 2026-07-10 20:26:30 +08:00
Tyke Chen 179281e49c fix: harden agent snapshot restore flow 2026-07-10 20:21:44 +08:00
hrz bc1aa1a089 Merge pull request #3267 from xinnan-tech/fix-agent-idor
fix: prevent agent IDOR access
2026-07-10 17:36:00 +08:00
wengzh c5b64c3ef0 Merge pull request #3270 from xinnan-tech/fix-voiceprint-dialogue-bug
fix: 修改提示词与说话人注入逻辑,减少模型回答带着说话人信息频率
2026-07-10 15:55:58 +08:00
DaGou12138 2aa6c5ee49 fix:
1.优化context注入方式,合并为一个system,解决qwen本地部署模型多system异常400问题
2026-07-10 15:05:02 +08:00
Tyke Chen 687b6db96b fix: resolve mobile type and lint errors 2026-07-10 11:45:16 +08:00
Tyke Chen a5aee109fe feat: improve agent snapshot history 2026-07-10 11:44:35 +08:00
hrz 8ec5851027 Merge pull request #3273 from xinnan-tech/agent-delete-name-confirm
feat: 删除智能体前需要用户进行确认
2026-07-10 10:39:47 +08:00
Tyke Chen 48d629c79c fix: localize agent deletion success message 2026-07-10 10:18:48 +08:00
wengzh 14e57a1add fix(powermem): 修复用户配置启用状态的类型判断问题
将配置的启用值统一转为小写字符串后判断是否为true,避免非布尔类型配置导致的判断失效
2026-07-10 10:06:07 +08:00
hrz 634967b2fe Merge pull request #3274 from xinnan-tech/i18n-duplicate-check
test: 增加 i18n 重复 key 检测
2026-07-10 09:59:33 +08:00
Tyke Chen e648453c67 fix: rename i18n check command 2026-07-10 09:55:53 +08:00
Tyke Chen d11cfc6923 Merge remote-tracking branch 'origin/main' into fix-agent-idor
# Conflicts:
#	main/manager-api/src/main/java/xiaozhi/modules/agent/controller/AgentController.java
#	main/manager-api/src/main/java/xiaozhi/modules/agent/service/AgentService.java
#	main/manager-api/src/main/java/xiaozhi/modules/agent/service/impl/AgentServiceImpl.java
2026-07-09 20:21:37 +08:00
DaGou12138 32fbbf92fe Merge branch 'refs/heads/main' into fix-voiceprint-dialogue-bug 2026-07-09 16:56:22 +08:00
wengzh bd981aac79 Merge pull request #3275 from xinnan-tech/py-audio-aec
feat: 服务端AEC功能
2026-07-09 16:24:03 +08:00
Sakura-RanChen adee070bb0 Merge branch 'main' into py-audio-aec 2026-07-09 15:51:01 +08:00
wengzh 03ec3bac3f Merge pull request #3271 from xinnan-tech/py-fix-bug
Py fix bug
2026-07-09 15:36:59 +08:00
Tyke Chen af5fbb5282 test: add i18n duplicate key check 2026-07-09 15:25:36 +08:00
Tyke Chen 9301c1889f feat: require agent name confirmation before delete 2026-07-09 15:03:12 +08:00
hrz ef2ed07c1c Merge pull request #3266 from xinnan-tech/agent-snapshot-history
Add agent configuration history
2026-07-09 14:54:27 +08:00
Tyke Chen 1e2c4c6a28 fix: 修复智能体快照基线与查看逻辑 2026-07-09 11:16:07 +08:00
Sakura-RanChen a32500a6a4 fix: 消除死锁隐患 2026-07-09 10:29:41 +08:00
hrz 772af6c918 Merge branch 'agent-snapshot-history' of https://github.com/xinnan-tech/xiaozhi-esp32-server into agent-snapshot-history 2026-07-08 17:46:50 +08:00
hrz 2a917c1e62 update:对齐i18n文件行数 2026-07-08 17:44:10 +08:00
Tyke Chen c32fbe69c5 Remove wakeup word audio asset 2026-07-08 16:33:36 +08:00
DaGou12138 3b6e8f0e4b fix:
1.优化声纹称呼人提示词和注入逻辑,减少次次都称呼的频率
2026-07-08 16:14:10 +08:00
Tyke Chen aae2ef2152 fix: prevent agent IDOR access 2026-07-08 15:52:47 +08:00
Tyke Chen c82b351457 Fix agent snapshot history issues 2026-07-08 15:15:26 +08:00
DaGou12138 04c10ef225 fix:
1.修改提示词与说话人注入,减少模型每次回答都带着说话人信息
2026-07-08 14:12:16 +08:00
Sakura-RanChen 0b8bb728e2 同步方法进行包装避免阻塞 2026-07-08 11:47:04 +08:00
mie 35ec3ba803 Add agent configuration history 2026-07-07 17:49:34 +08:00
wengzh 8808b69303 Merge pull request #3265 from xinnan-tech/ha-muisc-timeout
Ha muisc timeout
2026-07-07 16:25:12 +08:00
Sakura-RanChen fc573a72ea fix: 修复设备时间戳上报逻辑 2026-07-07 15:50:27 +08:00
Sakura-RanChen 59f6566cfb 同步工具异步化处理 2026-07-07 14:45:50 +08:00
Sakura-RanChen d8c97a9f81 fix: 修复携程死锁 2026-07-06 16:44:41 +08:00
Sakura-RanChen 3d204fb181 feat: 服务端AEC功能
ref: 入口解码为pcm音频数据,vad和asr可直接使用
2026-07-06 16:21:24 +08:00
hrz 209fd3fbc8 Merge pull request #3260 from xinnan-tech/test-funasr
FunASR识别指定语言
2026-07-06 13:56:48 +08:00
3030332422 bc0ffd1bd3 update:修改sql文件,删去测试文件 2026-07-01 15:02:26 +08:00
hrz f0d61442b8 Merge pull request #3259 from xinnan-tech/update-theme
顶部下拉菜单图标间距调整
2026-06-30 18:45:14 +08:00
CGD e6c2fac7ec Merge pull request #3255 from LauraGPT/funasr-language-option
fix: allow configuring FunASR recognition language
2026-06-30 14:57:31 +08:00
zhifu gao 5a9b5af441 fix: add FunASR language config migration 2026-06-30 14:07:16 +08:00
欣南科技 16e5dc8c78 Merge pull request #3258 from xinnan-tech/update-version
update:Bump to 0.9.5
2026-06-30 13:10:18 +08:00
hrz f77970dd7d update:Bump to 0.9.5 2026-06-30 13:10:26 +08:00
zhuoqinglian af3ca4f609 update: 顶部下拉菜单图标间距调整 2026-06-30 09:47:28 +08:00
hrz 1ebfcd1f13 Merge pull request #3256 from xinnan-tech/update-theme
解决智能体管理查询不展示的问题&删除知识库旧版废弃文件
2026-06-30 09:36:42 +08:00
zhuoqinglian 3ca322961e fix: 解决智能体管理查询不展示的问题 2026-06-30 09:22:05 +08:00
zhuoqinglian 0eb0a600a7 update: 删除知识库旧版废弃文件 2026-06-30 09:19:26 +08:00
wengzh 3ed06a9e34 Merge pull request #3244 from xinnan-tech/base_prompt-update
feat: 将系统提示词指令逻辑英文化,中文样本保留
2026-06-30 09:06:10 +08:00
LauraGPT 2fae96d60c fix: allow configuring FunASR recognition language 2026-06-29 20:58:51 +00:00
wengzh 67dde7cc47 Merge pull request #3254 from xinnan-tech/update-theme
智能体管理搜索恢复查询历史功能
2026-06-29 16:58:52 +08:00
zhuoqinglian efedc18f02 update: 智能体管理搜索恢复查询历史功能 2026-06-29 16:01:14 +08:00
zhuoqinglian 0270de1844 update: 滚动条样式抽离复用 2026-06-29 15:42:52 +08:00
wengzh 06b0ff139b Merge pull request #3247 from xinnan-tech/update-theme
Update theme
2026-06-26 17:58:18 +08:00
zhuoqinglian 799ea28d82 Merge branch 'main' into update-theme 2026-06-26 17:30:15 +08:00
zhuoqinglian 5a68252d22 update: 更新所有页面背景颜色 2026-06-26 17:28:06 +08:00
zhuoqinglian 97404e53fe update: 顶部导航栏搜索放到首页 2026-06-26 17:26:16 +08:00
zhuoqinglian b052158cee feat: 完成导航菜单组件样式优化 2026-06-26 17:10:49 +08:00
zhuoqinglian 979ef24b79 update: 音色克隆多语言配置 2026-06-26 15:32:48 +08:00
zhuoqinglian 0f4f0f01fd update: 音色克隆页面展现形式修改 2026-06-26 14:45:23 +08:00
wengzh 18e1de4f45 Merge pull request #3245 from xinnan-tech/py-add-minimax-voice
Py add minimax voice
2026-06-26 11:42:59 +08:00
Sakura-RanChen aa846a096f 名称长度调整 2026-06-26 11:41:46 +08:00
Sakura-RanChen a679412230 增加试听音色 2026-06-26 11:40:52 +08:00
3030332422 cf3e14bb07 feat: 将系统提示词指令逻辑英文化,中文样本保留 2026-06-26 11:06:18 +08:00
wengzh 3a73a17253 Merge pull request #3243 from xinnan-tech/py-add-minnimax-voice
调整minimax音色
2026-06-26 10:50:26 +08:00
Sakura-RanChen 3b27ebcd35 调整minimax音色 2026-06-26 10:29:07 +08:00
wengzh fa6074d284 Merge pull request #3242 from xinnan-tech/update-theme
Update theme
2026-06-25 17:29:55 +08:00
zhuoqinglian d809cfcb01 Merge branch 'main' into update-theme 2026-06-25 17:27:40 +08:00
zhuoqinglian eb2c09bbdc feat: 完成聊天记录、修改密码页面样式改造 2026-06-25 17:26:52 +08:00
zhuoqinglian 2cf8f79482 update: 模型配置新增、编辑弹窗样式优化 2026-06-25 17:09:29 +08:00
zhuoqinglian 1c94844bb9 feat: 完成声纹管理页面样式改造 2026-06-25 09:18:38 +08:00
wengzh 0c53b8f543 Merge pull request #3240 from xinnan-tech/update-theme
Update theme
2026-06-24 17:58:13 +08:00
zhuoqinglian 1518dfca30 feat: 完成设备管理页面样式改造 2026-06-24 17:37:37 +08:00
zhuoqinglian 7529c500a4 feat: 完成音色克隆页面样式改造 2026-06-24 11:43:36 +08:00
zhuoqinglian b6ed220e48 feat: 完成音色资源页面样式改造 2026-06-23 17:23:11 +08:00
wengzh b152c5dc23 Merge pull request #3239 from xinnan-tech/update-theme
模型配置页面样式调整
2026-06-23 16:31:30 +08:00
zhuoqinglian 04bfddf457 update: 按钮样式风格统一 2026-06-23 14:54:45 +08:00
zhuoqinglian 9d78ea29c3 Merge branch 'main' into update-theme 2026-06-23 14:15:57 +08:00
zhuoqinglian f82f364189 feat: 完成模型配置页面样式改造 2026-06-23 14:15:32 +08:00
zhuoqinglian 9705149b2b feat: 按钮组件新增add类型 2026-06-23 14:14:29 +08:00
wengzh 7986148013 Merge pull request #3238 from xinnan-tech/py-add-edge-config
补充edge相关配置
2026-06-23 10:40:40 +08:00
Sakura-RanChen 19f5cb5b4f 补充edge相关配置 2026-06-23 10:37:48 +08:00
zhuoqinglian ea48b948bf update: 模型配置新增弹窗样式调整 2026-06-22 17:58:35 +08:00
zhuoqinglian 3c5ca9f143 update: 模型配置编辑弹窗样式调整 2026-06-22 17:31:09 +08:00
wengzh da72ecaa1b Merge pull request #3235 from xinnan-tech/py-fix-tts
fix: 超时情况不与activate_session同一判断条件导致的监听任务未清除问题
2026-06-22 11:21:03 +08:00
Sakura-RanChen f297e1b648 fix: 超时情况不与activate_session同一判断条件导致的监听任务未清除问题 2026-06-22 11:16:25 +08:00
wengzh a1973e07b7 Merge pull request #3227 from xinnan-tech/update-theme
feat: 完成字典管理页面样式改造
2026-06-18 16:57:49 +08:00
zhuoqinglian cd58b2c3cb Merge branch 'main' into update-theme 2026-06-18 16:55:33 +08:00
zhuoqinglian a5be67d04b feat: 完成字典管理页面样式改造 2026-06-18 16:49:15 +08:00
wengzh e6dc6187c8 Merge pull request #3226 from xinnan-tech/update-theme
页面样式改造
2026-06-18 16:39:41 +08:00
zhuoqinglian 7f4011f9de feat: 完成字段管理页面样式改造 2026-06-18 16:37:42 +08:00
zhuoqinglian 211cf891af Merge branch 'main' into update-theme 2026-06-18 10:26:57 +08:00
zhuoqinglian 89a1229b53 feat: 完成服务端管理页面样式改造 2026-06-18 10:26:06 +08:00
wengzh 071ed99a32 Merge pull request #3224 from xinnan-tech/py-fix-bug
fix: 通讯录页面时间不显示
2026-06-17 14:36:48 +08:00
Sakura-RanChen ea14be27e6 fix: 通讯录页面时间不显示
fix: listen消息相关状态处理
2026-06-17 14:34:42 +08:00
zhuoqinglian a834f163b7 feat: 完成替换词页面样式改造 2026-06-17 14:20:48 +08:00
zhuoqinglian 7849e6b030 update: 模板保存添加必填校验 2026-06-17 10:36:02 +08:00
wengzh 0d869baaa0 Merge pull request #3223 from xinnan-tech/update-theme
页面样式调整
2026-06-16 17:01:10 +08:00
zhuoqinglian 164eb5481c feat: 完成默认角色管理页面样式改造,并添加编辑保存弹窗 2026-06-16 16:58:44 +08:00
zhuoqinglian 969ee3cb12 feat: 完成字段管理页面样式改造 2026-06-16 16:18:12 +08:00
zhuoqinglian ee54d87383 feat: 完成固件管理页面样式改造 2026-06-16 14:55:08 +08:00
wengzh 849026c015 Merge pull request #3222 from xinnan-tech/update-theme
Update theme
2026-06-16 10:42:24 +08:00
zhuoqinglian e5bb3e7791 update: 优化表格组件,增加代码的复用性 2026-06-16 10:40:44 +08:00
zhuoqinglian 1ffc40e71d feat: 完成用户管理页面样式改造 2026-06-16 10:39:52 +08:00
zhuoqinglian 45fd850757 update: 表格组件添加阴影效果 2026-06-16 09:46:25 +08:00
zhuoqinglian 8a5dac181e update: 知识库顶部标题合并 2026-06-15 17:37:53 +08:00
zhuoqinglian ecf20300ff update: 通讯录初始化加载默认选中第一项 2026-06-15 17:16:50 +08:00
zhuoqinglian 74def844f0 Merge branch 'main' into update-theme 2026-06-15 17:05:28 +08:00
zhuoqinglian a5a9bd1abe fix: 修复element组件多语言切换不生效的问题 2026-06-15 17:05:04 +08:00
zhuoqinglian 2d9b915d51 feat: 完成参数管理页面改造 2026-06-15 15:44:09 +08:00
hrz 5f87fc8361 Merge pull request #3204 from xinnan-tech/py-fix-calling
优化相关逻辑
2026-06-15 14:06:51 +08:00
zhuoqinglian 6859fdc90e update: 系统功能配置图标大小优化 2026-06-15 11:27:12 +08:00
zhuoqinglian b8ae0568e9 update: 系统功能配置页面适配调整 2026-06-15 11:23:31 +08:00
wengzh 1dc6f143f3 Merge pull request #3219 from xinnan-tech/update-theme
完成系统功能页面样式改造
2026-06-12 14:43:23 +08:00
zhuoqinglian 67532dc230 update: 系统功能配置完善多语言 2026-06-12 14:23:25 +08:00
zhuoqinglian b2876def99 Merge branch 'main' into update-theme 2026-06-12 14:14:59 +08:00
zhuoqinglian 0c73a8b6a4 feat: 完成系统功能页面样式改造 2026-06-12 14:13:21 +08:00
zhuoqinglian c857a3164d feat: 完成按钮组件抽离 2026-06-12 11:49:22 +08:00
hrz a964192eb0 Merge pull request #3213 from xinnan-tech/fix-knowledge-base-ragflow-bug
#3212 修复知识库RAGFlow相关逻辑BUG
2026-06-12 11:15:35 +08:00
DaGou12138 32da03818e #3212 fix:
1.修改知识库前端卡片提示样式,优化警示效果
2026-06-12 10:35:40 +08:00
DaGou12138 7f17e3e08e Merge branch 'refs/heads/main' into fix-knowledge-base-ragflow-bug 2026-06-12 10:29:17 +08:00
DaGou12138 c237e1f323 Merge remote-tracking branch 'origin/fix-knowledge-base-ragflow-bug' into fix-knowledge-base-ragflow-bug 2026-06-12 09:51:03 +08:00
DaGou12138 087a202e31 #3212 fix:
1.优化知识库连接RAGFlow异常提示,修改前端卡片提示样式
2026-06-12 09:50:01 +08:00
hrz 4b1460d938 update:调整同步数据集错误日志的级别 2026-06-11 19:10:21 +08:00
hrz f2aec524d2 del:无用类 2026-06-11 18:57:30 +08:00
wengzh 1a97ca137c Merge pull request #3215 from xinnan-tech/update-theme
Update theme
2026-06-11 17:45:58 +08:00
zhuoqinglian 2b6f457ae6 update: 知识库页面间距优化 2026-06-11 17:44:17 +08:00
DaGou12138 3f4e0b4aa0 #3212 fix:
1.修复RAGFlow连接异常导致级联删除问题
2.修复RAGFlow获取数量逻辑问题
3.远程失败不再删除本地影子记录,避免产生RAGFlow端孤儿文档
4.优化前端提示语
2026-06-11 17:17:59 +08:00
DaGou12138 e30ded4de9 fix:删除知识库rag相关非必要实体 2026-06-11 16:08:17 +08:00
hrz 07242a84ff Merge pull request #3210 from xinnan-tech/fix-knowledge-base-web-bug
fix:修复知识库模块弹框组件样式全局覆盖导致显示异常问题
2026-06-10 15:32:08 +08:00
DaGou12138 a2074dc0d8 fix:修复知识库模块弹框组件样式全局覆盖导致显示异常问题 2026-06-10 15:00:29 +08:00
zhuoqinglian 28b6c85650 fix: 修复弹窗展示位置异常的问题 2026-06-10 10:47:52 +08:00
wengzh af300138eb Merge pull request #3209 from xinnan-tech/update-theme
Update theme
2026-06-09 17:24:16 +08:00
zhuoqinglian 2bf5725048 update: 知识库页面复用弹窗、分页组件,同步修改样式 2026-06-09 17:22:18 +08:00
hrz 3d73141788 Merge pull request #3206 from xinnan-tech/web_search-Document
docs:新增联网搜索插件配置文档
2026-06-09 17:12:16 +08:00
hrz 945ab29bc3 add:增加联网搜索的教程和自编译docker镜像的教程 2026-06-09 17:11:35 +08:00
zhuoqinglian fe0f7dcf75 feat: 完成分页组件抽离 2026-06-09 15:05:16 +08:00
zhuoqinglian 1a3ffbc8e2 feat: 完成弹窗组件抽离 2026-06-09 14:25:49 +08:00
3030332422 71278b1898 docs:新增联网搜索插件配置文档 2026-06-08 11:38:11 +08:00
zhuoqinglian 0173c7abe1 update: 知识库样式优化、添加多语言配置 2026-06-05 17:37:21 +08:00
zhuoqinglian 8a50eea4ce update: 知识库整体页面样式调整 2026-06-03 16:49:08 +08:00
Sakura-RanChen b9891bb795 优化相关逻辑 2026-06-03 16:13:03 +08:00
wengzh b903a88cd5 Merge pull request #3203 from xinnan-tech/mac-address-mask
feat: mac地址添加马赛克功能
2026-06-03 14:41:34 +08:00
zhuoqinglian f3fcf2fedf feat: mac地址添加马赛克功能 2026-06-03 14:40:01 +08:00
欣南科技 a825f0874c Merge pull request #3202 from xinnan-tech/update-doc-video
update:更新演示视频
2026-06-03 13:58:14 +08:00
hrz 8a5ca46418 update:更新演示视频 2026-06-03 13:56:34 +08:00
欣南科技 5fcfc529c0 Merge pull request #3201 from xinnan-tech/0.9.4
Bump to 0.9.4
2026-06-03 09:08:43 +08:00
hrz 370d148a4e Bump to 0.9.4 2026-06-03 09:07:39 +08:00
欣南科技 3088ddce38 Merge pull request #3200 from xinnan-tech/update_doc
update:补充设备间通话开启备注
2026-06-02 18:23:49 +08:00
hrz 8e3faf20d3 update:补充设备间通话开启备注 2026-06-02 18:22:31 +08:00
欣南科技 f42088eff4 Merge pull request #3199 from xinnan-tech/py-device-calling
update:如果通讯权限 alias 为空,就用默认使用设备名称
2026-06-01 23:53:29 +08:00
hrz a0181cc723 update:如果通讯权限 alias 为空,就用默认使用设备名称 2026-06-01 23:52:38 +08:00
欣南科技 941aaf10ce Merge pull request #3198 from xinnan-tech/py-device-calling
fix:用户别名缺失bug
2026-06-01 22:57:24 +08:00
hrz 235ec8a25d update:更换通讯录图标 2026-06-01 22:55:03 +08:00
hrz effe95ba8f fix:用户别名缺失bug 2026-06-01 22:54:34 +08:00
CGD 4c4ba0aa08 Merge pull request #3196 from xinnan-tech/py-device-calling
Py device calling
2026-06-01 16:34:02 +08:00
3030332422 7af4b41881 update:设备呼叫工具的接听触发提示词约束 2026-06-01 16:25:52 +08:00
hrz cfcf68d4ce Merge branch 'py-device-calling' of https://github.com/xinnan-tech/xiaozhi-esp32-server into py-device-calling 2026-05-31 10:30:54 +08:00
hrz 7f2675f9df update:增加设备呼叫使用条件 2026-05-31 10:30:10 +08:00
Sakura-RanChen 69fd5a1ea8 不开启通讯录功能时不显示相关工具
关闭通讯录功能清除相关状态
2026-05-29 17:34:21 +08:00
Sakura-RanChen 4392edf27d 调整相关说明文档 2026-05-29 15:56:52 +08:00
zhuoqinglian e4ea9c8eff Merge branch 'style-device-calling' into py-device-calling 2026-05-29 14:37:55 +08:00
zhuoqinglian 2adf51ddf9 fix: 修复顶部导航栏遮挡的问题 2026-05-29 14:36:14 +08:00
CGD 3d1b133162 Merge pull request #3195 from xinnan-tech/fix-timezone-unify
fix:修复时区导致的设备状态不同步的问题
2026-05-29 10:29:02 +08:00
DaGou12138 f539d46de3 Merge pull request #3194 from xinnan-tech/fix(model-config)
fix(model-config): 修复合并JSON时的类型异常并添加日志
2026-05-29 10:10:21 +08:00
baigao 1701bfbfca fix(model-config): 修复合并JSON时的类型异常并添加日志
1. 为mergeJson方法添加空值检查和类型校验逻辑
2. 增加日志打印便于排查问题
3. 优化代码换行和注释格式
4. 修复原有代码中潜在的类型转换异常问题
2026-05-29 10:07:11 +08:00
zhuoqinglian bb48e7efab Merge branch 'style-device-calling' into py-device-calling 2026-05-29 10:05:24 +08:00
zhuoqinglian 9ea67d4306 update: 通讯录页面样式调整 2026-05-29 10:04:54 +08:00
rainv123 3a0ce6998e fix:修复设备型号不显示的问题 2026-05-29 09:50:09 +08:00
hrz 0678521b16 update:优化addressBook配置的的存储方式 2026-05-28 19:36:26 +08:00
hrz 84187d4e68 update:优化翻译 2026-05-28 15:18:54 +08:00
hrz 38958b2d06 add:通讯录i18n行数对齐 2026-05-28 15:02:20 +08:00
hrz 1e17e5ac8a fixed:当数据库的就数据不支持新特性时,增加新特性 2026-05-28 14:41:06 +08:00
rainv123 ec5063f050 fix: 设备最后连接时间返回 UTC 时间戳,修复时区显示问题 2026-05-27 17:35:10 +08:00
Sakura-RanChen d03583f493 增加工具配置说明 2026-05-27 17:28:58 +08:00
Sakura-RanChen 4a55662e91 fix: 修复智能体在线状态获取
补充相关国际化
2026-05-27 16:57:48 +08:00
Sakura-RanChen de78e87c77 调整逻辑使用java端作为中转发送 2026-05-27 15:34:17 +08:00
wengzh ff011d3c44 Merge pull request #3192 from xinnan-tech/fix_rag_page_size
fix:修复知识库文档切片的分页显示问题
2026-05-27 10:30:13 +08:00
rainv123 a90229a22a fix:修复知识库文档切片的分页显示问题 2026-05-27 10:26:50 +08:00
wengzh 559d47a229 Merge pull request #3189 from xinnan-tech/digital-human_prompt
update:根据数字人 emoji 开关动态渲染系统提示词
2026-05-26 16:38:46 +08:00
Sakura-RanChen ce68c83eea feat: 新增设备互相通讯功能
feat: 新增通讯录页面
2026-05-26 16:37:39 +08:00
3030332422 7e63b7e852 update:根据数字人 emoji 开关动态渲染系统提示词 2026-05-26 11:43:18 +08:00
wengzh 7f73dae1bf Merge pull request #3183 from xinnan-tech/digital-human_fix_emoji
fix:情绪表情默认改为false
2026-05-20 17:49:02 +08:00
3030332422 48e8839a1e fix:情绪表情默认改为false 2026-05-20 17:43:55 +08:00
wengzh 73f20ed9b0 Merge pull request #3182 from xinnan-tech/perf-tool-add-online-search
fix:修复deepseek模型下工具名重复导致错误问题
2026-05-20 15:03:26 +08:00
DaGou12138 48a2bd5a5a fix:修复deepseek模型下工具名重复导致错误问题 2026-05-20 14:52:21 +08:00
wengzh fb7e2e17a8 Merge pull request #3176 from xinnan-tech/perf-tool-add-online-search
Perf:新增联网搜索工具类、修复新闻描述配置获取源问题
2026-05-14 10:51:23 +08:00
欣南科技 bcce6a4ebe Merge pull request #3174 from xinnan-tech/faq-digital-human
add:faq增加数字人相关链接
2026-05-13 17:24:01 +08:00
hrz 7ef86eb3c6 add:faq增加数字人相关链接 2026-05-13 17:20:21 +08:00
wengzh 789b207a94 Merge pull request #3173 from xinnan-tech/chore-move-multilingual-readmes-to-docs
chore:迁移多语言 README 到 docs/readme
2026-05-12 15:20:06 +08:00
DaGou12138 db14d08c7b perf:
1、新增联网搜索工具类(web_search)
2、修复新闻描述配置获取源问题,改为从连接配置获取
3、新增web_search所需配置项数据集
2026-05-12 15:16:31 +08:00
DaGou12138 46e7f8508d perf:
1、新增联网搜索工具类(web_search)
2、修复新闻描述配置获取源问题,改为从连接配置获取
3、新增web_search所需配置项数据集
2026-05-12 15:16:08 +08:00
3030332422 f3583742c7 chore:迁移多语言 README 到 docs/readme 2026-05-12 15:15:08 +08:00
wengzh 27de4c40ea Merge pull request #3111 from xinnan-tech/py-test-wake
test测试页面唤醒功能
2026-05-12 11:00:58 +08:00
3030332422 98ba86d4a6 update:对文档的补充说明 2026-05-12 10:49:02 +08:00
3030332422 2aa1a8d590 update:补充 2026-05-12 10:24:19 +08:00
3030332422 a2c3da55cd update:关于对digital-human的markdown文档的修改 2026-05-12 09:45:43 +08:00
3030332422 41ad57ad4f update:test迁移重命名为digital-human 2026-05-12 09:27:39 +08:00
hrz 6e1ed9fa4d update:简化说明;默认开启唤醒词 2026-05-10 20:26:40 +08:00
wengzh 1b5c00231a Merge pull request #3165 from xinnan-tech/fix-chat-total
fix: 修复聊天记录左侧会话数量统计
2026-05-09 15:56:05 +08:00
zhuoqinglian 95acba1126 fix: 修复聊天记录左侧会话数量统计 2026-05-09 15:54:53 +08:00
wengzh 1e6f0fd1be Merge pull request #3162 from xinnan-tech/perf-tool-call-optimization
fix:修复音乐播放工具TTS上报记录问题
2026-05-08 16:40:37 +08:00
DaGou12138 eb571fae57 fix:修复音乐播放工具TTS上报记录问题 2026-05-08 16:38:23 +08:00
wengzh 2946be7284 Merge pull request #3161 from xinnan-tech/perf-tool-call-optimization
fix:优化天气工具调用提示词
2026-05-08 14:23:11 +08:00
DaGou12138 425cb1d6bd fix:优化天气工具调用提示词,本地上下文存在当前所在地的天气信息无需再度查询工具 2026-05-08 14:20:52 +08:00
wengzh cf9b0ec712 Merge pull request #3160 from xinnan-tech/fix-chat-title
fix: 修复聊天记录标题过长展示异常的问题
2026-05-08 11:08:23 +08:00
zhuoqinglian 1b053dc9c8 fix: 修复聊天记录标题过长展示异常的问题 2026-05-08 11:05:20 +08:00
wengzh d7ad656bd0 Merge pull request #3159 from xinnan-tech/perf-tool-call-optimization
fix:优化对话过程中JSON闭合符号清理的问题
2026-05-08 10:48:58 +08:00
DaGou12138 7dae67a57b fix:优化对话过程中JSON闭合符号清理的问题 2026-05-08 10:46:37 +08:00
wengzh 7b0f59a747 Merge pull request #3158 from xinnan-tech/chore/cleanup
chore: 暂时移除近期不可用或不稳定的TTS供应器及相关配置
2026-05-08 10:38:53 +08:00
DaGou12138 2eaa5bbb11 fix:优化对话过程中异常符号记录的问题 2026-05-08 10:29:50 +08:00
FAN-yeB c693ff1564 update:linkerai 近期多次测试,linkerai接口无法提供稳定服务,先暂时下架这个供应器 2026-05-08 10:18:26 +08:00
DaGou12138 e9bf006d9d Merge pull request #3154 from haixinke/Bug-get-user-prifile-null
fix:当打开用户画像开关时,修复获取用户画像为空的问题
2026-05-07 16:01:57 +08:00
wengzh bef96f36e2 Merge pull request #3157 from xinnan-tech/fix-streaming-tts-original-text
Fix streaming tts original text
2026-05-07 15:36:56 +08:00
wengzh bebb7eac69 Merge pull request #3156 from xinnan-tech/perf-tool-call-optimization
perf:优化新闻工具新闻源配置获取的逻辑
2026-05-07 11:43:17 +08:00
DaGou12138 a787130b74 perf:
1、优化新闻工具新闻源获取的逻辑
2、调整新闻工具描述内容
2026-05-07 11:41:17 +08:00
FAN-yeB 8b7f16b9c2 update:TTS_GizwitsTTS 近期机智云语音合成不成功,先暂时下架这个供应器 2026-05-07 11:27:32 +08:00
FAN-yeB 9ed66ec013 update:TTS_ACGNTTS 海豚配音截止5月7日未找到token入口,先暂时下架这个供应器 2026-05-07 10:54:03 +08:00
FAN-yeB 05c22cdf2a update:更新 .gitignore忽略.claude目录 2026-05-07 10:23:02 +08:00
FAN-yeB 4bc0b67cd7 update:去除无用的.claude目录 2026-05-07 10:20:48 +08:00
Sakura-RanChen 0a6f811448 fix: 修复流式情况下替换词失败 2026-05-07 10:20:10 +08:00
wengzh 5aeb55fffd Merge pull request #3155 from xinnan-tech/perf-tool-call-optimization
perf:优化工具调用问题、调整后端智能体搜索条件逻辑
2026-05-06 17:01:15 +08:00
DaGou12138 c33ee9442e perf:
1、优化小智后端智能体搜索条件逻辑
2、优化工具调用准确率问题,修改工具提示词,去除非必要的few-shot逻辑
3、优化工具描述内容
2026-05-06 16:53:06 +08:00
minwang 4670ad2925 delete log 2026-05-01 16:03:02 +08:00
minwangandClaude Sonnet 4.6 b542388053 🐛 修复PowerMem用户画像查询返回空的问题
修复 get_user_profile() 方法在缓存未命中时主动从 SDK 获取用户画
像,支持 profile_content 和 topics 两种格式,解决新对话中用户画像无法显示的问题

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 14:57:36 +08:00
3030332422 ba68c3d423 update:修改文档示例密码 2026-04-30 19:54:09 +08:00
3030332422 c30cd5ca79 update:添加“一体机数字人”配置说明 2026-04-30 18:04:44 +08:00
rainv123 a71ce53a68 fix:修复使用替换词时,火山引擎双向流式tts的文本显示/上报问题 2026-04-30 17:33:40 +08:00
rainv123 dd88107131 Revert "fix: 修复流式TTS替换词显示/上报问题"
This reverts commit 8b847a3826.
2026-04-30 15:57:03 +08:00
rainv123 8b847a3826 fix: 修复流式TTS替换词显示/上报问题 2026-04-30 14:32:36 +08:00
wengzh 036b89f24e Merge pull request #3151 from xinnan-tech/py-refactor-asr-tts
调整配置
2026-04-30 11:42:36 +08:00
Sakura-RanChen 354280d430 调整配置 2026-04-30 11:41:16 +08:00
hrz e2d16ec8d4 Merge pull request #3150 from Mirrorium227/main
Closes #Issue3149 修复文档序号
2026-04-30 11:27:15 +08:00
Radar Song 6a0192de27 Closes #Issue3149 修复文档序号 2026-04-29 20:18:17 +08:00
hrz ac6eeb5c1f Merge pull request #3145 from haixinke/fix-bug_prompt_cache_key_wrong
Fix:【设备快速提示词缓存key错误】
2026-04-29 14:32:42 +08:00
minwang 23e213f6ea 修复设备快速提示词缓存key错误 2026-04-28 15:52:23 +08:00
wengzh 382c948482 Merge pull request #3144 from xinnan-tech/fix-memory-config
fix:修复记忆模型配置保存字段未删除问题
2026-04-28 15:25:49 +08:00
DaGou12138 a67d938523 fix:修复记忆模型配置保存字段未删除问题 2026-04-28 15:23:57 +08:00
wengzh 26aca54f4b Merge pull request #3141 from xinnan-tech/add-correct-word
fix修复 asyncio.gather 吞掉业务异常的问题
2026-04-27 17:31:47 +08:00
rainv123 4717b8beac fix修复 asyncio.gather 吞掉业务异常的问题 2026-04-27 17:29:41 +08:00
wengzh cd580b6f81 Merge pull request #3139 from xinnan-tech/py-fix-mem0ai
fix: 修复Mem0AI记忆模式工具调用时保存记忆400错误
2026-04-27 16:57:26 +08:00
3030332422 70740af7a6 fix: 修复Mem0AI记忆模式工具调用时保存记忆400错误 2026-04-27 16:25:05 +08:00
wengzh 7b30c25923 Merge pull request #3132 from xinnan-tech/replacement-word
feat: 新增替换词管理功能
2026-04-27 16:23:54 +08:00
wengzh 7db13b8be2 Merge pull request #3130 from xinnan-tech/add-correct-word
add:替换词功能
2026-04-27 16:23:33 +08:00
wengzh a0df41ac24 Merge pull request #3136 from xinnan-tech/performance_tester
update:解决base类修改后测速工具兼容性问题
2026-04-27 15:08:26 +08:00
FAN-yeB a432c295d3 update:解决base类修改后测速工具兼容性问题 2026-04-27 15:06:40 +08:00
wengzh ab8a35e233 Merge pull request #3135 from xinnan-tech/perf-tool-call-optimization
Perf:优化工具调用准确性、禁用标题总结与记忆模型思考模式、拆分动态上下文
2026-04-27 14:54:16 +08:00
DaGou12138 160b67197f Merge branch 'refs/heads/main' into perf-tool-call-optimization 2026-04-27 14:23:28 +08:00
欣南科技 3510282cd5 Merge pull request #3134 from xinnan-tech/0.9.3
Bump to 0.9.3
2026-04-27 12:46:10 +08:00
hrz bdf7c7facf Bump to 0.9.3 2026-04-27 12:44:02 +08:00
DaGou12138 151a0c1b99 perf:
1、优化工具调用相关提示词,拆分静态system-prompt和动态上下文。
2、新增few-shot示例,使用"调用→结果→回复"流程闭环,显著提高多次会话后工具调用准确性。
3、去除对话标题总结和记忆的模型思考模式。
2026-04-27 11:08:25 +08:00
zhuoqinglian 52af722463 update: 更新替换词数量限制 2026-04-24 14:54:37 +08:00
wengzh fa91c97685 Merge pull request #3131 from xinnan-tech/fix-wifi-no-response
fix: 修复设备配网成功没有自动退出配网模式
2026-04-24 14:38:06 +08:00
zhuoqinglian a92d6ccdfb fix: 修复设备配网成功没有自动退出配网模式 2026-04-24 14:35:34 +08:00
zhuoqinglian f5ce755fea Merge pull request #3116 from wujingquan/main
Add WiFi password translations for multiple languages
2026-04-24 14:15:27 +08:00
rainv123 606a42cfb2 update:替换词功能 2026-04-24 10:06:26 +08:00
zhuoqinglian f85e863144 feat: 新增替换词管理功能 2026-04-23 17:50:57 +08:00
wengzh 550397de98 Merge pull request #3127 from xinnan-tech/py-fix-asr
Py fix asr
2026-04-23 10:46:09 +08:00
Sakura-RanChen ff66b815a5 fix: 在模型和工具调用期间进行打断后说完话错误退出问题 2026-04-23 10:17:43 +08:00
Sakura-RanChen 8c18e48b97 fix: 停止帧发送后与原先音频时序问题
回退退出工具打断处理(在大模型思考期间进行打断,然后大模型立马调用退出工具会造成死锁,因模型思考时间和是否调用工具不确定性,状态难管理)
2026-04-22 15:55:28 +08:00
3030332422 44587985dd fix: 优化唤醒词音频采集 2026-04-22 11:42:22 +08:00
wengzh 4acd02e2b0 Merge pull request #3125 from xinnan-tech/py-fix-llm-chunk
fix: 修复频繁打断下生成器不会清理
2026-04-22 11:30:35 +08:00
wengzh ddfbacc717 Merge pull request #3124 from xinnan-tech/py-add-ASRv2
使用双流优化版接口(速度提升),增加豆包流式语音识别模型2.0模型选择
2026-04-22 11:29:57 +08:00
Sakura-RanChen 2a4dbdb69b fix: 修复频繁打断下生成器不会清理 2026-04-22 11:24:43 +08:00
FAN-yeB e9a2489e48 update:测试工具更新豆包流式测试逻辑 2026-04-22 10:53:48 +08:00
3030332422 db73faeec4 fix: 优化唤醒词音频采集,使用队列解耦回调与推理处理 2026-04-22 10:35:48 +08:00
Sakura-RanChen ce13a7e3e6 使用双流优化版接口(速度提升),增加豆包流式语音识别模型2.0模型选择 2026-04-22 10:28:47 +08:00
wengzh 8a156e99f4 Merge pull request #3122 from xinnan-tech/py-add-TTS2.0
增加豆包语音合成模型2.0专属供应器
2026-04-21 15:44:04 +08:00
Sakura-RanChen 4313216816 增加豆包语音合成模型2.0专属供应器 2026-04-21 15:33:37 +08:00
DaGou12138 8c0899415a Merge branch 'refs/heads/main' into perf-tool-call-optimization 2026-04-21 15:02:31 +08:00
DaGou12138 3736a5845a perf:
1、优化提示词context动态变化导致的缓存失效问题,拆分静态system-prompt和动态上下文
2026-04-21 15:01:57 +08:00
wengzh dc7615c008 Merge pull request #3120 from xinnan-tech/update-theme-generator
update: 更新设备主题生成功能,添加新表情
2026-04-21 14:50:53 +08:00
wengzh b03f9374ea Merge pull request #3119 from xinnan-tech/fix-config-slm
feat: agent-models接口返回SLM模型,LLM类型同步返回所选SLM
2026-04-21 14:18:07 +08:00
欣南科技 cacfc9bfee Merge pull request #3121 from xinnan-tech/enhanced-prompts
update:优化提示词
2026-04-21 14:01:49 +08:00
hrz ce3c8a2bd8 update:优化提示词 2026-04-21 14:00:25 +08:00
zhuoqinglian e1a7e50e1f update: 更新设备主题生成功能,添加新表情 2026-04-21 11:44:03 +08:00
DaGou12138 90888de0ba perf:
1、新增工具调用Action枚举RECORD,通过完整的三段式 assistant(tool_calls) → tool → assistant(response)让模型准确学习到工具调用的正确行为,优化play_music的工具调用枚举NONE变更为RECORD,优化解决音乐播放工具调用缺陷问题。
2、处理悬空 tool_calls问题。
3、去除工具提示词强化注入逻辑。
2026-04-21 11:36:06 +08:00
3030332422 96f0412f94 update:测试页面支持关闭情绪表情功能 2026-04-21 11:15:31 +08:00
rainv123 ad4d32a05d feat: agent-models接口返回SLM模型,LLM类型同步返回所选SLM 2026-04-21 11:12:18 +08:00
hrz 08d6bb4056 Merge pull request #3118 from xinnan-tech/openai_nothinking
feat:默认禁用部分平台LLM思考模式
2026-04-20 17:39:20 +08:00
FAN-yeB 1f0db7dc47 feat:默认禁用部分平台LLM思考模式 2026-04-20 17:35:32 +08:00
wengzh 6f52d004e8 Merge pull request #3115 from xinnan-tech/fix-knowledge-sync-ragflow-documents
fix: 实现知识库与RAGFlow双向同步(知识库名称/简介的修改、知识库删除、文档上传/删除)
2026-04-20 16:51:06 +08:00
rainv123 0681550bab fix: 实现知识库与RAGFlow双向同步(知识库名称/简介的修改、知识库删除、文档上传/删除),移除冷却机制确保实时生效,修复文档影子表UNIQUE约束冲突 2026-04-20 16:49:24 +08:00
wengzh 97b9c95c2d Merge pull request #3117 from xinnan-tech/py-fix-ReportTime
fix: 修复聊天记录上报排序错乱的问题
2026-04-20 15:56:49 +08:00
wujingquan 6111c3b5a0 Add 'configuring' translation for multiple languages in device configuration 2026-04-20 15:24:53 +08:00
wujingquan 123c1e8880 Add WiFi password translations for multiple languages 2026-04-20 15:17:04 +08:00
rainv123 6ce4503b22 fix:实现知识库文档与RAGFlow双向同步,支持远端上传/删除自动对账 2026-04-20 14:49:01 +08:00
3030332422 b209de36f5 fix: 修复聊天记录上报排序错乱的问题 2026-04-20 11:00:04 +08:00
wengzh fc3e0869fd Merge pull request #3112 from xinnan-tech/py-fix-repeated-text
Py fix repeated text
2026-04-20 10:43:09 +08:00
Sakura-RanChen 3436d50b43 只有文本内容一致的时候不进行合成 2026-04-20 10:40:08 +08:00
Sakura-RanChen 7a58c04df3 Merge pull request #3077 from xuruiray/fix/duplicate-tts-on-tool-call
fix: LLM 流式输出文本后调用工具导致 TTS 重复播报
2026-04-20 10:08:06 +08:00
Sakura-RanChen 150be65436 Merge branch 'py-fix-repeated-text' into fix/duplicate-tts-on-tool-call 2026-04-20 10:07:28 +08:00
3030332422 dc2094ea8b update:说明文档的补充 2026-04-17 17:30:18 +08:00
wengzh bacf4cd5e8 Merge pull request #3110 from xinnan-tech/fix_rag_pagerank
fix: 修复更新知识库时 pagerank 为 null 导致 RAGFlow 校验失败的问题
2026-04-17 16:52:02 +08:00
rainv123 7049e8cce7 fix: 修复更新知识库时 pagerank 为 null 导致 RAGFlow 校验失败的问题 2026-04-17 16:41:42 +08:00
3030332422 b0e24732de update:更改说明文档 2026-04-17 16:12:02 +08:00
3030332422 fb469a3ee3 fix:唤醒词服务启动失败不阻塞测试页面 2026-04-17 15:09:57 +08:00
3030332422 725c562aa1 update:对齐测试页面唤醒流程与硬件设备 2026-04-17 11:33:28 +08:00
wengzh 61f137ce4b Merge pull request #3108 from xinnan-tech/fix-role-plugin
fix: 修复智能体tooltip高度超长的问题
2026-04-17 11:25:14 +08:00
zhuoqinglian 35ea14b95b fix: 修复智能体tooltip高度超长的问题 2026-04-17 11:10:33 +08:00
wengzh b7d07f92a2 Merge pull request #3107 from xinnan-tech/py-fix-tts-markdown
fix: 调整处理顺序保留英文空格的同时修复markdown被识别为纯ASCII 字符的错误返回
2026-04-17 11:02:53 +08:00
Sakura-RanChen d2cbec0981 fix: 调整处理顺序保留英文空格的同时修复markdown被识别为纯ASCII 字符的错误返回 2026-04-17 10:59:29 +08:00
wengzh 6348f12d60 Merge pull request #3106 from xinnan-tech/fix-role-plugin
fix: 修复角色配置-功能管理列表没有滚动条的问题
2026-04-17 10:51:45 +08:00
zhuoqinglian 11caf6bc8a fix: 修复角色配置-功能管理列表没有滚动条的问题 2026-04-17 10:42:47 +08:00
wengzh 3cae9e3c32 Merge pull request #3102 from xinnan-tech/fix-title-summary
fix:将标题总结从记忆流程中拆分为独立接口,支持异步执行
2026-04-16 17:24:56 +08:00
欣南科技 0161e642d0 Merge pull request #3104 from xinnan-tech/update-default-model
Update default model
2026-04-16 17:01:34 +08:00
hrz d41938fe07 update:这些文件已经完成使命,可移除 2026-04-16 16:54:33 +08:00
hrz e20c59e3df update:移除阿里云即将下架的模型 2026-04-16 16:42:34 +08:00
rainv123 fbaf28de5c fix:移除多余try和总结标题的异步 2026-04-16 14:43:16 +08:00
rainv123 2daf492e00 fix:将标题总结从记忆流程中拆分为独立接口,支持异步执行 2026-04-16 10:36:48 +08:00
wengzh ac9083cfc7 Merge pull request #3101 from xinnan-tech/fix-title-summary
fix:修复PowerMem和Mem0AI记忆模型不会总结会话标题的问题
2026-04-16 09:04:58 +08:00
3030332422 36ea5c5e13 fix:修复多个问题 2026-04-15 17:53:51 +08:00
rainv123 f36e605aed fix:修复PowerMem和Mem0AI记忆模型不会总结会话标题的问题 2026-04-15 15:26:52 +08:00
wengzh c1b836aaed Merge pull request #3100 from xinnan-tech/update-checkbox-style
update: 修改checkbox背景颜色
2026-04-15 14:32:46 +08:00
zhuoqinglian 07170f09eb update: 修改checkbox背景颜色 2026-04-15 14:28:20 +08:00
hrz 8211de242f Merge pull request #3098 from xinnan-tech/fix-title-summary
fix:修复mem_report_only记忆模型没有总结会话标题
2026-04-15 14:20:34 +08:00
欣南科技 eea6807355 Merge pull request #3099 from xinnan-tech/agent-base-prompt
update:优化提示词
2026-04-15 13:59:04 +08:00
hrz 0ba7e81438 update:优化提示词 2026-04-15 12:59:15 +08:00
rainv123 703c707b4c fix:修复mem_report_only记忆模型没有总结会话标题 2026-04-15 11:54:22 +08:00
hrz 3ab14578c2 Merge pull request #3097 from xinnan-tech/py-fix-stream-asr
fix: 修复豆包流式asr热词层级错误
2026-04-15 10:33:24 +08:00
Sakura-RanChen a15efa3090 fix: 修复豆包流式asr热词层级错误 2026-04-15 10:31:16 +08:00
3030332422 4225f1525f fix: 移除 RuntimeInstanceLock 锁文件机制 2026-04-14 17:59:29 +08:00
3030332422 d5b883817c update:前端增加可配置的本地唤醒词运行时 2026-04-14 16:54:37 +08:00
wengzh 8af8930997 Merge pull request #3094 from xinnan-tech/py-client-about
refactor:优化相关打断逻辑
2026-04-14 15:31:29 +08:00
Sakura-RanChen ad01b05940 打断判断优先级调整 2026-04-14 15:05:55 +08:00
Sakura-RanChen a0497fdc48 给复用型TTS增加相关过滤 2026-04-14 10:18:09 +08:00
Sakura-RanChen 9634f84a98 Merge remote-tracking branch 'origin/main' into py-client-about 2026-04-13 11:35:10 +08:00
Sakura-RanChen 27e5e1e9d7 refactor: 优化非流式音频过滤机制
fix: 修复等待中被打断队列被停止造成的死锁
2026-04-13 11:33:47 +08:00
wengzh 8fd92729f2 Merge pull request #3090 from xinnan-tech/slm-and-title
update:添加SLM模型配置和会话标题功能
2026-04-13 09:08:11 +08:00
wengzh 736ffd80bf Merge pull request #3086 from xinnan-tech/update-chat-history
角色配置表单标题添加Tooltip鼠标悬浮提示说明,调整小参数模型表单配置
2026-04-13 09:07:50 +08:00
wengzh 3783351f5d Merge pull request #3085 from xinnan-tech/update-mobile-chat-history
mobile 角色配置添加小参数模型选项,历史记录标题优化,聊天详情参数记录折叠处理
2026-04-13 09:07:31 +08:00
zhuoqinglian 2cd9484c54 update: mobile 聊天记录获取最新数据 2026-04-10 16:11:04 +08:00
Sakura-RanChen 0f1e0d0d2e 音频会在队列积压等待发送,等待协程结果返回不设置超时时间 2026-04-10 14:22:36 +08:00
3030332422 705b5732a6 update:为 test 页面增加可配置的本地唤醒词运行时 2026-04-10 09:17:22 +08:00
Sakura-RanChen 0e23672be7 fix: 修复直接回复是字幕未记录 2026-04-09 17:58:19 +08:00
Sakura-RanChen a6904f6ebd refactor: vad独立时间戳,相关残余状态处理 2026-04-09 17:15:35 +08:00
zhuoqinglian 784ba359f2 update: mobile 历史记录标题优化,聊天详情参数记录折叠处理 2026-04-09 14:32:32 +08:00
Sakura-RanChen e08eb48d9a refactor: 优化相关打断处理 2026-04-09 10:46:46 +08:00
rainv123 b9a1b154b7 update:添加SLM模型配置和会话标题功能 2026-04-09 09:23:54 +08:00
zhuoqinglian 64630e1b4e feat: mobile 角色配置添加小参数模型选项 2026-04-08 16:26:48 +08:00
zhuoqinglian 7dcc014ce4 update: 调整小参数模型表单配置 2026-04-08 11:41:12 +08:00
Rui Xu 389e8559aa fix: prevent duplicate TTS when LLM streams text before tool call 2026-04-06 21:49:28 +08:00
wengzh c5f82369e5 Merge pull request #3060 from xinnan-tech/py_test_end
Py test end
2026-04-03 17:31:56 +08:00
wengzh 52e63706a4 Merge pull request #3058 from xinnan-tech/py_fix_powermem
fix: 优化配置读取
2026-04-03 14:37:14 +08:00
Sakura-RanChen a42a8b2b4b Merge pull request #3069 from xinnan-tech/py_fix_wakeup
fix:停止音频发送循环(仅在流控器已初始化时调用)
2026-04-03 09:48:16 +08:00
baigao 11142104af fix:停止音频发送循环(仅在流控器已初始化时调用) 2026-04-03 09:46:55 +08:00
wengzh a9f5f1ab81 Merge pull request #3067 from xinnan-tech/memory-model-rename
fix:从有总结记忆的模型切换至仅上报聊天记录时,只清空记忆总结内容
2026-04-02 11:56:33 +08:00
rainv123 a8f68c42fc fix:从有总结记忆的模型切换至仅上报聊天记录时,只清空记忆 2026-04-02 11:27:11 +08:00
zhuoqinglian 1d3afb3c9a feat: 角色配置表单标题添加Tooltip鼠标悬浮提示说明 2026-04-02 09:45:33 +08:00
wengzh 943622969a Merge pull request #3063 from xinnan-tech/py-bug-fix
fix:修复intent_llm模式下工具不存在时未正确播放提示信息的问题
2026-04-01 17:46:23 +08:00
wengzh 9d39168a35 Merge pull request #3061 from xinnan-tech/memory-model-rename
选择仅上报聊天记录时,清除历史记忆;前端优化记忆模型切换体验
2026-04-01 15:57:44 +08:00
Sakura-RanChen 61ab7f25d7 Merge branch 'main' into py_test_end 2026-04-01 14:45:05 +08:00
Sakura-RanChen 140fcd5887 补充sql,增加错误详情上报 2026-04-01 14:32:47 +08:00
rainv123 3c2024d00d fix: 前端优化记忆模型切换体验 - 切换无记忆模型时暂存记忆、切换恢复、保存配置后才真正清除 2026-04-01 14:32:30 +08:00
rainv123 2aee1bf567 fix:选择仅上报聊天记录时,清除历史记忆 2026-04-01 11:35:26 +08:00
wengzh d89a26889c Merge pull request #3059 from xinnan-tech/memory-model-rename
fix:重命名记忆模型名称
2026-04-01 10:23:38 +08:00
rainv123 d611b2db73 update:重命名记忆模型名称 2026-04-01 09:54:46 +08:00
Sakura-RanChen b0e911b248 Merge pull request #2996 from lgy1027/fix-end
fix: 优化退出流程并添加超时保护
2026-04-01 09:35:34 +08:00
Sakura-RanChen fb3b7830ee Merge branch 'py_test_end' into fix-end 2026-04-01 09:35:23 +08:00
3030332422 10eeffdf31 fix:修复intent_llm模式下工具不存在时未正确播放提示信息的问题 2026-04-01 09:23:05 +08:00
rainv123 17847188dd Merge branch 'main' of https://github.com/xinnan-tech/xiaozhi-esp32-server into new 2026-04-01 09:12:39 +08:00
Sakura-RanChen 7b2bfc087d fix: 优化配置读取 2026-03-31 18:00:51 +08:00
wengzh 24a8839753 Merge pull request #3044 from xinnan-tech/py-display
添加“工具调用日志记录”与“设备端工具调用信息显示”
2026-03-31 15:12:11 +08:00
CGD 64d25533d9 Merge branch 'main' into py-display 2026-03-31 14:24:27 +08:00
3030332422 6f6c788cd5 update:修改ai_agent_chat_history表content字段类型为TEXT 2026-03-31 14:11:31 +08:00
欣南科技 8aa12d0961 Merge pull request #3057 from xinnan-tech/weather-faq
update:补齐天气文档链接
2026-03-31 12:11:36 +08:00
hrz d770632152 udpate:补齐天气文档链接 2026-03-31 12:08:30 +08:00
wengzh a30903b644 Merge pull request #3056 from xinnan-tech/update-manager-mobile
Update manager mobile
2026-03-31 11:02:40 +08:00
wengzh 5469168d18 Merge pull request #3055 from xinnan-tech/fix-digital-human
fix: 修复数字人摄像头展示区域裁剪的问题
2026-03-31 11:02:21 +08:00
zhuoqinglian 3c31231781 fix: 修复数字人摄像头展示区域裁剪的问题 2026-03-31 10:34:38 +08:00
3030332422 cc2e30205f fix:修复play_music工具调用信息显示异常 2026-03-31 10:03:50 +08:00
3030332422 f01de027cd fix:修复 intent_llm 模式下工具调用不上报的问题 2026-03-30 16:36:30 +08:00
zhuoqinglian fb195fc57b update: mobile 切换记忆模式判断是否需要禁用历史记忆内容输入 2026-03-26 17:25:44 +08:00
zhuoqinglian cc504fe542 update: mobile 添加智能体限制调整 2026-03-26 14:21:03 +08:00
wengzh fd93116d69 Merge pull request #3045 from xinnan-tech/fix_audioRateController
fix: 修复流式场景下的误判
2026-03-25 11:10:42 +08:00
Sakura-RanChen 5622a13735 fix: 修复流式场景下的误判 2026-03-25 11:07:43 +08:00
3030332422 079c8630d9 update:修改部分前端样式 2026-03-25 10:43:45 +08:00
wengzh 11830b2f22 Merge pull request #3043 from xinnan-tech/update-manager-mobile
mobile 优化聊天记录音频播放功能
2026-03-25 09:34:29 +08:00
wengzh 861b4dee4c Merge pull request #3042 from xinnan-tech/update-web-style
优化页面主体内容区域适配
2026-03-25 09:34:06 +08:00
zhuoqinglian 603fa67824 Merge branch 'main' into update-web-style 2026-03-25 09:27:30 +08:00
zhuoqinglian 4d8279d916 update: 优化页面主体内容区域适配 2026-03-25 09:26:33 +08:00
rainv123 f91c4acb8f Merge branch 'main' of https://github.com/xinnan-tech/xiaozhi-esp32-server into new 2026-03-24 11:12:32 +08:00
rainv123 1342eccf86 Merge pull request #3039 from xinnan-tech/revert-3038-agent-chat-history-audio-retention
Revert "fix:修复智能体记忆模型配置变更时错误删除历史聊天音频的问题"
2026-03-24 11:08:44 +08:00
wengzh 87f9ab7bb4 Revert "fix:修复智能体记忆模型配置变更时错误删除历史聊天音频的问题" 2026-03-24 11:05:35 +08:00
wengzh 94cc8564aa Merge pull request #3038 from xinnan-tech/agent-chat-history-audio-retention
fix:修复智能体记忆模型配置变更时错误删除历史聊天音频的问题
2026-03-24 10:53:51 +08:00
rainv123 8fc2d05c99 Merge branch 'main' of https://github.com/xinnan-tech/xiaozhi-esp32-server into new 2026-03-24 09:51:14 +08:00
rainv123 1a6d9d7490 fix: 修复智能体配置变更时错误删除历史音频的问题 2026-03-24 09:49:57 +08:00
wengzh 6c19f1bd26 Merge pull request #3035 from xinnan-tech/mem_report_only
fix:新增仅上报聊天记录的记忆模型供应器Mem report only
2026-03-24 09:01:31 +08:00
rainv123 6d89ae1c66 fix:增加仅上报模型说明和记忆模型排序 2026-03-23 16:58:51 +08:00
rainv123 2a49f0804c feat:新增仅上报聊天记录(mem_report_only)记忆供应器 2026-03-23 16:17:38 +08:00
zhuoqinglian d64bb36606 update: mobile 优化聊天记录音频播放功能 2026-03-23 14:18:53 +08:00
zhuoqinglian 852e097e0d fix: mobile 修复声纹管理-编辑模式下声纹向量展示异常的问题 2026-03-20 16:57:02 +08:00
zhuoqinglian 3aecb99f70 fix: 修复聊天记录播放音频重叠播放的问题 2026-03-20 16:21:58 +08:00
wengzh 6eecf5d0cb Merge pull request #3026 from xinnan-tech/fix_audioRateController_state
fix: 修复队列恢复时音频播放时间错乱问题
2026-03-20 14:49:42 +08:00
wengzh a4d02ca2ab Merge pull request #3029 from xinnan-tech/update-web-style
解决登录失败验证码刷新两次的问题
2026-03-20 14:48:45 +08:00
wengzh 7ef5a67119 Merge pull request #3028 from xinnan-tech/update-manager-mobile
mobile 智能体管理多语言调整、缓存用户信息
2026-03-20 14:48:26 +08:00
zhuoqinglian 1c5dacd2d5 Merge branch 'main' into update-web-style 2026-03-20 14:45:50 +08:00
zhuoqinglian 7732b7b1b9 update: mobile 缓存用户信息 2026-03-20 14:42:47 +08:00
zhuoqinglian 6e7b938f51 fix: 解决登录失败验证码刷新两次的问题 2026-03-20 10:42:35 +08:00
zhuoqinglian 52f5859b1f update: mobile 智能体管理多语言调整 2026-03-19 17:56:27 +08:00
Sakura-RanChen 73d174bc4a fix: 修复队列恢复时音频播放时间错乱问题 2026-03-19 16:18:30 +08:00
Sakura-RanChen 9512fe642c 恢复多工具 2026-03-19 16:04:14 +08:00
Sakura-RanChen 0ef3a60a4c 调整缩进 2026-03-19 11:57:40 +08:00
Sakura-RanChen f40dc4a658 修复双重回复问题 2026-03-19 11:56:29 +08:00
wengzh 4747e54f6c Merge pull request #3023 from xinnan-tech/update-manager-mobile
mobile 播放声纹弹窗样式调整,切换角色模板更新表单状态
2026-03-18 16:53:54 +08:00
wengzh 9278fdb016 Merge pull request #3024 from xinnan-tech/update-web-style
解决MCP接入点输入框没有显示报错信息的问题
2026-03-18 16:53:14 +08:00
zhuoqinglian 5b23804954 Merge branch 'main' into update-manager-mobile 2026-03-18 16:50:29 +08:00
zhuoqinglian b2ed5eb2c0 fix: 解决MCP接入点输入框没有显示报错信息的问题 2026-03-18 16:47:48 +08:00
zhuoqinglian 07113dc4d2 update: mobile 播放声纹弹窗样式调整,切换角色模板更新表单状态 2026-03-18 16:44:27 +08:00
Sakura-RanChen 1a5dd2d9d9 fix: 修复日志音频上报 2026-03-18 16:40:32 +08:00
wengzh 8a01432e9d Merge pull request #3022 from xinnan-tech/perf-userLanguage-complete
perf:补充MCP接入点国际化信息
2026-03-18 15:48:57 +08:00
DaGou12138 7a39fd0152 补充MCP接入点国际化信息 2026-03-18 09:48:11 +08:00
Sakura-RanChen 0d2cb7b3c9 增加工具调用日志记录,暂时单工具状态 2026-03-17 16:13:35 +08:00
wengzh 9123a174da Merge pull request #3020 from xinnan-tech/update-web-style
优化表格底部按钮边距统一调整,修复刷新页面菜单、页脚闪动的问题
2026-03-17 16:11:54 +08:00
zhuoqinglian 2e165747d6 update: 参数弹窗去掉多余代码 2026-03-17 16:08:36 +08:00
zhuoqinglian 3bdf634e37 Merge branch 'main' into update-web-style 2026-03-17 16:04:22 +08:00
zhuoqinglian 20ac6f3f96 fix: 修复刷新页面菜单、页脚闪动的问题 2026-03-17 16:01:10 +08:00
zhuoqinglian 818ac1eefb style: 表格底部按钮边距统一调整 2026-03-17 10:30:43 +08:00
3030332422 980930cd42 update:在设备端显示工具调用信息 2026-03-16 13:11:45 +08:00
wengzh 956ae37e56 Merge pull request #3015 from xinnan-tech/fix/update-sherpa-onnx
update:更新sherpa_onnx依赖至1.12.29版本
2026-03-16 11:51:01 +08:00
rainv123 15731892ad update:更新sherpa_onnx依赖至1.12.29版本 2026-03-16 11:43:07 +08:00
zhuoqinglian d6d50e0432 fix: 修复语言切换接口请求异常的问题 2026-03-16 10:37:47 +08:00
zhuoqinglian 286079c93a feat: 添加声纹管理-声纹向量音频播放功能 2026-03-16 09:50:24 +08:00
zhuoqinglian 860cbc36fd fix: 处理智能体管理全局触发下拉刷新问题 2026-03-16 09:46:43 +08:00
lgy1027 bcf03a0378 refactor: 优化超时配置,响应社区反馈
- 工具调用超时改为可配置项 tool_call_timeout,默认30秒
- OpenAI 超时恢复可配置机制,支持细粒度配置和单一值
- TTS 超时保护统一添加到所有流式TTS实现
- 将 ActionResponse 导入移到文件顶部
- 修复超时配置边界情况处理
2026-03-11 18:13:05 +08:00
lgy1027 4f615b3807 Merge remote-tracking branch 'origin/main' into fix-end 2026-03-11 17:15:59 +08:00
lgy1027 c2b95ca896 fix: 优化退出流程并添加超时保护
- 添加 is_exiting 标志防止退出流程被中断
- 工具调用添加 30 秒超时保护,避免流程卡死
- TTS 相关操作添加超时保护
- 优化 OpenAI 客户端超时配置
2026-03-10 11:45:38 +08:00
752 changed files with 47978 additions and 21922 deletions

No files matched your search

+37 -1
View File
@@ -3,4 +3,40 @@ updates:
- package-ecosystem: pip
directory: /main/xiaozhi-server
schedule:
interval: weekly
interval: monthly
open-pull-requests-limit: 5
groups:
ai-services:
patterns:
- "openai*"
- "google-generativeai"
- "dashscope"
- "baidu-aip"
- "cozepy"
- "markitdown"
- "mem0ai"
- "powermem"
ai-models:
patterns:
- "funasr"
- "vosk"
- "silero_vad"
- "opuslib_next"
- "sherpa_onnx"
- "modelscope"
web-stack:
patterns:
- "httpx"
- "aiohttp"
- "aiohttp_cors"
- "requests"
- "pysocks"
everything-else:
patterns:
- "*"
ignore:
- dependency-name: "torch"
- dependency-name: "torchaudio"
- dependency-name: "numpy"
- dependency-name: "portalocker"
- dependency-name: "websockets"
+66
View File
@@ -0,0 +1,66 @@
name: Tests
on:
push:
branches: [main]
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
python:
name: Python (xiaozhi-server)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.10'
cache: 'pip'
cache-dependency-path: 'main/xiaozhi-server/requirements.txt'
- name: Install dependencies
working-directory: main/xiaozhi-server
run: |
pip install -r requirements.txt
pip install pytest pytest-asyncio freezegun
- name: Run smoke test
working-directory: main/xiaozhi-server
run: |
if [ -f tests/test_smoke.py ]; then
pytest tests/test_smoke.py -x -q
else
echo "::warning::Slice 2 not done yet — tests/test_smoke.py missing"
exit 0
fi
java:
name: Java (manager-api)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '21'
cache: 'maven'
- name: Run tests
working-directory: main/manager-api
run: mvn -B -q test -DfailIfNoTests=false
web:
name: Vue (manager-web)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: main/manager-web/package-lock.json
- name: Run tests
working-directory: main/manager-web
run: |
npm ci --no-audit --no-fund
npm run test:unit
+11 -1
View File
@@ -112,12 +112,17 @@ celerybeat.pid
# Environments
.env
.venv
/.venv-*/
env/
venv/
ENV/
env.bak/
venv.bak/
# Repository-local runtimes and package-manager caches
/.runtime/
/main/manager-web/.npm-cache/
# Spyder project settings
.spyderproject
.spyproject
@@ -149,6 +154,7 @@ tmp
.history
.DS_Store
main/xiaozhi-server/data
main/xiaozhi-server/.claude
main/manager-web/node_modules
.config.yaml
@@ -186,4 +192,8 @@ uploadfile
!main/manager-mobile/**/env/
!main/manager-mobile/**/.env*
!main/manager-mobile/**/*.json
!main/xiaozhi-server/**/*.json
!main/xiaozhi-server/**/*.json
!main/digital-human/**/*.json
main/digital-human/wakeword_runtime/models
main/digital-human/wakeword_runtime/sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01
sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01.tar.bz2
+2 -1
View File
@@ -29,4 +29,5 @@ COPY main/xiaozhi-server/requirements.txt .
# 安装Python依赖
RUN pip install --no-cache-dir --upgrade pip setuptools wheel && \
pip install --no-cache-dir -r requirements.txt --default-timeout=120 --retries 5
# 阿里云 PyPI 镜像偶有元数据 404,加官方源作 fallback
pip install --no-cache-dir -r requirements.txt --default-timeout=120 --retries 5 --extra-index-url https://pypi.org/simple
+21
View File
@@ -0,0 +1,21 @@
.PHONY: test test-fast test-python test-java test-web help
help:
@echo "make test-fast - run all fast unit tests (default)"
@echo "make test - alias for test-fast"
@echo "make test-python - pytest in main/xiaozhi-server"
@echo "make test-java - mvn test in main/manager-api"
@echo "make test-web - npm run test:unit in main/manager-web"
test: test-fast
test-fast: test-python test-java test-web
test-python:
cd main/xiaozhi-server && python -m pytest -x -q
test-java:
cd main/manager-api && mvn -B -q test -DfailIfNoTests=false
test-web:
cd main/manager-web && npm ci --no-audit --no-fund && npm test
+79 -95
View File
@@ -18,10 +18,10 @@
<p align="center">
<a href="./README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DBEDFA"></a>
<a href="./README_en.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
<a href="./README_pt_BR.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-DFE0E5"></a>
<a href="./docs/readme/README_en.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./docs/readme/README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./docs/readme/README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
<a href="./docs/readme/README_pt_BR.md"><img alt="Português (Brasil)" src="https://img.shields.io/badge/Português (Brasil)-DFE0E5"></a>
<a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">
<img alt="GitHub Contributors" src="https://img.shields.io/github/v/release/xinnan-tech/xiaozhi-esp32-server?logo=docker" />
</a>
@@ -53,113 +53,98 @@ Spearheaded by Professor Siyuan Liu's Team (South China University of Technology
<table>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐" src="docs/images/demo7.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Z8XuYZEAS" target="_blank">
<picture>
<img alt="天气插件" src="docs/images/demo8.png" />
</picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="实时打断" src="docs/images/demo10.png" />
</picture>
</a>
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" />
</picture>
</a>
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV17LXWYvENb" target="_blank">
<picture>
<img alt="播报新闻" src="docs/images/demo0.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
</a>
</td>
</tr>
</table>
@@ -214,8 +199,8 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
|:---:|:---:|:---:|
| ASR(语音识别) | FunASR(本地) | 👍XunfeiStreamASR(讯飞流式) |
| LLM(大模型) | glm-4-flash(智谱) | 👍qwen-flash(阿里百炼) |
| VLLM(视觉大模型) | glm-4v-flash(智谱) | 👍qwen2.5-vl-3b-instructh(阿里百炼) |
| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山流式) |
| VLLM(视觉大模型) | glm-4v-flash(智谱) | 👍qwen3.5-flash(阿里百炼) |
| TTS(语音合成) | EdgeTTS(微软) | 👍HuoshanDoubleStreamTTS(火山流式) |
| Intent(意图识别) | function_call(函数调用) | function_call(函数调用) |
| Memory(记忆功能) | mem_local_short(本地短期记忆) | mem_local_short(本地短期记忆) |
@@ -226,7 +211,7 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| 工具名称 | 位置 | 使用方法 | 功能说明 |
|:---:|:---|:---:|:---:|
| 音频交互测试工具 | main》xiaozhi-server》test》test_page.html | 使用谷歌浏览器直接打开 | 测试音频播放和接收功能,验证Python端音频处理是否正常 |
| 音频交互测试工具 | main》digital-human》index.html | 在 `main/digital-human` 执行 `python start.py` 后访问 `http://127.0.0.1:8006/index.html` | 测试音频播放和接收功能,验证Python端音频处理是否正常 |
| 模型响应测试工具 | main》xiaozhi-server》performance_tester.py | 执行 `python performance_tester.py` | 测试ASR(语音识别)、LLM(大模型)、VLLM(视觉模型)、TTS(语音合成)三个核心模块的响应速度 |
> 💡 提示:测试模型速度时,只会测试配置了密钥的模型。
@@ -368,11 +353,10 @@ Websocket接口地址: wss://2662r3426b.vicp.fun/xiaozhi/v1/
| <img src="./docs/images/logo_contributors.png" width="160"> | [代码贡献者](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | 感谢[所有代码贡献者](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors)贡献者,你们的付出让项目更加健壮和强大。 |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
<a href="https://www.star-history.com/?repos=xinnan-tech%2Fxiaozhi-esp32-server&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=xinnan-tech/xiaozhi-esp32-server&type=Date" />
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&theme=dark&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=xinnan-tech/xiaozhi-esp32-server&type=date&legend=top-left&sealed_token=cQovHKgZjqEnQ-svfYfN392irNvGuq-6pyv4cA8nd2jQEhQLz1ETV4YHTVk2UZyLMFbCQuZA7jduRh3YbeK5WPYaRLrfmIimGQa3lram652jJL9oQk-UuSZA5H6L4dPIhZc8KCc-Ur_UAUNbly7TePpnTR2otGknBLCOjOliD4fk1st6z7tPEDVjSRx5" />
</picture>
</a>
+3 -3
View File
@@ -70,7 +70,7 @@ xiaozhi-server
## 2. 配置项目文件
接下里,程序还不能直接运行,你需要配置一下,你到底使用的是什么模型。你可以看这个教程:
接下来,程序还不能直接运行,你需要配置一下,你到底使用的是什么模型。你可以看这个教程:
[跳转到配置项目文件](#配置项目)
配置完项目文件后,回到本教程继续往下。
@@ -119,7 +119,7 @@ docker rmi ghcr.nju.edu.cn/xinnan-tech/xiaozhi-esp32-server:web_latest
如果确定使用`conda`,则安装好后,开始执行以下命令。
重要提示!windows 用户,可以通过安装`Anaconda`来管理环境。安装好`Anaconda`后,在`开始`那里搜索`anaconda`相关的关键词,
找到`Anaconda Prpmpt`,使用管理员身份运行它。如下图。
找到`Anaconda Prompt`,使用管理员身份运行它。如下图。
![conda_prompt](./images/conda_env_1.png)
@@ -241,7 +241,7 @@ LLM:
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-OTA接口是 http://192.168.4.123:8003/xiaozhi/ota/
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-Websocket地址是 ws://192.168.4.123:8000/xiaozhi/v1/
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
250427 13:04:20[0.3.11_SiFuChTTnofu][__main__]-INFO-=======================================================
```
+4 -4
View File
@@ -198,7 +198,7 @@ docker logs -f xiaozhi-esp32-server
```
25-02-23 12:01:09[core.websocket_server] - INFO - Websocket地址是 ws://xxx.xx.xx.xx:8000/xiaozhi/v1/
25-02-23 12:01:09[core.websocket_server] - INFO - =======上面的地址是websocket协议地址,请勿用浏览器访问=======
25-02-23 12:01:09[core.websocket_server] - INFO - 如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
25-02-23 12:01:09[core.websocket_server] - INFO - 如想测试websocket请启动digital-human模块,打开浏览器交互测试
25-02-23 12:01:09[core.websocket_server] - INFO - =======================================================
```
@@ -335,7 +335,7 @@ npm run serve
如果确定使用`conda`,则安装好后,开始执行以下命令。
重要提示!windows 用户,可以通过安装`Anaconda`来管理环境。安装好`Anaconda`后,在`开始`那里搜索`anaconda`相关的关键词,
找到`Anaconda Prpmpt`,使用管理员身份运行它。如下图。
找到`Anaconda Prompt`,使用管理员身份运行它。如下图。
![conda_prompt](./images/conda_env_1.png)
@@ -418,7 +418,7 @@ manager-api:
secret: 12345678-xxxx-xxxx-xxxx-123456789000
```
## 5.运行项目
## 9.运行项目
```
# 确保在xiaozhi-server目录下执行
@@ -431,7 +431,7 @@ python app.py
```
25-02-23 12:01:09[core.websocket_server] - INFO - Server is running at ws://xxx.xx.xx.xx:8000/xiaozhi/v1/
25-02-23 12:01:09[core.websocket_server] - INFO - =======上面的地址是websocket协议地址,请勿用浏览器访问=======
25-02-23 12:01:09[core.websocket_server] - INFO - 如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
25-02-23 12:01:09[core.websocket_server] - INFO - 如想测试websocket请启动digital-human模块,打开浏览器交互测试
25-02-23 12:01:09[core.websocket_server] - INFO - =======================================================
```
+12 -4
View File
@@ -40,8 +40,8 @@ conda install conda-forge::ffmpeg
|:---:|:---:|:---:|
| ASR(语音识别) | FunASR(本地) | 👍XunfeiStreamASR(讯飞流式) |
| LLM(大模型) | glm-4-flash(智谱) | 👍qwen-flash(阿里百炼) |
| VLLM(视觉大模型) | glm-4v-flash(智谱) | 👍qwen2.5-vl-3b-instructh(阿里百炼) |
| TTS(语音合成) | ✅LinkeraiTTS(灵犀流式) | 👍HuoshanDoubleStreamTTS(火山流式) |
| VLLM(视觉大模型) | glm-4v-flash(智谱) | 👍qwen3.5-flash(阿里百炼) |
| TTS(语音合成) | EdgeTTS(微软) | 👍HuoshanDoubleStreamTTS(火山流式) |
| Intent(意图识别) | function_call(函数调用) | function_call(函数调用) |
| Memory(记忆功能) | mem_local_short(本地短期记忆) | mem_local_short(本地短期记忆) |
@@ -65,13 +65,14 @@ VAD:
3、[如何部署MQTT网关开启MQTT+UDP协议](./mqtt-gateway-integration.md)<br/>
4、[如何自动拉取本项目最新代码自动编译和启动](./dev-ops-integration.md)<br/>
5、[如何与Nginx集成](https://github.com/xinnan-tech/xiaozhi-esp32-server/issues/791)<br/>
6、[修改代码后怎么编译自己的Docker镜像](./docker-build.md)<br/>
### 9、编译固件相关教程
### 8、编译固件相关教程
1、[如何自己编译小智固件](./firmware-build.md)<br/>
2、[如何基于虾哥编译好的固件修改OTA地址](./firmware-setting.md)<br/>
3、[单模块部署如何配置固件OTA自动升级](./ota-upgrade-guide.md)<br/>
### 10、拓展相关教程
### 9、拓展相关教程
1、[如何开启手机号码注册智控台](./ali-sms-integration.md)<br/>
2、[如何集成HomeAssistant实现智能家居控制](./homeassistant-integration.md)<br/>
3、[如何开启视觉模型实现拍照识物](./mcp-vision-integration.md)<br/>
@@ -83,6 +84,13 @@ VAD:
9、[知识库ragflow集成指南](./ragflow-integration.md)<br/>
10、[如何部署上下文源](./context-provider-integration.md)<br/>
11、[如何集成PowerMem智能记忆](./powermem-integration.md)<br/>
12、[如何配置天气插件查询天气](./weather-integration.md)<br/>
13、[如何开启设备呼叫插件](./device-call-guide.md)<br/>
14、[如何开启联网搜索功能](./web-search-integration.md)<br/>
### 10、数字人相关教程
1、[数字人digital-human启动方法](./digital-human-wakeword.md)<br/>
2、[如何在N100迷你主机上部署数字人digital-human](./all-in-one-digital-human-setup.md)<br/>
### 11、语音克隆、本地语音部署相关教程
1、[如何在智控台克隆音色](./huoshan-streamTTS-voice-cloning.md)<br/>
+442
View File
@@ -0,0 +1,442 @@
# 一体机数字人配置指南
本项目用于在 x86 架构设备(如迷你主机、工控机、普通电脑等)上部署一套完整的数字人展示系统,实现以下功能:
- 开机自动进入 Kiosk 全屏浏览器,展示指定网页
- 后台运行唤醒词检测服务,支持语音交互
> **说明**:本文档以 **Intel N100 迷你主机(天虹 QN10-100B4)** 为例进行部署演示,其他 x86 设备可参考调整(注意网络配置和声卡设备差异)。
## 适用环境
| 项目 | 说明 |
|------|------|
| 示例硬件 | 天虹 QN10-100B4(Intel N100) |
| 操作系统 | Ubuntu 24.04 LTS (Noble Numbat) |
| 示例用户 | xz(请根据实际情况替换) |
| 网络 | Wi-Fi 连接,固定 IP(可按需改为有线) |
## 部署流程
1. 系统初始化(换源、连网)
2. 安装图形组件与 Kiosk 浏览器
3. 配置自动登录与图形界面
4. 部署唤醒词服务(Python 环境 + 麦克风)
5. 优化开机速度与隐藏启动信息
---
### 系统初始化(换源、连网)
```
sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak
sudo tee /etc/apt/sources.list > /dev/null <<EOF
deb http://mirrors.aliyun.com/ubuntu/ noble main restricted universe multiverse
# deb-src http://mirrors.aliyun.com/ubuntu/ noble main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ noble-security main restricted universe multiverse
# deb-src http://mirrors.aliyun.com/ubuntu/ noble-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ noble-updates main restricted universe multiverse
# deb-src http://mirrors.aliyun.com/ubuntu/ noble-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ noble-proposed main restricted universe multiverse
# deb-src http://mirrors.aliyun.com/ubuntu/ noble-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ noble-backports main restricted universe multiverse
# deb-src http://mirrors.aliyun.com/ubuntu/ noble-backports main restricted universe multiverse
EOF
echo 'Acquire::ForceIPv4 "true";' | sudo tee /etc/apt/apt.conf.d/99force-ipv4
```
安装网络管理工具(若已存在可忽略)
Bash
```
sudo apt update
sudo apt install network-manager -y
sudo systemctl start NetworkManager
sudo systemctl enable NetworkManager
```
设置wifi密码,固定ip
> **提醒**:以下命令中的 Wi-Fi 名称、密码、IP 地址均为示例,请务必替换为你自己的实际信息。
Bash
```
sudo nmcli device wifi connect "MERCURY_1812" password "12345678"
sudo nmcli connection modify "MERCURY_1812" ipv4.addresses "192.168.0.86/24" ipv4.gateway "192.168.0.1" ipv4.dns "8.8.8.8,114.114.114.114" ipv4.method "manual"
sudo nmcli connection up "MERCURY_1812"
```
### 第一步:安装核心图形组件与浏览器
这里我们坚持“极简主义”,坚决不装多余的桌面环境(如 GNOME/KDE),只装底层驱动、最轻量的窗口管理器(Openbox)、隐藏鼠标工具以及 Chromium 浏览器。
Bash
```
sudo timedatectl set-timezone Asia/Shanghai
sudo apt install net-tools vim fonts-wqy-microhei fonts-wqy-zenhei alsa-utils pulseaudio -y
sudo apt install --no-install-recommends xserver-xorg x11-xserver-utils xinit openbox unclutter -y
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo apt install ./google-chrome-stable_current_amd64.deb -y
rm google-chrome-stable_current_amd64.deb
sudo apt purge snapd -y
```
### 第二步:配置 TTY1 开机免密自动登录
为了避免手动输入账号密码的尴尬,我们通过修改 systemd 服务,让系统一开机就自动以 `xz` 的身份登录。这里使用一键写入命令,彻底避开由于 `nano` 或 `vi` 操作不当导致的保存失败问题。
**1. 创建配置目录:**
Bash
```
sudo mkdir -p /etc/systemd/system/getty@tty1.service.d/
```
**2. 写入自动登录规则:**
Bash
```
echo -e "[Service]\nExecStart=\nExecStart=-/sbin/agetty --autologin xz --noclear %I \$TERM" | sudo tee /etc/systemd/system/getty@tty1.service.d/override.conf
```
**3. 重载服务并设置默认启动目标:**
Bash
```
sudo systemctl daemon-reload
sudo systemctl set-default multi-user.target
```
### 第三步:配置登录后自动启动图形界面
系统自动登录后默认停留在黑底白字的命令行,我们需要配置脚本,让它一旦登录立马启动 X11 图形环境。
**1. 触发 `startx` 的启动逻辑:**
直接将触发代码追加写入你的个人环境配置文件中:
Bash
```
cat << 'EOF' >> ~/.bash_profile
if [ -z "$DISPLAY" ] && [ "$(fgconsole)" -eq 1 ]; then
exec startx
fi
EOF
```
**2. 告诉 `startx` 去启动 Openbox:**
Bash
```
echo "exec openbox-session" > ~/.xinitrc
```
### 第四步:配置“铜墙铁壁”的 Openbox 与浏览器
这是最核心的一步:关闭屏幕休眠、隐藏鼠标、全屏锁定浏览器,并写一个“死循环”保证浏览器被意外关闭后也能瞬间复活。
**1. 创建 Openbox 配置目录:**
Bash
```
mkdir -p ~/.config/openbox
```
**2. 写入自启动脚本 (`autostart`):**
复制以下整段代码并回车(这会自动把所有保护规则写进文件里):
Bash
```
cat << 'EOF' > ~/.config/openbox/autostart
# 关闭屏幕保护
xset -dpms
xset s noblank
xset s off
# 隐藏鼠标
unclutter -idle 0.1 -root &
# 死循环启动 Chromium(崩溃或被关也能秒重启)
while true; do
google-chrome \
--kiosk \
--no-first-run \
--no-default-browser-check \
--disable-infobars \
--disable-session-crashed-bubble \
--disable-translate \
--disable-external-intent-requests \
--autoplay-policy=no-user-gesture-required \
--use-fake-ui-for-media-stream \
"https://www.douyin.com"
sleep 2
done &
EOF
```
**3. 屏蔽 `Alt+F4` 退出快捷键:**
为了防止别人插上键盘强行关掉窗口,我们把 Openbox 默认的系统快捷键干掉。
Bash
```
cp /etc/xdg/openbox/rc.xml ~/.config/openbox/
sed -i '/<keybind key="A-F4">/,/<\/keybind>/d' ~/.config/openbox/rc.xml
```
### 第五步:重启验收成果
如果拔掉网线或不需要等待所有网络上线,可禁用网络等待服务,避免开机卡顿
Bash
```
sudo systemctl mask systemd-networkd-wait-online.service
sudo systemctl mask NetworkManager-wait-online.service
```
隐藏开机信息(GRUB)
Bash
```
sudo sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="quiet loglevel=3 systemd.show_status=false vt.global_cursor_default=0"/g' /etc/default/grub
echo 'GRUB_TIMEOUT_STYLE="hidden"' | sudo tee -a /etc/default/grub
echo 'GRUB_RECORDFAIL_TIMEOUT=0' | sudo tee -a /etc/default/grub
sudo update-grub
```
声音设置成100%,然后 重启:
Bash
```
amixer -q sset Master 100% unmute
sudo reboot
```
### 部署唤醒词服务
在一体机上部署唤醒词检测服务,需要安装 Python 环境、上传项目文件、配置 Camera 麦克风和开机自启。
#### 1. 安装 Miniconda
```bash
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -p $HOME/miniconda3
~/miniconda3/bin/conda init bash
source ~/.bashrc
rm Miniconda3-latest-Linux-x86_64.sh
```
确保登录时自动进入conda环境
```bash
if ! grep -q '.bashrc' ~/.bash_profile; then
cat << 'EOF' >> ~/.bash_profile
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
EOF
fi
```
#### 2. 创建 Python 虚拟环境
```bash
conda create -n test python=3.10 -y
conda activate test
```
若出现 Terms of Service have not been accepted 错误,执行:
```bash
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r
```
#### 3. 上传项目文件
将开发机上的 `main/digital-human/` 整个目录上传到一体机的 `~/digital-human/` 目录:
```bash
# 在开发机上执行(将 <一体机IP> 替换为实际 IP)
scp -r main/digital-human/ xz@<一体机IP>:~/digital-human/
```
#### 4. 安装系统依赖
唤醒词服务需要音频采集库和 ALSA PulseAudio 插件:
```bash
sudo apt install libportaudio2 portaudio19-dev libasound2-plugins -y
```
#### 5. 安装 Python 依赖
```bash
cd ~/digital-human/wakeword_runtime
pip install numpy
pip install -r requirements.txt
```
#### 6. 下载唤醒词模型
模型文件不包含在项目中,需要单独下载配置,详见 [docs/digital-human-wakeword.md](digital-human-wakeword.md) 中的“模型下载”章节。
#### 7. 修改 Openbox 自启动脚本
需要在 autostart 中加上 PulseAudio 和 Camera 麦克风配置,并将 Chrome 地址改为测试页面。
先确认 Camera 麦克风在 PulseAudio 中的设备名:
```bash
pulseaudio --start
pactl list sources short
```
找到包含 `USB_Camera` 的那一行,记下完整名称,例如:
```
alsa_input.usb-SN0002_2K_USB_Camera_46435000_P030D00_SN0002-02.mono-fallback
```
然后用完整内容覆盖 autostart(将 `TARGET_MIC` 替换为你的实际设备名):
```bash
cat << 'EOF' > ~/.config/openbox/autostart
# 1. 启动声音服务并稍作等待
pulseaudio --start
sleep 1
# 2. 锁定 Camera 的麦克风(请替换为你的实际设备名)
TARGET_MIC="alsa_input.usb-SN0002_2K_USB_Camera_46435000_P030D00_SN0002-02.mono-fallback"
# 3. 设为系统默认麦克风
pactl set-default-source "$TARGET_MIC"
# 4. 解除静音
pactl set-source-mute "$TARGET_MIC" 0
# 5. 音量拉到 100%
pactl set-source-volume "$TARGET_MIC" 100%
# --- 极简桌面与浏览器环境配置 ---
# 关闭屏幕保护
xset -dpms
xset s noblank
xset s off
# 隐藏鼠标
unclutter -idle 0.1 -root &
# 死循环启动浏览器(崩溃或被关也能秒重启)
while true; do
google-chrome \
--kiosk \
--no-first-run \
--no-default-browser-check \
--disable-infobars \
--disable-session-crashed-bubble \
--disable-translate \
--disable-external-intent-requests \
--autoplay-policy=no-user-gesture-required \
--use-fake-ui-for-media-stream \
"http://127.0.0.1:8006/index.html"
sleep 2
done &
EOF
```
#### 8. 配置唤醒词服务开机自启
创建 systemd 服务文件,让唤醒词服务开机自动运行。
先确认当前用户的 UID:
```bash
id -u $(whoami)
```
然后用查到的 UID 替换下面 `1000`(通常第一个用户就是 1000):
```bash
sudo tee /etc/systemd/system/digital-human.service << 'EOF'
[Unit]
Description=Digital Human Runtime
After=network.target sound.target
[Service]
Type=simple
User=xz
Environment=XDG_RUNTIME_DIR=/run/user/1000
Environment=PULSE_SERVER=unix:/run/user/1000/pulse/native
WorkingDirectory=/home/xz/digital-human
ExecStartPre=/bin/sleep 10
ExecStart=/home/xz/miniconda3/envs/test/bin/python start.py
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
```
> **重要说明**:
> - `User=xz` — 替换为你的实际用户名
> - `/run/user/1000` — 替换为你实际的 UID
> - `WorkingDirectory` 和 `ExecStart` 中的路径 — 替换为你的实际部署路径
> - `Environment` 中的 PulseAudio 环境变量**必须保留**,否则唤醒词服务和浏览器无法同时使用 Camera 麦克风
启用并启动服务:
```bash
sudo systemctl daemon-reload
sudo systemctl enable digital-human
sudo systemctl start digital-human
```
#### 9. 常用服务管理命令
```bash
sudo systemctl start digital-human # 立即启动
sudo systemctl stop digital-human # 停止
sudo systemctl restart digital-human # 重启
sudo systemctl status digital-human # 查看状态
journalctl -u digital-human -f # 查看实时日志
```
+1 -1
View File
@@ -7,7 +7,7 @@
`上下文源` 在小智在唤醒那一刻,获取外部系统的数据,并将其动态注入到大模型的系统提示词(System Prompt)中。
让其做到唤醒时感知世界某个事物的状态。
它和MCP、记忆有本质的区别:`上下文源`是强制让小智感知世界的数据;`记忆(Mem)`是让他知道之前聊了什么内容;`MCP(functionc all)`是当需要调用某项能力/知识的时候使用调用。
它和MCP、记忆有本质的区别:`上下文源`是强制让小智感知世界的数据;`记忆(Mem)`是让他知道之前聊了什么内容;`MCP(function_call)`是当需要调用某项能力/知识的时候使用调用。
通过这个功能,在小智唤醒的一刹那,“感知”到:
- 人体健康传感器状态(体温、血压、血氧状态等)
+2 -2
View File
@@ -51,7 +51,7 @@ cp 你原来的model.pt完整路径 /home/system/xiaozhi/xiaozhi-esp32-server/ma
# 第四步 建立三个自动编译文件
## 4.1 自动编译mananger-web模块
## 4.1 自动编译manager-web模块
在`/home/system/xiaozhi/`目录下,创建名字为`update_8001.sh`的文件,内容如下
```
@@ -106,7 +106,7 @@ fi
nohup java -jar xiaozhi-esp32-api.jar --spring.profiles.active=dev &
tail tail -f nohup.out
tail -f nohup.out
```
保存好后执行赋权命令
+148
View File
@@ -0,0 +1,148 @@
# 设备间相互呼叫插件使用指南
## 概述
设备呼叫功能允许两个已配置设备之间通过语音/数据通道进行双向通信。设备A呼叫设备B时,系统通过以下流程实现:
```
设备A → 授权校验 → MQTT网关 → 设备B远程唤醒 → 建立连接 → 通话建立
```
## 使用这个功能的前提条件
1. 你必须要有至少两个设备,每个设备型号必须是`ESP32-S3`,因为只有`ESP32-S3`才支持远程唤醒功能。
2. 你的设备必须要有`两个麦克风`。但是如果你的设备只有`单个麦克风`,你只是想体验一下这个功能,也是可以的,但是会有强烈的卡顿感。
3. 你必须使用[全模块部署](Deployment_all.md)本项目,因为你需要`智控台`来管理设备的权限和通信。
4. 你必须安装并配置好`2026年5月27日`以后的[MQTT网关服务](mqtt-gateway-integration.md),如果你已经部署了MQTT网关服务,请确认代码的版本是`2026年5月27日`之后的版本。
以上是使用这个功能的硬性条件,接下来会详细介绍。
## 配置步骤
### 第一步:开启通讯录功能
1. 确认你的智控台版本是`0.9.4`或以上版本。
2. 登录智控台后台
3. 进入 **系统功能配置**
4. 在左侧功能列表中勾选 **通讯录**
5. 点击 **保存配置** 确认
### 第二步:配置设备间呼叫权限
1. 在智控台顶部菜单点击 **通讯录**
2. 在左侧智能体下,设备列表中选择你的设备A(支持按 MAC地址 或 备注名 搜索)
3. 在右侧详情面板中,找到目标设备B的称呼设置,例如 **"小王"**
4. 勾选设备B的 **呼叫权限** 复选框
5. 点击 **保存**
**双向授权说明:** 如需设备A和设备B互相通信,必须在两侧智控台分别配置对方权限。例如:
- 在设备A的配置中勾选设备B → 设备A可与设备B通信
- 在设备B的配置中勾选设备A → 设备B可与设备A通信
### 第三步:在智能体配置添加呼叫工具
1. 在智控台顶部菜单点击 **智能体管理**
2. 在刚刚配置设备联系人的相关智能体中点击 **编辑角色**
3. 在右侧详情面板中,点击 **编辑功能**
4. 勾选 **设备呼叫设备** 工具
5. 点击 **保存配置** 确认
6. 在外侧再次点击 **保存配置** ,随即重启设备
### 第四步:固件端添加远程唤醒工具
1. 在[xiaozhi-esp32](https://github.com/78/xiaozhi-esp32) 代码的基础上增加远程唤醒工具MCP,版本支持为2.1.0至2.2.6(2026年5月29日的版本)
2. 在application.h文件中添加远程唤醒函数声明
```cpp
void RemoteWakeup(const std::string& reason);
```
3. 在application.cc文件中添加远程唤醒函数
```cpp
void Application::RemoteWakeup(const std::string& reason){
if (!protocol_) {
return;
}
auto state = GetDeviceState();
if (state == kDeviceStateIdle) {
audio_service_.EncodeWakeWord();
if (!protocol_->IsAudioChannelOpened()) {
SetDeviceState(kDeviceStateConnecting);
if (!protocol_->OpenAudioChannel()) {
audio_service_.EnableWakeWordDetection(true);
return;
}
}
std::string wake_word = reason;
#if CONFIG_USE_AFE_WAKE_WORD || CONFIG_USE_CUSTOM_WAKE_WORD
// Encode and send the wake word data to the server
while (auto packet = audio_service_.PopWakeWordPacket()) {
protocol_->SendAudio(std::move(packet));
}
// Set the chat state to wake word detected
protocol_->SendWakeWordDetected(wake_word);
SetListeningMode(aec_mode_ == kAecOff ? kListeningModeAutoStop : kListeningModeRealtime);
#else
// Set flag to play popup sound after state changes to listening
// (PlaySound here would be cleared by ResetDecoder in EnableVoiceProcessing)
play_popup_on_listening_ = true;
SetListeningMode(aec_mode_ == kAecOff ? kListeningModeAutoStop : kListeningModeRealtime);
#endif
} else if (state == kDeviceStateSpeaking) {
AbortSpeaking(kAbortReasonWakeWordDetected);
SetDeviceState(kDeviceStateIdle);
} else if (state == kDeviceStateActivating) {
SetDeviceState(kDeviceStateIdle);
}
}
```
4. 在mcp_server.cc文件中添加远程唤醒工具
```cpp
AddUserOnlyTool("self.remote_wakeup", "Remote wakeup function with configurable parameters",
PropertyList({
Property("reason", kPropertyTypeString, "Wakeup reason"),
}),
[this](const PropertyList& properties) -> ReturnValue {
std::string reason = properties["reason"].value<std::string>();
ESP_LOGI(TAG, "Wakeup reason=%s", reason.c_str());
auto& app = Application::GetInstance();
app.RemoteWakeup(reason);
return true;
```
5. 按照 [固件编译烧录指南](firmware-build.md) 完成固件烧录
6. 无论你的设备是单麦还是双麦,请在编译环节,勾选开启AEC功能!
7. 无论你的设备是单麦还是双麦,请在编译环节,勾选开启AEC功能!
8. 无论你的设备是单麦还是双麦,请在编译环节,勾选开启AEC功能!
### 第五步:配置MQTT网关服务
1. 部署MQTT网关服务,参考 [MQTT网关集成文档](mqtt-gateway-integration.md)
2. 如果已经部署请确认代码的版本是2026年5月27日的版本
## 呼叫流程说明
准备两个设备,在智控台上面配置好通讯权限和在智能体中添加呼叫工具之后,在其中一个小智对话那里对他说:”呼叫XXX“,观察设备B是否响应。
## 常见问题
### Q: 设备B没有响应呼叫?
- 检查设备B是否在线(智控台设备状态)
- 确认设备B的固件已正确集成远程唤醒工具
- 检查MQTT网关连接是否正常
- 验证双向权限配置是否完整
### Q: 提示"无呼叫权限"?
- 在智控台确认设备A已勾选设备B的呼叫权限
- 确认配置已保存(非仅修改未保存)
### Q: 如何确认通讯录功能已开启?
- 智控台顶部菜单如显示"通讯录"入口,则表示已开启
### Q: 我叫他呼叫"张山",但是他老是识别成"张三",怎么办?
- 可以查阅你使用的asr服务的文档,确认是否支持热词识别。
- 如果你用的是`FunASRServer`,可以在容器里的`热词文件`里添加"张山",然后重启容器。
- 如果你用的是`火山引擎`的服务,可以在`火山引擎的控制台`里添加`热词文件`,然后回到智控台的`模型配置页面`,把`热词文件名称`配置在`火山引擎的tts`上去。
+220
View File
@@ -0,0 +1,220 @@
# 数字人digital-human启动方法
## 概述
测试页面集成了基于 **Sherpa-ONNX** 的高精度语音唤醒功能,支持自定义唤醒词和实时检测。使用轻量级关键词检测模型,提供毫秒级响应速度。
## 唤醒词模型
### 模型下载(必需)
**重要说明**: 项目不包含模型文件,需要提前下载配置。
### 官方模型下载地址
- **官方模型列表**: <https://csukuangfj.github.io/sherpa/onnx/kws/pretrained_models/index.html>
- **推荐模型**: `sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01`
### 下载和配置步骤
#### 1. 下载模型包
```bash
# 方法1:直接下载(推荐)
cd main/digital-human/wakeword_runtime/
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/kws-models/sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01.tar.bz2
# 解压
tar xvf sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01.tar.bz2
# 方法2:使用ModelScope
pip install modelscope
python -c "
from modelscope import snapshot_download
snapshot_download('pkufool/sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01', cache_dir='./models')
"
```
#### 2. 配置模型文件
模型包下载后包含以下文件:
```
sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01/
├── encoder-epoch-12-avg-2-chunk-16-left-64.int8.onnx # 速度优先
├── encoder-epoch-12-avg-2-chunk-16-left-64.onnx
├── encoder-epoch-99-avg-1-chunk-16-left-64.int8.onnx # 速度优先
├── encoder-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
├── decoder-epoch-12-avg-2-chunk-16-left-64.onnx
├── decoder-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
├── joiner-epoch-12-avg-2-chunk-16-left-64.int8.onnx # 速度优先
├── joiner-epoch-12-avg-2-chunk-16-left-64.onnx
├── joiner-epoch-99-avg-1-chunk-16-left-64.int8.onnx # 速度优先
├── joiner-epoch-99-avg-1-chunk-16-left-64.onnx # 精度优先
├── tokens.txt # Token映射表(必需)
├── keywords_raw.txt # 模型包里可能附带(可选,runtime 不依赖)
├── keywords.txt # 现成的
├── test_wavs/ # 测试音频(可选)
├── configuration.json # 模型元信息(可选)
└── README.md # 说明文档(可选)
```
#### 3. 选择配置方案
**方案一:精度优先(推荐)**
```bash
cd sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01
# 创建模型目录
mkdir -p ../models
# 复制精度优先的epoch-99 fp32三件套
cp encoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/encoder.onnx
cp decoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/decoder.onnx
cp joiner-epoch-99-avg-1-chunk-16-left-64.onnx ../models/joiner.onnx
# 复制配套文件
cp tokens.txt ../models/tokens.txt
# keywords_raw.txt 如果模型包里附带,可自行保留;runtime 不依赖它
```
**方案二:速度优先**
```bash
cd sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01
# 创建模型目录
mkdir -p ../models
# 复制速度优先的epoch-99 int8三件套
cp encoder-epoch-99-avg-1-chunk-16-left-64.int8.onnx ../models/encoder.onnx
cp decoder-epoch-99-avg-1-chunk-16-left-64.onnx ../models/decoder.onnx
cp joiner-epoch-99-avg-1-chunk-16-left-64.int8.onnx ../models/joiner.onnx
# 复制配套文件
cp tokens.txt ../models/tokens.txt
```
**注意事项**:
- **不要混用 fp32 与 int8**:三个模型文件必须保持一致的精度
- **优先选择 epoch-99**:比 epoch-12 训练更充分,精度更高
- **必需文件**:`encoder.onnx` + `decoder.onnx` + `joiner.onnx` + `tokens.txt` + `keywords.txt`
### 最终模型文件结构
配置完成后,模型文件应放在 `wakeword_runtime/models/` 目录下,完整路径为 `main/digital-human/wakeword_runtime/models/`:
```
wakeword_runtime/models/
├── encoder.onnx # 编码器模型(重命名后)
├── decoder.onnx # 解码器模型(重命名后)
├── joiner.onnx # 连接器模型(重命名后)
├── tokens.txt # 拼音 Token 映射表(228行版本)
├── keywords.txt # 关键词配置文件(首次启动自动生成)
└── keywords_raw.txt # 可选,runtime 不依赖
```
## 启动方式
在 `main/digital-human` 目录执行:
```bash
pip install -r wakeword_runtime/requirements.txt
python start.py
```
启动后默认地址:
- 页面地址:`http://127.0.0.1:8006/index.html`
- 事件桥地址:`ws://127.0.0.1:8006/wakeword-ws`
- 健康检查:`http://127.0.0.1:8006/health`
停止方式:
- 在运行终端按 `Ctrl+C`
- 会同时停止静态页面服务、事件桥和唤醒词检测流程
## 配置文件说明
配置文件位于 [main/digital-human/wakeword_runtime/config.json](../main/digital-human/wakeword_runtime/config.json)。
当前主要配置项:
```json
{
"wakeword": {
"enabled": true
},
"model_dir": "models",
"audio": {
"input_device": null,
"sample_rate": 16000,
"channels": 1
},
"detector": {
"num_threads": 4,
"provider": "cpu",
"max_active_paths": 2,
"keywords_score": 1.8,
"keywords_threshold": 0.1,
"num_trailing_blanks": 1,
"cooldown_seconds": 1.5
},
"logging": {
"level": "INFO",
"dir": "logs",
"file": "wakeword-runtime.log"
}
}
```
各字段含义:
| 参数 | 说明 |
| --- | --- |
| `wakeword.enabled` | 是否启用本地唤醒词检测 |
| `model_dir` | 模型和词表所在目录 |
| `audio.input_device` | 麦克风输入设备,默认使用系统默认设备 |
| `audio.sample_rate` | 采样率,默认 `16000` |
| `audio.channels` | 声道数,默认 `1` |
| `detector.num_threads` | 检测器线程数 |
| `detector.provider` | 推理 provider,当前通常为 `cpu` |
| `detector.max_active_paths` | 搜索路径数 |
| `detector.keywords_score` | 关键词增强分数 |
| `detector.keywords_threshold` | 检测阈值 |
| `detector.num_trailing_blanks` | 尾随空白数量 |
| `detector.cooldown_seconds` | 连续触发冷却时间 |
| `logging.level` | 日志等级 |
| `logging.dir` | 日志目录 |
| `logging.file` | 日志文件名 |
## 推荐使用流程
### 首次使用
1. 准备 `models/` 目录下的模型文件和 `tokens.txt`
2. 确认 `models/keywords.txt` 存在
3. 在 `digital-human` 目录运行 `python start.py`
4. 浏览器打开 `http://127.0.0.1:8006/index.html`
5. 进入设置页检查“唤醒词”配置
### 修改唤醒词
1. 打开数字人页面设置
2. 切到“唤醒词”页签
3. 修改启用状态或唤醒词列表
4. 点击“应用唤醒词”
5. 根据提示决定是否立即重启
### 禁用唤醒词
1. 将“启用本地唤醒词”改成禁用
2. 点击“应用唤醒词”
3. 建议立即重启一次
禁用后:
- 页面与事件桥仍然可用
- 唤醒词检测不会继续运行
+66 -13
View File
@@ -1,21 +1,74 @@
# 本地编译docker镜像方法
现在本项目已经使用github自动编译docker功能,本文档是提供给有本地编译docker镜像需求的朋友准备的。
现在本项目已经使用`github`的`自动编译docker镜像`功能,如果您拉取的是项目发行的镜像,您没有自己编译镜像的需求,那就忽略本文档。
1、安装docker
```
如果您修改了源码,然后想采用`docker`的方式部署运行,可以参照以下步骤操作:
## 1、环境准备
安装docker:
```bash
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```
2、编译docker镜像
```
#进入项目根目录
# 编译server
docker build -t xiaozhi-esp32-server:server_latest -f ./Dockerfile-server .
# 编译web
docker build -t xiaozhi-esp32-server:web_latest -f ./Dockerfile-web .
# 编译完成后,可以使用docker-compose启动项目
# docker-compose.yml你需要修改成自己编译的镜像版本
## 2、编译镜像
当你修改好代码后,需要编译新的镜像时,需要按照以下步骤操作:
准备好你的`你的用户名`和`新的版本号`。
- 这个`你的用户名`是你在`docker hub`注册的用户名,例如`xiaozhi`。当然,如果你不需要推送到`docker hub`,你可以自由定义。
- 这个`新的版本号`是你编译的镜像版本,例如`1.2.3`,你可以根据需要自定义或者使用日期格式(例如`20260609`)主要是方便和现在运行的版本号做区分,同时也方便下次回忆你是什么时候构建的,不要和现在你本机运行的版本号相同。
进入`xiaozhi-esp32-server`项目根目录,编译 server 和 web 两个镜像:
```bash
cd 项目根目录
# 编译server镜像
docker build -f Dockerfile-server -t 你的用户名/xiaozhi-esp32-server:新的版本号 .
# 编译web镜像
docker build -f Dockerfile-web -t 你的用户名/xiaozhi-esp32-server-web:新的版本号 .
```
## 3、修改docker-compose配置
```bash
cd main/xiaozhi-server
docker compose up -d
```
编辑 `docker-compose_all.yml` 文件,将镜像版本替换为你刚才编译的版本:
```yaml
services:
xiaozhi-esp32-server:
image: 你的用户名/xiaozhi-esp32-server:新的版本号 # 修改为你的镜像地址
...
xiaozhi-esp32-server-web:
image: 你的用户名/xiaozhi-esp32-server-web:新的版本号 #修改为你的镜像地址
...
```
## 4、重启服务
```bash
# 停止旧容器
docker compose -f docker-compose_all.yml down
# 启动新容器
docker compose -f docker-compose_all.yml up -d
```
## 5、验证
查看日志确认服务启动正常:
```bash
# 查看server日志
docker logs -f -n 50 xiaozhi-esp32-server
# 查看web日志
docker logs -f -n 50 xiaozhi-esp32-server-web
```
+2 -2
View File
@@ -13,9 +13,9 @@ http://192.168.1.25:8003/xiaozhi/ota/
```
如果显示“OTA接口运行正常,向设备发送的websocket地址是:ws://xxx:8000/xiaozhi/v1/
你可以使用项目自带的`test_page.html`测试一下,是否能连上ota页面输出的websocket地址。
你可以启动`digital-human`模块后打开`index.html`测试一下,是否能连上ota页面输出的websocket地址。
如果访问不到,你需要到配置文件`.config.yaml`里修改`server.websocket`的地址,重启后再重新测试,直到`test_page.html`能正常访问。
如果访问不到,你需要到配置文件`.config.yaml`里修改`server.websocket`的地址,重启后再重新测试,直到`index.html`能正常访问。
成功后,请往下进行第2步
+1 -1
View File
@@ -45,7 +45,7 @@ python download_models.py
python -m tools.api_server --listen 0.0.0.0:6006
```
然后用浏览器去到aotodl实例页面
然后用浏览器去到autodl实例页面
```
https://autodl.com/console/instance/list
```
+2 -2
View File
@@ -68,7 +68,7 @@ http://homeassistant.local:8123
然后在实体中搜索你相关控制的开关,结果出来后,在列表中,点击其中一个结果,这是会出现一个开关的界面。
在开关的界面,我们尝试点击开关,看看是开发会随着我们的点击开/关。如果能操作,说明是正常联网的。
在开关的界面,我们尝试点击开关,看看是否会随着我们的点击开/关。如果能操作,说明是正常联网的。
接着在开关面板找到设置按钮,点击后,可以查看这个开关的`实体标识符`。
@@ -107,7 +107,7 @@ http://homeassistant.local:8123
保存成功后,即可唤醒设备操作。
#### 3. 唤醒设别进行控制
#### 3. 唤醒设备进行控制
尝试和esp32说,“打开XXX灯”
+1 -1
View File
@@ -3,7 +3,7 @@
本教程分为4个阶段:准备阶段、配置阶段、克隆阶段、使用阶段。主要是介绍通过智控台配置火山双流式语音合成+音色克隆的过程。
## 第一阶段:准备阶段
超级管理员先预先把火山引擎服务开通好,获取到App Id,Access Token。默认火上引擎会赠送一个音色资源。这个音色资源需要把它复制到本项目里。
超级管理员先预先把火山引擎服务开通好,获取到App Id,Access Token。默认火山引擎会赠送一个音色资源。这个音色资源需要把它复制到本项目里。
如果你想克隆多个音色,需要购买开通多个音色资源。只要把每个音色资源的声音ID(S_xxxxx)复制到本项目。然后分配给系统的账号使用即可。以下是详细步骤:
Binary file not shown.

Before

Width:  |  Height:  |  Size: 289 KiB

After

Width:  |  Height:  |  Size: 152 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 325 KiB

After

Width:  |  Height:  |  Size: 143 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 280 KiB

After

Width:  |  Height:  |  Size: 108 KiB

+1 -1
View File
@@ -118,7 +118,7 @@ mcp_endpoint: ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=def
250705[__main__]-INFO-mcp接入点是 ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
250705[__main__]-INFO-Websocket地址是 ws://192.168.1.25:8000/xiaozhi/v1/
250705[__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250705[__main__]-INFO-如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
250705[__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
250705[__main__]-INFO-=============================================================
```
+2 -2
View File
@@ -23,7 +23,7 @@
250705[__main__]-INFO-mcp接入点是 ws://192.168.1.25:8004/mcp_endpoint/mcp/?token=abc
250705[__main__]-INFO-Websocket地址是 ws://192.168.1.25:8000/xiaozhi/v1/
250705[__main__]-INFO-=======上面的地址是websocket协议地址,请勿用浏览器访问=======
250705[__main__]-INFO-如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
250705[__main__]-INFO-如想测试websocket请启动digital-human模块,打开浏览器交互测试
250705[__main__]-INFO-=============================================================
```
@@ -37,7 +37,7 @@
打开完,找到页面中一个绿色的按钮,写着`Code`的按钮,点开它,然后你就看到`Download ZIP`的按钮。
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`mcp-calculatorr-main`
点击它,下载本项目源码压缩包。下载到你电脑后,解压它,此时它的名字可能叫`mcp-calculator-main`
你需要把它重命名成`mcp-calculator`。接下来,我们用命令行进入项目目录即安装依赖
+3 -3
View File
@@ -58,7 +58,7 @@ docker restart xiaozhi-esp32-server
2025-06-01 **** - 视觉分析接口是 http://192.168.4.7:8003/mcp/vision/explain
2025-06-01 **** - Websocket地址是 ws://192.168.4.7:8000/xiaozhi/v1/
2025-06-01 **** - =======上面的地址是websocket协议地址,请勿用浏览器访问=======
2025-06-01 **** - 如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
2025-06-01 **** - 如想测试websocket请启动digital-human模块,打开浏览器交互测试
2025-06-01 **** - =============================================================
```
@@ -113,7 +113,7 @@ server:
那我们需要先登录[智谱AI](https://bigmodel.cn/usercenter/proj-mgmt/apikeys)网站,申请密钥。如果你之前已经申请过了密钥,可以复用这个密钥。
登录`智控台`,顶部菜单点击`模型配置`,在左侧栏点击`视觉打语言模型`,找到`VLLM_ChatGLMVLLM`,点击修改按钮,在弹框中,在`API密钥`输入你密钥,点击保存。
登录`智控台`,顶部菜单点击`模型配置`,在左侧栏点击`视觉大语言模型`,找到`VLLM_ChatGLMVLLM`,点击修改按钮,在弹框中,在`API密钥`输入你密钥,点击保存。
保存成功后,去到你需要测试的智能体哪里,点击`配置角色`,在打开的内容里,查看`视觉大语言模型(VLLM)`是否选择了刚才的视觉模型。点击保存。
@@ -133,7 +133,7 @@ docker restart xiaozhi-esp32-server
2025-06-01 **** - 视觉分析接口是 http://192.168.4.7:8003/mcp/vision/explain
2025-06-01 **** - Websocket地址是 ws://192.168.4.7:8000/xiaozhi/v1/
2025-06-01 **** - =======上面的地址是websocket协议地址,请勿用浏览器访问=======
2025-06-01 **** - 如想测试websocket请用谷歌浏览器打开test目录下的test_page.html
2025-06-01 **** - 如想测试websocket请启动digital-human模块,打开浏览器交互测试
2025-06-01 **** - =============================================================
```
+1 -1
View File
@@ -86,7 +86,7 @@ SERVER_SECRET=Te1st12134 # 服务器密钥,请保持和智控台(server.sec
- 注意不要用简单的密码,比如`123456`、`test`等。
- 注意不要用简单的密码,比如`123456`、`test`等。
`SERVER_SECRET` 是用生成websocket连接的认证信息。
`SERVER_SECRET` 是用于生成websocket连接的认证信息。
1、如果你是全模块部署,且你的智控台的参数管理里`server.auth.enabled`设置成了`true`,那么,`SERVER_SECRET`需要和智控台(`server.secret`)保持一致。
+1 -1
View File
@@ -91,7 +91,7 @@ TTS:
```py
python app.py
```
打开test目录下的test_page.html,测试连接和发送消息时paddlespeech端是否有输出日志
启动`main/digital-human`下的`python start.py`后,打开`http://127.0.0.1:8006/index.html`,测试连接和发送消息时paddlespeech端是否有输出日志
输出日志参考:
```
+2 -2
View File
@@ -1,6 +1,6 @@
# ragflow 集成指南
本教程主要是是两部分
本教程主要是两部分
- 一、如何部署ragflow
- 二、如何在智控台配置ragflow接口
@@ -12,7 +12,7 @@
ragflow需要依赖`mysql`数据库。如果你之前已经部署`智控台`,说明你已经安装了`mysql`。你可以共用它。
你可以你试一下在宿主机使用`telnet`命令,看看能不能正常访问`mysql`的`3306`端口。
你可以试一下在宿主机使用`telnet`命令,看看能不能正常访问`mysql`的`3306`端口。
``` shell
telnet 127.0.0.1 3306
+91 -106
View File
@@ -1,4 +1,4 @@
[![Banners](docs/images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
[![Banners](../images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
<h1 align="center">Xiaozhi Backend-Service xiaozhi-esp32-server</h1>
@@ -10,14 +10,14 @@ Unterstützt MQTT+UDP-Protokoll, Websocket-Protokoll, MCP-Endpunkte und Stimmabd
</p>
<p align="center">
<a href="./docs/FAQ.md">Häufige Fragen</a>
<a href="../FAQ.md">Häufige Fragen</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/issues">Probleme melden</a>
· <a href="./README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Deployment-Dokumentation</a>
· <a href="../../README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Deployment-Dokumentation</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">Release-Hinweise</a>
</p>
<p align="center">
<a href="./README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="../../README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_en.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DBEDFA"></a>
@@ -38,7 +38,7 @@ Geleitet vom Team von Professor Siyuan Liu (South China University of Technology
</br>
刘思源教授团队主导研发(华南理工大学)
</br>
<img src="./docs/images/hnlg.jpg" alt="South China University of Technology" width="50%">
<img src="../images/hnlg.jpg" alt="South China University of Technology" width="50%">
</p>
---
@@ -52,113 +52,98 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
<table>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐" src="docs/images/demo7.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Z8XuYZEAS" target="_blank">
<picture>
<img alt="天气插件" src="docs/images/demo8.png" />
</picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="实时打断" src="docs/images/demo10.png" />
</picture>
</a>
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" />
</picture>
</a>
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV17LXWYvENb" target="_blank">
<picture>
<img alt="播报新闻" src="docs/images/demo0.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
</a>
</td>
</tr>
</table>
@@ -175,17 +160,17 @@ Möchten Sie die Nutzungseffekte sehen? Klicken Sie auf die Videos unten 🎥
## Deployment-Dokumentation
![Banners](docs/images/banner2.png)
![Banners](../images/banner2.png)
Dieses Projekt bietet zwei Bereitstellungsmethoden. Bitte wählen Sie basierend auf Ihren spezifischen Anforderungen:
#### 🚀 Auswahl der Bereitstellungsmethode
| Bereitstellungsmethode | Funktionen | Anwendungsszenarien | Deployment-Dokumente | Konfigurationsanforderungen | Video-Tutorials |
|---------|------|---------|---------|---------|---------|
| **Vereinfachte Installation** | Intelligenter Dialog, Einzel-Agenten-Verwaltung | Umgebungen mit geringer Konfiguration, Daten in Konfigurationsdateien gespeichert, keine Datenbank erforderlich | [①Docker-Version](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Quellcode-Deployment](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 Kerne 4GB bei Verwendung von `FunASR`, 2 Kerne 2GB bei allen APIs | - |
| **Vollständige Modulinstallation** | Intelligenter Dialog, Mehrbenutzerverwaltung, Mehr-Agenten-Verwaltung, Intelligente Steuerkonsole-Bedienung | Vollständige Funktionserfahrung, Daten in Datenbank gespeichert |[①Docker-Version](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Quellcode-Deployment](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Quellcode-Deployment Auto-Update-Tutorial](./docs/dev-ops-integration.md) | 4 Kerne 8GB bei Verwendung von `FunASR`, 2 Kerne 4GB bei allen APIs| [Video-Tutorial für lokalen Quellcode-Start](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **Vereinfachte Installation** | Intelligenter Dialog, Einzel-Agenten-Verwaltung | Umgebungen mit geringer Konfiguration, Daten in Konfigurationsdateien gespeichert, keine Datenbank erforderlich | [①Docker-Version](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Quellcode-Deployment](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 Kerne 4GB bei Verwendung von `FunASR`, 2 Kerne 2GB bei allen APIs | - |
| **Vollständige Modulinstallation** | Intelligenter Dialog, Mehrbenutzerverwaltung, Mehr-Agenten-Verwaltung, Intelligente Steuerkonsole-Bedienung | Vollständige Funktionserfahrung, Daten in Datenbank gespeichert |[①Docker-Version](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Quellcode-Deployment](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Quellcode-Deployment Auto-Update-Tutorial](../dev-ops-integration.md) | 4 Kerne 8GB bei Verwendung von `FunASR`, 2 Kerne 4GB bei allen APIs| [Video-Tutorial für lokalen Quellcode-Start](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
Häufige Fragen und entsprechende Tutorials finden Sie unter [diesem Link](./docs/FAQ.md)
Häufige Fragen und entsprechende Tutorials finden Sie unter [diesem Link](../FAQ.md)
> 💡 Hinweis: Unten ist eine Testplattform, die mit dem neuesten Code bereitgestellt wurde. Sie können bei Bedarf brennen und testen. Gleichzeitige Benutzer: 6, Daten werden täglich gelöscht.
@@ -212,8 +197,8 @@ Websocket-Schnittstellenadresse: wss://2662r3426b.vicp.fun/xiaozhi/v1/
|:---:|:---:|:---:|
| ASR (Spracherkennung) | FunASR (Lokal) | 👍XunfeiStreamASR (Xunfei-Streaming) |
| LLM (Großes Modell) | glm-4-flash (Zhipu) | 👍qwen-flash (Alibaba Bailian) |
| VLLM (Vision Large Model) | glm-4v-flash (Zhipu) | 👍qwen2.5-vl-3b-instructh (Alibaba Bailian) |
| TTS (Sprachsynthese) | ✅LinkeraiTTS (Lingxi-Streaming) | 👍HuoshanDoubleStreamTTS (Volcano-Streaming) |
| VLLM (Vision Large Model) | glm-4v-flash (Zhipu) | 👍qwen3.5-flash (Alibaba Bailian) |
| TTS (Sprachsynthese) | EdgeTTS (Microsoft) | 👍HuoshanDoubleStreamTTS (Volcano-Streaming) |
| Intent (Absichtserkennung) | function_call (Funktionsaufruf) | function_call (Funktionsaufruf) |
| Memory (Gedächtnisfunktion) | mem_local_short (Lokales Kurzzeitgedächtnis) | mem_local_short (Lokales Kurzzeitgedächtnis) |
@@ -224,7 +209,7 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
| Werkzeugname | Standort | Verwendungsmethode | Funktionsbeschreibung |
|:---:|:---|:---:|:---:|
| Audio-Interaktionstesttool | main》xiaozhi-server》test》test_page.html | Direkt mit Google Chrome öffnen | Testet Audio-Wiedergabe- und Empfangsfunktionen, überprüft, ob die Python-seitige Audioverarbeitung normal ist |
| Audio-Interaktionstesttool | main》digital-human》index.html | In `main/digital-human` `python start.py` ausführen und dann `http://127.0.0.1:8006/index.html` öffnen | Testet Audio-Wiedergabe- und Empfangsfunktionen, überprüft, ob die Python-seitige Audioverarbeitung normal ist |
| Modell-Reaktionstesttool | main》xiaozhi-server》performance_tester.py | Ausführen `python performance_tester.py` | Testet die Reaktionsgeschwindigkeit von ASR (Spracherkennung), LLM (großes Modell), VLLM (Vision-Modell), TTS (Sprachsynthese) drei Kernmodulen |
> 💡 Hinweis: Beim Testen der Modellgeschwindigkeit werden nur Modelle mit konfigurierten Schlüsseln getestet.
@@ -232,7 +217,7 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
---
## Funktionsliste ✨
### Implementiert ✅
![请参考-全模块安装架构图](docs/images/deploy2.png)
![请参考-全模块安装架构图](../images/deploy2.png)
| Funktionsmodul | Beschreibung |
|:---:|:---|
| Kernarchitektur | Basierend auf [MQTT+UDP-Gateway](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket und HTTP-Servern, bietet vollständiges Konsolenverwaltungs- und Authentifizierungssystem |
@@ -252,9 +237,9 @@ Dieses Projekt bietet die folgenden Testwerkzeuge, um Ihnen bei der Überprüfun
### In Entwicklung 🚧
Um über spezifische Entwicklungsplanfortschritte zu erfahren, [klicken Sie hier](https://github.com/users/xinnan-tech/projects/3). Häufige Fragen und entsprechende Tutorials finden Sie unter [diesem Link](./docs/FAQ.md)
Um über spezifische Entwicklungsplanfortschritte zu erfahren, [klicken Sie hier](https://github.com/users/xinnan-tech/projects/3). Häufige Fragen und entsprechende Tutorials finden Sie unter [diesem Link](../FAQ.md)
Wenn Sie ein Softwareentwickler sind, finden Sie hier einen [Offenen Brief an Entwickler](docs/contributor_open_letter.md). Willkommen beim Beitritt!
Wenn Sie ein Softwareentwickler sind, finden Sie hier einen [Offenen Brief an Entwickler](../contributor_open_letter.md). Willkommen beim Beitritt!
---
@@ -329,7 +314,7 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int
| Typ | Plattformname | Verwendungsmethode | Preismodell | Hinweise |
|:------:|:---------------:|:----:|:---------:|:--:|
| Memory | mem0ai | Schnittstellenaufrufe | 1000 Mal/Monat Kontingent | |
| Memory | [powermem](./docs/powermem-integration.md) | Lokale Zusammenfassung | Abhängig von LLM und DB | OceanBase Open Source, unterstützt intelligente Abfrage |
| Memory | [powermem](../powermem-integration.md) | Lokale Zusammenfassung | Abhängig von LLM und DB | OceanBase Open Source, unterstützt intelligente Abfrage |
| Memory | mem_local_short | Lokale Zusammenfassung | Kostenlos | |
| Memory | nomem | Kein Gedächtnismodus | Kostenlos | |
@@ -357,13 +342,13 @@ Tatsächlich kann jedes VLLM, das OpenAI-Schnittstellenaufrufe unterstützt, int
| Logo | Projekt/Unternehmen | Beschreibung |
|:---:|:---:|:---|
| <img src="./docs/images/logo_bailing.png" width="160"> | [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) | Dieses Projekt wurde von [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) inspiriert und auf dessen Basis implementiert |
| <img src="./docs/images/logo_tenclass.png" width="160"> | [Tenclass](https://www.tenclass.com/) | Dank an [Tenclass](https://www.tenclass.com/) für die Formulierung von Standardkommunikationsprotokollen, Multi-Geräte-Kompatibilitätslösungen und High-Concurrency-Szenario-Praxisdemonstrationen für das Xiaozhi-Ökosystem; für die Bereitstellung vollständiger technischer Dokumentationsunterstützung für dieses Projekt |
| <img src="./docs/images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology](https://github.com/Eric0308) | Dank an [Xuanfeng Technology](https://github.com/Eric0308) für den Beitrag des Funktionsaufruf-Frameworks, des MCP-Kommunikationsprotokolls und der plugin-basierten Aufrufmechanismus-Implementierungscode. Durch standardisiertes Befehlsplanungssystem und dynamische Erweiterungsfähigkeiten wird die Interaktionseffizienz und funktionale Erweiterbarkeit von Frontend-Geräten (IoT) erheblich verbessert |
| <img src="./docs/images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Dank an [huangjunsen](https://github.com/huangjunsen0406) für den Beitrag des `Smart Control Console Mobile`-Moduls, das eine effiziente Steuerung und Echtzeit-Interaktion über mobile Geräte ermöglicht und die Betriebsbequemlichkeit und Verwaltungseffizienz des Systems in mobilen Szenarien erheblich verbessert |
| <img src="./docs/images/logo_huiyuan.png" width="160"> | [Huiyuan Design](http://ui.kwd988.net/) | Dank an [Huiyuan Design](http://ui.kwd988.net/) für die Bereitstellung professioneller visueller Lösungen für dieses Projekt, unter Verwendung ihrer Design-Praxiserfahrung im Dienst von über tausend Unternehmen, um die Produktbenutzererfahrung dieses Projekts zu stärken |
| <img src="./docs/images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology](https://www.029app.com/) | Dank an [Xi'an Qinren Information Technology](https://www.029app.com/) für die Vertiefung des visuellen Systems dieses Projekts und die Sicherstellung der Konsistenz und Erweiterbarkeit des Gesamtdesignstils in Multi-Szenario-Anwendungen |
| <img src="./docs/images/logo_contributors.png" width="160"> | [Code-Mitwirkende](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Dank an [alle Code-Mitwirkenden](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), Ihre Bemühungen haben das Projekt robuster und leistungsfähiger gemacht. |
| <img src="../images/logo_bailing.png" width="160"> | [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) | Dieses Projekt wurde von [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) inspiriert und auf dessen Basis implementiert |
| <img src="../images/logo_tenclass.png" width="160"> | [Tenclass](https://www.tenclass.com/) | Dank an [Tenclass](https://www.tenclass.com/) für die Formulierung von Standardkommunikationsprotokollen, Multi-Geräte-Kompatibilitätslösungen und High-Concurrency-Szenario-Praxisdemonstrationen für das Xiaozhi-Ökosystem; für die Bereitstellung vollständiger technischer Dokumentationsunterstützung für dieses Projekt |
| <img src="../images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology](https://github.com/Eric0308) | Dank an [Xuanfeng Technology](https://github.com/Eric0308) für den Beitrag des Funktionsaufruf-Frameworks, des MCP-Kommunikationsprotokolls und der plugin-basierten Aufrufmechanismus-Implementierungscode. Durch standardisiertes Befehlsplanungssystem und dynamische Erweiterungsfähigkeiten wird die Interaktionseffizienz und funktionale Erweiterbarkeit von Frontend-Geräten (IoT) erheblich verbessert |
| <img src="../images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Dank an [huangjunsen](https://github.com/huangjunsen0406) für den Beitrag des `Smart Control Console Mobile`-Moduls, das eine effiziente Steuerung und Echtzeit-Interaktion über mobile Geräte ermöglicht und die Betriebsbequemlichkeit und Verwaltungseffizienz des Systems in mobilen Szenarien erheblich verbessert |
| <img src="../images/logo_huiyuan.png" width="160"> | [Huiyuan Design](http://ui.kwd988.net/) | Dank an [Huiyuan Design](http://ui.kwd988.net/) für die Bereitstellung professioneller visueller Lösungen für dieses Projekt, unter Verwendung ihrer Design-Praxiserfahrung im Dienst von über tausend Unternehmen, um die Produktbenutzererfahrung dieses Projekts zu stärken |
| <img src="../images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology](https://www.029app.com/) | Dank an [Xi'an Qinren Information Technology](https://www.029app.com/) für die Vertiefung des visuellen Systems dieses Projekts und die Sicherstellung der Konsistenz und Erweiterbarkeit des Gesamtdesignstils in Multi-Szenario-Anwendungen |
| <img src="../images/logo_contributors.png" width="160"> | [Code-Mitwirkende](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Dank an [alle Code-Mitwirkenden](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), Ihre Bemühungen haben das Projekt robuster und leistungsfähiger gemacht. |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
+91 -106
View File
@@ -1,4 +1,4 @@
[![Banners](docs/images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
[![Banners](../images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
<h1 align="center">Xiaozhi Backend Service xiaozhi-esp32-server</h1>
@@ -10,14 +10,14 @@ Support for MQTT+UDP protocol, Websocket protocol, MCP access point, voiceprint
</p>
<p align="center">
<a href="./docs/FAQ.md">FAQ</a>
<a href="../FAQ.md">FAQ</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/issues">Report Issues</a>
· <a href="./README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Deployment Docs</a>
· <a href="../../README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Deployment Docs</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">Release Notes</a>
</p>
<p align="center">
<a href="./README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="../../README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_en.md"><img alt="README in English" src="https://img.shields.io/badge/English-DBEDFA"></a>
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
@@ -38,7 +38,7 @@ Spearheaded by Professor Siyuan Liu's Team (South China University of Technology
</br>
刘思源教授团队主导研发(华南理工大学)
</br>
<img src="./docs/images/hnlg.jpg" alt="South China University of Technology" width="50%">
<img src="../images/hnlg.jpg" alt="South China University of Technology" width="50%">
</p>
---
@@ -52,113 +52,98 @@ Want to see the usage effects? Click the videos below 🎥
<table>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐" src="docs/images/demo7.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Z8XuYZEAS" target="_blank">
<picture>
<img alt="天气插件" src="docs/images/demo8.png" />
</picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="实时打断" src="docs/images/demo10.png" />
</picture>
</a>
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" />
</picture>
</a>
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV17LXWYvENb" target="_blank">
<picture>
<img alt="播报新闻" src="docs/images/demo0.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
</a>
</td>
</tr>
</table>
@@ -175,17 +160,17 @@ Want to see the usage effects? Click the videos below 🎥
## Deployment Documentation
![Banners](docs/images/banner2.png)
![Banners](../images/banner2.png)
This project provides two deployment methods. Please choose based on your specific needs:
#### 🚀 Deployment Method Selection
| Deployment Method | Features | Applicable Scenarios | Deployment Docs | Configuration Requirements | Video Tutorials |
|---------|------|---------|---------|---------|---------|
| **Simplified Installation** | Intelligent dialogue, single agent management | Low-configuration environments, data stored in config files, no database required | [①Docker Version](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Source Code Deployment](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 cores 4GB if using `FunASR`, 2 cores 2GB if all APIs | - |
| **Full Module Installation** | Intelligent dialogue, multi-user management, multi-agent management, intelligent console interface operation | Complete functionality experience, data stored in database |[①Docker Version](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Source Code Deployment](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Source Code Deployment Auto-Update Tutorial](./docs/dev-ops-integration.md) | 4 cores 8GB if using `FunASR`, 2 cores 4GB if all APIs| [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **Simplified Installation** | Intelligent dialogue, single agent management | Low-configuration environments, data stored in config files, no database required | [①Docker Version](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Source Code Deployment](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 cores 4GB if using `FunASR`, 2 cores 2GB if all APIs | - |
| **Full Module Installation** | Intelligent dialogue, multi-user management, multi-agent management, intelligent console interface operation | Complete functionality experience, data stored in database |[①Docker Version](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Source Code Deployment](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Source Code Deployment Auto-Update Tutorial](../dev-ops-integration.md) | 4 cores 8GB if using `FunASR`, 2 cores 4GB if all APIs| [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
For frequently asked questions and related tutorials, please refer to [this link](./docs/FAQ.md)
For frequently asked questions and related tutorials, please refer to [this link](../FAQ.md)
> 💡 Note: Below is a test platform deployed with the latest code. You can burn and test if needed. Concurrent users: 6, data will be cleared daily.
@@ -212,8 +197,8 @@ Websocket Interface Address: wss://2662r3426b.vicp.fun/xiaozhi/v1/
|:---:|:---:|:---:|
| ASR(Speech Recognition) | FunASR(Local) | 👍XunfeiStreamASR(Xunfei Streaming) |
| LLM(Large Model) | glm-4-flash(Zhipu) | 👍qwen-flash(Alibaba Bailian) |
| VLLM(Vision Large Model) | glm-4v-flash(Zhipu) | 👍qwen2.5-vl-3b-instructh(Alibaba Bailian) |
| TTS(Speech Synthesis) | ✅LinkeraiTTS(Lingxi streaming) | 👍HuoshanDoubleStreamTTS(Volcano Streaming) |
| VLLM(Vision Large Model) | glm-4v-flash(Zhipu) | 👍qwen3.5-flash(Alibaba Bailian) |
| TTS(Speech Synthesis) | EdgeTTS(Microsoft) | 👍HuoshanDoubleStreamTTS(Volcano Streaming) |
| Intent(Intent Recognition) | function_call(Function calling) | function_call(Function calling) |
| Memory(Memory function) | mem_local_short(Local short-term memory) | mem_local_short(Local short-term memory) |
@@ -224,7 +209,7 @@ This project provides the following testing tools to help you verify the system
| Tool Name | Location | Usage Method | Function Description |
|:---:|:---|:---:|:---:|
| Audio Interaction Test Tool | main》xiaozhi-server》test》test_page.html | Open directly with Google Chrome | Tests audio playback and reception functions, verifies if Python-side audio processing is normal |
| Audio Interaction Test Tool | main》digital-human》index.html | Run `python start.py` in `main/digital-human`, then open `http://127.0.0.1:8006/index.html` | Tests audio playback and reception functions, verifies if Python-side audio processing is normal |
| Model Response Test Tool | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Tests response speed of three core modules: ASR(speech recognition), LLM(large model), VLLM(vision model), TTS(speech synthesis) |
> 💡 Note: When testing model speed, only models with configured keys will be tested.
@@ -232,7 +217,7 @@ This project provides the following testing tools to help you verify the system
---
## Feature List ✨
### Implemented ✅
![请参考-全模块安装架构图](docs/images/deploy2.png)
![请参考-全模块安装架构图](../images/deploy2.png)
| Feature Module | Description |
|:---:|:---|
| Core Architecture | Based on [MQTT+UDP gateway](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket and HTTP servers, provides complete console management and authentication system |
@@ -252,9 +237,9 @@ This project provides the following testing tools to help you verify the system
### Under Development 🚧
To learn about specific development plan progress, [click here](https://github.com/users/xinnan-tech/projects/3). For frequently asked questions and related tutorials, please refer to [this link](./docs/FAQ.md)
To learn about specific development plan progress, [click here](https://github.com/users/xinnan-tech/projects/3). For frequently asked questions and related tutorials, please refer to [this link](../FAQ.md)
If you are a software developer, here is an [Open Letter to Developers](docs/contributor_open_letter.md). Welcome to join!
If you are a software developer, here is an [Open Letter to Developers](../contributor_open_letter.md). Welcome to join!
---
@@ -329,7 +314,7 @@ In fact, any VLLM that supports OpenAI interface calls can be integrated and use
| Type | Platform Name | Usage Method | Pricing Model | Notes |
|:------:|:---------------:|:----:|:---------:|:--:|
| Memory | mem0ai | Interface calls | 1000 times/month quota | |
| Memory | [powermem](./docs/powermem-integration.md) | Local summarization | Depends on LLM and DB | OceanBase open source, supports intelligent retrieval |
| Memory | [powermem](../powermem-integration.md) | Local summarization | Depends on LLM and DB | OceanBase open source, supports intelligent retrieval |
| Memory | mem_local_short | Local summarization | Free | |
| Memory | nomem | No memory mode | Free | |
@@ -357,13 +342,13 @@ In fact, any VLLM that supports OpenAI interface calls can be integrated and use
| Logo | Project/Company | Description |
|:---:|:---:|:---|
| <img src="./docs/images/logo_bailing.png" width="160"> | [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) | This project is inspired by [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) and implemented on its basis |
| <img src="./docs/images/logo_tenclass.png" width="160"> | [Tenclass](https://www.tenclass.com/) | Thanks to [Tenclass](https://www.tenclass.com/) for formulating standard communication protocols, multi-device compatibility solutions, and high-concurrency scenario practice demonstrations for the Xiaozhi ecosystem; providing full-link technical documentation support for this project |
| <img src="./docs/images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology](https://github.com/Eric0308) | Thanks to [Xuanfeng Technology](https://github.com/Eric0308) for contributing function calling framework, MCP communication protocol, and plugin-based calling mechanism implementation code. Through standardized instruction scheduling system and dynamic expansion capabilities, it significantly improves the interaction efficiency and functional extensibility of frontend devices (IoT) |
| <img src="./docs/images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Thanks to [huangjunsen](https://github.com/huangjunsen0406) for contributing the `Smart Control Console Mobile` module, which enables efficient control and real-time interaction across mobile devices, significantly enhancing the system's operational convenience and management efficiency in mobile scenarios. |
| <img src="./docs/images/logo_huiyuan.png" width="160"> | [Huiyuan Design](http://ui.kwd988.net/) | Thanks to [Huiyuan Design](http://ui.kwd988.net/) for providing professional visual solutions for this project, using their design practical experience serving over a thousand enterprises to empower this project's product user experience |
| <img src="./docs/images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology](https://www.029app.com/) | Thanks to [Xi'an Qinren Information Technology](https://www.029app.com/) for deepening this project's visual system, ensuring consistency and extensibility of overall design style in multi-scenario applications |
| <img src="./docs/images/logo_contributors.png" width="160"> | [Code Contributors](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Thanks to [all code contributors](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), your efforts have made the project more robust and powerful. |
| <img src="../images/logo_bailing.png" width="160"> | [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) | This project is inspired by [Bailing Voice Dialogue Robot](https://github.com/wwbin2017/bailing) and implemented on its basis |
| <img src="../images/logo_tenclass.png" width="160"> | [Tenclass](https://www.tenclass.com/) | Thanks to [Tenclass](https://www.tenclass.com/) for formulating standard communication protocols, multi-device compatibility solutions, and high-concurrency scenario practice demonstrations for the Xiaozhi ecosystem; providing full-link technical documentation support for this project |
| <img src="../images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology](https://github.com/Eric0308) | Thanks to [Xuanfeng Technology](https://github.com/Eric0308) for contributing function calling framework, MCP communication protocol, and plugin-based calling mechanism implementation code. Through standardized instruction scheduling system and dynamic expansion capabilities, it significantly improves the interaction efficiency and functional extensibility of frontend devices (IoT) |
| <img src="../images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Thanks to [huangjunsen](https://github.com/huangjunsen0406) for contributing the `Smart Control Console Mobile` module, which enables efficient control and real-time interaction across mobile devices, significantly enhancing the system's operational convenience and management efficiency in mobile scenarios. |
| <img src="../images/logo_huiyuan.png" width="160"> | [Huiyuan Design](http://ui.kwd988.net/) | Thanks to [Huiyuan Design](http://ui.kwd988.net/) for providing professional visual solutions for this project, using their design practical experience serving over a thousand enterprises to empower this project's product user experience |
| <img src="../images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology](https://www.029app.com/) | Thanks to [Xi'an Qinren Information Technology](https://www.029app.com/) for deepening this project's visual system, ensuring consistency and extensibility of overall design style in multi-scenario applications |
| <img src="../images/logo_contributors.png" width="160"> | [Code Contributors](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Thanks to [all code contributors](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), your efforts have made the project more robust and powerful. |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
+91 -106
View File
@@ -1,4 +1,4 @@
[![Banners](docs/images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
[![Banners](../images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
<h1 align="center">Serviço Backend Xiaozhi xiaozhi-esp32-server</h1>
@@ -10,14 +10,14 @@ Suporte ao protocolo MQTT+UDP, protocolo WebSocket, ponto de acesso MCP, reconhe
</p>
<p align="center">
<a href="./docs/FAQ.md">Perguntas Frequentes</a>
<a href="../FAQ.md">Perguntas Frequentes</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/issues">Reportar Problemas</a>
· <a href="./README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Documentação de Implantação</a>
· <a href="../../README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Documentação de Implantação</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">Notas de Lançamento</a>
</p>
<p align="center">
<a href="./README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="../../README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_en.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DFE0E5"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
@@ -38,7 +38,7 @@ Liderado pela Equipe do Professor Siyuan Liu (Universidade de Tecnologia do Sul
</br>
刘思源教授团队主导研发(华南理工大学)
</br>
<img src="./docs/images/hnlg.jpg" alt="Universidade de Tecnologia do Sul da China (华南理工大学)" width="50%">
<img src="../images/hnlg.jpg" alt="Universidade de Tecnologia do Sul da China (华南理工大学)" width="50%">
</p>
---
@@ -52,113 +52,98 @@ Quer ver os efeitos de uso? Clique nos vídeos abaixo 🎥
<table>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="Experiência de velocidade de resposta" src="docs/images/demo9.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="Segredo da otimização de velocidade" src="docs/images/demo6.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="Cenário médico complexo" src="docs/images/demo1.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="Envio de comandos MQTT" src="docs/images/demo4.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="Reconhecimento de impressão vocal" src="docs/images/demo14.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="Controle de interruptores de eletrodomésticos" src="docs/images/demo5.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="Ponto de acesso MCP" src="docs/images/demo13.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="Tarefas com múltiplos comandos" src="docs/images/demo11.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="Reproduzir música" src="docs/images/demo7.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Z8XuYZEAS" target="_blank">
<picture>
<img alt="Plugin de clima" src="docs/images/demo8.png" />
</picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="Interrupção em tempo real" src="docs/images/demo10.png" />
</picture>
</a>
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="Fotografar e identificar objetos" src="docs/images/demo12.png" />
</picture>
</a>
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="Timbre de voz personalizado" src="docs/images/demo2.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="Comunicação em cantonês" src="docs/images/demo3.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV17LXWYvENb" target="_blank">
<picture>
<img alt="Transmissão de notícias" src="docs/images/demo0.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
</a>
</td>
</tr>
</table>
@@ -175,17 +160,17 @@ Quer ver os efeitos de uso? Clique nos vídeos abaixo 🎥
## Documentação de Implantação
![Banners](docs/images/banner2.png)
![Banners](../images/banner2.png)
Este projeto oferece dois métodos de implantação. Por favor, escolha de acordo com suas necessidades específicas:
#### 🚀 Seleção do Método de Implantação
| Método de Implantação | Funcionalidades | Cenários Aplicáveis | Documentação de Implantação | Requisitos de Configuração | Tutoriais em Vídeo |
|---------|------|---------|---------|---------|---------|
| **Instalação Simplificada** | Diálogo inteligente, gerenciamento de agente único | Ambientes de baixa configuração, dados armazenados em arquivos de configuração, sem necessidade de banco de dados | [①Versão Docker](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Implantação via Código-Fonte](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 núcleos 4GB se usar `FunASR`, 2 núcleos 2GB se todas APIs | - |
| **Instalação de Módulo Completo** | Diálogo inteligente, gerenciamento multiusuário, gerenciamento de múltiplos agentes, operação de interface do console inteligente | Experiência com funcionalidade completa, dados armazenados em banco de dados |[①Versão Docker](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Implantação via Código-Fonte](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Tutorial de Atualização Automática via Código-Fonte](./docs/dev-ops-integration.md) | 4 núcleos 8GB se usar `FunASR`, 2 núcleos 4GB se todas APIs| [Tutorial em Vídeo de Inicialização via Código-Fonte Local](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **Instalação Simplificada** | Diálogo inteligente, gerenciamento de agente único | Ambientes de baixa configuração, dados armazenados em arquivos de configuração, sem necessidade de banco de dados | [①Versão Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Implantação via Código-Fonte](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 núcleos 4GB se usar `FunASR`, 2 núcleos 2GB se todas APIs | - |
| **Instalação de Módulo Completo** | Diálogo inteligente, gerenciamento multiusuário, gerenciamento de múltiplos agentes, operação de interface do console inteligente | Experiência com funcionalidade completa, dados armazenados em banco de dados |[①Versão Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Implantação via Código-Fonte](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Tutorial de Atualização Automática via Código-Fonte](../dev-ops-integration.md) | 4 núcleos 8GB se usar `FunASR`, 2 núcleos 4GB se todas APIs| [Tutorial em Vídeo de Inicialização via Código-Fonte Local](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
Perguntas frequentes e tutoriais relacionados podem ser consultados [neste link](./docs/FAQ.md)
Perguntas frequentes e tutoriais relacionados podem ser consultados [neste link](../FAQ.md)
> 💡 Nota: Abaixo está uma plataforma de teste implantada com o código mais recente. Você pode gravar e testar se necessário. Usuários simultâneos: 6, os dados serão limpos diariamente.
@@ -212,8 +197,8 @@ Endereço da Interface WebSocket: wss://2662r3426b.vicp.fun/xiaozhi/v1/
|:---:|:---:|:---:|
| ASR(Reconhecimento de Fala) | FunASR(Local) | 👍XunfeiStreamASR(Xunfei Streaming) |
| LLM(Modelo de Linguagem) | glm-4-flash(Zhipu) | 👍qwen-flash(Alibaba Bailian) |
| VLLM(Modelo de Visão) | glm-4v-flash(Zhipu) | 👍qwen2.5-vl-3b-instructh(Alibaba Bailian) |
| TTS(Síntese de Voz) | ✅LinkeraiTTS(Lingxi streaming) | 👍HuoshanDoubleStreamTTS(Volcano Streaming) |
| VLLM(Modelo de Visão) | glm-4v-flash(Zhipu) | 👍qwen3.5-flash(Alibaba Bailian) |
| TTS(Síntese de Voz) | EdgeTTS(Microsoft) | 👍HuoshanDoubleStreamTTS(Volcano Streaming) |
| Intent(Reconhecimento de Intenção) | function_call(Chamada de função) | function_call(Chamada de função) |
| Memory(Função de Memória) | mem_local_short(Memória local de curto prazo) | mem_local_short(Memória local de curto prazo) |
@@ -224,7 +209,7 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
| Nome da Ferramenta | Localização | Método de Uso | Descrição da Função |
|:---:|:---|:---:|:---:|
| Ferramenta de Teste de Interação por Áudio | main》xiaozhi-server》test》test_page.html | Abrir diretamente com Google Chrome | Testa as funções de reprodução e recepção de áudio, verifica se o processamento de áudio no lado Python está normal |
| Ferramenta de Teste de Interação por Áudio | main》digital-human》index.html | Execute `python start.py` em `main/digital-human` e depois abra `http://127.0.0.1:8006/index.html` | Testa as funções de reprodução e recepção de áudio, verifica se o processamento de áudio no lado Python está normal |
| Ferramenta de Teste de Resposta de Modelo | main》xiaozhi-server》performance_tester.py | Execute `python performance_tester.py` | Testa a velocidade de resposta dos três módulos principais: ASR(reconhecimento de fala), LLM(modelo de linguagem), VLLM(modelo de visão), TTS(síntese de voz) |
> 💡 Nota: Ao testar a velocidade dos modelos, apenas os modelos com chaves configuradas serão testados.
@@ -232,7 +217,7 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
---
## Lista de Funcionalidades ✨
### Implementado ✅
![请参考-全模块安装架构图](docs/images/deploy2.png)
![请参考-全模块安装架构图](../images/deploy2.png)
| Módulo de Funcionalidade | Descrição |
|:---:|:---|
| Arquitetura Principal | Baseado em [gateway MQTT+UDP](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), servidores WebSocket e HTTP, fornece sistema completo de gerenciamento de console e autenticação |
@@ -252,9 +237,9 @@ Este projeto fornece as seguintes ferramentas de teste para ajudá-lo a verifica
### Em Desenvolvimento 🚧
Para conhecer o progresso específico do plano de desenvolvimento, [clique aqui](https://github.com/users/xinnan-tech/projects/3). Perguntas frequentes e tutoriais relacionados podem ser consultados [neste link](./docs/FAQ.md)
Para conhecer o progresso específico do plano de desenvolvimento, [clique aqui](https://github.com/users/xinnan-tech/projects/3). Perguntas frequentes e tutoriais relacionados podem ser consultados [neste link](../FAQ.md)
Se você é um desenvolvedor de software, aqui está uma [Carta Aberta aos Desenvolvedores](docs/contributor_open_letter.md). Seja bem-vindo a participar!
Se você é um desenvolvedor de software, aqui está uma [Carta Aberta aos Desenvolvedores](../contributor_open_letter.md). Seja bem-vindo a participar!
---
@@ -329,7 +314,7 @@ Na verdade, qualquer VLLM que suporte chamadas via interface OpenAI pode ser int
| Tipo | Nome da Plataforma | Método de Uso | Modelo de Preço | Observações |
|:------:|:---------------:|:----:|:---------:|:--:|
| Memória | mem0ai | Chamadas via interface | Cota de 1000 vezes/mês | |
| Memória | [powermem](./docs/powermem-integration.md) | Resumo local | Depende do LLM e BD | OceanBase de código aberto, suporta busca inteligente |
| Memória | [powermem](../powermem-integration.md) | Resumo local | Depende do LLM e BD | OceanBase de código aberto, suporta busca inteligente |
| Memória | mem_local_short | Resumo local | Gratuito | |
| Memória | nomem | Modo sem memória | Gratuito | |
@@ -357,13 +342,13 @@ Na verdade, qualquer VLLM que suporte chamadas via interface OpenAI pode ser int
| Logo | Projeto/Empresa | Descrição |
|:---:|:---:|:---|
| <img src="./docs/images/logo_bailing.png" width="160"> | [Robô de Diálogo por Voz Bailing](https://github.com/wwbin2017/bailing) | Este projeto foi inspirado pelo [Robô de Diálogo por Voz Bailing](https://github.com/wwbin2017/bailing) e implementado com base nele |
| <img src="./docs/images/logo_tenclass.png" width="160"> | [Tenclass](https://www.tenclass.com/) | Agradecimentos à [Tenclass](https://www.tenclass.com/) por formular protocolos de comunicação padrão, soluções de compatibilidade multidispositivo e demonstrações práticas de cenários de alta concorrência para o ecossistema Xiaozhi; fornecendo suporte completo de documentação técnica para este projeto |
| <img src="./docs/images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology (玄凤科技)](https://github.com/Eric0308) | Agradecimentos à [Xuanfeng Technology](https://github.com/Eric0308) por contribuir com o framework de chamada de função, protocolo de comunicação MCP e implementação do mecanismo de chamada baseado em plugins. Através de um sistema padronizado de agendamento de instruções e capacidades de expansão dinâmica, melhora significativamente a eficiência de interação e extensibilidade funcional dos dispositivos de frontend (IoT) |
| <img src="./docs/images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Agradecimentos a [huangjunsen](https://github.com/huangjunsen0406) por contribuir com o módulo `Console de Controle Inteligente Mobile`, que permite controle eficiente e interação em tempo real em dispositivos móveis, melhorando significativamente a conveniência operacional e a eficiência de gerenciamento do sistema em cenários móveis. |
| <img src="./docs/images/logo_huiyuan.png" width="160"> | [Huiyuan Design (汇远设计)](http://ui.kwd988.net/) | Agradecimentos à [Huiyuan Design](http://ui.kwd988.net/) por fornecer soluções visuais profissionais para este projeto, utilizando sua experiência prática de design atendendo mais de mil empresas para potencializar a experiência do usuário deste produto |
| <img src="./docs/images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology (西安勤人信息科技)](https://www.029app.com/) | Agradecimentos à [Xi'an Qinren Information Technology](https://www.029app.com/) por aprofundar o sistema visual deste projeto, garantindo consistência e extensibilidade do estilo de design geral em aplicações de múltiplos cenários |
| <img src="./docs/images/logo_contributors.png" width="160"> | [Contribuidores de Código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Agradecimentos a [todos os contribuidores de código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), seus esforços tornaram o projeto mais robusto e poderoso. |
| <img src="../images/logo_bailing.png" width="160"> | [Robô de Diálogo por Voz Bailing](https://github.com/wwbin2017/bailing) | Este projeto foi inspirado pelo [Robô de Diálogo por Voz Bailing](https://github.com/wwbin2017/bailing) e implementado com base nele |
| <img src="../images/logo_tenclass.png" width="160"> | [Tenclass](https://www.tenclass.com/) | Agradecimentos à [Tenclass](https://www.tenclass.com/) por formular protocolos de comunicação padrão, soluções de compatibilidade multidispositivo e demonstrações práticas de cenários de alta concorrência para o ecossistema Xiaozhi; fornecendo suporte completo de documentação técnica para este projeto |
| <img src="../images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology (玄凤科技)](https://github.com/Eric0308) | Agradecimentos à [Xuanfeng Technology](https://github.com/Eric0308) por contribuir com o framework de chamada de função, protocolo de comunicação MCP e implementação do mecanismo de chamada baseado em plugins. Através de um sistema padronizado de agendamento de instruções e capacidades de expansão dinâmica, melhora significativamente a eficiência de interação e extensibilidade funcional dos dispositivos de frontend (IoT) |
| <img src="../images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Agradecimentos a [huangjunsen](https://github.com/huangjunsen0406) por contribuir com o módulo `Console de Controle Inteligente Mobile`, que permite controle eficiente e interação em tempo real em dispositivos móveis, melhorando significativamente a conveniência operacional e a eficiência de gerenciamento do sistema em cenários móveis. |
| <img src="../images/logo_huiyuan.png" width="160"> | [Huiyuan Design (汇远设计)](http://ui.kwd988.net/) | Agradecimentos à [Huiyuan Design](http://ui.kwd988.net/) por fornecer soluções visuais profissionais para este projeto, utilizando sua experiência prática de design atendendo mais de mil empresas para potencializar a experiência do usuário deste produto |
| <img src="../images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology (西安勤人信息科技)](https://www.029app.com/) | Agradecimentos à [Xi'an Qinren Information Technology](https://www.029app.com/) por aprofundar o sistema visual deste projeto, garantindo consistência e extensibilidade do estilo de design geral em aplicações de múltiplos cenários |
| <img src="../images/logo_contributors.png" width="160"> | [Contribuidores de Código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Agradecimentos a [todos os contribuidores de código](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), seus esforços tornaram o projeto mais robusto e poderoso. |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
+91 -106
View File
@@ -1,4 +1,4 @@
[![Banners](docs/images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
[![Banners](../images/banner1.png)](https://github.com/xinnan-tech/xiaozhi-esp32-server)
<h1 align="center">Dịch vụ Backend Xiaozhi xiaozhi-esp32-server</h1>
@@ -10,14 +10,14 @@ Hỗ trợ giao thức MQTT+UDP, giao thức Websocket, điểm truy cập MCP,
</p>
<p align="center">
<a href="./docs/FAQ.md">Câu hỏi thường gặp</a>
<a href="../FAQ.md">Câu hỏi thường gặp</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/issues">Báo cáo vấn đề</a>
· <a href="./README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Tài liệu triển khai</a>
· <a href="../../README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3">Tài liệu triển khai</a>
· <a href="https://github.com/xinnan-tech/xiaozhi-esp32-server/releases">Nhật ký cập nhật</a>
</p>
<p align="center">
<a href="./README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="../../README.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_en.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_vi.md"><img alt="Tiếng Việt" src="https://img.shields.io/badge/Tiếng Việt-DBEDFA"></a>
<a href="./README_de.md"><img alt="Deutsch" src="https://img.shields.io/badge/Deutsch-DFE0E5"></a>
@@ -38,7 +38,7 @@ Spearheaded by Professor Siyuan Liu's Team (South China University of Technology
</br>
Được dẫn dắt bởi nhóm Giáo sư Lưu Tư Nguyên (Đại học Bách khoa Nam Trung Quốc)
</br>
<img src="./docs/images/hnlg.jpg" alt="华南理工大学" width="50%">
<img src="../images/hnlg.jpg" alt="华南理工大学" width="50%">
</p>
---
@@ -52,113 +52,98 @@ Muốn xem hiệu quả sử dụng? Hãy xem video 🎥
<table>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1FMFyejExX" target="_blank">
<picture>
<img alt="响应速度感受" src="docs/images/demo9.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1vchQzaEse" target="_blank">
<picture>
<img alt="速度优化秘诀" src="docs/images/demo6.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1WEcxzFEAT" target="_blank">
<picture>
<img alt="小智数字人 支持语音唤醒" src="docs/images/demo8.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1CKVz6UEuB" target="_blank">
<picture>
<img alt="设备呼叫设备,打电话" src="docs/images/demo0.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1C1tCzUEZh" target="_blank">
<picture>
<img alt="复杂医疗场景" src="docs/images/demo1.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐查询天气播报新闻" src="docs/images/demo7.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1VC96Y5EMH" target="_blank">
<picture>
<img alt="播放音乐" src="docs/images/demo7.png" />
</picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Z8XuYZEAS" target="_blank">
<picture>
<img alt="天气插件" src="docs/images/demo8.png" />
</picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV12J7WzBEaH" target="_blank">
<picture>
<img alt="实时打断" src="docs/images/demo10.png" />
</picture>
</a>
<picture>
<img alt="实时打断" src="docs/images/demo10.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Co76z7EvK" target="_blank">
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" />
</picture>
</a>
<picture>
<img alt="拍照识物品" src="docs/images/demo12.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1pNXWYGEx1" target="_blank">
<picture>
<img alt="控制家电开关" src="docs/images/demo5.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1TJ7WzzEo6" target="_blank">
<picture>
<img alt="多指令任务" src="docs/images/demo11.png" /></picture>
</a>
</td>
</tr>
<tr>
<td>
<a href="https://www.bilibili.com/video/BV1ZQKUzYExM" target="_blank">
<picture>
<img alt="MCP接入点" src="docs/images/demo13.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV17LXWYvENb" target="_blank">
<picture>
<img alt="播报新闻" src="docs/images/demo0.png" />
</picture>
</a>
<a href="https://www.bilibili.com/video/BV1zUW5zJEkq" target="_blank">
<picture>
<img alt="MQTT指令下发" src="docs/images/demo4.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1Exu3zqEDe" target="_blank">
<picture>
<img alt="声纹识别" src="docs/images/demo14.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV1CDKWemEU6" target="_blank">
<picture>
<img alt="自定义音色" src="docs/images/demo2.png" /></picture>
</a>
</td>
<td>
<a href="https://www.bilibili.com/video/BV12yA2egEaC" target="_blank">
<picture>
<img alt="使用粤语交流" src="docs/images/demo3.png" /></picture>
</a>
</td>
</tr>
</table>
@@ -176,17 +161,17 @@ Khuyến nghị người dùng ưu tiên lựa chọn nhà cung cấp dịch v
## Tài liệu triển khai
![Banners](docs/images/banner2.png)
![Banners](../images/banner2.png)
Dự án này cung cấp hai phương pháp triển khai, vui lòng chọn theo nhu cầu cụ thể của bạn:
#### 🚀 Lựa chọn phương pháp triển khai
| Phương pháp triển khai | Đặc điểm | Tình huống áp dụng | Tài liệu triển khai | Yêu cầu cấu hình | Video hướng dẫn |
|---------|------|---------|---------|---------|---------|
| **Cài đặt tối giản** | Đối thoại thông minh, quản lý đơn tác nhân | Môi trường cấu hình thấp, dữ liệu lưu trong tệp cấu hình, không cần cơ sở dữ liệu | [①Phiên bản Docker](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Triển khai mã nguồn](./docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 nhân 4GB nếu dùng `FunASR`, 2 nhân 2GB nếu toàn API | - |
| **Cài đặt toàn bộ module** | Đối thoại thông minh, quản lý đa người dùng, quản lý đa tác nhân, bảng điều khiển thông minh | Trải nghiệm đầy đủ tính năng, dữ liệu lưu trong cơ sở dữ liệu |[①Phiên bản Docker](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Triển khai mã nguồn](./docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Hướng dẫn tự động cập nhật triển khai mã nguồn](./docs/dev-ops-integration.md) | 4 nhân 8GB nếu dùng `FunASR`, 2 nhân 4GB nếu toàn API| [Video hướng dẫn khởi động mã nguồn cục bộ](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
| **Cài đặt tối giản** | Đối thoại thông minh, quản lý đơn tác nhân | Môi trường cấu hình thấp, dữ liệu lưu trong tệp cấu hình, không cần cơ sở dữ liệu | [①Phiên bản Docker](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [②Triển khai mã nguồn](../Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)| 2 nhân 4GB nếu dùng `FunASR`, 2 nhân 2GB nếu toàn API | - |
| **Cài đặt toàn bộ module** | Đối thoại thông minh, quản lý đa người dùng, quản lý đa tác nhân, bảng điều khiển thông minh | Trải nghiệm đầy đủ tính năng, dữ liệu lưu trong cơ sở dữ liệu |[①Phiên bản Docker](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [②Triển khai mã nguồn](../Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③Hướng dẫn tự động cập nhật triển khai mã nguồn](../dev-ops-integration.md) | 4 nhân 8GB nếu dùng `FunASR`, 2 nhân 4GB nếu toàn API| [Video hướng dẫn khởi động mã nguồn cục bộ](https://www.bilibili.com/video/BV1wBJhz4Ewe) |
Câu hỏi thường gặp và hướng dẫn liên quan, vui lòng tham khảo [liên kết này](./docs/FAQ.md)
Câu hỏi thường gặp và hướng dẫn liên quan, vui lòng tham khảo [liên kết này](../FAQ.md)
> 💡 Gợi ý: Dưới đây là nền tảng thử nghiệm được triển khai theo mã mới nhất, có thể flash để thử nghiệm nếu cần, đồng thời là 6, dữ liệu sẽ được xóa mỗi ngày,
@@ -213,8 +198,8 @@ Công cụ kiểm tra dịch vụ: https://2662r3426b.vicp.fun/test/
|:---:|:---:|:---:|
| ASR(Nhận dạng giọng nói) | FunASR(Local) | 👍XunfeiStreamASR(Xunfei Streaming) |
| LLM(Mô hình lớn) | glm-4-flash(Zhipu) | 👍qwen-flash(Alibaba Bailian) |
| VLLM(Mô hình lớn thị giác) | glm-4v-flash(Zhipu) | 👍qwen2.5-vl-3b-instructh(Alibaba Bailian) |
| TTS(Tổng hợp giọng nói) | ✅LinkeraiTTS(Lingxi streaming) | 👍HuoshanDoubleStreamTTS(Volcano Streaming) |
| VLLM(Mô hình lớn thị giác) | glm-4v-flash(Zhipu) | 👍qwen3.5-flash(Alibaba Bailian) |
| TTS(Tổng hợp giọng nói) | EdgeTTS(Microsoft) | 👍HuoshanDoubleStreamTTS(Volcano Streaming) |
| Intent(Nhận dạng ý định) | function_call(Gọi hàm) | function_call(Gọi hàm) |
| Memory(Chức năng bộ nhớ) | mem_local_short(Bộ nhớ ngắn hạn cục bộ) | mem_local_short(Bộ nhớ ngắn hạn cục bộ) |
@@ -225,7 +210,7 @@ Dự án này cung cấp các công cụ kiểm tra sau để giúp bạn xác m
| Tên công cụ | Vị trí | Phương pháp sử dụng | Mô tả chức năng |
|:---:|:---|:---:|:---:|
| Công cụ kiểm tra tương tác âm thanh | main》xiaozhi-server》test》test_page.html | Mở trực tiếp bằng trình duyệt Google Chrome | Kiểm tra chức năng phát và nhận âm thanh, xác minh xử lý âm thanh phía Python có bình thường không |
| Công cụ kiểm tra tương tác âm thanh | main》digital-human》index.html | Chạy `python start.py` trong `main/digital-human`, sau đó mở `http://127.0.0.1:8006/index.html` | Kiểm tra chức năng phát và nhận âm thanh, xác minh xử lý âm thanh phía Python có bình thường không |
| Công cụ kiểm tra phản hồi mô hình | main》xiaozhi-server》performance_tester.py | Thực hiện `python performance_tester.py` | Kiểm tra tốc độ phản hồi của ba module cốt lõi ASR(Nhận dạng giọng nói), LLM(Mô hình lớn), VLLM(Mô hình thị giác), TTS(Tổng hợp giọng nói) |
> 💡 Gợi ý: Khi kiểm tra tốc độ mô hình, chỉ kiểm tra các mô hình đã cấu hình khóa.
@@ -233,7 +218,7 @@ Dự án này cung cấp các công cụ kiểm tra sau để giúp bạn xác m
---
## Danh sách tính năng ✨
### Đã thực hiện ✅
![请参考-全模块安装架构图](docs/images/deploy2.png)
![请参考-全模块安装架构图](../images/deploy2.png)
| Module chức năng | Mô tả |
|:---:|:---|
| Kiến trúc cốt lõi | Dựa trên [cổng MQTT+UDP](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/docs/mqtt-gateway-integration.md), WebSocket, máy chủ HTTP, cung cấp hệ thống quản lý bảng điều khiển và xác thực hoàn chỉnh |
@@ -253,9 +238,9 @@ Dự án này cung cấp các công cụ kiểm tra sau để giúp bạn xác m
### Đang phát triển 🚧
Muốn hiểu tiến độ kế hoạch phát triển cụ thể, [vui lòng nhấp vào đây](https://github.com/users/xinnan-tech/projects/3). Câu hỏi thường gặp và hướng dẫn liên quan, vui lòng tham khảo [liên kết này](./docs/FAQ.md)
Muốn hiểu tiến độ kế hoạch phát triển cụ thể, [vui lòng nhấp vào đây](https://github.com/users/xinnan-tech/projects/3). Câu hỏi thường gặp và hướng dẫn liên quan, vui lòng tham khảo [liên kết này](../FAQ.md)
Nếu bạn là một nhà phát triển phần mềm, đây có một [Lá thư mở gửi các nhà phát triển](docs/contributor_open_letter.md), chào mừng tham gia!
Nếu bạn là một nhà phát triển phần mềm, đây có một [Lá thư mở gửi các nhà phát triển](../contributor_open_letter.md), chào mừng tham gia!
---
@@ -330,7 +315,7 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
| Loại | Tên nền tảng | Phương pháp sử dụng | Mô hình thu phí | Ghi chú |
|:------:|:---------------:|:----:|:---------:|:--:|
| Memory | mem0ai | Gọi giao diện | Hạn mức 1000 lần/tháng | |
| Memory | [powermem](./docs/powermem-integration.md) | Tóm tắt cục bộ | Phụ thuộc vào LLM và DB | OceanBase mã nguồn mở, hỗ trợ tìm kiếm thông minh |
| Memory | [powermem](../powermem-integration.md) | Tóm tắt cục bộ | Phụ thuộc vào LLM và DB | OceanBase mã nguồn mở, hỗ trợ tìm kiếm thông minh |
| Memory | mem_local_short | Tóm tắt cục bộ | Miễn phí | |
| Memory | nomem | Chế độ không có bộ nhớ | Miễn phí | |
@@ -358,13 +343,13 @@ Trên thực tế, bất kỳ VLLM nào hỗ trợ gọi giao diện openai đ
| Logo | Dự án/Công ty | Mô tả |
|:---:|:---:|:---|
| <img src="./docs/images/logo_bailing.png" width="160"> | [Robot đối thoại giọng nói Bailing](https://github.com/wwbin2017/bailing) | Dự án này được lấy cảm hứng từ [Robot đối thoại giọng nói Bailing](https://github.com/wwbin2017/bailing) và được triển khai trên cơ sở đó |
| <img src="./docs/images/logo_tenclass.png" width="160"> | [Shifang Ronghai](https://www.tenclass.com/) | Cảm ơn [Shifang Ronghai](https://www.tenclass.com/) đã xây dựng giao thức truyền thông tiêu chuẩn, phương án tương thích đa thiết bị và mô phạm thực hành tình huống đồng thời cao cho hệ sinh thái Xiaozhi; cung cấp tài liệu hỗ trợ kỹ thuật toàn diện cho dự án này |
| <img src="./docs/images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology](https://github.com/Eric0308) | Cảm ơn [Xuanfeng Technology](https://github.com/Eric0308) đã đóng góp khung gọi hàm, giao thức truyền thông MCP và mã triển khai cơ chế gọi dạng plugin, thông qua hệ thống điều phối lệnh tiêu chuẩn hóa và khả năng mở rộng động, đã cải thiện đáng kể hiệu suất tương tác và khả năng mở rộng chức năng của thiết bị front-end(IoT) |
| <img src="./docs/images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Cảm ơn [huangjunsen](https://github.com/huangjunsen0406) đã đóng góp module `Bảng điều khiển thông minh di động`, thực hiện điều khiển hiệu quả và tương tác thời gian thực trên thiết bị di động đa nền tảng, cải thiện đáng kể sự tiện lợi vận hành và hiệu quả quản lý của hệ thống trong tình huống di động |
| <img src="./docs/images/logo_huiyuan.png" width="160"> | [Huiyuan Design](http://ui.kwd988.net/) | Cảm ơn [Huiyuan Design](http://ui.kwd988.net/) đã cung cấp giải pháp thị giác chuyên nghiệp cho dự án này, sử dụng kinh nghiệm thực tế thiết kế phục vụ hơn nghìn doanh nghiệp, trao quyền cho trải nghiệm người dùng sản phẩm của dự án này |
| <img src="./docs/images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology](https://www.029app.com/) | Cảm ơn [Xi'an Qinren Information Technology](https://www.029app.com/) đã làm sâu sắc hệ thống thị giác của dự án này, đảm bảo tính nhất quán và khả năng mở rộng của phong cách thiết kế tổng thể trong ứng dụng đa tình huống |
| <img src="./docs/images/logo_contributors.png" width="160"> | [Người đóng góp mã](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Cảm ơn [tất cả người đóng góp mã](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), sự cống hiến của bạn khiến dự án mạnh mẽ và vững chắc hơn. |
| <img src="../images/logo_bailing.png" width="160"> | [Robot đối thoại giọng nói Bailing](https://github.com/wwbin2017/bailing) | Dự án này được lấy cảm hứng từ [Robot đối thoại giọng nói Bailing](https://github.com/wwbin2017/bailing) và được triển khai trên cơ sở đó |
| <img src="../images/logo_tenclass.png" width="160"> | [Shifang Ronghai](https://www.tenclass.com/) | Cảm ơn [Shifang Ronghai](https://www.tenclass.com/) đã xây dựng giao thức truyền thông tiêu chuẩn, phương án tương thích đa thiết bị và mô phạm thực hành tình huống đồng thời cao cho hệ sinh thái Xiaozhi; cung cấp tài liệu hỗ trợ kỹ thuật toàn diện cho dự án này |
| <img src="../images/logo_xuanfeng.png" width="160"> | [Xuanfeng Technology](https://github.com/Eric0308) | Cảm ơn [Xuanfeng Technology](https://github.com/Eric0308) đã đóng góp khung gọi hàm, giao thức truyền thông MCP và mã triển khai cơ chế gọi dạng plugin, thông qua hệ thống điều phối lệnh tiêu chuẩn hóa và khả năng mở rộng động, đã cải thiện đáng kể hiệu suất tương tác và khả năng mở rộng chức năng của thiết bị front-end(IoT) |
| <img src="../images/logo_junsen.png" width="160"> | [huangjunsen](https://github.com/huangjunsen0406) | Cảm ơn [huangjunsen](https://github.com/huangjunsen0406) đã đóng góp module `Bảng điều khiển thông minh di động`, thực hiện điều khiển hiệu quả và tương tác thời gian thực trên thiết bị di động đa nền tảng, cải thiện đáng kể sự tiện lợi vận hành và hiệu quả quản lý của hệ thống trong tình huống di động |
| <img src="../images/logo_huiyuan.png" width="160"> | [Huiyuan Design](http://ui.kwd988.net/) | Cảm ơn [Huiyuan Design](http://ui.kwd988.net/) đã cung cấp giải pháp thị giác chuyên nghiệp cho dự án này, sử dụng kinh nghiệm thực tế thiết kế phục vụ hơn nghìn doanh nghiệp, trao quyền cho trải nghiệm người dùng sản phẩm của dự án này |
| <img src="../images/logo_qinren.png" width="160"> | [Xi'an Qinren Information Technology](https://www.029app.com/) | Cảm ơn [Xi'an Qinren Information Technology](https://www.029app.com/) đã làm sâu sắc hệ thống thị giác của dự án này, đảm bảo tính nhất quán và khả năng mở rộng của phong cách thiết kế tổng thể trong ứng dụng đa tình huống |
| <img src="../images/logo_contributors.png" width="160"> | [Người đóng góp mã](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors) | Cảm ơn [tất cả người đóng góp mã](https://github.com/xinnan-tech/xiaozhi-esp32-server/graphs/contributors), sự cống hiến của bạn khiến dự án mạnh mẽ và vững chắc hơn. |
<a href="https://star-history.com/#xinnan-tech/xiaozhi-esp32-server&Date">
+1 -1
View File
@@ -20,7 +20,7 @@
声纹识别需要依赖`mysql`数据库。如果你之前已经部署`智控台`,说明你已经安装了`mysql`。你可以共用它。
你可以你试一下在宿主机使用`telnet`命令,看看能不能正常访问`mysql`的`3306`端口。
你可以试一下在宿主机使用`telnet`命令,看看能不能正常访问`mysql`的`3306`端口。
```
telnet 127.0.0.1 3306
```
+79
View File
@@ -0,0 +1,79 @@
# 联网搜索插件使用指南
## 功能简介
联网搜索插件 `web_search` 支持在对话过程中实时联网搜索信息并返回结果。插件支持三个搜索源:秘塔(Metaso)、Tavily和Serply,用户可根据需要选择其中一个。
## API Key申请指南
目前我们适配了`秘塔搜索`、`Tavily搜索`和`Serply搜索`。
- Tavily搜索:每个月1000次免费额度。
- 秘塔搜索:拥有较为优质的国内数据源。
- Serply搜索:返回谷歌搜索结果,注册赠送2500次免费额度,额度用完后需要付费。
## API Key申请指南
### 方式一:使用秘塔搜索
- 访问 [秘塔搜索API](https://metaso.cn/search-api/api-keys),注册并登录账号
- 在API密钥管理页面,点击"创建新的Key"
- 复制生成的API Key(以 `mk-` 为前缀),这是配置所需的关键信息
### 方式二:使用Tavily搜索
- 访问 [Tavily控制台](https://app.tavily.com/home),注册并登录账号
- 在控制台中创建API Key
- 复制生成的API Key(以 `tvly-` 为前缀),这是配置所需的关键信息
### 方式三:使用Serply搜索
- 访问 [Serply](https://serply.io),注册并登录账号
- 在控制台中创建API Key
- 复制生成的API Key,这是配置所需的关键信息
- 接口说明见 [Serply文档](https://serply.io/docs)
## 配置方式
### 方式1. 使用智控台部署(推荐)
- 登录智控台
- 进入"配置角色"页面,选择要配置的智能体
- 点击"编辑功能"按钮,在右侧参数配置区域找到"联网搜索"插件
- 勾选"联网搜索"
- 填入搜索源(`metaso`、`tavily`或`serply`),并将对应的`API Key`填入配置项
- 保存配置,再保存智能体配置
### 方式2. 单模块xiaozhi-server部署
在 `data/.config.yaml` 中配置:
- 将搜索源填入 `provider`,可选值为 `metaso`、`tavily` 或 `serply`
- 将申请到的API Key填入 `api_key`
```yaml
plugins:
web_search:
provider: "metaso"
api_key: "你的API Key"
```
如需自定义返回结果数量和工具描述,可额外配置 `max_results` 和 `description`:
```yaml
plugins:
web_search:
provider: "metaso"
description: "联网搜索工具。当用户明确需要联网搜索问题时使用此工具。"
max_results: 5
api_key: "你的API Key"
```
同时在 `functions` 列表中确保已启用 `web_search`:
```yaml
plugins:
functions:
- web_search
```
配置完成后重启服务即可生效。
+52 -1
View File
@@ -9,6 +9,7 @@
* [3.2. `manager-api` (管理后端 - Java Spring Boot实现)](#32-manager-api-管理后端---java-spring-boot实现)
* [3.3. `manager-web` (Web管理前端 - Vue.js实现)](#33-manager-web-web管理前端---vuejs实现)
* [3.4. `manager-mobile` (移动管理端 - uni-app+Vue3实现)](#34-manager-mobile-移动管理端---uni-appvue3实现)
* [3.5. `digital-human` (数字人测试模块 - Python+Web实现)](#35-digital-human-数字人测试模块---pythonweb实现)
4. [数据流与交互机制](#4-数据流与交互机制)
5. [核心功能概要](#5-核心功能概要)
6. [部署与配置概述](#6-部署与配置概述)
@@ -67,6 +68,13 @@
* 基于alova + @alova/adapter-uniapp实现网络请求,与manager-api无缝集成。
* 使用pinia进行状态管理,确保数据一致性。
6. **`digital-human` (数字人测试模块 - Python+Web实现):**
这是一个独立的数字人测试模块,提供本地测试页面、前端交互资源、唤醒词运行时与事件桥能力,用于联调整个数字人交互链路。其主要能力包括:
* 提供浏览器端的数字人测试页面,用于验证音频播放、接收和交互流程。
* 集成本地唤醒词运行时,支持基于Sherpa-ONNX的关键词检测。
* 通过事件桥将页面状态与本地运行时打通,便于调试唤醒词触发与交互效果。
* 作为独立模块与`xiaozhi-server`、`manager-web`、`manager-api`并列,便于单独开发和部署。
**高层交互流程概述:**
* **语音交互主线:** **ESP32设备**捕捉到用户语音后,通过**WebSocket**将音频数据实时传输给**`xiaozhi-server`**。`xiaozhi-server`完成一系列AI处理(VAD、ASR、LLM交互、TTS)后,再通过WebSocket将合成的语音回复发送回ESP32设备进行播放。所有与语音直接相关的实时交互均在此链路完成。
@@ -80,7 +88,8 @@ xiaozhi-esp32-server
├─ xiaozhi-server 8000 端口 Python语言开发 负责与esp32通信
├─ manager-web 8001 端口 Node.js+Vue开发 负责提供控制台的web界面
├─ manager-api 8002 端口 Java语言开发 负责提供控制台的api
└─ manager-mobile 跨平台移动应用 uni-app+Vue3开发 负责提供移动版智控台管理
├─ manager-mobile 跨平台移动应用 uni-app+Vue3开发 负责提供移动版智控台管理
└─ digital-human 数字人测试模块 Python+Web实现 负责本地测试页面、唤醒词运行时与事件桥
```
---
@@ -366,6 +375,48 @@ xiaozhi-esp32-server
---
### 3.5. `digital-human` (数字人测试模块 - Python+Web实现)
`digital-human` 组件是一个独立的数字人测试模块,负责提供本地测试页面、页面资源、唤醒词运行时和事件桥能力。它主要用于联调整个数字人交互链路,帮助开发者验证页面交互、音频能力和本地唤醒词流程。
* **核心目标:**
* 提供独立的本地测试页面,用于验证数字人交互效果。
* 提供本地唤醒词运行时,支持关键词检测与事件上报。
* 为前端页面和本地运行时提供事件桥,打通测试链路。
* 以独立模块形式存在,便于单独开发、调试和部署。
* **核心技术栈:**
* **Python 3:** 作为测试运行时的主要语言,负责启动本地服务、管理唤醒词运行时和事件桥。
* **原生HTML/CSS/JavaScript:** 用于构建数字人测试页面与交互逻辑。
* **Sherpa-ONNX:** 用于本地唤醒词检测。
* **ThreadingHTTPServer / WebSocket桥接:** 用于承载测试页面、健康检查和本地事件通信。
* **关键实现细节:**
1. **模块入口与页面资源:**
* `start.py` 是模块启动入口,负责初始化测试运行时。
* `index.html` 是数字人测试页面入口。
* `js/`、`css/`、`images/`、`resources/` 目录提供页面脚本、样式和资源文件。
2. **本地测试运行时:**
* `wakeword_runtime/` 目录承载本地唤醒词运行时实现。
* 运行时负责配置加载、日志初始化、音频采集、关键词检测和服务生命周期管理。
* 模块默认通过本地HTTP服务暴露页面地址、事件桥地址和健康检查接口。
3. **唤醒词检测链路:**
* 基于Sherpa-ONNX关键词检测模型,支持本地唤醒词触发。
* 模型文件和关键词配置位于 `wakeword_runtime/models/` 及相应配置文件中。
* 页面侧可结合事件桥观察唤醒词服务状态和触发结果。
4. **调试与联调定位:**
* 启动后可通过浏览器访问本地页面,验证音频播放、接收和交互逻辑。
* 通过事件桥可以将唤醒词状态同步到页面,便于排查本地链路问题。
* 详细的模型下载、运行时配置和使用说明已整理在 `docs/digital-human-wakeword.md` 中。
`digital-human` 的存在使得数字人相关能力可以脱离主服务独立验证,降低了页面调试、唤醒词联调和本地环境搭建的复杂度。
---
## 4. 数据流与交互机制
`xiaozhi-esp32-server` 系统通过各组件间定义清晰的数据流和交互协议来协同工作。主要的通信方式依赖于针对实时交互优化的WebSocket协议和适用于客户端-服务器请求的RESTful API。
+58 -1
View File
@@ -9,6 +9,7 @@
* [3.2. `manager-api` (Management Backend - Java Spring Boot Implementation)](#32-manager-api-management-backend---java-spring-boot-implementation)
* [3.3. `manager-web` (Web Management Frontend - Vue.js Implementation)](#33-manager-web-web-management-frontend---vuejs-implementation)
* [3.4. `manager-mobile` (Mobile Management Console - uni-app Implementation)](#34-manager-mobile-mobile-management-console---uniapp-implementation)
* [3.5. `digital-human` (Digital Human Test Module - Python+Web Implementation)](#35-digital-human-digital-human-test-module---pythonweb-implementation)
4. [Data Flow and Interaction Mechanisms](#4-data-flow-and-interaction-mechanisms)
5. [Key Features Summary](#5-key-features-summary)
6. [Deployment and Configuration Overview](#6-deployment-and-configuration-overview)
@@ -60,6 +61,21 @@ The `xiaozhi-esp32-server` system adopts a **distributed, multi-component collab
* (Potential functionality) Monitoring system operation status, viewing logs, troubleshooting, etc.
* Comprehensive interaction with all backend management functions provided by `manager-api`.
5. **`manager-mobile` (Mobile Management Console - uni-app Implementation):**
This is a cross-platform mobile management application based on uni-app v3 + Vue 3 + Vite, supporting App (Android & iOS) and WeChat Mini Program. Its main capabilities include:
* Providing a convenient management interface on mobile devices, similar in functionality to manager-web but optimized for mobile platforms.
* Supporting core functions such as user login, device management, and AI service configuration.
* Cross-platform adaptation, allowing a single codebase to run on iOS, Android, and WeChat Mini Programs.
* Integrating with `manager-api` through alova + `@alova/adapter-uniapp`.
* Using pinia for state management to ensure data consistency.
6. **`digital-human` (Digital Human Test Module - Python+Web Implementation):**
This is an independent digital human test module that provides a local test page, frontend interaction resources, a wake word runtime, and an event bridge for end-to-end debugging of the digital human interaction flow. Its main capabilities include:
* Providing a browser-based digital human test page for validating audio playback, audio reception, and interaction flow.
* Integrating a local wake word runtime with keyword detection based on Sherpa-ONNX.
* Bridging page state and the local runtime through an event bridge to simplify wake word debugging.
* Existing as a standalone module alongside `xiaozhi-server`, `manager-web`, and `manager-api`, making independent development and deployment easier.
**High-Level Interaction Flow Overview:**
* **Voice Interaction Main Line:** After the **ESP32 device** captures user voice, it transmits audio data in real-time to **`xiaozhi-server`** through **WebSocket**. After `xiaozhi-server` completes a series of AI processing (VAD, ASR, LLM interaction, TTS), it sends the synthesized voice response back to the ESP32 device for playback through WebSocket. All real-time interactions directly related to voice are completed in this link.
@@ -73,7 +89,8 @@ xiaozhi-esp32-server
├─ xiaozhi-server Port 8000 Python development Responsible for ESP32 communication
├─ manager-web Port 8001 Node.js+Vue development Responsible for providing web interface for console
├─ manager-api Port 8002 Java development Responsible for providing console API
└─ manager-mobile Cross-platform mobile application uni-app+Vue3 development Responsible for providing mobile console management
├─ manager-mobile Cross-platform mobile application uni-app+Vue3 development Responsible for providing mobile console management
└─ digital-human Digital human test module Python+Web implementation Responsible for local test page, wake word runtime, and event bridge
```
---
@@ -257,6 +274,46 @@ The `manager-mobile` component is a cross-platform mobile management application
`manager-mobile` provides users with a fully functional, smooth mobile management tool through the application of these technologies, allowing administrators to perform system management and configuration anytime, anywhere.
### 3.5. `digital-human` (Digital Human Test Module - Python+Web Implementation)
The `digital-human` component is an independent digital human test module responsible for providing a local test page, page resources, a wake word runtime, and an event bridge. It is primarily used to debug the complete digital human interaction flow and to validate frontend interaction, audio capability, and local wake word behavior.
* **Core Objectives:**
* Provide an independent local test page for validating digital human interaction effects.
* Provide a local wake word runtime for keyword detection and event reporting.
* Provide an event bridge between the frontend page and the local runtime to connect the test flow.
* Exist as a standalone module for easier independent development, debugging, and deployment.
* **Core Technologies:**
* **Python 3:** Used for the local test runtime, service startup, wake word lifecycle management, and event bridge coordination.
* **Native HTML/CSS/JavaScript:** Used to build the digital human test page and interaction logic.
* **Sherpa-ONNX:** Used for local wake word keyword detection.
* **ThreadingHTTPServer / WebSocket bridge:** Used to serve the local page, health check endpoint, and event communication.
* **Key Implementation Aspects:**
1. **Module Entry and Page Resources:**
* `start.py` is the module entry point responsible for starting the local test runtime.
* `index.html` is the entry page for digital human testing.
* The `js/`, `css/`, `images/`, and `resources/` directories provide frontend scripts, styles, and static resources.
2. **Local Test Runtime:**
* The `wakeword_runtime/` directory contains the local wake word runtime implementation.
* The runtime is responsible for configuration loading, logging initialization, audio capture, keyword detection, and lifecycle management.
* The module exposes a local page URL, event bridge URL, and health check endpoint through a local HTTP service.
3. **Wake Word Detection Flow:**
* Uses Sherpa-ONNX keyword detection models to support local wake word triggering.
* Model files and keyword configurations are stored in `wakeword_runtime/models/` and related configuration files.
* The page can observe runtime state and trigger results through the event bridge.
4. **Debugging and Integration Positioning:**
* After startup, developers can open the local page in a browser to validate playback, reception, and interaction logic.
* The event bridge synchronizes wake word state to the page, which helps with local debugging.
* Detailed model download, runtime configuration, and usage instructions are documented in `docs/digital-human-wakeword.md`.
The `digital-human` module allows digital human related capabilities to be validated independently from the main service, reducing the complexity of page debugging, wake word integration, and local environment setup.
---
## 4. Data Flow and Interaction Mechanisms
+46
View File
@@ -0,0 +1,46 @@
本文档是开发类文档,如需部署小智服务端,[点击这里查看部署教程](../../README.md#%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3)
如需查看一体机数字人落地部署、Kiosk 全屏启动和系统环境配置,[点击这里查看一体机部署指南](../../docs/all-in-one-digital-human-setup.md)
如需查看唤醒词模型下载、运行时配置和详细使用说明,[点击这里查看唤醒词专题文档](../../docs/digital-human-wakeword.md)
# 项目介绍
digital-human 是独立的数字人测试模块,负责提供本地测试页面、前端交互资源、唤醒词运行时和事件桥能力,用于联调整个数字人交互链路。
# 快速启动
安装依赖:
```bash
pip install -r wakeword_runtime/requirements.txt
```
启动模块:
```bash
python start.py
```
# 访问地址
启动后可访问:
- 页面地址:http://127.0.0.1:8006/index.html
- 事件桥地址:ws://127.0.0.1:8006/wakeword-ws
- 健康检查:http://127.0.0.1:8006/health
# 目录说明
- `start.py`:模块启动入口
- `index.html`:数字人测试页面入口
- `wakeword_runtime`:本地唤醒词运行时与配置目录
- `js`、`css`:页面前端脚本与样式
- `images`、`resources`:页面资源文件
# 相关文档
- 一体机部署指南:适用于 x86 设备整机落地部署、Kiosk 展示和开机自启动配置
[../../docs/all-in-one-digital-human-setup.md](../../docs/all-in-one-digital-human-setup.md)
- 唤醒词专题文档:适用于唤醒词模型下载、运行时配置和本地调试说明
[../../docs/digital-human-wakeword.md](../../docs/digital-human-wakeword.md)
File renamed without changes.

Before

Width:  |  Height:  |  Size: 185 B

After

Width:  |  Height:  |  Size: 185 B

@@ -812,6 +812,20 @@ body {
font-size: 14px;
}
.config-item textarea {
width: 100%;
padding: 12px;
border: 1px solid #40444b;
border-radius: 6px;
background: #40444b;
color: white;
min-height: 100px;
resize: vertical;
box-sizing: border-box;
font-family: inherit;
font-size: 14px;
}
.model-select {
width: 100%;
padding: 10px 40px 10px 14px;
File renamed without changes.

Before

Width:  |  Height:  |  Size: 4.2 KiB

After

Width:  |  Height:  |  Size: 4.2 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 135 KiB

After

Width:  |  Height:  |  Size: 135 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 321 KiB

After

Width:  |  Height:  |  Size: 321 KiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 290 KiB

After

Width:  |  Height:  |  Size: 290 KiB

@@ -4,8 +4,8 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>小智服务器测试页面</title>
<link rel="stylesheet" href="css/test_page.css?v=0205">
<title>小智数字人页面</title>
<link rel="stylesheet" href="css/index.css?v=0206">
<script>
// 检测是否使用file://协议打开
if (window.location.protocol === 'file:') {
@@ -21,14 +21,14 @@
warningDiv.innerHTML = `
<h2>⚠️ 警告:请使用HTTP服务器打开此页面</h2>
<p>您当前使用的是本地文件方式打开页面(file://协议),这可能导致页面功能异常。</p>
<p>您可以使用nginx映射启动测试页面,也可以请按照以下步骤使用python启动测试http服务:</p>
<p>您可以使用nginx映射启动小智数字人页面,也可以请按照以下步骤启动测试运行时:</p>
<ol>
<li>打开命令行终端</li>
<li>命令行进入到 xiaozhi-server/test 目录</li>
<li>执行以下命令启动HTTP服务器:</li>
<li>命令行进入到 digital-human 目录</li>
<li>执行以下命令启动小智数字人页面运行时:</li>
</ol>
<pre>python -m http.server 8006</pre>
<p>然后在浏览器中访问:<strong>http://localhost:8006/test_page.html</strong></p>
<pre>python start.py</pre>
<p>然后在浏览器中访问:<strong>http://localhost:8006/index.html</strong></p>
`;
document.body.appendChild(warningDiv);
});
@@ -137,6 +137,7 @@
<div class="modal-body">
<div class="settings-tabs">
<button class="tab-btn active" data-tab="device">设备配置</button>
<button class="tab-btn" data-tab="wakeword">唤醒词</button>
<button class="tab-btn" data-tab="mcp">MCP工具</button>
<button class="tab-btn" data-tab="other">数字人皮肤</button>
</div>
@@ -162,6 +163,15 @@
placeholder="deviceName">
</div>
</div>
<div class="config-row">
<div class="config-item">
<label for="emojiEnabled">情绪表情:</label>
<select id="emojiEnabled" class="model-select">
<option value="true">启用</option>
<option value="false" selected>禁用</option>
</select>
</div>
</div>
</div>
</div>
@@ -184,6 +194,37 @@
</div>
</div>
<div class="tab-content" id="wakewordTab">
<div class="config-panel">
<div class="control-panel">
<div class="config-row">
<div class="config-item">
<label for="wakewordEnabled">启用本地唤醒词:</label>
<select id="wakewordEnabled" class="model-select">
<option value="true">启用</option>
<option value="false">禁用</option>
</select>
</div>
</div>
<div class="config-row">
<div class="config-item">
<label for="wakewordWsUrl">唤醒词服务地址:</label>
<input type="text" id="wakewordWsUrl" placeholder="ws://127.0.0.1:8006/wakeword-ws">
</div>
</div>
<div class="config-row">
<div class="config-item">
<label for="wakewordList">唤醒词配置 (一行一个):</label>
<textarea id="wakewordList" placeholder="例如:&#10;小智小智&#10;你好小智"></textarea>
</div>
</div>
<div class="modal-actions">
<button type="button" class="btn-primary" id="applyWakewordBtn">应用唤醒词</button>
</div>
</div>
</div>
</div>
<div class="tab-content" id="mcpTab">
<!-- MCP 工具管理区域 -->
<div class="mcp-tools-container">
@@ -3,6 +3,7 @@ import { checkOpusLoaded, initOpusEncoder } from './core/audio/opus-codec.js?v=0
import { getAudioPlayer } from './core/audio/player.js?v=0205';
import { checkMicrophoneAvailability, isHttpNonLocalhost } from './core/audio/recorder.js?v=0205';
import { initMcpTools } from './core/mcp/tools.js?v=0205';
import { startWakewordBridgeListener } from './core/network/wakeword-bridge.js?v=0205';
import { uiController } from './ui/controller.js?v=0205';
import { log } from './utils/logger.js?v=0205';
@@ -43,6 +44,8 @@ class App {
await this.audioPlayer.start();
// 初始化MCP工具
initMcpTools();
// 初始化本地唤醒事件监听
startWakewordBridgeListener();
// 检查麦克风可用性
await this.checkMicrophoneAvailability();
// 检查摄像头可用性
@@ -192,7 +195,7 @@ class App {
}
log('正在请求摄像头权限...', 'info');
this.cameraStream = await navigator.mediaDevices.getUserMedia({
video: { width: 320, height: 240, facingMode: this.currentFacingMode },
video: { width: 180, height: 240, facingMode: this.currentFacingMode },
audio: false
});
cameraVideo.srcObject = this.cameraStream;
@@ -274,7 +277,7 @@ class App {
return;
}
canvas.width = video.videoWidth || 320;
canvas.width = video.videoWidth || 180;
canvas.height = video.videoHeight || 240;
const ctx = canvas.getContext('2d');
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
@@ -1,5 +1,8 @@
// 配置管理模块
// 默认唤醒词列表
export const DEFAULT_WAKE_WORDS = '你好小智\n你好小志\n小爱同学\n你好小鑫\n你好小新\n小美同学\n小龙小龙\n喵喵同学\n小滨小滨\n小冰小冰\n嘿你好呀';
// 生成随机MAC地址
function generateRandomMac() {
const hexDigits = '0123456789ABCDEF';
@@ -19,6 +22,9 @@ export function loadConfig() {
const deviceNameInput = document.getElementById('deviceName');
const clientIdInput = document.getElementById('clientId');
const otaUrlInput = document.getElementById('otaUrl');
const wakewordWsUrlInput = document.getElementById('wakewordWsUrl');
const wakewordEnabledInput = document.getElementById('wakewordEnabled');
const wakewordListInput = document.getElementById('wakewordList');
// 从localStorage加载MAC地址,如果没有则生成新的
let savedMac = localStorage.getItem('xz_tester_deviceMac');
@@ -43,6 +49,29 @@ export function loadConfig() {
if (savedOtaUrl) {
otaUrlInput.value = savedOtaUrl;
}
const savedWakewordWsUrl = localStorage.getItem('xz_tester_wakewordWsUrl');
if (savedWakewordWsUrl !== null && wakewordWsUrlInput) {
wakewordWsUrlInput.value = savedWakewordWsUrl;
}
const savedWakewordEnabled = localStorage.getItem('xz_tester_wakewordEnabled');
if (savedWakewordEnabled !== null && wakewordEnabledInput) {
wakewordEnabledInput.value = savedWakewordEnabled;
}
const savedWakewordList = localStorage.getItem('xz_tester_wakewordList');
if (savedWakewordList !== null && wakewordListInput) {
wakewordListInput.value = savedWakewordList;
} else if (wakewordListInput) {
wakewordListInput.value = DEFAULT_WAKE_WORDS;
}
const emojiEnabledInput = document.getElementById('emojiEnabled');
const savedEmojiEnabled = localStorage.getItem('xz_tester_emojiEnabled');
if (savedEmojiEnabled !== null && emojiEnabledInput) {
emojiEnabledInput.value = savedEmojiEnabled;
}
}
// 保存配置
@@ -50,10 +79,26 @@ export function saveConfig() {
const deviceMacInput = document.getElementById('deviceMac');
const deviceNameInput = document.getElementById('deviceName');
const clientIdInput = document.getElementById('clientId');
const wakewordWsUrlInput = document.getElementById('wakewordWsUrl');
const wakewordEnabledInput = document.getElementById('wakewordEnabled');
const wakewordListInput = document.getElementById('wakewordList');
localStorage.setItem('xz_tester_deviceMac', deviceMacInput.value);
localStorage.setItem('xz_tester_deviceName', deviceNameInput.value);
localStorage.setItem('xz_tester_clientId', clientIdInput.value);
const emojiEnabledInput = document.getElementById('emojiEnabled');
if (emojiEnabledInput) {
localStorage.setItem('xz_tester_emojiEnabled', emojiEnabledInput.value);
}
if (wakewordEnabledInput) {
localStorage.setItem('xz_tester_wakewordEnabled', wakewordEnabledInput.value);
}
if (wakewordListInput) {
localStorage.setItem('xz_tester_wakewordList', wakewordListInput.value);
}
if (wakewordWsUrlInput && wakewordWsUrlInput.value.trim()) {
localStorage.setItem('xz_tester_wakewordWsUrl', wakewordWsUrlInput.value.trim());
}
}
// 获取配置值
@@ -62,12 +107,14 @@ export function getConfig() {
const deviceMac = document.getElementById('deviceMac')?.value.trim() || '';
const deviceName = document.getElementById('deviceName')?.value.trim() || '';
const clientId = document.getElementById('clientId')?.value.trim() || '';
const emojiEnabled = document.getElementById('emojiEnabled')?.value !== 'false';
return {
deviceId: deviceMac, // 使用MAC地址作为deviceId
deviceName,
deviceMac,
clientId
clientId,
emojiEnabled
};
}
@@ -203,17 +203,6 @@ export class AudioRecorder {
async start() {
if (this.isRecording) return false;
try {
// Check if WebSocketHandler instance exists
const { getWebSocketHandler } = await import('../network/websocket.js?v=0205');
const wsHandler = getWebSocketHandler();
// If machine is speaking, send abort message
if (wsHandler && wsHandler.isRemoteSpeaking && wsHandler.currentSessionId) {
const abortMessage = { session_id: wsHandler.currentSessionId, type: 'abort', reason: 'wake_word_detected' };
if (this.websocket && this.websocket.readyState === WebSocket.OPEN) {
this.websocket.send(JSON.stringify(abortMessage));
log('已发送中止消息', 'info');
}
}
if (!this.initEncoder()) {
log('无法开始录音: Opus编码器初始化失败', 'error');
return false;
@@ -0,0 +1,225 @@
import { uiController } from '../../ui/controller.js?v=0205';
import { log } from '../../utils/logger.js?v=0205';
let wakewordSocket = null;
let reconnectTimer = null;
let reconnectAttempts = 0;
let shouldReconnect = true;
let wakewordRequestSeq = 0;
let onNextBridgeConnectedCallback = null;
const pendingWakewordRequests = new Map();
export function startWakewordBridgeListener() {
if (wakewordSocket) {
return wakewordSocket;
}
shouldReconnect = true;
log('正在连接本地唤醒事件桥...', 'info');
tryConnect();
return wakewordSocket;
}
function tryConnect() {
const bridgeUrl = buildWakewordBridgeUrl();
try {
wakewordSocket = new WebSocket(bridgeUrl);
wakewordSocket.onopen = () => {
reconnectAttempts = 0;
log(`本地唤醒事件桥已连接: ${bridgeUrl}`, 'success');
// 连接成功后自动保存地址,刷新后仍能记住
localStorage.setItem('xz_tester_wakewordWsUrl', bridgeUrl);
const urlInput = document.getElementById('wakewordWsUrl');
if (urlInput) urlInput.value = bridgeUrl;
};
wakewordSocket.onerror = () => {
log(`本地唤醒事件桥连接失败: ${bridgeUrl}`, 'error');
};
wakewordSocket.onmessage = async (event) => {
try {
const message = parseWakewordBridgeMessage(event.data);
if (message.requestId && pendingWakewordRequests.has(message.requestId)) {
settleWakewordRequest(message);
return;
}
if (message.success === false) {
log(`本地唤醒事件桥返回错误: ${message.error || '未知错误'}`, 'error');
return;
}
if (message.type === 'bridge_connected') {
log('本地唤醒监听已就绪', 'info');
if (onNextBridgeConnectedCallback) {
const cb = onNextBridgeConnectedCallback;
onNextBridgeConnectedCallback = null;
cb(message);
}
return;
}
if (message.type === 'service_ready') {
log('本地唤醒服务已启动', 'info');
return;
}
if (message.type === 'wakeword_config') {
uiController.applyWakewordConfig(message.payload || {});
log('已同步本地唤醒词配置', 'info');
return;
}
if (message.type === 'service_stopping') {
log('本地唤醒服务正在停止', 'warning');
return;
}
if (message.type === 'wake_word_detected') {
const wakeWord = message.payload?.wake_word || '唤醒词';
log(`检测到本地唤醒事件: ${wakeWord}`, 'info');
await uiController.triggerWakewordDial(wakeWord);
}
} catch (error) {
log(`解析本地唤醒事件失败: ${error.message}`, 'error');
}
};
wakewordSocket.onclose = () => {
if (wakewordSocket) {
wakewordSocket = null;
}
rejectAllWakewordRequests('本地唤醒事件桥已断开');
if (!shouldReconnect) {
return;
}
if (reconnectTimer) {
return;
}
reconnectAttempts += 1;
const delay = Math.min(1000 * reconnectAttempts, 5000);
log(`本地唤醒事件桥将在 ${delay}ms 后重连: ${bridgeUrl}`, 'warning');
reconnectTimer = window.setTimeout(() => {
reconnectTimer = null;
tryConnect();
}, delay);
};
return wakewordSocket;
} catch (error) {
log(`启动本地唤醒监听失败: ${error.message}`, 'error');
return null;
}
}
export function stopWakewordBridgeListener() {
shouldReconnect = false;
if (reconnectTimer) {
window.clearTimeout(reconnectTimer);
reconnectTimer = null;
}
if (!wakewordSocket) {
return;
}
wakewordSocket.onclose = null;
wakewordSocket.close();
wakewordSocket = null;
}
export function sendWakewordBridgeMessage(type, payload = {}, requestId = null) {
if (!wakewordSocket || wakewordSocket.readyState !== WebSocket.OPEN) {
log('本地唤醒事件桥未连接,无法发送消息', 'warning');
return false;
}
wakewordSocket.send(JSON.stringify({
type,
requestId,
payload,
}));
return true;
}
export function requestWakewordBridge(type, payload = {}, timeout = 5000) {
const requestId = `wakeword-${Date.now()}-${++wakewordRequestSeq}`;
return new Promise((resolve, reject) => {
const timer = window.setTimeout(() => {
pendingWakewordRequests.delete(requestId);
reject(new Error('本地唤醒服务响应超时'));
}, timeout);
pendingWakewordRequests.set(requestId, { resolve, reject, timer });
if (!sendWakewordBridgeMessage(type, payload, requestId)) {
window.clearTimeout(timer);
pendingWakewordRequests.delete(requestId);
reject(new Error('本地唤醒事件桥未连接'));
}
});
}
export function getWakewordBridgeUrl() {
if (wakewordSocket && wakewordSocket.url) {
return wakewordSocket.url;
}
return buildWakewordBridgeUrl();
}
export function onNextBridgeConnected(callback) {
onNextBridgeConnectedCallback = callback;
}
function buildWakewordBridgeUrl() {
const configured = localStorage.getItem('xz_tester_wakewordWsUrl');
if (configured && configured.trim()) {
return configured.trim();
}
return 'ws://127.0.0.1:8006/wakeword-ws';
}
function parseWakewordBridgeMessage(rawData) {
const message = JSON.parse(rawData);
return {
type: message.type || '',
requestId: message.requestId || null,
success: message.success !== false,
payload: message.payload || {},
error: message.error || null,
};
}
function settleWakewordRequest(message) {
const pendingRequest = pendingWakewordRequests.get(message.requestId);
if (!pendingRequest) {
return;
}
window.clearTimeout(pendingRequest.timer);
pendingWakewordRequests.delete(message.requestId);
if (message.success === false) {
pendingRequest.reject(new Error(message.error || '本地唤醒服务返回失败'));
return;
}
pendingRequest.resolve(message);
}
function rejectAllWakewordRequests(errorMessage) {
pendingWakewordRequests.forEach((pendingRequest) => {
window.clearTimeout(pendingRequest.timer);
pendingRequest.reject(new Error(errorMessage));
});
pendingWakewordRequests.clear();
}
@@ -34,7 +34,8 @@ export class WebSocketHandler {
device_mac: config.deviceMac,
token: config.token,
features: {
mcp: true
mcp: true,
emoji: config.emojiEnabled
}
};
@@ -70,6 +71,28 @@ export class WebSocketHandler {
}
}
_sendWakeupMessages(sessionId) {
if (!this.websocket || this.websocket.readyState !== WebSocket.OPEN) return;
// listen detect
this.websocket.send(JSON.stringify({
session_id: sessionId,
type: 'listen',
state: 'detect',
text: '嘿,你好呀'
}));
log('发送listen detect消息,唤醒词: 嘿,你好呀', 'info');
// listen start:开始监听
this.websocket.send(JSON.stringify({
session_id: sessionId,
type: 'listen',
state: 'start',
mode: 'auto'
}));
log('发送listen start消息', 'info');
}
// 处理文本消息
handleTextMessage(message) {
if (message.type === 'hello') {
@@ -77,6 +100,9 @@ export class WebSocketHandler {
window.cameraAvailable = true;
log('连接成功,摄像头已可用', 'success');
uiController.updateDialButton(true);
this._sendWakeupMessages(message.session_id);
uiController.startAIChatSession();
} else if (message.type === 'tts') {
this.handleTTSMessage(message);
@@ -2,7 +2,9 @@
import { loadConfig, saveConfig } from '../config/manager.js?v=0205';
import { getAudioPlayer } from '../core/audio/player.js?v=0205';
import { getAudioRecorder } from '../core/audio/recorder.js?v=0205';
import { requestWakewordBridge, stopWakewordBridgeListener, startWakewordBridgeListener, getWakewordBridgeUrl, onNextBridgeConnected } from '../core/network/wakeword-bridge.js?v=0205';
import { getWebSocketHandler } from '../core/network/websocket.js?v=0205';
import { log } from '../utils/logger.js?v=0205';
// UI controller class
class UIController {
@@ -14,6 +16,8 @@ class UIController {
this.currentBackgroundIndex = localStorage.getItem('backgroundIndex') ? parseInt(localStorage.getItem('backgroundIndex')) : 0;
this.backgroundImages = ['1.png', '2.png', '3.png'];
this.dialBtnDisabled = false;
this.isConnecting = false;
this.lastWakewordDialTime = 0;
// Bind methods
this.init = this.init.bind(this);
@@ -25,6 +29,9 @@ class UIController {
this.showModal = this.showModal.bind(this);
this.hideModal = this.hideModal.bind(this);
this.switchTab = this.switchTab.bind(this);
this.applyWakewordConfig = this.applyWakewordConfig.bind(this);
this.handleApplyWakeword = this.handleApplyWakeword.bind(this);
this.triggerWakewordDial = this.triggerWakewordDial.bind(this);
}
// Initialize
@@ -262,6 +269,11 @@ class UIController {
});
});
const applyWakewordBtn = document.getElementById('applyWakewordBtn');
if (applyWakewordBtn) {
applyWakewordBtn.addEventListener('click', this.handleApplyWakeword);
}
// 点击模态框背景关闭(仅对特定模态框禁用此功能)
const modals = document.querySelectorAll('.modal');
modals.forEach(modal => {
@@ -512,6 +524,121 @@ class UIController {
}
}
applyWakewordConfig(config = {}) {
const wakewordEnabledInput = document.getElementById('wakewordEnabled');
const wakewordListInput = document.getElementById('wakewordList');
if (!wakewordEnabledInput || !wakewordListInput) {
return;
}
const wakeWords = Array.isArray(config.wakeWords)
? config.wakeWords.filter(item => typeof item === 'string' && item.trim())
: [];
wakewordEnabledInput.value = config.enabled === false ? 'false' : 'true';
wakewordListInput.value = wakeWords.join('\n');
saveConfig();
}
async handleApplyWakeword() {
const wakewordEnabledInput = document.getElementById('wakewordEnabled');
const wakewordListInput = document.getElementById('wakewordList');
const wakewordWsUrlInput = document.getElementById('wakewordWsUrl');
if (!wakewordEnabledInput || !wakewordListInput) {
return;
}
const wakeWords = wakewordListInput.value
.split(/\r?\n/u)
.map(item => item.trim())
.filter(Boolean)
.filter((item, index, items) => items.indexOf(item) === index);
const payload = {
enabled: wakewordEnabledInput.value !== 'false',
wakeWords,
};
if (payload.enabled && payload.wakeWords.length === 0) {
this.addChatMessage('启用唤醒词时,至少需要填写一个唤醒词。', false);
return;
}
const applyWakewordBtn = document.getElementById('applyWakewordBtn');
if (applyWakewordBtn) {
applyWakewordBtn.disabled = true;
applyWakewordBtn.textContent = '应用中...';
}
try {
// 保存地址到 localStorage
if (wakewordWsUrlInput && wakewordWsUrlInput.value.trim()) {
localStorage.setItem('xz_tester_wakewordWsUrl', wakewordWsUrlInput.value.trim());
}
// 比较新地址和当前连接地址
const newWsUrl = localStorage.getItem('xz_tester_wakewordWsUrl');
const currentWsUrl = getWakewordBridgeUrl();
const urlChanged = newWsUrl !== currentWsUrl;
if (urlChanged) {
// 地址变了,先确认
const shouldRestart = window.confirm('地址已变更,是否继续?(将断开旧连接并重新连接)');
if (!shouldRestart) {
// 恢复 localStorage 里的旧地址
localStorage.setItem('xz_tester_wakewordWsUrl', currentWsUrl);
this.addChatMessage('已取消地址变更。', false);
return;
}
// 断开旧连接
stopWakewordBridgeListener();
// 启动新连接(自动用新地址)
startWakewordBridgeListener();
// 等 bridge_connected
await new Promise((resolve, reject) => {
const timeout = setTimeout(() => {
reject(new Error('连接新服务器超时'));
}, 5000);
onNextBridgeConnected(() => {
clearTimeout(timeout);
resolve();
});
});
// 发配置和重启
await requestWakewordBridge('set_wakeword_config', payload);
this.applyWakewordConfig(payload);
await requestWakewordBridge('restart_wakeword_service');
this.addChatMessage('唤醒词配置已保存,唤醒词服务正在重启。', false);
} else {
// 地址没变:直接在当前连接操作
const response = await requestWakewordBridge('set_wakeword_config', payload);
this.applyWakewordConfig(response.payload || payload);
const shouldRestart = window.confirm('唤醒词已保存。是否现在重启唤醒词服务以立即生效?');
if (!shouldRestart) {
this.addChatMessage('唤醒词配置已保存,可稍后手动重启服务后生效。', false);
return;
}
await requestWakewordBridge('restart_wakeword_service');
this.addChatMessage('唤醒词配置已保存,唤醒词服务正在重启。', false);
}
} catch (error) {
this.addChatMessage(`应用唤醒词失败: ${error.message}`, false);
} finally {
if (applyWakewordBtn) {
applyWakewordBtn.disabled = false;
applyWakewordBtn.textContent = '应用唤醒词';
}
}
}
// Start AI chat session after connection
startAIChatSession() {
this.addChatMessage('连接成功,开始聊天吧~😊', false);
@@ -550,47 +677,51 @@ class UIController {
// Handle connect button click
async handleConnect() {
console.log('handleConnect called');
// Switch to device settings tab
this.switchTab('device');
// Wait for DOM update
await new Promise(resolve => setTimeout(resolve, 50));
const otaUrlInput = document.getElementById('otaUrl');
console.log('otaUrl element:', otaUrlInput);
if (!otaUrlInput || !otaUrlInput.value) {
this.addChatMessage('请输入OTA服务器地址', false);
const wsHandler = getWebSocketHandler();
if (this.isConnecting || (wsHandler && wsHandler.isConnected())) {
log('连接已存在或正在进行,忽略本次拨号请求', 'info');
return;
}
const otaUrl = otaUrlInput.value;
console.log('otaUrl value:', otaUrl);
// Update dial button state to connecting
const dialBtn = document.getElementById('dialBtn');
if (dialBtn) {
dialBtn.classList.add('dial-active');
dialBtn.querySelector('.btn-text').textContent = '连接中...';
dialBtn.disabled = true;
}
// Show connecting message
this.addChatMessage('正在连接服务器...', false);
const chatIpt = document.getElementById('chatIpt');
if (chatIpt) {
chatIpt.style.display = 'flex';
}
this.isConnecting = true;
console.log('handleConnect called');
try {
// Switch to device settings tab
this.switchTab('device');
// Wait for DOM update
await new Promise(resolve => setTimeout(resolve, 50));
const otaUrlInput = document.getElementById('otaUrl');
console.log('otaUrl element:', otaUrlInput);
if (!otaUrlInput || !otaUrlInput.value) {
this.addChatMessage('请输入OTA服务器地址', false);
return;
}
const otaUrl = otaUrlInput.value;
console.log('otaUrl value:', otaUrl);
// Update dial button state to connecting
const dialBtn = document.getElementById('dialBtn');
if (dialBtn) {
dialBtn.classList.add('dial-active');
dialBtn.querySelector('.btn-text').textContent = '连接中...';
dialBtn.disabled = true;
}
// Show connecting message
this.addChatMessage('正在连接服务器...', false);
const chatIpt = document.getElementById('chatIpt');
if (chatIpt) {
chatIpt.style.display = 'flex';
}
// Get WebSocket handler instance
const wsHandler = getWebSocketHandler();
// Register connection state callback BEFORE connecting
wsHandler.onConnectionStateChange = (isConnected) => {
this.updateConnectionUI(isConnected);
@@ -670,9 +801,36 @@ class UIController {
dialBtn.classList.remove('dial-active');
console.log('Dial button state restored successfully');
}
} finally {
this.isConnecting = false;
}
}
async triggerWakewordDial(wakeWord = '唤醒词') {
const wsHandler = getWebSocketHandler();
const now = Date.now();
if (wsHandler && wsHandler.isConnected()) {
log('页面已连接,忽略自动拨号', 'info');
return false;
}
if (this.isConnecting || this.dialBtnDisabled) {
log('页面正在连接中,忽略重复唤醒', 'info');
return false;
}
if (now - this.lastWakewordDialTime < 3000) {
log('唤醒触发过于频繁,忽略本次自动拨号', 'warning');
return false;
}
this.lastWakewordDialTime = now;
this.addChatMessage(`检测到唤醒词“${wakeWord}”,准备连接服务器...`, false);
await this.handleConnect();
return true;
}
// Add MCP tool
addMCPTool() {
const mcpToolsList = document.getElementById('mcpToolsList');
File renamed without changes.

Before

Width:  |  Height:  |  Size: 1.7 MiB

After

Width:  |  Height:  |  Size: 1.7 MiB

File renamed without changes.

Before

Width:  |  Height:  |  Size: 2.4 MiB

After

Width:  |  Height:  |  Size: 2.4 MiB

Loaded 100 of 752 files, more files were not shown because too many files have changed in this diff. Show more