Compare commits

...
Author SHA1 Message Date
hrz 1a4b8d76c5 TTS 用例说明里的名字改成真存在的
说明里写的那个用例名仓库里没有,改成实际跑的那条。
2026-10-24 08:06:12 +00:00
hrz 57dffb23e9 三条 LLM 用例改成查请求体
原来只看回应非空,provider 把消息丢掉、自己编一句回来,测试照样绿。现在检查请求里带了模型名和这次对话,回的内容也要跟假接口给的一致。
2026-10-24 08:06:12 +00:00
hrz 379185caed 盯住连接模块里那次插件自动导入
原来两条用例自己调自动导入,连接模块里那行被删掉它们照样绿。加一条查源码的用例,盯住模块加载时那次调用本身还在。
2026-10-24 08:06:12 +00:00
hrz 03b7452daf CI 里缺东西直接失败,不许静默跳过
测试里原来有几处配置缺失就跳过的分支,CI 上会静悄悄跳掉,门禁看着是绿的其实没跑。加了个统一的守卫:本地缺东西照旧跳过,CI 里直接失败,口径跟 manager-api 的集成测试守卫一样。
2026-10-24 07:31:19 +00:00
hrz aff47ee6de 要真凭据的用例打标记统一剔除
Edge TTS 那条要连微软公网,runner 的出口 IP 会被风控挡掉。以前在 CI 命令行上用 -k 排除,看不出排掉了什么。现在给它打 live_api 标记,由 conftest 统一剔除,日志里列出剔的是哪几条。
2026-10-24 07:27:15 +00:00
hrz e3e043a9ed 几组用例不再整份跳过,真跑起来
插件自动导入、三个 create_instance、LLM 链路这几组用例以前整份跳过,现在都真跑。LLM 那边装了个假接口,provider 拼请求和解析流式回应那套照跑,不发真请求也不用真 key。data/.config.yaml 缺的时候现造一个,免得用例收集不起来。
2026-10-24 07:27:06 +00:00
hrz b3810fb7cb 本地没起库也能构建,CI 里集成测试不许跳过
三个要连 MySQL 和 Redis 的集成测试加了守卫:依赖连不上就整类跳过,日志写明缺什么,构建照样是绿的;CI 上依赖不通就直接失败。另外加了一个门禁检查脚本和 CI 步骤,盯住这三个类有没有真跑,守卫哪天手滑把它们全跳过了,CI 必须变红。README 里补了本地怎么起依赖、怎么只编译不跑测试。
2026-10-24 06:30:22 +00:00
hrz 4eb586b597 给 runner 补上 libopus,修掉音频用例报错
opuslib_next 是 ctypes 绑定,真正编解码靠系统里的 libopus。runner 上默认
没有,import 失败后测试里换上的是自己写的桩,桩里的编码器只有构造方法,
一调就报错。正式镜像里是装了这个库的,CI 少了这一步。

顺手把桩的报错改成一句人话,别再让人对着缺方法的报错猜根因。
2026-10-24 00:59:59 +00:00
hrz 49c908db56 让 CI 真正跑起三类测试并修掉挡路的测试缺陷
之前 CI 只跑了一小部分,Java 那边的测试被 pom 里的开关整个跳过了。
这条把它打开,并补上 MySQL、Redis 两个容器,Java、契约、Python 三类
测试都在 CI 里真跑起来。

另外修了两个挡路的地方:一条 Java 用例单跑会因为实体信息没注册而报错,
现在自己注册;Python 的假网络层把整个 httpx 类换掉了,第三方库拼请求
时报错,改成只换发请求那一步。
2026-10-24 00:46:53 +00:00
15 changed files with 606 additions and 89 deletions

No files matched your search

+49 -8
View File
@@ -20,24 +20,58 @@ jobs:
python-version: '3.10'
cache: 'pip'
cache-dependency-path: 'main/xiaozhi-server/requirements.txt'
- name: Install system dependencies
run: |
# opuslib_next 是 ctypes 绑定,pip 装上的是纯 Python 包,真正的编解码
# 靠系统里的 libopus.so。runner 上默认没有,import 会失败,音频相关的
# 用例直接报错。正式镜像 Dockerfile-server-base 里也是装 libopus0 的
sudo apt-get update
sudo apt-get install -y --no-install-recommends libopus0
- name: Install dependencies
working-directory: main/xiaozhi-server
run: |
pip install -r requirements.txt
pip install pytest pytest-asyncio freezegun
- name: Run smoke test
- name: Prepare test config
working-directory: main/xiaozhi-server
run: |
if [ -f tests/test_smoke.py ]; then
pytest tests/test_smoke.py -x -q
else
echo "::warning::Slice 2 not done yet — tests/test_smoke.py missing"
exit 0
fi
# config/settings.py 在 import 期就要这个文件,缺了整套用例连收集都过不去。
# data/ 在 .gitignore 里,只能由这一步现造
mkdir -p data
printf '{}\n' > data/.config.yaml
- name: Run tests
working-directory: main/xiaozhi-server
run: |
# 要真凭据/真网络的用例由 tests/conftest.py 按 live_api 标记统一剔除,
# 剔掉的是哪几条会打在日志里
pytest tests/ -v
java:
name: Java (manager-api)
runs-on: ubuntu-latest
# 三个 @SpringBootTest 要真连 MySQL 和 Redis,空库即可,Liquibase 会自己建表
services:
mysql:
image: mysql:8.0
env:
MYSQL_ROOT_PASSWORD: 123456
MYSQL_DATABASE: xiaozhi_esp32_server
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping -h 127.0.0.1 -uroot -p123456"
--health-interval=10s
--health-timeout=5s
--health-retries=20
redis:
image: redis:8.0
ports:
- 6379:6379
options: >-
--health-cmd="redis-cli ping"
--health-interval=10s
--health-timeout=5s
--health-retries=20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
@@ -47,7 +81,12 @@ jobs:
cache: 'maven'
- name: Run tests
working-directory: main/manager-api
run: mvn -B -q test -DfailIfNoTests=false
run: mvn -B test -DskipTests=false -DfailIfNoTests=false
- name: Check the integration tests really ran
working-directory: main/manager-api
# 守卫在依赖不齐时会把整类集成测试跳过,这里盯一下它们有没有真跑,
# 免得门禁被悄悄掏空
run: bash scripts/check-integration-tests.sh
web:
name: Vue (manager-web)
@@ -63,4 +102,6 @@ jobs:
working-directory: main/manager-web
run: |
npm ci --no-audit --no-fund
npm run test:contract
npm run test:unit
npm run build
+25
View File
@@ -16,3 +16,28 @@ Vue 3.x
# 接口文档
启动后打开:http://localhost:8002/xiaozhi/doc.html
# 跑测试
测试分两种:纯单元测试,和要真连 MySQL、Redis 的集成测试(`DeviceTest`、`loginControllerTest`、`ModelConfigPersistenceTest`)。
集成测试需要先起好 MySQL 8 和 Redis 8。没起的时候这类测试整类跳过,不会让构建变红,日志里会写明缺的是哪个依赖。CI 上不一样:依赖连不上就直接让构建失败,不许跳过,免得门禁被悄悄掏空。
本地用 Docker 起这两个依赖:
```bash
docker run -d --name xiaozhi-mysql -p 3306:3306 \
-e MYSQL_ROOT_PASSWORD=123456 -e MYSQL_DATABASE=xiaozhi_esp32_server mysql:8.0
docker run -d --name xiaozhi-redis -p 6379:6379 redis:8.0
```
跑全部测试:
```bash
mvn -B test -DskipTests=false
```
只想编译、不跑测试:
```bash
mvn -B package -DskipTests=true
```
+1 -1
View File
@@ -32,7 +32,7 @@
<liquibase-core.version>4.20.0</liquibase-core.version>
<aliyun-sms-version>4.1.0</aliyun-sms-version>
<okio-version>3.4.0</okio-version>
<skipTests>true</skipTests>
<skipTests>false</skipTests>
<argLine></argLine>
</properties>
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# 检查那几个要连 MySQL 和 Redis 的集成测试是不是真的跑了。
# 守卫要是哪天手滑把它们全跳过了,CI 必须红,不能安安静静少跑一半门禁。
set -uo pipefail
REPORT_DIR="${1:-target/surefire-reports}"
WATCHED=(
"xiaozhi.modules.device.DeviceTest"
"xiaozhi.modules.model.service.impl.ModelConfigPersistenceTest"
"xiaozhi.modules.sys.loginControllerTest"
)
if [ ! -d "$REPORT_DIR" ]; then
echo "找不到测试报告目录:$REPORT_DIR"
exit 1
fi
missing=()
for class in "${WATCHED[@]}"; do
count=0
for report in "$REPORT_DIR"/TEST-*.xml; do
[ -e "$report" ] || continue
name=$(sed -n 's/.*<testsuite[^>]*name="\([^"]*\)".*/\1/p' "$report" | head -1)
if [ "$name" = "$class" ]; then
count=$(sed -n 's/.*<testsuite[^>]*tests="\([0-9]*\)".*/\1/p' "$report" | head -1)
count=${count:-0}
break
fi
done
if [ "$count" -eq 0 ]; then
missing+=("$class")
else
echo "OK $class 跑了 $count 条"
fi
done
if [ ${#missing[@]} -ne 0 ]; then
echo "这几个集成测试一条都没跑:${missing[*]}"
echo "守卫把门禁掏空了,构建必须失败"
exit 1
fi
echo "集成测试都跑过了"
@@ -14,9 +14,14 @@ import java.util.Collections;
import java.util.Date;
import java.util.List;
import org.apache.ibatis.builder.MapperBuilderAssistant;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.springframework.test.util.ReflectionTestUtils;
import com.baomidou.mybatisplus.core.MybatisConfiguration;
import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
import xiaozhi.modules.agent.dao.AgentCorrectWordMappingDao;
import xiaozhi.modules.agent.entity.AgentCorrectWordMappingEntity;
import xiaozhi.modules.correctword.dao.CorrectWordFileDao;
@@ -37,6 +42,20 @@ class CorrectWordFileServiceImplTest {
private final CorrectWordFileServiceImpl service =
new CorrectWordFileServiceImpl(fileDao, itemDao, mappingDao);
@BeforeAll
static void registerEntityMetadata() {
// 这条用例不起 Spring 上下文,MyBatis-Plus 的实体元数据没人注册,
// LambdaQueryWrapper 拼条件时找不到 lambda 缓存就抛异常。
// 全量跑的时候是前面的上下文测试顺带注册好的,单独跑就露馅。
// 这里自己补上,单独跑和跟着全量跑结果一样。
TableInfoHelper.initTableInfo(
new MapperBuilderAssistant(new MybatisConfiguration(), ""),
CorrectWordItemEntity.class);
TableInfoHelper.initTableInfo(
new MapperBuilderAssistant(new MybatisConfiguration(), ""),
CorrectWordFileEntity.class);
}
@Test
void getAllItemsByAgentId_returns_empty_list_when_no_mappings() {
when(mappingDao.selectByAgentId(AGENT_ID)).thenReturn(Collections.emptyList());
@@ -3,6 +3,7 @@ package xiaozhi.modules.device;
import java.util.HashMap;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
@@ -15,6 +16,7 @@ import xiaozhi.common.exception.RenException;
import xiaozhi.common.redis.RedisUtils;
import xiaozhi.modules.sys.dto.SysUserDTO;
import xiaozhi.modules.sys.service.SysUserService;
import xiaozhi.testsupport.IntegrationTestEnvironment;
@Slf4j
@SpringBootTest
@@ -22,6 +24,11 @@ import xiaozhi.modules.sys.service.SysUserService;
@DisplayName("设备测试")
public class DeviceTest {
@BeforeAll
static void checkDependencies() {
IntegrationTestEnvironment.requireDatabaseAndRedis();
}
@Autowired
private RedisUtils redisUtils;
@Autowired
@@ -5,6 +5,7 @@ import static org.junit.jupiter.api.Assertions.assertNull;
import java.util.UUID;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.springframework.beans.factory.annotation.Autowired;
@@ -19,12 +20,18 @@ import xiaozhi.common.redis.RedisUtils;
import xiaozhi.modules.model.dto.ModelConfigBodyDTO;
import xiaozhi.modules.model.dto.ModelConfigDTO;
import xiaozhi.modules.model.service.ModelConfigService;
import xiaozhi.testsupport.IntegrationTestEnvironment;
@SpringBootTest
@ActiveProfiles("dev")
@Transactional
class ModelConfigPersistenceTest {
@BeforeAll
static void checkDependencies() {
IntegrationTestEnvironment.requireDatabaseAndRedis();
}
@Autowired
private ModelConfigService service;
@@ -4,6 +4,7 @@ import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.Mockito.when;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
@@ -18,12 +19,18 @@ import xiaozhi.modules.security.dto.LoginDTO;
import xiaozhi.modules.security.dto.SmsVerificationDTO;
import xiaozhi.modules.sys.dto.RetrievePasswordDTO;
import xiaozhi.modules.sys.service.SysUserService;
import xiaozhi.testsupport.IntegrationTestEnvironment;
@Slf4j
@SpringBootTest
@ActiveProfiles("dev")
class loginControllerTest {
@BeforeAll
static void checkDependencies() {
IntegrationTestEnvironment.requireDatabaseAndRedis();
}
@Autowired
LoginController loginController;
@@ -0,0 +1,149 @@
package xiaozhi.testsupport;
import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.time.Duration;
import org.junit.jupiter.api.Assumptions;
import org.springframework.boot.env.YamlPropertySourceLoader;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.PropertySource;
import org.springframework.core.env.PropertySourcesPropertyResolver;
import org.springframework.core.env.StandardEnvironment;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import io.lettuce.core.RedisClient;
import io.lettuce.core.RedisURI;
import io.lettuce.core.api.StatefulRedisConnection;
import lombok.extern.slf4j.Slf4j;
/**
* 几个 @SpringBootTest 要真连 MySQL 和 Redis。本地没起这两个依赖时,
* 整类测试跳过,别让 Spring 上下文起不来变成一堆看不懂的报错;
* CI 里依赖必须真的在,探不通就直接让构建红。
*/
@Slf4j
public final class IntegrationTestEnvironment {
/** 集成测试用的配置文件,跟应用本身读的是同一套 */
private static final String[] CONFIG_FILES = {"application.yml", "application-dev.yml"};
private static Boolean available;
private static String problem;
private IntegrationTestEnvironment() {
}
/**
* 放在要连库的测试类的 @BeforeAll 里。
* 本地依赖不齐就整类跳过,CI 里依赖不齐就直接失败。
*/
public static void requireDatabaseAndRedis() {
String issue = probe();
if (issue == null) {
return;
}
if (isCi()) {
throw new IllegalStateException(
"CI 环境里" + issue + ",这几个集成测试必须真的跑起来,不能跳过");
}
log.warn("跳过依赖 MySQL 和 Redis 的集成测试:{}", issue);
Assumptions.assumeTrue(false, issue);
}
/** GitHub Actions 上会带 CI=true */
public static boolean isCi() {
String value = System.getenv("CI");
if (value == null) {
return false;
}
String trimmed = value.trim();
return !trimmed.isEmpty() && !"false".equalsIgnoreCase(trimmed);
}
/** 依赖都齐返回 null,不齐返回缺什么 */
static synchronized String probe() {
if (available != null) {
return problem;
}
PropertySourcesPropertyResolver resolver = resolver();
StringBuilder issues = new StringBuilder();
String url = firstNonBlank(resolver.getProperty("spring.datasource.druid.url", ""),
resolver.getProperty("spring.datasource.url", ""));
String username = firstNonBlank(resolver.getProperty("spring.datasource.druid.username", ""),
resolver.getProperty("spring.datasource.username", ""));
String password = firstNonBlank(resolver.getProperty("spring.datasource.druid.password", ""),
resolver.getProperty("spring.datasource.password", ""));
if (url.isBlank()) {
issues.append("配置里找不到数据源地址;");
} else {
try {
DriverManager.setLoginTimeout(5);
try (Connection connection = DriverManager.getConnection(url, username, password)) {
connection.isValid(3);
}
} catch (Exception e) {
issues.append("连不上 MySQL(").append(e.getMessage()).append(");");
}
}
String redisHost = resolver.getProperty("spring.data.redis.host", "localhost");
int redisPort = resolver.getProperty("spring.data.redis.port", Integer.class, 6379);
String redisPassword = resolver.getProperty("spring.data.redis.password", "");
try {
RedisURI.Builder builder = RedisURI.builder()
.withHost(redisHost)
.withPort(redisPort)
.withTimeout(Duration.ofSeconds(3));
if (redisPassword != null && !redisPassword.isBlank()) {
builder.withPassword(redisPassword.toCharArray());
}
RedisClient client = RedisClient.create(builder.build());
try {
StatefulRedisConnection<String, String> connection = client.connect();
connection.sync().ping();
connection.close();
} finally {
client.shutdown();
}
} catch (Exception e) {
issues.append("连不上 Redis(").append(e.getMessage()).append(");");
}
if (issues.length() == 0) {
available = Boolean.TRUE;
problem = null;
} else {
available = Boolean.FALSE;
problem = issues.toString();
}
return problem;
}
/** 读应用自己那份配置,命令行上的 -D 参数优先级更高,跟应用保持一致 */
private static PropertySourcesPropertyResolver resolver() {
ConfigurableEnvironment environment = new StandardEnvironment();
YamlPropertySourceLoader loader = new YamlPropertySourceLoader();
for (String name : CONFIG_FILES) {
Resource resource = new ClassPathResource(name);
if (!resource.exists()) {
continue;
}
try {
for (PropertySource<?> source : loader.load(name, resource)) {
environment.getPropertySources().addLast(source);
}
} catch (IOException e) {
log.warn("读配置 {} 失败:{}", name, e.getMessage());
}
}
return new PropertySourcesPropertyResolver(environment.getPropertySources());
}
private static String firstNonBlank(String first, String second) {
return first != null && !first.isBlank() ? first : (second == null ? "" : second);
}
}
+3
View File
@@ -5,6 +5,9 @@ python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = "-ra"
markers = [
"live_api: 需要真凭据或真网络的用例,mock 模式下由 conftest 剔除并列出",
]
filterwarnings = [
"ignore::DeprecationWarning",
]
+92 -28
View File
@@ -47,10 +47,29 @@ if str(_PROJECT_ROOT) not in sys.path:
# ---------------------------------------------------------------------------
try:
import opuslib_next # noqa: F401
except Exception:
except Exception as _opus_import_error:
# opuslib_next 是 ctypes 绑定,pip 装上的是纯 Python 包,真正的编解码靠系统里的
# libopus.so。runner 上没装 libopus 时 import 会失败,这里换桩顶上。
# 桩只能撑住 import 链路,真要编解码得先把 libopus 装上
# (apt-get install libopus0,正式镜像 Dockerfile-server-base 里也是这么装的)。
print(
f"[conftest] opuslib_next 不可用,改用桩:{_opus_import_error}",
flush=True,
)
def _opus_unavailable(_self, *_args, **_kwargs):
raise RuntimeError(
"libopus 没装,Opus 编解码跑不了。"
"装法:apt-get install libopus0(正式镜像 Dockerfile-server-base 里也是这么装的)"
)
_stub = types.ModuleType("opuslib_next")
_stub.Encoder = type("Encoder", (), {"__init__": lambda *a, **kw: None})
_stub.Decoder = type("Decoder", (), {"__init__": lambda *a, **kw: None})
_stub.Encoder = type(
"Encoder", (), {"__init__": lambda *a, **kw: None, "encode": _opus_unavailable}
)
_stub.Decoder = type(
"Decoder", (), {"__init__": lambda *a, **kw: None, "decode": _opus_unavailable}
)
_stub.APPLICATION_AUDIO = 0
_stub.OpusError = type("OpusError", (Exception,), {})
_stub.constants = types.SimpleNamespace()
@@ -70,6 +89,13 @@ _CUSTOM_CONFIG = Path(os.environ["PYTEST_CONFIG_FILE"]) if os.environ.get(
"PYTEST_CONFIG_FILE"
) else _DEFAULT_CUSTOM
# data/.config.yaml 在 .gitignore 里,本地没这个文件时现造一个空的。
# 项目里好几个模块 import 的时候就要读它(config/logger.py 的 setup_logging
# 直接抛 FileNotFoundError),缺了整份用例会收集不起来。CI 里也是这么造的
_DEFAULT_CUSTOM.parent.mkdir(parents=True, exist_ok=True)
if not _DEFAULT_CUSTOM.exists():
_DEFAULT_CUSTOM.write_text("{}\n", encoding="utf-8")
def _load_via_file_merge() -> dict:
"""纯文件合并:config.yaml + data/.config.yaml。和生产单模块部署一致。"""
@@ -323,35 +349,25 @@ def _install_mock_layer() -> None:
asyncio.sleep = _fast_sleep # type: ignore[assignment]
# 网络层 mock:如果真实 httpx 已加载,把它的 AsyncClient.send 替换成
# 网络层 mock:如果真实 httpx 已加载,把它的 send 替换成
# 立即抛 ConnectError 的桩,触发 ``_should_retry`` 走到 ``raise``,
# 但避免真发请求。Live 模式完全不安装这层。
if not LIVE_API_TESTS:
try:
import httpx as _httpx # noqa: F401
import httpx as _httpx
class _FakeAsyncClient:
def __init__(self, *args, **kwargs):
pass
async def _fake_async_send(self, request, **kwargs):
raise _httpx.ConnectError("MOCK: connect refused")
async def send(self, *args, **kwargs):
raise _httpx.ConnectError("MOCK: connect refused")
def _fake_send(self, request, **kwargs):
raise _httpx.ConnectError("MOCK: connect refused")
async def aclose(self):
return None
class _FakeClient:
def __init__(self, *args, **kwargs):
pass
def send(self, *args, **kwargs):
raise _httpx.ConnectError("MOCK: connect refused")
def close(self):
return None
_httpx.AsyncClient = _FakeAsyncClient # type: ignore[misc]
_httpx.Client = _FakeClient # type: ignore[misc]
# 只换 send 这一个方法,不换整个类。有些库(openai 2.x 的
# SyncHttpxClientWrapper)继承 httpx.Client 再自己拼请求,
# 整个类被换掉的话它们继承到的是假类,拼请求时直接
# AttributeError,报错还完全看不出是 mock 层的问题
_httpx.AsyncClient.send = _fake_async_send # type: ignore[assignment]
_httpx.Client.send = _fake_send # type: ignore[assignment]
except ImportError:
pass
@@ -437,9 +453,57 @@ if not LIVE_API_TESTS:
import pytest # noqa: E402 (after conftest hooks so pytest is initialised)
def is_ci() -> bool:
"""GitHub Actions 上会带 CI=true"""
value = os.environ.get("CI")
if value is None:
return False
trimmed = value.strip()
return bool(trimmed) and trimmed.lower() != "false"
def skip_or_fail(reason: str) -> None:
"""本地缺东西就跳过,CI 里缺东西直接让 job 挂掉。
口径跟 manager-api 那边的 IntegrationTestEnvironment 一致:本地没起依赖的
机器上跳过就行,不该把构建从绿变成红;CI 里依赖必须真的在,缺了就失败,
不许静默跳过 —— 静默跳过等于门禁被掏空,还看不出来。
"""
if is_ci():
raise RuntimeError(f"CI 环境里{reason},这条用例必须真的跑起来,不能跳过")
pytest.skip(reason)
def pytest_collection_modifyitems(config, items):
"""Live 模式下:``live_api`` 标记的测试必须配齐密钥才能跑(否则 skip)。"""
"""Mock 模式下:``live_api`` 标记的用例要真凭据或真网络,不收,并列出是哪几条。
以前是在 CI 命令行上用 ``-k`` 排除,看不出排掉了什么、为什么排。
改成打标记 + 这里统一剔除,日志里会一条条列出来,剔了什么一目了然。
"""
if LIVE_API_TESTS:
return
# Mock 模式下不强行 skip:测试应该都能跑(用 shim)。
return
keep = []
drop = []
for item in items:
if "live_api" in item.keywords:
drop.append(item)
else:
keep.append(item)
if not drop:
return
print(
"\n[conftest] 下面这些用例要真凭据或真网络,mock 模式下不收:",
flush=True,
)
for item in drop:
print(f"[conftest] - {item.nodeid}", flush=True)
print(
"[conftest] 真请求模式:export RUN_LIVE_API_TESTS=1 并配好凭据后再跑。\n",
flush=True,
)
config.hook.pytest_deselected(items=drop)
items[:] = keep
@@ -5,22 +5,13 @@ on disk. We only test:
- the ValueError path for unknown names (no providers need to exist)
- that the signature accepts *args / **kwargs (so all three modules work)
NOTE: This test file is currently skipped at module level because importing
`core.utils.intent` triggers `setup_logging()` at module load time, which
requires `data/.config.yaml` to exist. That file is .gitignore'd and not
present in CI. To enable this test suite, either:
1. Make `setup_logging()` lazy (move the call out of module-level scope).
2. Provide a stub `data/.config.yaml` in the test environment.
Tracked as deferred-minor in progress.md.
早先这个文件整份跳过,理由是导入 `core.utils.intent` 会触发
`setup_logging()`,而它要 `data/.config.yaml` 在场。CI 里这个文件是现造的,
本地跑 pytest 时 conftest 也会补上,所以那份跳过理由已经不成立了。
"""
import pytest
pytest.skip(
"setup_logging() at module import requires data/.config.yaml; skipped",
allow_module_level=True,
)
from core.utils import intent, llm, memory # noqa: E402 (unreachable when skipped)
from core.utils import intent, llm, memory
@pytest.mark.parametrize("factory,module_type", [
@@ -46,4 +37,4 @@ def test_create_instance_accepts_args_and_kwargs(factory, module_type):
is variadic. We use a sentinel that will fail the path check.
"""
with pytest.raises(ValueError):
factory("__missing__", "positional-arg", kwarg="value")
factory("__missing__", "positional-arg", kwarg="value")
@@ -1,26 +1,65 @@
"""Test for the auto-import mechanism in plugins_func/loadplugins.py.
"""插件自动导入机制的测试。
We import a real subpackage (`plugins_func.functions`) and verify the
import side-effect runs. This protects against accidentally deleting
`auto_import_modules` from `app.py` startup code.
``core/connection.py`` 启动时会调 ``auto_import_modules("plugins_func.functions")``,
把 ``plugins_func/functions`` 下的模块逐个导入一遍。导入过程中各插件用
``@register_function`` 把自己登记进注册表。这段自动导入万一被删掉,
插件一个都不会被加载,工具链直接空掉,而且不报错,很难发现。这里守住这条链路。
NOTE: This test is currently skipped because importing
`plugins_func.functions` triggers heavy module imports (cache_manager,
etc.) that may not initialise in the test environment. To enable this
test, ensure `data/.config.yaml` exists in the test environment or refactor
plugin discovery to be lazy.
最后一条用例查的是 ``core/connection.py`` 的源码本身:自动导入这个动作做得对
不对,跟连接模块有没有真的调它,是两回事,两条都要守。
早先这个文件整份跳过,理由是自动导入会拉起一串重量级依赖。CI 里
``data/.config.yaml`` 是现造的,那个理由已经不成立了。
"""
import pytest
import sys
from pathlib import Path
pytest.skip(
"plugins_func.functions import has side effects that need data/.config.yaml; skipped",
allow_module_level=True,
)
from plugins_func.functions import get_time # noqa: F401 (unreachable when skipped)
from plugins_func.loadplugins import auto_import_modules
from plugins_func.register import all_function_registry
def test_functions_package_importable():
"""If this test runs, plugins_func.functions.__init__.py has been imported."""
import plugins_func.functions as fns
assert hasattr(fns, "get_time")
def test_auto_import_loads_plugin_modules():
"""自动导入跑完之后,插件模块应该已经在 sys.modules 里。"""
auto_import_modules("plugins_func.functions")
assert "plugins_func.functions.get_time" in sys.modules
assert "plugins_func.functions.get_weather" in sys.modules
def test_auto_import_registers_plugin_functions():
"""自动导入的时候,插件函数应该已经登记进注册表。"""
auto_import_modules("plugins_func.functions")
assert "get_lunar" in all_function_registry
assert "get_weather" in all_function_registry
def test_connection_module_triggers_auto_import():
"""core/connection.py 模块加载时那次自动导入得在
<p>
这行被删掉或者改成注释,插件一个都不会被加载:启动不报错,工具链空掉,
上面两条用例照样绿。这里查源码,盯住那一次调用。
"""
import ast
source_path = Path(__file__).resolve().parents[2] / "core" / "connection.py"
tree = ast.parse(source_path.read_text(encoding="utf-8"))
imported_packages = []
for node in tree.body:
if not isinstance(node, ast.Expr) or not isinstance(node.value, ast.Call):
continue
call = node.value
func = call.func
name = func.id if isinstance(func, ast.Name) else getattr(func, "attr", "")
if name != "auto_import_modules":
continue
for arg in call.args:
if isinstance(arg, ast.Constant) and isinstance(arg.value, str):
imported_packages.append(arg.value)
assert "plugins_func.functions" in imported_packages, (
"core/connection.py 模块加载时的那次自动导入不见了,"
"插件不会被加载,工具链会空掉"
)
+134 -17
View File
@@ -1,7 +1,8 @@
"""LLM 链路测试。
Mock 模式(默认):从 data/.config.yaml 拿配置构造真实 provider,
网络调用走 conftest 注入的 shim —— 不发任何请求。
网络那一段换成本文件里的假接口 —— provider 自己拼请求、自己解析
流式回应那套代码照跑,只是不发真请求,也不需要真 key。
Live 模式(``RUN_LIVE_API_TESTS=1``):调真实 LLM provider,参考
``performance_tester_llm.py`` 的 ThreadPoolExecutor + 10s 超时模式。
@@ -18,6 +19,7 @@ Live 模式(``RUN_LIVE_API_TESTS=1``):调真实 LLM provider,参考
from __future__ import annotations
import concurrent.futures
import json
import sys
from pathlib import Path
@@ -30,41 +32,144 @@ from tests.conftest import (
get_provider_type,
get_selected_provider_name,
has_real_key,
skip_or_fail,
)
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
# 假接口回的固定内容。够长,能看出是流式拼出来的。
_MOCK_ANSWER = "这是 mock 出来的回应"
@pytest.fixture(autouse=True)
def _mock_chat_completions(monkeypatch):
"""Mock 模式下给 chat/completions 装一个假的回应。
装在 httpx 这一层:provider 拼请求、走流式解析那套代码都是真的,
只有网络那一段是假的。这样不用真 key 也能验证这条链路。
发出去的请求体记在返回的列表里,用例要检查 provider 真的把这次对话
装进了请求里。
"""
if LIVE_API_TESTS:
yield None
return
import httpx
stream_body = (
"data: "
+ json.dumps(
{
"id": "chatcmpl-mock",
"object": "chat.completion.chunk",
"created": 1700000000,
"model": "mock-model",
"choices": [
{
"index": 0,
"delta": {"role": "assistant", "content": _MOCK_ANSWER},
"finish_reason": None,
}
],
},
ensure_ascii=False,
)
+ "\n\n"
+ "data: [DONE]\n\n"
).encode("utf-8")
plain_body = json.dumps(
{
"id": "chatcmpl-mock",
"object": "chat.completion",
"created": 1700000000,
"model": "mock-model",
"choices": [
{
"index": 0,
"message": {"role": "assistant", "content": _MOCK_ANSWER},
"finish_reason": "stop",
}
],
},
ensure_ascii=False,
).encode("utf-8")
sent_requests = []
def _fake_send(self, request, **kwargs):
content = getattr(request, "content", b"") or b""
sent_requests.append(content)
if b'"stream":true' in content.replace(b" ", b""):
payload = stream_body
headers = {"content-type": "text/event-stream"}
else:
payload = plain_body
headers = {"content-type": "application/json"}
response = httpx.Response(200, content=payload, headers=headers)
response.request = request
return response
monkeypatch.setattr(httpx.Client, "send", _fake_send)
yield sent_requests
def _sent_body(requests):
"""最后一次发出去的请求体。live 模式没有假接口,返回 None"""
if requests is None:
return None
assert requests, "provider 一次请求都没发出去"
return json.loads(requests[-1].decode("utf-8"))
def _assert_request(body, provider, expected_contents):
"""检查请求体:模型名在、这次对话在
<p>
只看回应非空的话,provider 把消息丢掉、自己编一句话回来,测试照样绿。
"""
if body is None:
return
assert body.get("model"), "请求里得带上模型名"
expected_model = getattr(provider, "model_name", None)
if expected_model:
assert body["model"] == expected_model, "请求里的模型名跟配置对不上"
messages = body.get("messages") or []
assert messages, "请求里得带上对话"
contents = [message.get("content") for message in messages]
for text in expected_contents:
assert text in contents, f"请求里没有把「{text}」发出去"
def _build_provider():
name = get_selected_provider_name("LLM")
if not name:
pytest.skip("selected_module.LLM 未设置")
skip_or_fail("selected_module.LLM 未设置")
cfg = get_provider_config("LLM", name)
if not cfg:
pytest.skip(f"LLM provider '{name}' 配置缺失")
skip_or_fail(f"LLM provider '{name}' 配置缺失")
# 任何模式下:key 是占位符 → skip。真请求需要真 key;mock 模式下
# 没有真 key 跑下去也会撞真 SDK 的网络栈(openai/edge_tts)报错。
# 任何模式下:key 是占位符 → 真请求没法发。mock 模式有假接口顶上,
# 但占位符本身是中文,httpx 拼 Authorization 头的时候会直接报编码错,
# 所以 mock 模式把它换成一把假的 ascii key
if name == "CozeLLM":
if any(x in cfg.get("bot_id", "") for x in ["你的"]) or any(
x in cfg.get("user_id", "") for x in ["你的"]
):
pytest.skip("LLM 'CozeLLM' 的 bot_id/user_id 未配置")
skip_or_fail("LLM 'CozeLLM' 的 bot_id/user_id 未配置")
elif name == "Ollama":
if not cfg.get("model_name"):
pytest.skip("LLM 'Ollama' 的 model_name 未配置")
skip_or_fail("LLM 'Ollama' 的 model_name 未配置")
elif not has_real_key(cfg, "api_key"):
if LIVE_API_TESTS:
pytest.skip(f"LLM '{name}' 的 api_key 未配置")
else:
pytest.skip(
f"LLM '{name}' 的 api_key 是占位符(mock 模式不会真发请求)。"
"真请求模式:export RUN_LIVE_API_TESTS=1 并填真 key 后再跑。"
)
skip_or_fail(f"LLM '{name}' 的 api_key 未配置")
cfg = dict(cfg)
cfg["api_key"] = "mock-api-key"
ptype = get_provider_type("LLM", name)
if not ptype:
pytest.skip(f"LLM '{name}' 的 type 字段缺失")
skip_or_fail(f"LLM '{name}' 的 type 字段缺失")
from core.utils.llm import create_instance
return create_instance(ptype, cfg)
@@ -77,7 +182,7 @@ def _collect_with_timeout(provider, session_id: str, dialogue, timeout: float =
return future.result(timeout=timeout)
def test_llm_response_streams_non_empty() -> None:
def test_llm_response_streams_non_empty(_mock_chat_completions) -> None:
"""response(session_id, dialogue) 应产生非空 token 流。"""
provider = _build_provider()
dialogue = [
@@ -89,21 +194,33 @@ def test_llm_response_streams_non_empty() -> None:
full = "".join(chunks).strip()
assert isinstance(full, str), f"LLM chunks join 应是 str,得到 {type(full).__name__}"
assert full, f"LLM 返回空响应(provider={type(provider).__name__})"
# 回的得是假接口给的那句,不是随便一个非空串
assert full == _MOCK_ANSWER
_assert_request(
_sent_body(_mock_chat_completions),
provider,
["你是助手", "用一句话说 hi"],
)
def test_llm_response_no_stream_returns_string() -> None:
def test_llm_response_no_stream_returns_string(_mock_chat_completions) -> None:
"""response_no_stream 应返回非空字符串。"""
provider = _build_provider()
text = provider.response_no_stream("你是助手", "介绍一下北京").strip()
assert isinstance(text, str), f"response_no_stream 应返回 str,得到 {type(text).__name__}"
assert text, "response_no_stream 返回空字符串"
assert text == _MOCK_ANSWER
_assert_request(
_sent_body(_mock_chat_completions), provider, ["你是助手", "介绍一下北京"]
)
def test_llm_dialogue_min_length_one_user() -> None:
def test_llm_dialogue_min_length_one_user(_mock_chat_completions) -> None:
"""至少一条 user 消息的 dialogue 应能跑通。"""
provider = _build_provider()
dialogue = [{"role": "user", "content": "ping"}]
chunks = _collect_with_timeout(provider, "s1", dialogue, timeout=10.0)
assert "".join(chunks).strip(), "单 user 消息应返回内容"
_assert_request(_sent_body(_mock_chat_completions), provider, ["ping"])
+5 -1
View File
@@ -8,7 +8,7 @@ Live 模式(``RUN_LIVE_API_TESTS=1``):调真实 TTS provider。
text_to_speak(text, output_file) -> 异步,把合成结果写到 output_file
注意:
- 3 个构造测试不依赖云服务;第 4 个 test_tts_text_to_speak_writes_nonempty_file
- 3 个构造测试不依赖云服务;第 4 个 test_tts_text_to_speak_writes_file
才是真链路测试,会真发 HTTP。
- Mock 模式下:第 4 个测试断言宽松 —— 文件存在即可(shim 创建空文件),
不强求 size > 0。Live 模式下严格校验 size > 0。
@@ -81,9 +81,13 @@ def test_tts_provider_has_output_dir() -> None:
assert provider.output_file, "output_file 不应为空"
@pytest.mark.live_api
def test_tts_text_to_speak_writes_file() -> None:
"""Mock 模式:断言 file 存在即可。Live 模式:断言 size > 0。
这条要连微软公网,runner 的出口 IP 会被风控挡掉,所以打了 live_api 标记:
mock 模式下由 conftest 统一剔除(会列出剔的是哪几条),live 模式下才收。
参考 performance_tester_tts.py 的实现:
- mock conn(sample_rate/audio_format/stop_event/client_abort/headers)
和 opus_encoder(SimpleNamespace 占位即可)