Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a69ef627c7 | ||
|
|
d4de462d44 | ||
|
|
0276bceab5 | ||
|
|
c2b2943a3a | ||
|
|
55b12df373 | ||
|
|
27d465c250 | ||
|
|
4a2c4aa638 |
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Architecture-and-Code-Map
|
||||
wiki_url: https://git.ilapage.cn/ila/yovision/wiki/Architecture-and-Code-Map.-
|
||||
wiki_revision: b25498f58c690f787f0b572788617ef6b0d8ea7d
|
||||
synchronized_at: 2026-08-31T07:23:02Z
|
||||
wiki_revision: 77af1bb5a71d7b89e74b1905ca11b6040dc0d1eb
|
||||
synchronized_at: 2026-08-31T08:25:26Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 架构与代码地图
|
||||
@@ -359,3 +359,26 @@ v1 使用 HTTPS 上的 Ed25519 短期请求绑定 JWS。每个部署实例拥有
|
||||
|
||||
Sense/Bell 使用 Go 标准库 Ed25519,Brain 冻结 `cryptography==50.0.1`。固定跨语言向量证明 Go/Python 可互相验签。#151 只提供身份、注册表、传输策略及可注入 replay 接口;#152/#153 才注册业务 endpoint,并必须使用各产品独立的持久原子 replay store验证重启,不能共享数据库。
|
||||
<!-- machine-identity-v1:end -->
|
||||
|
||||
<!-- coordination-deployment-v1:start -->
|
||||
## 可选协调部署层
|
||||
|
||||
工单 #154 已于 2026-08-31 验收。根级协调部署层位于 `deploy/coordination/**` 与 `scripts/runtime/coordination/**`,它只负责声明、校验和调用三个独立产品入口:
|
||||
|
||||
```text
|
||||
仓库外 coordination.json
|
||||
├─ Sense 独立包 / env / DB / 端口 / 数据 / 日志 / 机器身份
|
||||
├─ Brain 独立包 / env / 端口 / 数据 / 日志 / 机器身份
|
||||
└─ Bell 独立包 / env / DB / 端口 / 数据 / 日志 / 机器身份
|
||||
↓
|
||||
coordination-common.ps1
|
||||
├─ 封闭清单与隔离断言
|
||||
├─ 单端或选择性组合 start/stop/status
|
||||
├─ 健康检查、版本和清单摘要
|
||||
└─ PID + 启动器 + 命令令牌归属保护
|
||||
```
|
||||
|
||||
`coordination.schema.json` 定义版本 `yovision.coordination/v1`;`coordination.example.json` 只提供不可投产占位。公共实现 `coordination-common.ps1` 解析外部 env 数据但不执行其内容,校验产品命令位于各自包内、秘密路径位于仓库和包外,并拒绝路径、端口、数据库身份、JWT、Cookie 或机器身份复用。三个薄入口脚本分别调用公共实现,BAT 只透传参数和退出码。
|
||||
|
||||
协调层不拥有业务数据或契约,不共享用户表、JWT、Cookie、数据库内部模型、摄像头凭据或产品实现。它不替代 `contracts/**`,也不让任一产品成为另一产品的启动前置;connector 可关闭,三端核心能力继续独立运行。根级运行状态位于清单指定的 `runtime_root\state`,产品日志仍归各自日志目录和产品入口管理。
|
||||
<!-- coordination-deployment-v1:end -->
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Local-Development-and-Verification
|
||||
wiki_url: https://git.ilapage.cn/ila/yovision/wiki/Local-Development-and-Verification.-
|
||||
wiki_revision: 59f6eb5816b5d3f78c85ef902053c5fc45b8e6e9
|
||||
synchronized_at: 2026-08-31T07:33:54Z
|
||||
wiki_revision: 889ffccc30a998a5cc8c4a5d91af3c72771c093a
|
||||
synchronized_at: 2026-08-31T13:00:20Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 本地开发与验证
|
||||
@@ -751,3 +751,68 @@ git diff --check
|
||||
|
||||
工单验收未使用客户 PKI、生产 PostgreSQL、真实三端网络或生产流量;这些结果只能由 #154 部署和 #155 E2E 补充。Sense 本地候选原子 Outbox 当前没有生产创建 caller,也不得据此声明本地产生链已完整接通。
|
||||
<!-- integration-connectors-v1:end -->
|
||||
|
||||
<!-- coordination-deployment-v1:start -->
|
||||
## 根级编排本地验证
|
||||
|
||||
工单 #154 的编排验证只使用仓库外临时目录和假交付包,不需要真实密码、数据库或客户 PKI。先运行清单和脚本验证:
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -File scripts/runtime/coordination/start-yovision.ps1 -Manifest C:\YoVision\config\coordination.json -ValidateOnly
|
||||
pwsh -NoLogo -NoProfile -File deploy/coordination/tests/coordination-smoke.ps1
|
||||
```
|
||||
|
||||
冒烟测试必须返回 0,并覆盖 Sense、Brain、Bell 分别启动/状态/停止、Brain+Bell 选择性组合、停止单端不影响另一端、单端启动失败隔离、端口占用与重复端口拒绝、重复数据库身份拒绝、无归属 PID 保护、清单漂移诊断、退出码和状态清理。测试创建的目录必须解析在系统临时目录内,结束时只清理该测试目录。
|
||||
|
||||
仓库闭环验证:
|
||||
|
||||
```powershell
|
||||
python dev_scripts/harness.py check --strict
|
||||
python -m unittest discover -s tests -v
|
||||
python dev_scripts/harness.py sync --check
|
||||
git diff --check
|
||||
```
|
||||
|
||||
这些测试证明编排和隔离控制,不证明真实产品包、生产 PostgreSQL、客户 PKI、真实摄像头/GPU、供应商服务、容量或长稳表现。真实契约闭环与 Brain/Bell 离线、重启、重复/冲突及证据降级由后续 E2E 工单验证。
|
||||
<!-- coordination-deployment-v1:end -->
|
||||
|
||||
<!-- coordination-e2e-v1:start -->
|
||||
## 三项目协调 E2E 验证
|
||||
|
||||
从仓库根目录执行唯一默认验收入口:
|
||||
|
||||
```powershell
|
||||
pwsh scripts/e2e/coordination/run-coordination-e2e.ps1
|
||||
```
|
||||
|
||||
可复制的工具参数:
|
||||
|
||||
```powershell
|
||||
pwsh scripts/e2e/coordination/run-coordination-e2e.ps1 -PostgresBin D:\pgsql17\bin -Python Brain\.venv\Scripts\python.exe
|
||||
```
|
||||
|
||||
默认完整运行按以下阶段串行收敛:
|
||||
|
||||
1. 创建动态 loopback 端口的临时 PostgreSQL,为 Sense/Bell 建立不同的随机 owner/database,并执行 Bell 正式迁移。
|
||||
2. 在一次性 Python venv 中验证 source-config、runtime-status、machine-identity、events、evidence 五类冻结 v1 契约。
|
||||
3. 验证 Sense 配置/状态、Brain 事件接入/证据/Outbox 与 PostgreSQL 并发恢复。
|
||||
4. 验证 Brain 配置/状态 connector、匿名事件导出和持久重放。
|
||||
5. 串行验证 Bell ingress/evidence、Rule/Alert 投影和 ack/close 生命周期。共享有状态 Bell 数据库的包不得并行运行。
|
||||
6. 默认重新运行 Sense isolated E2E、Brain 全套测试和 Bell isolated E2E。
|
||||
7. 核对 Sense/Bell database owner/database 不同、随机秘密未进入临时日志、所属端口/进程已停止,并安全清理专属临时目录。
|
||||
|
||||
#155 的完整通过记录为:source-config 11 项、runtime-status 14 项、machine-identity 10 项、events 7 项、evidence 4 项;Brain connector/event export 25 项、Brain 全套 74 项;Sense/Bell connector 与独立 isolated E2E 均通过。不同提交必须以当次真实输出为准,不复用这些数量冒充新结果。
|
||||
|
||||
只有默认完整入口退出码为 0、末行出现 `COORDINATION_E2E passed`、三端独立回归均成功并且工作区无计划外改动,才满足协调 E2E 技术验收。`-SkipIndependentProductE2E` 仅供 harness 调试,不能作为 #155/MVP 验收。`-KeepTemporary` 仅用于失败诊断,保留内容不得提交或共享。
|
||||
|
||||
仓库级闭环继续执行:
|
||||
|
||||
```powershell
|
||||
python dev_scripts/harness.py check --strict
|
||||
python -m unittest discover -s tests -v
|
||||
python dev_scripts/harness.py sync --check
|
||||
git diff --check
|
||||
```
|
||||
|
||||
已验证范围不包含真实 GPU/生产模型、真实摄像头、通知供应商、生产迁移、16 路长稳和客户现场效果;这些项目必须在对应环境和独立工单中验收。
|
||||
<!-- coordination-e2e-v1:end -->
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Troubleshooting
|
||||
wiki_url: https://git.ilapage.cn/ila/yovision/wiki/Troubleshooting
|
||||
wiki_revision: fdd44bfe589d65cbce6ec085878abae2702d59e7
|
||||
synchronized_at: 2026-08-31T07:23:47Z
|
||||
wiki_revision: eec4beb8e5b9b8ff18401b272fd249aff7711509
|
||||
synchronized_at: 2026-08-31T13:00:41Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 故障排查
|
||||
@@ -195,3 +195,56 @@ synchronized_at: 2026-08-31T07:23:47Z
|
||||
|
||||
日志只记录稳定错误码、request/correlation ID、已认证 principal/kid 和脱敏业务引用;不得记录私钥、令牌、完整 Authorization、摄像头凭据、内部证据路径或事件完整敏感载荷。
|
||||
<!-- integration-connectors-v1:end -->
|
||||
|
||||
<!-- coordination-deployment-v1:start -->
|
||||
## 根级协调编排排错
|
||||
|
||||
工单 #154 已于 2026-08-31 验收;本节适用于已验收的可选根级编排。
|
||||
|
||||
先使用与启动时相同的仓库外清单查询状态:
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -File scripts/runtime/coordination/status-yovision.ps1 -Manifest C:\YoVision\config\coordination.json -Product all
|
||||
```
|
||||
|
||||
| 现象/状态 | 检查 | 安全处理 |
|
||||
|---|---|---|
|
||||
| 清单校验失败 | 检查包、env、私钥路径是否存在且位于仓库和包外;检查三端端口、目录、数据库、Cookie、JWT、账户和身份是否独立 | 修正仓库外清单或环境文件;不得放宽隔离断言或把秘密写进仓库 |
|
||||
| `port ... is already in use` | 用状态命令确认是否已有受管实例,再检查对应监听端口 | 先按归属停止旧实例或为产品分配独立端口;不得终止未知进程 |
|
||||
| `stopped` | 没有该产品状态文件 | 按需单独启动;不代表其他产品异常 |
|
||||
| `unhealthy` | 进程仍归属本实例,但 HTTP/进程健康检查失败 | 查看该产品独立 `coordination.err.log`、`coordination.out.log` 和产品日志;修复该端,不自动重启其他端 |
|
||||
| `stale` / `ownership-mismatch` | PID 已复用、启动器或命令令牌与状态不符 | 不会停止该进程;人工核对进程与状态文件,确认归属后再处理 |
|
||||
| `stale` / `manifest-drift` | 运行中的实例来自不同清单摘要 | 使用原清单安全停止,或确认归属后停止再以新清单启动;不得用新清单覆盖运行事实 |
|
||||
| 单端启动失败 | 查看该端协调日志、产品日志和退出码 | 编排只清理该端新进程;确认其他端状态,修复失败端后单独重试 |
|
||||
| connector 使对端成为启动强依赖 | connector 开关或产品配置错误 | 关闭对应 event export、ingress、relay 或 evidence connector,恢复三端独立运行;保留 Outbox/Receipt/Event 等持久事实 |
|
||||
| 停止命令拒绝执行 | 状态归属不匹配,或产品停止入口返回非零 | 不使用无条件 taskkill;先核对 PID、启动器、命令令牌和产品停止日志 |
|
||||
|
||||
日志和状态不得包含环境变量值、密码、JWT、token、私钥、完整 Authorization、摄像头凭据或客户数据。协调层故障时可停止使用根级入口并恢复三个产品各自的已验收启动脚本,不删除数据或共享事实。
|
||||
<!-- coordination-deployment-v1:end -->
|
||||
|
||||
<!-- coordination-e2e-v1:start -->
|
||||
## 三项目协调 E2E 排错
|
||||
|
||||
先从完整输出定位第一个失败阶段,不同时修改多个猜测原因。默认入口是:
|
||||
|
||||
```powershell
|
||||
pwsh scripts/e2e/coordination/run-coordination-e2e.ps1
|
||||
```
|
||||
|
||||
| 现象 | 检查 | 安全处理 |
|
||||
|---|---|---|
|
||||
| 缺少 `initdb.exe`、`pg_ctl.exe`、`createdb.exe` 或 `psql.exe` | 检查 `-PostgresBin` 是否指向同一 PostgreSQL 安装的 `bin` | 修正工具路径;不要改用生产数据库或默认 5432 |
|
||||
| Python 缺少契约依赖 | 确认所选 Python 可创建 venv 且能安装仓库冻结依赖 | 修复 Python/依赖源后重试;不要把依赖临时装进产品环境充当固定基线 |
|
||||
| Bell 正式迁移失败 | 查看当轮临时目录的 `bell-migrate.log`,核对首个 SQLSTATE | 修复迁移或工具链;不得跳过迁移、手工补表或连接业务库 |
|
||||
| Sense/Bell 数据库隔离断言失败 | 检查生成的 owner/database 是否不同 | 停止测试;不得共享数据库、角色、JWT、Cookie 或账户空间 |
|
||||
| Bell Rule/Alert 评估数偶发增加 | 检查是否把 ingress、rule、lifecycle 多个有状态 Go 包放在同一数据库并行运行 | 恢复 root harness 的串行阶段;不要降低断言或清除不可变事实 |
|
||||
| `machine_token_*`、duplicate 或 conflict 与预期不符 | 核对 audience/scope/kid、请求绑定、jti 和 producer/source 业务键 | 保持稳定错误和原业务键;不得记录 Authorization、复用网页登录态或为重试改 source ID |
|
||||
| Outbox 在 Bell 离线后未补投 | 核对 retry/lease/available_at、Bell 恢复和新 relay 进程 | 保留消息和历史,恢复 Bell 后重试;不得清空 Outbox/Receipt/Event |
|
||||
| evidence timeout/unavailable 导致整条事件失败 | 检查 evidence resolver 与降级状态 | Event/Receipt/Alert 应继续保留;不得把失败伪装为 success |
|
||||
| 端口仍占用或测试结束后有进程 | 确认进程是否由本轮测试启动,查看所属临时目录和状态 | 只停止能证明归属的进程;不得无条件 taskkill 或终止其他实例 |
|
||||
| 秘密扫描失败 | 在本轮临时日志中搜索该随机测试值,禁止把值粘贴到工单 | 修复日志输出后重新运行;不得仅关闭扫描 |
|
||||
| 需要保留失败现场 | 使用 `-KeepTemporary` 并记录明确绝对临时路径 | 目录只用于本机受控诊断,完成后按已核对路径清理;不得提交或共享 |
|
||||
| 使用 `-SkipIndependentProductE2E` 后通过 | 只证明协调 harness 主体,不证明三端独立回归 | 修复依赖后重新运行默认完整入口,不能据此通过 MVP 验收 |
|
||||
|
||||
若失败发生在 Sense、Brain 或 Bell 的独立 E2E,转到对应产品章节按该端首个错误排查;不得在协调文档工单中顺手修改产品代码。真实 GPU、真机、通知供应商、生产迁移、16 路长稳或客户现场问题不属于该隔离 E2E 的结论。
|
||||
<!-- coordination-e2e-v1:end -->
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Deployment-and-Operations
|
||||
wiki_url: https://git.ilapage.cn/ila/yovision/wiki/Deployment-and-Operations.-
|
||||
wiki_revision: fa7e2031338cbca0f669f5d5a72a073670114882
|
||||
synchronized_at: 2026-08-31T07:35:46Z
|
||||
wiki_revision: c376d69227aa159a7839a829f5bfb5a7beaf89e6
|
||||
synchronized_at: 2026-08-31T13:12:00Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# YoVision 部署与运维
|
||||
@@ -188,3 +188,49 @@ Bell 运行变量:
|
||||
|
||||
回退时关闭 Brain `event_export.enabled`、Sense 两个 connector 开关和 Bell ingress/evidence 开关;保留 last-known-good、运行投影、InboundEvent、EvidenceRecord、Outbox、ReplayToken、Receipt、Event 与审计。不得删除事实、关闭 TLS/验签或改用网页登录态。
|
||||
<!-- integration-connectors-v1:end -->
|
||||
|
||||
<!-- coordination-deployment-v1:start -->
|
||||
## 三项目可选根级部署编排
|
||||
|
||||
工单 #154 已于 2026-08-31 验收。根级编排只组合 Sense、Brain、Bell 已审核交付包,不复制产品实现,也不改变三端独立交付边界。事实入口如下:
|
||||
|
||||
- 清单 Schema:`deploy/coordination/coordination.schema.json`
|
||||
- 无秘密示例:`deploy/coordination/coordination.example.json`
|
||||
- 操作说明:`deploy/coordination/README.md`
|
||||
- 启动、停止、状态入口:`scripts/runtime/coordination/{start,stop,status}-yovision.ps1`,并提供同名 BAT 包装器
|
||||
|
||||
生产或验收环境必须先把示例清单复制到仓库外受控目录,并分别准备仓库外 Sense、Brain、Bell 环境文件和每实例独立私钥。Sense/Bell 使用不同数据库、数据库角色、账户空间、浏览器 origin、Cookie、JWT、端口、包、数据目录和日志目录;三端机器 principal、key id 和私钥文件也不得复用。清单只记录版本、路径和公开标识,不保存密码、JWT、token 或私钥内容。
|
||||
|
||||
从仓库根目录使用:
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -File scripts/runtime/coordination/start-yovision.ps1 -Manifest C:\YoVision\config\coordination.json -ValidateOnly
|
||||
pwsh -NoProfile -File scripts/runtime/coordination/start-yovision.ps1 -Manifest C:\YoVision\config\coordination.json -Product bell,sense,brain
|
||||
pwsh -NoProfile -File scripts/runtime/coordination/status-yovision.ps1 -Manifest C:\YoVision\config\coordination.json -Product all
|
||||
pwsh -NoProfile -File scripts/runtime/coordination/stop-yovision.ps1 -Manifest C:\YoVision\config\coordination.json -Product brain
|
||||
```
|
||||
|
||||
启动顺序为 Bell → Sense → Brain,停止顺序反向。启动时 `all` 只包含 `enabled=true` 的产品;停止和状态查询时 `all` 覆盖三端,避免停用配置后遗留进程。编排状态只保存 PID、版本、清单摘要和命令归属元数据;停止前必须核对 PID、启动器与命令令牌,归属不匹配时拒绝终止。单端启动失败只清理该端新进程,不自动停止其他端。
|
||||
|
||||
升级时每次只替换一个独立包并更新精确版本,先备份 Sense/Bell,再按 Bell → Sense → Brain 验证,最后启用 connector。回退时先停用 Brain event export、Sense ingress/relay 与 Bell ingress/evidence connector,再使用各产品独立入口回退包或恢复数据库;不得删除 Outbox、Receipt、Event、运行投影、replay 或审计事实。16 路只是当前交付配额,不是编排器硬上限;真实生产包、PostgreSQL、客户 PKI、真机容量与长稳仍需部署环境验收。
|
||||
<!-- coordination-deployment-v1:end -->
|
||||
|
||||
<!-- coordination-e2e-v1:start -->
|
||||
## 三项目协调 E2E 验收入口
|
||||
|
||||
工单 #155 已于 2026-08-31 验收,并通过 PR #170 合入 `dev@0276bce`。该入口只用于隔离开发/验收,不是生产部署、生产迁移或现场容量测试:
|
||||
|
||||
```powershell
|
||||
pwsh scripts/e2e/coordination/run-coordination-e2e.ps1
|
||||
```
|
||||
|
||||
入口要求 PowerShell 7、冻结的 Go 1.26.5 工具链、Brain 可用 Python 环境、PostgreSQL 17 命令行工具,以及 Sense/Bell 各自独立 E2E 已记录的本机依赖。PostgreSQL 工具默认从 `D:\pgsql17\bin` 读取,可通过 `-PostgresBin` 指定其他安装目录;Python 默认优先使用 `Brain\.venv\Scripts\python.exe`,也可通过 `-Python` 指定。不得为通过测试而连接生产数据库、客户设备或生产服务。
|
||||
|
||||
每次运行会在系统临时目录创建专属 PostgreSQL cluster,使用非 5432 动态 loopback 端口,并为 Sense、Bell 创建随机且不同的 database owner 和 database。测试凭据只存在于当前进程和临时测试范围,不进入仓库。入口按顺序执行冻结契约、三端 connector/持久化故障验证,并默认继续执行 Sense、Brain、Bell 各自已有的独立 E2E。
|
||||
|
||||
`-SkipIndependentProductE2E` 只用于定位协调 harness 自身故障;使用该参数的结果不能作为 #155 或 MVP #156 验收证据。`-KeepTemporary` 只用于受控保留失败诊断,目录可能包含一次性测试数据,排查完成后按明确绝对路径清理,不得提交或共享。
|
||||
|
||||
入口在 `finally` 中只停止自己启动的临时 PostgreSQL 和下游测试拥有的进程,核对端口关闭,并扫描临时日志是否出现本轮随机秘密。成功的最终判据是全部阶段退出码为 0,输出末行包含 `COORDINATION_E2E passed`,且 `git status --short` 没有产品源码或配置改动。
|
||||
|
||||
该验收证明版本化契约、配置/状态、匿名事件、Sense Outbox、Bell Receipt/Event/Alert/ack/close、离线恢复、身份/重放/冲突/证据降级及三端独立运行;不证明真实 GPU/生产模型、真实摄像头、通知供应商、生产迁移、16 路长稳或客户现场效果。
|
||||
<!-- coordination-e2e-v1:end -->
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
@echo off
|
||||
setlocal
|
||||
pwsh.exe -NoProfile -File "%~dp0run-coordination-e2e.ps1" %*
|
||||
exit /b %ERRORLEVEL%
|
||||
@@ -0,0 +1,191 @@
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string]$PostgresBin = 'D:\pgsql17\bin',
|
||||
[string]$Python = '',
|
||||
[switch]$SkipIndependentProductE2E,
|
||||
[switch]$KeepTemporary
|
||||
)
|
||||
|
||||
Set-StrictMode -Version 3.0
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
$repositoryRoot = [IO.Path]::GetFullPath((Join-Path $PSScriptRoot '..\..\..'))
|
||||
$temporaryRoot = [IO.Path]::GetFullPath((Join-Path ([IO.Path]::GetTempPath()) ('yovision-coordination-e2e-' + [guid]::NewGuid().ToString('N'))))
|
||||
$postgresData = Join-Path $temporaryRoot 'postgres'
|
||||
$postgresLog = Join-Path $temporaryRoot 'postgres.log'
|
||||
$postgresStarted = $false
|
||||
$savedEnvironment = @{}
|
||||
$sensitiveValues = [Collections.Generic.List[string]]::new()
|
||||
|
||||
function Get-FreeTcpPort {
|
||||
$listener = [Net.Sockets.TcpListener]::new([Net.IPAddress]::Loopback, 0)
|
||||
try { $listener.Start(); return ([Net.IPEndPoint]$listener.LocalEndpoint).Port } finally { $listener.Stop() }
|
||||
}
|
||||
|
||||
function Wait-Tcp([int]$Port, [bool]$Open, [int]$Attempts = 120) {
|
||||
for ($attempt = 0; $attempt -lt $Attempts; $attempt++) {
|
||||
$client = [Net.Sockets.TcpClient]::new()
|
||||
try { $connected = $client.ConnectAsync('127.0.0.1', $Port).Wait(250) -and $client.Connected } catch { $connected = $false } finally { $client.Dispose() }
|
||||
if ($connected -eq $Open) { return }
|
||||
Start-Sleep -Milliseconds 250
|
||||
}
|
||||
throw "TCP port $Port did not reach open=$Open"
|
||||
}
|
||||
|
||||
function New-RandomName([string]$Prefix) {
|
||||
return $Prefix + '_' + [guid]::NewGuid().ToString('N').Substring(0, 12)
|
||||
}
|
||||
|
||||
function New-RandomSecret {
|
||||
$buffer = New-Object byte[] 48
|
||||
$generator = [Security.Cryptography.RandomNumberGenerator]::Create()
|
||||
try { $generator.GetBytes($buffer) } finally { $generator.Dispose() }
|
||||
return [Convert]::ToBase64String($buffer).Replace('+', 'A').Replace('/', 'B')
|
||||
}
|
||||
|
||||
function Set-TestEnvironment([string]$Name, [string]$Value, [bool]$Sensitive = $false) {
|
||||
if (-not $script:savedEnvironment.ContainsKey($Name)) {
|
||||
$script:savedEnvironment[$Name] = [Environment]::GetEnvironmentVariable($Name, 'Process')
|
||||
}
|
||||
[Environment]::SetEnvironmentVariable($Name, $Value, 'Process')
|
||||
if ($Sensitive) { $script:sensitiveValues.Add($Value) }
|
||||
}
|
||||
|
||||
function Invoke-Checked {
|
||||
param([string]$Name, [string]$WorkingDirectory, [scriptblock]$Command)
|
||||
Write-Host "[coordination-e2e] $Name"
|
||||
Push-Location $WorkingDirectory
|
||||
try {
|
||||
& $Command
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Name failed with exit code $LASTEXITCODE" }
|
||||
} finally { Pop-Location }
|
||||
}
|
||||
|
||||
function Assert-NoSecretInLogs {
|
||||
$logs = @(Get-ChildItem -LiteralPath $temporaryRoot -File -Recurse -ErrorAction SilentlyContinue)
|
||||
foreach ($log in $logs) {
|
||||
$stream = [IO.File]::Open($log.FullName, [IO.FileMode]::Open, [IO.FileAccess]::Read, [IO.FileShare]::ReadWrite -bor [IO.FileShare]::Delete)
|
||||
try {
|
||||
$reader = [IO.StreamReader]::new($stream, [Text.Encoding]::UTF8, $true)
|
||||
try { $content = $reader.ReadToEnd() } finally { $reader.Dispose() }
|
||||
} finally { $stream.Dispose() }
|
||||
foreach ($secret in $sensitiveValues) {
|
||||
if ($secret.Length -ge 8 -and $content.Contains($secret)) {
|
||||
throw "Temporary log exposed a generated E2E secret: $($log.Name)"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
New-Item -ItemType Directory -Path $temporaryRoot | Out-Null
|
||||
|
||||
try {
|
||||
foreach ($tool in @('initdb.exe', 'pg_ctl.exe', 'createdb.exe', 'psql.exe')) {
|
||||
$path = Join-Path $PostgresBin $tool
|
||||
if (-not (Test-Path -LiteralPath $path -PathType Leaf)) { throw "Required PostgreSQL tool not found: $path" }
|
||||
}
|
||||
if ([string]::IsNullOrWhiteSpace($Python)) {
|
||||
$candidate = Join-Path $repositoryRoot 'Brain\.venv\Scripts\python.exe'
|
||||
$Python = if (Test-Path -LiteralPath $candidate -PathType Leaf) { $candidate } else { 'python.exe' }
|
||||
}
|
||||
$contractEnvironment = Join-Path $temporaryRoot 'contract-venv'
|
||||
& $Python -m venv $contractEnvironment
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Could not create the isolated contract-test environment.' }
|
||||
$contractPython = Join-Path $contractEnvironment 'Scripts\python.exe'
|
||||
$env:PIP_DISABLE_PIP_VERSION_CHECK = '1'
|
||||
& $contractPython -m pip install --quiet -r (Join-Path $repositoryRoot 'contracts\tests\source-config-v1\requirements.txt') 'cryptography==50.0.1'
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Could not install the pinned contract-test dependencies.' }
|
||||
|
||||
$postgresPort = Get-FreeTcpPort
|
||||
if ($postgresPort -eq 5432) { throw 'Coordination E2E refuses the default PostgreSQL port.' }
|
||||
$clusterUser = New-RandomName 'yvcoord'
|
||||
$senseRole = New-RandomName 'sense_owner'
|
||||
$bellRole = New-RandomName 'bell_owner'
|
||||
$senseDatabase = New-RandomName 'sense_e2e'
|
||||
$bellDatabase = New-RandomName 'bell_e2e'
|
||||
|
||||
& (Join-Path $PostgresBin 'initdb.exe') -D $postgresData -U $clusterUser -A trust --encoding=UTF8 --no-locale | Out-Null
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Isolated PostgreSQL initdb failed.' }
|
||||
$startArguments = "-D `"$postgresData`" -l `"$postgresLog`" -o `"-p $postgresPort -h 127.0.0.1`" start"
|
||||
Start-Process -FilePath (Join-Path $PostgresBin 'pg_ctl.exe') -ArgumentList $startArguments -RedirectStandardOutput (Join-Path $temporaryRoot 'pg-ctl.out.log') -RedirectStandardError (Join-Path $temporaryRoot 'pg-ctl.err.log') -WindowStyle Hidden | Out-Null
|
||||
Wait-Tcp -Port $postgresPort -Open $true
|
||||
$postgresStarted = $true
|
||||
|
||||
$psql = Join-Path $PostgresBin 'psql.exe'
|
||||
foreach ($role in @($senseRole, $bellRole)) {
|
||||
& $psql -X -h 127.0.0.1 -p $postgresPort -U $clusterUser -d postgres -v ON_ERROR_STOP=1 -c "CREATE ROLE $role LOGIN;" | Out-Null
|
||||
if ($LASTEXITCODE -ne 0) { throw "Could not create isolated role $role" }
|
||||
}
|
||||
& (Join-Path $PostgresBin 'createdb.exe') -h 127.0.0.1 -p $postgresPort -U $clusterUser -O $senseRole $senseDatabase
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Could not create isolated Sense database.' }
|
||||
& (Join-Path $PostgresBin 'createdb.exe') -h 127.0.0.1 -p $postgresPort -U $clusterUser -O $bellRole $bellDatabase
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Could not create isolated Bell database.' }
|
||||
|
||||
$senseDsn = "host=127.0.0.1 port=$postgresPort user=$senseRole dbname=$senseDatabase sslmode=disable"
|
||||
$bellDsn = "host=127.0.0.1 port=$postgresPort user=$bellRole dbname=$bellDatabase sslmode=disable"
|
||||
if ($senseDsn -eq $bellDsn -or $senseRole -eq $bellRole -or $senseDatabase -eq $bellDatabase) { throw 'Sense and Bell isolation invariant failed.' }
|
||||
|
||||
Set-TestEnvironment 'GOTOOLCHAIN' 'go1.26.5'
|
||||
Set-TestEnvironment 'PYTHONDONTWRITEBYTECODE' '1'
|
||||
Set-TestEnvironment 'SENSE_OUTBOX_TEST_DATABASE_URL' $senseDsn
|
||||
Set-TestEnvironment 'BELL_DATABASE_URL' $bellDsn
|
||||
Set-TestEnvironment 'BELL_EVENT_INGRESS_TEST_DATABASE_URL' $bellDsn
|
||||
Set-TestEnvironment 'BELL_RULE_ALERT_TEST_DATABASE_URL' $bellDsn
|
||||
Set-TestEnvironment 'BELL_ALERT_LIFECYCLE_TEST_DATABASE_URL' $bellDsn
|
||||
Set-TestEnvironment 'BELL_JWT_SECRET' (New-RandomSecret) $true
|
||||
Set-TestEnvironment 'BELL_BOOTSTRAP_USERNAME' (New-RandomName 'coord_admin')
|
||||
Set-TestEnvironment 'BELL_BOOTSTRAP_PASSWORD' (New-RandomSecret) $true
|
||||
Set-TestEnvironment 'BELL_RULE_ALERT_OPERATOR_PASSWORD' (New-RandomSecret) $true
|
||||
Set-TestEnvironment 'BELL_HOST' '127.0.0.1'
|
||||
Set-TestEnvironment 'BELL_PORT' (Get-FreeTcpPort).ToString()
|
||||
|
||||
Invoke-Checked 'Bell formal migrations' (Join-Path $repositoryRoot 'Bell\server') { go run . migrate -c config/settings.demo.yml *> (Join-Path $temporaryRoot 'bell-migrate.log') }
|
||||
|
||||
Invoke-Checked 'source-config v1 contract' $repositoryRoot { & $contractPython -m unittest discover -s contracts/tests/source-config-v1 -p 'test_*.py' -v }
|
||||
Invoke-Checked 'runtime-status v1 contract' $repositoryRoot { & $contractPython contracts/tests/runtime-status-v1/test_contract.py }
|
||||
Invoke-Checked 'machine-identity v1 cross-language contract' $repositoryRoot { & $contractPython contracts/tests/machine-identity-v1/test_contract.py }
|
||||
Invoke-Checked 'events v1 contract' $repositoryRoot { & $contractPython contracts/tests/events-v1/test_contract.py }
|
||||
Invoke-Checked 'evidence v1 contract' $repositoryRoot { & $contractPython contracts/tests/evidence-v1/test_contract.py }
|
||||
|
||||
Invoke-Checked 'Sense source/status integration' (Join-Path $repositoryRoot 'Sense\tests\integration\brain_control') { go test . -count=1 -v }
|
||||
Invoke-Checked 'Sense Brain-event/evidence/Outbox integration' (Join-Path $repositoryRoot 'Sense\tests\integration\bell_connector') { go test . -count=1 -v }
|
||||
Invoke-Checked 'Sense PostgreSQL Outbox recovery' (Join-Path $repositoryRoot 'Sense\server') { go test ./app/sense/outbox -count=1 -v }
|
||||
Invoke-Checked 'Brain source/status connector and anonymous event export' $repositoryRoot {
|
||||
& $Python -m pytest Brain/tests/integration/sense_control Brain/tests/integration/event_export -q
|
||||
}
|
||||
Invoke-Checked 'Bell ingress and evidence degradation' (Join-Path $repositoryRoot 'Bell\server') { go test ./tests/integration/event_ingress -count=1 -v }
|
||||
Invoke-Checked 'Bell rule and alert projection' (Join-Path $repositoryRoot 'Bell\server') { go test ./tests/bell_rule_alert -count=1 -v }
|
||||
Invoke-Checked 'Bell alert lifecycle' (Join-Path $repositoryRoot 'Bell\server') { go test ./tests/bell_alert_lifecycle -count=1 -v }
|
||||
|
||||
$identityFacts = (& $psql -X -h 127.0.0.1 -p $postgresPort -U $clusterUser -d postgres -tAc "select datname||':'||pg_get_userbyid(datdba) from pg_database where datname in ('$senseDatabase','$bellDatabase') order by datname;")
|
||||
if (@($identityFacts).Count -ne 2 -or ($identityFacts -join '|') -notmatch [regex]::Escape($senseRole) -or ($identityFacts -join '|') -notmatch [regex]::Escape($bellRole)) {
|
||||
throw 'PostgreSQL ownership isolation evidence is incomplete.'
|
||||
}
|
||||
|
||||
if (-not $SkipIndependentProductE2E) {
|
||||
Invoke-Checked 'Sense independent isolated E2E regression' $repositoryRoot { & (Join-Path $repositoryRoot 'Sense\tests\e2e\run-isolated-e2e.ps1') -PostgresBin $PostgresBin }
|
||||
Invoke-Checked 'Brain independent test regression' $repositoryRoot { & $Python -m pytest Brain/tests -q }
|
||||
Invoke-Checked 'Bell independent isolated E2E regression' $repositoryRoot { & (Join-Path $repositoryRoot 'Bell\tests\e2e\run-isolated-e2e.ps1') -PostgresBin $PostgresBin }
|
||||
}
|
||||
|
||||
Assert-NoSecretInLogs
|
||||
Write-Host "COORDINATION_E2E passed: versioned contracts, source/status, anonymous event, durable Outbox recovery, Bell Receipt/Event/Alert lifecycle, identity/replay/conflict/evidence faults, separate Sense/Bell databases. postgres_port=$postgresPort"
|
||||
} finally {
|
||||
if ($postgresStarted) {
|
||||
Start-Process -FilePath (Join-Path $PostgresBin 'pg_ctl.exe') -ArgumentList "-D `"$postgresData`" -m fast stop" -RedirectStandardOutput (Join-Path $temporaryRoot 'pg-stop.out.log') -RedirectStandardError (Join-Path $temporaryRoot 'pg-stop.err.log') -WindowStyle Hidden -Wait | Out-Null
|
||||
try { Wait-Tcp -Port $postgresPort -Open $false -Attempts 40 } catch {}
|
||||
}
|
||||
foreach ($entry in $savedEnvironment.GetEnumerator()) {
|
||||
[Environment]::SetEnvironmentVariable($entry.Key, $entry.Value, 'Process')
|
||||
}
|
||||
if ($KeepTemporary) {
|
||||
Write-Host "Kept coordination E2E directory: $temporaryRoot"
|
||||
} elseif (Test-Path -LiteralPath $temporaryRoot) {
|
||||
$resolved = [IO.Path]::GetFullPath($temporaryRoot)
|
||||
$tempPrefix = [IO.Path]::GetFullPath([IO.Path]::GetTempPath())
|
||||
if (-not $resolved.StartsWith($tempPrefix, [StringComparison]::OrdinalIgnoreCase) -or -not ([IO.Path]::GetFileName($resolved)).StartsWith('yovision-coordination-e2e-')) {
|
||||
throw "Refusing unsafe temporary cleanup: $resolved"
|
||||
}
|
||||
Remove-Item -LiteralPath $resolved -Recurse -Force
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
# Coordination E2E
|
||||
|
||||
Run the complete isolated coordination acceptance from the repository root:
|
||||
|
||||
```powershell
|
||||
pwsh scripts/e2e/coordination/run-coordination-e2e.ps1
|
||||
```
|
||||
|
||||
The runner creates a temporary PostgreSQL cluster on a dynamic loopback port,
|
||||
uses distinct random owners and databases for Sense and Bell, exercises the
|
||||
frozen contracts and the three connector chains, runs each product's existing
|
||||
independent regression, checks generated secrets are absent from temporary
|
||||
logs, and removes only the processes and directory it created.
|
||||
|
||||
`-SkipIndependentProductE2E` is intended only for local harness debugging and
|
||||
does not satisfy issue #155 acceptance. `-KeepTemporary` preserves disposable
|
||||
diagnostics after a failed run; the directory contains test-only generated
|
||||
credentials and must not be committed or shared.
|
||||
Reference in New Issue
Block a user