Compare commits
31
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
79929036bb | ||
|
|
e4677069d7 | ||
|
|
8cfd32babe | ||
|
|
3181302c16 | ||
|
|
2b01974fe5 | ||
|
|
305062dba0 | ||
|
|
ea38ced58c | ||
|
|
2cb78ad8cd | ||
|
|
ca997f6108 | ||
|
|
b1594dc7bd | ||
|
|
08d939a9b8 | ||
|
|
9088e6b734 | ||
|
|
fdd26afaf0 | ||
|
|
d403f3b3a0 | ||
|
|
c6a962d522 | ||
|
|
9d5242b371 | ||
|
|
9a10d82cf4 | ||
|
|
ac3c63ead6 | ||
|
|
71f7751754 | ||
|
|
76c94e33e0 | ||
|
|
57b0f1595f | ||
|
|
29ba16e4f9 | ||
|
|
ff2d0ca0e5 | ||
|
|
bf58ad0005 | ||
|
|
81c1a7ead2 | ||
|
|
aa2ecc6ef7 | ||
|
|
f27ec16307 | ||
|
|
1d029d34c7 | ||
|
|
b12460825c | ||
|
|
d7924619b6 | ||
|
|
041cd8d03f |
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Project-Profile
|
||||
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Project-Profile.-
|
||||
wiki_revision: 7468b9fbdd4d0bbbb9a73580c22ec868b3085753
|
||||
synchronized_at: 2026-09-05T07:16:39Z
|
||||
wiki_revision: 3b78360779ae520f1ff9e51be3118a04cd549f51
|
||||
synchronized_at: 2026-09-07T09:27:40Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 项目档案
|
||||
@@ -61,6 +61,7 @@ GoAuto 默认采用轻量治理:文案、注释、格式、局部样式或布
|
||||
## 环境与凭据
|
||||
|
||||
- 服务端正式环境默认必须使用 HTTPS。仅当管理员接受 Device Token、任务内容和执行结果明文传输风险,并显式设置 `GOAUTO_ALLOW_INSECURE_AGENT_HTTP=true` 时,Agent `/api/agent/v1/**` 可通过 HTTP;管理端和第三方服务不因此放宽。
|
||||
- #237 客户端密钥例外(用户 2026-09-07 明确接受风险):提交 `71f7751` 起,管理员密钥管理及 `/api/client/v1` 默认兼容 HTTP/HTTPS,无开关;HTTP 明文传输密钥及业务数据,建议优先 HTTPS。实际迁移部署仍须单独授权;此例外不改变其他第三方服务的安全边界。
|
||||
- 每台设备使用独立 Device Token;Token 只存安全配置,不进入仓库。
|
||||
- PDD 账号密码、Cookie、验证码和用户个人数据不得进入日志。
|
||||
- 根目录 `gitea.env` 是本机工单访问配置,已被 Git 忽略。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Architecture-and-Code-Map
|
||||
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Architecture-and-Code-Map.-
|
||||
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
|
||||
synchronized_at: 2026-09-04T11:29:50Z
|
||||
wiki_revision: aea36cd232cc24af80fd8576ef206fe92e6161f2
|
||||
synchronized_at: 2026-09-10T08:09:12Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 架构与代码地图
|
||||
@@ -68,6 +68,8 @@ Android Portal/Agent
|
||||
| `pdd_account` | 可选的账号调度引用,只保存名称和状态,不保存凭据 |
|
||||
| `purchase_task` | 商品外键和不可变快照、执行模式、状态/租约 guard、价格边界、订单、人工支付复核、物流与回填事实 |
|
||||
| `purchase_task_attempt` | `task_id + attempt_id` 幂等执行记录、阶段、规则哈希、固化规格决策和结构化错误 |
|
||||
|
||||
> #241 新增:`server/app/goauto/purchase/order_backfill.go` 与 `order_backfill_handler.go` 提供 `POST /api/agent/v1/purchase-tasks/order-backfill`(Device Token 鉴权,逐条事务、逐条结果);`server/app/goauto/models/purchase_order_guard.go` 在 `PurchaseTask.BeforeSave` 上全局强制订单号唯一,以 `purchase_rule_setting` 单例行串行化订单号分配,避免新增唯一索引迁移;`purchasecontract.ParseAddressSuffix` 为 `AddressSuffix` 的反解,通过回比而非负向前瞻实现(Go RE2 不支持前瞻)。
|
||||
| `ai_matching_setting` | 唯一单例的启用状态、OpenAI-compatible Base URL、模型、超时、内部部署明文 API Key 和更新人;仅管理员设置接口可以读取该字段 |
|
||||
|
||||
`collection_task` 的状态仅为 `pending`、`running`、`completed`、`completed_partial`、`failed`。设备身份和心跳表属于 Agent 领取任务的必要基础,不承载 PDD 业务数据。
|
||||
@@ -333,3 +335,97 @@ PddProductDetailCollector
|
||||
- `GET /api/v1/sysjob/:id/execution-logs` 由 `server/app/jobs/service/execution_log.go`、`apis/execution_log.go` 和 `router/sys_job.go` 提供任务级分页、状态与开始时间过滤;沿用隐藏菜单 `JobLog` 的角色菜单绑定和精确 GET 权限。Web 入口为 `web/src/views/schedule/index.vue` 的单选“日志”按钮,详情页为 `web/src/views/schedule/log.vue`。
|
||||
- 执行历史明确不保存任务参数、AI Provider 地址或密钥、第三方原始响应和业务原始载荷;当前不提供删除、保留期限自动化、WebSocket 实时流或立即执行动作。
|
||||
- 追加迁移为 `server/cmd/migrate/migration/version-local/1788357000000_sys_job_execution_log.go`:创建执行历史表、登记只读 API、关联 `JobLog` 菜单,并只给迁移前已绑定该菜单的角色补充精确 Casbin 权限。
|
||||
|
||||
## 客户端密钥访问架构(#237)
|
||||
|
||||
代码基线 `71f7751`(含用户确认的默认 HTTP/HTTPS 兼容),2026-09-07 完成 Server/Web 代码与隔离测试;本机迁移、管理员菜单写入与 Server/Web 启动已于 2026-09-07 授权完成,真实 HTTP 管理列表和模块目录加载通过;线上仍未部署。
|
||||
|
||||
- `server/app/goauto/clientkey` 管理独立密钥、授权及审计;`clientapi/routes.go` 的显式 Inventory 将 `/api/client/v1` 映射到既有业务处理器,绝不转发任意 Admin 路由。
|
||||
- `clientapi/gateway.go` 默认接受 HTTP 与 HTTPS,无协议开关或转发协议头门禁,按 Bearer 密钥、模块及能力顺序校验,每次请求读取数据库,无授权缓存。`common/clientprincipal` 传递独立客户端身份,不生成或伪装管理员 JWT。
|
||||
- `client_api_key` 保存名称、随机 256 位密钥的 SHA-256 摘要、前缀、授权 JSON、启用状态、版本、创建/修改人和最后使用时间;完整密钥仅创建响应一次返回。`client_api_key_audit` 保存管理变更和客户端请求元数据,不保存请求正文、查询参数、响应正文或凭据。
|
||||
- 编辑与停用采用启用状态和 version 条件更新,并与变更审计同事务提交;冲突返回 409。业务执行前先持久化请求审计意图,失败则不执行业务。完成后更新状态;更新失败或进程中断可能留下 status=0,表示结果待核对,不能据此自动重放。
|
||||
- 部分既有业务操作人字段使用该密钥最近授权管理员的 ID 兼容现有外键;实际调用方以独立审计的 key_id 为准,不能把业务字段当成人工操作证据。
|
||||
- Admin 页面 `web/src/views/goauto/client-keys/index.vue` 复用创建/编辑授权弹窗;菜单位于“采采管理”,仅管理员可见。新追加迁移 `1788798000000_client_api_key.go` 创建两表及管理员菜单,不改 Android。
|
||||
|
||||
## 采购规格面板预滑动兼容(#238)
|
||||
|
||||
代码基线 `58a6c1c`,Android 0.9.60 / versionCode 73(构建完成不等同于已安装/发布)。`PurchaseRehearsalExecutor.applyPostAction` 对 `openSpecPanel.swipeAfter` 只兼容解析、不执行机械预滑动,`waitAfterMs` 保留;首趟继续原 `probeSpecs` 遍历,第二趟继续原 `selectSpec` 精确查找与容器内有界滚动。其他动作的后置滑动仍沿用既有执行语义,失败不会被统一忽略。
|
||||
|
||||
`GoAutoAccessibilityService.swipePurchase` 的失败分类由 `PurchaseSwipeFailureReason` 枚举提供;共享 `swipeNode` 仅增加可选分类回调,不改变手势目标、轨迹、1500ms 回调等待或其他调用者行为。`GoAutoPurchasePanel` 日志经 `AgentForegroundService` 关联 task、attempt、device 与规则快照哈希,新增预滑动跳过/必需滑动失败标量;不记录节点文字、坐标、原始控件树、截图或凭据。
|
||||
|
||||
Server/Web、数据库和任务快照不变;旧 APK 仍有预滑动行为,必须更新 Agent 才生效。相关验证在 `PurchaseRehearsalExecutorTest`,Android 全量测试与 APK 构建入口不变。
|
||||
|
||||
## SYB 逐页保存与部分成功(#239)
|
||||
|
||||
实现绑定 c6a962d;代码已实现不代表当前线上已部署。每页完整明细在外部请求结束后按页事务保存;页回滚不累计明细/新增/覆盖数,已提交页保留。日期局部读取失败继续下一日期,全局数据库/进度/会话/取消故障停止。当天漂移不在一次运行内重扫;后续运行重新扫描并幂等补齐。
|
||||
|
||||
同步状态增加 `partial_success`(部分成功,15 字符,复用现有 varchar(16),无需迁移)。有错误且 created+updated>0 为部分成功;有错误无已提交明细为 failed;完整且无错误为 succeeded(包括无符合店铺的数据)。中断仍为 interrupted,不把中断追认为成功。所有终态沿用活动槽释放规则。
|
||||
|
||||
`orderCount` 是已验证页的原始列表读取数量;`detailCount`、`created`、`updated` 为已提交明细及其新增/覆盖数量;`daysProcessed` 是完整通过的日期数,不是已尝试日期数。失败日期/页码/阶段写入现有脱敏限长 errorMessage。部分成功不刷新店铺的完整同步统计。
|
||||
|
||||
Web 唯一展示位置为“采集采购 → SYB 同步记录”:列表状态、状态筛选及详情支持部分成功,详情保留已保存数量、错误原因与重新同步补齐提示。定时任务日志只表示异步任务受理,不等于最终业务同步成功。
|
||||
|
||||
入口为 `SyncWithShopSnapshot → loadDailyList → importSyncPage`;`server/app/goauto/sybimport/sync_page.go` 封装页事务和提交后计数;`import_handler.go` 判定终态,`sync_run.go` 保存及筛选,Web 复用 `web/src/views/goauto/syb-sync-runs/index.vue`。
|
||||
|
||||
## Android 连续临时采集收尾(#250)
|
||||
|
||||
当前实现绑定 `37e8b56`(Agent 0.9.72 / 85),用户确认 v5 按次数返回后切回 Agent。仅 Android,无 Admin/API/权限/数据库迁移;取代 v1~v4 首页导航。
|
||||
|
||||
- `AgentSettingsFragment` 在采集间隔后复用卡片和 SwitchMaterial;仅开启显示“返回次数”按钮,Material 单选框选择 1~5 次(默认 3),立即保存或取消。成功无常驻反馈,错误/忙碌显示原因。任务忙碌禁用,弹窗选定时再检查。
|
||||
- `AgentSettingsStore` 在私有 `goauto_agent_settings` 保存 `continuous_collection`(false)与 `continuous_collection_return_count`(3);次数越界或读取错误回落 3,写入失败恢复旧值,无网络加载。
|
||||
- `requestCurrentPageCollection` 保留原范围判断;`executeTask` 在采集开始固定次数。`executeTaskWhileAwake` 提交结果、结束图片上传后返回 resultStatus;`ContinuousCollectionPolicy.finish` 仅对完整确认成功调用新 `PddCollectionReturnNavigator`。失败/部分成功不导航。冷却间隔不变,连续模式不启动旧 idle-return。
|
||||
- 删除 `PddCollectionHomeNavigator` 和专属首页/搜索层判据。新导航器仅使用 capture/back、当前包名、openAgentPreservingTab;1~5 次 Back,每次后 500ms,前台离开 PDD 终止剩余 Back;保留风险场景停止。最后打开 Agent 并等待 500ms 验证包名,不验证 PDD 首页。仍在原任务互斥与唤醒锁范围内完成,结束后原 finally 释放。
|
||||
- `GoAutoCollector` 日志关联 taskId、attemptNumber、deviceId、规则快照 SHA-256;记录 continuousReturn、configuredBacks、固定 reason 和 actions。原因含 count_completed、left_pdd、back_failed、unsafe_page、agent_launch_failed、agent_not_confirmed、navigation_exception、invalid_count;actions 为 Back 尝试数,不代表最终页面。无节点文字、树、截图或个人数据落盘;不扩大采购诊断存储。
|
||||
- `ContinuousCollectionTest` 覆盖范围/终态、1~5 次、无变化仍按次数、前台变化、风险、动作/启动失败、包名确认和异常;`ContinuousCollectionSettingsTest` 覆盖开关与次数持久化、默认/越界/写入失败。`:app:testDebugUnitTest :app:assembleDebug` 376 项通过。旧首页专属测试随旧行为替换,新版未安装,UI 大字体/横屏、多设备和完整采集真机链路尚未验证。
|
||||
|
||||
## SYB 存疑规格候选解析(#251)
|
||||
|
||||
实现绑定 `64b4205`,仅 Server `app/goauto/sybimport`;无 API 字段、数据库迁移、权限、并发模式或 UI 变化。
|
||||
|
||||
- `parse_candidates.go: ParseWithCandidates` 先运行原 `Parse`,仅 uncertain 时解码并校验已存 SpecsJSON;复用 `closedShopeeCandidates` 提取颜色/尺码封闭候选,拒绝未知有值角色/重复角色。完整候选组合与原文归一比较唯一时确认,其他情况返回原 ParseResult。
|
||||
- `ApplyDetail` 在加载既有蝦皮商品后、写入规格前调用;`Reparse` 在保留人工/AI 确认保护后加载关联候选调用。新增 ParseResult 私有 matchedCandidates 标记,使 `mergeParsedSpec` 对已经存在的候选直接返回,避免按维度名重复合并或改变映射。
|
||||
- #198 `ProcessSpecAIParseRun → Reparse` 自然接入确定性优化,仍需 AI 的记录沿用原后续步骤,不新增异步任务/Provider 调用/接口。
|
||||
- 验证 `go test ./app/goauto/sybimport ./app/goauto/aimatching ./app/goauto/shopeeproduct ./app/goauto/purchase` 和 `go build ./...` 通过;新增候选正反/格式/组合色/单维度、歧义/空值、同步重解析一致、人工保护及无需调用 AI 测试。AI 原有置信度/失败重试测试改用仍需语义理解的合成输入,继续验证原分支。
|
||||
- 本轮只使用本地测试数据与 httptest AI,未触发真实 SYB 同步、外部 AI、采购、数据库迁移或服务重启;线上及历史数据效果尚待授权验证。
|
||||
|
||||
|
||||
## 蝦皮详情匹配等待与回读(#254)
|
||||
|
||||
实现绑定 `9088e6b`(2026-09-10,分支 fix/254-match-loading);已通过合成数据测试,尚未合并 main 或发布线上。
|
||||
|
||||
- Server `shopeeproduct.Service.DetailWithAutoMatchBudget` 为详情 GET 附加派生的整次匹配预算,不返回 Provider、Key 等配置。`AutoMatchMappings` 读取 AI 设置 T,使用 2T+10 秒的单个父 context;颜色、尺码、保存及结果回读共用剩余时间。初始配置读取也受有界 context 约束,读取耗时从后续预算扣除。
|
||||
- 批量定时匹配调用内部 `autoMatchMappings` 并继续使用原 MaxProviderTimeout=600 秒,不随本次详情入口调整;候选建议与其他 AI 功能不改。
|
||||
- Web `ShopeeProductDetailDrawer` 在提交前以普通 GET 刷新商品与预算,仅 auto-match POST 使用 (2T+20) 秒超时。准备失败/缺少预算时不发 POST;旧 Server 必须先更新,不能只发布新版 Web。
|
||||
- 加载状态同时绑定商品加载代次和操作序号。关闭/切换立即结束该页面等待状态;同商品重开读取详情,旧响应/旧 finally 不得污染新操作。匹配期间禁止在当前抽屉编辑映射,原人工草稿放弃确认保留。
|
||||
- POST 未正常返回时只回读详情及关联 PDD,不自动重发 POST;回读失败结束转圈并保留刷新入口。GET 沿用单请求10秒超时,完整结果回读最多两次串行 GET,不是无限等待。
|
||||
- 验证:Go shopeeproduct/aimatching/clientapi 测试与 go build;Web `node --test tests/unit/shopee-match-lifecycle.test.cjs`,Playwright shopee-match-lifecycle 与 shopee-pdd-picker,受影响 ESLint 与 build:prod。
|
||||
|
||||
|
||||
## SYB 行内一键关联最近采集(#253)
|
||||
|
||||
实现绑定 `b1594dc`,2026-09-10;分支 feat/253-one-click-link,包含 #254 依赖。已完成合成数据测试及构建,未合并 main、未部署,不代表当前线上已具备此能力。
|
||||
|
||||
- Web `syb-products/index.vue` 在“去关联”旁增加“一键关联+下拉箭头”,共享 `quick-link.js` 页面级状态。`quick-link-device-preference.js` 按当前浏览器来源、API环境、用户ID保存设备ID,独立于批量采购偏好;不存密钥或商品数据。首次主动作选机继续原行,箭头选机只保存偏好。
|
||||
- Server `shopeeproduct/latest_collection.go` 复用 LinkPDD 的可选 Admin 模式。短事务核对 SYB→Shopee、未关联/规格版本、设备状态,选择并锁定最新 completed/completed_partial 的 agent_current_page 任务及其 PDD;仅完整采集、active且有可用颜色或尺码时更新,WHERE pdd_product_id IS NULL 不覆盖其他操作者的关联。响应保留该事务产生的规格版本及来源task/device/PDD,不能用后续GET替换冻结上下文再自动匹配。
|
||||
- 不新增表、迁移、路由或权限对账。原手动关联/替换请求不变;共享 Handler 明确拒绝 Client API 使用此跨模块模式,不能借 Shopee-only write grant 读取 SYB/设备上下文。来源任务原规则快照不改,taskId 可回溯既有采集证据。
|
||||
- 关联后读取现有采购准备;需要人工确认或不具备AI资格时停止自动链路并保留关联,其他情况调用原 AutoMatchMappings,沿用 #254 动态预算及原事务保存/上下文冲突保护。最后刷新实际处理阶段,不调用创建采购/订单接口。
|
||||
- 运行期间冻结行、商品、手机;前端重复主动作禁用,切换偏好仍只影响未来操作。卸载或 keep-alive deactivated 令旧操作失效,禁止旧回调继续下一步;不声称已经取消服务端在途写入。异常仅有限GET回读和刷新,不自动重发POST。
|
||||
- 验证命令:Server `go test -p 1 ./...`、`go build ./...`;Web `node --test tests/unit/syb-quick-link.test.cjs tests/unit/shopee-match-lifecycle.test.cjs`、`pnpm exec playwright test tests/e2e/syb-quick-link.spec.ts tests/e2e/shopee-match-lifecycle.spec.ts --workers=1`、受影响文件 ESLint、`pnpm build:prod`。SQLite 单连接并发测试不代替生产 MySQL 行锁验收;真实AI、生产数据、跨设备人工闭环待验证。
|
||||
|
||||
## SYB 行内一键替换(#258)
|
||||
|
||||
实现绑定 `2b01974`;部署与验收见 #258 发布证据。本节补充 #253,不改变原未关联入口。
|
||||
|
||||
- `syb-products/quick-link.js` 复用分体按钮、设备偏好和匹配/回读流程;已关联且允许处理阶段时同位置显示一键替换。先只读预览,固定任务、目标商品及规格版本;不同商品一次确认,同商品不重绑。
|
||||
- `shopeeproduct/latest_collection.go` 在原 latestCollection 增加可选 replacement 模式。预览和提交均短事务锁定 SYB、Shopee、原PDD、设备、来源采集与目标PDD,核对旧关联及上下文。提交按预览 taskId 读取,不重新挑选后来采集的商品。
|
||||
- Admin 路由注入 `purchase.ValidateQuickReplacement`,在事务内复用采购准备数据集及处理阶段计算,不调用 AI。缺少校验函数拒绝替换;客户端密钥不注入且仍拒绝 latestCollection。无新路由、权限、表或迁移。
|
||||
- 关联完成后才执行现有 AI 匹配,不在数据库事务中等待 Provider;同一 SYB 正式采购创建与替换共用 SYB 行锁。历史采购快照不变。SQLite 合成测试覆盖冲突与串行并发请求,不能替代生产 MySQL 多连接锁验收。
|
||||
|
||||
## 辅助信息缺失的采集关联(#259)
|
||||
|
||||
实现绑定 `e467706`,仅 Server;部署/验收见 #259。替代 #253/#258 中“部分完成一律拒绝”的限制,不改变 completed 原有路径。
|
||||
|
||||
- completed_partial 的 missing 必须是非空合法列表且仅包含 salesText、shopName、reviewCount,才进入辅助信息兼容检查;title、规格/价格缺失、遍历截断、SKU数量上限及未知原因仍拒绝。
|
||||
- `shopeeproduct/partial_collection.go` 核对本次 task 的维度和值、可售SKU的完整标记/规格引用/非负价格、本次颜色价格,以及当前档案可选值由本次有效SKU覆盖,不能借旧档案合并残留的规格或价格通过。真实单维度允许,不虚构另一维度;本次明确报告缺少规格仍拒绝。
|
||||
- 一键关联、替换预览及确认提交共用验证。失败沿用409 LATEST_COLLECTION_UNAVAILABLE及现有提示,不回退更早采集、不换手机。状态 completed_partial 和 missing 原样保留;辅助字段缺失仍可在任务详情查看。
|
||||
- 无接口字段、数据库迁移、Android/Web或权限变更;普通人工关联与采购/AI资格不变。不将“允许关联”视为“已匹配或已采购”。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Business-Rules-and-Glossary
|
||||
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Business-Rules-and-Glossary.-
|
||||
wiki_revision: 670a5592a6db8301cd115295bf820f7f4b6e06d7
|
||||
synchronized_at: 2026-09-07T03:21:47Z
|
||||
wiki_revision: ab871d61c53fe5edae1b404adaebf2495d60f099
|
||||
synchronized_at: 2026-09-10T08:09:15Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 业务规则与术语
|
||||
@@ -37,6 +37,8 @@ synchronized_at: 2026-09-07T03:21:47Z
|
||||
- PDD 重新采集或更换关联后,目标规格仍存在则映射继续有效;目标规格消失时详情标记“已失效”,服务端拒绝保存不存在的目标,采购预检和创建也拒绝使用失效映射并提示重新选择。
|
||||
- 批量软删除逐条校验引用(虾皮商品被 SYB 明细或采购任务引用时不可删除)并逐条返回结果;已删除商品默认不出现在列表,可筛选查看并恢复。
|
||||
|
||||
- #257:蝦皮颜色与尺码建议/一键匹配先保留仍有效的已确认映射,再复用相同标准化比较:简体、空白、全半角与大小写统一。颜色标准化后仅命中一个当前可选 PDD 候选时,不调用 AI,建议标为 matched,一键匹配按既有流程保存 exact_match + confirmed;未命中或多个不同候选标准化后相同,仍走原 AI 与置信度/候选校验。只转换比较键,蝦皮/PDD 原规格名不改,映射目标保存 PDD 原始候选名称;不引入颜色同义词猜测、不自动重算历史映射。
|
||||
|
||||
## SYB 商品明细
|
||||
|
||||
- 一行对应 SYB 一条明确的商品/颜色/尺码/数量明细;唯一键为「蝦皮订单号 `code` + 来源明细 `id`」组合,不是全局唯一 ID(未在多货运单样本中验证过全局唯一性)。
|
||||
@@ -58,6 +60,13 @@ synchronized_at: 2026-09-07T03:21:47Z
|
||||
- 对采购人员展示的阶段固定为:待人工处理、未关联 PDD、PDD 待采集、PDD 采集中、PDD 采集失败、规格待匹配、可创建采购、已创建任务、采购成功、待人工核对。
|
||||
- 主阶段优先级为:待人工核对 → 采购成功 → 已创建任务 → 待人工处理 → 未关联 PDD → PDD 待采集/采集中/采集失败 → 规格待匹配 → 可创建采购。每行只显示一个阶段和对应下一步。
|
||||
- “待人工核对”表示订单结果不明确,必须先人工核查并禁止自动重试;“采购成功”表示已取得 PDD 订单号和下单时间,不代表已经支付。
|
||||
- 一个 PDD 订单号只能属于一个采购任务,该唯一性在采购任务保存路径上全局强制(#241)。人工处理结果未知、取消及 lifecycle 保存路径撞号时返回 `PURCHASE_ORDER_NUMBER_ALREADY_USED`,提示订单号已属于哪个任务,由采购员人工核对,不静默覆盖原值。
|
||||
- 不可逆边界之后的 `order_created` 结果回传是上述规则的例外:此时 PDD 真单已创建,发现订单号已属于其他任务时不回滚、不判失败,而是把任务降级为 `order_result_unknown`,冲突订单号以「读到订单号 X,但该号已属于任务 CG-yy」保存在任务与 attempt 的 `error_message`,`pdd_order_no` 留空以维持唯一性,保留下单时间与不可逆时间,进入既有人工处理结果未知通道。首要目标是保住「真单已存在」这一事实,不制造无记录的真实订单。
|
||||
- Agent 可按收货地址后缀 `_cg<任务号>` 批量回填订单号与下单时间(#241)。
|
||||
- Agent 侧回填为**人工触发的只读扫描**(#242):入口在采购记录页搜索按钮右侧,输入天数(默认 2)后确认启动。天数口径为滚动 N×24 小时,基准是页面读到的下单时间。采集、采购、回填三者互斥,复用既有任务互斥锁并加原子防重入,忙碌时拒绝启动而不排队、不抢占。
|
||||
- 回填扫描不假设订单列表有序:只有连续观察到至少 5 单且下单时间严格递减、期间无缺失时间时,才允许「超过指定时间即停止」;一旦出现时间回升或缺失,改为扫至内部上限并明确标记「未完整扫描」,不得把不完整结果显示为已扫完。内部上限用于限制设备占用,不作为用户可配置的门禁。
|
||||
- 回填扫描全程只读:确认收货、申请退款、催发货、去支付、立即支付、提交订单、付款、退款、取消订单、再次购买、删除订单永久排除为点击目标;点击目标必须自身可点、子树不含上述词,且不与任何含上述词的可见控件几何重叠;不得按固定坐标盲点。遇登录、验证码或风控立即停止并报告。
|
||||
- 解析不出规范后缀的订单一律跳过,其订单数据不缓存、不上传、不入日志;上传载荷只含后缀、订单号与可选下单时间。本地缓存只采纳服务端确认的事实,并区分页面读到的真实时间与 `irreversible_at` 回落估算值。后缀只承载任务号,请求不含地址全文或收件人信息;只允许回填该设备自己的正式采购任务;页面下单时间优先,缺失时回落该任务的 `irreversible_at` 并标记时间来源,两者皆空则该条失败。
|
||||
- 已失败、已取消或演练完成的旧采购任务不单独占用主阶段;当前数据仍满足条件时恢复显示“可创建采购”,旧任务继续在采购管理留痕和按既有规则处理。
|
||||
- 未选择处理阶段时,商品列表仍先返回,当前页阶段和采购准备继续异步批量读取且不调用 AI Provider;选择阶段筛选时,服务端必须先对完整查询结果派生并筛选阶段,再计算总数和分页,不能只过滤当前页。
|
||||
|
||||
@@ -67,13 +76,13 @@ synchronized_at: 2026-09-07T03:21:47Z
|
||||
- 版本迁移会对历史存活店铺回填 `normalized_name = Normalize(display_name)`;回填只改匹配键且可重复执行。若两个存活店铺回填后会得到同一键,迁移必须整体失败并保留原数据,管理员先人工消除歧义后再执行,不能静默合并、删除或改变店铺启用状态。
|
||||
- 店铺可以从 SYB 真实货运单列表发现,也允许管理员手工补充。只有管理员可以新增、改名、启停和软删除,采购员等其他角色只读。
|
||||
- 没有任何启用店铺时,导入必须在读取凭据、建立会话、验证码 OCR 和任意 SYB 网络请求之前失败,并给出“请先启用店铺”的可读提示。
|
||||
- 同步必须先拉取并校验当天原始全量列表的总数、分页和唯一 ID,再按本次同步开始时固定的启用店铺快照过滤;过滤不能降低完整性校验的请求范围或容量上限。
|
||||
- 同步先预检全范围总数及上限,每页校验原始列表条数、合法 ID 和重复,再按冻结店铺快照获取明细并页事务保存;整日结束核对总数。过滤不能降低完整性校验范围,已保存不能冒充整日完整(#239,c6a962d,已于 2026-09-08 随 d403f3b 部署线上)。
|
||||
- 只有列表与明细响应的店铺名都非空且命中启用快照时才允许写入。明细店铺名为空、变化为未启用店铺或无法匹配时跳过,并计入跳过数量。
|
||||
- 停用或软删除店铺只影响后续导入,不删除历史 SYB 商品、虾皮商品或任务数据。已有错误导入数据的清理必须先给出精确 SQL 和影响行数,再由用户单独确认。
|
||||
|
||||
## SYB 后台导入记录
|
||||
|
||||
- 导入由管理员创建,创建成功后立即转入后台执行;页面关闭不取消任务。采购员等其他已登录角色可以查看同步记录,不能开始导入。
|
||||
- 导入允许管理员和采购员(purchaser)创建,创建成功后立即转入后台执行;页面关闭不取消任务。其他已登录角色只能查看同步记录,不能开始导入(#236)。
|
||||
- 系统同一时刻只运行一个 SYB 导入任务。`syb_sync_run` 的唯一执行槽负责跨进程互斥,内存锁减少同一进程内的竞争;服务重启后遗留的执行中记录标记为“已中断”。
|
||||
- 导入失败或中断时保留已写入商品,记录明确的失败原因和已处理进度;重新导入相同范围按「订单号 + 明细 ID」覆盖,不产生重复商品。
|
||||
- 创建同步记录时冻结本次启用店铺的规范化匹配集合、展示名称快照及 SHA-256 哈希;后台执行必须只使用这一份快照,不得在开始后重新读取 `syb_shop`。同步详情保存并返回快照可用标记、快照店铺名称、哈希及各店铺“已导入/已跳过”数量。#212 之前的历史记录只有哈希和统计,明确标记为无完整快照。上述记录不保存账号、密码、Cookie、Token、验证码图片或 SYB 原始响应。
|
||||
@@ -97,6 +106,8 @@ synchronized_at: 2026-09-07T03:21:47Z
|
||||
- PDD 完整规格选择器若无主滚动容器且尚未选择规格而没有已选摘要,仍可识别为非滚动规格面板,但必须同时出现至少两个已解析维度、至少两个可选项、唯一数量控件和唯一底部订单动作;缺少其中任一强证据时保持未识别。该订单动作仍仅作页面证据,永久禁止点击。
|
||||
- Agent 可扩展,但规则必须按任务类型授权:采集规则不能创建订单,采购规则只能使用独立审核的创建订单能力。当前项目不实现支付动作、入口或测试;支付、下单和订单相关文字可以作为只读页面证据配置,但不得成为点击目标。后续支付能力必须单独评估并至少具备显式能力位、服务端开关、单笔金额上限和人工授权。
|
||||
|
||||
- #256(Android 0.9.73,实现 43703e0):采集执行器可识别无“已选/请选择”摘要且当前仅一个规格标题的滚动规格面板。必须匹配 PDD 包名、任务详情 Activity 与选择器,无登录/验证等异常,并同时具备唯一有界规格滚动区域、唯一标题及非空候选、关闭、唯一数量输入/增减、支付区和唯一下方提交订单只读动作;缺失任一证据仍拒绝。此分支只由采集上下文启用,不生成虚假选中摘要,采购与其他解析调用维持原条件。订单动作仍仅作只读证据,采集绝不点击。
|
||||
|
||||
## 采集任务
|
||||
|
||||
- 任务与结果共用 `collection_task` 主表,不建立独立 `collection_result` 主表。
|
||||
@@ -168,7 +179,7 @@ synchronized_at: 2026-09-07T03:21:47Z
|
||||
|
||||
- 管理端固定支持 `admin`(管理员)和 `purchaser`(采购员)两类业务角色;用户必须绑定一个存在且启用的角色,`role_id=0` 或停用角色不能创建或保存。
|
||||
- 采购员可读取设备状态,维护 PDD/虾皮商品和规格映射,查看与修正 SYB 商品,读取 SYB 店铺及同步记录,读取采集规则,创建/重置/删除采集任务,并创建、查看、重试及人工处理采购任务。
|
||||
- 仅管理员可管理用户、角色、菜单、接口、部门和岗位;停用设备或吊销 Device Token;新增、改名、启停、删除或发现 SYB 店铺;手动启动 SYB 同步;新增、编辑或删除采集规则;保存或测试 AI Provider 配置。
|
||||
- 仅管理员可管理用户、角色、菜单、接口、部门和岗位;停用设备或吊销 Device Token;新增、改名、启停、删除或发现 SYB 店铺;新增、编辑或删除采集规则;保存或测试 AI Provider 配置。
|
||||
- AI 规格匹配菜单对采购员硬排除:采购员不显示该菜单,角色配置也不能为采购员选中该模块;既有 API 权限不变,仍只允许读取是否启用,不得读取 Provider 地址、模型、API Key,也不得保存或测试。
|
||||
- GoAuto 菜单由代码维护的模块定义幂等写入系统菜单和菜单/API 关联;迁移只为采购员追加首次引入且默认开放的模块,不会把采购员人工取消勾选的既有模块重新加回。
|
||||
- 采购员 Casbin API 白名单由代码权限矩阵全量重建,不从角色菜单勾选反推;减少权限时旧策略必须删除。菜单勾选只控制可见模块,不能扩大采购员 API 权限。
|
||||
@@ -230,7 +241,7 @@ synchronized_at: 2026-09-07T03:21:47Z
|
||||
- 定时任务失败时明确记录失败原因,不自动重试。服务重启后由既有启动恢复逻辑处理遗留的运行中记录。
|
||||
- 单次同步内部的 SYB 只读请求(货运单总数、列表和明细)遇到暂时网络故障、5xx 或异常响应时最多执行 3 次,分别退避 1 秒、2 秒,单次 HTTP 总超时 60 秒;这不等于失败任务自动重试。明确未登录、业务失败、数据完整性错误以及任何写操作均不自动重试。
|
||||
- SYB 商品页不再提供“导入”和“同步记录”快捷按钮,也不查询、展示或轮询同步状态;后台同步成功、失败或中断均不在商品页弹出消息,商品数据由用户主动查询或刷新获取。
|
||||
- 独立“SYB 同步记录”页面是同步结果的唯一 Web 展示位置,保留运行状态、进度、数量、失败原因和店铺统计;go-admin 定时任务中的 `GoAutoSYBHourlySync` 提供“执行记录”入口。管理员可在同步记录页人工发起覆盖昨天和今天的同步,采购员只读;人工与定时同步共用导入服务、执行记录和互斥,不排队、不并发,也不立即重试。
|
||||
- 独立“SYB 同步记录”页面是同步结果的唯一 Web 展示位置,保留运行状态、进度、数量、失败原因和店铺统计;go-admin 定时任务中的 `GoAutoSYBHourlySync` 提供“执行记录”入口。管理员和采购员可在同步记录页人工发起覆盖昨天和今天的同步,其他角色只读(#236);人工与定时同步共用导入服务、执行记录和互斥,不排队、不并发,也不立即重试。
|
||||
|
||||
|
||||
## cmautobuy 商品导入
|
||||
@@ -397,7 +408,7 @@ synchronized_at: 2026-09-07T03:21:47Z
|
||||
|
||||
- 规格入口与精确规格选择统一按“重新定位唯一目标 → 执行一次无障碍点击 → 读取新页面验证”执行;不得复用旧无障碍节点,不选择相近规格,也不在多候选时默认点击第一个。
|
||||
- 无障碍点击后页面完全无变化时,只允许对解析器已确认的安全规格入口或服务端下发且唯一命中的精确规格执行一次中心手势兜底。目标必须可见、启用、边界有效,手势后仍须以规格面板强证据或精确选中证据确认结果。
|
||||
- 页面发生变化但规格面板强证据不足时,不再继续手势或猜测页面,明确失败并只记录面板类型、候选数量和证据布尔值等无敏感标量。原始控件树、节点文字集合和整屏截图仍不得保存或上传。
|
||||
- 页面发生变化但规格面板强证据不足时,默认明确失败;#243 的同商品页安全入口恢复是限定例外:确认包/Activity 未变、商品标题非空且未变、入口来源与语义未变,无面板、关闭、数量、已选、支付、订单或页面异常信号时,重新定位入口并最多执行一次受控手势。整页普通动态节点变化不单独阻断该恢复;入口缺失或显式/嵌套入口歧义仍拒绝。商品标题一致是当前上下文约束,不宣称已从页面核验 goods_id。诊断只记录入口来源、点击结果/原因、恢复结果和面板数量/布尔等无敏感标量。原始控件树、节点文字集合和整屏截图仍不得保存或上传。
|
||||
- 规格已处于精确选中状态时不得重复点击。规格查找只在解析器唯一识别的规格面板容器内有限滚动,每次滚动后重新定位容器与目标;容器缺失、歧义、到边或验证失败均 fail-closed。
|
||||
- 中心手势兜底不得用于修改/保存地址、创建或提交订单、订单详情入口以及任何支付/付款目标;正式创建订单的一次性不可逆门禁与永久禁止支付规则不变。
|
||||
## PDD 商品反向关联与继续订单采购(#161)
|
||||
@@ -444,3 +455,127 @@ synchronized_at: 2026-09-07T03:21:47Z
|
||||
- 弹窗打开后先按现有在线/可选/采购能力条件加载设备,再恢复选择并以相同设备预检。记忆设备当前不可用时保留偏好并提示用户重新选择或明确清空,不静默切换设备或自动领取。
|
||||
- 预检加载中、失败或记忆设备不可用时不能提交;过期预检结果不覆盖新的设备选择。服务端原有设备及采购资格校验不变。
|
||||
- 偏好只作用于此入口,不影响采集、其他创建入口和采购重试。浏览器存储失败时仍允许手动操作;刷新或关闭再打开浏览器可恢复,清理浏览器数据或沿用现有退出登录清理存储行为后需重新选择。
|
||||
|
||||
## 客户端密钥与可编辑模块授权(#237)
|
||||
|
||||
以下为提交 `71f7751` 已实现的规则;2026-09-07 本机已迁移并验证管理页面可用,2026-09-08 已随 d403f3b 完成线上迁移与部署,真实客户端密钥执行闭环尚未验证。
|
||||
|
||||
- 仅管理员创建、查看、编辑授权或停用密钥。首版不提供密钥改名、到期、轮换、恢复启用、删除或任意接口授权。
|
||||
- 模块选择复用“采集采购/采采管理”现有 12 个业务模块;分组勾选只影响当前子模块,新菜单不会自动获得授权。客户端密钥管理、系统账号权限及支付不在可授权模块中。
|
||||
- 勾选模块默认只读,至少保留一个模块。读写仅开放清单中的普通写操作;同步、采集、采购、删除、导入、重解析、匹配、回写及切换当前采购规则分别独立授权,默认关闭。没有普通写接口的模块不允许勾选读写。
|
||||
- 编辑既有密钥不会更换密钥,名称和前缀只读;移除模块同时移除其动作。取消保留原授权;失败保留输入;版本冲突要求刷新重开。停用不可编辑或恢复。
|
||||
- 保存后新请求按最新授权校验,已经通过校验的在途请求可能完成,不追溯回滚。模块授权不是行级、店铺级或租户隔离,授予读取即允许读取该模块明确接口可返回的业务范围。
|
||||
- 用户于 2026-09-07 明确接受明文风险:密钥管理及客户端接口默认兼容 HTTP/HTTPS,无需开关。HTTP 会明文传输密钥与业务数据,建议优先使用 HTTPS;兼容不改变管理员身份与模块授权边界。
|
||||
- 客户端与 Admin 登录 JWT、Agent Device Token 独立。响应排除凭据及原始载荷字段;AI 设置只返回 enabled,不开放 Provider 配置读写或连接测试。设备仅开放列表,不开放身份重置、令牌或解锁接口。
|
||||
- 执行动作复用既有业务门禁、幂等参数及状态机;客户端采购 batch-retry 沿用 Admin 原有语义,不等同于 Agent 就地 reset。授权重采购、支付复核、取消订单等未列入接口不开放;永久禁止付款。
|
||||
- 创建响应丢失时不可找回完整密钥,应核对列表并停用可能已创建的记录,再明确创建新密钥,不能盲目自动重试。完整密钥只在创建结果弹窗内存中显示,关闭或离开页面清空,不写浏览器持久存储。
|
||||
|
||||
## 打开采购规格面板后按需滚动(#238)
|
||||
|
||||
- Android 0.9.60 / versionCode 73,代码 `58a6c1c` 起,规则 `openSpecPanel.swipeAfter` 保留格式校验与旧快照兼容,但不执行打开面板后的固定次数预滑动;不以“必须滑两次成功”作为进入规格探测/选择的条件。动作后的 `waitAfterMs` 仍生效。
|
||||
- 首趟规格探测和第二趟精确选择仍使用各自既有的按需横向/纵向、有界与稳定终止策略。取消预滑动不等于不探测隐藏规格,也不等于只看首屏。目标不存在、歧义、页面证据不足或必要的有界查找失败时仍明确失败。
|
||||
- 此调整覆盖所有已经安全识别打开的面板,不再仅特判 NON_SCROLLABLE_CONFIRMATION;不弱化面板验证、精确选中、地址、价格、任务租约、创建订单边界或禁止支付规则。
|
||||
- 其他动作的后置滑动沿用原行为,必需滑动失败仍返回 RULE_ACTION_FAILED。旧规则 JSON 不回写、不迁移;原任务 ID、历史 attempt、商品与规格快照不变。旧 APK 行为不变,需升级 Agent;本单未改线上规则或执行 CG68 真机采购。
|
||||
|
||||
## SYB 逐页保存与部分成功(#239)
|
||||
|
||||
实现绑定 c6a962d;2026-09-08 已随 d403f3b 部署线上,未手动触发真实同步验收;#240 上游尾页超时尚未修复。每页完整明细在外部请求结束后按页事务保存;页回滚不累计明细/新增/覆盖数,已提交页保留。日期局部读取失败继续下一日期,全局数据库/进度/会话/取消故障停止。当天漂移不在一次运行内重扫;后续运行重新扫描并幂等补齐。
|
||||
|
||||
同步状态增加 `partial_success`(部分成功,15 字符,复用现有 varchar(16),无需迁移)。有错误且 created+updated>0 为部分成功;有错误无已提交明细为 failed;完整且无错误为 succeeded(包括无符合店铺的数据)。中断仍为 interrupted,不把中断追认为成功。所有终态沿用活动槽释放规则。
|
||||
|
||||
`orderCount` 是已验证页的原始列表读取数量;`detailCount`、`created`、`updated` 为已提交明细及其新增/覆盖数量;`daysProcessed` 是完整通过的日期数,不是已尝试日期数。失败日期/页码/阶段写入现有脱敏限长 errorMessage。部分成功不刷新店铺的完整同步统计。
|
||||
|
||||
Web 唯一展示位置为“采集采购 → SYB 同步记录”:列表状态、状态筛选及详情支持部分成功,详情保留已保存数量、错误原因与重新同步补齐提示。定时任务日志只表示异步任务受理,不等于最终业务同步成功。
|
||||
|
||||
|
||||
## 商品页动态变化下的采购入口恢复(#243)
|
||||
|
||||
实现 ec8b14a,Android 0.9.61/versionCode 74;已构建、未安装或真机验收。规格探测和正式采购共用入口函数,面板已打开时不新增点击;仍在同一商品上下文且安全入口存在时,一次重新定位手势恢复不再依赖整页完全静止。手势后仍未打开则报告入口无效果,不循环点击。恢复不能用于地址、创建订单、支付或未知面板,不修改面板分类器及正式下单边界。CG82/CG85 原首次点击未生效的底层原因尚待新日志和授权真机验证。
|
||||
|
||||
|
||||
## 同一颜色卡片重复选中节点(#244)
|
||||
|
||||
实现 93aab6a,Android 0.9.62/versionCode 75;已构建并授权覆盖安装,真实采购效果待验收。仅在颜色维度完整精确目标已选中、但存在非目标已选候选时,识别同一卡片重复表达:卡片必须是滚动容器直接子级的可点击非滚动 android.view.ViewGroup,节点父链明确、可见启用且边界包含;其他冲突节点仅可为该卡片自身或其中的 ImageView。另一文字选项、另一张卡片、多个精确目标、目标未选中、父链/边界不明均不适用例外。额外核对内存源节点以拒绝被解析器按同名去重隐藏的跨卡片目标;不保存原始树。
|
||||
|
||||
不删除规格数字、不模糊匹配、不改变通用解析器、尺码确认、摘要兜底、入口/滚动/地址/价格/创建订单。没有非目标选中冲突时继续原判断;仅把同卡片容器/图片的重复选中表达从冲突中排除,不新增点击。
|
||||
|
||||
|
||||
## 精确规格不可用与入口恢复诊断(#245)
|
||||
|
||||
实现 ca815c3,Android 0.9.63/versionCode 76;已通过单元测试并构建,未安装、未进行真实采购验收。入口恢复沿用 #243 条件,拒绝时通过 entryRecovery=rejected;reason=<固定原因码> 区分页面身份、商品上下文、面板/支付区域、数量/摘要、标题、入口来源/文案、歧义、规则别名及可用性;不放宽条件,不新增点击。
|
||||
|
||||
在规格面板识别出唯一完整精确候选且 available=false 时,立即返回既有 PURCHASE_SPEC_SAFE_TARGET_MISSING,提示目标颜色/尺码已售罄或当前不可选,未创建订单;已选中目标同样不能绕过不可用检查。只记录 dimension 和 exact_target_unavailable 固定结构证据,不记录规格原文或原始树。原有纵向与横向定位在此条件下本就停止,不能将本改动描述为修复“已识别售罄仍继续搜索”。未识别精确候选时继续既有有界搜索,不推断售罄,不替换规格;解析器、匹配规则、滚动预算、地址与下单流程不变。
|
||||
|
||||
## 改地址返回后的动态提交按钮(#248)
|
||||
|
||||
实现绑定 `c80c746`,Android 0.9.66 / versionCode 79;已通过 353 项 Android 单元测试并构建,尚未安装或进行真实采购验收。
|
||||
|
||||
- 地址回读与最终提交按钮定位分开判断。旧“提交订单”“现在买,仅”“确认购买”识别继续优先;存在旧文案候选但歧义时不使用结构补充选择其中一个。地址确实缺失时保留既有有限恢复;地址及采购面板上下文已确认后,不因底部按钮文字变化而滚动规格列表。按钮缺失或不唯一使用 `PURCHASE_SUBMIT_TARGET_AMBIGUOUS` 提示按钮问题,不误报地址保存超时。
|
||||
- 结构补充只属于当前执行器中已完成地址修改后的返回流程,不能由其他页面或旧执行的地址证明启用。当前须为 PDD、地址后缀按既有规则回读唯一、地址入口唯一,并存在唯一数量输入或已选摘要;通过地址入口与该证据的最近共同祖先确定整个采购弹层,不把颜色/尺码滚动区域当成弹层。排除全屏大祖先、地址编辑、已知支付 Activity 及订单结果/支付页面,后续定位要求 Activity 与地址返回时一致。
|
||||
- 旧文案无候选时,只接受弹层底部固定区域内一个独立、可见启用、非滚动的可点击容器;不在规格滚动区内选择按钮,不取控件树末项,不使用固定屏幕点击坐标。文字与人民币金额子节点合并为一个按钮;支持如“复购价,”及非支付展示文字变化,但单独价格、多个操作按钮或支付/付款/地址/返回等非提交动作均不放行。
|
||||
- 地址返回检查、最终复核和提交共用目标定位。结构路径保留原文字节点为重新定位锚点,由已有无障碍点击重新查找其可点击父级;不新增下单手势兜底。最终复核成功后才允许该结构路径提交;提交前再次检查路径、类别、文字与边界,发生变化即停止;复核失败清除本地结构提交锚点。
|
||||
- 既有规格、数量、价格和地址复核,任务租约、服务端/本地不可逆边界及单次提交规则不变。结构识别不支持支付,不增加采购员操作、配置或审批;不改变 Admin、接口、数据库、规则 JSON、通用解析器及滑动执行器。原始控件树、整屏截图、地址全文和凭据不得落盘或上传。
|
||||
|
||||
## 临时采集连续模式(#250)
|
||||
|
||||
当前实现绑定 `37e8b56`,Agent 0.9.72 / versionCode 85。2026-09-09 用户批准 v5:按配置次数返回后切回 Agent,替代此前“识别并停留 PDD 首页”的方案;不保证 PDD 最终停在首页。代码、376 项 Android 单测及 Debug 构建通过,新版设置与完整真机闭环待验收。
|
||||
|
||||
- 连续采集默认关闭,本机即时保存,失败恢复原值;开启后显示返回次数选择,1~5 次、默认 3,取消不改,写入失败恢复旧值。任务忙碌时开关和次数均禁用,选择提交时再次检查;无新增权限。只用于本次人工新建普通临时采集(source=agent_current_page、attemptNumber=1、无替换来源),Admin 下发、重试、替代与采购不参与。
|
||||
- 单任务执行开始固定返回次数。完整 completed、服务端提交确认且既有图片上传流程结束后,执行有限 Back,每次后等待 500ms;不要求识别商品/搜索/相机/首页,也不要求画面发生变化。图片上传仍沿用既有非致命行为,不因收尾更改结果。
|
||||
- 每次 Back 前检查前台仍为 PDD,读取页面后再次检查;离开 PDD 则停止余下 Back。次数完成、离开 PDD 或 Back 动作失败后,尝试打开 Agent 保持原 Tab,等待 500ms 并验证前台包名。动作失败不重试,不把请求启动成功直接当作切回成功。
|
||||
- 沿用风险页安全边界:登录/验证码/风控、地址编辑、订单详情或支付等证据时保留现场,不发送 Back 或强行切回;Activity 缺失、读取异常明确停止。以上例外不引入首页门禁。
|
||||
- 导航在原任务互斥和唤醒锁内完成,不变更租约/调度。返回失败只显示人工处理提示,不把成功采集改为失败或自动重采。
|
||||
- 失败、部分成功、提交未确认保留现场,不运行返回步骤,也不触发旧空闲返回倒计时。关闭开关保持原流程,采集间隔不变。
|
||||
- 不自动拍照、搜索或创建下一任务,不涉及采购/订单/付款;不使用深链、浏览器、OCR/VLM 或坐标兜底。
|
||||
|
||||
## SYB 存疑规格候选核对(#251)
|
||||
|
||||
实现绑定 `64b4205`,基线 main `fdd26af`;2026-09-09 代码与本地测试/构建通过,未部署、未重跑生产历史数据。
|
||||
|
||||
- 原 Parse 的成功与失败行为不变,仅 uncertain 明细增加既有蝦皮规格候选核对。比较完整原文,沿用【备注】清理,允许全角逗号、逗号两侧空白及连续空白归一;颜色尺码可正序或倒序,使用逗号或空白分隔,只有唯一完整组合才改为 success。返回既有候选标签,不改写标签,不删除加号、不截断体重说明、不做近似匹配。
|
||||
- 单规格只有在候选确实仅包含该角色时通过;有两个角色时不能用单值补猜另一个角色。空/损坏候选、校验不通过、同角色多维、未知有值维度、多种组合或归一化碰撞,均保留原结果。候选必须已存在,不能用本次待确认输入生成候选再自我证明。
|
||||
- 同步入库 ApplyDetail、手动 Reparse 共用;成功核对不再次合并规格,因此不新增重复维度、不覆盖已有映射。人工确认仍优先,默认 Reparse 跳过人工与 AI 已确认值;原文保留。
|
||||
- #198 批处理原本先 Reparse,所以可直接复用本优化:能确定的不用调用 AI;仍不确定的按原 AI 流程处理。没有新增调度、调用额度或同步等待,不自动开启已有任务。
|
||||
- 不取消 uncertain/failed 状态,不改变采购可信度/可采购规则,不实现一键关联 PDD。部署后新同步或显式重新解析会使用新逻辑;本次没有执行生产批量更新。
|
||||
|
||||
|
||||
## 蝦皮详情匹配等待与回读(#254)
|
||||
|
||||
实现绑定 `9088e6b`(2026-09-10,分支 fix/254-match-loading);已通过合成数据测试,尚未合并 main 或发布线上。
|
||||
|
||||
- AI 设置中的 timeoutSeconds=T 仍是单次 Provider 请求的上限。详情一键匹配的整次预算为 2T+10 秒,浏览器 POST 等待上限为 2T+20 秒;例如 T=60 时130/140秒,T=180时370/380秒。只保留现有一个 AI 设置入口,不新增时间配置框,也不自动改变管理员已保存的 T。
|
||||
- 超时上限不是固定等待。颜色/尺码串行执行且共用整次预算;正常部分匹配仍按原事务保存合格结果,AI异常在保存前返回时不提交本轮新映射。既有置信度、候选、人工确认映射保护和采购资格不变。
|
||||
- 前端超时、关闭或切换详情不代表后端已取消、未保存或已回滚;必须以回读结果为准,无法回读显示结果未确认,不自动重复匹配、不自动创建采购。
|
||||
- 单商品详情的成功/失败/超时/关闭重开都必须结束失效的等待状态;旧请求不得覆盖新商品或新操作。匹配进行期间本抽屉不能人工改映射,已有未保存草稿仍按原确认流程处理。
|
||||
- UI匹配提示的等待上限仅指POST,不含提交前GET及结果核对GET;结果核对也有限时,不能让失败提示再次陷入无限转圈。
|
||||
|
||||
|
||||
## SYB 行内一键关联最近采集(#253)
|
||||
|
||||
实现绑定 `b1594dc`,2026-09-10;分支 feat/253-one-click-link,包含 #254 依赖。已完成合成数据测试及构建,未合并 main、未部署,不代表当前线上已具备此能力。
|
||||
|
||||
- 未关联 PDD 且既有处理阶段提供“去关联”的 SYB 行可用一键关联;原人工入口不删除。首次点击选择采集手机后继续当前行,后续使用记忆手机;小箭头查看/切换手机,保存只修改全行共用偏好。取消不改变偏好、不执行关联。与批量采购设备选择互不影响。
|
||||
- 以所选手机的临时采集 source=agent_current_page 为范围,从 completed、completed_partial 中按 finished_at DESC、id DESC 取最新一条;失败/待执行/执行中不参与。最新为部分成功或商品停用、无可用规格时明确停止,不回退更旧商品,不切换其他手机。离线手机允许使用已有成功结果,停用或已删除手机需要重新选择。
|
||||
- 主动作开始冻结行、蝦皮商品和设备;服务端关联事务固定来源采集任务、PDD商品与规格版本。仅未关联时自动写入,已有或被其他操作改变的关联不覆盖。关联作用于蝦皮商品,因此共用该蝦皮档案的其他 SYB 明细也受影响;切换手机不会重绑已有商品,也不会改变在途操作。
|
||||
- 先保存关联,再复用现有颜色/尺码自动匹配及保存。有效已确认映射保留;普通部分匹配保存合格结果,AI故障不提交本轮新映射,但已保存关联保留。后续可从“去匹配”或详情人工继续。failed 且无人工/AI确认时不能自动跳过人工解析;uncertain 不统一拦截,仍以原采购准备规则为准。
|
||||
- 操作完成不保证立即可采购,必须刷新现有处理阶段;缺规格、缺映射、解析需确认或其他原资格不满足时继续显示真实待处理阶段。不自动创建采购任务、不下单、不支付。
|
||||
- 请求超时/网络中断只做有限回读,不重复提交;读回失败明确结果未确认,避免无限转圈。离开页面不代表后端停止或回滚。普通用户沿用现有采购员权限,无新管理员专属开关。
|
||||
|
||||
## SYB 已关联商品的一键替换(#258)
|
||||
|
||||
实现绑定 `2b01974`;部署与验收见 #258 发布证据。
|
||||
|
||||
- 复用一键关联及小箭头的位置,保留原查看 PDD 商品。PDD 待采集、采集失败、停用或缺价格的对应处理阶段可替换;采集中、已创建/运行采购、成功或结果未知不开放。服务端提交时再次检查,不能只相信按钮。
|
||||
- 复用每用户/浏览器/API环境的手机偏好;首次选择继续当前行,小箭头仅切换偏好。预览仅选所选手机最新 completed/completed_partial 临时采集,部分成功或不可用不回退旧商品。确认后固定来源任务和目标,不改用新采集;来源状态、设备、目标规格或旧关联改变时拒绝。
|
||||
- 不同商品显示一次确认,包含旧/新商品、手机和共享蝦皮档案影响;取消不写入。同商品不弹替换确认且不重写关联,只继续匹配。共享该蝦皮档案的 SYB 明细使用新关联,历史采购任务快照不改。
|
||||
- 沿用既有更换关联和有效映射检查,再复用自动匹配保存。部分匹配、AI失败或解析需人工处理时保留已保存关联,显示真实处理阶段,不保证立即可采购、不自动创建采购。
|
||||
- 网络异常只有限回读,区分目标商品、仍为原商品和其他关联,不把“已有某个关联”误报为本次替换成功;不自动重发写请求。没有新增后台作业、订单或付款操作。
|
||||
|
||||
## 辅助信息缺失的采集关联(#259)
|
||||
|
||||
实现绑定 `e467706`,仅 Server;部署/验收见 #259。替代 #253/#258 中“部分完成一律拒绝”的限制,不改变 completed 原有路径。
|
||||
|
||||
- completed_partial 的 missing 必须是非空合法列表且仅包含 salesText、shopName、reviewCount,才进入辅助信息兼容检查;title、规格/价格缺失、遍历截断、SKU数量上限及未知原因仍拒绝。
|
||||
- `shopeeproduct/partial_collection.go` 核对本次 task 的维度和值、可售SKU的完整标记/规格引用/非负价格、本次颜色价格,以及当前档案可选值由本次有效SKU覆盖,不能借旧档案合并残留的规格或价格通过。真实单维度允许,不虚构另一维度;本次明确报告缺少规格仍拒绝。
|
||||
- 一键关联、替换预览及确认提交共用验证。失败沿用409 LATEST_COLLECTION_UNAVAILABLE及现有提示,不回退更早采集、不换手机。状态 completed_partial 和 missing 原样保留;辅助字段缺失仍可在任务详情查看。
|
||||
- 无接口字段、数据库迁移、Android/Web或权限变更;普通人工关联与采购/AI资格不变。不将“允许关联”视为“已匹配或已采购”。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Android-Agent-API-Contract
|
||||
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Android-Agent-API-Contract.-
|
||||
wiki_revision: f3242938de4ac55c47f5c6eebd69184024e6a0ea
|
||||
synchronized_at: 2026-09-05T09:04:42Z
|
||||
wiki_revision: a2e73f5b892385a3c306dfdeca543a3aeb89a32e
|
||||
synchronized_at: 2026-09-10T08:09:35Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# MVP 共享 API 契约
|
||||
@@ -75,9 +75,9 @@ POST /api/admin/v1/syb-products/reparse-batch
|
||||
PATCH /api/admin/v1/syb-products/{productId}/correction
|
||||
```
|
||||
|
||||
`import` 仅允许管理员调用。请求体提交 `dateFrom`、`dateTo` 后创建持久化后台任务并立即以 `202` 返回 `runId` 和 `status=running`;关闭弹窗、刷新或离开页面不影响任务。没有启用店铺时必须在读取凭据、登录、验证码 OCR 和任意 SYB 网络请求之前返回 `422`。任意时刻只能有一条 `running` 记录,内存锁与数据库唯一执行槽共同阻止单进程和跨进程重复导入;冲突时返回正在执行任务的日期范围。
|
||||
`import` 允许已认证的管理员(admin)和采购员(purchaser)调用(#236),仍须通过 Casbin 权限校验;其他角色返回 403。采购员的 POST 权限由既有启动权限对账写入,不需要新增数据库迁移。操作人取已认证 claims,不接受客户端冒名。请求体提交 `dateFrom`、`dateTo` 后创建持久化后台任务并立即以 `202` 返回 `runId` 和 `status=running`;关闭弹窗、刷新或离开页面不影响任务。没有启用店铺时必须在读取凭据、登录、验证码 OCR 和任意 SYB 网络请求之前返回 `422`。任意时刻只能有一条 `running` 记录,内存锁与数据库唯一执行槽共同阻止单进程和跨进程重复导入;冲突时返回正在执行任务的日期范围。
|
||||
|
||||
`sync-runs` 列表支持 `page`、`pageSize`、`status`、`dateFrom`、`dateTo`,详情返回日期范围、状态(`running` / `succeeded` / `failed` / `interrupted`)、处理天数、货运单/明细/新增/覆盖数量、店铺准入与跳过数量、店铺筛选快照哈希、按店铺的 `accepted` / `skipped` 统计、操作人和起止时间。列表和详情对已登录角色只读开放。服务启动时遗留的 `running` 任务改为 `interrupted`;中途失败或中断已经写入的数据保留,重新导入仍按唯一键覆盖。
|
||||
`sync-runs` 列表支持 `page`、`pageSize`、`status`、`dateFrom`、`dateTo`,详情返回日期范围、状态(`running` / `succeeded` / `partial_success` / `failed` / `interrupted`)、处理天数、货运单/明细/新增/覆盖数量、店铺准入与跳过数量、店铺筛选快照哈希、按店铺的 `accepted` / `skipped` 统计、操作人和起止时间。列表和详情对已登录角色只读开放。服务启动时遗留的 `running` 任务改为 `interrupted`;中途失败或中断已经写入的数据保留,重新导入仍按唯一键覆盖。
|
||||
|
||||
列表返回结构化字段(`orderCode`、`shopeeItemId`、`productTitle`、`targetColor`、`targetSize`、`quantity`、`unitPriceCent`、`imageUrl`、`parseStatus`、`parseNote`、`manuallyConfirmed`),不含原始 JSON;`keyword` 匹配订单号、虾皮商品ID 或商品标题,`parseStatus` 筛选 `success`/`uncertain`/`failed`。详情额外返回 `rawJson`(原始 `details[]` 元素,未做任何改写)。
|
||||
|
||||
@@ -472,7 +472,7 @@ POST /api/agent/v1/tasks/{taskId}/fail
|
||||
|---|---:|---:|---:|---:|
|
||||
| `openProduct` | 是 | 是 | 是 | 否 |
|
||||
| `verifyProduct` | 是 | 是 | 否 | 否 |
|
||||
| `openSpecPanel` | 是 | 是 | 是 | 否 |
|
||||
| `openSpecPanel` | 是 | 是 | 兼容读取、不执行(0.9.60+,见 #238) | 否 |
|
||||
| `selectSpec` | 是 | 是 | 是 | 否 |
|
||||
| `setQuantity` | 是 | 是 | 否 | 否 |
|
||||
| `verifyUnitPrice` | 是 | 是 | 否 | 否 |
|
||||
@@ -486,7 +486,7 @@ POST /api/agent/v1/tasks/{taskId}/fail
|
||||
- 文字候选按控件文字或内容描述**精确匹配**;候选合并后必须唯一命中。点击动作只允许点击唯一文字节点或其最近的可点击父容器,不允许模糊匹配、猜测相近候选、改点兄弟节点。
|
||||
- 动作 `textAliases` 不能包含地址修改、创建/提交订单、订单号或支付相关文字,防止用安全 action 绕过危险动作类型和能力门禁。只读识别字段使用独立校验:允许订单和支付证据,仍拒绝修改地址、收货地址,并沿用各字段声明的数量、长度、去重和空白限制。
|
||||
- `waitAfterMs` 表示动作成功后的等待时间,范围为 0~30000 毫秒;省略时为 0。
|
||||
- `swipeAfter` 表示动作成功后执行一个有限滑动计划。`direction` 只能为 `up` / `down` / `left` / `right`,`count` 为 1~10,`durationMs` 为 100~2000,`intervalMs` 为 0~5000 且省略时为 0。
|
||||
- `swipeAfter` 通常表示动作成功后执行一个有限滑动计划;Android 0.9.60+ 的 `openSpecPanel` 例外,只兼容读取而不执行预滑动(见 #238)。`direction` 只能为 `up` / `down` / `left` / `right`,`count` 为 1~10,`durationMs` 为 100~2000,`intervalMs` 为 0~5000 且省略时为 0。
|
||||
- 未在矩阵中授权的 action/参数组合、未知字段、空候选和越界值一律拒绝。`updateShippingAddress`、`createOrder`、`readOrderResult` 等正式动作在其独立高风险契约完成前不接受上述参数。
|
||||
- 旧的仅含 `actions[].type` 的规则继续有效:候选使用 Agent 内置语义,等待为 0,不执行动作后滑动。
|
||||
- 服务端保存创建任务时收到的完整原始规则快照;规则后来更新为规则 B,不会改变已有任务中的规则 A 快照。
|
||||
@@ -556,7 +556,9 @@ Android #42/#36 使用本地 SQLite 保存恢复与重传所需的任务、attem
|
||||
创建订单前先在本地事务保存 `order_submit_started`、不可逆时间、稳定的 `orderSubmitRequestId` 和脱敏最终确认快照,再调用服务端同名接口;只有两侧标记完成并重新校验商品、规格、数量、单价、地址后缀、PDD 包/Activity 与唯一创建订单按钮后,才点击一次。重启时重放同一标记请求并只读核单;无法取得唯一未付款订单号和 PDD 下单时间时提交 `order_result_unknown`。支付文字仅用于识别未付款/离开支付页,永不点击。服务端数据库仍是最终事实来源;双方均不保存原始控件树、截图、PDD 凭据或完整收货地址。
|
||||
|
||||
创建订单后若出现 Android 多微信应用选择器,只读核单器必须同时确认前台包为 `android` / `com.android.intentresolver`、Activity 为白名单 `ChooserActivity` / `ResolverActivity`、页面出现已知系统选择器标题且至少一个候选以“微信”开头,才允许执行一次系统返回;不得点击任何微信候选。若前台已经是精确微信包 `com.tencent.mm`,只读核单器不得点击、输入、登录、支付或强制停止微信,只允许执行一次无参数 PDD 启动 Intent 并等待既有 PDD 任务栈回到前台;`startActivity()` 成功只表示恢复请求已发起。请求后在固定最多 15 次、每次 200ms 的宽限期内允许微信或空窗口短暂残留,不执行点击、返回、输入或滑动;观察到 PDD 后结束宽限,宽限超时、已经观察到 PDD 后再次进入微信或出现稳定未知应用时返回未知结果。随后若前台为 PDD `com.xunmeng.pinduoduo.app_pay.core.PayActivity` 或当前页面出现支付动作文字,最多再返回一次。返回后只读解析唯一订单号和下单时间;选择器或支付页重复出现、白名单不成立、恢复动作重复、无法到达订单详情、结果不唯一或超时均返回未知结果,禁止再次点击创建订单、取消订单或支付。
|
||||
`order_result_unknown` 保持订单号和下单时间为空,但允许携带创建订单前已经严格验证的 `actualUnitPriceCent`。Agent 同时提交脱敏稳定失败阶段,服务端只接受白名单并按错误码写入固定提示,不信任或保存页面原文;阶段覆盖空窗口超时、选择器返回失败、微信恢复失败/超时、未知应用、支付页返回失败/重复、订单上下文缺失、订单号缺失/歧义、下单时间缺失/无效和待付款证据缺失。任务与 attempt 保存同一错误阶段,Admin 和 Agent 历史读取数据库最终事实;旧 Agent 未提交阶段时归一为 `PURCHASE_ORDER_RESULT_UNKNOWN`。
|
||||
Agent 主动提交 `order_result_unknown` 时保持订单号和下单时间为空,但允许携带创建订单前已经严格验证的 `actualUnitPriceCent`。Agent 同时提交脱敏稳定失败阶段,服务端只接受白名单并按错误码写入固定提示,不信任或保存页面原文;阶段覆盖空窗口超时、选择器返回失败、微信恢复失败/超时、未知应用、支付页返回失败/重复、订单上下文缺失、订单号缺失/歧义、下单时间缺失/无效和待付款证据缺失。任务与 attempt 保存同一错误阶段,Admin 和 Agent 历史读取数据库最终事实;旧 Agent 未提交阶段时归一为 `PURCHASE_ORDER_RESULT_UNKNOWN`。
|
||||
|
||||
#241 追加:全局订单号唯一性校验在人工处理结果未知、取消及 lifecycle 保存路径返回 `PURCHASE_ORDER_NUMBER_ALREADY_USED`(HTTP 409、`retryable=false`),提示“订单号已属于任务 CG-任务ID”;批量回填继续使用原有 `PURCHASE_BACKFILL_ORDER_ALREADY_USED`。不可逆边界后的 `order_created` 结果回传为例外:发现该号已属于其他任务时成功受理结果,将 `order_submit_started` 降级为 `order_result_unknown`,不回滚或自动重派。冲突订单号不写入 `pdd_order_no`,而以“读到订单号 X,但该号已属于任务 CG-yy”保存到任务和 attempt 的 `error_message`,两者 `error_code` 均为 `PURCHASE_ORDER_NUMBER_ALREADY_USED`;保留下单时间、不可逆时间及实际单价。attempt 以 failed 结束并保留原始 `order_created` 结果类型、请求 ID 和摘要,重复提交按原幂等协议返回;任务释放租约和运行槽,进入既有人工处理结果未知通道,权限不变。
|
||||
|
||||
| 错误码 | 普通提示 |
|
||||
|---|---|
|
||||
@@ -867,3 +869,259 @@ X-GoAuto-Device-Recovery-Code: <one-time-code>
|
||||
Agent 携带既有 Token(可已失效)及恢复码重新调用注册接口。服务端必须同时校验同一 `installId`、未停用状态、恢复码摘要、未过期和未使用;成功后使用原 `deviceId` 写入新 Token 摘要并返回一次新 Token,清除恢复码摘要和有效期。旧 Token 与恢复码都立即失效,已分配的 pending 采集或采购任务保持原 `deviceId`,不创建替代设备记录。缺少或错误恢复码仍为 `DEVICE_INSTALL_ID_CONFLICT`;过期码为 `DEVICE_RECOVERY_EXPIRED`;停用设备为 `DEVICE_DISABLED`。
|
||||
|
||||
自 #223 起,新建 SYB 任务在保持 `mappedColor` / `mappedSize` 为空和首趟 `spec_probe` 不变的同时,把创建时与目标规格对应的 confirmed 商品映射冻结为仅供服务端决策的指导快照。服务端收到当次候选后按角色验证该快照:只有规范化后唯一对应当次候选时才复用,并固化当次候选原文;否则该角色继续执行确定性匹配,仍未解决才把该角色及其封闭候选交给 AI。已解决角色不得重复发送给 AI,最终颜色和尺码仍须逐字属于各自当次候选。任务决策快照通过 `roleSources` 记录每个角色的 `manual_mapping` / `exact_match` / `ai_match` 来源;任务级 `specSource` 使用现有枚举汇总,不新增 Agent 决策权限。
|
||||
|
||||
## 客户端 API 与管理员密钥管理(#237)
|
||||
|
||||
实现基线 `71f7751`;以下接口已通过隔离测试;2026-09-07 本机 MySQL 迁移及管理员通过 HTTP 读取列表、模块目录已验证,真实密钥创建/编辑/停用及业务访问仍未联调,线上尚未部署。Android 接口不变。
|
||||
|
||||
### 管理接口
|
||||
|
||||
前缀 `/api/admin/v1/client-keys`,要求 Admin JWT 和 admin 角色,默认接受 HTTP/HTTPS,响应 `Cache-Control: no-store`。
|
||||
|
||||
| 方法与相对路径 | 输入 | 成功 data |
|
||||
|---|---|---|
|
||||
| GET 空路径 | page,固定每页 20 | items、total、page、pageSize |
|
||||
| GET /modules | 无 | 模块数组:key、title、group、writable、actions |
|
||||
| POST 空路径 | name(1~80 字符)、grants | key 元数据与仅本次返回的 secret |
|
||||
| PATCH /:keyId/grants | version、grants | 更新后的元数据 |
|
||||
| POST /:keyId/disable | version | 停用后的元数据 |
|
||||
|
||||
授权示例:`{"module":"pdd_products","write":false,"actions":[]}`;grants 为非空数组。元数据包括 id、name、prefix、enabled、version、grants、createdBy、updatedBy、createdAt、updatedAt、lastUsedAt,不返回摘要或完整密钥。管理请求只接受单个 JSON 对象、拒绝未知字段、最大 64 KiB。成功 code=200;无效输入 422、不存在 404、授权版本冲突或已停用 409。
|
||||
|
||||
### 客户端访问与错误语义
|
||||
|
||||
- 前缀 `/api/client/v1`,只接受 `Authorization: Bearer <客户端密钥>`,不接受 Cookie 或 URL 凭据,不与 JWT、Device Token 通用。
|
||||
- 根据用户 2026-09-07 的明确确认,管理与客户端接口在所有环境默认接受 HTTP/HTTPS,不设置协议开关,也不依赖 X-Forwarded-Proto 或 GOAUTO_TRUST_FORWARDED_PROTO。HTTP 明文传输密钥及业务数据,优先使用 HTTPS;不影响其他接口各自的协议要求。
|
||||
- 不再因 HTTP 返回 426;401 为缺失、无效或停用密钥;403 为模块/动作未授权;400 为查询参数携带凭据;503 为认证或审计暂不可用。业务错误沿用各既有接口。
|
||||
- 一般请求体最大 16 MiB;响应只支持有限 JSON(32 MiB),递归过滤凭据与原始载荷字段。不支持直接流式/二进制文件接口。响应不可序列化或超限时返回 502;业务可能已执行,必须先核对结果,不要自动重试。
|
||||
- 通过密钥认证的请求由服务端生成 `X-Client-Request-Id` 关联审计;它不是业务幂等键,原业务接口要求的 requestId 等字段仍须提供。允许请求必须先落审计意图;完成状态更新失败可留下 status=0。无效密钥没有 key_id 关联审计;拒绝授权的 403 审计为尽力记录。
|
||||
- GET `/ai-matching-settings` 只返回 `data.enabled`。POST `/ai-matching-settings/resolve` 接受 targetColor、targetSize、colors、sizes;每组最多 200 项,每个值最多 255 字符,总请求最大 64 KiB;确定性优先,必要时使用当前 Provider,返回 mappedColor、mappedSize、source;不保存映射、不创建任务,无法可靠匹配返回 422 安全提示。
|
||||
|
||||
### 明确开放的接口清单
|
||||
|
||||
下表路径均相对 `/api/client/v1`;普通业务请求字段沿用本文对应 Admin 业务契约。read=选中模块,write=模块读写,其他值均需 actions 逐项授权。没有列出的 Admin 接口不能用客户端密钥调用;新增 Admin 路由不会自动开放。
|
||||
|
||||
| 方法 | 路径 | 模块键 | 能力 |
|
||||
|---|---|---|---|
|
||||
| POST | `/ai-matching-settings/resolve` | ai_matching | match |
|
||||
| GET | `/devices` | devices | read |
|
||||
| GET | `/pdd-products` | pdd_products | read |
|
||||
| GET | `/pdd-products/:productId` | pdd_products | read |
|
||||
| POST | `/pdd-products` | pdd_products | write |
|
||||
| PATCH | `/pdd-products/:productId` | pdd_products | write |
|
||||
| GET | `/shopee-products` | shopee_products | read |
|
||||
| GET | `/shopee-products/:productId` | shopee_products | read |
|
||||
| POST | `/shopee-products` | shopee_products | write |
|
||||
| PATCH | `/shopee-products/:productId` | shopee_products | write |
|
||||
| POST | `/shopee-products/:productId/link-pdd` | shopee_products | write |
|
||||
| POST | `/shopee-products/:productId/specs/values` | shopee_products | write |
|
||||
| PUT | `/shopee-products/:productId/specs/mapping` | shopee_products | write |
|
||||
| DELETE | `/shopee-products/:productId/specs/values` | shopee_products | delete |
|
||||
| DELETE | `/shopee-products/:productId/specs/mapping` | shopee_products | delete |
|
||||
| POST | `/shopee-products/batch-delete` | shopee_products | delete |
|
||||
| POST | `/shopee-products/:productId/specs/mapping/auto-match` | shopee_products | match |
|
||||
| POST | `/shopee-products/:productId/specs/mapping/confirm` | shopee_products | match |
|
||||
| GET | `/syb-products` | syb_products | read |
|
||||
| GET | `/syb-products/:productId` | syb_products | read |
|
||||
| PATCH | `/syb-products/:productId/correction` | syb_products | write |
|
||||
| POST | `/syb-products/:productId/reparse` | syb_products | reparse |
|
||||
| POST | `/syb-products/reparse-batch` | syb_products | reparse |
|
||||
| GET | `/syb-products/sync-runs` | syb_sync_runs | read |
|
||||
| GET | `/syb-products/sync-runs/:runId` | syb_sync_runs | read |
|
||||
| POST | `/syb-products/import` | syb_sync_runs | sync |
|
||||
| GET | `/syb-inner-codes` | syb_inner_codes | read |
|
||||
| GET | `/syb-inner-codes/:recordId` | syb_inner_codes | read |
|
||||
| GET | `/syb-inner-codes/match-jobs/:jobId` | syb_inner_codes | read |
|
||||
| GET | `/syb-inner-codes/apply-batches/:batchId` | syb_inner_codes | read |
|
||||
| POST | `/syb-inner-codes/rematch` | syb_inner_codes | match |
|
||||
| POST | `/syb-inner-codes/import` | syb_inner_codes | import |
|
||||
| POST | `/syb-inner-codes/batch-delete` | syb_inner_codes | delete |
|
||||
| POST | `/syb-inner-codes/apply-preview` | syb_inner_codes | writeback |
|
||||
| POST | `/syb-inner-codes/apply` | syb_inner_codes | writeback |
|
||||
| POST | `/syb-inner-codes/:recordId/recheck` | syb_inner_codes | match |
|
||||
| GET | `/syb-shops` | syb_shops | read |
|
||||
| POST | `/syb-shops` | syb_shops | write |
|
||||
| PATCH | `/syb-shops/:shopId/name` | syb_shops | write |
|
||||
| PATCH | `/syb-shops/:shopId/enabled` | syb_shops | write |
|
||||
| DELETE | `/syb-shops/:shopId` | syb_shops | delete |
|
||||
| GET | `/collection-rules` | collection_rules | read |
|
||||
| POST | `/collection-rules` | collection_rules | write |
|
||||
| PATCH | `/collection-rules/:ruleId` | collection_rules | write |
|
||||
| DELETE | `/collection-rules/:ruleId` | collection_rules | delete |
|
||||
| GET | `/purchase-rules` | purchase_rules | read |
|
||||
| GET | `/purchase-rules/current` | purchase_rules | read |
|
||||
| POST | `/purchase-rules` | purchase_rules | write |
|
||||
| PATCH | `/purchase-rules/:ruleId` | purchase_rules | write |
|
||||
| DELETE | `/purchase-rules/:ruleId` | purchase_rules | delete |
|
||||
| PUT | `/purchase-rules/current` | purchase_rules | activate |
|
||||
| GET | `/collection-tasks` | collection_tasks | read |
|
||||
| GET | `/collection-tasks/:taskId` | collection_tasks | read |
|
||||
| POST | `/collection-tasks` | collection_tasks | collect |
|
||||
| POST | `/collection-tasks/batch` | collection_tasks | collect |
|
||||
| POST | `/collection-tasks/:taskId/reset` | collection_tasks | collect |
|
||||
| DELETE | `/collection-tasks/:taskId` | collection_tasks | delete |
|
||||
| GET | `/purchase-tasks` | purchase_tasks | read |
|
||||
| GET | `/purchase-tasks/:taskId` | purchase_tasks | read |
|
||||
| POST | `/purchase-tasks/batch-preview` | purchase_tasks | purchase |
|
||||
| POST | `/purchase-tasks` | purchase_tasks | purchase |
|
||||
| POST | `/purchase-tasks/batch` | purchase_tasks | purchase |
|
||||
| POST | `/purchase-tasks/batch-retry` | purchase_tasks | purchase |
|
||||
| POST | `/purchase-tasks/stock` | purchase_tasks | purchase |
|
||||
| GET | `/ai-matching-settings` | ai_matching | read |
|
||||
|
||||
### openSpecPanel 后置滑动兼容与诊断(#238)
|
||||
|
||||
版本边界:Android 0.9.60 / versionCode 73,代码 `58a6c1c`。不修改 JSON schema、能力标识、任务接口或已有快照哈希。`openSpecPanel.swipeAfter` 仍按 direction/count/durationMs/intervalMs 原约束校验;解析成功后不执行该准备性滑动,`waitAfterMs` 保留。其他动作后置滑动沿用旧执行语义。旧 Server 可继续下发原快照;旧 APK 仍按原策略执行,不能将本契约描述当作旧设备已获得兼容。
|
||||
|
||||
规格探测与精确选择自行负责按需有界滚动,原始候选、精确点击和选中复核不变。跳过预滑动不作为规格探测成功或订单创建证据。
|
||||
|
||||
本地 `GoAutoPurchasePanel` 脱敏结构日志关联 task、attempt、device、rule(规则 SHA-256),不上传原始页面。新增事件:`postSwipe=skipped;action=openSpecPanel;reason=spec_panel_on_demand;panel=<枚举>`;其他必需滑动失败为 `postSwipe=failed;action=<动作枚举>;direction=<方向枚举>;swipeIndex=<本动作内第几次滑动>;reason=<固定分类>`。
|
||||
固定失败分类:unknown、root_unavailable、no_scrollable、invalid_bounds、gesture_unsupported、gesture_rejected、gesture_cancelled、gesture_timeout。日志不含规格值、节点文本、坐标、地址、订单、凭据、原始树或截图;结果错误码仍为 RULE_ACTION_FAILED,现有结果提交字段不变。
|
||||
|
||||
## SYB 逐页保存与部分成功(#239)
|
||||
|
||||
实现绑定 c6a962d;代码已实现不代表当前线上已部署。每页完整明细在外部请求结束后按页事务保存;页回滚不累计明细/新增/覆盖数,已提交页保留。日期局部读取失败继续下一日期,全局数据库/进度/会话/取消故障停止。当天漂移不在一次运行内重扫;后续运行重新扫描并幂等补齐。
|
||||
|
||||
同步状态增加 `partial_success`(部分成功,15 字符,复用现有 varchar(16),无需迁移)。有错误且 created+updated>0 为部分成功;有错误无已提交明细为 failed;完整且无错误为 succeeded(包括无符合店铺的数据)。中断仍为 interrupted,不把中断追认为成功。所有终态沿用活动槽释放规则。
|
||||
|
||||
`orderCount` 是已验证页的原始列表读取数量;`detailCount`、`created`、`updated` 为已提交明细及其新增/覆盖数量;`daysProcessed` 是完整通过的日期数,不是已尝试日期数。失败日期/页码/阶段写入现有脱敏限长 errorMessage。部分成功不刷新店铺的完整同步统计。
|
||||
|
||||
Web 唯一展示位置为“采集采购 → SYB 同步记录”:列表状态、状态筛选及详情支持部分成功,详情保留已保存数量、错误原因与重新同步补齐提示。定时任务日志只表示异步任务受理,不等于最终业务同步成功。
|
||||
|
||||
## Agent 采购订单批量回填(#241)
|
||||
|
||||
本节为 #241 服务端实现契约,2026-09-08 按用户授权直接更新本地镜像;线上 Wiki 与其他长期文档由审核阶段同步。本节不表示已经部署或完成真机验收。
|
||||
|
||||
`POST /api/agent/v1/purchase-tasks/order-backfill`
|
||||
|
||||
使用 `Authorization: Bearer <Device Token>`,沿用 `RequireAgentHTTPS`、`GOAUTO_ALLOW_INSECURE_AGENT_HTTP` 与既有可信转发协议策略。无需 Admin JWT、claim、start 或 attempt。设备号只从认证读取,请求不得指定 deviceId、地址全文、收件人、手机号或原始控件树;未知 JSON 字段拒绝。此接口只记录已观察到的订单事实,不执行设备动作、创建订单或付款。
|
||||
|
||||
请求示例(页面时间先按 Asia/Shanghai 理解,再以带时区 RFC3339/RFC3339Nano 发送):
|
||||
|
||||
```json
|
||||
{
|
||||
"requestId": "5826cdda-dcd6-442e-90c3-9b75ba6fb8d8",
|
||||
"items": [
|
||||
{"addressSuffix": "_cg7", "pddOrderNo": "EXAMPLE-ORDER-7", "orderSubmittedAt": "2026-09-08T20:30:00+08:00"},
|
||||
{"addressSuffix": "_cg72", "pddOrderNo": "EXAMPLE-ORDER-72"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- requestId 必须为 UUID;items 为 1~50 条,保持输入顺序;请求体上限沿用 1 MiB。
|
||||
- addressSuffix 只接受 `AddressSuffix(id)` 生成的完整字符串。任务号为非零 uint64,拒绝前导零、正负号、空格、尾随文本、多个后缀与溢出;`_cg7` 与 `_cg72` 分别定位任务 7 和 72。
|
||||
- pddOrderNo 必填,最多 100 个 Unicode 字符,不接受首尾空白、换行或制表符,不自动裁剪后覆盖旧值。
|
||||
- orderSubmittedAt 缺失或 null 时回落任务 irreversible_at;空字符串、无时区文本及非法时间是条目错误,不触发回落。存储统一 UTC。页面值与 irreversible_at 都缺失时该条失败。
|
||||
|
||||
有效批次返回 HTTP 200,包括全部条目失败的批次;每条独立事务,失败不撤销其他条目已提交的数据。响应包裹为 `data`,并设 `Cache-Control: no-store`:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"requestId": "5826cdda-dcd6-442e-90c3-9b75ba6fb8d8",
|
||||
"items": [
|
||||
{"index": 0, "taskId": 7, "result": "backfilled", "code": "BACKFILLED", "status": "order_created", "statusVersion": 5, "pddOrderNo": "EXAMPLE-ORDER-7", "orderSubmittedAt": "2026-09-08T12:30:00Z", "timeSource": "page", "retryable": false},
|
||||
{"index": 1, "taskId": 72, "result": "backfilled", "code": "BACKFILLED", "status": "order_created", "statusVersion": 4, "pddOrderNo": "EXAMPLE-ORDER-72", "orderSubmittedAt": "2026-09-08T12:31:00Z", "timeSource": "irreversible_at", "retryable": false}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
index 从 0 开始;后缀无法解析时不返回 taskId。result 为 `backfilled`、`already_backfilled`、`conflict` 或 `failed`。已认证设备所属任务可返回提交后的状态、版本、已保存订单号及时间;拒绝条目尽可能返回当前已提交事实。跨设备任务和不存在任务不返回这些业务字段,事务回滚后的内存值绝不作为最终事实返回。
|
||||
|
||||
timeSource 说明已保存时间的来源:`page` 为页面值,`irreversible_at` 为估算回落,`existing_unknown` 为原先已创建的历史订单且没有可证明的来源。没有已保存时间时省略 timeSource。客户端必须保留估算标记,不得把回落值或 unknown 宣称为页面真实时间。重复回填不会用新页面时间自动校正旧时间。
|
||||
|
||||
| 条目 code | result | 含义 |
|
||||
|---|---|---|
|
||||
| `BACKFILLED` | backfilled | 本条完成回填 |
|
||||
| `ALREADY_BACKFILLED` | already_backfilled | 正式任务已为 order_created 且订单号相同,无写入 |
|
||||
| `PURCHASE_BACKFILL_SUFFIX_INVALID` | failed | 非法、非规范、零、溢出或歧义后缀 |
|
||||
| `PURCHASE_TASK_NOT_FOUND` | failed | 任务不存在 |
|
||||
| `PURCHASE_BACKFILL_DEVICE_MISMATCH` | failed | 未绑定设备或不属于认证设备 |
|
||||
| `PURCHASE_STATE_CONFLICT` | failed | 非正式采购,或状态不允许回填 |
|
||||
| `PURCHASE_INVALID_REQUEST` | failed | 订单号非法 |
|
||||
| `PURCHASE_ORDER_TIME_INVALID` | failed | 提供的页面时间无效 |
|
||||
| `PURCHASE_ORDER_TIME_MISSING` | failed | 页面时间与 irreversible_at 均无有效值 |
|
||||
| `PURCHASE_BACKFILL_ORDER_CONFLICT` | conflict | 任务已有不同订单号 |
|
||||
| `PURCHASE_BACKFILL_BATCH_CONFLICT` | conflict | 同批同任务出现多个不同订单号,该任务所有条目均拒绝 |
|
||||
| `PURCHASE_BACKFILL_ORDER_ALREADY_USED` | conflict | 同一订单号已对应其他任务 |
|
||||
| `INTERNAL_ERROR` | failed | 数据库失败、死锁等,retryable=true,可安全重放 |
|
||||
|
||||
批级 JSON/UUID/数量错误为 HTTP 422 `PURCHASE_INVALID_REQUEST`;鉴权、停用设备和 HTTPS 限制复用既有错误(401 `DEVICE_TOKEN_INVALID`、403 `DEVICE_DISABLED`、426 `HTTPS_REQUIRED`)。批级失败使用既有 `{code,message,retryable}` 包裹,未开始条目写入。
|
||||
|
||||
### 状态、幂等与并发
|
||||
|
||||
新服务在事务中锁定任务并检查来源状态,仅允许当前设备的 `live + order_result_unknown` 首次写入;`live + order_created` 只在订单号相同时返回已回填。其他状态(包括 running、failed、cancelled 与演练)均拒绝。复用 SetStatus 同步占用字段,同一事务递增 statusVersion、设置 statusChangedAt、清空主任务当前错误及租约;原始 attempt、规则快照、支付与物流、SYB 回写字段不变。
|
||||
|
||||
requestId 沿用 UUID 约定,不增加批次表或全局幂等缓存。既有 unknown_resolve_request_id 槽存储 `backfill:<page|irreversible_at>:<由 requestId 和后缀派生的 UUID>`(最多 61 字符),用于任务级关联和保留本功能时间来源。相同任务和订单号即使更换 requestId 也无写入;同 requestId 改内容仍重新执行设备、状态和订单冲突检查,不凭 requestId 直接放行。批内不同任务提交同一订单号时,先成功提交者占用,其余条目返回订单已被使用;已存在的历史重复订单号不自动修复。
|
||||
|
||||
订单号尚无唯一索引,本实现不迁移数据库。共享模型保存钩子复用 `purchase_rule_setting.id=1` 行作短事务互斥锁,再以锁定读检查订单号归属,覆盖回填、原人工解除和旧结果提交路径;单例缺失时拒绝写入,数据库死锁时回滚失败事务。原 ResolveUnknown 与 Admin 鉴权代码保持不变。禁止通过跳过模型钩子的直接 SQL 写入宣称具备此保证。
|
||||
|
||||
新回填路径禁用包含绑定参数的 SQL 日志,不记录请求正文、订单号、地址或原始树;任务号和认证设备号沿既有任务关联不可变 ruleSnapshot。此接口未新增日志载荷或任务/attempt。
|
||||
|
||||
本地测试覆盖事务回滚、并发服务调用和旧写入路径,使用 SQLite;MySQL 8.4 多连接/多进程的实际行锁、生产数据和真机端到端回填尚待环境验收,不以单元测试替代。
|
||||
|
||||
|
||||
## 蝦皮详情匹配等待与回读(#254)
|
||||
|
||||
实现绑定 `9088e6b`(2026-09-10,分支 fix/254-match-loading);已通过合成数据测试,尚未合并 main 或发布线上。
|
||||
|
||||
本节是 Admin 蝦皮详情接口补充,不修改 Android 请求或采购协议。
|
||||
|
||||
- `GET /api/admin/v1/shopee-products/:productId` 的 `data` 增加 `autoMatchTimeoutSeconds` 整数,为后端详情一键匹配整次预算 `2 × AI timeoutSeconds + 10`,配置范围3~600对应16~1210秒。它是非敏感派生预算,沿用商品详情读取权限;不是开放 AI 管理配置读取,响应不包含 Key、Provider、BaseURL 或 Model。现有客户端路由若复用本 Handler 同样仅获得这个数字。
|
||||
- `POST /api/admin/v1/shopee-products/:productId/specs/mapping/auto-match` 请求字段、业务权限、版本核对与响应映射语义不变。服务端按收到请求时的 AI 配置确定整次父预算,读取配置耗时扣除;Provider 请求仍受自身超时及父 context 剩余时间中更短者约束。
|
||||
- Admin 点击前先GET新详情取得预算,POST timeout=(autoMatchTimeoutSeconds+10)×1000毫秒;缺字段、非整数或超范围时明确拒绝开始匹配,不猜配置。前后端需配套升级。
|
||||
- 准备后配置可能变化,浏览器仍可能先超时;不要据超时响应断言事务是否提交。POST失败/断连/超时只执行有限GET回读,不自动重发POST;失败回读显示结果未知。回读结果表示当前已保存状态,不保证这次请求已经永久不再写入,也不保证全部项目都匹配成功。
|
||||
- 普通GET默认10秒;本轮未修改建议接口610秒、其他AI功能或后台批量匹配600秒预算。T=60时130/140秒、T=180时370/380秒。
|
||||
|
||||
|
||||
## SYB 行内一键关联最近采集(#253)
|
||||
|
||||
实现绑定 `b1594dc`,2026-09-10;分支 feat/253-one-click-link,包含 #254 依赖。已完成合成数据测试及构建,未合并 main、未部署,不代表当前线上已具备此能力。
|
||||
|
||||
本节仅补充 Admin API;Android 和客户端密钥旧请求不变。
|
||||
|
||||
`POST /api/admin/v1/shopee-products/{productId}/link-pdd` 新增与非零 pddProductId 互斥的可选模式:
|
||||
|
||||
```json
|
||||
{"requestId":"UUID","latestCollection":{"sybProductId":1,"deviceId":2,"specContextVersion":"详情返回的未关联版本"}}
|
||||
```
|
||||
|
||||
- 校验当前 SYB→Shopee、未关联、规格版本和设备存在/未停用;离线可用。短事务选择最新已结束成功或部分成功的临时采集,排序 finished_at DESC、id DESC;部分成功不跳过回退旧记录。只接受 completed、存在 finishedAt/PDD、PDD active 且存在可选颜色或尺码。
|
||||
- 成功返回原 `product`,及 `collectionSource: {taskId, deviceId, pddProductId, requiresSpecConfirmation}`。product.specContextVersion 是此次事务写入后的冻结版本;requiresSpecConfirmation 对应 failed 且没有人工或AI确认,完整资格仍由现有 batch-preview 派生。不得把后续其他人的商品版本当作本次自动匹配上下文。
|
||||
- 409 `PDD_LINK_CONFLICT` 表示已有关联;409 `LATEST_COLLECTION_UNAVAILABLE` 表示设备/最近采集/候选不可用;409 `SPEC_CONTEXT_VERSION_STALE` 沿用现有语义。非法参数422。重复请求不会覆盖已有关联,成功响应丢失后重发也会冲突,客户端应回读而非自动重试。last_update_request_id 保留本次请求标识;不承诺额外持久化完整重放响应。
|
||||
- `/api/client/v1/.../link-pdd` 带 latestCollection 时返回403 FORBIDDEN,不扩大API Key授权模块;原手动 pddProductId 模式行为不变。
|
||||
- Web 先GET详情及 #254 autoMatchTimeoutSeconds,再提交关联;关联响应中的冻结版本交给现有 auto-match。其匹配仍自动保存,无额外“保存修改”写请求。普通请求/回读每次10秒,auto-match 单独使用派生前端预算2T+20秒;业务拒绝直接提示,网络/匹配失败至多一次详情回读,再刷新准备状态,禁止自动POST重试。
|
||||
- Server/Web 需配套更新并包含 #254;无需迁移/Android升级。现有人工关联、匹配阈值、规格/采购资格、任务与支付边界不变。
|
||||
|
||||
## Admin 一键替换最近采集候选(#258)
|
||||
|
||||
实现绑定 `2b01974`;仅补充 Admin,Android/API Key 不变。沿用 `POST /api/admin/v1/shopee-products/{productId}/link-pdd` 和原权限,不新增路由或迁移。
|
||||
|
||||
先请求只读预览(仍需 UUID requestId):
|
||||
|
||||
```json
|
||||
{"requestId":"UUID","latestCollection":{"sybProductId":1,"deviceId":2,"specContextVersion":"当前详情版本","replacement":{"preview":true,"expectedPddProductId":3}}}
|
||||
```
|
||||
|
||||
响应 `product` 保持原关联,`collectionSource` 返回 taskId、deviceId、pddProductId、requiresSpecConfirmation,并增加 targetVersion、title、goodsId、previousTitle、previousGoodsId、deviceName。预览没有关联写入。
|
||||
|
||||
确认提交沿用同一 SYB/设备/详情版本与原商品ID,replacement 改为:
|
||||
|
||||
```json
|
||||
{"preview":false,"expectedPddProductId":3,"taskId":4,"targetPddProductId":5,"targetVersion":"预览返回值"}
|
||||
```
|
||||
|
||||
- 提交按指定 taskId 核对临时采集来源、设备、completed/finishedAt、目标ID、可用规格及 targetVersion;不重新选择最新任务。旧关联、旧/目标规格版本变更返回现有409上下文冲突;阶段不再允许返回409 PDD_LINK_CONFLICT;来源不可用409 LATEST_COLLECTION_UNAVAILABLE;缺字段422。
|
||||
- 不同商品条件更新旧关联,返回更新后的 product.specContextVersion;同商品不更新关联及 last_update_request_id,但仍返回用于后续匹配的当前版本。请求不是持久化重放协议;客户端超时只回读、不自动重发。
|
||||
- 预览与提交都校验当前允许阶段;采集中拒绝。复用现有人工更换关联的映射语义,后续 auto-match 自动保存及预算沿用 #254。部分匹配/AI故障不回滚已提交关联。
|
||||
- 不提供 replacement 时 #253 行为不变;`/api/client/v1` 携带 latestCollection(包括 replacement)仍403,不扩大客户端密钥授权。
|
||||
|
||||
## 辅助信息缺失的采集关联(#259)
|
||||
|
||||
实现绑定 `e467706`,仅 Server;部署/验收见 #259。替代 #253/#258 中“部分完成一律拒绝”的限制,不改变 completed 原有路径。
|
||||
|
||||
- completed_partial 的 missing 必须是非空合法列表且仅包含 salesText、shopName、reviewCount,才进入辅助信息兼容检查;title、规格/价格缺失、遍历截断、SKU数量上限及未知原因仍拒绝。
|
||||
- `shopeeproduct/partial_collection.go` 核对本次 task 的维度和值、可售SKU的完整标记/规格引用/非负价格、本次颜色价格,以及当前档案可选值由本次有效SKU覆盖,不能借旧档案合并残留的规格或价格通过。真实单维度允许,不虚构另一维度;本次明确报告缺少规格仍拒绝。
|
||||
- 一键关联、替换预览及确认提交共用验证。失败沿用409 LATEST_COLLECTION_UNAVAILABLE及现有提示,不回退更早采集、不换手机。状态 completed_partial 和 missing 原样保留;辅助字段缺失仍可在任务详情查看。
|
||||
- 无接口字段、数据库迁移、Android/Web或权限变更;普通人工关联与采购/AI资格不变。不将“允许关联”视为“已匹配或已采购”。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: SYB-ERP-Interface-Contract
|
||||
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/SYB-ERP-Interface-Contract.-
|
||||
wiki_revision: ad9adc22e69e48c9a8fd87d7121066eecca55fd6
|
||||
synchronized_at: 2026-09-04T11:30:50Z
|
||||
wiki_revision: 214485db630141aeb03d4b49b22a65e572fb592a
|
||||
synchronized_at: 2026-09-08T02:24:10Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 12 顺云宝(SYB)ERP 接口契约
|
||||
@@ -220,17 +220,19 @@ Admin 默认 `max_matches = 10000`,可以在配置中调整;上限针对整
|
||||
有限重试:单次 HTTP 总超时 60 秒,最多执行 3 次,重试前分别等待 1 秒、2 秒,等待须
|
||||
响应 context 取消。HTTP 401/403、明确会话失效、业务失败、接口数据完整性错误和本地
|
||||
校验失败不得重试;登录、验证码、单件码写入和任何回填请求也不得使用该机制。重试
|
||||
耗尽后返回最后一次错误,外层继续保留日期、页码和已获取数量上下文并将同步标为失败。
|
||||
耗尽后返回最后一次错误,外层保留日期、页码和已获取数量上下文,按最终已保存成果将同步标为失败或部分成功。
|
||||
|
||||
今天的货运单会在同步期间持续新增。只有 UTC+8 下的今天发生上述快照漂移时,
|
||||
允许只重试今天的列表分页,最多 3 次;已经完成的历史日期不得重复拉取,每次尝试
|
||||
也必须使用独立 ID 集合。第三次仍不稳定时,可以对最后一次取得的合法唯一 ID
|
||||
读取完整明细并按既有 upsert 保存,但本次同步仍记为失败、明确提示当天未形成
|
||||
稳定快照且不推进游标,下一次继续覆盖今天。任何尝试都不得突破 `max_matches`;
|
||||
网络/业务错误、非法 ID 或不完整明细不属于可放宽的快照漂移。
|
||||
自 #239(实现提交 c6a962d,已于 2026-09-08 随 d403f3b 部署线上)起,改为按页验证和保存:每页先验证条数、合法 ID、页内/跨页重复,再按冻结店铺快照获取并验证全部明细;外部请求完成后才开启页事务。页内任一入库失败回滚整页,已提交的前页保留,计数在事务提交后累计。
|
||||
|
||||
当天和历史日期统一处理:分页/明细失败或最终总数漂移时停止该日期并继续后续日期,不在同次运行中重新扫描当天,以避免重复写入和计数。数据库故障、进度持久化失败、明确会话失效、上下文取消或总任务超时停止整个范围。下一次人工或定时同步重新扫描日期,按 (order_code, detail_id) 幂等覆盖并保留人工确认。跨页重复不去重后冒充完整数据;最终日期完整性校验仍覆盖所有店铺。此流程替代此前 #235 的当天三次整日快照重扫。
|
||||
|
||||
重复诊断只记录日期、首次/当前页码和行号、start、pageSize、expectedTotal、已获取唯一数量,不记录真实重复 ID、原始响应或个人数据。
|
||||
|
||||
### 4.4 统一日期范围同步与覆盖游标
|
||||
|
||||
GoAuto 同步记录页的立即同步允许管理员和采购员(purchaser)使用(#236),固定覆盖昨天和今天;其他已登录角色仅可查看记录。复用既有同步互斥、日期校验、启用店铺筛选及操作人审计,不授予店铺配置、凭据或定时任务管理权限。权限代码发布并完成启动对账后生效。
|
||||
|
||||
|
||||
页面只有一个同步入口,操作员确认工具条上的开始日和结束日后发起。首次打开页面
|
||||
固定默认昨天到今天,不因 `last_synced_at` 更早而自动扩大范围;需要补历史缺口时
|
||||
由操作员明确选择日期,单次仍不得超过 31 天。
|
||||
@@ -247,7 +249,7 @@ Admin 默认 `max_matches = 10000`,可以在配置中调整;上限针对整
|
||||
或超过 `max_matches` 时不推进游标。
|
||||
|
||||
`[必须]` 每批明细响应必须与请求的货运单 ID 一一对应。缺失、重复、出现未请求
|
||||
ID,或某张货运单返回空商品明细,都视为不完整并停止同步;已经写入的幂等数据
|
||||
ID,或某张货运单返回空商品明细,都视为不完整并停止当前日期,继续后续日期;已经写入的幂等数据
|
||||
可以保留,但只有所有日期全部成功才推进游标。
|
||||
|
||||
`[必须]` 登录和验证码只是同步前置步骤。日期范围经过自动 OCR 降级、手工
|
||||
@@ -489,9 +491,9 @@ POST /am/stock/detail/updateDetailCode?t=0&id={stockID}&detailId={detailID}&code
|
||||
> 以下「店铺准入」已由 GoAuto #49 采纳并实现;店铺规范化还包括全角/半角统一和忽略大小写。
|
||||
|
||||
`[必须]` **同步先校验原始全量,再做店铺准入。** 顺序固定为:按日期查询原始总数
|
||||
并执行单次容量熔断 → 拉完当天原始列表并核对分页前后总数、页长和唯一 ID → 按
|
||||
并执行单次容量熔断 → 逐页核对页长和唯一 ID,整日结束复核总数 → 按
|
||||
`shopName` 去除首尾空白后与启用店铺精确匹配 → 只为接受的货运单请求明细和入库。
|
||||
不能先过滤再做完整性校验,否则非目标店铺的分页漂移会被掩盖。
|
||||
店铺过滤只决定明细获取和入库,不能减少原始列表完整性校验范围;已保存不代表整日完整。
|
||||
|
||||
`[必须]` 同步开始时只读取一次启用店铺,整次运行使用同一个快照。列表允许但明细
|
||||
响应中的 `shopName` 变为空或非允许店铺时再次拦截。没有启用店铺时在会话/OCR/
|
||||
@@ -646,3 +648,13 @@ settings:
|
||||
- 客户端实现:`server/app/goauto/sybclient/`
|
||||
- 落库与解析:`server/app/goauto/sybimport/`
|
||||
- 相关工单:[#48 客户端移植](https://git.ilapage.cn/OPC/goauto/issues/48)、[#41 SYB 货运单商品导入](https://git.ilapage.cn/OPC/goauto/issues/41)、[#37 物流调度与货运宝回填](https://git.ilapage.cn/OPC/goauto/issues/37)
|
||||
|
||||
## SYB 逐页保存与部分成功(#239)
|
||||
|
||||
实现绑定 c6a962d;2026-09-08 已随 d403f3b 部署线上,未手动触发真实同步验收;#240 上游尾页超时尚未修复。每页完整明细在外部请求结束后按页事务保存;页回滚不累计明细/新增/覆盖数,已提交页保留。日期局部读取失败继续下一日期,全局数据库/进度/会话/取消故障停止。当天漂移不在一次运行内重扫;后续运行重新扫描并幂等补齐。
|
||||
|
||||
同步状态增加 `partial_success`(部分成功,15 字符,复用现有 varchar(16),无需迁移)。有错误且 created+updated>0 为部分成功;有错误无已提交明细为 failed;完整且无错误为 succeeded(包括无符合店铺的数据)。中断仍为 interrupted,不把中断追认为成功。所有终态沿用活动槽释放规则。
|
||||
|
||||
`orderCount` 是已验证页的原始列表读取数量;`detailCount`、`created`、`updated` 为已提交明细及其新增/覆盖数量;`daysProcessed` 是完整通过的日期数,不是已尝试日期数。失败日期/页码/阶段写入现有脱敏限长 errorMessage。部分成功不刷新店铺的完整同步统计。
|
||||
|
||||
Web 唯一展示位置为“采集采购 → SYB 同步记录”:列表状态、状态筛选及详情支持部分成功,详情保留已保存数量、错误原因与重新同步补齐提示。定时任务日志只表示异步任务受理,不等于最终业务同步成功。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Deployment-and-Operations
|
||||
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Deployment-and-Operations.-
|
||||
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
|
||||
synchronized_at: 2026-09-04T11:30:08Z
|
||||
wiki_revision: 933eaf6cc02fed843a6898269fdb5f6848ac2a8a
|
||||
synchronized_at: 2026-09-10T01:42:05Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 部署与运维
|
||||
@@ -41,7 +41,7 @@ GOAUTO_ALLOW_INSECURE_AGENT_HTTP=true
|
||||
|
||||
### API 长请求超时约束
|
||||
|
||||
Admin 蝦皮规格 AI 匹配会同步等待外部 Provider:Provider 配置允许 3~600 秒,复合的一键匹配共享 600 秒总预算,Admin 客户端等待 610 秒。因此服务端 `settings.application.writetimeout` 必须至少为 620 秒,受版本管理的标准配置固定为 `620`;启动时低于该值必须明确失败,禁止以 2 秒等短写超时运行,否则连接会在领域错误返回前被截断并表现为代理 502。反向代理的读/发送超时也必须大于服务端窗口。
|
||||
Admin 蝦皮规格 AI 匹配会同步等待外部 Provider:Provider 配置允许 3~600 秒,旧版详情复合一键匹配共享600秒总预算,旧Admin客户端等待610秒;#254未部署前仍适用,部署后详情入口改用下节动态预算。因此服务端 `settings.application.writetimeout` 必须至少为 620 秒,受版本管理的标准配置固定为 `620`;启动时低于该值必须明确失败,禁止以 2 秒等短写超时运行,否则连接会在领域错误返回前被截断并表现为代理 502。反向代理的读/发送超时也必须大于服务端窗口。
|
||||
|
||||
Provider 故障日志只允许记录调用关联 ID、操作类型、耗时、上游 HTTP 状态码或网络错误分类;不得记录 API Key、Authorization、Provider URL、模型输入、候选规格、商品原始内容或响应正文。Provider 失败时接口返回 HTTP 503 与 `AI_MATCHING_UNAVAILABLE`,供 Admin 显示安全中文提示。
|
||||
|
||||
@@ -70,3 +70,37 @@ Provider 故障日志只允许记录调用关联 ID、操作类型、耗时、
|
||||
- 服务启动会将上次进程遗留的 `running` 执行记录标记为 `interrupted`。该恢复依赖当前线上每个数据库只运行一个调度器实例;扩展为多调度器前必须另建工单引入实例租约,不能直接复用此判断。
|
||||
- 排错从 Admin 定时任务页单选任务后进入“日志”,按状态和开始时间查询。记录只含稳定错误码和脱敏摘要;需要定位细节时查看受控服务日志,不得把任务参数、Provider 配置/响应、密钥或业务原始数据复制进执行历史。
|
||||
- 当前没有执行历史删除接口和自动保留策略;删除定时任务不删除历史。数据库容量治理需要另建工单评估。#198 的 `GoAutoSYBSpecAIParse` 在 #199 发布和迁移后仍保持关闭,启用必须由管理员另行确认。
|
||||
|
||||
## 客户端密钥部署与验证(#237)
|
||||
|
||||
实现基线 `71f7751`;2026-09-07 已按用户授权完成本机 MySQL 迁移、管理员菜单写入和 Server/Web 构建重启;管理页面 HTTP 列表与模块加载已验证。2026-09-08 经用户授权,线上已发布 Server/Web 基线 `d403f3b`,执行迁移 `1788798000000` 并重启 GoAuto/Nginx;迁移记录、两张密钥表、仅管理员菜单关联已回读。HTTP 页面与健康检查通过;未登录管理请求 HTTP 200、业务码 401。真实密钥创建及客户端业务读写闭环尚未验证。
|
||||
|
||||
- 发布前须单独授权追加迁移 `server/cmd/migrate/migration/version-local/1788798000000_client_api_key.go`,按既有迁移流程创建 client_api_key、client_api_key_audit 和“采采管理/客户端密钥”管理员菜单。前置父菜单必须存在;不应以赋予普通用户管理员角色代替迁移或权限核验。
|
||||
- 用户于 2026-09-07 明确确认默认兼容 HTTP/HTTPS、不设开关并接受明文风险;部署本版本并执行迁移后,现有 `http://185.216.248.75:9527` 可以使用客户端密钥管理及客户端 API。2026-09-08 线上部署及未登录拒绝检查已通过,真实密钥读写验收仍待进行。HTTP 会明文传输密钥和业务数据,仍建议使用 HTTPS。
|
||||
- 客户端密钥功能不依赖 GOAUTO_TRUST_FORWARDED_PROTO、X-Forwarded-Proto 或 Agent HTTP 例外。既有其他路由的协议和代理配置保持不变;不伪造协议头,不新增明文放行开关。
|
||||
- 代理、APM、应用日志均不得记录 Authorization、创建响应 secret 或原始业务载荷。应用对两类客户端密钥路由跳过旧请求/响应正文日志,使用专用元数据审计;实际代理日志脱敏仍须部署验收。
|
||||
- 请求审计 status=0 可能表示在途、进程中断或结果审计更新失败;先按请求关联号核对业务结果,不自动重试采购、采集、同步等操作。停用阻止后续认证,不保证取消已开始的业务操作。
|
||||
- 隔离验证:Server `go test ./app/goauto/clientkey ./app/goauto/clientapi ./cmd/migrate/migration/version-local`;Web `pnpm exec jest tests/unit/client-keys.spec.js --runInBand` 与 `pnpm run build:prod`。浏览器模拟入口 `/tests/fixtures/client-keys.html` 仅由本地 Vite 开发服务承载,使用内存模拟请求和无效示例密钥,不连接真实数据库,不证明线上鉴权已验收。
|
||||
- 真实部署验收须另行验证实际 HTTP/HTTPS 入口、管理员创建/编辑/停用、普通用户拒绝、读写/独立动作隔离、停用后的后续请求拒绝及日志无密钥;任何真实业务执行继续按独立授权范围进行。
|
||||
|
||||
|
||||
## Windows 本机运行目录与 #237 迁移验证(2026-09-07)
|
||||
|
||||
- Supervisor 实际配置 `D:/supervisor/programs/goauto.conf`;程序仅 `goauto-admin-api` 和 `goauto-admin-ui`。从已有干净工作区 `D:/OPC/goauto-worktrees/main-runtime` 的 main 分支运行;源码运行基线 `ac3c63e`,版本化启动模板更新提交 `9a10d82`。
|
||||
- 原目录 `D:/OPC/goauto` 的用户改动保持原样;本机敏感配置继续读取 `D:/OPC/goauto/config.yaml`,不复制凭据到运行工作区或版本库。数据库 `127.0.0.1:3307/goauto`,API `http://127.0.0.1:8010`,Web `http://127.0.0.1:9527`。
|
||||
- API 命令:`pwsh.exe -NoLogo -NoProfile -File "D:/OPC/goauto-worktrees/main-runtime/scripts/start-server.ps1" -ConfigPath "D:/OPC/goauto/config.yaml" -SkipMigration`;Web 同目录 `scripts/start-web.ps1` 与相同 ConfigPath。已验证无需 ExecutionPolicy Bypass。
|
||||
- 真实迁移必须单独授权并先确认唯一待执行版本;常驻 API 加 `-SkipMigration`,日常重启不自动执行未来待审核迁移。#237 迁移 `1788798000000_client_api_key.go` 已执行,sys_migration 记录 `1788798000000`、两张密钥表及管理员菜单已回读;其他角色未新增菜单关联。任务启停状态未改。
|
||||
- 范围明确的服务控制:`D:/supervisor/supervisord.exe -c D:/supervisor/supervisord.conf ctl status goauto-admin-api goauto-admin-ui`,启动/停止/重启将 status 分别替换为 start/stop/restart。不得为了 GoAuto 重启整个 Supervisor 或其他项目。配置内容变化后需重新读取配置;本次调用 `supervisor.reloadConfig`,再对上述两个程序定向 start,已通过进程命令行核对新目录。
|
||||
- 日志:`D:/supervisor/logs/goauto-admin-api.log`、`D:/supervisor/logs/goauto-admin-ui.log`;对外分享只能保留脱敏结构摘要。迁移输出不得暴露凭据或业务原文。
|
||||
- 管理员刷新页面后可进入 `http://127.0.0.1:9527/#/client-keys/index`;菜单缓存未更新时重新登录。已验证列表空态、创建弹窗加载 12 模块及 HTTP 风险提示;未实际创建密钥,真实客户端读写及停用闭环仍待范围明确的验证。本节为 2026-09-07 本机验证记录;线上已于 2026-09-08 另行授权发布,见上节。
|
||||
|
||||
|
||||
## 蝦皮详情匹配等待与回读(#254)
|
||||
|
||||
实现绑定 `9088e6b`(2026-09-10,分支 fix/254-match-loading);已通过合成数据测试,尚未合并 main 或发布线上。
|
||||
|
||||
- 本次只完成代码构建与合成测试,未改线上配置、AI设置、常驻服务或Nginx;部署与重启需另行授权。
|
||||
- 详情匹配新窗口是后端 2T+10 秒、Web POST 2T+20 秒。当前默认API最低写超时校验仍为620秒,这是历史最低值,不保证容纳所有动态预算;发布前读取实际 AI timeoutSeconds、application.writetimeout 和生效代理配置核对。
|
||||
- 建议API写超时至少为 2T+30 秒,且仍满足原最低620秒;代理读/发送超时需留足对应窗口。T=60/180时原620秒足够容纳应用预算;T=600时至少需要1230秒API写超时及适当更长的代理窗口。不得仅设置较大的AI超时却忽略传输层更早截断。
|
||||
- 上述是发布核对要求,不是本单已修改/验证线上代理的结论。超大T带来长连接占用,不能用关闭超时来代替正确预算。新增失败回读不代表必须延长任何线上请求。
|
||||
- 必须配套发布 Server/Web;仅上新版Web连接旧Server时,因缺失派生预算会提示等待时间不可用并停止匹配。原Android不需升级。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[program:goauto-admin-api]
|
||||
command=pwsh.exe -NoLogo -NoProfile -ExecutionPolicy Bypass -File "D:/OPC/goauto/scripts/start-server.ps1" -ConfigPath "D:/OPC/goauto/config.yaml"
|
||||
directory=D:/OPC/goauto
|
||||
command=pwsh.exe -NoLogo -NoProfile -File "D:/OPC/goauto-worktrees/main-runtime/scripts/start-server.ps1" -ConfigPath "D:/OPC/goauto/config.yaml" -SkipMigration
|
||||
directory=D:/OPC/goauto-worktrees/main-runtime
|
||||
autostart=true
|
||||
autorestart=true
|
||||
startsecs=3
|
||||
@@ -14,8 +14,8 @@ stdout_logfile_maxbytes=50MB
|
||||
stdout_logfile_backups=5
|
||||
|
||||
[program:goauto-admin-ui]
|
||||
command=pwsh.exe -NoLogo -NoProfile -ExecutionPolicy Bypass -File "D:/OPC/goauto/scripts/start-web.ps1" -ConfigPath "D:/OPC/goauto/config.yaml"
|
||||
directory=D:/OPC/goauto
|
||||
command=pwsh.exe -NoLogo -NoProfile -File "D:/OPC/goauto-worktrees/main-runtime/scripts/start-web.ps1" -ConfigPath "D:/OPC/goauto/config.yaml"
|
||||
directory=D:/OPC/goauto-worktrees/main-runtime
|
||||
autostart=true
|
||||
autorestart=true
|
||||
startsecs=3
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
goautoclientapi "go-admin/app/goauto/clientapi"
|
||||
"os"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
@@ -53,6 +54,7 @@ func InitRouter() {
|
||||
|
||||
// 注册 GoAuto Agent 与设备管理路由。
|
||||
goautodevice.InitRouter(r, authMiddleware)
|
||||
goautoclientapi.InitRouter(r, authMiddleware)
|
||||
goautoapprelease.InitRouter(r, authMiddleware)
|
||||
goautoaimatching.InitRouter(r, authMiddleware)
|
||||
goautotask.InitRouter(r, authMiddleware)
|
||||
@@ -61,7 +63,7 @@ func InitRouter() {
|
||||
goautoproduct.InitRouter(r, authMiddleware)
|
||||
goautoreplacement.InitRouter(r, authMiddleware)
|
||||
goautorule.InitRouter(r, authMiddleware)
|
||||
goautoshopeeproduct.InitRouter(r, authMiddleware)
|
||||
goautoshopeeproduct.InitRouter(r, authMiddleware, goautopurchase.ValidateQuickReplacement)
|
||||
goautosybimport.InitRouter(r, authMiddleware)
|
||||
goautosybinnercode.InitRouter(r, authMiddleware)
|
||||
goautosybshop.InitRouter(r, authMiddleware)
|
||||
|
||||
@@ -136,6 +136,8 @@ func moduleKeyForAPI(path string) string {
|
||||
return ModulePDDProducts
|
||||
case strings.HasPrefix(path, "/api/admin/v1/shopee-products"):
|
||||
return ModuleShopeeProducts
|
||||
case strings.HasPrefix(path, "/api/admin/v1/shopee-spec-auto-match"):
|
||||
return ModuleShopeeProducts
|
||||
case strings.HasPrefix(path, "/api/admin/v1/syb-products/sync-runs"):
|
||||
return ModuleSYBSyncRuns
|
||||
case strings.HasPrefix(path, "/api/admin/v1/syb-products"):
|
||||
|
||||
@@ -48,13 +48,15 @@ var AdminAPIs = []APIPermission{
|
||||
{"AI 建议颜色映射", "/api/admin/v1/shopee-products/:productId/specs/mapping/suggest-colors", "POST", true},
|
||||
{"AI 建议尺码映射", "/api/admin/v1/shopee-products/:productId/specs/mapping/suggest-sizes", "POST", true},
|
||||
{"一键匹配并确认颜色尺码", "/api/admin/v1/shopee-products/:productId/specs/mapping/auto-match", "POST", true},
|
||||
{"手动执行虾皮规格自动匹配", "/api/admin/v1/shopee-spec-auto-match/runs", "POST", false},
|
||||
{"查看最近虾皮规格自动匹配", "/api/admin/v1/shopee-spec-auto-match/runs/latest", "GET", false},
|
||||
|
||||
{"查看 SYB 商品", "/api/admin/v1/syb-products", "GET", true},
|
||||
{"查看 SYB 商品详情", "/api/admin/v1/syb-products/:productId", "GET", true},
|
||||
{"重新解析 SYB 商品", "/api/admin/v1/syb-products/:productId/reparse", "POST", true},
|
||||
{"批量重新解析 SYB 商品", "/api/admin/v1/syb-products/reparse-batch", "POST", true},
|
||||
{"人工修正 SYB 商品", "/api/admin/v1/syb-products/:productId/correction", "PATCH", true},
|
||||
{"手动同步 SYB 商品", "/api/admin/v1/syb-products/import", "POST", false},
|
||||
{"手动同步 SYB 商品", "/api/admin/v1/syb-products/import", "POST", true},
|
||||
{"查看 SYB 同步记录", "/api/admin/v1/syb-products/sync-runs", "GET", true},
|
||||
{"查看 SYB 同步详情", "/api/admin/v1/syb-products/sync-runs/:runId", "GET", true},
|
||||
|
||||
|
||||
@@ -15,10 +15,12 @@ func TestPurchaserPermissionMatrixHasNoDuplicates(t *testing.T) {
|
||||
|
||||
func TestPurchaserExcludesAdministratorOperations(t *testing.T) {
|
||||
denied := map[string]bool{
|
||||
"POST /api/admin/v1/devices/:deviceId/disable": true,
|
||||
"POST /api/admin/v1/syb-products/import": true,
|
||||
"POST /api/admin/v1/collection-rules": true,
|
||||
"PUT /api/admin/v1/ai-matching-settings": true,
|
||||
"POST /api/admin/v1/devices/:deviceId/disable": true,
|
||||
"POST /api/admin/v1/syb-shops/discover": true,
|
||||
"POST /api/admin/v1/collection-rules": true,
|
||||
"PUT /api/admin/v1/ai-matching-settings": true,
|
||||
"POST /api/admin/v1/shopee-spec-auto-match/runs": true,
|
||||
"GET /api/admin/v1/shopee-spec-auto-match/runs/latest": true,
|
||||
}
|
||||
for _, permission := range PurchaserAPIs() {
|
||||
if denied[permission.Method+" "+permission.Path] {
|
||||
|
||||
@@ -55,7 +55,9 @@ func TestReconcilePurchaserPermissions(t *testing.T) {
|
||||
}
|
||||
|
||||
assertPolicyCount(t, db, "custom-role", "/custom", "GET", 1)
|
||||
assertPolicyCount(t, db, RolePurchaser, "/api/admin/v1/syb-products/import", "POST", 0)
|
||||
assertPolicyCount(t, db, RolePurchaser, "/api/admin/v1/syb-products/import", "POST", 1)
|
||||
assertPolicyCount(t, db, RolePurchaser, "/api/admin/v1/syb-shops/discover", "POST", 0)
|
||||
assertPolicyCount(t, db, RolePurchaser, "/api/admin/v1/syb-shops", "POST", 0)
|
||||
}
|
||||
|
||||
func TestReconcilePurchaserPermissionsRemovesOnlyStalePurchaserGrant(t *testing.T) {
|
||||
|
||||
@@ -25,12 +25,28 @@ const (
|
||||
defaultAutoConfirmMinConfidence = 0.9
|
||||
)
|
||||
|
||||
// MaxProviderTimeout is also the total budget used by composite synchronous
|
||||
// AI operations. This keeps their HTTP response inside the Admin and API
|
||||
// transport windows even when an operation needs more than one provider call.
|
||||
const MaxProviderTimeout = 600 * time.Second
|
||||
|
||||
// Service owns the internal AI Provider configuration. The API key exception
|
||||
// is deliberately narrow: it is plain text only in the dedicated settings
|
||||
// table and is returned only by the administrator settings handler.
|
||||
type Service struct {
|
||||
DB *gorm.DB
|
||||
HTTPClient *http.Client
|
||||
DB *gorm.DB
|
||||
HTTPClient *http.Client
|
||||
ProviderFailureLogger func(ProviderFailureDiagnostic)
|
||||
}
|
||||
|
||||
// ProviderFailureDiagnostic deliberately contains no URL, model, prompt,
|
||||
// candidates, response body or credential. It is safe for operational logs.
|
||||
type ProviderFailureDiagnostic struct {
|
||||
CallID string
|
||||
Operation string
|
||||
Kind string
|
||||
StatusCode int
|
||||
Duration time.Duration
|
||||
}
|
||||
|
||||
func NewService(db *gorm.DB) *Service {
|
||||
@@ -179,6 +195,58 @@ func (s *Service) Resolve(ctx context.Context, request MatchRequest) (MatchResul
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// ResolveSYBSpec parses one SYB productSpec into the linked Shopee product's
|
||||
// exact color/size labels. Unlike Resolve, this operation does not map to PDD:
|
||||
// every non-empty answer must be an exact member of the supplied Shopee set.
|
||||
func (s *Service) ResolveSYBSpec(ctx context.Context, request SYBSpecParseRequest) (SYBSpecParseResult, error) {
|
||||
request.ProductSpec = strings.TrimSpace(request.ProductSpec)
|
||||
request.Colors = usableCandidates(request.Colors)
|
||||
request.Sizes = usableCandidates(request.Sizes)
|
||||
if request.ProductSpec == "" || (len(request.Colors) == 0 && len(request.Sizes) == 0) {
|
||||
return SYBSpecParseResult{}, fail(CodeNoMatch, "SYB 采购规格缺少可判断的原文或蝦皮候选")
|
||||
}
|
||||
setting, apiKey, err := s.activeSetting(ctx)
|
||||
if err != nil {
|
||||
return SYBSpecParseResult{}, err
|
||||
}
|
||||
payload := openAIChatRequest{Model: setting.Model, Temperature: 0, Messages: []openAIMessage{
|
||||
{Role: "system", Content: "你只负责把一条 SYB 商品规格原文解析成给定蝦皮候选中的原始颜色和尺码。不得猜测、不得改写候选、不得返回候选外文本。只返回 JSON:{\"color\":\"颜色候选原文或空\",\"size\":\"尺码候选原文或空\",\"reason\":\"简短原因\",\"confidence\":0到1}。提供了某角色候选时必须唯一可靠地选择一个,否则对应字段留空。"},
|
||||
{Role: "user", Content: sybSpecParsePrompt(request)},
|
||||
}}
|
||||
body, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return SYBSpecParseResult{}, &Error{Code: CodeProviderUnavailable, Message: "SYB 规格 AI 解析请求生成失败", Cause: err}
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(ctx, time.Duration(setting.TimeoutSeconds)*time.Second)
|
||||
defer cancel()
|
||||
httpRequest, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint(setting.BaseURL, "chat/completions"), bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return SYBSpecParseResult{}, fail(CodeInvalidSetting, "AI 服务地址无效")
|
||||
}
|
||||
httpRequest.Header.Set("Authorization", "Bearer "+apiKey)
|
||||
httpRequest.Header.Set("Content-Type", "application/json")
|
||||
response, err := s.httpClient().Do(httpRequest)
|
||||
if err != nil {
|
||||
return SYBSpecParseResult{}, &Error{Code: CodeProviderUnavailable, Message: "SYB 规格 AI 解析服务暂时不可用", Cause: err}
|
||||
}
|
||||
defer response.Body.Close()
|
||||
responseBody, readErr := io.ReadAll(io.LimitReader(response.Body, 1<<20))
|
||||
if readErr != nil || response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
|
||||
return SYBSpecParseResult{}, fail(CodeProviderUnavailable, "SYB 规格 AI 解析服务暂时不可用")
|
||||
}
|
||||
choice, err := parseProviderChoice(responseBody)
|
||||
if err != nil || !validClosedChoice(choice.Color, request.Colors) || !validClosedChoice(choice.Size, request.Sizes) {
|
||||
return SYBSpecParseResult{}, fail(CodeNoMatch, "AI 未能在蝦皮候选中唯一解析采购规格")
|
||||
}
|
||||
if choice.Confidence == nil || *choice.Confidence < 0 || *choice.Confidence > 1 || strings.TrimSpace(choice.Reason) == "" {
|
||||
return SYBSpecParseResult{}, fail(CodeNoMatch, "AI 解析结果缺少有效置信度或理由")
|
||||
}
|
||||
return SYBSpecParseResult{
|
||||
Color: choice.Color, Size: choice.Size, Provider: ProviderOpenAICompatible,
|
||||
Model: setting.Model, Reason: safeReason(choice.Reason), Confidence: choice.Confidence,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Service) activeSetting(ctx context.Context) (models.AIMatchingSetting, string, error) {
|
||||
setting, err := s.setting(ctx)
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) || !setting.Enabled {
|
||||
@@ -294,6 +362,22 @@ func validChoice(target, selected string, candidates []string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func validClosedChoice(selected string, candidates []string) bool {
|
||||
selected = strings.TrimSpace(selected)
|
||||
if len(candidates) == 0 {
|
||||
return selected == ""
|
||||
}
|
||||
if selected == "" {
|
||||
return false
|
||||
}
|
||||
for _, candidate := range candidates {
|
||||
if candidate == selected {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func safeReason(reason string) string {
|
||||
reason = strings.TrimSpace(reason)
|
||||
if reason == "" {
|
||||
@@ -316,6 +400,16 @@ func matchPrompt(request MatchRequest) string {
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
func sybSpecParsePrompt(request SYBSpecParseRequest) string {
|
||||
payload := struct {
|
||||
ProductSpec string `json:"productSpec"`
|
||||
Colors []string `json:"shopeeColorCandidates,omitempty"`
|
||||
Sizes []string `json:"shopeeSizeCandidates,omitempty"`
|
||||
}{request.ProductSpec, request.Colors, request.Sizes}
|
||||
raw, _ := json.Marshal(payload)
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
type openAIMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
|
||||
@@ -2,6 +2,7 @@ package aimatching
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
@@ -135,3 +136,61 @@ func TestAutoConfirmThresholdDefaultsPersistsAndValidates(t *testing.T) {
|
||||
t.Fatalf("invalid threshold err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveSYBSpecUsesOnlyProductSpecAndClosedShopeeCandidates(t *testing.T) {
|
||||
service := matcherTestService(t)
|
||||
if _, err := service.SaveSettings(context.Background(), SaveSettingsRequest{Enabled: true, BaseURL: "https://provider.example/v1", Model: "test-model", APIKey: "test-secret", TimeoutSeconds: 8}, 7); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var sent map[string]any
|
||||
service.HTTPClient = &http.Client{Transport: roundTripper(func(request *http.Request) (*http.Response, error) {
|
||||
raw, err := io.ReadAll(request.Body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := json.Unmarshal(raw, &sent); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body := `{"choices":[{"message":{"content":"{\"color\":\"黑色\",\"size\":\"XL\",\"reason\":\"原文对应唯一候选\",\"confidence\":0.95}"}}]}`
|
||||
return &http.Response{StatusCode: http.StatusOK, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(body)), Request: request}, nil
|
||||
})}
|
||||
result, err := service.ResolveSYBSpec(context.Background(), SYBSpecParseRequest{ProductSpec: "黑色 XL【备注】", Colors: []string{"黑色", "白色"}, Sizes: []string{"L", "XL"}})
|
||||
if err != nil || result.Color != "黑色" || result.Size != "XL" || result.Confidence == nil || *result.Confidence != 0.95 {
|
||||
t.Fatalf("result=%+v err=%v", result, err)
|
||||
}
|
||||
encoded, _ := json.Marshal(sent)
|
||||
for _, forbidden := range []string{"orderCode", "address", "rawJson", "price", "test-secret"} {
|
||||
if strings.Contains(string(encoded), forbidden) {
|
||||
t.Fatalf("provider payload leaked forbidden field %q: %s", forbidden, encoded)
|
||||
}
|
||||
}
|
||||
for _, required := range []string{"productSpec", "shopeeColorCandidates", "shopeeSizeCandidates"} {
|
||||
if !strings.Contains(string(encoded), required) {
|
||||
t.Fatalf("provider payload missing %q: %s", required, encoded)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveSYBSpecRejectsCandidateOutsideClosedSetAndMissingConfidence(t *testing.T) {
|
||||
service := matcherTestService(t)
|
||||
if _, err := service.SaveSettings(context.Background(), SaveSettingsRequest{Enabled: true, BaseURL: "https://provider.example/v1", Model: "test-model", APIKey: "test-secret", TimeoutSeconds: 8}, 7); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
responses := []string{
|
||||
`{"choices":[{"message":{"content":"{\"color\":\"灰色\",\"size\":\"XL\",\"reason\":\"猜测\",\"confidence\":0.99}"}}]}`,
|
||||
`{"choices":[{"message":{"content":"{\"color\":\"黑色\",\"size\":\"XL\",\"reason\":\"候选\"}"}}]}`,
|
||||
}
|
||||
service.HTTPClient = &http.Client{Transport: roundTripper(func(request *http.Request) (*http.Response, error) {
|
||||
body := responses[0]
|
||||
responses = responses[1:]
|
||||
return &http.Response{StatusCode: http.StatusOK, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(body)), Request: request}, nil
|
||||
})}
|
||||
request := SYBSpecParseRequest{ProductSpec: "黑 XL", Colors: []string{"黑色"}, Sizes: []string{"XL"}}
|
||||
for i := 0; i < 2; i++ {
|
||||
_, err := service.ResolveSYBSpec(context.Background(), request)
|
||||
var target *Error
|
||||
if !errors.As(err, &target) || target.Code != CodeNoMatch {
|
||||
t.Fatalf("attempt %d err=%v", i, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,6 +10,9 @@ import (
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
log "github.com/go-admin-team/go-admin-core/logger"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// Suggestion limits are enforced defensively here too, even though callers
|
||||
@@ -95,18 +98,26 @@ func (s *Service) SuggestBatch(ctx context.Context, request SuggestRequest) (Sug
|
||||
}
|
||||
httpRequest.Header.Set("Authorization", "Bearer "+apiKey)
|
||||
httpRequest.Header.Set("Content-Type", "application/json")
|
||||
callID, startedAt := uuid.NewString(), time.Now()
|
||||
response, err := s.httpClient().Do(httpRequest)
|
||||
if err != nil {
|
||||
s.logProviderFailure(ProviderFailureDiagnostic{CallID: callID, Operation: "suggest_batch", Kind: providerNetworkErrorKind(err), Duration: time.Since(startedAt)})
|
||||
return SuggestResult{}, &Error{Code: CodeProviderUnavailable, Message: "AI 建议服务暂时不可用", Cause: err}
|
||||
}
|
||||
defer response.Body.Close()
|
||||
limited := io.LimitReader(response.Body, 1<<20)
|
||||
responseBody, readErr := io.ReadAll(limited)
|
||||
if readErr != nil || response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
|
||||
if readErr != nil {
|
||||
s.logProviderFailure(ProviderFailureDiagnostic{CallID: callID, Operation: "suggest_batch", Kind: "read_error", StatusCode: response.StatusCode, Duration: time.Since(startedAt)})
|
||||
return SuggestResult{}, fail(CodeProviderUnavailable, "AI 建议服务暂时不可用")
|
||||
}
|
||||
if response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
|
||||
s.logProviderFailure(ProviderFailureDiagnostic{CallID: callID, Operation: "suggest_batch", Kind: "http_status", StatusCode: response.StatusCode, Duration: time.Since(startedAt)})
|
||||
return SuggestResult{}, fail(CodeProviderUnavailable, "AI 建议服务暂时不可用")
|
||||
}
|
||||
raw, err := parseSuggestChoices(responseBody)
|
||||
if err != nil {
|
||||
s.logProviderFailure(ProviderFailureDiagnostic{CallID: callID, Operation: "suggest_batch", Kind: "invalid_response", StatusCode: response.StatusCode, Duration: time.Since(startedAt)})
|
||||
return SuggestResult{}, fail(CodeProviderUnavailable, "AI 建议响应无效")
|
||||
}
|
||||
|
||||
@@ -145,6 +156,22 @@ func (s *Service) SuggestBatch(ctx context.Context, request SuggestRequest) (Sug
|
||||
return SuggestResult{Decisions: decisions, Provider: ProviderOpenAICompatible, Model: setting.Model}, nil
|
||||
}
|
||||
|
||||
func (s *Service) logProviderFailure(diagnostic ProviderFailureDiagnostic) {
|
||||
if s.ProviderFailureLogger != nil {
|
||||
s.ProviderFailureLogger(diagnostic)
|
||||
return
|
||||
}
|
||||
log.Warnf("AI provider call failed: call_id=%s operation=%s kind=%s status=%d duration_ms=%d",
|
||||
diagnostic.CallID, diagnostic.Operation, diagnostic.Kind, diagnostic.StatusCode, diagnostic.Duration.Milliseconds())
|
||||
}
|
||||
|
||||
func providerNetworkErrorKind(err error) string {
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return "timeout"
|
||||
}
|
||||
return "network_error"
|
||||
}
|
||||
|
||||
func suggestSystemPrompt(dimension string) string {
|
||||
noun := "颜色或尺码"
|
||||
switch dimension {
|
||||
|
||||
@@ -6,7 +6,9 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
|
||||
@@ -134,3 +136,93 @@ func TestSuggestBatchRequiresConfiguredProvider(t *testing.T) {
|
||||
t.Fatalf("expected CodeNotConfigured, got %v", target.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuggestBatchLogsSafeDiagnosticForProvider502(t *testing.T) {
|
||||
const sensitiveBody = "api-key-and-provider-body-must-not-be-logged"
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
http.Error(w, sensitiveBody, http.StatusBadGateway)
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
db := openSuggestTestDB(t)
|
||||
seedEnabledSetting(t, db, server.URL)
|
||||
var diagnostic ProviderFailureDiagnostic
|
||||
service := NewService(db)
|
||||
service.ProviderFailureLogger = func(value ProviderFailureDiagnostic) { diagnostic = value }
|
||||
|
||||
_, err := service.SuggestBatch(context.Background(), SuggestRequest{
|
||||
Sources: []SuggestSource{{ID: "s1", Label: "sensitive-source"}},
|
||||
Candidates: []SuggestCandidate{{ID: "c1", Label: "sensitive-candidate"}},
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("expected provider failure")
|
||||
}
|
||||
if diagnostic.Operation != "suggest_batch" || diagnostic.Kind != "http_status" || diagnostic.StatusCode != http.StatusBadGateway || diagnostic.CallID == "" {
|
||||
t.Fatalf("unexpected diagnostic: %+v", diagnostic)
|
||||
}
|
||||
printed := fmt.Sprintf("%+v", diagnostic)
|
||||
for _, secret := range []string{sensitiveBody, "sensitive-source", "sensitive-candidate", "test-key", server.URL} {
|
||||
if strings.Contains(printed, secret) {
|
||||
t.Fatalf("diagnostic leaked %q: %s", secret, printed)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuggestBatchClassifiesProviderTimeout(t *testing.T) {
|
||||
db := openSuggestTestDB(t)
|
||||
seedEnabledSetting(t, db, "http://provider.invalid")
|
||||
var diagnostic ProviderFailureDiagnostic
|
||||
service := NewService(db)
|
||||
service.HTTPClient = &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||
return nil, context.DeadlineExceeded
|
||||
})}
|
||||
service.ProviderFailureLogger = func(value ProviderFailureDiagnostic) { diagnostic = value }
|
||||
|
||||
_, err := service.SuggestBatch(context.Background(), SuggestRequest{
|
||||
Sources: []SuggestSource{{ID: "s1", Label: "黑色"}}, Candidates: []SuggestCandidate{{ID: "c1", Label: "黑色"}},
|
||||
})
|
||||
if err == nil || diagnostic.Kind != "timeout" || diagnostic.StatusCode != 0 {
|
||||
t.Fatalf("timeout was not safely classified: diagnostic=%+v err=%v", diagnostic, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuggestBatchClassifiesInvalidResponseWithoutLoggingBody(t *testing.T) {
|
||||
const sensitiveBody = "not-json-with-sensitive-provider-details"
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
_, _ = w.Write([]byte(sensitiveBody))
|
||||
}))
|
||||
defer server.Close()
|
||||
db := openSuggestTestDB(t)
|
||||
seedEnabledSetting(t, db, server.URL)
|
||||
var diagnostic ProviderFailureDiagnostic
|
||||
service := NewService(db)
|
||||
service.ProviderFailureLogger = func(value ProviderFailureDiagnostic) { diagnostic = value }
|
||||
|
||||
_, err := service.SuggestBatch(context.Background(), SuggestRequest{
|
||||
Sources: []SuggestSource{{ID: "s1", Label: "黑色"}}, Candidates: []SuggestCandidate{{ID: "c1", Label: "黑色"}},
|
||||
})
|
||||
if err == nil || diagnostic.Kind != "invalid_response" || strings.Contains(fmt.Sprintf("%+v", diagnostic), sensitiveBody) {
|
||||
t.Fatalf("invalid response diagnostic is unsafe or missing: diagnostic=%+v err=%v", diagnostic, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuggestBatchAllowsProviderResponseAfterTwoSeconds(t *testing.T) {
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
time.Sleep(2100 * time.Millisecond)
|
||||
chatCompletionResponder(`{"suggestions":[{"sourceId":"s1","candidateId":"c1","confidence":0.95,"reason":"match"}]}`)(w, r)
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
db := openSuggestTestDB(t)
|
||||
seedEnabledSetting(t, db, server.URL)
|
||||
result, err := NewService(db).SuggestBatch(context.Background(), SuggestRequest{
|
||||
Sources: []SuggestSource{{ID: "s1", Label: "黑色"}}, Candidates: []SuggestCandidate{{ID: "c1", Label: "黑色"}},
|
||||
})
|
||||
if err != nil || result.Decisions["s1"].CandidateID != "c1" {
|
||||
t.Fatalf("delayed provider response failed: result=%+v err=%v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
type roundTripFunc func(*http.Request) (*http.Response, error)
|
||||
|
||||
func (fn roundTripFunc) RoundTrip(request *http.Request) (*http.Response, error) { return fn(request) }
|
||||
|
||||
@@ -29,6 +29,24 @@ type MatchRequest struct {
|
||||
Sizes []string
|
||||
}
|
||||
|
||||
// SYBSpecParseRequest contains the only source text and closed Shopee
|
||||
// candidate sets that may leave GoAuto for an AI-assisted SYB parse. It must
|
||||
// never contain the shipment/order, account, address, price or full raw JSON.
|
||||
type SYBSpecParseRequest struct {
|
||||
ProductSpec string
|
||||
Colors []string
|
||||
Sizes []string
|
||||
}
|
||||
|
||||
type SYBSpecParseResult struct {
|
||||
Color string
|
||||
Size string
|
||||
Provider string
|
||||
Model string
|
||||
Reason string
|
||||
Confidence *float64
|
||||
}
|
||||
|
||||
type CandidateSnapshot struct {
|
||||
Colors []string `json:"colors,omitempty"`
|
||||
Sizes []string `json:"sizes,omitempty"`
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
package clientapi
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/sdk/pkg"
|
||||
"go-admin/app/goauto/clientkey"
|
||||
"go-admin/common/clientprincipal"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Both HTTP and HTTPS are supported by explicit operator decision (#237).
|
||||
// Transport does not grant identity or permissions; secrets remain non-cacheable.
|
||||
func NoStore(c *gin.Context) {
|
||||
c.Header("Cache-Control", "no-store")
|
||||
c.Next()
|
||||
}
|
||||
func Gate(db *gorm.DB, e Endpoint) gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
c.Header("Cache-Control", "no-store")
|
||||
deny := func(status int, message string) {
|
||||
c.AbortWithStatusJSON(status, gin.H{"code": status, "message": message})
|
||||
}
|
||||
// Never accept a credential supplied through a URL or cookie fallback.
|
||||
for k := range c.Request.URL.Query() {
|
||||
if sensitive(k) || strings.EqualFold(k, "token") {
|
||||
deny(400, "密钥只能通过 Authorization 请求头发送")
|
||||
return
|
||||
}
|
||||
}
|
||||
header := strings.Fields(c.GetHeader("Authorization"))
|
||||
if len(header) != 2 || !strings.EqualFold(header[0], "Bearer") {
|
||||
deny(401, "需要客户端密钥")
|
||||
return
|
||||
}
|
||||
connection := db
|
||||
var err error
|
||||
if connection == nil {
|
||||
connection, err = pkg.GetOrm(c)
|
||||
}
|
||||
if err != nil || connection == nil {
|
||||
deny(503, "客户端认证暂不可用")
|
||||
return
|
||||
}
|
||||
key, err := (clientkey.Service{DB: connection}).Authenticate(c.Request.Context(), header[1])
|
||||
if err != nil {
|
||||
if errors.Is(err, clientkey.ErrCredential) {
|
||||
deny(401, "客户端密钥无效或已停用")
|
||||
} else {
|
||||
deny(503, "客户端认证暂不可用")
|
||||
}
|
||||
return
|
||||
}
|
||||
random := make([]byte, 16)
|
||||
if _, err = rand.Read(random); err != nil {
|
||||
deny(503, "审计暂不可用")
|
||||
return
|
||||
}
|
||||
requestID := hex.EncodeToString(random)
|
||||
c.Header("X-Client-Request-Id", requestID)
|
||||
record := clientkey.Audit{KeyID: key.ID, Event: "request", RequestID: requestID, Method: e.Method, Route: "/api/client/v1" + e.Path}
|
||||
if !key.Allows(e.Module, e.Capability) {
|
||||
record.Status = 403
|
||||
_ = connection.Create(&record).Error
|
||||
deny(403, "密钥未授权此模块或执行动作")
|
||||
return
|
||||
}
|
||||
// The intent must be durable before any business handler can run.
|
||||
if err = connection.WithContext(c.Request.Context()).Create(&record).Error; err != nil {
|
||||
deny(503, "审计暂不可用,未执行操作")
|
||||
return
|
||||
}
|
||||
clientprincipal.Set(c, clientprincipal.Identity{KeyID: key.ID, RequestID: requestID, AuthorizedBy: key.UpdatedBy})
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, 16<<20)
|
||||
original := c.Writer
|
||||
buffer := &responseBuffer{ResponseWriter: original, status: 200}
|
||||
c.Writer = buffer
|
||||
defer func() { c.Writer = original }()
|
||||
c.Next()
|
||||
c.Writer = original
|
||||
status := buffer.status
|
||||
var value any
|
||||
if buffer.overflow || json.Unmarshal(buffer.body.Bytes(), &value) != nil {
|
||||
status = 502
|
||||
value = gin.H{"code": 502, "message": "无法生成客户端响应,请先核对操作结果,不要自动重试"}
|
||||
} else {
|
||||
value = redact(value)
|
||||
}
|
||||
auditCtx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
_ = connection.WithContext(auditCtx).Model(&clientkey.Audit{}).Where("id = ?", record.ID).Update("status", status).Error
|
||||
now := time.Now().UTC()
|
||||
_ = connection.WithContext(auditCtx).Model(&clientkey.Key{}).Where("id = ?", key.ID).UpdateColumn("last_used_at", now).Error
|
||||
c.JSON(status, value)
|
||||
}
|
||||
}
|
||||
|
||||
type responseBuffer struct {
|
||||
gin.ResponseWriter
|
||||
body bytes.Buffer
|
||||
status int
|
||||
overflow bool
|
||||
}
|
||||
|
||||
func (w *responseBuffer) WriteHeader(status int) { w.status = status }
|
||||
func (w *responseBuffer) WriteHeaderNow() {}
|
||||
func (w *responseBuffer) Status() int { return w.status }
|
||||
func (w *responseBuffer) Size() int { return w.body.Len() }
|
||||
func (w *responseBuffer) Written() bool { return w.body.Len() > 0 }
|
||||
func (w *responseBuffer) Write(b []byte) (int, error) {
|
||||
if w.body.Len()+len(b) > 32<<20 {
|
||||
w.overflow = true
|
||||
return len(b), nil
|
||||
}
|
||||
return w.body.Write(b)
|
||||
}
|
||||
func (w *responseBuffer) WriteString(s string) (int, error) { return w.Write([]byte(s)) }
|
||||
func (w *responseBuffer) Flush() {}
|
||||
|
||||
func sensitive(k string) bool {
|
||||
key := strings.ToLower(strings.ReplaceAll(strings.ReplaceAll(k, "_", ""), "-", ""))
|
||||
for _, part := range []string{"password", "passwd", "secret", "credential", "cookie", "authorization", "apikey", "accesstoken", "devicetoken", "refreshtoken", "recoverycode"} {
|
||||
if strings.Contains(key, part) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
switch key {
|
||||
case "token", "tokenhash", "digest", "rawjson", "rawpayload", "rawresponse", "requestheaders", "responseheaders":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
func redact(v any) any {
|
||||
switch value := v.(type) {
|
||||
case map[string]any:
|
||||
for k, item := range value {
|
||||
if sensitive(k) {
|
||||
delete(value, k)
|
||||
} else {
|
||||
value[k] = redact(item)
|
||||
}
|
||||
}
|
||||
case []any:
|
||||
for i, item := range value {
|
||||
value[i] = redact(item)
|
||||
}
|
||||
}
|
||||
return v
|
||||
}
|
||||
@@ -0,0 +1,165 @@
|
||||
package clientapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"encoding/json"
|
||||
"github.com/gin-gonic/gin"
|
||||
"go-admin/app/goauto/clientkey"
|
||||
"go-admin/common/clientprincipal"
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func fixture(t *testing.T) (*gorm.DB, clientkey.Service) {
|
||||
t.Helper()
|
||||
gin.SetMode(gin.TestMode)
|
||||
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sql, _ := db.DB()
|
||||
sql.SetMaxOpenConns(1)
|
||||
t.Cleanup(func() { sql.Close() })
|
||||
if err = db.AutoMigrate(&clientkey.Key{}, &clientkey.Audit{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return db, clientkey.Service{DB: db, Modules: Catalog(Inventory())}
|
||||
}
|
||||
func TestEveryRouteIsExplicitlyScoped(t *testing.T) {
|
||||
db, s := fixture(t)
|
||||
routes := Inventory()
|
||||
if len(s.Modules) != 12 {
|
||||
t.Fatal("menu groups lost")
|
||||
}
|
||||
router := gin.New()
|
||||
seen := map[string]bool{}
|
||||
for _, e := range routes {
|
||||
key := e.Method + e.Path
|
||||
if seen[key] {
|
||||
t.Fatal("duplicate route")
|
||||
}
|
||||
seen[key] = true
|
||||
if strings.Contains(e.Path, "payment") || strings.Contains(e.Path, "token") || strings.Contains(e.Path, "client-keys") || e.Handle == nil {
|
||||
t.Fatal("forbidden route")
|
||||
}
|
||||
router.Handle(e.Method, "/api/client/v1"+e.Path, Gate(db, e), func(c *gin.Context) { c.JSON(200, gin.H{"data": "ok"}) })
|
||||
if e.Method != "GET" && e.Capability == "read" {
|
||||
t.Fatal("mutating read permission")
|
||||
}
|
||||
}
|
||||
for _, e := range routes {
|
||||
for _, scheme := range []string{"http", "https"} {
|
||||
grant := clientkey.Grant{Module: e.Module}
|
||||
v, token, err := s.Create(context.Background(), "test", []clientkey.Grant{grant}, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
path := e.Path
|
||||
for _, param := range []string{":productId", ":runId", ":recordId", ":jobId", ":batchId", ":shopId", ":ruleId", ":taskId"} {
|
||||
path = strings.ReplaceAll(path, param, "1")
|
||||
}
|
||||
req := httptest.NewRequest(e.Method, scheme+"://example.test/api/client/v1"+path, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
rr := httptest.NewRecorder()
|
||||
router.ServeHTTP(rr, req)
|
||||
want := 200
|
||||
if e.Capability != "read" {
|
||||
want = 403
|
||||
}
|
||||
if rr.Code != want {
|
||||
t.Fatalf("%s got %d want %d", keyFor(e), rr.Code, want)
|
||||
}
|
||||
if e.Capability == "write" {
|
||||
grant.Write = true
|
||||
} else if e.Capability != "read" {
|
||||
grant.Actions = []string{e.Capability}
|
||||
}
|
||||
if _, err = s.Update(context.Background(), v.ID, 1, 1, []clientkey.Grant{grant}, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rr = httptest.NewRecorder()
|
||||
router.ServeHTTP(rr, req.Clone(context.Background()))
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("authorized %s failed: %d", keyFor(e), rr.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
func keyFor(e Endpoint) string { return e.Method + " " + e.Path }
|
||||
func TestRevocationTransportRedactionAndAudit(t *testing.T) {
|
||||
db, s := fixture(t)
|
||||
v, token, err := s.Create(context.Background(), "test", []clientkey.Grant{{Module: "pdd_products"}}, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
e := Endpoint{Method: "GET", Path: "/pdd-products", Module: "pdd_products", Capability: "read"}
|
||||
router := gin.New()
|
||||
calls := 0
|
||||
router.GET("/api/client/v1/pdd-products", Gate(db, e), func(c *gin.Context) {
|
||||
calls++
|
||||
id, ok := clientprincipal.Get(c)
|
||||
if !ok || id.KeyID != v.ID {
|
||||
t.Error("client attribution missing")
|
||||
}
|
||||
c.JSON(200, gin.H{"code": 200, "data": gin.H{"apiKey": "sentinel-secret", "rawJson": "sentinel-raw", "title": "product", "nested": []any{gin.H{"device_token": "sentinel-token"}}}})
|
||||
})
|
||||
send := func(tlsOn bool, credential, path string) *httptest.ResponseRecorder {
|
||||
req := httptest.NewRequest("GET", "http://example.test"+path, nil)
|
||||
if tlsOn {
|
||||
req.TLS = &tls.ConnectionState{}
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+credential)
|
||||
rr := httptest.NewRecorder()
|
||||
router.ServeHTTP(rr, req)
|
||||
return rr
|
||||
}
|
||||
path := "/api/client/v1/pdd-products"
|
||||
for _, tlsOn := range []bool{false, true} {
|
||||
if send(tlsOn, "jwt", path).Code != 401 || send(tlsOn, "", path).Code != 401 || send(tlsOn, token, path+"?token=invalid").Code != 400 {
|
||||
t.Fatal("credential fallback accepted")
|
||||
}
|
||||
rr := send(tlsOn, token, path)
|
||||
if rr.Code != 200 || strings.Contains(rr.Body.String(), "sentinel") || !strings.Contains(rr.Body.String(), "product") {
|
||||
t.Fatal("redaction failed")
|
||||
}
|
||||
if rr.Header().Get("Cache-Control") != "no-store" || rr.Header().Get("X-Client-Request-Id") == "" {
|
||||
t.Fatal("cache or audit headers missing")
|
||||
}
|
||||
}
|
||||
if _, err = s.Update(context.Background(), v.ID, 1, 1, nil, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if send(true, token, path).Code != 401 || send(false, token, path).Code != 401 || calls != 2 {
|
||||
t.Fatal("revocation not immediate")
|
||||
}
|
||||
var logs []clientkey.Audit
|
||||
db.Find(&logs)
|
||||
raw, _ := json.Marshal(logs)
|
||||
if strings.Contains(string(raw), token) || strings.Contains(string(raw), "sentinel") {
|
||||
t.Fatal("audit leaked payload")
|
||||
}
|
||||
}
|
||||
func TestAuditUnavailableDoesNotExecute(t *testing.T) {
|
||||
db, s := fixture(t)
|
||||
_, token, err := s.Create(context.Background(), "test", []clientkey.Grant{{Module: "pdd_products", Write: true}}, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
db.Migrator().DropTable(&clientkey.Audit{})
|
||||
router := gin.New()
|
||||
called := false
|
||||
e := Endpoint{Method: "POST", Path: "/pdd-products", Module: "pdd_products", Capability: "write"}
|
||||
router.POST(e.Path, Gate(db, e), func(c *gin.Context) { called = true; c.JSON(200, gin.H{}) })
|
||||
req := httptest.NewRequest("POST", "https://example.test"+e.Path, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
rr := httptest.NewRecorder()
|
||||
router.ServeHTTP(rr, req)
|
||||
if rr.Code != 503 || called {
|
||||
t.Fatal("executed without audit")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
package clientapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"github.com/gin-gonic/gin"
|
||||
jwt "github.com/go-admin-team/go-admin-core/sdk/pkg/jwtauth"
|
||||
"go-admin/app/goauto/clientkey"
|
||||
"go-admin/app/goauto/models"
|
||||
"go-admin/app/goauto/product"
|
||||
"go-admin/common/middleware"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestClientCanCreateProductWithoutLoginAndReplay(t *testing.T) {
|
||||
db, s := fixture(t)
|
||||
if err := db.AutoMigrate(&models.PDDProduct{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, token, err := s.Create(context.Background(), "test", []clientkey.Grant{{Module: "pdd_products", Write: true}}, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
e := Endpoint{Method: "POST", Path: "/pdd-products", Module: "pdd_products", Capability: "write"}
|
||||
router := gin.New()
|
||||
router.Use(func(c *gin.Context) { c.Set("db", db); c.Next() })
|
||||
router.POST("/api/client/v1/pdd-products", Gate(db, e), product.Handler{}.Create)
|
||||
for n := 0; n < 2; n++ {
|
||||
req := httptest.NewRequest("POST", "https://example.test/api/client/v1/pdd-products", strings.NewReader(`{"requestId":"00000000-0000-4000-8000-000000000237","url":"https://mobile.yangkeduo.com/goods.html?goods_id=100000000001"}`))
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
rr := httptest.NewRecorder()
|
||||
router.ServeHTTP(rr, req)
|
||||
if rr.Code != 200 {
|
||||
t.Fatalf("product create failed status %d", rr.Code)
|
||||
}
|
||||
var body struct {
|
||||
Data struct {
|
||||
Replayed bool `json:"replayed"`
|
||||
}
|
||||
}
|
||||
json.Unmarshal(rr.Body.Bytes(), &body)
|
||||
if (n == 1) != body.Data.Replayed {
|
||||
t.Fatal("business idempotency changed")
|
||||
}
|
||||
}
|
||||
var count int64
|
||||
db.Model(&models.PDDProduct{}).Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatal("duplicate business write")
|
||||
}
|
||||
}
|
||||
|
||||
func TestManagementRequiresAdminNotClientIdentity(t *testing.T) {
|
||||
db, s := fixture(t)
|
||||
h := clientkey.Handler{DB: db, Modules: s.Modules}
|
||||
for _, role := range []string{"admin", "purchaser", "client", ""} {
|
||||
for _, scheme := range []string{"http", "https"} {
|
||||
router := gin.New()
|
||||
router.Use(func(c *gin.Context) {
|
||||
c.Set(jwt.JwtPayloadKey, jwt.MapClaims{"rolekey": role, "identity": float64(7)})
|
||||
c.Next()
|
||||
})
|
||||
router.POST("/keys", middleware.RequireRoleKey("admin"), NoStore, h.Create)
|
||||
req := httptest.NewRequest("POST", scheme+"://example.test/keys", strings.NewReader(`{"name":"test","grants":[{"module":"pdd_products","write":false,"actions":[]}]}`))
|
||||
rr := httptest.NewRecorder()
|
||||
router.ServeHTTP(rr, req)
|
||||
want := 403
|
||||
if role == "admin" {
|
||||
want = 200
|
||||
}
|
||||
if rr.Code != want {
|
||||
t.Fatalf("role %q status %d", role, rr.Code)
|
||||
}
|
||||
if role == "admin" && rr.Header().Get("Cache-Control") != "no-store" {
|
||||
t.Fatal("one-time secret cacheable")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
package clientapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/sdk/pkg"
|
||||
"go-admin/app/goauto/aimatching"
|
||||
"io"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
// resolveSpecs accepts only specification text, never a provider URL/key,
|
||||
// prompt, raw order, address or account. It returns a suggestion, not an order.
|
||||
func resolveSpecs(c *gin.Context) {
|
||||
var req struct {
|
||||
TargetColor string `json:"targetColor"`
|
||||
TargetSize string `json:"targetSize"`
|
||||
Colors []string `json:"colors"`
|
||||
Sizes []string `json:"sizes"`
|
||||
}
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, 64<<10)
|
||||
d := json.NewDecoder(c.Request.Body)
|
||||
d.DisallowUnknownFields()
|
||||
if d.Decode(&req) != nil || d.Decode(&struct{}{}) != io.EOF || len(req.Colors) > 200 || len(req.Sizes) > 200 {
|
||||
c.JSON(422, gin.H{"code": 422, "message": "规格请求无效"})
|
||||
return
|
||||
}
|
||||
for _, s := range append(append([]string{req.TargetColor, req.TargetSize}, req.Colors...), req.Sizes...) {
|
||||
if len([]rune(s)) > 255 {
|
||||
c.JSON(422, gin.H{"code": 422, "message": "规格值过长"})
|
||||
return
|
||||
}
|
||||
}
|
||||
db, err := pkg.GetOrm(c)
|
||||
if err != nil {
|
||||
c.JSON(503, gin.H{"code": 503})
|
||||
return
|
||||
}
|
||||
v, err := aimatching.NewService(db).Resolve(c.Request.Context(), aimatching.MatchRequest{TargetColor: req.TargetColor, TargetSize: req.TargetSize, Colors: req.Colors, Sizes: req.Sizes})
|
||||
if err != nil {
|
||||
c.JSON(422, gin.H{"code": 422, "message": "未获得可靠匹配,请检查候选规格或联系管理员"})
|
||||
return
|
||||
}
|
||||
c.JSON(200, gin.H{"code": 200, "data": gin.H{"mappedColor": v.MappedColor, "mappedSize": v.MappedSize, "source": v.Source}})
|
||||
}
|
||||
@@ -0,0 +1,170 @@
|
||||
// Package clientapi exposes only the reviewed client route inventory. It never
|
||||
// forwards arbitrary URLs or synthesizes a logged-in administrator identity.
|
||||
package clientapi
|
||||
|
||||
import (
|
||||
"go-admin/app/goauto/access"
|
||||
"go-admin/app/goauto/aimatching"
|
||||
"go-admin/app/goauto/clientkey"
|
||||
"go-admin/app/goauto/device"
|
||||
"go-admin/app/goauto/product"
|
||||
"go-admin/app/goauto/purchase"
|
||||
"go-admin/app/goauto/purchaserule"
|
||||
"go-admin/app/goauto/rule"
|
||||
"go-admin/app/goauto/shopeeproduct"
|
||||
"go-admin/app/goauto/sybimport"
|
||||
"go-admin/app/goauto/sybinnercode"
|
||||
"go-admin/app/goauto/sybshop"
|
||||
"go-admin/app/goauto/task"
|
||||
"go-admin/common/middleware"
|
||||
"sort"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/sdk/pkg"
|
||||
jwt "github.com/go-admin-team/go-admin-core/sdk/pkg/jwtauth"
|
||||
)
|
||||
|
||||
type Endpoint struct {
|
||||
Method, Path, Module, Capability string
|
||||
Handle gin.HandlerFunc
|
||||
}
|
||||
|
||||
func Inventory() []Endpoint {
|
||||
p := product.Handler{}
|
||||
s := shopeeproduct.Handler{}
|
||||
y := sybimport.Handler{}
|
||||
i := sybinnercode.Handler{}
|
||||
shop := sybshop.Handler{}
|
||||
r := rule.Handler{}
|
||||
pr := purchaserule.Handler{}
|
||||
t := task.Handler{}
|
||||
buy := purchase.Handler{}
|
||||
return []Endpoint{
|
||||
{"POST", "/ai-matching-settings/resolve", access.ModuleAIMatching, "match", resolveSpecs},
|
||||
{"GET", "/devices", access.ModuleDevices, "read", device.Handler{}.List},
|
||||
{"GET", "/pdd-products", access.ModulePDDProducts, "read", p.List},
|
||||
{"GET", "/pdd-products/:productId", access.ModulePDDProducts, "read", p.Detail},
|
||||
{"POST", "/pdd-products", access.ModulePDDProducts, "write", p.Create},
|
||||
{"PATCH", "/pdd-products/:productId", access.ModulePDDProducts, "write", p.Update},
|
||||
{"GET", "/shopee-products", access.ModuleShopeeProducts, "read", s.List},
|
||||
{"GET", "/shopee-products/:productId", access.ModuleShopeeProducts, "read", s.Detail},
|
||||
{"POST", "/shopee-products", access.ModuleShopeeProducts, "write", s.Create},
|
||||
{"PATCH", "/shopee-products/:productId", access.ModuleShopeeProducts, "write", s.Update},
|
||||
{"POST", "/shopee-products/:productId/link-pdd", access.ModuleShopeeProducts, "write", s.LinkPDD},
|
||||
{"POST", "/shopee-products/:productId/specs/values", access.ModuleShopeeProducts, "write", s.AddSpecValue},
|
||||
{"PUT", "/shopee-products/:productId/specs/mapping", access.ModuleShopeeProducts, "write", s.SetMapping},
|
||||
{"DELETE", "/shopee-products/:productId/specs/values", access.ModuleShopeeProducts, "delete", s.RemoveSpecValue},
|
||||
{"DELETE", "/shopee-products/:productId/specs/mapping", access.ModuleShopeeProducts, "delete", s.ClearMapping},
|
||||
{"POST", "/shopee-products/batch-delete", access.ModuleShopeeProducts, "delete", s.BatchDelete},
|
||||
{"POST", "/shopee-products/:productId/specs/mapping/auto-match", access.ModuleShopeeProducts, "match", s.AutoMatchMappings},
|
||||
{"POST", "/shopee-products/:productId/specs/mapping/confirm", access.ModuleShopeeProducts, "match", s.ConfirmMapping},
|
||||
{"GET", "/syb-products", access.ModuleSYBProducts, "read", y.List},
|
||||
{"GET", "/syb-products/:productId", access.ModuleSYBProducts, "read", y.Detail},
|
||||
{"PATCH", "/syb-products/:productId/correction", access.ModuleSYBProducts, "write", y.ManualCorrect},
|
||||
{"POST", "/syb-products/:productId/reparse", access.ModuleSYBProducts, "reparse", y.Reparse},
|
||||
{"POST", "/syb-products/reparse-batch", access.ModuleSYBProducts, "reparse", y.ReparseBatch},
|
||||
{"GET", "/syb-products/sync-runs", access.ModuleSYBSyncRuns, "read", y.ListSyncRuns},
|
||||
{"GET", "/syb-products/sync-runs/:runId", access.ModuleSYBSyncRuns, "read", y.SyncRunDetail},
|
||||
{"POST", "/syb-products/import", access.ModuleSYBSyncRuns, "sync", y.Import},
|
||||
{"GET", "/syb-inner-codes", access.ModuleSYBInnerCodes, "read", i.List},
|
||||
{"GET", "/syb-inner-codes/:recordId", access.ModuleSYBInnerCodes, "read", i.Detail},
|
||||
{"GET", "/syb-inner-codes/match-jobs/:jobId", access.ModuleSYBInnerCodes, "read", i.MatchJob},
|
||||
{"GET", "/syb-inner-codes/apply-batches/:batchId", access.ModuleSYBInnerCodes, "read", i.ApplyBatch},
|
||||
{"POST", "/syb-inner-codes/rematch", access.ModuleSYBInnerCodes, "match", i.Rematch},
|
||||
{"POST", "/syb-inner-codes/import", access.ModuleSYBInnerCodes, "import", i.Import},
|
||||
{"POST", "/syb-inner-codes/batch-delete", access.ModuleSYBInnerCodes, "delete", i.Delete},
|
||||
{"POST", "/syb-inner-codes/apply-preview", access.ModuleSYBInnerCodes, "writeback", i.ApplyPreview},
|
||||
{"POST", "/syb-inner-codes/apply", access.ModuleSYBInnerCodes, "writeback", i.Apply},
|
||||
{"POST", "/syb-inner-codes/:recordId/recheck", access.ModuleSYBInnerCodes, "match", i.Recheck},
|
||||
{"GET", "/syb-shops", access.ModuleSYBShops, "read", shop.List},
|
||||
{"POST", "/syb-shops", access.ModuleSYBShops, "write", shop.Create},
|
||||
{"PATCH", "/syb-shops/:shopId/name", access.ModuleSYBShops, "write", shop.Rename},
|
||||
{"PATCH", "/syb-shops/:shopId/enabled", access.ModuleSYBShops, "write", shop.SetEnabled},
|
||||
{"DELETE", "/syb-shops/:shopId", access.ModuleSYBShops, "delete", shop.Delete},
|
||||
{"GET", "/collection-rules", access.ModuleCollectionRules, "read", r.List},
|
||||
{"POST", "/collection-rules", access.ModuleCollectionRules, "write", r.Create},
|
||||
{"PATCH", "/collection-rules/:ruleId", access.ModuleCollectionRules, "write", r.Update},
|
||||
{"DELETE", "/collection-rules/:ruleId", access.ModuleCollectionRules, "delete", r.Delete},
|
||||
{"GET", "/purchase-rules", access.ModulePurchaseRules, "read", pr.List},
|
||||
{"GET", "/purchase-rules/current", access.ModulePurchaseRules, "read", pr.Current},
|
||||
{"POST", "/purchase-rules", access.ModulePurchaseRules, "write", pr.Create},
|
||||
{"PATCH", "/purchase-rules/:ruleId", access.ModulePurchaseRules, "write", pr.Update},
|
||||
{"DELETE", "/purchase-rules/:ruleId", access.ModulePurchaseRules, "delete", pr.Delete},
|
||||
{"PUT", "/purchase-rules/current", access.ModulePurchaseRules, "activate", pr.SetCurrent},
|
||||
{"GET", "/collection-tasks", access.ModuleCollectionTasks, "read", t.AdminList},
|
||||
{"GET", "/collection-tasks/:taskId", access.ModuleCollectionTasks, "read", t.AdminDetail},
|
||||
{"POST", "/collection-tasks", access.ModuleCollectionTasks, "collect", t.AdminCreate},
|
||||
{"POST", "/collection-tasks/batch", access.ModuleCollectionTasks, "collect", t.AdminBatchCreate},
|
||||
{"POST", "/collection-tasks/:taskId/reset", access.ModuleCollectionTasks, "collect", t.AdminReset},
|
||||
{"DELETE", "/collection-tasks/:taskId", access.ModuleCollectionTasks, "delete", t.AdminDelete},
|
||||
{"GET", "/purchase-tasks", access.ModulePurchaseTasks, "read", buy.AdminList},
|
||||
{"GET", "/purchase-tasks/:taskId", access.ModulePurchaseTasks, "read", buy.AdminDetail},
|
||||
{"POST", "/purchase-tasks/batch-preview", access.ModulePurchaseTasks, "purchase", buy.AdminBatchPreview},
|
||||
{"POST", "/purchase-tasks", access.ModulePurchaseTasks, "purchase", buy.AdminCreate},
|
||||
{"POST", "/purchase-tasks/batch", access.ModulePurchaseTasks, "purchase", buy.AdminBatchCreate},
|
||||
{"POST", "/purchase-tasks/batch-retry", access.ModulePurchaseTasks, "purchase", buy.AdminBatchRetry},
|
||||
{"POST", "/purchase-tasks/stock", access.ModulePurchaseTasks, "purchase", buy.AdminCreateStock},
|
||||
{"GET", "/ai-matching-settings", access.ModuleAIMatching, "read", func(c *gin.Context) {
|
||||
db, err := pkg.GetOrm(c)
|
||||
if err != nil {
|
||||
c.JSON(500, gin.H{"code": 500})
|
||||
return
|
||||
}
|
||||
v, err := aimatching.NewService(db).Settings(c.Request.Context())
|
||||
if err != nil {
|
||||
c.JSON(500, gin.H{"code": 500})
|
||||
return
|
||||
}
|
||||
c.JSON(200, gin.H{"code": 200, "data": gin.H{"enabled": v.Enabled}})
|
||||
}},
|
||||
}
|
||||
}
|
||||
|
||||
func Catalog(routes []Endpoint) []clientkey.Module {
|
||||
mods := map[string]clientkey.Module{}
|
||||
for _, m := range access.GoAutoModules() {
|
||||
mods[m.Key] = clientkey.Module{Key: m.Key, Title: m.Title, Actions: []string{}}
|
||||
}
|
||||
for _, e := range routes {
|
||||
m := mods[e.Module]
|
||||
if e.Capability == "write" {
|
||||
m.Writable = true
|
||||
} else if e.Capability != "read" {
|
||||
found := false
|
||||
for _, a := range m.Actions {
|
||||
if a == e.Capability {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
m.Actions = append(m.Actions, e.Capability)
|
||||
}
|
||||
}
|
||||
mods[e.Module] = m
|
||||
}
|
||||
out := []clientkey.Module{}
|
||||
for _, g := range access.GoAutoMenuGroups() {
|
||||
for _, key := range g.ModuleKeys {
|
||||
m := mods[key]
|
||||
m.Group = g.Title
|
||||
sort.Strings(m.Actions)
|
||||
out = append(out, m)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func InitRouter(engine *gin.Engine, auth *jwt.GinJWTMiddleware) {
|
||||
routes := Inventory()
|
||||
catalog := Catalog(routes)
|
||||
h := clientkey.Handler{Modules: catalog}
|
||||
management := engine.Group("/api/admin/v1/client-keys", auth.MiddlewareFunc(), middleware.RequireRoleKey("admin"), NoStore)
|
||||
management.GET("", h.List)
|
||||
management.GET("/modules", h.Catalog)
|
||||
management.POST("", h.Create)
|
||||
management.PATCH("/:keyId/grants", h.Edit)
|
||||
management.POST("/:keyId/disable", h.Disable)
|
||||
for _, e := range routes {
|
||||
engine.Handle(e.Method, "/api/client/v1"+e.Path, Gate(nil, e), e.Handle)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
package clientkey
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/sdk/pkg"
|
||||
"github.com/go-admin-team/go-admin-core/sdk/pkg/jwtauth/user"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type Handler struct {
|
||||
DB *gorm.DB
|
||||
Modules []Module
|
||||
}
|
||||
|
||||
func (h Handler) service(c *gin.Context) (Service, bool) {
|
||||
db := h.DB
|
||||
var err error
|
||||
if db == nil {
|
||||
db, err = pkg.GetOrm(c)
|
||||
}
|
||||
if err != nil || db == nil {
|
||||
failure(c, errors.New("database unavailable"))
|
||||
return Service{}, false
|
||||
}
|
||||
c.Header("Cache-Control", "no-store")
|
||||
return Service{db, h.Modules}, true
|
||||
}
|
||||
func failure(c *gin.Context, err error) {
|
||||
status, message := 500, "服务端处理失败"
|
||||
switch {
|
||||
case errors.Is(err, ErrInvalid):
|
||||
status, message = 422, ErrInvalid.Error()
|
||||
case errors.Is(err, ErrConflict):
|
||||
status, message = 409, ErrConflict.Error()
|
||||
case errors.Is(err, gorm.ErrRecordNotFound):
|
||||
status, message = 404, "密钥不存在"
|
||||
}
|
||||
c.AbortWithStatusJSON(status, gin.H{"code": status, "message": message})
|
||||
}
|
||||
func (h Handler) Catalog(c *gin.Context) { c.JSON(200, gin.H{"code": 200, "data": h.Modules}) }
|
||||
func (h Handler) List(c *gin.Context) {
|
||||
s, ok := h.service(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
|
||||
if page < 1 {
|
||||
page = 1
|
||||
}
|
||||
size := 20
|
||||
var total int64
|
||||
var keys []Key
|
||||
if err := s.DB.WithContext(c.Request.Context()).Model(&Key{}).Count(&total).Error; err != nil {
|
||||
failure(c, err)
|
||||
return
|
||||
}
|
||||
if err := s.DB.WithContext(c.Request.Context()).Order("id DESC").Offset((page - 1) * size).Limit(size).Find(&keys).Error; err != nil {
|
||||
failure(c, err)
|
||||
return
|
||||
}
|
||||
items := make([]View, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
items = append(items, view(k))
|
||||
}
|
||||
c.JSON(200, gin.H{"code": 200, "data": gin.H{"items": items, "total": total, "page": page, "pageSize": size}})
|
||||
}
|
||||
func decode(c *gin.Context, v any) bool {
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, 64<<10)
|
||||
d := json.NewDecoder(c.Request.Body)
|
||||
d.DisallowUnknownFields()
|
||||
if d.Decode(v) != nil || d.Decode(&struct{}{}) != io.EOF {
|
||||
failure(c, ErrInvalid)
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
func (h Handler) Create(c *gin.Context) {
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
Grants []Grant `json:"grants"`
|
||||
}
|
||||
if !decode(c, &req) {
|
||||
return
|
||||
}
|
||||
s, ok := h.service(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
result, secret, err := s.Create(c.Request.Context(), req.Name, req.Grants, uint64(user.GetUserId(c)))
|
||||
if err != nil {
|
||||
failure(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(200, gin.H{"code": 200, "data": gin.H{"key": result, "secret": secret}})
|
||||
}
|
||||
func (h Handler) Edit(c *gin.Context) { h.update(c, false) }
|
||||
func (h Handler) Disable(c *gin.Context) { h.update(c, true) }
|
||||
func (h Handler) update(c *gin.Context, disable bool) {
|
||||
id, err := strconv.ParseUint(c.Param("keyId"), 10, 64)
|
||||
if err != nil {
|
||||
failure(c, ErrInvalid)
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Version uint64 `json:"version"`
|
||||
Grants []Grant `json:"grants"`
|
||||
}
|
||||
if !decode(c, &req) {
|
||||
return
|
||||
}
|
||||
s, ok := h.service(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
result, err := s.Update(c.Request.Context(), id, req.Version, uint64(user.GetUserId(c)), req.Grants, disable)
|
||||
if err != nil {
|
||||
failure(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(200, gin.H{"code": 200, "data": result})
|
||||
}
|
||||
@@ -0,0 +1,222 @@
|
||||
// Package clientkey implements independent, revocable client credentials (#237).
|
||||
package clientkey
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
var ErrInvalid = errors.New("名称或授权无效")
|
||||
var ErrConflict = errors.New("密钥已停用或授权已被修改,请刷新后重试")
|
||||
var ErrCredential = errors.New("客户端密钥无效或已停用")
|
||||
|
||||
type Grant struct {
|
||||
Module string `json:"module"`
|
||||
Write bool `json:"write"`
|
||||
Actions []string `json:"actions"`
|
||||
}
|
||||
type Module struct {
|
||||
Key string `json:"key"`
|
||||
Title string `json:"title"`
|
||||
Group string `json:"group"`
|
||||
Writable bool `json:"writable"`
|
||||
Actions []string `json:"actions"`
|
||||
}
|
||||
type Key struct {
|
||||
ID uint64 `gorm:"primaryKey" json:"id"`
|
||||
Name string `gorm:"size:80;not null" json:"name"`
|
||||
Prefix string `gorm:"size:24;not null" json:"prefix"`
|
||||
Digest string `gorm:"size:64;uniqueIndex;not null" json:"-"`
|
||||
GrantsJSON string `gorm:"type:text;not null" json:"-"`
|
||||
Enabled bool `gorm:"not null" json:"enabled"`
|
||||
Version uint64 `gorm:"not null" json:"version"`
|
||||
CreatedBy uint64 `json:"createdBy"`
|
||||
UpdatedBy uint64 `json:"updatedBy"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
LastUsedAt *time.Time `json:"lastUsedAt"`
|
||||
}
|
||||
|
||||
func (Key) TableName() string { return "client_api_key" }
|
||||
|
||||
type Audit struct {
|
||||
ID uint64 `gorm:"primaryKey"`
|
||||
KeyID uint64 `gorm:"index;not null"`
|
||||
ActorID uint64
|
||||
Event string `gorm:"size:32;not null"`
|
||||
RequestID string `gorm:"size:32;index"`
|
||||
Method string `gorm:"size:10"`
|
||||
Route string `gorm:"size:180"`
|
||||
Status int
|
||||
Changes string `gorm:"type:text"`
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
func (Audit) TableName() string { return "client_api_key_audit" }
|
||||
|
||||
type View struct {
|
||||
Key
|
||||
Grants []Grant `json:"grants"`
|
||||
}
|
||||
|
||||
func view(k Key) View {
|
||||
var g []Grant
|
||||
_ = json.Unmarshal([]byte(k.GrantsJSON), &g)
|
||||
return View{Key: k, Grants: g}
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
DB *gorm.DB
|
||||
Modules []Module
|
||||
}
|
||||
|
||||
func (s Service) Normalize(in []Grant) ([]Grant, error) {
|
||||
if len(in) == 0 || len(in) > len(s.Modules) {
|
||||
return nil, ErrInvalid
|
||||
}
|
||||
catalog := map[string]Module{}
|
||||
for _, m := range s.Modules {
|
||||
catalog[m.Key] = m
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
out := make([]Grant, 0, len(in))
|
||||
for _, g := range in {
|
||||
m, ok := catalog[g.Module]
|
||||
if !ok || seen[g.Module] || (g.Write && !m.Writable) {
|
||||
return nil, ErrInvalid
|
||||
}
|
||||
seen[g.Module] = true
|
||||
allowed := map[string]bool{}
|
||||
for _, a := range m.Actions {
|
||||
allowed[a] = true
|
||||
}
|
||||
used := map[string]bool{}
|
||||
actions := []string{}
|
||||
for _, a := range g.Actions {
|
||||
if !allowed[a] || used[a] {
|
||||
return nil, ErrInvalid
|
||||
}
|
||||
used[a] = true
|
||||
actions = append(actions, a)
|
||||
}
|
||||
sort.Strings(actions)
|
||||
out = append(out, Grant{g.Module, g.Write, actions})
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Module < out[j].Module })
|
||||
return out, nil
|
||||
}
|
||||
func (s Service) Create(ctx context.Context, name string, grants []Grant, actor uint64) (View, string, error) {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" || len([]rune(name)) > 80 || actor == 0 {
|
||||
return View{}, "", ErrInvalid
|
||||
}
|
||||
g, err := s.Normalize(grants)
|
||||
if err != nil {
|
||||
return View{}, "", err
|
||||
}
|
||||
raw, _ := json.Marshal(g)
|
||||
secret := make([]byte, 32)
|
||||
if _, err = rand.Read(secret); err != nil {
|
||||
return View{}, "", err
|
||||
}
|
||||
token := "gak_" + hex.EncodeToString(secret)
|
||||
digest := sha256.Sum256([]byte(token))
|
||||
k := Key{Name: name, Prefix: token[:16], Digest: hex.EncodeToString(digest[:]), GrantsJSON: string(raw), Enabled: true, Version: 1, CreatedBy: actor, UpdatedBy: actor}
|
||||
err = s.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
if err := tx.Create(&k).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Create(&Audit{KeyID: k.ID, ActorID: actor, Event: "create", Changes: string(raw)}).Error
|
||||
})
|
||||
if err != nil {
|
||||
return View{}, "", err
|
||||
}
|
||||
return view(k), token, nil
|
||||
}
|
||||
func (s Service) Update(ctx context.Context, id, version, actor uint64, grants []Grant, disable bool) (View, error) {
|
||||
if id == 0 || version == 0 || actor == 0 {
|
||||
return View{}, ErrInvalid
|
||||
}
|
||||
var raw []byte
|
||||
if !disable {
|
||||
g, err := s.Normalize(grants)
|
||||
if err != nil {
|
||||
return View{}, err
|
||||
}
|
||||
raw, _ = json.Marshal(g)
|
||||
}
|
||||
var k Key
|
||||
err := s.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
if err := tx.First(&k, id).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
changes := map[string]any{"version": version + 1, "updated_by": actor, "updated_at": time.Now().UTC()}
|
||||
event := "edit"
|
||||
if disable {
|
||||
changes["enabled"] = false
|
||||
event = "disable"
|
||||
} else {
|
||||
changes["grants_json"] = string(raw)
|
||||
}
|
||||
result := tx.Model(&Key{}).Where("id = ? AND version = ? AND enabled = ?", id, version, true).Updates(changes)
|
||||
if result.Error != nil {
|
||||
return result.Error
|
||||
}
|
||||
if result.RowsAffected != 1 {
|
||||
return ErrConflict
|
||||
}
|
||||
diff, _ := json.Marshal(map[string]string{"before": k.GrantsJSON, "after": string(raw)})
|
||||
if err := tx.Create(&Audit{KeyID: id, ActorID: actor, Event: event, Changes: string(diff)}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.First(&k, id).Error
|
||||
})
|
||||
return view(k), err
|
||||
}
|
||||
func (s Service) Authenticate(ctx context.Context, token string) (Key, error) {
|
||||
if len(token) != 68 || !strings.HasPrefix(token, "gak_") {
|
||||
return Key{}, ErrCredential
|
||||
}
|
||||
if _, err := hex.DecodeString(token[4:]); err != nil {
|
||||
return Key{}, ErrCredential
|
||||
}
|
||||
digest := sha256.Sum256([]byte(token))
|
||||
var k Key
|
||||
err := s.DB.WithContext(ctx).Where("digest = ? AND enabled = ?", hex.EncodeToString(digest[:]), true).First(&k).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return Key{}, ErrCredential
|
||||
}
|
||||
return k, err
|
||||
}
|
||||
func (k Key) Allows(module, capability string) bool {
|
||||
var grants []Grant
|
||||
if json.Unmarshal([]byte(k.GrantsJSON), &grants) != nil {
|
||||
return false
|
||||
}
|
||||
for _, g := range grants {
|
||||
if g.Module != module {
|
||||
continue
|
||||
}
|
||||
if capability == "read" {
|
||||
return true
|
||||
}
|
||||
if capability == "write" {
|
||||
return g.Write
|
||||
}
|
||||
for _, a := range g.Actions {
|
||||
if a == capability {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
package clientkey
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func testService(t *testing.T) Service {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sql, _ := db.DB()
|
||||
sql.SetMaxOpenConns(1)
|
||||
t.Cleanup(func() { sql.Close() })
|
||||
if err = db.AutoMigrate(&Key{}, &Audit{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return Service{db, []Module{{Key: "products", Writable: true, Actions: []string{"delete"}}, {Key: "devices", Actions: []string{}}}}
|
||||
}
|
||||
func TestCredentialLifecycle(t *testing.T) {
|
||||
s := testService(t)
|
||||
ctx := context.Background()
|
||||
v, token, err := s.Create(ctx, "Tool", []Grant{{Module: "products"}}, 7)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(token) != 68 || v.Digest == token {
|
||||
t.Fatal("invalid credential generation")
|
||||
}
|
||||
wire, _ := json.Marshal(v)
|
||||
if strings.Contains(string(wire), token) || strings.Contains(string(wire), v.Digest) {
|
||||
t.Fatal("secret serialized")
|
||||
}
|
||||
k, err := s.Authenticate(ctx, token)
|
||||
if err != nil || !k.Allows("products", "read") || k.Allows("products", "write") || k.Allows("devices", "read") {
|
||||
t.Fatal("default grants")
|
||||
}
|
||||
updated, err := s.Update(ctx, k.ID, 1, 8, []Grant{{Module: "products", Write: true, Actions: []string{"delete"}}}, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
k, err = s.Authenticate(ctx, token)
|
||||
if err != nil || !k.Allows("products", "write") || !k.Allows("products", "delete") || updated.Version != 2 || k.Digest != v.Digest {
|
||||
t.Fatal("edit changed credential or failed to apply grants")
|
||||
}
|
||||
if _, err = s.Update(ctx, k.ID, 1, 8, []Grant{{Module: "devices"}}, false); !errors.Is(err, ErrConflict) {
|
||||
t.Fatal("stale edit accepted")
|
||||
}
|
||||
if _, err = s.Update(ctx, k.ID, 2, 8, nil, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, token); !errors.Is(err, ErrCredential) {
|
||||
t.Fatal("disabled credential accepted")
|
||||
}
|
||||
if _, err = s.Update(ctx, k.ID, 3, 8, []Grant{{Module: "products"}}, false); !errors.Is(err, ErrConflict) {
|
||||
t.Fatal("disabled key edited")
|
||||
}
|
||||
var logs []Audit
|
||||
s.DB.Find(&logs)
|
||||
if len(logs) != 3 {
|
||||
t.Fatalf("audit count %d", len(logs))
|
||||
}
|
||||
data, _ := json.Marshal(logs)
|
||||
if strings.Contains(string(data), token) {
|
||||
t.Fatal("secret in audit")
|
||||
}
|
||||
}
|
||||
func TestInvalidGrantsAndCredentials(t *testing.T) {
|
||||
s := testService(t)
|
||||
for _, g := range [][]Grant{nil, {{Module: "admin"}}, {{Module: "devices", Write: true}}, {{Module: "products", Actions: []string{"payment"}}}, {{Module: "products"}, {Module: "products"}}} {
|
||||
if _, err := s.Normalize(g); err == nil {
|
||||
t.Fatal("invalid grant accepted")
|
||||
}
|
||||
}
|
||||
for _, token := range []string{"", "jwt", "gak_" + strings.Repeat("z", 64), strings.Repeat("a", 68)} {
|
||||
if _, err := s.Authenticate(context.Background(), token); !errors.Is(err, ErrCredential) {
|
||||
t.Fatal("invalid credential accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestAuditFailureRollsBack(t *testing.T) {
|
||||
s := testService(t)
|
||||
ctx := context.Background()
|
||||
v, _, err := s.Create(ctx, "Tool", []Grant{{Module: "products"}}, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = s.DB.Migrator().DropTable(&Audit{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Update(ctx, v.ID, 1, 1, nil, true); err == nil {
|
||||
t.Fatal("audit failure ignored")
|
||||
}
|
||||
var k Key
|
||||
s.DB.First(&k, v.ID)
|
||||
if !k.Enabled || k.Version != 1 {
|
||||
t.Fatal("mutation not rolled back")
|
||||
}
|
||||
if _, secret, err := s.Create(ctx, "Failed", []Grant{{Module: "products"}}, 1); err == nil || secret != "" {
|
||||
t.Fatal("creation audit failure ignored")
|
||||
}
|
||||
var count int64
|
||||
s.DB.Model(&Key{}).Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatal("key persisted without audit")
|
||||
}
|
||||
}
|
||||
@@ -34,7 +34,11 @@ func MigratedModels() []any {
|
||||
&models.PDDProduct{},
|
||||
&models.AIMatchingSetting{},
|
||||
&models.ShopeeProduct{},
|
||||
&models.ShopeeSpecAutoMatchRun{},
|
||||
&models.ShopeeSpecAutoMatchWorkItem{},
|
||||
&models.SYBProduct{},
|
||||
&models.SYBSpecAIParseRun{},
|
||||
&models.SYBSpecAIParseWorkItem{},
|
||||
&models.SYBSession{},
|
||||
&models.SYBShop{},
|
||||
&models.SYBSyncRun{},
|
||||
|
||||
@@ -536,6 +536,15 @@ type SYBProduct struct {
|
||||
// recording the parser's own last output for audit even after a manual
|
||||
// correction; it is not overwritten by the correction itself.
|
||||
ManuallyConfirmed bool `json:"manuallyConfirmed" gorm:"not null;default:false"`
|
||||
// AIConfirmed is independent of ParseStatus: ParseStatus remains the
|
||||
// deterministic parser's audit result, while these fields record a closed-
|
||||
// candidate, high-confidence AI decision. Human correction always clears
|
||||
// and supersedes this decision.
|
||||
AIConfirmed bool `json:"aiConfirmed" gorm:"not null;default:false;index"`
|
||||
AIConfidence *float64 `json:"aiConfidence,omitempty"`
|
||||
AIReason string `json:"aiReason,omitempty" gorm:"size:500;not null;default:''"`
|
||||
AIConfirmedAt *time.Time `json:"aiConfirmedAt,omitempty"`
|
||||
AIInputFingerprint string `json:"-" gorm:"size:64;not null;default:'';index"`
|
||||
|
||||
// RawJSON is the untouched `details[]` element as SYB returned it. It is
|
||||
// what reparse (#41: "适用于解析规则更新后批量重跑,只读取已保存的原始
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
package models
|
||||
|
||||
import "time"
|
||||
|
||||
// ShopeeSpecAutoMatchRun is one scheduled or administrator-triggered batch.
|
||||
// ActiveSlot is 1 only while running; its nullable unique index is the
|
||||
// database-level cross-process mutex shared by both trigger paths.
|
||||
type ShopeeSpecAutoMatchRun struct {
|
||||
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
|
||||
RequestID string `json:"requestId" gorm:"size:36;not null;uniqueIndex:ux_shopee_spec_auto_match_run_request"`
|
||||
Trigger string `json:"trigger" gorm:"size:16;not null;index"`
|
||||
Status string `json:"status" gorm:"size:24;not null;index"`
|
||||
ActiveSlot *uint8 `json:"-" gorm:"uniqueIndex:ux_shopee_spec_auto_match_run_active"`
|
||||
LeaseOwner string `json:"-" gorm:"size:64;not null;default:''"`
|
||||
LeaseExpiresAt *time.Time `json:"-" gorm:"index"`
|
||||
RequestedBy *uint64 `json:"requestedBy,omitempty"`
|
||||
BatchLimit int `json:"batchLimit" gorm:"not null;default:20"`
|
||||
ScannedCount int `json:"scannedCount" gorm:"not null;default:0"`
|
||||
EligibleCount int `json:"eligibleCount" gorm:"not null;default:0"`
|
||||
ProcessedCount int `json:"processedCount" gorm:"not null;default:0"`
|
||||
ConfirmedCount int `json:"confirmedCount" gorm:"not null;default:0"`
|
||||
UnmatchedCount int `json:"unmatchedCount" gorm:"not null;default:0"`
|
||||
FailedCount int `json:"failedCount" gorm:"not null;default:0"`
|
||||
ErrorSummary string `json:"errorSummary,omitempty" gorm:"size:500;not null;default:''"`
|
||||
StartedAt time.Time `json:"startedAt" gorm:"not null"`
|
||||
FinishedAt *time.Time `json:"finishedAt,omitempty"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func (ShopeeSpecAutoMatchRun) TableName() string { return "shopee_spec_auto_match_run" }
|
||||
|
||||
// ShopeeSpecAutoMatchWorkItem remembers the last input fingerprint and retry
|
||||
// state for each product, preventing unchanged low-confidence inputs from
|
||||
// repeatedly spending AI calls.
|
||||
type ShopeeSpecAutoMatchWorkItem struct {
|
||||
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
|
||||
ShopeeProductID uint64 `json:"shopeeProductId" gorm:"not null;uniqueIndex:ux_shopee_spec_auto_match_work_product"`
|
||||
RunID *uint64 `json:"runId,omitempty" gorm:"index"`
|
||||
InputFingerprint string `json:"inputFingerprint" gorm:"size:128;not null;default:'';index"`
|
||||
Status string `json:"status" gorm:"size:24;not null;index"`
|
||||
AttemptCount int `json:"attemptCount" gorm:"not null;default:0"`
|
||||
NextAttemptAt *time.Time `json:"nextAttemptAt,omitempty" gorm:"index"`
|
||||
LeaseOwner string `json:"-" gorm:"size:64;not null;default:''"`
|
||||
LeaseExpiresAt *time.Time `json:"-" gorm:"index"`
|
||||
ConfirmedCount int `json:"confirmedCount" gorm:"not null;default:0"`
|
||||
UnmatchedCount int `json:"unmatchedCount" gorm:"not null;default:0"`
|
||||
LastErrorCode string `json:"lastErrorCode,omitempty" gorm:"size:64;not null;default:''"`
|
||||
LastError string `json:"lastError,omitempty" gorm:"size:500;not null;default:''"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func (ShopeeSpecAutoMatchWorkItem) TableName() string {
|
||||
return "shopee_spec_auto_match_work_item"
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
package models
|
||||
|
||||
import "time"
|
||||
|
||||
// SYBSpecAIParseRun is one globally serialized scheduled batch.
|
||||
type SYBSpecAIParseRun struct {
|
||||
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
|
||||
RequestID string `json:"requestId" gorm:"size:36;not null;uniqueIndex:ux_syb_spec_ai_parse_run_request"`
|
||||
Trigger string `json:"trigger" gorm:"size:16;not null;index"`
|
||||
Status string `json:"status" gorm:"size:24;not null;index"`
|
||||
ActiveSlot *uint8 `json:"-" gorm:"uniqueIndex:ux_syb_spec_ai_parse_run_active"`
|
||||
LeaseOwner string `json:"-" gorm:"size:64;not null;default:''"`
|
||||
LeaseExpiresAt *time.Time `json:"-" gorm:"index"`
|
||||
BatchLimit int `json:"batchLimit" gorm:"not null;default:20"`
|
||||
ScannedCount int `json:"scannedCount" gorm:"not null;default:0"`
|
||||
EligibleCount int `json:"eligibleCount" gorm:"not null;default:0"`
|
||||
ProcessedCount int `json:"processedCount" gorm:"not null;default:0"`
|
||||
ConfirmedCount int `json:"confirmedCount" gorm:"not null;default:0"`
|
||||
UnmatchedCount int `json:"unmatchedCount" gorm:"not null;default:0"`
|
||||
FailedCount int `json:"failedCount" gorm:"not null;default:0"`
|
||||
ErrorSummary string `json:"errorSummary,omitempty" gorm:"size:500;not null;default:''"`
|
||||
StartedAt time.Time `json:"startedAt" gorm:"not null"`
|
||||
FinishedAt *time.Time `json:"finishedAt,omitempty"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func (SYBSpecAIParseRun) TableName() string { return "syb_spec_ai_parse_run" }
|
||||
|
||||
// SYBSpecAIParseWorkItem prevents unchanged ambiguous input from repeatedly
|
||||
// spending provider calls and owns the per-row recovery lease.
|
||||
type SYBSpecAIParseWorkItem struct {
|
||||
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
|
||||
SYBProductID uint64 `json:"sybProductId" gorm:"not null;uniqueIndex:ux_syb_spec_ai_parse_work_product"`
|
||||
RunID *uint64 `json:"runId,omitempty" gorm:"index"`
|
||||
InputFingerprint string `json:"inputFingerprint" gorm:"size:64;not null;default:'';index"`
|
||||
Status string `json:"status" gorm:"size:24;not null;index"`
|
||||
AttemptCount int `json:"attemptCount" gorm:"not null;default:0"`
|
||||
NextAttemptAt *time.Time `json:"nextAttemptAt,omitempty" gorm:"index"`
|
||||
LeaseOwner string `json:"-" gorm:"size:64;not null;default:''"`
|
||||
LeaseExpiresAt *time.Time `json:"-" gorm:"index"`
|
||||
LastErrorCode string `json:"lastErrorCode,omitempty" gorm:"size:64;not null;default:''"`
|
||||
LastError string `json:"lastError,omitempty" gorm:"size:500;not null;default:''"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func (SYBSpecAIParseWorkItem) TableName() string { return "syb_spec_ai_parse_work_item" }
|
||||
@@ -32,10 +32,44 @@ type skuCombinationRow struct {
|
||||
}
|
||||
|
||||
func sybSpecsTrusted(syb models.SYBProduct) bool {
|
||||
if syb.ParseStatus == models.SYBParseStatusFailed {
|
||||
if strings.TrimSpace(syb.TargetColor) == "" && strings.TrimSpace(syb.TargetSize) == "" {
|
||||
return false
|
||||
}
|
||||
return strings.TrimSpace(syb.TargetColor) != "" || strings.TrimSpace(syb.TargetSize) != ""
|
||||
return syb.ParseStatus != models.SYBParseStatusFailed || syb.ManuallyConfirmed || syb.AIConfirmed
|
||||
}
|
||||
|
||||
func aiConfirmedSpecsCurrent(syb models.SYBProduct, shopee models.ShopeeProduct) bool {
|
||||
if !syb.AIConfirmed {
|
||||
return true
|
||||
}
|
||||
specs, err := shopeeproduct.Unmarshal(shopee.SpecsJSON)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
wanted := map[string]string{shopeeproduct.RoleColor: strings.TrimSpace(syb.TargetColor), shopeeproduct.RoleSize: strings.TrimSpace(syb.TargetSize)}
|
||||
foundAny := false
|
||||
for role, target := range wanted {
|
||||
if target == "" {
|
||||
continue
|
||||
}
|
||||
foundAny = true
|
||||
found := false
|
||||
for _, dimension := range specs {
|
||||
if dimension.Role != role {
|
||||
continue
|
||||
}
|
||||
for _, value := range dimension.Values {
|
||||
if value.Name == target {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return foundAny
|
||||
}
|
||||
|
||||
func (s *Service) loadLatestSKUCombinations(ctx context.Context, pddIDs []uint64, dataset *batchPreviewDataset) error {
|
||||
|
||||
@@ -388,6 +388,10 @@ func (s *Service) previewFromDataset(id uint64, dataset batchPreviewDataset, gua
|
||||
item.ReasonCode, item.Reason, item.NextAction = "SHOPEE_NOT_FOUND", "关联的蝦皮商品不存在,请先处理商品档案", "open_shopee"
|
||||
return item
|
||||
}
|
||||
if !aiConfirmedSpecsCurrent(syb, shopee) {
|
||||
item.ReasonCode, item.Reason, item.NextAction = "SYB_PARSE_FAILED", "AI 解析依据已变化,请等待重新解析或人工修正", "reparse"
|
||||
return item
|
||||
}
|
||||
if shopee.PDDProductID == nil {
|
||||
item.ReasonCode, item.Reason, item.NextAction = "PDD_NOT_LINKED", "尚未关联 PDD 商品,请先关联", processActionOpenPDDLink
|
||||
return item
|
||||
|
||||
@@ -54,6 +54,9 @@ func TestSybSpecsTrustedOnlyBlocksFailedOrEmptyExtraction(t *testing.T) {
|
||||
{"uncertain with color", models.SYBProduct{ParseStatus: models.SYBParseStatusUncertain, TargetColor: "套装"}, true},
|
||||
{"uncertain with size", models.SYBProduct{ParseStatus: models.SYBParseStatusUncertain, TargetSize: "均码"}, true},
|
||||
{"failed with values", models.SYBProduct{ParseStatus: models.SYBParseStatusFailed, TargetColor: "黑色"}, false},
|
||||
{"failed AI confirmed", models.SYBProduct{ParseStatus: models.SYBParseStatusFailed, TargetColor: "黑色", AIConfirmed: true}, true},
|
||||
{"failed manually confirmed", models.SYBProduct{ParseStatus: models.SYBParseStatusFailed, TargetColor: "黑色", ManuallyConfirmed: true}, true},
|
||||
{"AI confirmed without values", models.SYBProduct{ParseStatus: models.SYBParseStatusFailed, AIConfirmed: true}, false},
|
||||
{"uncertain without values", models.SYBProduct{ParseStatus: models.SYBParseStatusUncertain}, false},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
@@ -65,6 +68,28 @@ func TestSybSpecsTrustedOnlyBlocksFailedOrEmptyExtraction(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestSYBSpecsTrustedAcceptsAIWithoutCallingItManual(t *testing.T) {
|
||||
row := models.SYBProduct{ParseStatus: models.SYBParseStatusUncertain, TargetColor: "黑色", AIConfirmed: true}
|
||||
if !sybSpecsTrusted(row) {
|
||||
t.Fatal("a valid AI-confirmed parse must pass the purchase parse gate")
|
||||
}
|
||||
if row.ManuallyConfirmed {
|
||||
t.Fatal("AI confirmation must not be represented as human confirmation")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIConfirmedSpecsBecomeUntrustedWhenShopeeCandidateDisappears(t *testing.T) {
|
||||
row := models.SYBProduct{TargetColor: "黑色", TargetSize: "XL", ParseStatus: models.SYBParseStatusUncertain, AIConfirmed: true}
|
||||
product := models.ShopeeProduct{SpecsJSON: `[{"name":"颜色","role":"color","values":[{"name":"白色","source":"import"}]},{"name":"尺码","role":"size","values":[{"name":"XL","source":"import"}]}]`}
|
||||
if aiConfirmedSpecsCurrent(row, product) {
|
||||
t.Fatal("removed Shopee candidate must invalidate the AI parse gate")
|
||||
}
|
||||
product.SpecsJSON = `[{"name":"颜色","role":"color","values":[{"name":"黑色","source":"import"}]},{"name":"尺码","role":"size","values":[{"name":"XL","source":"import"}]}]`
|
||||
if !aiConfirmedSpecsCurrent(row, product) {
|
||||
t.Fatal("unchanged exact Shopee candidates should keep AI parse valid")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBatchPreviewExposesIndependentCollectionEligibility(t *testing.T) {
|
||||
db := testDB(t)
|
||||
fixture := seed(t, db, liveCaps(), true)
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"go-admin/common/clientprincipal"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
@@ -470,6 +471,9 @@ func writeAdminReplay(c *gin.Context, data any, replayed bool) {
|
||||
}
|
||||
|
||||
func allowedOperator(c *gin.Context) bool {
|
||||
if _, ok := clientprincipal.Get(c); ok {
|
||||
return true
|
||||
}
|
||||
role, _ := jwt.ExtractClaims(c)["rolekey"].(string)
|
||||
if role == "admin" || role == "purchaser" {
|
||||
return true
|
||||
@@ -544,6 +548,9 @@ func writeError(c *gin.Context, err error) {
|
||||
c.JSON(status, gin.H{"code": code, "message": msg, "retryable": retryable})
|
||||
}
|
||||
func operatorID(c *gin.Context) uint64 {
|
||||
if id, ok := clientprincipal.Get(c); ok {
|
||||
return id.AuthorizedBy
|
||||
}
|
||||
claims := jwt.ExtractClaims(c)
|
||||
switch v := claims["identity"].(type) {
|
||||
case float64:
|
||||
|
||||
@@ -136,7 +136,7 @@ func processStageFromDataset(id uint64, dataset batchPreviewDataset, preview Bat
|
||||
if !ok {
|
||||
return stage(ProcessStageManualAction, "SYB 商品不存在或已删除", "refresh")
|
||||
}
|
||||
if syb.ParseStatus == models.SYBParseStatusFailed {
|
||||
if syb.ParseStatus == models.SYBParseStatusFailed && !syb.ManuallyConfirmed && !syb.AIConfirmed {
|
||||
return stage(ProcessStageManualAction, "解析失败,请先处理", "reparse")
|
||||
}
|
||||
if syb.ShopeeProductID == nil {
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
package purchase
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"go-admin/app/goauto/shopeeproduct"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Called inside the link transaction, after locking the SYB and Shopee rows.
|
||||
// Reuse the authoritative stage calculation, without AI or task creation.
|
||||
func ValidateQuickReplacement(ctx context.Context, tx *gorm.DB, id uint64) error {
|
||||
s := NewService(tx)
|
||||
dataset, err := s.loadBatchPreviewDataset(ctx, []uint64{id})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
guard, _, err := s.readOnlyPriceGuard(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
preview := s.previewFromDataset(id, dataset, guard)
|
||||
qualification := aiMatchQualificationForDataset(id, dataset)
|
||||
preview.AIMatchEligible = qualification.Eligible
|
||||
preview.AIMatchDisabledReason = qualification.DisabledReason
|
||||
stage := processStageFromDataset(id, dataset, preview)
|
||||
syb := dataset.sybByID[id]
|
||||
disabled := false
|
||||
if syb.ShopeeProductID != nil {
|
||||
shopee := dataset.shopeeByID[*syb.ShopeeProductID]
|
||||
if shopee.PDDProductID != nil {
|
||||
pdd := dataset.pddByID[*shopee.PDDProductID]
|
||||
disabled = pdd.Status == "disabled"
|
||||
if _, active := dataset.activeCollectionByPDD[pdd.ID]; active {
|
||||
return &shopeeproduct.ServiceError{Code: shopeeproduct.CodeLinkConflict, Message: "商品正在采集,请结束后刷新再操作"}
|
||||
}
|
||||
}
|
||||
}
|
||||
if stage.NextAction == "open_pdd" && (stage.Stage == ProcessStagePDDPending || stage.Stage == ProcessStagePDDCollectionFail ||
|
||||
(stage.Stage == ProcessStageManualAction && (disabled || preview.ReasonCode == "PDD_PRICE_MISSING"))) {
|
||||
return nil
|
||||
}
|
||||
return &shopeeproduct.ServiceError{Code: shopeeproduct.CodeLinkConflict, Message: "处理阶段已变化,当前不能一键替换,请刷新列表"}
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
package purchase
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func TestQuickReplacementUsesCurrentStage(t *testing.T) {
|
||||
for _, name := range []string{"pending", "disabled", "price-missing", "ready", "parse-failed", "collecting", "purchase-pending", "purchase-running", "order-created", "order-unknown"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
db := testDB(t)
|
||||
f := seed(t, db, liveCaps(), true)
|
||||
allow := name == "pending" || name == "disabled" || name == "price-missing"
|
||||
switch name {
|
||||
case "pending", "disabled":
|
||||
db.Model(&f.pdd).Update("status", name)
|
||||
case "ready":
|
||||
setCollectedPDDPrice(t, db, f.pdd.ID)
|
||||
case "parse-failed":
|
||||
db.Model(&f.syb).Update("parse_status", "failed")
|
||||
case "collecting":
|
||||
rule := models.CollectionRule{Name: "synthetic", ContentJSON: "{}"}
|
||||
if err := db.Create(&rule).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Create(&models.CollectionTask{DeviceID: &f.device.ID, PDDProductID: &f.pdd.ID, RuleID: rule.ID, Status: models.TaskStatusRunning, RuleSnapshot: "{}"}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "purchase-pending", "purchase-running", "order-created", "order-unknown":
|
||||
status := map[string]string{"purchase-pending": "pending", "purchase-running": "running", "order-created": "order_created", "order-unknown": "order_result_unknown"}[name]
|
||||
if err := db.Session(&gorm.Session{SkipHooks: true}).Create(&models.PurchaseTask{ExecutionMode: "live", Quantity: 1, SYBProductID: &f.syb.ID, PDDProductID: f.pdd.ID, Status: status, CreateRequestID: name}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
err := ValidateQuickReplacement(context.Background(), db, f.syb.ID)
|
||||
if (err == nil) != allow {
|
||||
t.Fatalf("allowed=%v: %v", allow, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -131,6 +131,12 @@ func (service *Service) suggestMappings(ctx context.Context, id uint64, requestC
|
||||
item.Reason = "保留已确认映射"
|
||||
item.Apply = true
|
||||
response.MatchedCount++
|
||||
case role == RoleColor && deterministicColorMatch(value.Name, candidateNames) != "":
|
||||
item.PDDValue = deterministicColorMatch(value.Name, candidateNames)
|
||||
item.Status = "matched"
|
||||
item.Reason = "格式统一后唯一匹配"
|
||||
item.Apply = true
|
||||
response.MatchedCount++
|
||||
case role == RoleSize && deterministicSizeMatch(value.Name, candidateNames) != "":
|
||||
item.PDDValue = deterministicSizeMatch(value.Name, candidateNames)
|
||||
item.Status = "matched"
|
||||
@@ -237,6 +243,15 @@ func deterministicSizeMatch(value string, candidates []string) string {
|
||||
return match.MappedSize
|
||||
}
|
||||
|
||||
// Keep the original candidate label; normalization is only a comparison key.
|
||||
func deterministicColorMatch(value string, candidates []string) string {
|
||||
match, ok := aimatching.DeterministicMatch(aimatching.MatchRequest{TargetColor: value, Colors: candidates})
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
return match.MappedColor
|
||||
}
|
||||
|
||||
// aiSuggestErrorMessage turns an aimatching error into a message safe to
|
||||
// return to the admin UI (no credential, no raw provider body).
|
||||
func aiSuggestErrorMessage(err error) string {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"go-admin/app/goauto/aimatching"
|
||||
"go-admin/app/goauto/models"
|
||||
@@ -48,6 +49,22 @@ type autoMatchRef struct {
|
||||
// before the transaction; the transaction rechecks the complete spec context
|
||||
// and PDD candidate set so a stale decision can never be written.
|
||||
func (service *Service) AutoMatchMappings(ctx context.Context, id uint64, request AutoMatchRequest) (AutoMatchResponse, error) {
|
||||
started := time.Now()
|
||||
// Bound the initial settings lookup too. This budget is specific to this
|
||||
// composite endpoint; other AI operations retain their existing limits.
|
||||
ctx, cancel := context.WithTimeout(ctx, 1210*time.Second)
|
||||
defer cancel()
|
||||
seconds, err := service.autoMatchTimeout(ctx)
|
||||
if err != nil {
|
||||
return AutoMatchResponse{}, err
|
||||
}
|
||||
return service.autoMatchMappings(ctx, id, request, time.Duration(seconds)*time.Second-time.Since(started))
|
||||
}
|
||||
|
||||
func (service *Service) autoMatchMappings(ctx context.Context, id uint64, request AutoMatchRequest, budget time.Duration) (AutoMatchResponse, error) {
|
||||
ctx, cancel := context.WithTimeout(ctx, budget)
|
||||
defer cancel()
|
||||
|
||||
requestID := strings.TrimSpace(request.RequestID)
|
||||
if _, err := uuid.Parse(requestID); err != nil {
|
||||
return AutoMatchResponse{}, invalidRequest("requestId 必须是 UUID")
|
||||
@@ -245,6 +262,21 @@ func (service *Service) AutoMatchMappings(ctx context.Context, id uint64, reques
|
||||
return response, nil
|
||||
}
|
||||
|
||||
func autoMatchBudgetSeconds(providerSeconds int) int {
|
||||
if providerSeconds < 3 || providerSeconds > 600 {
|
||||
providerSeconds = 15 // same default as AI settings; never allow unbounded waits
|
||||
}
|
||||
return providerSeconds*2 + 10
|
||||
}
|
||||
|
||||
func (service *Service) autoMatchTimeout(ctx context.Context) (int, error) {
|
||||
settings, err := aimatching.NewService(service.DB).Settings(ctx)
|
||||
if err != nil {
|
||||
return 0, internalError(err)
|
||||
}
|
||||
return autoMatchBudgetSeconds(settings.TimeoutSeconds), nil
|
||||
}
|
||||
|
||||
func autoMatchRefs(specs []SpecDimension, role string) []autoMatchRef {
|
||||
refs := make([]autoMatchRef, 0)
|
||||
for _, dimension := range specs {
|
||||
|
||||
@@ -0,0 +1,361 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"go-admin/app/goauto/aimatching"
|
||||
"go-admin/app/goauto/models"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
const (
|
||||
SpecAutoMatchInvokeTarget = "GoAutoShopeeSpecAutoMatch"
|
||||
defaultAutoMatchBatchLimit = 20
|
||||
autoMatchLeaseDuration = 30 * time.Minute
|
||||
autoMatchRetryDelay = time.Hour
|
||||
maxAutoMatchAttempts = 3
|
||||
)
|
||||
|
||||
type AutoMatchRunView struct {
|
||||
models.ShopeeSpecAutoMatchRun
|
||||
AlreadyRunning bool `json:"alreadyRunning,omitempty"`
|
||||
Replayed bool `json:"replayed,omitempty"`
|
||||
}
|
||||
|
||||
// StartAutoMatchRun acquires the single database-backed activity slot. A
|
||||
// repeated requestId is idempotent; a concurrent trigger receives the current
|
||||
// run instead of starting a second batch.
|
||||
func (service *Service) StartAutoMatchRun(ctx context.Context, trigger, requestID string, requestedBy *uint64, batchLimit int) (AutoMatchRunView, bool, error) {
|
||||
if _, err := uuid.Parse(strings.TrimSpace(requestID)); err != nil {
|
||||
return AutoMatchRunView{}, false, invalidRequest("requestId 必须是 UUID")
|
||||
}
|
||||
if trigger != "manual" && trigger != "scheduled" {
|
||||
return AutoMatchRunView{}, false, invalidRequest("trigger 无效")
|
||||
}
|
||||
if batchLimit <= 0 {
|
||||
batchLimit = defaultAutoMatchBatchLimit
|
||||
}
|
||||
if batchLimit > 100 {
|
||||
return AutoMatchRunView{}, false, invalidRequest("batchLimit 不能超过 100")
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
lease := now.Add(autoMatchLeaseDuration)
|
||||
owner := uuid.NewString()
|
||||
one := uint8(1)
|
||||
var result models.ShopeeSpecAutoMatchRun
|
||||
created := false
|
||||
err := service.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
if err := tx.Model(&models.ShopeeSpecAutoMatchRun{}).
|
||||
Where("status = ? AND active_slot = ? AND lease_expires_at < ?", "running", 1, now).
|
||||
Updates(map[string]any{"status": "failed", "active_slot": nil, "lease_owner": "", "lease_expires_at": nil, "error_summary": "上次运行租约过期,已安全释放", "finished_at": now}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if err := tx.Where("request_id = ?", requestID).First(&result).Error; err == nil {
|
||||
return nil
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return err
|
||||
}
|
||||
if err := tx.Where("status = ? AND active_slot = ?", "running", 1).First(&result).Error; err == nil {
|
||||
result.ActiveSlot = &one
|
||||
return nil
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return err
|
||||
}
|
||||
result = models.ShopeeSpecAutoMatchRun{RequestID: requestID, Trigger: trigger, Status: "running", ActiveSlot: &one, LeaseOwner: owner, LeaseExpiresAt: &lease, RequestedBy: requestedBy, BatchLimit: batchLimit, StartedAt: now}
|
||||
if err := tx.Create(&result).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
created = true
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
// A unique-slot race means another instance won after our read. Return
|
||||
// its run as the stable, non-error result.
|
||||
if findErr := service.DB.WithContext(ctx).Where("status = ? AND active_slot = ?", "running", 1).First(&result).Error; findErr == nil {
|
||||
return AutoMatchRunView{ShopeeSpecAutoMatchRun: result, AlreadyRunning: true}, false, nil
|
||||
}
|
||||
return AutoMatchRunView{}, false, internalError(err)
|
||||
}
|
||||
view := AutoMatchRunView{ShopeeSpecAutoMatchRun: result}
|
||||
if !created {
|
||||
view.AlreadyRunning = result.RequestID != requestID
|
||||
view.Replayed = result.RequestID == requestID
|
||||
}
|
||||
return view, created, nil
|
||||
}
|
||||
|
||||
func (service *Service) LatestAutoMatchRun(ctx context.Context) (*AutoMatchRunView, error) {
|
||||
var run models.ShopeeSpecAutoMatchRun
|
||||
err := service.DB.WithContext(ctx).Order("id DESC").First(&run).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, internalError(err)
|
||||
}
|
||||
return &AutoMatchRunView{ShopeeSpecAutoMatchRun: run}, nil
|
||||
}
|
||||
|
||||
// ProcessAutoMatchRun performs a bounded batch. It is safe to call from an
|
||||
// HTTP-launched goroutine or the scheduler because only the run owning the
|
||||
// active slot may update and finish itself.
|
||||
func (service *Service) ProcessAutoMatchRun(ctx context.Context, runID uint64) error {
|
||||
var run models.ShopeeSpecAutoMatchRun
|
||||
if err := service.DB.WithContext(ctx).First(&run, runID).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if run.Status != "running" || run.ActiveSlot == nil || *run.ActiveSlot != 1 {
|
||||
return nil
|
||||
}
|
||||
limit := run.BatchLimit
|
||||
if limit <= 0 || limit > 100 {
|
||||
limit = defaultAutoMatchBatchLimit
|
||||
}
|
||||
var candidates []models.ShopeeProduct
|
||||
queryLimit := limit * 25
|
||||
if queryLimit < 100 {
|
||||
queryLimit = 100
|
||||
}
|
||||
if queryLimit > 1000 {
|
||||
queryLimit = 1000
|
||||
}
|
||||
if err := service.DB.WithContext(ctx).
|
||||
Joins("JOIN pdd_product ON pdd_product.id = shopee_product.pdd_product_id AND pdd_product.status = ?", "active").
|
||||
Where("shopee_product.pdd_product_id IS NOT NULL").
|
||||
Order("shopee_product.updated_at ASC, shopee_product.id ASC").Limit(queryLimit).Find(&candidates).Error; err != nil {
|
||||
service.finishAutoMatchRun(run, "failed", 0, 0, 0, 0, 0, 1, "扫描符合条件的商品失败")
|
||||
return err
|
||||
}
|
||||
|
||||
eligible, processed, confirmed, unmatched, failed := 0, 0, 0, 0, 0
|
||||
firstError := ""
|
||||
for _, product := range candidates {
|
||||
if processed >= limit {
|
||||
break
|
||||
}
|
||||
fingerprint, ok, err := service.autoMatchEligibility(ctx, product)
|
||||
if err != nil {
|
||||
failed++
|
||||
if firstError == "" {
|
||||
firstError = safeBatchError(err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
eligible++
|
||||
work, claimed, err := service.claimAutoMatchWork(ctx, run, product.ID, fingerprint)
|
||||
if err != nil {
|
||||
failed++
|
||||
if firstError == "" {
|
||||
firstError = safeBatchError(err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !claimed {
|
||||
continue
|
||||
}
|
||||
processed++
|
||||
service.renewAutoMatchRun(run)
|
||||
response, matchErr := service.autoMatchMappings(ctx, product.ID, AutoMatchRequest{RequestID: uuid.NewString(), SpecContextVersion: fingerprint[:64]}, aimatching.MaxProviderTimeout)
|
||||
// fingerprint begins with the 64-character context version.
|
||||
postFingerprint := fingerprint
|
||||
if next, _, nextErr := service.autoMatchEligibility(ctx, product); nextErr == nil && next != "" {
|
||||
postFingerprint = next
|
||||
}
|
||||
if matchErr != nil {
|
||||
failed++
|
||||
if firstError == "" {
|
||||
firstError = safeBatchError(matchErr)
|
||||
}
|
||||
service.completeAutoMatchWork(work, postFingerprint, 0, 0, matchErr)
|
||||
continue
|
||||
}
|
||||
confirmed += response.ConfirmedCount
|
||||
unmatched += response.UnmatchedCount
|
||||
service.completeAutoMatchWork(work, postFingerprint, response.ConfirmedCount, response.UnmatchedCount, nil)
|
||||
}
|
||||
status := "completed"
|
||||
if failed > 0 {
|
||||
status = "completed_partial"
|
||||
}
|
||||
return service.finishAutoMatchRun(run, status, len(candidates), eligible, processed, confirmed, unmatched, failed, firstError)
|
||||
}
|
||||
|
||||
func (service *Service) autoMatchEligibility(ctx context.Context, product models.ShopeeProduct) (string, bool, error) {
|
||||
if product.PDDProductID == nil {
|
||||
return "", false, nil
|
||||
}
|
||||
var pdd models.PDDProduct
|
||||
if err := service.DB.WithContext(ctx).First(&pdd, *product.PDDProductID).Error; err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
if pdd.Status != "active" {
|
||||
return "", false, nil
|
||||
}
|
||||
shopeeSpecs, err := Unmarshal(product.SpecsJSON)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
shared, needsMatch := false, false
|
||||
for _, role := range []string{RoleColor, RoleSize} {
|
||||
pddValues, err := selectablePDDValues(pdd.SpecsJSON, role)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
if len(pddValues) == 0 {
|
||||
continue
|
||||
}
|
||||
for _, dimension := range shopeeSpecs {
|
||||
if dimension.Role != role || len(dimension.Values) == 0 {
|
||||
continue
|
||||
}
|
||||
shared = true
|
||||
for _, value := range dimension.Values {
|
||||
if value.Mapping == nil || value.Mapping.Status != MappingStatusConfirmed || !pddValues[value.Mapping.PDDValue] {
|
||||
needsMatch = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if !shared || !needsMatch {
|
||||
return "", false, nil
|
||||
}
|
||||
contextVersion := computeSpecContextVersion(product.PDDProductID, product.SpecsJSON, pdd.SpecsJSON)
|
||||
var setting struct{ UpdatedAt time.Time }
|
||||
_ = service.DB.WithContext(ctx).Table((models.AIMatchingSetting{}).TableName()).Select("updated_at").Where("id = ?", 1).Scan(&setting).Error
|
||||
h := sha256.Sum256([]byte(contextVersion + "\x00" + setting.UpdatedAt.UTC().Format(time.RFC3339Nano)))
|
||||
// Keeping the context version as a prefix lets ProcessAutoMatchRun pass the
|
||||
// exact version to #194 without re-reading a potentially drifting input.
|
||||
return contextVersion + hex.EncodeToString(h[:]), true, nil
|
||||
}
|
||||
|
||||
func (service *Service) claimAutoMatchWork(ctx context.Context, run models.ShopeeSpecAutoMatchRun, productID uint64, fingerprint string) (models.ShopeeSpecAutoMatchWorkItem, bool, error) {
|
||||
now := time.Now().UTC()
|
||||
lease := now.Add(autoMatchLeaseDuration)
|
||||
var work models.ShopeeSpecAutoMatchWorkItem
|
||||
err := service.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
err := tx.Where("shopee_product_id = ?", productID).First(&work).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
work = models.ShopeeSpecAutoMatchWorkItem{ShopeeProductID: productID, RunID: &run.ID, InputFingerprint: fingerprint, Status: "running", AttemptCount: 1, LeaseOwner: run.LeaseOwner, LeaseExpiresAt: &lease}
|
||||
return tx.Create(&work).Error
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if work.InputFingerprint == fingerprint {
|
||||
if work.Status == "completed" || work.Status == "unmatched" || work.AttemptCount >= maxAutoMatchAttempts || (work.NextAttemptAt != nil && work.NextAttemptAt.After(now)) || (work.Status == "running" && work.LeaseExpiresAt != nil && work.LeaseExpiresAt.After(now)) {
|
||||
return errWorkNotClaimed
|
||||
}
|
||||
} else {
|
||||
work.AttemptCount = 0
|
||||
}
|
||||
updates := map[string]any{"run_id": run.ID, "input_fingerprint": fingerprint, "status": "running", "attempt_count": work.AttemptCount + 1, "next_attempt_at": nil, "lease_owner": run.LeaseOwner, "lease_expires_at": lease, "last_error_code": "", "last_error": ""}
|
||||
if err := tx.Model(&models.ShopeeSpecAutoMatchWorkItem{}).Where("id = ?", work.ID).Updates(updates).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.First(&work, work.ID).Error
|
||||
})
|
||||
if errors.Is(err, errWorkNotClaimed) {
|
||||
return work, false, nil
|
||||
}
|
||||
return work, err == nil, err
|
||||
}
|
||||
|
||||
var errWorkNotClaimed = errors.New("auto match work not claimed")
|
||||
|
||||
func (service *Service) completeAutoMatchWork(work models.ShopeeSpecAutoMatchWorkItem, fingerprint string, confirmed, unmatched int, matchErr error) {
|
||||
now := time.Now().UTC()
|
||||
updates := map[string]any{"input_fingerprint": fingerprint, "lease_owner": "", "lease_expires_at": nil, "confirmed_count": confirmed, "unmatched_count": unmatched}
|
||||
if matchErr == nil {
|
||||
if unmatched > 0 {
|
||||
updates["status"] = "unmatched"
|
||||
} else {
|
||||
updates["status"] = "completed"
|
||||
}
|
||||
updates["next_attempt_at"], updates["last_error_code"], updates["last_error"] = nil, "", ""
|
||||
} else {
|
||||
code := batchErrorCode(matchErr)
|
||||
updates["status"], updates["last_error_code"], updates["last_error"] = "failed", code, safeBatchError(matchErr)
|
||||
if code == CodeAIUnavailable && work.AttemptCount < maxAutoMatchAttempts {
|
||||
next := now.Add(autoMatchRetryDelay)
|
||||
updates["next_attempt_at"] = next
|
||||
} else {
|
||||
updates["next_attempt_at"] = nil
|
||||
}
|
||||
}
|
||||
_ = service.DB.Model(&models.ShopeeSpecAutoMatchWorkItem{}).Where("id = ?", work.ID).Updates(updates).Error
|
||||
}
|
||||
|
||||
func (service *Service) renewAutoMatchRun(run models.ShopeeSpecAutoMatchRun) {
|
||||
lease := time.Now().UTC().Add(autoMatchLeaseDuration)
|
||||
_ = service.DB.Model(&models.ShopeeSpecAutoMatchRun{}).Where("id = ? AND status = ? AND lease_owner = ?", run.ID, "running", run.LeaseOwner).Update("lease_expires_at", lease).Error
|
||||
}
|
||||
|
||||
func (service *Service) finishAutoMatchRun(run models.ShopeeSpecAutoMatchRun, status string, scanned, eligible, processed, confirmed, unmatched, failed int, summary string) error {
|
||||
now := time.Now().UTC()
|
||||
updates := map[string]any{"status": status, "active_slot": nil, "lease_owner": "", "lease_expires_at": nil, "scanned_count": scanned, "eligible_count": eligible, "processed_count": processed, "confirmed_count": confirmed, "unmatched_count": unmatched, "failed_count": failed, "error_summary": truncateBatchText(summary), "finished_at": now}
|
||||
return service.DB.Model(&models.ShopeeSpecAutoMatchRun{}).Where("id = ? AND status = ? AND lease_owner = ?", run.ID, "running", run.LeaseOwner).Updates(updates).Error
|
||||
}
|
||||
|
||||
func batchErrorCode(err error) string {
|
||||
var serviceErr *ServiceError
|
||||
if errors.As(err, &serviceErr) {
|
||||
return serviceErr.Code
|
||||
}
|
||||
return CodeInternal
|
||||
}
|
||||
|
||||
func safeBatchError(err error) string {
|
||||
var serviceErr *ServiceError
|
||||
if errors.As(err, &serviceErr) {
|
||||
return truncateBatchText(serviceErr.Message)
|
||||
}
|
||||
return "服务端处理失败"
|
||||
}
|
||||
|
||||
func truncateBatchText(value string) string {
|
||||
runes := []rune(strings.TrimSpace(value))
|
||||
if len(runes) > 500 {
|
||||
runes = runes[:500]
|
||||
}
|
||||
return string(runes)
|
||||
}
|
||||
|
||||
type scheduledAutoMatchArgs struct {
|
||||
BatchLimit int `json:"batchLimit"`
|
||||
}
|
||||
|
||||
type ScheduledAutoMatchJob struct{}
|
||||
|
||||
func (ScheduledAutoMatchJob) Exec(_ interface{}) error {
|
||||
return errors.New("规格自动匹配定时任务缺少数据库连接")
|
||||
}
|
||||
|
||||
func (ScheduledAutoMatchJob) ExecWithDB(db *gorm.DB, arg interface{}) error {
|
||||
args := scheduledAutoMatchArgs{BatchLimit: defaultAutoMatchBatchLimit}
|
||||
if raw, ok := arg.(string); ok && strings.TrimSpace(raw) != "" {
|
||||
if err := json.Unmarshal([]byte(raw), &args); err != nil {
|
||||
return fmt.Errorf("规格自动匹配参数不是合法 JSON: %w", err)
|
||||
}
|
||||
}
|
||||
if args.BatchLimit < 1 || args.BatchLimit > 100 {
|
||||
return errors.New("规格自动匹配 batchLimit 必须在 1 到 100 之间")
|
||||
}
|
||||
service := NewService(db)
|
||||
run, created, err := service.StartAutoMatchRun(context.Background(), "scheduled", uuid.NewString(), nil, args.BatchLimit)
|
||||
if err != nil || !created {
|
||||
return err
|
||||
}
|
||||
return service.ProcessAutoMatchRun(context.Background(), run.ID)
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
func TestAutoMatchRunIsIdempotentAndGloballySerialized(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
service := NewService(db)
|
||||
requestID := uuid.NewString()
|
||||
first, created, err := service.StartAutoMatchRun(context.Background(), "manual", requestID, nil, 20)
|
||||
if err != nil || !created {
|
||||
t.Fatalf("first=%+v created=%v err=%v", first, created, err)
|
||||
}
|
||||
replay, created, err := service.StartAutoMatchRun(context.Background(), "manual", requestID, nil, 20)
|
||||
if err != nil || created || !replay.Replayed || replay.ID != first.ID {
|
||||
t.Fatalf("replay=%+v created=%v err=%v", replay, created, err)
|
||||
}
|
||||
concurrent, created, err := service.StartAutoMatchRun(context.Background(), "scheduled", uuid.NewString(), nil, 20)
|
||||
if err != nil || created || !concurrent.AlreadyRunning || concurrent.ID != first.ID {
|
||||
t.Fatalf("concurrent=%+v created=%v err=%v", concurrent, created, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProcessAutoMatchRunConfirmsExactSizeAndFinishes(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
pdd := seedPDDProduct(t, db, "active")
|
||||
service := NewService(db)
|
||||
createdProduct, err := service.Create(context.Background(), CreateRequest{
|
||||
RequestID: uuid.NewString(), ShopeeItemID: "SP-BATCH-EXACT", PDDProductID: &pdd.ID,
|
||||
Specs: []SpecDimension{{Name: "尺码", Role: RoleSize, Values: []SpecValue{{Name: " xl ", Source: ValueSourceImport}}}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
run, started, err := service.StartAutoMatchRun(context.Background(), "manual", uuid.NewString(), nil, 20)
|
||||
if err != nil || !started {
|
||||
t.Fatalf("run=%+v started=%v err=%v", run, started, err)
|
||||
}
|
||||
if err := service.ProcessAutoMatchRun(context.Background(), run.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
latest, err := service.LatestAutoMatchRun(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if latest == nil || latest.Status != "completed" || latest.ProcessedCount != 1 || latest.ConfirmedCount != 1 || latest.ActiveSlot != nil {
|
||||
t.Fatalf("latest=%+v", latest)
|
||||
}
|
||||
detail, err := service.Detail(context.Background(), createdProduct.Product.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
mapping := detail.Product.Specs[0].Values[0].Mapping
|
||||
if mapping == nil || mapping.Status != MappingStatusConfirmed || mapping.PDDValue != "XL" {
|
||||
t.Fatalf("mapping=%+v", mapping)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnchangedUnmatchedWorkIsNotClaimedAgain(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
service := NewService(db)
|
||||
one := uint8(1)
|
||||
run := models.ShopeeSpecAutoMatchRun{RequestID: uuid.NewString(), Trigger: "manual", Status: "running", ActiveSlot: &one, LeaseOwner: uuid.NewString(), BatchLimit: 20}
|
||||
if err := db.Create(&run).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
work, claimed, err := service.claimAutoMatchWork(context.Background(), run, 99, "fingerprint")
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("work=%+v claimed=%v err=%v", work, claimed, err)
|
||||
}
|
||||
service.completeAutoMatchWork(work, "fingerprint", 0, 1, nil)
|
||||
_, claimed, err = service.claimAutoMatchWork(context.Background(), run, 99, "fingerprint")
|
||||
if err != nil || claimed {
|
||||
t.Fatalf("unchanged unmatched claimed=%v err=%v", claimed, err)
|
||||
}
|
||||
_, claimed, err = service.claimAutoMatchWork(context.Background(), run, 99, "changed")
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("changed input claimed=%v err=%v", claimed, err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
func TestAutoMatchBudgetFollowsProviderSetting(t *testing.T) {
|
||||
for _, tc := range []struct{ ai, total int }{{3, 16}, {60, 130}, {180, 370}, {600, 1210}, {0, 40}, {601, 40}} {
|
||||
if got := autoMatchBudgetSeconds(tc.ai); got != tc.total {
|
||||
t.Fatalf("AI %d: got %d want %d", tc.ai, got, tc.total)
|
||||
}
|
||||
}
|
||||
db := openTestDB(t)
|
||||
setting := seedEnabledAISetting(t, "https://example.invalid", 0.9)
|
||||
setting.TimeoutSeconds = 60
|
||||
if err := db.Create(&setting).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := NewService(db)
|
||||
created, err := s.Create(context.Background(), CreateRequest{RequestID: uuid.NewString(), ShopeeItemID: "budget-test"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
detail, err := s.DetailWithAutoMatchBudget(context.Background(), created.Product.ID)
|
||||
if err != nil || detail.AutoMatchTimeoutSeconds != 130 {
|
||||
t.Fatalf("budget detail: %d %v", detail.AutoMatchTimeoutSeconds, err)
|
||||
}
|
||||
body, _ := json.Marshal(detail)
|
||||
if strings.Contains(string(body), "apiKey") || strings.Contains(string(body), "example.invalid") {
|
||||
t.Fatal("AI configuration leaked")
|
||||
}
|
||||
if err := db.Model(&setting).Update("timeout_seconds", 180).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
detail, err = s.DetailWithAutoMatchBudget(context.Background(), created.Product.ID)
|
||||
if err != nil || detail.AutoMatchTimeoutSeconds != 370 {
|
||||
t.Fatalf("fresh budget: %d %v", detail.AutoMatchTimeoutSeconds, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAutoMatchCompositeBudgetCancelsWithoutSaving(t *testing.T) {
|
||||
var calls atomic.Int32
|
||||
responder := chatCompletionResponder(`{"suggestions":[{"sourceId":"s1","candidateId":"c1","confidence":0.96,"reason":"unique match"}]}`)
|
||||
provider := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if calls.Add(1) == 1 {
|
||||
responder(w, r)
|
||||
return
|
||||
}
|
||||
select {
|
||||
case <-r.Context().Done():
|
||||
case <-time.After(time.Second):
|
||||
}
|
||||
}))
|
||||
defer provider.Close()
|
||||
db := openTestDB(t)
|
||||
pdd := seedPDDProduct(t, db, "active")
|
||||
setting := seedEnabledAISetting(t, provider.URL, 0.9)
|
||||
if err := db.Create(&setting).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := NewService(db)
|
||||
created, err := s.Create(context.Background(), CreateRequest{RequestID: uuid.NewString(), ShopeeItemID: "budget-cancel", PDDProductID: &pdd.ID,
|
||||
Specs: []SpecDimension{{Name: "颜色", Role: RoleColor, Values: []SpecValue{{Name: "深黑", Source: ValueSourceImport}}}, {Name: "尺码", Role: RoleSize, Values: []SpecValue{{Name: "加大码", Source: ValueSourceImport}}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
started := time.Now()
|
||||
_, err = s.autoMatchMappings(context.Background(), created.Product.ID, AutoMatchRequest{RequestID: uuid.NewString(), SpecContextVersion: created.Product.SpecContextVersion}, 100*time.Millisecond)
|
||||
if err == nil || time.Since(started) > time.Second {
|
||||
t.Fatalf("budget not enforced: %v", err)
|
||||
}
|
||||
if calls.Load() != 2 {
|
||||
t.Fatalf("expected sequential color and size calls, got %d", calls.Load())
|
||||
}
|
||||
detail, err := s.Detail(context.Background(), created.Product.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if detail.Product.Specs[0].Values[0].Mapping != nil {
|
||||
t.Fatal("timed-out operation saved mappings")
|
||||
}
|
||||
}
|
||||
@@ -5,11 +5,13 @@ import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
@@ -116,6 +118,20 @@ func TestAutoMatchMappingsProviderFailureDoesNotWrite(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteErrorMapsAIUnavailableToStructured503(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
recorder := httptest.NewRecorder()
|
||||
context, _ := gin.CreateTestContext(recorder)
|
||||
writeError(context, aiUnavailable("AI 匹配服务暂时不可用,请稍后重试"))
|
||||
|
||||
if recorder.Code != http.StatusServiceUnavailable {
|
||||
t.Fatalf("status = %d, body = %s", recorder.Code, recorder.Body.String())
|
||||
}
|
||||
if !strings.Contains(recorder.Body.String(), `"code":"AI_MATCHING_UNAVAILABLE"`) {
|
||||
t.Fatalf("response is not structured: %s", recorder.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestAutoMatchMappingsRejectsContextDriftBeforeAtomicWrite(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
pdd := seedPDDProduct(t, db, "active")
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"go-admin/app/goauto/models"
|
||||
)
|
||||
|
||||
func TestColorNormalizedMatchWithoutAIKeepsOriginalLabels(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
pdd := seedPDDProduct(t, db, "active")
|
||||
pdd.SpecsJSON = `[{"name":"颜色","role":"color","values":[{"name":"260#大红","selectable":true},{"name":"260#墨绿","selectable":true},{"name":"AbC黑色","selectable":true}]}]`
|
||||
if err := db.Save(&pdd).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
service := NewService(db)
|
||||
sources := []string{"260#大紅", "260#墨綠", "abc 黑色"}
|
||||
targets := []string{"260#大红", "260#墨绿", "AbC黑色"}
|
||||
values := []SpecValue{}
|
||||
for _, name := range sources {
|
||||
values = append(values, SpecValue{Name: name, Source: ValueSourceImport})
|
||||
}
|
||||
created, err := service.Create(context.Background(), CreateRequest{RequestID: uuid.NewString(), ShopeeItemID: "normalized-color", PDDProductID: &pdd.ID, Specs: []SpecDimension{{Name: "颜色", Role: RoleColor, Values: values}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
preview, err := service.SuggestColorMappings(context.Background(), created.Product.ID, created.Product.SpecContextVersion)
|
||||
if err != nil {
|
||||
t.Fatalf("exact colors must not require AI: %v", err)
|
||||
}
|
||||
for i, item := range preview.Items {
|
||||
if item.Status != "matched" || !item.Apply || item.ValueName != sources[i] || item.PDDValue != targets[i] {
|
||||
t.Fatalf("unexpected preview: %+v", item)
|
||||
}
|
||||
}
|
||||
if len(preview.Items) != 3 || preview.MatchedCount != 3 {
|
||||
t.Fatal("missing exact matches")
|
||||
}
|
||||
var stored models.ShopeeProduct
|
||||
if err := db.First(&stored, created.Product.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
specs, err := Unmarshal(stored.SpecsJSON)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, value := range specs[0].Values {
|
||||
if value.Mapping != nil {
|
||||
t.Fatal("preview wrote mappings")
|
||||
}
|
||||
}
|
||||
result, err := service.AutoMatchMappings(context.Background(), created.Product.ID, AutoMatchRequest{RequestID: uuid.NewString(), SpecContextVersion: created.Product.SpecContextVersion})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if result.ConfirmedCount != 3 || result.UnmatchedCount != 0 {
|
||||
t.Fatalf("unexpected counts: %+v", result)
|
||||
}
|
||||
for i, value := range result.Product.Specs[0].Values {
|
||||
m := value.Mapping
|
||||
if value.Name != sources[i] || m == nil || m.PDDValue != targets[i] || m.Source != MappingSourceExactMatch || m.Status != MappingStatusConfirmed {
|
||||
t.Fatalf("incorrect persisted mapping: %+v", value)
|
||||
}
|
||||
}
|
||||
var after models.PDDProduct
|
||||
if err := db.First(&after, pdd.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after.SpecsJSON != pdd.SpecsJSON {
|
||||
t.Fatal("changed original PDD specs")
|
||||
}
|
||||
}
|
||||
|
||||
func TestColorNormalizedAmbiguityAndUnavailableStillRequireAI(t *testing.T) {
|
||||
for _, tc := range []struct{ name, source, specs string }{
|
||||
{"ambiguous", "大紅", `[{"name":"颜色","role":"color","values":[{"name":"大红","selectable":true},{"name":"大紅","selectable":true}]}]`},
|
||||
{"unavailable", "大紅", `[{"name":"颜色","role":"color","values":[{"name":"大红","selectable":false},{"name":"黑色","selectable":true}]}]`},
|
||||
{"unmatched", "深紅", `[{"name":"颜色","role":"color","values":[{"name":"大红","selectable":true}]}]`},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
pdd := seedPDDProduct(t, db, "active")
|
||||
pdd.SpecsJSON = tc.specs
|
||||
if err := db.Save(&pdd).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
service := NewService(db)
|
||||
created, err := service.Create(context.Background(), CreateRequest{RequestID: uuid.NewString(), ShopeeItemID: tc.name, PDDProductID: &pdd.ID, Specs: []SpecDimension{{Name: "颜色", Role: RoleColor, Values: []SpecValue{{Name: tc.source, Source: ValueSourceImport}}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, err = service.SuggestColorMappings(context.Background(), created.Product.ID, created.Product.SpecContextVersion)
|
||||
if err == nil || errCode(t, err) != CodeAIUnavailable {
|
||||
t.Fatalf("must fall back to AI, not guess: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
@@ -13,7 +14,10 @@ import (
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type Handler struct{ DB *gorm.DB }
|
||||
type Handler struct {
|
||||
DB *gorm.DB
|
||||
ReplacementEligibility ReplacementEligibility
|
||||
}
|
||||
|
||||
func (handler Handler) List(c *gin.Context) {
|
||||
page, err := queryInt(c.Query("page"), 1)
|
||||
@@ -63,7 +67,7 @@ func (handler Handler) Detail(c *gin.Context) {
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
response, err := service.Detail(c.Request.Context(), id)
|
||||
response, err := service.DetailWithAutoMatchBudget(c.Request.Context(), id)
|
||||
respond(c, response, err)
|
||||
}
|
||||
|
||||
@@ -95,10 +99,17 @@ func (handler Handler) LinkPDD(c *gin.Context) {
|
||||
writeError(c, invalidRequest("请求 JSON 无效"))
|
||||
return
|
||||
}
|
||||
// This composite workflow reads SYB and device context. It must not inherit
|
||||
// a client key's Shopee-only write grant through the shared handler.
|
||||
if request.LatestCollection != nil && !strings.HasPrefix(c.FullPath(), "/api/admin/v1/") {
|
||||
c.JSON(http.StatusForbidden, gin.H{"code": "FORBIDDEN", "message": "一键关联仅限 Admin 入口"})
|
||||
return
|
||||
}
|
||||
service, ok := handler.service(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
service.ReplacementEligibility = handler.ReplacementEligibility
|
||||
response, err := service.LinkPDD(c.Request.Context(), id, request)
|
||||
respond(c, response, err)
|
||||
}
|
||||
@@ -300,6 +311,45 @@ func (handler Handler) AutoMatchMappings(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, gin.H{"code": 200, "data": response})
|
||||
}
|
||||
|
||||
func (handler Handler) StartAutoMatchRun(c *gin.Context) {
|
||||
var request struct {
|
||||
RequestID string `json:"requestId"`
|
||||
}
|
||||
if err := decodeJSON(c, &request); err != nil {
|
||||
writeError(c, invalidRequest("请求 JSON 无效"))
|
||||
return
|
||||
}
|
||||
service, ok := handler.service(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
operator := currentUserID(c)
|
||||
run, created, err := service.StartAutoMatchRun(c.Request.Context(), "manual", request.RequestID, &operator, defaultAutoMatchBatchLimit)
|
||||
if err != nil {
|
||||
writeError(c, err)
|
||||
return
|
||||
}
|
||||
if created {
|
||||
go func(runID uint64, db *gorm.DB) {
|
||||
_ = NewService(db).ProcessAutoMatchRun(context.Background(), runID)
|
||||
}(run.ID, service.DB)
|
||||
}
|
||||
c.JSON(http.StatusAccepted, gin.H{"code": 200, "data": gin.H{"run": run}})
|
||||
}
|
||||
|
||||
func (handler Handler) LatestAutoMatchRun(c *gin.Context) {
|
||||
service, ok := handler.service(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
run, err := service.LatestAutoMatchRun(c.Request.Context())
|
||||
if err != nil {
|
||||
writeError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"code": 200, "data": gin.H{"run": run}})
|
||||
}
|
||||
|
||||
func (handler Handler) BatchDelete(c *gin.Context) {
|
||||
var request BatchDeleteRequest
|
||||
if err := decodeJSON(c, &request); err != nil {
|
||||
@@ -427,7 +477,7 @@ func writeError(c *gin.Context, err error) {
|
||||
status = http.StatusConflict
|
||||
case CodeProductNotFound, CodePDDProductNotFound, CodeMappingNotFound, CodeValueNotFound:
|
||||
status = http.StatusNotFound
|
||||
case CodePDDProductDisabled, CodeSpecContextStale:
|
||||
case CodePDDProductDisabled, CodeSpecContextStale, CodeLatestCollectionUnavailable, CodeLinkConflict:
|
||||
status = http.StatusConflict
|
||||
case CodeAIUnavailable:
|
||||
status = http.StatusServiceUnavailable
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
const CodeLatestCollectionUnavailable = "LATEST_COLLECTION_UNAVAILABLE"
|
||||
const CodeLinkConflict = "PDD_LINK_CONFLICT"
|
||||
|
||||
type LatestCollectionRequest struct {
|
||||
SYBProductID uint64 `json:"sybProductId"`
|
||||
DeviceID uint64 `json:"deviceId"`
|
||||
SpecContextVersion string `json:"specContextVersion"`
|
||||
Replacement *LatestCollectionReplacement `json:"replacement,omitempty"`
|
||||
}
|
||||
|
||||
type LatestCollectionReplacement struct {
|
||||
Preview bool `json:"preview"`
|
||||
ExpectedPDDProductID uint64 `json:"expectedPddProductId"`
|
||||
TaskID uint64 `json:"taskId"`
|
||||
TargetPDDProductID uint64 `json:"targetPddProductId"`
|
||||
TargetVersion string `json:"targetVersion"`
|
||||
}
|
||||
|
||||
type LatestCollectionSource struct {
|
||||
TaskID uint64 `json:"taskId"`
|
||||
DeviceID uint64 `json:"deviceId"`
|
||||
PDDProductID uint64 `json:"pddProductId"`
|
||||
RequiresSpecConfirmation bool `json:"requiresSpecConfirmation"`
|
||||
TargetVersion string `json:"targetVersion,omitempty"`
|
||||
Title string `json:"title,omitempty"`
|
||||
GoodsID string `json:"goodsId,omitempty"`
|
||||
PreviousTitle string `json:"previousTitle,omitempty"`
|
||||
PreviousGoodsID string `json:"previousGoodsId,omitempty"`
|
||||
DeviceName string `json:"deviceName,omitempty"`
|
||||
}
|
||||
|
||||
func latestUnavailable(message string) error {
|
||||
return &ServiceError{Code: CodeLatestCollectionUnavailable, Message: message}
|
||||
}
|
||||
|
||||
// Resolve the candidate exactly once under short row locks; never hold a
|
||||
// transaction while calling AI. The returned context belongs to this write,
|
||||
// not a later GET that could silently adopt another operator's association.
|
||||
func (service *Service) linkLatestCollection(ctx context.Context, id uint64, request LinkPDDRequest) (SaveResponse, error) {
|
||||
r := request.LatestCollection
|
||||
if request.PDDProductID != 0 || r.DeviceID == 0 || r.SYBProductID == 0 || r.SpecContextVersion == "" {
|
||||
return SaveResponse{}, invalidRequest("一键关联需要 SYB、设备及规格版本,不能同时指定 PDD 商品")
|
||||
}
|
||||
var response SaveResponse
|
||||
err := service.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
var syb models.SYBProduct
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).First(&syb, r.SYBProductID).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return latestUnavailable("SYB 明细不存在,请刷新列表")
|
||||
}
|
||||
return err
|
||||
}
|
||||
if syb.ShopeeProductID == nil || *syb.ShopeeProductID != id {
|
||||
return contextVersionStale()
|
||||
}
|
||||
var shopee models.ShopeeProduct
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).First(&shopee, id).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return productNotFound()
|
||||
}
|
||||
return err
|
||||
}
|
||||
replacement := r.Replacement
|
||||
var old models.PDDProduct
|
||||
if replacement != nil {
|
||||
if shopee.PDDProductID == nil || replacement.ExpectedPDDProductID == 0 || *shopee.PDDProductID != replacement.ExpectedPDDProductID {
|
||||
return contextVersionStale()
|
||||
}
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).First(&old, *shopee.PDDProductID).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if service.ReplacementEligibility == nil {
|
||||
return invalidRequest("一键替换入口未配置")
|
||||
}
|
||||
if err := service.ReplacementEligibility(ctx, tx, syb.ID); err != nil {
|
||||
return err
|
||||
}
|
||||
if !replacement.Preview && (replacement.TaskID == 0 || replacement.TargetPDDProductID == 0 || replacement.TargetVersion == "") {
|
||||
return invalidRequest("请先预览并确认替换商品")
|
||||
}
|
||||
} else if shopee.PDDProductID != nil {
|
||||
return &ServiceError{Code: CodeLinkConflict, Message: "商品已有关联,请刷新后查看;一键关联不会覆盖"}
|
||||
}
|
||||
if computeSpecContextVersion(shopee.PDDProductID, shopee.SpecsJSON, old.SpecsJSON) != r.SpecContextVersion {
|
||||
return contextVersionStale()
|
||||
}
|
||||
var device models.AgentDevice
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).First(&device, r.DeviceID).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return latestUnavailable("手机已不存在,请重新选择")
|
||||
}
|
||||
return err
|
||||
}
|
||||
if device.Status != models.DeviceStatusOnline && device.Status != models.DeviceStatusOffline {
|
||||
return latestUnavailable("手机已停用,请重新选择")
|
||||
}
|
||||
var task models.CollectionTask
|
||||
query := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("device_id = ? AND source = ? AND status IN ?", r.DeviceID, models.CollectionTaskSourceAgentCurrentPage, []string{models.TaskStatusCompleted, models.TaskStatusCompletedPartial})
|
||||
if replacement != nil && !replacement.Preview {
|
||||
query = query.Where("id = ?", replacement.TaskID)
|
||||
}
|
||||
err := query.Order("finished_at DESC, id DESC").First(&task).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return latestUnavailable("这台手机暂无成功的临时采集,请先采集商品")
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if task.FinishedAt == nil || task.PDDProductID == nil {
|
||||
return latestUnavailable("最近一次临时采集不完整,请先补采;不会回退关联更早的商品")
|
||||
}
|
||||
var pdd models.PDDProduct
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).First(&pdd, *task.PDDProductID).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return latestUnavailable("最近采集的 PDD 商品已不存在")
|
||||
}
|
||||
return err
|
||||
}
|
||||
if pdd.Status != "active" {
|
||||
return latestUnavailable("最近采集的 PDD 商品尚不可用,请先检查采集结果")
|
||||
}
|
||||
color, err := selectablePDDValues(pdd.SpecsJSON, RoleColor)
|
||||
if err != nil {
|
||||
return latestUnavailable("最近采集的 PDD 规格无效,请先补采")
|
||||
}
|
||||
size, err := selectablePDDValues(pdd.SpecsJSON, RoleSize)
|
||||
if err != nil || len(color)+len(size) == 0 {
|
||||
return latestUnavailable("最近采集的 PDD 缺少可用规格,请先补采")
|
||||
}
|
||||
if task.Status == models.TaskStatusCompletedPartial {
|
||||
if err := validatePartialLinkResult(tx, task, color, size); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
targetVersion := computeSpecContextVersion(&pdd.ID, "", pdd.SpecsJSON)
|
||||
if replacement != nil && !replacement.Preview && (replacement.TargetPDDProductID != pdd.ID || replacement.TargetVersion != targetVersion) {
|
||||
return contextVersionStale()
|
||||
}
|
||||
// Preview never writes. Same-product continuation also preserves the
|
||||
// existing association and request marker; matching is a separate call.
|
||||
if replacement == nil || (!replacement.Preview && old.ID != pdd.ID) {
|
||||
query := tx.Model(&models.ShopeeProduct{}).Where("id = ?", id)
|
||||
if replacement == nil {
|
||||
query = query.Where("pdd_product_id IS NULL")
|
||||
} else {
|
||||
query = query.Where("pdd_product_id = ?", old.ID)
|
||||
}
|
||||
result := query.Updates(map[string]any{"pdd_product_id": pdd.ID, "last_update_request_id": request.RequestID})
|
||||
if result.Error != nil {
|
||||
return result.Error
|
||||
}
|
||||
if result.RowsAffected != 1 {
|
||||
return contextVersionStale()
|
||||
}
|
||||
}
|
||||
response, err = NewService(tx).Detail(ctx, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
response.CollectionSource = &LatestCollectionSource{TaskID: task.ID, DeviceID: device.ID, PDDProductID: pdd.ID, RequiresSpecConfirmation: syb.ParseStatus == models.SYBParseStatusFailed && !syb.ManuallyConfirmed && !syb.AIConfirmed}
|
||||
if replacement != nil {
|
||||
response.CollectionSource.TargetVersion = targetVersion
|
||||
response.CollectionSource.Title = pdd.Title
|
||||
response.CollectionSource.GoodsID = pdd.GoodsID
|
||||
response.CollectionSource.PreviousTitle = old.Title
|
||||
response.CollectionSource.PreviousGoodsID = old.GoodsID
|
||||
response.CollectionSource.DeviceName = device.Name
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
var domain *ServiceError
|
||||
if errors.As(err, &domain) {
|
||||
return SaveResponse{}, err
|
||||
}
|
||||
return SaveResponse{}, internalError(err)
|
||||
}
|
||||
return response, nil
|
||||
}
|
||||
@@ -0,0 +1,194 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/google/uuid"
|
||||
"go-admin/app/goauto/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func latestFixture(t *testing.T) (*Service, uint64, LinkPDDRequest, models.CollectionTask) {
|
||||
t.Helper()
|
||||
db := openTestDB(t)
|
||||
device := models.AgentDevice{InstallID: uuid.NewString(), TokenDigest: uuid.NewString(), Status: models.DeviceStatusOffline}
|
||||
if err := db.Create(&device).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rule := models.CollectionRule{Name: "synthetic", ContentJSON: "{}"}
|
||||
if err := db.Create(&rule).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pdd := seedPDDProduct(t, db, "active")
|
||||
s := NewService(db)
|
||||
created, err := s.Create(context.Background(), CreateRequest{RequestID: uuid.NewString(), ShopeeItemID: uuid.NewString(), Specs: []SpecDimension{{Name: "颜色", Role: RoleColor, Values: []SpecValue{{Name: "黑色", Source: ValueSourceManual}}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
syb := models.SYBProduct{OrderCode: "synthetic", DetailID: 1, Quantity: 1, ParseStatus: "success", ShopeeProductID: &created.Product.ID}
|
||||
if err := db.Create(&syb).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
task := models.CollectionTask{DeviceID: &device.ID, RuleID: rule.ID, PDDProductID: &pdd.ID, Source: models.CollectionTaskSourceAgentCurrentPage, Status: models.TaskStatusCompleted, RuleSnapshot: "{}", FinishedAt: &now}
|
||||
if err := db.Create(&task).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return s, created.Product.ID, LinkPDDRequest{RequestID: uuid.NewString(), LatestCollection: &LatestCollectionRequest{DeviceID: device.ID, SYBProductID: syb.ID, SpecContextVersion: created.Product.SpecContextVersion}}, task
|
||||
}
|
||||
|
||||
func TestLatestCollectionOfflineAndOnlyIfUnlinked(t *testing.T) {
|
||||
s, id, req, task := latestFixture(t)
|
||||
result, err := s.LinkPDD(context.Background(), id, req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if result.CollectionSource.TaskID != task.ID || *result.Product.PDDProductID != *task.PDDProductID || result.Product.SpecContextVersion == req.LatestCollection.SpecContextVersion {
|
||||
t.Fatalf("invalid frozen response: %+v", result)
|
||||
}
|
||||
// Another request (or an ambiguous network retry) cannot overwrite the link.
|
||||
req.RequestID = uuid.NewString()
|
||||
if _, err := s.LinkPDD(context.Background(), id, req); err == nil || errCode(t, err) != CodeLinkConflict {
|
||||
t.Fatalf("expected conflict: %v", err)
|
||||
}
|
||||
other := seedPDDProduct(t, s.DB, "active")
|
||||
if _, err := s.LinkPDD(context.Background(), id, LinkPDDRequest{RequestID: uuid.NewString(), PDDProductID: other.ID}); err != nil {
|
||||
t.Fatalf("manual replacement regressed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLatestCollectionRejectsInvalidContextWithoutWrite(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
change func(*gorm.DB, *LinkPDDRequest, models.CollectionTask)
|
||||
}{
|
||||
{"partial", func(db *gorm.DB, _ *LinkPDDRequest, task models.CollectionTask) {
|
||||
db.Model(&task).Update("status", "completed_partial")
|
||||
}},
|
||||
{"disabled-device", func(db *gorm.DB, r *LinkPDDRequest, _ models.CollectionTask) {
|
||||
db.Model(&models.AgentDevice{}).Where("id = ?", r.LatestCollection.DeviceID).Update("status", "disabled")
|
||||
}},
|
||||
{"missing-device", func(_ *gorm.DB, r *LinkPDDRequest, _ models.CollectionTask) { r.LatestCollection.DeviceID += 100 }},
|
||||
{"admin-source", func(db *gorm.DB, _ *LinkPDDRequest, task models.CollectionTask) {
|
||||
db.Model(&task).Update("source", "admin")
|
||||
}},
|
||||
{"disabled-pdd", func(db *gorm.DB, _ *LinkPDDRequest, task models.CollectionTask) {
|
||||
db.Model(&models.PDDProduct{}).Where("id = ?", *task.PDDProductID).Update("status", "disabled")
|
||||
}},
|
||||
{"empty-specs", func(db *gorm.DB, _ *LinkPDDRequest, task models.CollectionTask) {
|
||||
db.Model(&models.PDDProduct{}).Where("id = ?", *task.PDDProductID).Update("specs_json", "[]")
|
||||
}},
|
||||
{"stale-version", func(_ *gorm.DB, r *LinkPDDRequest, _ models.CollectionTask) {
|
||||
r.LatestCollection.SpecContextVersion = "stale"
|
||||
}},
|
||||
{"syb-link-changed", func(db *gorm.DB, r *LinkPDDRequest, _ models.CollectionTask) {
|
||||
db.Model(&models.SYBProduct{}).Where("id = ?", r.LatestCollection.SYBProductID).Update("shopee_product_id", nil)
|
||||
}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
s, id, r, task := latestFixture(t)
|
||||
tc.change(s.DB, &r, task)
|
||||
if _, err := s.LinkPDD(context.Background(), id, r); err == nil {
|
||||
t.Fatal("unexpected successful write")
|
||||
}
|
||||
var product models.ShopeeProduct
|
||||
s.DB.First(&product, id)
|
||||
if product.PDDProductID != nil {
|
||||
t.Fatal("rejected request wrote association")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLatestCollectionOrdersByFinishThenIDAndDoesNotSkipPartial(t *testing.T) {
|
||||
s, id, r, first := latestFixture(t)
|
||||
newer := first
|
||||
newer.ID = 0
|
||||
newer.Status = models.TaskStatusCompletedPartial
|
||||
if err := s.DB.Create(&newer).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := s.LinkPDD(context.Background(), id, r); err == nil {
|
||||
t.Fatal("skipped latest partial")
|
||||
}
|
||||
// A newer ID with an older completion time must not win.
|
||||
older := newer
|
||||
older.ID = 0
|
||||
older.Status = models.TaskStatusCompleted
|
||||
ts := first.FinishedAt.Add(-time.Hour)
|
||||
older.FinishedAt = &ts
|
||||
if err := s.DB.Create(&older).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := s.LinkPDD(context.Background(), id, r); err == nil {
|
||||
t.Fatal("sorted by id instead of completion")
|
||||
}
|
||||
if err := s.DB.Model(&newer).Update("status", models.TaskStatusCompleted).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
result, err := s.LinkPDD(context.Background(), id, r)
|
||||
if err != nil || result.CollectionSource.TaskID != newer.ID {
|
||||
t.Fatalf("wrong latest result: %+v %v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLatestCollectionKeepsParseConfirmationBoundary(t *testing.T) {
|
||||
for _, status := range []string{"success", "uncertain", "failed"} {
|
||||
t.Run(status, func(t *testing.T) {
|
||||
s, id, r, _ := latestFixture(t)
|
||||
s.DB.Model(&models.SYBProduct{}).Where("id = ?", r.LatestCollection.SYBProductID).Update("parse_status", status)
|
||||
result, err := s.LinkPDD(context.Background(), id, r)
|
||||
if err != nil || result.CollectionSource.RequiresSpecConfirmation != (status == "failed") {
|
||||
t.Fatalf("parse boundary: %+v %v", result, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientKeyCannotUseAdminCompositeLink(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
r := gin.New()
|
||||
r.POST("/api/client/v1/shopee-products/:productId/link-pdd", Handler{}.LinkPDD)
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, httptest.NewRequest(http.MethodPost, "/api/client/v1/shopee-products/1/link-pdd", strings.NewReader(`{"latestCollection":{"deviceId":1,"sybProductId":1,"specContextVersion":"x"}}`)))
|
||||
if w.Code != http.StatusForbidden {
|
||||
t.Fatalf("unexpected status: %d %s", w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// SQLite has no row-level FOR UPDATE. A single test connection serializes
|
||||
// transactions to exercise concurrent callers; production MySQL locking must
|
||||
// additionally be verified in its deployment environment.
|
||||
func TestLatestCollectionConcurrentCallersOnlyOneWins(t *testing.T) {
|
||||
s, id, r, _ := latestFixture(t)
|
||||
sqlDB, err := s.DB.DB()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sqlDB.SetMaxOpenConns(1)
|
||||
errors := make(chan error, 2)
|
||||
start := make(chan struct{})
|
||||
for range 2 {
|
||||
go func() { <-start; _, err := s.LinkPDD(context.Background(), id, r); errors <- err }()
|
||||
}
|
||||
close(start)
|
||||
wins := 0
|
||||
for range 2 {
|
||||
err := <-errors
|
||||
if err == nil {
|
||||
wins++
|
||||
} else if errCode(t, err) != CodeLinkConflict {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if wins != 1 {
|
||||
t.Fatalf("expected one write, got %d", wins)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Only auxiliary metadata may be missing. Validate the source task, not the
|
||||
// merged product archive, which can retain data from an older collection.
|
||||
func validatePartialLinkResult(tx *gorm.DB, task models.CollectionTask, colors, sizes map[string]bool) error {
|
||||
reject := func() error {
|
||||
return latestUnavailable("最近一次采集的规格或价格仍不完整,请先补采;不会回退关联更早的商品")
|
||||
}
|
||||
var missing []string
|
||||
if task.MissingJSON == nil || json.Unmarshal([]byte(*task.MissingJSON), &missing) != nil || len(missing) == 0 {
|
||||
return reject()
|
||||
}
|
||||
for _, field := range missing {
|
||||
switch field {
|
||||
case "salesText", "shopName", "reviewCount":
|
||||
default:
|
||||
return reject()
|
||||
}
|
||||
}
|
||||
var dimensions []models.CollectionDimension
|
||||
if err := tx.Where("task_id = ?", task.ID).Find(&dimensions).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
values := map[string]map[string]bool{}
|
||||
for _, dimension := range dimensions {
|
||||
if (dimension.Key != RoleColor && dimension.Key != RoleSize) || values[dimension.Key] != nil {
|
||||
return reject()
|
||||
}
|
||||
var rows []models.CollectionDimensionValue
|
||||
if err := tx.Where("dimension_id = ?", dimension.ID).Find(&rows).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(rows) == 0 {
|
||||
return reject()
|
||||
}
|
||||
values[dimension.Key] = map[string]bool{}
|
||||
for _, row := range rows {
|
||||
if strings.TrimSpace(row.Value) == "" {
|
||||
return reject()
|
||||
}
|
||||
values[dimension.Key][row.Value] = true
|
||||
}
|
||||
}
|
||||
if len(values) == 0 {
|
||||
return reject()
|
||||
}
|
||||
var skus []models.CollectionSKU
|
||||
if err := tx.Where("task_id = ? AND available = ?", task.ID, true).Find(&skus).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(skus) == 0 {
|
||||
return reject()
|
||||
}
|
||||
covered := map[string]map[string]bool{RoleColor: {}, RoleSize: {}}
|
||||
for _, sku := range skus {
|
||||
var specs map[string]string
|
||||
if !sku.Complete || sku.PriceCent < 0 || json.Unmarshal([]byte(sku.SpecKey), &specs) != nil || len(specs) != len(values) {
|
||||
return reject()
|
||||
}
|
||||
for role, value := range specs {
|
||||
if !values[role][value] {
|
||||
return reject()
|
||||
}
|
||||
covered[role][value] = true
|
||||
}
|
||||
}
|
||||
var prices []models.CollectionColorPrice
|
||||
if err := tx.Where("task_id = ?", task.ID).Find(&prices).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
priced := map[string]bool{}
|
||||
for _, price := range prices {
|
||||
if price.PriceCent >= 0 {
|
||||
priced[price.Color] = true
|
||||
}
|
||||
}
|
||||
for role, candidates := range map[string]map[string]bool{RoleColor: colors, RoleSize: sizes} {
|
||||
for value := range candidates {
|
||||
if !covered[role][value] || (role == RoleColor && !priced[value]) {
|
||||
return reject()
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func seedPartialResult(t *testing.T, db *gorm.DB, task models.CollectionTask, missing string, single bool) {
|
||||
t.Helper()
|
||||
if err := db.Model(&task).Updates(map[string]any{"status": models.TaskStatusCompletedPartial, "missing_json": missing}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dimensions := map[string][]string{"color": {"黑色"}, "size": {"M", "L", "XL", "2XL"}}
|
||||
if single {
|
||||
delete(dimensions, "size")
|
||||
if err := db.Model(&models.PDDProduct{}).Where("id = ?", *task.PDDProductID).Update("specs_json", `[{"role":"color","name":"颜色","values":[{"name":"黑色","selectable":true,"priceCent":1000}]}]`).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for role, values := range dimensions {
|
||||
d := models.CollectionDimension{TaskID: task.ID, Key: role, Name: role}
|
||||
if err := db.Create(&d).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, value := range values {
|
||||
if err := db.Create(&models.CollectionDimensionValue{DimensionID: d.ID, Value: value}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := db.Create(&models.CollectionColorPrice{TaskID: task.ID, Color: "黑色", PriceCent: 1000}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sizes := dimensions["size"]
|
||||
if single {
|
||||
sizes = []string{""}
|
||||
}
|
||||
for _, size := range sizes {
|
||||
specs := map[string]string{"color": "黑色"}
|
||||
if size != "" {
|
||||
specs["size"] = size
|
||||
}
|
||||
raw, _ := json.Marshal(specs)
|
||||
if err := db.Create(&models.CollectionSKU{TaskID: task.ID, SpecKey: string(raw), PriceCent: 1000, Available: true, Complete: true}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPartialAuxiliaryMetadataAllowsLinkAndKeepsStatus(t *testing.T) {
|
||||
for _, missing := range []string{`["salesText"]`, `["shopName","reviewCount","salesText"]`} {
|
||||
t.Run(missing, func(t *testing.T) {
|
||||
s, id, req, task := latestFixture(t)
|
||||
seedPartialResult(t, s.DB, task, missing, false)
|
||||
result, err := s.LinkPDD(context.Background(), id, req)
|
||||
if err != nil || result.CollectionSource.TaskID != task.ID {
|
||||
t.Fatalf("link: %+v %v", result, err)
|
||||
}
|
||||
s.DB.First(&task, task.ID)
|
||||
if task.Status != models.TaskStatusCompletedPartial || *task.MissingJSON != missing {
|
||||
t.Fatal("changed historical completeness")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPartialMetadataAllowsSingleDimensionAndReplacement(t *testing.T) {
|
||||
s, id, req, task, old := replacementFixture(t)
|
||||
seedPartialResult(t, s.DB, task, `["salesText"]`, true)
|
||||
preview := previewReplacement(t, s, id, &req)
|
||||
if *preview.Product.PDDProductID != old.ID {
|
||||
t.Fatal("preview wrote")
|
||||
}
|
||||
result, err := s.LinkPDD(context.Background(), id, req)
|
||||
if err != nil || *result.Product.PDDProductID != *task.PDDProductID {
|
||||
t.Fatalf("replacement: %+v %v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPartialMissingOrResultDefectsStillReject(t *testing.T) {
|
||||
for _, scenario := range []string{"[]", "null", "bad-json", `["salesText","size"]`, `["title"]`, `["price:黑色"]`, `["skuLimit:12>8"]`, `["unknown"]`, "sku-incomplete", "sku-unavailable", "sku-invalid", "no-current-price", "old-archive-value"} {
|
||||
t.Run(scenario, func(t *testing.T) {
|
||||
s, id, req, task := latestFixture(t)
|
||||
missing := scenario
|
||||
if scenario[0] != '[' && scenario != "null" && scenario != "bad-json" {
|
||||
missing = `["salesText"]`
|
||||
}
|
||||
seedPartialResult(t, s.DB, task, missing, false)
|
||||
switch scenario {
|
||||
case "sku-incomplete":
|
||||
s.DB.Model(&models.CollectionSKU{}).Where("task_id = ?", task.ID).Update("complete", false)
|
||||
case "sku-unavailable":
|
||||
s.DB.Model(&models.CollectionSKU{}).Where("task_id = ?", task.ID).Update("available", false)
|
||||
case "sku-invalid":
|
||||
var sku models.CollectionSKU
|
||||
if err := s.DB.Where("task_id = ?", task.ID).First(&sku).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := s.DB.Model(&sku).Update("spec_key", `{}`).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "no-current-price":
|
||||
s.DB.Model(&models.CollectionColorPrice{}).Where("task_id = ?", task.ID).Update("color", "不是本次颜色")
|
||||
case "old-archive-value":
|
||||
s.DB.Model(&models.PDDProduct{}).Where("id = ?", *task.PDDProductID).Update("specs_json", `[{"role":"color","values":[{"name":"旧颜色","selectable":true,"priceCent":1000}]}]`)
|
||||
}
|
||||
if _, err := s.LinkPDD(context.Background(), id, req); err == nil {
|
||||
t.Fatal("accepted incomplete current result")
|
||||
}
|
||||
var product models.ShopeeProduct
|
||||
s.DB.First(&product, id)
|
||||
if product.PDDProductID != nil {
|
||||
t.Fatal("rejected request wrote association")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
package shopeeproduct
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func replacementFixture(t *testing.T) (*Service, uint64, LinkPDDRequest, models.CollectionTask, models.PDDProduct) {
|
||||
s, id, req, task := latestFixture(t)
|
||||
old := seedPDDProduct(t, s.DB, "pending")
|
||||
if err := s.DB.Model(&models.ShopeeProduct{}).Where("id = ?", id).Update("pdd_product_id", old.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
detail, err := s.Detail(context.Background(), id)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
req.LatestCollection.SpecContextVersion = detail.Product.SpecContextVersion
|
||||
req.LatestCollection.Replacement = &LatestCollectionReplacement{Preview: true, ExpectedPDDProductID: old.ID}
|
||||
s.ReplacementEligibility = func(context.Context, *gorm.DB, uint64) error { return nil }
|
||||
return s, id, req, task, old
|
||||
}
|
||||
|
||||
func previewReplacement(t *testing.T, s *Service, id uint64, req *LinkPDDRequest) SaveResponse {
|
||||
t.Helper()
|
||||
preview, err := s.LinkPDD(context.Background(), id, *req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := req.LatestCollection.Replacement
|
||||
r.Preview = false
|
||||
r.TaskID = preview.CollectionSource.TaskID
|
||||
r.TargetPDDProductID = preview.CollectionSource.PDDProductID
|
||||
r.TargetVersion = preview.CollectionSource.TargetVersion
|
||||
return preview
|
||||
}
|
||||
|
||||
func TestReplacementPreviewIsReadOnlyAndFreezesCandidate(t *testing.T) {
|
||||
s, id, req, task, old := replacementFixture(t)
|
||||
preview := previewReplacement(t, s, id, &req)
|
||||
if *preview.Product.PDDProductID != old.ID {
|
||||
t.Fatal("preview changed association")
|
||||
}
|
||||
var record models.ShopeeProduct
|
||||
s.DB.First(&record, id)
|
||||
if *record.PDDProductID != old.ID || record.LastUpdateRequestID != nil {
|
||||
t.Fatal("preview wrote")
|
||||
}
|
||||
newer := task
|
||||
newer.ID = 0
|
||||
newer.Status = models.TaskStatusCompletedPartial
|
||||
if err := s.DB.Create(&newer).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
result, err := s.LinkPDD(context.Background(), id, req)
|
||||
if err != nil || *result.Product.PDDProductID != *task.PDDProductID || result.CollectionSource.TaskID != task.ID {
|
||||
t.Fatalf("frozen candidate: %+v %v", result, err)
|
||||
}
|
||||
if _, err := s.LinkPDD(context.Background(), id, req); err == nil {
|
||||
t.Fatal("duplicate replaced stale association")
|
||||
}
|
||||
}
|
||||
|
||||
func TestReplacementSameProductDoesNotRebind(t *testing.T) {
|
||||
s, id, req, task, _ := replacementFixture(t)
|
||||
s.DB.Model(&models.ShopeeProduct{}).Where("id = ?", id).Update("pdd_product_id", *task.PDDProductID)
|
||||
detail, _ := s.Detail(context.Background(), id)
|
||||
req.LatestCollection.SpecContextVersion = detail.Product.SpecContextVersion
|
||||
req.LatestCollection.Replacement.ExpectedPDDProductID = *task.PDDProductID
|
||||
previewReplacement(t, s, id, &req)
|
||||
if _, err := s.LinkPDD(context.Background(), id, req); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var record models.ShopeeProduct
|
||||
s.DB.First(&record, id)
|
||||
if record.LastUpdateRequestID != nil {
|
||||
t.Fatal("same product wrote association")
|
||||
}
|
||||
}
|
||||
|
||||
func TestReplacementRejectsChangesAfterPreview(t *testing.T) {
|
||||
for _, change := range []string{"association", "specs", "target-specs", "task-status", "device", "stage", "missing-validator", "target-id", "task-id"} {
|
||||
t.Run(change, func(t *testing.T) {
|
||||
s, id, req, task, old := replacementFixture(t)
|
||||
previewReplacement(t, s, id, &req)
|
||||
expected := old.ID
|
||||
switch change {
|
||||
case "association":
|
||||
other := seedPDDProduct(t, s.DB, "active")
|
||||
expected = other.ID
|
||||
s.DB.Model(&models.ShopeeProduct{}).Where("id = ?", id).Update("pdd_product_id", expected)
|
||||
case "specs":
|
||||
s.DB.Model(&models.ShopeeProduct{}).Where("id = ?", id).Update("specs_json", "[]")
|
||||
case "target-specs":
|
||||
s.DB.Model(&models.PDDProduct{}).Where("id = ?", *task.PDDProductID).Update("specs_json", `[{"role":"color","name":"颜色","values":[{"name":"白色","selectable":true}]}]`)
|
||||
case "task-status":
|
||||
s.DB.Model(&task).Update("status", "completed_partial")
|
||||
case "device":
|
||||
s.DB.Model(&models.AgentDevice{}).Where("id = ?", req.LatestCollection.DeviceID).Update("status", "disabled")
|
||||
case "stage":
|
||||
s.ReplacementEligibility = func(context.Context, *gorm.DB, uint64) error { return &ServiceError{Code: CodeLinkConflict} }
|
||||
case "missing-validator":
|
||||
s.ReplacementEligibility = nil
|
||||
case "target-id":
|
||||
req.LatestCollection.Replacement.TargetPDDProductID++
|
||||
case "task-id":
|
||||
req.LatestCollection.Replacement.TaskID++
|
||||
}
|
||||
if _, err := s.LinkPDD(context.Background(), id, req); err == nil {
|
||||
t.Fatal("accepted changed context")
|
||||
}
|
||||
var record models.ShopeeProduct
|
||||
s.DB.First(&record, id)
|
||||
if *record.PDDProductID != expected {
|
||||
t.Fatal("overwrote rejected association")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestReplacementConcurrentConfirmationsOnlyOneWrites(t *testing.T) {
|
||||
s, id, req, _, _ := replacementFixture(t)
|
||||
previewReplacement(t, s, id, &req)
|
||||
connection, err := s.DB.DB()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
connection.SetMaxOpenConns(1) // SQLite serialization, not a MySQL lock claim.
|
||||
results := make(chan error, 2)
|
||||
for range 2 {
|
||||
go func() { _, err := s.LinkPDD(context.Background(), id, req); results <- err }()
|
||||
}
|
||||
wins := 0
|
||||
for range 2 {
|
||||
if <-results == nil {
|
||||
wins++
|
||||
}
|
||||
}
|
||||
if wins != 1 {
|
||||
t.Fatalf("writes=%d", wins)
|
||||
}
|
||||
}
|
||||
@@ -7,9 +7,12 @@ import (
|
||||
jwt "github.com/go-admin-team/go-admin-core/sdk/pkg/jwtauth"
|
||||
)
|
||||
|
||||
func InitRouter(engine *gin.Engine, auth *jwt.GinJWTMiddleware) {
|
||||
handler := Handler{}
|
||||
func InitRouter(engine *gin.Engine, auth *jwt.GinJWTMiddleware, eligibility ReplacementEligibility) {
|
||||
handler := Handler{ReplacementEligibility: eligibility}
|
||||
admin := engine.Group("/api/admin/v1/shopee-products").Use(auth.MiddlewareFunc()).Use(middleware.AuthCheckRole())
|
||||
adminOnlyRuns := engine.Group("/api/admin/v1/shopee-spec-auto-match/runs").Use(auth.MiddlewareFunc()).Use(middleware.AuthCheckRole()).Use(middleware.RequireRoleKey("admin"))
|
||||
adminOnlyRuns.POST("", handler.StartAutoMatchRun)
|
||||
adminOnlyRuns.GET("/latest", handler.LatestAutoMatchRun)
|
||||
admin.GET("", handler.List)
|
||||
admin.POST("", handler.Create)
|
||||
admin.POST("/batch-delete", handler.BatchDelete)
|
||||
|
||||
@@ -57,7 +57,12 @@ func (err *ServiceError) Error() string {
|
||||
}
|
||||
func (err *ServiceError) Unwrap() error { return err.Cause }
|
||||
|
||||
type Service struct{ DB *gorm.DB }
|
||||
type ReplacementEligibility func(context.Context, *gorm.DB, uint64) error
|
||||
|
||||
type Service struct {
|
||||
DB *gorm.DB
|
||||
ReplacementEligibility ReplacementEligibility
|
||||
}
|
||||
|
||||
func NewService(db *gorm.DB) *Service { return &Service{DB: db} }
|
||||
|
||||
@@ -76,8 +81,10 @@ type ProductView struct {
|
||||
}
|
||||
|
||||
type SaveResponse struct {
|
||||
Product ProductView `json:"product"`
|
||||
Replayed bool `json:"replayed,omitempty"`
|
||||
Product ProductView `json:"product"`
|
||||
Replayed bool `json:"replayed,omitempty"`
|
||||
AutoMatchTimeoutSeconds int `json:"autoMatchTimeoutSeconds,omitempty"`
|
||||
CollectionSource *LatestCollectionSource `json:"collectionSource,omitempty"`
|
||||
}
|
||||
|
||||
type ListItemView struct {
|
||||
@@ -156,6 +163,16 @@ func (service *Service) Create(ctx context.Context, request CreateRequest) (Save
|
||||
return SaveResponse{Product: view}, err
|
||||
}
|
||||
|
||||
// DetailWithAutoMatchBudget exposes only a derived time budget, never AI settings.
|
||||
func (service *Service) DetailWithAutoMatchBudget(ctx context.Context, id uint64) (SaveResponse, error) {
|
||||
response, err := service.Detail(ctx, id)
|
||||
if err != nil {
|
||||
return SaveResponse{}, err
|
||||
}
|
||||
response.AutoMatchTimeoutSeconds, err = service.autoMatchTimeout(ctx)
|
||||
return response, err
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- update (archive fields only)
|
||||
|
||||
type UpdateRequest struct {
|
||||
@@ -262,8 +279,9 @@ func (service *Service) List(ctx context.Context, request ListRequest) (ListResp
|
||||
// ---------------------------------------------------------------- link PDD
|
||||
|
||||
type LinkPDDRequest struct {
|
||||
RequestID string `json:"requestId"`
|
||||
PDDProductID uint64 `json:"pddProductId"`
|
||||
RequestID string `json:"requestId"`
|
||||
PDDProductID uint64 `json:"pddProductId"`
|
||||
LatestCollection *LatestCollectionRequest `json:"latestCollection,omitempty"`
|
||||
}
|
||||
|
||||
// LinkPDD sets or changes the linked PDD product via the search-and-select
|
||||
@@ -274,6 +292,9 @@ func (service *Service) LinkPDD(ctx context.Context, id uint64, request LinkPDDR
|
||||
if _, err := uuid.Parse(strings.TrimSpace(request.RequestID)); err != nil {
|
||||
return SaveResponse{}, invalidRequest("requestId 必须是 UUID")
|
||||
}
|
||||
if request.LatestCollection != nil {
|
||||
return service.linkLatestCollection(ctx, id, request)
|
||||
}
|
||||
if request.PDDProductID == 0 {
|
||||
return SaveResponse{}, invalidRequest("pddProductId 必填")
|
||||
}
|
||||
|
||||
@@ -0,0 +1,489 @@
|
||||
package sybimport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"go-admin/app/goauto/aimatching"
|
||||
"go-admin/app/goauto/models"
|
||||
"go-admin/app/goauto/shopeeproduct"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
const (
|
||||
SpecAIParseInvokeTarget = "GoAutoSYBSpecAIParse"
|
||||
defaultAIParseBatchLimit = 20
|
||||
aiParseLeaseDuration = 30 * time.Minute
|
||||
aiParseRetryDelay = time.Hour
|
||||
maxAIParseAttempts = 3
|
||||
)
|
||||
|
||||
var (
|
||||
errAIParseWorkNotClaimed = errors.New("syb spec ai parse work not claimed")
|
||||
errAIParseInputChanged = errors.New("syb spec ai parse input changed")
|
||||
)
|
||||
|
||||
type aiParseInput struct {
|
||||
ProductSpec string
|
||||
Colors []string
|
||||
Sizes []string
|
||||
Fingerprint string
|
||||
}
|
||||
|
||||
func StartSpecAIParseRun(ctx context.Context, db *gorm.DB, requestID string, batchLimit int) (models.SYBSpecAIParseRun, bool, error) {
|
||||
if _, err := uuid.Parse(strings.TrimSpace(requestID)); err != nil {
|
||||
return models.SYBSpecAIParseRun{}, false, fmt.Errorf("requestId 必须是 UUID")
|
||||
}
|
||||
if batchLimit <= 0 {
|
||||
batchLimit = defaultAIParseBatchLimit
|
||||
}
|
||||
if batchLimit > 100 {
|
||||
return models.SYBSpecAIParseRun{}, false, fmt.Errorf("batchLimit 不能超过 100")
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
lease := now.Add(aiParseLeaseDuration)
|
||||
one := uint8(1)
|
||||
owner := uuid.NewString()
|
||||
var run models.SYBSpecAIParseRun
|
||||
created := false
|
||||
err := db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
if err := tx.Model(&models.SYBSpecAIParseRun{}).
|
||||
Where("status = ? AND active_slot = ? AND lease_expires_at < ?", "running", 1, now).
|
||||
Updates(map[string]any{"status": "failed", "active_slot": nil, "lease_owner": "", "lease_expires_at": nil, "error_summary": "上次运行租约过期,已安全释放", "finished_at": now}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if err := tx.Where("request_id = ?", requestID).First(&run).Error; err == nil {
|
||||
return nil
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return err
|
||||
}
|
||||
if err := tx.Where("status = ? AND active_slot = ?", "running", 1).First(&run).Error; err == nil {
|
||||
return nil
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return err
|
||||
}
|
||||
run = models.SYBSpecAIParseRun{
|
||||
RequestID: requestID, Trigger: "scheduled", Status: "running", ActiveSlot: &one,
|
||||
LeaseOwner: owner, LeaseExpiresAt: &lease, BatchLimit: batchLimit, StartedAt: now,
|
||||
}
|
||||
if err := tx.Create(&run).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
created = true
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
if findErr := db.WithContext(ctx).Where("status = ? AND active_slot = ?", "running", 1).First(&run).Error; findErr == nil {
|
||||
return run, false, nil
|
||||
}
|
||||
return models.SYBSpecAIParseRun{}, false, err
|
||||
}
|
||||
return run, created, nil
|
||||
}
|
||||
|
||||
func ProcessSpecAIParseRun(ctx context.Context, db *gorm.DB, runID uint64) error {
|
||||
var run models.SYBSpecAIParseRun
|
||||
if err := db.WithContext(ctx).First(&run, runID).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if run.Status != "running" || run.ActiveSlot == nil || *run.ActiveSlot != 1 {
|
||||
return nil
|
||||
}
|
||||
limit := run.BatchLimit
|
||||
if limit <= 0 || limit > 100 {
|
||||
limit = defaultAIParseBatchLimit
|
||||
}
|
||||
queryLimit := limit * 25
|
||||
if queryLimit < 100 {
|
||||
queryLimit = 100
|
||||
}
|
||||
if queryLimit > 1000 {
|
||||
queryLimit = 1000
|
||||
}
|
||||
var candidates []models.SYBProduct
|
||||
if err := db.WithContext(ctx).
|
||||
Where("manually_confirmed = ?", false).
|
||||
Where("parse_status IN ?", []string{models.SYBParseStatusUncertain, models.SYBParseStatusFailed}).
|
||||
Order("updated_at ASC, id ASC").Limit(queryLimit).Find(&candidates).Error; err != nil {
|
||||
finishSpecAIParseRun(db, run, "failed", 0, 0, 0, 0, 0, 1, "扫描异常规格失败")
|
||||
return err
|
||||
}
|
||||
|
||||
eligible, processed, confirmed, unmatched, failed := 0, 0, 0, 0, 0
|
||||
firstError := ""
|
||||
for _, candidate := range candidates {
|
||||
if processed >= limit {
|
||||
break
|
||||
}
|
||||
if candidate.AIConfirmed {
|
||||
current, currentErr := aiConfirmationTargetsCurrent(ctx, db, candidate)
|
||||
if currentErr != nil {
|
||||
failed++
|
||||
continue
|
||||
}
|
||||
if current {
|
||||
continue
|
||||
}
|
||||
if err := db.WithContext(ctx).Model(&models.SYBProduct{}).
|
||||
Where("id = ? AND manually_confirmed = ?", candidate.ID, false).
|
||||
Updates(map[string]any{"ai_confirmed": false, "ai_confidence": nil, "ai_reason": "", "ai_confirmed_at": nil, "ai_input_fingerprint": ""}).Error; err != nil {
|
||||
failed++
|
||||
continue
|
||||
}
|
||||
candidate.AIConfirmed = false
|
||||
}
|
||||
outcome, err := Reparse(ctx, db, candidate.ID, false)
|
||||
if err != nil {
|
||||
failed++
|
||||
if firstError == "" {
|
||||
firstError = "确定性重新解析失败"
|
||||
}
|
||||
continue
|
||||
}
|
||||
if outcome.NewStatus == models.SYBParseStatusSuccess {
|
||||
processed++
|
||||
confirmed++
|
||||
continue
|
||||
}
|
||||
if err := db.WithContext(ctx).First(&candidate, candidate.ID).Error; err != nil {
|
||||
failed++
|
||||
continue
|
||||
}
|
||||
input, ok, err := buildAIParseInput(ctx, db, candidate)
|
||||
if err != nil {
|
||||
failed++
|
||||
if firstError == "" {
|
||||
firstError = "读取 AI 解析上下文失败"
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
eligible++
|
||||
work, claimed, err := claimSpecAIParseWork(ctx, db, run, candidate.ID, input.Fingerprint)
|
||||
if err != nil {
|
||||
failed++
|
||||
continue
|
||||
}
|
||||
if !claimed {
|
||||
continue
|
||||
}
|
||||
processed++
|
||||
renewSpecAIParseRun(db, run)
|
||||
matcher := aimatching.NewService(db)
|
||||
result, matchErr := matcher.ResolveSYBSpec(ctx, aimatching.SYBSpecParseRequest{
|
||||
ProductSpec: input.ProductSpec, Colors: input.Colors, Sizes: input.Sizes,
|
||||
})
|
||||
if matchErr == nil {
|
||||
settings, settingsErr := matcher.Settings(ctx)
|
||||
if settingsErr != nil {
|
||||
matchErr = settingsErr
|
||||
} else if result.Confidence == nil || *result.Confidence < settings.AutoConfirmMinConfidence || strings.TrimSpace(result.Reason) == "" {
|
||||
unmatched++
|
||||
completeSpecAIParseWork(db, work, input.Fingerprint, false, nil)
|
||||
continue
|
||||
} else {
|
||||
matchErr = applyAIParseResult(ctx, db, candidate.ID, input.Fingerprint, result)
|
||||
}
|
||||
}
|
||||
if matchErr != nil {
|
||||
if isNoAIParseMatch(matchErr) || errors.Is(matchErr, errAIParseInputChanged) {
|
||||
unmatched++
|
||||
completeSpecAIParseWork(db, work, input.Fingerprint, false, nil)
|
||||
continue
|
||||
}
|
||||
failed++
|
||||
if firstError == "" {
|
||||
firstError = safeAIParseError(matchErr)
|
||||
}
|
||||
completeSpecAIParseWork(db, work, input.Fingerprint, false, matchErr)
|
||||
continue
|
||||
}
|
||||
confirmed++
|
||||
completeSpecAIParseWork(db, work, input.Fingerprint, true, nil)
|
||||
}
|
||||
status := "completed"
|
||||
if failed > 0 {
|
||||
status = "completed_partial"
|
||||
}
|
||||
return finishSpecAIParseRun(db, run, status, len(candidates), eligible, processed, confirmed, unmatched, failed, firstError)
|
||||
}
|
||||
|
||||
func buildAIParseInput(ctx context.Context, db *gorm.DB, record models.SYBProduct) (aiParseInput, bool, error) {
|
||||
if record.ManuallyConfirmed || (record.ParseStatus != models.SYBParseStatusUncertain && record.ParseStatus != models.SYBParseStatusFailed) || record.ShopeeProductID == nil {
|
||||
return aiParseInput{}, false, nil
|
||||
}
|
||||
var raw rawDetailSpec
|
||||
if err := json.Unmarshal([]byte(record.RawJSON), &raw); err != nil {
|
||||
return aiParseInput{}, false, err
|
||||
}
|
||||
raw.ProductSpec = strings.TrimSpace(raw.ProductSpec)
|
||||
if raw.ProductSpec == "" {
|
||||
return aiParseInput{}, false, nil
|
||||
}
|
||||
var product models.ShopeeProduct
|
||||
if err := db.WithContext(ctx).First(&product, *record.ShopeeProductID).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return aiParseInput{}, false, nil
|
||||
}
|
||||
return aiParseInput{}, false, err
|
||||
}
|
||||
specs, err := shopeeproduct.Unmarshal(product.SpecsJSON)
|
||||
if err != nil {
|
||||
return aiParseInput{}, false, err
|
||||
}
|
||||
colors, sizes, ambiguous := closedShopeeCandidates(specs)
|
||||
if ambiguous || (len(colors) == 0 && len(sizes) == 0) {
|
||||
return aiParseInput{}, false, nil
|
||||
}
|
||||
var setting models.AIMatchingSetting
|
||||
if err := db.WithContext(ctx).First(&setting, 1).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return aiParseInput{}, false, nil
|
||||
}
|
||||
return aiParseInput{}, false, err
|
||||
}
|
||||
if !setting.Enabled || strings.TrimSpace(setting.APIKey) == "" {
|
||||
return aiParseInput{}, false, nil
|
||||
}
|
||||
fingerprintPayload := struct {
|
||||
ProductSpec string
|
||||
ShopeeProductID uint64
|
||||
ShopeeSpecsJSON string
|
||||
SettingUpdatedAt string
|
||||
}{raw.ProductSpec, product.ID, product.SpecsJSON, setting.UpdatedAt.UTC().Format(time.RFC3339Nano)}
|
||||
encoded, _ := json.Marshal(fingerprintPayload)
|
||||
hash := sha256.Sum256(encoded)
|
||||
return aiParseInput{ProductSpec: raw.ProductSpec, Colors: colors, Sizes: sizes, Fingerprint: hex.EncodeToString(hash[:])}, true, nil
|
||||
}
|
||||
|
||||
func aiConfirmationTargetsCurrent(ctx context.Context, db *gorm.DB, record models.SYBProduct) (bool, error) {
|
||||
if !record.AIConfirmed || record.ShopeeProductID == nil {
|
||||
return false, nil
|
||||
}
|
||||
var product models.ShopeeProduct
|
||||
if err := db.WithContext(ctx).First(&product, *record.ShopeeProductID).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return false, nil
|
||||
}
|
||||
return false, err
|
||||
}
|
||||
specs, err := shopeeproduct.Unmarshal(product.SpecsJSON)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
colors, sizes, ambiguous := closedShopeeCandidates(specs)
|
||||
if ambiguous {
|
||||
return false, nil
|
||||
}
|
||||
return closedCandidateContains(record.TargetColor, colors) && closedCandidateContains(record.TargetSize, sizes) &&
|
||||
(strings.TrimSpace(record.TargetColor) != "" || strings.TrimSpace(record.TargetSize) != ""), nil
|
||||
}
|
||||
|
||||
func closedCandidateContains(value string, candidates []string) bool {
|
||||
value = strings.TrimSpace(value)
|
||||
if len(candidates) == 0 {
|
||||
return value == ""
|
||||
}
|
||||
for _, candidate := range candidates {
|
||||
if candidate == value {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func closedShopeeCandidates(specs []shopeeproduct.SpecDimension) (colors, sizes []string, ambiguous bool) {
|
||||
roleDimensions := map[string]int{}
|
||||
for _, dimension := range specs {
|
||||
if dimension.Role != shopeeproduct.RoleColor && dimension.Role != shopeeproduct.RoleSize {
|
||||
continue
|
||||
}
|
||||
values := make([]string, 0, len(dimension.Values))
|
||||
seen := map[string]bool{}
|
||||
for _, value := range dimension.Values {
|
||||
name := strings.TrimSpace(value.Name)
|
||||
if name != "" && !seen[name] {
|
||||
seen[name] = true
|
||||
values = append(values, name)
|
||||
}
|
||||
}
|
||||
if len(values) == 0 {
|
||||
continue
|
||||
}
|
||||
roleDimensions[dimension.Role]++
|
||||
if roleDimensions[dimension.Role] > 1 {
|
||||
return nil, nil, true
|
||||
}
|
||||
if dimension.Role == shopeeproduct.RoleColor {
|
||||
colors = values
|
||||
} else {
|
||||
sizes = values
|
||||
}
|
||||
}
|
||||
return colors, sizes, false
|
||||
}
|
||||
|
||||
func applyAIParseResult(ctx context.Context, db *gorm.DB, id uint64, fingerprint string, result aimatching.SYBSpecParseResult) error {
|
||||
now := time.Now().UTC()
|
||||
return db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
var record models.SYBProduct
|
||||
if err := tx.First(&record, id).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
input, ok, err := buildAIParseInput(ctx, tx, record)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !ok || input.Fingerprint != fingerprint {
|
||||
return errAIParseInputChanged
|
||||
}
|
||||
if result.Confidence == nil || strings.TrimSpace(result.Reason) == "" {
|
||||
return errAIParseInputChanged
|
||||
}
|
||||
write := tx.Model(&models.SYBProduct{}).Where("id = ? AND manually_confirmed = ? AND ai_confirmed = ?", id, false, false).Updates(map[string]any{
|
||||
"target_color": result.Color, "target_size": result.Size,
|
||||
"ai_confirmed": true, "ai_confidence": *result.Confidence, "ai_reason": truncateAIParseText(result.Reason),
|
||||
"ai_confirmed_at": now, "ai_input_fingerprint": fingerprint,
|
||||
})
|
||||
if write.Error != nil {
|
||||
return write.Error
|
||||
}
|
||||
if write.RowsAffected != 1 {
|
||||
return errAIParseInputChanged
|
||||
}
|
||||
return mergeParsedSpec(tx, *record.ShopeeProductID, ParseResult{Color: result.Color, Size: result.Size, Status: models.SYBParseStatusSuccess})
|
||||
})
|
||||
}
|
||||
|
||||
func claimSpecAIParseWork(ctx context.Context, db *gorm.DB, run models.SYBSpecAIParseRun, productID uint64, fingerprint string) (models.SYBSpecAIParseWorkItem, bool, error) {
|
||||
now := time.Now().UTC()
|
||||
lease := now.Add(aiParseLeaseDuration)
|
||||
var work models.SYBSpecAIParseWorkItem
|
||||
err := db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
err := tx.Where("syb_product_id = ?", productID).First(&work).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
work = models.SYBSpecAIParseWorkItem{SYBProductID: productID, RunID: &run.ID, InputFingerprint: fingerprint, Status: "running", AttemptCount: 1, LeaseOwner: run.LeaseOwner, LeaseExpiresAt: &lease}
|
||||
return tx.Create(&work).Error
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if work.InputFingerprint == fingerprint {
|
||||
if work.Status == "completed" || work.Status == "unmatched" || work.AttemptCount >= maxAIParseAttempts || (work.NextAttemptAt != nil && work.NextAttemptAt.After(now)) || (work.Status == "running" && work.LeaseExpiresAt != nil && work.LeaseExpiresAt.After(now)) {
|
||||
return errAIParseWorkNotClaimed
|
||||
}
|
||||
} else {
|
||||
work.AttemptCount = 0
|
||||
}
|
||||
updates := map[string]any{"run_id": run.ID, "input_fingerprint": fingerprint, "status": "running", "attempt_count": work.AttemptCount + 1, "next_attempt_at": nil, "lease_owner": run.LeaseOwner, "lease_expires_at": lease, "last_error_code": "", "last_error": ""}
|
||||
if err := tx.Model(&models.SYBSpecAIParseWorkItem{}).Where("id = ?", work.ID).Updates(updates).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.First(&work, work.ID).Error
|
||||
})
|
||||
if errors.Is(err, errAIParseWorkNotClaimed) {
|
||||
return work, false, nil
|
||||
}
|
||||
return work, err == nil, err
|
||||
}
|
||||
|
||||
func completeSpecAIParseWork(db *gorm.DB, work models.SYBSpecAIParseWorkItem, fingerprint string, confirmed bool, parseErr error) {
|
||||
now := time.Now().UTC()
|
||||
updates := map[string]any{"input_fingerprint": fingerprint, "lease_owner": "", "lease_expires_at": nil}
|
||||
if parseErr == nil {
|
||||
if confirmed {
|
||||
updates["status"] = "completed"
|
||||
} else {
|
||||
updates["status"] = "unmatched"
|
||||
}
|
||||
updates["next_attempt_at"], updates["last_error_code"], updates["last_error"] = nil, "", ""
|
||||
} else {
|
||||
code := aiParseErrorCode(parseErr)
|
||||
updates["status"], updates["last_error_code"], updates["last_error"] = "failed", code, safeAIParseError(parseErr)
|
||||
if code == aimatching.CodeProviderUnavailable && work.AttemptCount < maxAIParseAttempts {
|
||||
next := now.Add(aiParseRetryDelay)
|
||||
updates["next_attempt_at"] = next
|
||||
} else {
|
||||
updates["next_attempt_at"] = nil
|
||||
}
|
||||
}
|
||||
_ = db.Model(&models.SYBSpecAIParseWorkItem{}).Where("id = ?", work.ID).Updates(updates).Error
|
||||
}
|
||||
|
||||
func renewSpecAIParseRun(db *gorm.DB, run models.SYBSpecAIParseRun) {
|
||||
lease := time.Now().UTC().Add(aiParseLeaseDuration)
|
||||
_ = db.Model(&models.SYBSpecAIParseRun{}).Where("id = ? AND status = ? AND lease_owner = ?", run.ID, "running", run.LeaseOwner).Update("lease_expires_at", lease).Error
|
||||
}
|
||||
|
||||
func finishSpecAIParseRun(db *gorm.DB, run models.SYBSpecAIParseRun, status string, scanned, eligible, processed, confirmed, unmatched, failed int, summary string) error {
|
||||
now := time.Now().UTC()
|
||||
return db.Model(&models.SYBSpecAIParseRun{}).Where("id = ? AND status = ? AND lease_owner = ?", run.ID, "running", run.LeaseOwner).Updates(map[string]any{
|
||||
"status": status, "active_slot": nil, "lease_owner": "", "lease_expires_at": nil,
|
||||
"scanned_count": scanned, "eligible_count": eligible, "processed_count": processed,
|
||||
"confirmed_count": confirmed, "unmatched_count": unmatched, "failed_count": failed,
|
||||
"error_summary": truncateAIParseText(summary), "finished_at": now,
|
||||
}).Error
|
||||
}
|
||||
|
||||
func isNoAIParseMatch(err error) bool { return aiParseErrorCode(err) == aimatching.CodeNoMatch }
|
||||
|
||||
func aiParseErrorCode(err error) string {
|
||||
var target *aimatching.Error
|
||||
if errors.As(err, &target) {
|
||||
return target.Code
|
||||
}
|
||||
return CodeInternal
|
||||
}
|
||||
|
||||
func safeAIParseError(err error) string {
|
||||
var target *aimatching.Error
|
||||
if errors.As(err, &target) {
|
||||
return truncateAIParseText(target.Message)
|
||||
}
|
||||
return "服务端处理失败"
|
||||
}
|
||||
|
||||
func truncateAIParseText(value string) string {
|
||||
runes := []rune(strings.TrimSpace(value))
|
||||
if len(runes) > 500 {
|
||||
runes = runes[:500]
|
||||
}
|
||||
return string(runes)
|
||||
}
|
||||
|
||||
type scheduledSpecAIParseArgs struct {
|
||||
BatchLimit int `json:"batchLimit"`
|
||||
}
|
||||
|
||||
type ScheduledSpecAIParseJob struct{}
|
||||
|
||||
func (ScheduledSpecAIParseJob) Exec(_ interface{}) error {
|
||||
return errors.New("SYB 规格 AI 解析定时任务缺少数据库连接")
|
||||
}
|
||||
|
||||
func (ScheduledSpecAIParseJob) ExecWithDB(db *gorm.DB, arg interface{}) error {
|
||||
args := scheduledSpecAIParseArgs{BatchLimit: defaultAIParseBatchLimit}
|
||||
if raw, ok := arg.(string); ok && strings.TrimSpace(raw) != "" {
|
||||
if err := json.Unmarshal([]byte(raw), &args); err != nil {
|
||||
return fmt.Errorf("SYB 规格 AI 解析参数不是合法 JSON: %w", err)
|
||||
}
|
||||
}
|
||||
if args.BatchLimit < 1 || args.BatchLimit > 100 {
|
||||
return errors.New("SYB 规格 AI 解析 batchLimit 必须在 1 到 100 之间")
|
||||
}
|
||||
run, created, err := StartSpecAIParseRun(context.Background(), db, uuid.NewString(), args.BatchLimit)
|
||||
if err != nil || !created {
|
||||
return err
|
||||
}
|
||||
return ProcessSpecAIParseRun(context.Background(), db, run.ID)
|
||||
}
|
||||
@@ -0,0 +1,204 @@
|
||||
package sybimport_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
"go-admin/app/goauto/shopeeproduct"
|
||||
"go-admin/app/goauto/sybimport"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func seedAIParseCandidate(t *testing.T, db *gorm.DB, serverURL string, detailID uint64, productSpec string) models.SYBProduct {
|
||||
t.Helper()
|
||||
detail := realDetailA()
|
||||
detail.ID = detailID
|
||||
detail.ProductSpec = productSpec
|
||||
detail.Raw = []byte(fmt.Sprintf(`{"id":%d,"productSpec":%q}`, detail.ID, productSpec))
|
||||
applied, err := sybimport.ApplyDetail(context.Background(), db, realOrder(), detail)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
specs, err := shopeeproduct.Marshal([]shopeeproduct.SpecDimension{
|
||||
{Name: "颜色", Role: shopeeproduct.RoleColor, Values: []shopeeproduct.SpecValue{{Name: "黑色", Source: shopeeproduct.ValueSourceImport}, {Name: "白色", Source: shopeeproduct.ValueSourceImport}}},
|
||||
{Name: "尺码", Role: shopeeproduct.RoleSize, Values: []shopeeproduct.SpecValue{{Name: "L", Source: shopeeproduct.ValueSourceImport}, {Name: "XL", Source: shopeeproduct.ValueSourceImport}}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Model(&models.ShopeeProduct{}).Where("id = ?", *applied.SYBProduct.ShopeeProductID).Update("specs_json", specs).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
setting := models.AIMatchingSetting{ID: 1, Enabled: true, Provider: "openai_compatible", BaseURL: serverURL, Model: "test-model", APIKey: "test-key", TimeoutSeconds: 5, AutoConfirmMinConfidence: 0.9}
|
||||
if err := db.Save(&setting).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return applied.SYBProduct
|
||||
}
|
||||
|
||||
func TestScheduledAIParseConfirmsClosedCandidatesAndDoesNotRepeat(t *testing.T) {
|
||||
var calls atomic.Int32
|
||||
provider := httptest.NewServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
|
||||
calls.Add(1)
|
||||
response.Header().Set("Content-Type", "application/json")
|
||||
_, _ = response.Write([]byte(`{"choices":[{"message":{"content":"{\"color\":\"黑色\",\"size\":\"XL\",\"reason\":\"原文对应唯一候选\",\"confidence\":0.95}"}}]}`))
|
||||
}))
|
||||
defer provider.Close()
|
||||
db := openTestDB(t)
|
||||
record := seedAIParseCandidate(t, db, provider.URL, 19801, "黑色 XL")
|
||||
rawBefore := record.RawJSON
|
||||
run, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
|
||||
if err != nil || !created {
|
||||
t.Fatalf("start run: created=%v err=%v", created, err)
|
||||
}
|
||||
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, run.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.First(&record, record.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !record.AIConfirmed || record.ManuallyConfirmed || record.ParseStatus != models.SYBParseStatusUncertain || record.TargetColor != "黑色" || record.TargetSize != "XL" || record.AIConfidence == nil || *record.AIConfidence != 0.95 || record.AIReason == "" || record.AIInputFingerprint == "" {
|
||||
t.Fatalf("unexpected confirmed record: %+v", record)
|
||||
}
|
||||
if record.RawJSON != rawBefore {
|
||||
t.Fatal("AI confirmation must not rewrite RawJSON")
|
||||
}
|
||||
var finished models.SYBSpecAIParseRun
|
||||
if err := db.First(&finished, run.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if finished.Status != "completed" || finished.ConfirmedCount != 1 || finished.ProcessedCount != 1 {
|
||||
t.Fatalf("unexpected run: %+v", finished)
|
||||
}
|
||||
second, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
|
||||
if err != nil || !created {
|
||||
t.Fatalf("second start: created=%v err=%v", created, err)
|
||||
}
|
||||
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, second.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 1 {
|
||||
t.Fatalf("unchanged confirmed input called provider %d times", calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestScheduledAIParseLeavesLowConfidenceUnmatchedForSameFingerprint(t *testing.T) {
|
||||
var calls atomic.Int32
|
||||
provider := httptest.NewServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
|
||||
calls.Add(1)
|
||||
_, _ = response.Write([]byte(`{"choices":[{"message":{"content":"{\"color\":\"黑色\",\"size\":\"XL\",\"reason\":\"仍有歧义\",\"confidence\":0.4}"}}]}`))
|
||||
}))
|
||||
defer provider.Close()
|
||||
db := openTestDB(t)
|
||||
record := seedAIParseCandidate(t, db, provider.URL, 19802, "黑色 XL")
|
||||
for i := 0; i < 2; i++ {
|
||||
run, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
|
||||
if err != nil || !created {
|
||||
t.Fatalf("start %d: created=%v err=%v", i, created, err)
|
||||
}
|
||||
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, run.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.First(&record, record.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record.AIConfirmed || calls.Load() != 1 {
|
||||
t.Fatalf("low confidence must remain unconfirmed and not repeat: confirmed=%v calls=%d", record.AIConfirmed, calls.Load())
|
||||
}
|
||||
var work models.SYBSpecAIParseWorkItem
|
||||
if err := db.Where("syb_product_id = ?", record.ID).First(&work).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if work.Status != "unmatched" {
|
||||
t.Fatalf("work status=%s", work.Status)
|
||||
}
|
||||
}
|
||||
|
||||
func TestScheduledAIParseSkipsEmptySourceAndManualConfirmation(t *testing.T) {
|
||||
var calls atomic.Int32
|
||||
provider := httptest.NewServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
|
||||
calls.Add(1)
|
||||
response.WriteHeader(http.StatusInternalServerError)
|
||||
}))
|
||||
defer provider.Close()
|
||||
db := openTestDB(t)
|
||||
empty := seedAIParseCandidate(t, db, provider.URL, 19803, "")
|
||||
manual := seedAIParseCandidate(t, db, provider.URL, 19804, "黑色 XL")
|
||||
if _, err := sybimport.ManualCorrect(context.Background(), db, manual.ID, "黑色", "XL"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
run, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
|
||||
if err != nil || !created {
|
||||
t.Fatalf("start: created=%v err=%v", created, err)
|
||||
}
|
||||
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, run.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 0 {
|
||||
t.Fatalf("ineligible rows called provider %d times", calls.Load())
|
||||
}
|
||||
var workCount int64
|
||||
if err := db.Model(&models.SYBSpecAIParseWorkItem{}).Where("syb_product_id IN ?", []uint64{empty.ID, manual.ID}).Count(&workCount).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if workCount != 0 {
|
||||
t.Fatalf("ineligible rows created %d work items", workCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSpecAIParseRunHasSingleGlobalActiveSlot(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
first, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
|
||||
if err != nil || !created {
|
||||
t.Fatalf("first: created=%v err=%v", created, err)
|
||||
}
|
||||
second, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
|
||||
if err != nil || created || second.ID != first.ID {
|
||||
t.Fatalf("second must reuse active run: first=%d second=%d created=%v err=%v", first.ID, second.ID, created, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestScheduledAIParseRetriesProviderFailureAtMostThreeTimes(t *testing.T) {
|
||||
var calls atomic.Int32
|
||||
provider := httptest.NewServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
|
||||
calls.Add(1)
|
||||
response.WriteHeader(http.StatusBadGateway)
|
||||
}))
|
||||
defer provider.Close()
|
||||
db := openTestDB(t)
|
||||
record := seedAIParseCandidate(t, db, provider.URL, 19805, "黑色 XL")
|
||||
for attempt := 1; attempt <= 4; attempt++ {
|
||||
run, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
|
||||
if err != nil || !created {
|
||||
t.Fatalf("start %d: created=%v err=%v", attempt, created, err)
|
||||
}
|
||||
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, run.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if attempt < 3 {
|
||||
past := time.Now().UTC().Add(-time.Minute)
|
||||
if err := db.Model(&models.SYBSpecAIParseWorkItem{}).Where("syb_product_id = ?", record.ID).Update("next_attempt_at", past).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
if calls.Load() != 3 {
|
||||
t.Fatalf("provider calls=%d, want 3", calls.Load())
|
||||
}
|
||||
var work models.SYBSpecAIParseWorkItem
|
||||
if err := db.Where("syb_product_id = ?", record.ID).First(&work).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if work.AttemptCount != 3 || work.NextAttemptAt != nil {
|
||||
t.Fatalf("retry state=%+v", work)
|
||||
}
|
||||
}
|
||||
@@ -130,13 +130,37 @@ func ApplyDetail(ctx context.Context, db *gorm.DB, order OrderInput, detail Deta
|
||||
result.Outcome = OutcomeCreated
|
||||
case err == nil:
|
||||
record.ID = existing.ID
|
||||
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", existing.ID).Updates(map[string]any{
|
||||
// Human-confirmed target values are authoritative and survive every
|
||||
// source re-import. ParseStatus/ParseNote below still record what the
|
||||
// current deterministic parser observed for audit.
|
||||
if existing.ManuallyConfirmed {
|
||||
record.TargetColor, record.TargetSize = existing.TargetColor, existing.TargetSize
|
||||
record.ManuallyConfirmed = true
|
||||
}
|
||||
updates := map[string]any{
|
||||
"stock_id": record.StockID, "shop_name": record.ShopName, "shopee_item_id": record.ShopeeItemID,
|
||||
"shopee_product_id": record.ShopeeProductID, "product_title": record.ProductTitle,
|
||||
"target_color": record.TargetColor, "target_size": record.TargetSize,
|
||||
"quantity": record.Quantity, "unit_price_cent": record.UnitPriceCent, "image_url": record.ImageURL,
|
||||
"parse_status": record.ParseStatus, "parse_note": record.ParseNote, "raw_json": record.RawJSON,
|
||||
}).Error; err != nil {
|
||||
}
|
||||
// An identical re-import keeps a valid AI decision. Changed source,
|
||||
// link, or a newly deterministic parse invalidates it atomically.
|
||||
preserveAI := existing.AIConfirmed && !existing.ManuallyConfirmed && parsed.Status != models.SYBParseStatusSuccess &&
|
||||
existing.RawJSON == record.RawJSON && sameOptionalID(existing.ShopeeProductID, record.ShopeeProductID)
|
||||
if preserveAI {
|
||||
record.TargetColor, record.TargetSize = existing.TargetColor, existing.TargetSize
|
||||
updates["target_color"], updates["target_size"] = existing.TargetColor, existing.TargetSize
|
||||
record.AIConfirmed, record.AIConfidence, record.AIReason = true, existing.AIConfidence, existing.AIReason
|
||||
record.AIConfirmedAt, record.AIInputFingerprint = existing.AIConfirmedAt, existing.AIInputFingerprint
|
||||
} else {
|
||||
updates["ai_confirmed"], updates["ai_confidence"], updates["ai_reason"] = false, nil, ""
|
||||
updates["ai_confirmed_at"], updates["ai_input_fingerprint"] = nil, ""
|
||||
}
|
||||
if existing.ManuallyConfirmed {
|
||||
updates["target_color"], updates["target_size"] = existing.TargetColor, existing.TargetSize
|
||||
}
|
||||
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", existing.ID).Updates(updates).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
result.Outcome = OutcomeUpdated
|
||||
@@ -161,6 +185,13 @@ func ApplyDetail(ctx context.Context, db *gorm.DB, order OrderInput, detail Deta
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func sameOptionalID(left, right *uint64) bool {
|
||||
if left == nil || right == nil {
|
||||
return left == nil && right == nil
|
||||
}
|
||||
return *left == *right
|
||||
}
|
||||
|
||||
// findOrCreateShopeeProduct implements #40's revival rule: a live match wins,
|
||||
// a soft-deleted match is revived (keeping its prior mapping), and only when
|
||||
// neither exists does the import create a minimal archive. On an existing
|
||||
|
||||
@@ -3,6 +3,8 @@ package sybimport
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go-admin/common/clientprincipal"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -113,8 +115,10 @@ func isImportAlreadyRunning(err error) bool {
|
||||
// `[必须]` This remains the authenticated manual entry point. Both it and the
|
||||
// scheduler delegate to StartImport, and every downstream SYB call is read-only.
|
||||
func (handler Handler) Import(c *gin.Context) {
|
||||
if claimString(jwt.ExtractClaims(c)["rolekey"]) != "admin" {
|
||||
c.JSON(http.StatusForbidden, gin.H{"code": "FORBIDDEN", "message": "只有管理员可以开始导入"})
|
||||
role := claimString(jwt.ExtractClaims(c)["rolekey"])
|
||||
client, clientOK := clientprincipal.Get(c)
|
||||
if role != "admin" && role != "purchaser" && !clientOK {
|
||||
c.JSON(http.StatusForbidden, gin.H{"code": "FORBIDDEN", "message": "只有管理员或采购员可以开始同步"})
|
||||
return
|
||||
}
|
||||
var request ImportRequest
|
||||
@@ -127,9 +131,13 @@ func (handler Handler) Import(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
claims := jwt.ExtractClaims(c)
|
||||
result, err := StartImport(c.Request.Context(), service.DB, request, ImportActor{
|
||||
actor := ImportActor{
|
||||
ID: claimUint64(claims["identity"]), Name: claimString(claims["nice"]),
|
||||
}, false)
|
||||
}
|
||||
if clientOK {
|
||||
actor = ImportActor{ID: client.AuthorizedBy, Name: fmt.Sprintf("client-key:%d", client.KeyID)}
|
||||
}
|
||||
result, err := StartImport(c.Request.Context(), service.DB, request, actor, false)
|
||||
if err != nil {
|
||||
var serviceErr *ServiceError
|
||||
if errors.As(err, &serviceErr) {
|
||||
@@ -163,6 +171,9 @@ func runImport(db *gorm.DB, runID uint64, request ImportRequest, settings config
|
||||
status := SyncRunSucceeded
|
||||
if err != nil {
|
||||
status = SyncRunFailed
|
||||
if report.Created+report.Updated > 0 {
|
||||
status = SyncRunPartial
|
||||
}
|
||||
}
|
||||
finishCtx, finishCancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer finishCancel()
|
||||
|
||||
@@ -53,17 +53,26 @@ func TestImportRejectsNoEnabledShopBeforeConnect(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestImportRequiresAdminRole(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
engine := gin.New()
|
||||
engine.Use(func(c *gin.Context) { c.Set(jwt.JwtPayloadKey, jwt.MapClaims{"rolekey": "purchaser"}); c.Next() })
|
||||
engine.POST("/api/admin/v1/syb-products/import", Handler{}.Import)
|
||||
request := httptest.NewRequest(http.MethodPost, "/api/admin/v1/syb-products/import", bytes.NewBufferString(`{"dateFrom":"2026-08-19","dateTo":"2026-08-19"}`))
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
recorder := httptest.NewRecorder()
|
||||
engine.ServeHTTP(recorder, request)
|
||||
if recorder.Code != http.StatusForbidden || !strings.Contains(recorder.Body.String(), "只有管理员") {
|
||||
t.Fatalf("采购员应被拒绝: status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||
func TestImportRoleBoundary(t *testing.T) {
|
||||
for _, role := range []string{"admin", "purchaser", "viewer", "", "custom-role"} {
|
||||
t.Run(role, func(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
engine := gin.New()
|
||||
engine.Use(func(c *gin.Context) { c.Set(jwt.JwtPayloadKey, jwt.MapClaims{"rolekey": role}); c.Next() })
|
||||
engine.POST("/api/admin/v1/syb-products/import", Handler{}.Import)
|
||||
// Authorized roles reach JSON validation; no DB or external SYB call is made.
|
||||
request := httptest.NewRequest(http.MethodPost, "/api/admin/v1/syb-products/import", bytes.NewBufferString(`{`))
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
recorder := httptest.NewRecorder()
|
||||
engine.ServeHTTP(recorder, request)
|
||||
want := http.StatusForbidden
|
||||
if role == "admin" || role == "purchaser" {
|
||||
want = http.StatusUnprocessableEntity
|
||||
}
|
||||
if recorder.Code != want {
|
||||
t.Fatalf("role=%q status=%d want=%d body=%s", role, recorder.Code, want, recorder.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
package sybimport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestTodayOverlapPreservesCommittedPagesWithoutImportingOverlappingPage(t *testing.T) {
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-28": 2, "2026-08-29": 4}}
|
||||
f.pageIDs = func(date string, start, _ int) []int64 {
|
||||
if date == "2026-08-29" && start == 2 {
|
||||
return []int64{1001, 1002}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
db := newSyncTestDB(t)
|
||||
report, err := Sync(context.Background(), db, newSyncClient(t, f), SyncConfig{PageSize: 2, MaxMatches: 100}, "2026-08-28", "2026-08-29")
|
||||
if err == nil {
|
||||
t.Fatal("overlap was treated as successful sync")
|
||||
}
|
||||
for _, token := range []string{"跨页重复", "firstPage=1", "firstRow=2", "page=2", "row=1", "start=2", "pageSize=2", "expectedTotal=4", "unique=2"} {
|
||||
if !strings.Contains(err.Error(), token) {
|
||||
t.Fatalf("missing %s in %v", token, err)
|
||||
}
|
||||
}
|
||||
if strings.Contains(err.Error(), "1001") || strings.Contains(err.Error(), "已保存") {
|
||||
t.Fatalf("unsafe diagnosis/degraded save: %v", err)
|
||||
}
|
||||
if len(f.listCalls) != 3 || f.detailCalls != 2 || report.OrderCount != 4 || report.Created != 2 || report.Updated != 2 {
|
||||
t.Fatalf("unexpected retry/import boundary: pages=%d details=%d orders=%d", len(f.listCalls), f.detailCalls, report.OrderCount)
|
||||
}
|
||||
var count int64
|
||||
if e := db.Table("syb_product").Count(&count).Error; e != nil || count != 2 {
|
||||
t.Fatalf("yesterday not preserved: count=%d err=%v", count, e)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPaginationHardErrorsDoNotUseOverlapRecovery(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name, date, message string
|
||||
page int
|
||||
ids []int64
|
||||
calls int
|
||||
}{
|
||||
{"same page", "2026-08-29", "同页重复", 0, []int64{1000, 1000}, 1},
|
||||
{"mixed overlap and same page", "2026-08-29", "同页重复", 2, []int64{1001, 1001}, 2},
|
||||
{"overlap and invalid ID", "2026-08-29", "非法 id", 2, []int64{1001, 0}, 2},
|
||||
{"historical overlap", "2026-08-28", "跨页重复", 2, []int64{1001, 1002}, 2},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := &fakeSYB{perDay: map[string]int{tc.date: 4}}
|
||||
f.pageIDs = func(_ string, start, _ int) []int64 {
|
||||
if start == tc.page {
|
||||
return tc.ids
|
||||
}
|
||||
return nil
|
||||
}
|
||||
rows, err := loadDailyList(context.Background(), newSyncClient(t, f), tc.date, 2, 4)
|
||||
if rows != nil || err == nil || !strings.Contains(err.Error(), tc.message) || len(f.listCalls) != tc.calls || f.listTotalCalls != 0 {
|
||||
t.Fatalf("unexpected recovery: rows=%d pages=%d totals=%d err=%v", len(rows), len(f.listCalls), f.listTotalCalls, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -28,7 +28,7 @@ type rawDetailSpec struct {
|
||||
// result page's per-line feedback.
|
||||
type ReparseOutcome struct {
|
||||
SYBProductID uint64 `json:"sybProductId"`
|
||||
Outcome string `json:"outcome"` // reparsed | skipped_manual | unchanged
|
||||
Outcome string `json:"outcome"` // reparsed | skipped_manual | skipped_ai | unchanged
|
||||
OldStatus string `json:"oldStatus"`
|
||||
NewStatus string `json:"newStatus"`
|
||||
}
|
||||
@@ -36,6 +36,7 @@ type ReparseOutcome struct {
|
||||
const (
|
||||
ReparseOutcomeReparsed = "reparsed"
|
||||
ReparseOutcomeSkippedManual = "skipped_manual"
|
||||
ReparseOutcomeSkippedAI = "skipped_ai"
|
||||
ReparseOutcomeUnchanged = "unchanged"
|
||||
)
|
||||
|
||||
@@ -65,6 +66,11 @@ func Reparse(ctx context.Context, db *gorm.DB, sybProductID uint64, force bool)
|
||||
outcome.NewStatus = record.ParseStatus
|
||||
return nil
|
||||
}
|
||||
if record.AIConfirmed && !force {
|
||||
outcome.Outcome = ReparseOutcomeSkippedAI
|
||||
outcome.NewStatus = record.ParseStatus
|
||||
return nil
|
||||
}
|
||||
|
||||
var raw rawDetailSpec
|
||||
if err := json.Unmarshal([]byte(record.RawJSON), &raw); err != nil {
|
||||
@@ -76,8 +82,11 @@ func Reparse(ctx context.Context, db *gorm.DB, sybProductID uint64, force bool)
|
||||
if parsed.Color == record.TargetColor && parsed.Size == record.TargetSize && parsed.Status == record.ParseStatus {
|
||||
outcome.Outcome = ReparseOutcomeUnchanged
|
||||
if force {
|
||||
record.ManuallyConfirmed = false
|
||||
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", record.ID).Update("manually_confirmed", false).Error; err != nil {
|
||||
record.ManuallyConfirmed, record.AIConfirmed = false, false
|
||||
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", record.ID).Updates(map[string]any{
|
||||
"manually_confirmed": false, "ai_confirmed": false, "ai_confidence": nil,
|
||||
"ai_reason": "", "ai_confirmed_at": nil, "ai_input_fingerprint": "",
|
||||
}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
@@ -87,6 +96,8 @@ func Reparse(ctx context.Context, db *gorm.DB, sybProductID uint64, force bool)
|
||||
updates := map[string]any{
|
||||
"target_color": parsed.Color, "target_size": parsed.Size,
|
||||
"parse_status": parsed.Status, "parse_note": parsed.Note, "manually_confirmed": false,
|
||||
"ai_confirmed": false, "ai_confidence": nil, "ai_reason": "",
|
||||
"ai_confirmed_at": nil, "ai_input_fingerprint": "",
|
||||
}
|
||||
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", record.ID).Updates(updates).Error; err != nil {
|
||||
return err
|
||||
@@ -140,8 +151,12 @@ func ManualCorrect(ctx context.Context, db *gorm.DB, sybProductID uint64, color,
|
||||
return err
|
||||
}
|
||||
record.TargetColor, record.TargetSize, record.ManuallyConfirmed = color, size, true
|
||||
record.AIConfirmed, record.AIConfidence, record.AIReason = false, nil, ""
|
||||
record.AIConfirmedAt, record.AIInputFingerprint = nil, ""
|
||||
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", sybProductID).Updates(map[string]any{
|
||||
"target_color": color, "target_size": size, "manually_confirmed": true,
|
||||
"ai_confirmed": false, "ai_confidence": nil, "ai_reason": "",
|
||||
"ai_confirmed_at": nil, "ai_input_fingerprint": "",
|
||||
}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -60,6 +60,35 @@ func TestReparseSkipsManuallyConfirmedRowByDefault(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestReparseSkipsAIConfirmationUnlessForced(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
applied, err := sybimport.ApplyDetail(context.Background(), db, realOrder(), realDetailB())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
confidence := 0.95
|
||||
if err := db.Model(&models.SYBProduct{}).Where("id = ?", applied.SYBProduct.ID).Updates(map[string]any{
|
||||
"target_color": "AI颜色", "target_size": "AI尺码", "ai_confirmed": true,
|
||||
"ai_confidence": confidence, "ai_reason": "AI 结果", "ai_input_fingerprint": strings.Repeat("c", 64),
|
||||
}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
outcome, err := sybimport.Reparse(context.Background(), db, applied.SYBProduct.ID, false)
|
||||
if err != nil || outcome.Outcome != sybimport.ReparseOutcomeSkippedAI {
|
||||
t.Fatalf("unforced outcome=%+v err=%v", outcome, err)
|
||||
}
|
||||
if _, err := sybimport.Reparse(context.Background(), db, applied.SYBProduct.ID, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var record models.SYBProduct
|
||||
if err := db.First(&record, applied.SYBProduct.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record.AIConfirmed || record.AIConfidence != nil || record.AIReason != "" || record.AIInputFingerprint != "" {
|
||||
t.Fatalf("forced reparse retained AI state: %+v", record)
|
||||
}
|
||||
}
|
||||
|
||||
// 可勾选强制覆盖.
|
||||
func TestReparseWithForceOverridesManualCorrection(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
@@ -151,6 +180,83 @@ func TestManualCorrectMergesIntoArchiveLikeASuccessfulParse(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestManualCorrectSupersedesAIConfirmation(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
applied, err := sybimport.ApplyDetail(context.Background(), db, realOrder(), realDetailB())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
confidence := 0.96
|
||||
if err := db.Model(&models.SYBProduct{}).Where("id = ?", applied.SYBProduct.ID).Updates(map[string]any{
|
||||
"ai_confirmed": true, "ai_confidence": confidence, "ai_reason": "旧 AI 结果", "ai_input_fingerprint": strings.Repeat("a", 64),
|
||||
}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
corrected, err := sybimport.ManualCorrect(context.Background(), db, applied.SYBProduct.ID, "人工颜色", "人工尺码")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !corrected.ManuallyConfirmed || corrected.AIConfirmed || corrected.AIConfidence != nil || corrected.AIReason != "" || corrected.AIInputFingerprint != "" {
|
||||
t.Fatalf("human correction did not supersede AI state: %+v", corrected)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReimportPreservesIdenticalAIInputAndInvalidatesChangedSource(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
order, detail := realOrder(), realDetailB()
|
||||
first, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
confidence := 0.95
|
||||
if err := db.Model(&models.SYBProduct{}).Where("id = ?", first.SYBProduct.ID).Updates(map[string]any{
|
||||
"target_color": "AI颜色", "target_size": "AI尺码", "ai_confirmed": true,
|
||||
"ai_confidence": confidence, "ai_reason": "已确认", "ai_input_fingerprint": strings.Repeat("b", 64),
|
||||
}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
same, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !same.SYBProduct.AIConfirmed {
|
||||
t.Fatal("identical re-import must preserve AI confirmation")
|
||||
}
|
||||
if same.SYBProduct.TargetColor != "AI颜色" || same.SYBProduct.TargetSize != "AI尺码" {
|
||||
t.Fatal("identical re-import must preserve AI-confirmed target values")
|
||||
}
|
||||
detail.ProductSpec += " 新备注"
|
||||
detail.Raw = []byte(`{"productSpec":"changed"}`)
|
||||
changed, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if changed.SYBProduct.AIConfirmed || changed.SYBProduct.AIConfidence != nil || changed.SYBProduct.AIReason != "" {
|
||||
t.Fatalf("changed source retained stale AI state: %+v", changed.SYBProduct)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReimportNeverOverwritesManualTargetValues(t *testing.T) {
|
||||
db := openTestDB(t)
|
||||
order, detail := realOrder(), realDetailB()
|
||||
first, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := sybimport.ManualCorrect(context.Background(), db, first.SYBProduct.ID, "人工颜色", "人工尺码"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
detail.ProductSpec = "来源新颜色,来源新尺码"
|
||||
detail.Raw = []byte(`{"productSpec":"来源新颜色,来源新尺码"}`)
|
||||
updated, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !updated.SYBProduct.ManuallyConfirmed || updated.SYBProduct.TargetColor != "人工颜色" || updated.SYBProduct.TargetSize != "人工尺码" {
|
||||
t.Fatalf("re-import overwrote human decision: %+v", updated.SYBProduct)
|
||||
}
|
||||
}
|
||||
|
||||
// Regression test: ReparseOutcome originally had no json tags at all, so Go's
|
||||
// default marshaling produced PascalCase keys ("SYBProductID", "OldStatus")
|
||||
// instead of the camelCase the rest of this API and the admin frontend use.
|
||||
|
||||
@@ -19,7 +19,7 @@ func InitRouter(engine *gin.Engine, auth *jwt.GinJWTMiddleware) {
|
||||
admin.POST("/reparse-batch", handler.ReparseBatch)
|
||||
admin.POST("/import", handler.Import)
|
||||
// Run history is intentionally authenticated but not Casbin-gated: #50
|
||||
// makes it read-only for every role, while Import enforces admin itself.
|
||||
// makes it readable for every role; Import also requires admin/purchaser.
|
||||
readonly := engine.Group("/api/admin/v1/syb-products").Use(auth.MiddlewareFunc())
|
||||
readonly.GET("/sync-runs", handler.ListSyncRuns)
|
||||
readonly.GET("/sync-runs/:runId", handler.SyncRunDetail)
|
||||
|
||||
@@ -41,8 +41,7 @@ const (
|
||||
// maxSyncDays bounds one request's window. It is a guard against a typo in
|
||||
// the date range turning into tens of thousands of remote reads before
|
||||
// MaxMatches trips.
|
||||
maxSyncDays = 31
|
||||
maxTodaySnapshotAttempts = 3
|
||||
maxSyncDays = 31
|
||||
)
|
||||
|
||||
// SyncReport summarises one sync run.
|
||||
@@ -82,8 +81,6 @@ type SyncProgress struct {
|
||||
DaysProcessed int
|
||||
}
|
||||
|
||||
var syncNow = time.Now
|
||||
|
||||
type ProgressFunc func(SyncProgress) error
|
||||
|
||||
type snapshotDriftError struct {
|
||||
@@ -94,14 +91,6 @@ type snapshotDriftError struct {
|
||||
|
||||
func (err *snapshotDriftError) Error() string { return err.message }
|
||||
|
||||
func shanghaiToday() string {
|
||||
location, err := time.LoadLocation("Asia/Shanghai")
|
||||
if err != nil {
|
||||
location = time.FixedZone("Asia/Shanghai", 8*60*60)
|
||||
}
|
||||
return syncNow().In(location).Format("2006-01-02")
|
||||
}
|
||||
|
||||
// Sync pulls every shipment order in [dateFrom, dateTo] and folds each detail
|
||||
// line into the SYB/Shopee archive through ApplyDetail.
|
||||
//
|
||||
@@ -109,16 +98,10 @@ func shanghaiToday() string {
|
||||
// a separate concern (see Connect); a session that dies mid-run surfaces as
|
||||
// sybclient.ErrSessionInvalid and is treated like any other mid-run failure.
|
||||
//
|
||||
// `[必须]` Failure stops the run immediately. Rows already written are NOT
|
||||
// rolled back — ApplyDetail is idempotent on (order_code, detail_id), so a
|
||||
// re-run overwrites them rather than duplicating. What must not happen is
|
||||
// reporting a partial run as a complete one, which would let the missing
|
||||
// orders go unnoticed forever.
|
||||
//
|
||||
// `[必须]` Every day is verified for completeness before anything is written:
|
||||
// the per-day total is re-read after paging and must not have drifted. SYB's
|
||||
// list endpoint returns the *page* size in `total` (§4.3, confirmed against
|
||||
// live data), so the paging loop is driven by listTotal, never by list.total.
|
||||
// Each validated page is committed independently. Date-local read failures
|
||||
// preserve committed pages and allow later dates to run. Database, cancellation
|
||||
// and progress persistence failures stop the run. Completeness is verified at
|
||||
// the end of each date; partial results must never be reported as full success.
|
||||
func Sync(ctx context.Context, db *gorm.DB, client *sybclient.Client, cfg SyncConfig, dateFrom, dateTo string) (SyncReport, error) {
|
||||
return SyncWithProgress(ctx, db, client, cfg, dateFrom, dateTo, nil)
|
||||
}
|
||||
@@ -190,106 +173,38 @@ func SyncWithShopSnapshot(ctx context.Context, db *gorm.DB, client *sybclient.Cl
|
||||
plans = append(plans, dayPlan{date: date, total: total})
|
||||
}
|
||||
|
||||
for dayIndex, plan := range plans {
|
||||
if plan.total == 0 {
|
||||
if err := emit(dayIndex + 1); err != nil {
|
||||
return report, err
|
||||
}
|
||||
continue
|
||||
}
|
||||
// `[必须]` Completeness first, filtering second. loadDailyList proves the
|
||||
// day's snapshot is whole; filtering before that would let drift among
|
||||
// other shops' orders hide a hole in the ones we do want
|
||||
// (docs/12-syb-erp-interface.md §8).
|
||||
rows, listErr := loadDailyListWithRecovery(ctx, client, plan.date, pageSize, plan.total, maxMatches)
|
||||
if listErr != nil && len(rows) == 0 {
|
||||
return report, listErr
|
||||
}
|
||||
report.OrderCount += len(rows)
|
||||
|
||||
byID := make(map[int64]sybclient.StockRow, len(rows))
|
||||
ids := make([]int64, 0, len(rows))
|
||||
for _, row := range rows {
|
||||
name := stringField(row.Raw, "shopName")
|
||||
label := name
|
||||
if sybshop.IsBlank(label) {
|
||||
label = "(无店铺名)"
|
||||
}
|
||||
display, ok := allowed[sybshop.Normalize(name)]
|
||||
if ok {
|
||||
label = display
|
||||
}
|
||||
if !ok {
|
||||
entry := report.ShopBreakdown[label]
|
||||
entry.Skipped++
|
||||
report.ShopBreakdown[label] = entry
|
||||
report.ShopSkipped++
|
||||
continue
|
||||
}
|
||||
entry := report.ShopBreakdown[label]
|
||||
entry.Accepted++
|
||||
report.ShopBreakdown[label] = entry
|
||||
report.AcceptedCount++
|
||||
byID[row.ID] = row
|
||||
ids = append(ids, row.ID)
|
||||
}
|
||||
if len(ids) == 0 {
|
||||
if err := emit(dayIndex + 1); err != nil {
|
||||
return report, err
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
for start := 0; start < len(ids); start += detailBatch {
|
||||
end := start + detailBatch
|
||||
if end > len(ids) {
|
||||
end = len(ids)
|
||||
}
|
||||
batch := ids[start:end]
|
||||
details, err := client.DetailListByStock(ctx, batch)
|
||||
if err != nil {
|
||||
return report, fmt.Errorf("拉取 %s 货运单明细失败(本次同步停止;"+
|
||||
"已写入的数据保留,重跑会按 (order_code, detail_id) 覆盖): %w", plan.date, err)
|
||||
}
|
||||
if err := validateDetailBatch(batch, details); err != nil {
|
||||
return report, fmt.Errorf("%s 货运单明细不完整:%w;本次同步停止", plan.date, err)
|
||||
}
|
||||
for _, detail := range details {
|
||||
// `[必须]` Re-check the shop on the detail response. The list said
|
||||
// this order belongs to an enabled shop; if the detail disagrees,
|
||||
// the two views are inconsistent and importing it would write a
|
||||
// row for a shop nobody enabled.
|
||||
name := stringField(detail.Raw, "shopName")
|
||||
if sybshop.IsBlank(name) {
|
||||
moveAcceptedToSkipped(&report, stringField(byID[detail.ID].Raw, "shopName"), "(无店铺名)", allowed)
|
||||
report.AcceptedCount--
|
||||
report.ShopSkipped++
|
||||
continue
|
||||
}
|
||||
if _, ok := allowed[sybshop.Normalize(name)]; !ok {
|
||||
moveAcceptedToSkipped(&report, stringField(byID[detail.ID].Raw, "shopName"), name, allowed)
|
||||
report.AcceptedCount--
|
||||
report.ShopSkipped++
|
||||
continue
|
||||
}
|
||||
if err := applyStockDetail(ctx, db, byID[detail.ID], detail, &report); err != nil {
|
||||
return report, err
|
||||
}
|
||||
}
|
||||
if err := emit(dayIndex); err != nil {
|
||||
return report, err
|
||||
}
|
||||
}
|
||||
if listErr != nil {
|
||||
return report, listErr
|
||||
}
|
||||
if err := emit(dayIndex + 1); err != nil {
|
||||
daysCompleted := 0
|
||||
var failures []error
|
||||
for _, plan := range plans {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return report, err
|
||||
}
|
||||
_, dayErr := loadDailyList(ctx, client, plan.date, pageSize, plan.total, func(rows []sybclient.StockRow, page int) error {
|
||||
report.OrderCount += len(rows)
|
||||
err := importSyncPage(ctx, db, client, rows, allowed, &report)
|
||||
if err != nil {
|
||||
return fmt.Errorf("%s 第 %d 页明细/入库阶段失败: %w", plan.date, page, err)
|
||||
}
|
||||
if err := emit(daysCompleted); err != nil {
|
||||
return &syncFatalError{err}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if dayErr != nil {
|
||||
failures = append(failures, dayErr)
|
||||
var fatal *syncFatalError
|
||||
if ctx.Err() != nil || errors.As(dayErr, &fatal) || errors.Is(dayErr, sybclient.ErrSessionInvalid) {
|
||||
return report, errors.Join(failures...)
|
||||
}
|
||||
continue
|
||||
}
|
||||
daysCompleted++
|
||||
if err := emit(daysCompleted); err != nil {
|
||||
return report, errors.Join(append(failures, err)...)
|
||||
}
|
||||
}
|
||||
|
||||
report.FinishedAt = time.Now().UTC()
|
||||
return report, nil
|
||||
return report, errors.Join(failures...)
|
||||
}
|
||||
|
||||
func shopSnapshotNames(allowed map[string]string) []string {
|
||||
@@ -336,41 +251,10 @@ func moveAcceptedToSkipped(report *SyncReport, listName, detailName string, allo
|
||||
// `[必须]` The loop bound comes from expectedTotal (listTotal), because
|
||||
// list.total is the current page's row count, not the filtered total (§4.3).
|
||||
// Driving the loop with the response's own total would stop after page one.
|
||||
func loadDailyListWithRecovery(ctx context.Context, client *sybclient.Client, date string, pageSize, expectedTotal, maxMatches int) ([]sybclient.StockRow, error) {
|
||||
if date != shanghaiToday() {
|
||||
return loadDailyList(ctx, client, date, pageSize, expectedTotal)
|
||||
}
|
||||
var last *snapshotDriftError
|
||||
for attempt := 1; attempt <= maxTodaySnapshotAttempts; attempt++ {
|
||||
if attempt > 1 {
|
||||
total, err := client.ListTotal(ctx, date, date, pageSize)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("重新查询今天 %s 货运单总数失败: %w", date, err)
|
||||
}
|
||||
if total > maxMatches {
|
||||
return nil, fmt.Errorf("今天 %s 的货运单总数 %d 超过单次同步上限 %d", date, total, maxMatches)
|
||||
}
|
||||
expectedTotal = total
|
||||
}
|
||||
rows, err := loadDailyList(ctx, client, date, pageSize, expectedTotal)
|
||||
if err == nil {
|
||||
return rows, nil
|
||||
}
|
||||
var drift *snapshotDriftError
|
||||
if !errors.As(err, &drift) {
|
||||
return nil, err
|
||||
}
|
||||
last = drift
|
||||
}
|
||||
if last != nil && last.valid && len(last.rows) > 0 {
|
||||
return last.rows, fmt.Errorf("今天持续变化,已保存本次取得的完整明细,但未形成稳定快照;下次同步继续覆盖: %w", last)
|
||||
}
|
||||
return nil, fmt.Errorf("今天持续变化,连续 %d 次未形成稳定快照,本次同步停止: %w", maxTodaySnapshotAttempts, last)
|
||||
}
|
||||
|
||||
func loadDailyList(ctx context.Context, client *sybclient.Client, date string, pageSize, expectedTotal int) ([]sybclient.StockRow, error) {
|
||||
func loadDailyList(ctx context.Context, client *sybclient.Client, date string, pageSize, expectedTotal int, consume ...func([]sybclient.StockRow, int) error) ([]sybclient.StockRow, error) {
|
||||
rows := make([]sybclient.StockRow, 0, expectedTotal)
|
||||
seen := make(map[int64]struct{}, expectedTotal)
|
||||
type position struct{ page, row int }
|
||||
seen := make(map[int64]position, expectedTotal)
|
||||
|
||||
for start := 0; start < expectedTotal; start += pageSize {
|
||||
pageIndex := start/pageSize + 1
|
||||
@@ -386,19 +270,37 @@ func loadDailyList(ctx context.Context, client *sybclient.Client, date string, p
|
||||
if pageCount != len(page) {
|
||||
return nil, fmt.Errorf("%s 货运单列表第 %d 页响应条数不自洽:total=%d,list=%d", date, pageIndex, pageCount, len(page))
|
||||
}
|
||||
for _, row := range page {
|
||||
// Validate the whole page first: a same-page duplicate must not be
|
||||
// hidden behind an earlier, recoverable cross-page overlap.
|
||||
pageSeen := make(map[int64]int, len(page))
|
||||
for index, row := range page {
|
||||
if row.ID <= 0 {
|
||||
return nil, fmt.Errorf("%s 货运单列表包含非法 id=%d", date, row.ID)
|
||||
}
|
||||
if _, duplicate := seen[row.ID]; duplicate {
|
||||
return nil, fmt.Errorf("%s 货运单列表重复返回 id=%d", date, row.ID)
|
||||
if firstRow, duplicate := pageSeen[row.ID]; duplicate {
|
||||
return nil, fmt.Errorf("%s 货运单列表同页重复 [firstPage=%d,firstRow=%d,page=%d,row=%d,start=%d,pageSize=%d,expectedTotal=%d,unique=%d]",
|
||||
date, pageIndex, firstRow, pageIndex, index+1, start, pageSize, expectedTotal, len(rows))
|
||||
}
|
||||
seen[row.ID] = struct{}{}
|
||||
pageSeen[row.ID] = index + 1
|
||||
}
|
||||
for index, row := range page {
|
||||
if first, duplicate := seen[row.ID]; duplicate {
|
||||
// No rows/valid flag: overlapping pages are never eligible for
|
||||
// the existing final-attempt degraded save, even after deduping.
|
||||
return nil, &snapshotDriftError{message: fmt.Sprintf("%s 货运单列表跨页重复 [firstPage=%d,firstRow=%d,page=%d,row=%d,start=%d,pageSize=%d,expectedTotal=%d,unique=%d]",
|
||||
date, first.page, first.row, pageIndex, index+1, start, pageSize, expectedTotal, len(rows))}
|
||||
}
|
||||
seen[row.ID] = position{page: pageIndex, row: index + 1}
|
||||
rows = append(rows, row)
|
||||
}
|
||||
if len(page) != expectedPageCount {
|
||||
return nil, &snapshotDriftError{message: fmt.Sprintf("%s 货运单列表第 %d 页不完整且相对初始总数发生变化:预期 %d 行,实际 %d 行", date, pageIndex, expectedPageCount, len(page)), rows: rows, valid: len(page) > expectedPageCount}
|
||||
}
|
||||
if len(consume) > 0 {
|
||||
if err := consume[0](page, pageIndex); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Re-read the total: if it moved while we paged, some order was inserted or
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
package sybimport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"go-admin/app/goauto/sybclient"
|
||||
"go-admin/app/goauto/sybshop"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Database/progress errors stop the entire run, not merely the current date.
|
||||
type syncFatalError struct{ error }
|
||||
|
||||
func (e *syncFatalError) Unwrap() error { return e.error }
|
||||
|
||||
// Read all remote details before opening a page transaction. Publish counters
|
||||
// only after commit, so a rollback never reports records as saved.
|
||||
func importSyncPage(ctx context.Context, db *gorm.DB, client *sybclient.Client, rows []sybclient.StockRow, allowed map[string]string, report *SyncReport) error {
|
||||
next := *report
|
||||
next.ShopBreakdown = make(map[string]ShopBreakdown, len(report.ShopBreakdown))
|
||||
for k, v := range report.ShopBreakdown {
|
||||
next.ShopBreakdown[k] = v
|
||||
}
|
||||
byID := make(map[int64]sybclient.StockRow)
|
||||
var ids []int64
|
||||
for _, row := range rows {
|
||||
name := stringField(row.Raw, "shopName")
|
||||
label := name
|
||||
if sybshop.IsBlank(label) {
|
||||
label = "(无店铺名)"
|
||||
}
|
||||
display, ok := allowed[sybshop.Normalize(name)]
|
||||
if ok {
|
||||
label = display
|
||||
}
|
||||
entry := next.ShopBreakdown[label]
|
||||
if ok {
|
||||
entry.Accepted++
|
||||
next.AcceptedCount++
|
||||
ids = append(ids, row.ID)
|
||||
byID[row.ID] = row
|
||||
} else {
|
||||
entry.Skipped++
|
||||
next.ShopSkipped++
|
||||
}
|
||||
next.ShopBreakdown[label] = entry
|
||||
}
|
||||
var details []sybclient.StockDetail
|
||||
for start := 0; start < len(ids); start += detailBatch {
|
||||
end := start + detailBatch
|
||||
if end > len(ids) {
|
||||
end = len(ids)
|
||||
}
|
||||
batch, err := client.DetailListByStock(ctx, ids[start:end])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := validateDetailBatch(ids[start:end], batch); err != nil {
|
||||
return err
|
||||
}
|
||||
details = append(details, batch...)
|
||||
}
|
||||
err := db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
for _, detail := range details {
|
||||
name := stringField(detail.Raw, "shopName")
|
||||
_, ok := allowed[sybshop.Normalize(name)]
|
||||
if sybshop.IsBlank(name) || !ok {
|
||||
moveAcceptedToSkipped(&next, stringField(byID[detail.ID].Raw, "shopName"), name, allowed)
|
||||
next.AcceptedCount--
|
||||
next.ShopSkipped++
|
||||
continue
|
||||
}
|
||||
if err := applyStockDetail(ctx, tx, byID[detail.ID], detail, &next); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return &syncFatalError{err}
|
||||
}
|
||||
*report = next
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
package sybimport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"go-admin/app/goauto/models"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestPageFailurePreservesEarlierPagesContinuesDatesAndRerunsIdempotently(t *testing.T) {
|
||||
db := newSyncTestDB(t)
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-01": 3, "2026-08-02": 1}, shortPageAtIndex: 2}
|
||||
var last SyncProgress
|
||||
report, err := SyncWithProgress(context.Background(), db, newSyncClient(t, f), SyncConfig{PageSize: 2, MaxMatches: 100}, "2026-08-01", "2026-08-02", func(p SyncProgress) error { last = p; return nil })
|
||||
if err == nil || !strings.Contains(err.Error(), "第 2 页") || report.Created != 2 || report.Updated != 1 || last.DaysProcessed != 1 {
|
||||
t.Fatalf("report=%+v progress=%+v err=%v", report, last, err)
|
||||
}
|
||||
var count int64
|
||||
db.Model(&models.SYBProduct{}).Count(&count)
|
||||
if count != 2 {
|
||||
t.Fatal(count)
|
||||
}
|
||||
f.shortPageAtIndex = 0
|
||||
report, err = Sync(context.Background(), db, newSyncClient(t, f), SyncConfig{PageSize: 2, MaxMatches: 100}, "2026-08-01", "2026-08-02")
|
||||
if err != nil || report.Created != 1 || report.Updated != 3 {
|
||||
t.Fatalf("rerun=%+v err=%v", report, err)
|
||||
}
|
||||
db.Model(&models.SYBProduct{}).Count(&count)
|
||||
if count != 3 {
|
||||
t.Fatal(count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPageTransactionRollbackDoesNotPublishCountersAndStopsDates(t *testing.T) {
|
||||
db := newSyncTestDB(t)
|
||||
if err := db.Exec("CREATE TRIGGER reject_second BEFORE INSERT ON syb_product WHEN NEW.detail_id = 10011 BEGIN SELECT RAISE(ABORT, 'test database failure'); END").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-01": 3, "2026-08-02": 1}}
|
||||
report, err := Sync(context.Background(), db, newSyncClient(t, f), SyncConfig{PageSize: 2, MaxMatches: 100}, "2026-08-01", "2026-08-02")
|
||||
var fatal *syncFatalError
|
||||
if !errors.As(err, &fatal) || report.Created != 0 || report.DetailCount != 0 || len(f.listCalls) != 1 {
|
||||
t.Fatalf("report=%+v calls=%v err=%v", report, f.listCalls, err)
|
||||
}
|
||||
var count int64
|
||||
db.Model(&models.SYBProduct{}).Count(&count)
|
||||
if count != 0 {
|
||||
t.Fatal(count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPageProgressFailureStopsAfterCommittedPage(t *testing.T) {
|
||||
db := newSyncTestDB(t)
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-01": 3, "2026-08-02": 1}}
|
||||
calls := 0
|
||||
report, err := SyncWithProgress(context.Background(), db, newSyncClient(t, f), SyncConfig{PageSize: 2, MaxMatches: 100}, "2026-08-01", "2026-08-02", func(SyncProgress) error {
|
||||
calls++
|
||||
if calls == 2 {
|
||||
return errors.New("progress unavailable")
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err == nil || report.Created != 2 || len(f.listCalls) != 1 {
|
||||
t.Fatalf("report=%+v err=%v", report, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPageMissingDetailPreservesPriorPageAndContinuesNextDate(t *testing.T) {
|
||||
db := newSyncTestDB(t)
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-01": 3, "2026-08-02": 1}, detailDropID: 1002}
|
||||
report, err := Sync(context.Background(), db, newSyncClient(t, f), SyncConfig{PageSize: 2, MaxMatches: 100}, "2026-08-01", "2026-08-02")
|
||||
if err == nil || report.Created != 2 || report.Updated != 1 || report.OrderCount != 4 || len(f.listCalls) != 3 {
|
||||
t.Fatalf("report=%+v err=%v", report, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPageCancellationPreservesCommitAndStopsNextPage(t *testing.T) {
|
||||
db := newSyncTestDB(t)
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-01": 3, "2026-08-02": 1}}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
report, err := SyncWithProgress(ctx, db, newSyncClient(t, f), SyncConfig{PageSize: 2, MaxMatches: 100}, "2026-08-01", "2026-08-02", func(p SyncProgress) error {
|
||||
if p.Report.Created > 0 {
|
||||
cancel()
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if !errors.Is(err, context.Canceled) || report.Created != 2 || len(f.listCalls) != 1 {
|
||||
t.Fatalf("report=%+v err=%v", report, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPartialFinishPreservesProgressReleasesSlotAndIsFilterable(t *testing.T) {
|
||||
db := newSyncTestDB(t)
|
||||
ctx := context.Background()
|
||||
s := NewSyncRunService(db)
|
||||
run, err := s.Create(ctx, CreateSyncRunInput{DateFrom: "2026-08-01", DateTo: "2026-08-02"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
report := SyncReport{Created: 2, DetailCount: 2, ShopBreakdown: map[string]ShopBreakdown{}}
|
||||
if err := s.UpdateProgress(ctx, run.ID, SyncProgress{Report: report, DaysTotal: 2, DaysProcessed: 1}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := s.Finish(ctx, run.ID, SyncRunPartial, report, errors.New("第 2 页失败")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
row, err := s.Detail(ctx, run.ID)
|
||||
if err != nil || row.Status != SyncRunPartial || row.ActiveSlot != nil || row.ProgressPercent != 50 || row.Created != 2 {
|
||||
t.Fatalf("row=%+v err=%v", row, err)
|
||||
}
|
||||
result, err := s.List(ctx, SyncRunListRequest{Page: 1, PageSize: 20, Status: SyncRunPartial})
|
||||
if err != nil || result.Total != 1 {
|
||||
t.Fatalf("result=%+v err=%v", result, err)
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,7 @@ import (
|
||||
const (
|
||||
SyncRunRunning = "running"
|
||||
SyncRunSucceeded = "succeeded"
|
||||
SyncRunPartial = "partial_success"
|
||||
SyncRunFailed = "failed"
|
||||
SyncRunInterrupted = "interrupted"
|
||||
)
|
||||
@@ -75,7 +76,7 @@ func (s *SyncRunService) UpdateProgress(ctx context.Context, id uint64, progress
|
||||
}
|
||||
|
||||
func (s *SyncRunService) Finish(ctx context.Context, id uint64, status string, report SyncReport, runErr error) error {
|
||||
if status != SyncRunSucceeded && status != SyncRunFailed {
|
||||
if status != SyncRunSucceeded && status != SyncRunPartial && status != SyncRunFailed {
|
||||
return fmt.Errorf("invalid terminal sync status %q", status)
|
||||
}
|
||||
payload, err := marshalSyncRunPayload(report.ShopBreakdown, report.ShopFilterSnapshot)
|
||||
|
||||
@@ -20,7 +20,7 @@ func (handler Handler) ListSyncRuns(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
status := strings.TrimSpace(c.Query("status"))
|
||||
if status != "" && status != SyncRunRunning && status != SyncRunSucceeded && status != SyncRunFailed && status != SyncRunInterrupted {
|
||||
if status != "" && status != SyncRunRunning && status != SyncRunSucceeded && status != SyncRunPartial && status != SyncRunFailed && status != SyncRunInterrupted {
|
||||
writeError(c, invalidRequest("status 无效"))
|
||||
return
|
||||
}
|
||||
|
||||
@@ -36,4 +36,12 @@ func TestSyncRunListAndDetailHandlers(t *testing.T) {
|
||||
if detail.Code != http.StatusOK || !strings.Contains(detail.Body.String(), `"operatorName":"管理员"`) {
|
||||
t.Fatalf("详情响应不正确: %d %s", detail.Code, detail.Body.String())
|
||||
}
|
||||
if err := NewSyncRunService(db).Finish(context.Background(), run.ID, SyncRunPartial, SyncReport{Created: 1}, nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
partial := httptest.NewRecorder()
|
||||
engine.ServeHTTP(partial, httptest.NewRequest(http.MethodGet, "/sync-runs?status=partial_success", nil))
|
||||
if partial.Code != http.StatusOK || !strings.Contains(partial.Body.String(), `"total":1`) {
|
||||
t.Fatalf("partial filter: %d %s", partial.Code, partial.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,7 +9,6 @@ import (
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go-admin/app/goauto/migrations"
|
||||
"go-admin/app/goauto/models"
|
||||
@@ -60,6 +59,9 @@ type fakeSYB struct {
|
||||
totalOverride map[int]int
|
||||
shortPageAtIndex int
|
||||
detailDropID int64
|
||||
listCalls []string
|
||||
pageIDs func(date string, start, call int) []int64
|
||||
detailCalls int
|
||||
// shopNames 按货运单序号轮换;留空表示全部用「测试店铺」。
|
||||
shopNames []string
|
||||
// detailShopName 非空时,明细响应里的 shopName 用它覆盖,
|
||||
@@ -96,6 +98,7 @@ func (f *fakeSYB) server(t *testing.T) *httptest.Server {
|
||||
start := int(body["start"].(float64))
|
||||
pageSize := int(body["length"].(float64))
|
||||
total := f.perDay[date]
|
||||
f.listCalls = append(f.listCalls, fmt.Sprintf("%s:%d", date, start))
|
||||
|
||||
rows := []map[string]any{}
|
||||
for i := start; i < total && len(rows) < pageSize; i++ {
|
||||
@@ -108,9 +111,18 @@ func (f *fakeSYB) server(t *testing.T) *httptest.Server {
|
||||
if f.shortPageAtIndex > 0 && start/pageSize+1 == f.shortPageAtIndex && len(rows) > 0 {
|
||||
rows = rows[:len(rows)-1]
|
||||
}
|
||||
if f.pageIDs != nil {
|
||||
if ids := f.pageIDs(date, start, len(f.listCalls)); ids != nil {
|
||||
rows = nil
|
||||
for _, id := range ids {
|
||||
rows = append(rows, map[string]any{"id": id, "code": "TEST", "shopName": f.shopFor(0)})
|
||||
}
|
||||
}
|
||||
}
|
||||
writeEnvelope(w, map[string]any{"list": rows, "total": len(rows)})
|
||||
})
|
||||
mux.HandleFunc("/am/stock/detail/listByStock", func(w http.ResponseWriter, r *http.Request) {
|
||||
f.detailCalls++
|
||||
var body struct {
|
||||
IDs []int64 `json:"ids"`
|
||||
}
|
||||
@@ -263,39 +275,6 @@ func TestSyncStopsWhenTotalDriftsDuringPaging(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTodaySnapshotDriftRetriesFromFirstPage(t *testing.T) {
|
||||
originalNow := syncNow
|
||||
syncNow = func() time.Time { return time.Date(2026, 8, 29, 12, 0, 0, 0, time.FixedZone("Asia/Shanghai", 8*60*60)) }
|
||||
t.Cleanup(func() { syncNow = originalNow })
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-29": 10}, totalOverride: map[int]int{1: 11, 2: 10, 3: 10}}
|
||||
rows, err := loadDailyListWithRecovery(context.Background(), newSyncClient(t, f), "2026-08-29", 10, 10, 1000)
|
||||
if err != nil || len(rows) != 10 || f.listTotalCalls != 3 {
|
||||
t.Fatalf("rows=%d totalCalls=%d err=%v", len(rows), f.listTotalCalls, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTodayThirdDriftReturnsValidLastListForDegradedSave(t *testing.T) {
|
||||
originalNow := syncNow
|
||||
syncNow = func() time.Time { return time.Date(2026, 8, 29, 12, 0, 0, 0, time.FixedZone("Asia/Shanghai", 8*60*60)) }
|
||||
t.Cleanup(func() { syncNow = originalNow })
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-29": 10}, totalOverride: map[int]int{1: 11, 2: 10, 3: 11, 4: 10, 5: 11}}
|
||||
rows, err := loadDailyListWithRecovery(context.Background(), newSyncClient(t, f), "2026-08-29", 10, 10, 1000)
|
||||
if err == nil || len(rows) != 10 || !strings.Contains(err.Error(), "已保存本次取得的完整明细") {
|
||||
t.Fatalf("rows=%d err=%v", len(rows), err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHistoricalSnapshotDriftDoesNotRetry(t *testing.T) {
|
||||
originalNow := syncNow
|
||||
syncNow = func() time.Time { return time.Date(2026, 8, 29, 12, 0, 0, 0, time.FixedZone("Asia/Shanghai", 8*60*60)) }
|
||||
t.Cleanup(func() { syncNow = originalNow })
|
||||
f := &fakeSYB{perDay: map[string]int{"2026-08-28": 10}, totalOverride: map[int]int{1: 11}}
|
||||
rows, err := loadDailyListWithRecovery(context.Background(), newSyncClient(t, f), "2026-08-28", 10, 10, 1000)
|
||||
if err == nil || rows != nil || f.listTotalCalls != 1 {
|
||||
t.Fatalf("rows=%v totalCalls=%d err=%v", rows, f.listTotalCalls, err)
|
||||
}
|
||||
}
|
||||
|
||||
// 某页行数少于预期同样是不完整快照。
|
||||
func TestSyncStopsOnShortPage(t *testing.T) {
|
||||
db := newSyncTestDB(t)
|
||||
|
||||
@@ -3,6 +3,7 @@ package sybinnercode
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"go-admin/common/clientprincipal"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
@@ -81,13 +82,19 @@ func (h Handler) Import(c *gin.Context) {
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
result, err := service.Import(c.Request.Context(), src, file.Filename, uint64(user.GetUserId(c)), c.PostForm("requestId"))
|
||||
result, err := service.Import(c.Request.Context(), src, file.Filename, clientOperator(c), c.PostForm("requestId"))
|
||||
if err != nil {
|
||||
writeError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"code": 200, "data": result})
|
||||
}
|
||||
func clientOperator(c *gin.Context) uint64 {
|
||||
if p, ok := clientprincipal.Get(c); ok {
|
||||
return p.AuthorizedBy
|
||||
}
|
||||
return uint64(user.GetUserId(c))
|
||||
}
|
||||
func (h Handler) Delete(c *gin.Context) {
|
||||
var request DeleteRequest
|
||||
if err := decode(c, &request); err != nil {
|
||||
@@ -98,7 +105,7 @@ func (h Handler) Delete(c *gin.Context) {
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
result, err := service.Delete(c.Request.Context(), uint64(user.GetUserId(c)), request)
|
||||
result, err := service.Delete(c.Request.Context(), clientOperator(c), request)
|
||||
if err != nil {
|
||||
writeError(c, err)
|
||||
return
|
||||
@@ -150,7 +157,7 @@ func (h Handler) Apply(c *gin.Context) {
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
result, err := service.QueueApply(c.Request.Context(), uint64(user.GetUserId(c)), request)
|
||||
result, err := service.QueueApply(c.Request.Context(), clientOperator(c), request)
|
||||
if err != nil {
|
||||
writeError(c, err)
|
||||
return
|
||||
|
||||
@@ -255,7 +255,9 @@ func (service *Service) IdentifyCurrentPage(ctx context.Context, taskID uint64,
|
||||
return serviceError(CodeCurrentPageIdentityConflict, "当前商品与任务已识别商品不一致")
|
||||
}
|
||||
now := service.Now()
|
||||
if err := tx.Model(&models.CollectionTask{}).Where("id = ?", record.ID).Updates(map[string]any{
|
||||
// Validate the locked task's real state; an empty model fails BeforeSave
|
||||
// when a retried current-page task already has a product binding.
|
||||
if err := tx.Model(&record).Where("id = ?", record.ID).Updates(map[string]any{
|
||||
"identify_request_id": request.RequestID, "identity_resolved_at": now,
|
||||
}).Error; err != nil {
|
||||
return internalError(err)
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
package task
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"go-admin/app/goauto/models"
|
||||
"go-admin/app/goauto/rulecontract"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
func TestCurrentPageReidentifyAfterSameTaskRetry(t *testing.T) {
|
||||
db := openTaskDatabase(t)
|
||||
_, token := registerTaskDeviceWithCapabilities(t, db, "retry-device", []string{
|
||||
rulecontract.CapabilitySchemaV2,
|
||||
rulecontract.CapabilityPDDProductDetailV1,
|
||||
rulecontract.CapabilityPDDCurrentPageShareV1,
|
||||
})
|
||||
_, otherToken := registerTaskDevice(t, db, "other-device")
|
||||
rule := models.CollectionRule{Name: "retry-rule", ContentJSON: v2TaskRuleSnapshot()}
|
||||
if err := db.Create(&rule).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Create(&models.AgentManualCollectionSetting{ID: 1, RuleID: rule.ID}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
service := newTaskService(db)
|
||||
ctx := context.Background()
|
||||
created, err := service.CreateCurrentPage(ctx, CurrentPageCreateRequest{RequestID: uuid.NewString()}, token)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
load := func() models.CollectionTask {
|
||||
t.Helper()
|
||||
var record models.CollectionTask
|
||||
if err := db.First(&record, created.TaskID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return record
|
||||
}
|
||||
request := CurrentPageIdentifyRequest{
|
||||
RequestID: uuid.NewString(), ShareURL: "https://mobile.yangkeduo.com/goods.html?goods_id=700000000255",
|
||||
}
|
||||
// Direct links resolve locally: the regression never calls PDD or a live DB.
|
||||
identity, err := service.IdentifyCurrentPage(ctx, created.TaskID, request, token)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
const failureCode = "SPEC_PANEL_EVIDENCE_NOT_MATCHED"
|
||||
if _, err := service.SubmitFailure(ctx, created.TaskID, FailRequest{
|
||||
RequestID: uuid.NewString(), Code: failureCode, Message: "Synthetic panel failure",
|
||||
}, token); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
reset, err := service.ResetForDevice(ctx, created.TaskID, ActionRequest{RequestID: uuid.NewString()}, token)
|
||||
if err != nil || reset.TaskID != created.TaskID || reset.AttemptNumber != 2 || reset.Status != models.TaskStatusPending {
|
||||
t.Fatalf("same-task reset: %+v, %v", reset, err)
|
||||
}
|
||||
resetRecord := load()
|
||||
if resetRecord.PDDProductID == nil || *resetRecord.PDDProductID != identity.PDDProductID ||
|
||||
resetRecord.IdentifyRequestID != nil || resetRecord.IdentityResolvedAt != nil {
|
||||
t.Fatal("reset must keep the product and clear per-attempt identification")
|
||||
}
|
||||
if _, err := service.Claim(ctx, created.TaskID, ActionRequest{RequestID: uuid.NewString()}, token); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := service.Start(ctx, created.TaskID, ActionRequest{RequestID: uuid.NewString()}, token); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := load()
|
||||
request.RequestID = uuid.NewString()
|
||||
for _, tc := range []struct {
|
||||
name, token, url, code string
|
||||
}{
|
||||
{"other device", otherToken, request.ShareURL, CodeTaskNotFound},
|
||||
{"other product", token, "https://mobile.yangkeduo.com/goods.html?goods_id=700000000256", CodeCurrentPageIdentityConflict},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := service.IdentifyCurrentPage(ctx, created.TaskID, CurrentPageIdentifyRequest{RequestID: uuid.NewString(), ShareURL: tc.url}, tc.token)
|
||||
if taskErrorCode(t, err) != tc.code {
|
||||
t.Fatalf("expected %s, got %v", tc.code, err)
|
||||
}
|
||||
if !reflect.DeepEqual(before, load()) {
|
||||
t.Fatal("rejected identification changed the task")
|
||||
}
|
||||
})
|
||||
}
|
||||
reidentified, err := service.IdentifyCurrentPage(ctx, created.TaskID, request, token)
|
||||
if err != nil {
|
||||
t.Fatalf("reidentify same product after retry: %v", err)
|
||||
}
|
||||
if reidentified.TaskID != created.TaskID || reidentified.PDDProductID != identity.PDDProductID ||
|
||||
reidentified.GoodsID != identity.GoodsID || reidentified.URL != identity.URL || !reidentified.Replayed {
|
||||
t.Fatalf("identity changed: %+v", reidentified)
|
||||
}
|
||||
after := load()
|
||||
if after.IdentifyRequestID == nil || *after.IdentifyRequestID != request.RequestID || after.IdentityResolvedAt == nil {
|
||||
t.Fatal("new attempt identification was not persisted")
|
||||
}
|
||||
// Apart from identity metadata and the normal update timestamp, every field
|
||||
// (including status, device, rule, lease and uniqueness slots) must survive.
|
||||
expected := before
|
||||
expected.IdentifyRequestID = after.IdentifyRequestID
|
||||
expected.IdentityResolvedAt = after.IdentityResolvedAt
|
||||
expected.UpdatedAt = after.UpdatedAt
|
||||
if !reflect.DeepEqual(expected, after) {
|
||||
t.Fatal("reidentification changed unrelated task fields")
|
||||
}
|
||||
service.ResolveCurrentPageShare = func(context.Context, string) (ResolvedPDDShare, error) {
|
||||
t.Fatal("replay must not resolve the link again")
|
||||
return ResolvedPDDShare{}, nil
|
||||
}
|
||||
replay, err := service.IdentifyCurrentPage(ctx, created.TaskID, request, token)
|
||||
if err != nil || replay != reidentified || !reflect.DeepEqual(after, load()) {
|
||||
t.Fatalf("replay changed identity or task: %+v, %v", replay, err)
|
||||
}
|
||||
service.ResolveCurrentPageShare = nil
|
||||
title := "Synthetic collected product"
|
||||
result, err := service.SubmitResult(ctx, created.TaskID, ResultRequest{
|
||||
RequestID: uuid.NewString(), Status: models.TaskStatusCompleted,
|
||||
Product: ResultProduct{PDDGoodsID: identity.GoodsID, Title: &title},
|
||||
Dimensions: []ResultDimension{{Key: "color", Name: "颜色", Values: []string{"黑色"}}},
|
||||
ColorPrices: []ResultColorPrice{{Color: "黑色", PriceCent: 1200}},
|
||||
SKUs: []ResultSKU{{Specs: map[string]string{"color": "黑色"}, PriceCent: 1200, Available: true}},
|
||||
}, token)
|
||||
if err != nil || result.Task.ID != created.TaskID || result.Task.Status != models.TaskStatusCompleted {
|
||||
t.Fatalf("complete retried collection: %+v, %v", result, err)
|
||||
}
|
||||
terminal := load()
|
||||
request.RequestID = uuid.NewString()
|
||||
if _, err := service.IdentifyCurrentPage(ctx, created.TaskID, request, token); taskErrorCode(t, err) != CodeTaskStateConflict {
|
||||
t.Fatalf("expected terminal-state rejection: %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(terminal, load()) {
|
||||
t.Fatal("terminal identification changed the task")
|
||||
}
|
||||
var archived []models.CollectionTaskAttempt
|
||||
if err := db.Where("task_id = ?", created.TaskID).Find(&archived).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(archived) != 1 || archived[0].AttemptNumber != 1 || archived[0].Status != models.TaskStatusFailed ||
|
||||
archived[0].ErrorCode == nil || *archived[0].ErrorCode != failureCode {
|
||||
t.Fatal("original failed attempt was not preserved")
|
||||
}
|
||||
for _, model := range []any{&models.CollectionTask{}, &models.PDDProduct{}} {
|
||||
var count int64
|
||||
if err := db.Model(model).Count(&count).Error; err != nil || count != 1 {
|
||||
t.Fatalf("retry duplicated task/product: count=%d, %v", count, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package apis
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
jobservice "go-admin/app/jobs/service"
|
||||
)
|
||||
|
||||
func (e SysJob) ListExecutionLogs(c *gin.Context) {
|
||||
jobID, err := strconv.Atoi(c.Param("id"))
|
||||
if err != nil || jobID < 1 {
|
||||
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "jobId 无效"})
|
||||
return
|
||||
}
|
||||
page, err := positiveQueryInt(c.Query("pageIndex"), 1)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "pageIndex 必须是正整数"})
|
||||
return
|
||||
}
|
||||
pageSize, err := positiveQueryInt(c.Query("pageSize"), 20)
|
||||
if err != nil || pageSize > 100 {
|
||||
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "pageSize 必须是 1 到 100 的整数"})
|
||||
return
|
||||
}
|
||||
startedFrom, err := optionalTime(c.Query("startedFrom"))
|
||||
if err != nil {
|
||||
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "startedFrom 必须是 RFC3339 时间"})
|
||||
return
|
||||
}
|
||||
startedTo, err := optionalTime(c.Query("startedTo"))
|
||||
if err != nil {
|
||||
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "startedTo 必须是 RFC3339 时间"})
|
||||
return
|
||||
}
|
||||
|
||||
e.MakeContext(c)
|
||||
db, err := e.GetOrm()
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"code": 500, "msg": "服务端处理失败"})
|
||||
return
|
||||
}
|
||||
response, err := jobservice.NewExecutionLogService(db).List(c.Request.Context(), jobID, jobservice.ExecutionLogListRequest{
|
||||
Page: page, PageSize: pageSize, Status: strings.TrimSpace(c.Query("status")),
|
||||
StartedFrom: startedFrom, StartedTo: startedTo,
|
||||
})
|
||||
if err != nil {
|
||||
switch {
|
||||
case errors.Is(err, jobservice.ErrExecutionLogInvalidRequest):
|
||||
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": err.Error()})
|
||||
case errors.Is(err, jobservice.ErrExecutionLogJobNotFound):
|
||||
c.JSON(http.StatusNotFound, gin.H{"code": 404, "msg": "定时任务不存在"})
|
||||
default:
|
||||
e.GetLogger().Errorf("list scheduled job execution logs failed job_id=%d: %v", jobID, err)
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"code": 500, "msg": "服务端处理失败"})
|
||||
}
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"code": 200, "data": response})
|
||||
}
|
||||
|
||||
func positiveQueryInt(value string, fallback int) (int, error) {
|
||||
if strings.TrimSpace(value) == "" {
|
||||
return fallback, nil
|
||||
}
|
||||
parsed, err := strconv.Atoi(value)
|
||||
if err != nil || parsed < 1 {
|
||||
return 0, errors.New("invalid positive integer")
|
||||
}
|
||||
return parsed, nil
|
||||
}
|
||||
|
||||
func optionalTime(value string) (*time.Time, error) {
|
||||
if strings.TrimSpace(value) == "" {
|
||||
return nil, nil
|
||||
}
|
||||
parsed, err := time.Parse(time.RFC3339, value)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &parsed, nil
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"go-admin/app/goauto/shopeeproduct"
|
||||
"go-admin/app/goauto/sybimport"
|
||||
)
|
||||
|
||||
@@ -12,8 +13,10 @@ import (
|
||||
// 字典 key 可以配置到 自动任务 调用目标 中;
|
||||
func InitJob() {
|
||||
jobList = map[string]JobExec{
|
||||
"ExamplesOne": ExamplesOne{},
|
||||
sybimport.HourlySyncInvokeTarget: sybimport.HourlySyncJob{},
|
||||
"ExamplesOne": ExamplesOne{},
|
||||
sybimport.HourlySyncInvokeTarget: sybimport.HourlySyncJob{},
|
||||
sybimport.SpecAIParseInvokeTarget: sybimport.ScheduledSpecAIParseJob{},
|
||||
shopeeproduct.SpecAutoMatchInvokeTarget: shopeeproduct.ScheduledAutoMatchJob{},
|
||||
// ...
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
package jobs
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
log "github.com/go-admin-team/go-admin-core/logger"
|
||||
"github.com/google/uuid"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/jobs/models"
|
||||
)
|
||||
|
||||
const (
|
||||
executionErrorTargetMissing = "JOB_TARGET_NOT_FOUND"
|
||||
executionErrorExecFailed = "JOB_EXECUTION_FAILED"
|
||||
executionErrorHTTPFailed = "JOB_HTTP_FAILED"
|
||||
executionErrorPanicked = "JOB_EXECUTION_PANICKED"
|
||||
executionErrorInterrupted = "JOB_INTERRUPTED"
|
||||
)
|
||||
|
||||
type executionFailure struct {
|
||||
code string
|
||||
message string
|
||||
cause error
|
||||
}
|
||||
|
||||
func (failure *executionFailure) Error() string {
|
||||
if failure.cause != nil {
|
||||
return failure.cause.Error()
|
||||
}
|
||||
return failure.message
|
||||
}
|
||||
|
||||
func newExecutionFailure(code, message string, cause error) error {
|
||||
return &executionFailure{code: code, message: message, cause: cause}
|
||||
}
|
||||
|
||||
func runWithExecutionLog(db *gorm.DB, core JobCore, execute func() error) (executionErr error) {
|
||||
startedAt := time.Now().UTC()
|
||||
record := &models.SysJobExecutionLog{
|
||||
ExecutionID: uuid.NewString(), JobID: core.JobId,
|
||||
JobNameSnapshot: core.Name, InvokeTargetSnapshot: core.InvokeTarget,
|
||||
TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionRunning,
|
||||
StartedAt: startedAt,
|
||||
}
|
||||
created := false
|
||||
if db != nil {
|
||||
if err := db.WithContext(context.Background()).Create(record).Error; err != nil {
|
||||
log.Errorf("[Job] execution log create failed job_id=%d: %v", core.JobId, err)
|
||||
} else {
|
||||
created = true
|
||||
}
|
||||
}
|
||||
|
||||
defer func() {
|
||||
if recovered := recover(); recovered != nil {
|
||||
executionErr = newExecutionFailure(executionErrorPanicked, "任务执行异常中断", nil)
|
||||
if created {
|
||||
finishExecutionLog(db, core.JobId, record, startedAt, executionErr)
|
||||
}
|
||||
return
|
||||
}
|
||||
if created {
|
||||
finishExecutionLog(db, core.JobId, record, startedAt, executionErr)
|
||||
}
|
||||
}()
|
||||
executionErr = execute()
|
||||
return executionErr
|
||||
}
|
||||
|
||||
func finishExecutionLog(db *gorm.DB, jobID int, record *models.SysJobExecutionLog, startedAt time.Time, executionErr error) {
|
||||
finishedAt := time.Now().UTC()
|
||||
updates := map[string]any{
|
||||
"status": models.JobExecutionSucceeded, "finished_at": finishedAt,
|
||||
"duration_ms": finishedAt.Sub(startedAt).Milliseconds(), "error_code": "", "error_message": "",
|
||||
}
|
||||
if executionErr != nil {
|
||||
code, message := publicExecutionFailure(executionErr)
|
||||
updates["status"] = models.JobExecutionFailed
|
||||
updates["error_code"] = code
|
||||
updates["error_message"] = message
|
||||
}
|
||||
if err := db.WithContext(context.Background()).Model(&models.SysJobExecutionLog{}).
|
||||
Where("id = ? AND status = ?", record.ID, models.JobExecutionRunning).Updates(updates).Error; err != nil {
|
||||
log.Errorf("[Job] execution log finish failed job_id=%d execution_id=%s: %v", jobID, record.ExecutionID, err)
|
||||
}
|
||||
}
|
||||
|
||||
func publicExecutionFailure(err error) (string, string) {
|
||||
var failure *executionFailure
|
||||
if errors.As(err, &failure) {
|
||||
return failure.code, failure.message
|
||||
}
|
||||
return executionErrorExecFailed, "任务执行失败,请查看受控服务日志"
|
||||
}
|
||||
|
||||
// RecoverInterruptedExecutionLogs closes invocations left running by the
|
||||
// previous process. Production currently runs one scheduler per database.
|
||||
func RecoverInterruptedExecutionLogs(db *gorm.DB) error {
|
||||
if db == nil {
|
||||
return nil
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
var records []models.SysJobExecutionLog
|
||||
if err := db.WithContext(context.Background()).Where("status = ?", models.JobExecutionRunning).Find(&records).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return db.WithContext(context.Background()).Transaction(func(tx *gorm.DB) error {
|
||||
for _, record := range records {
|
||||
duration := now.Sub(record.StartedAt).Milliseconds()
|
||||
if duration < 0 {
|
||||
duration = 0
|
||||
}
|
||||
if err := tx.Model(&models.SysJobExecutionLog{}).
|
||||
Where("id = ? AND status = ?", record.ID, models.JobExecutionRunning).Updates(map[string]any{
|
||||
"status": models.JobExecutionInterrupted, "finished_at": now,
|
||||
"duration_ms": duration, "error_code": executionErrorInterrupted,
|
||||
"error_message": "服务重启前任务未完成",
|
||||
}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func missingExecutionTarget(target string) error {
|
||||
return newExecutionFailure(executionErrorTargetMissing, "任务调用目标未注册", fmt.Errorf("job target %q is not registered", target))
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
package jobs
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/jobs/models"
|
||||
)
|
||||
|
||||
func jobExecutionTestDB(t *testing.T) *gorm.DB {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.AutoMigrate(&models.SysJobExecutionLog{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
func TestRunWithExecutionLogRecordsSuccessAndSanitizedFailure(t *testing.T) {
|
||||
db := jobExecutionTestDB(t)
|
||||
core := JobCore{JobId: 7, Name: "测试任务", InvokeTarget: "TestTarget"}
|
||||
if err := runWithExecutionLog(db, core, func() error { return nil }); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rawSecret := "token=secret-value productSpec=private"
|
||||
if err := runWithExecutionLog(db, core, func() error { return errors.New(rawSecret) }); err == nil {
|
||||
t.Fatal("failed execution must return its error")
|
||||
}
|
||||
var records []models.SysJobExecutionLog
|
||||
if err := db.Order("id asc").Find(&records).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(records) != 2 || records[0].Status != models.JobExecutionSucceeded || records[1].Status != models.JobExecutionFailed {
|
||||
t.Fatalf("unexpected records: %+v", records)
|
||||
}
|
||||
if records[1].ErrorCode != executionErrorExecFailed || records[1].ErrorMessage == rawSecret || records[1].ErrorMessage == "" {
|
||||
t.Fatalf("failure was not safely summarized: %+v", records[1])
|
||||
}
|
||||
if records[0].FinishedAt == nil || records[1].FinishedAt == nil || records[0].ExecutionID == records[1].ExecutionID {
|
||||
t.Fatal("execution lifecycle or unique IDs were not recorded")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecoverInterruptedExecutionLogs(t *testing.T) {
|
||||
db := jobExecutionTestDB(t)
|
||||
started := time.Now().UTC().Add(-2 * time.Second)
|
||||
record := models.SysJobExecutionLog{
|
||||
ExecutionID: "00000000-0000-4000-8000-000000000010", JobID: 9,
|
||||
JobNameSnapshot: "中断任务", InvokeTargetSnapshot: "Interrupted",
|
||||
TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionRunning, StartedAt: started,
|
||||
}
|
||||
if err := db.Create(&record).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := RecoverInterruptedExecutionLogs(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.First(&record, record.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record.Status != models.JobExecutionInterrupted || record.FinishedAt == nil || record.ErrorCode != executionErrorInterrupted || record.DurationMS < 1000 {
|
||||
t.Fatalf("record was not safely interrupted: %+v", record)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMissingExecutionTargetHasPublicSafeMessage(t *testing.T) {
|
||||
code, message := publicExecutionFailure(missingExecutionTarget("SecretTarget"))
|
||||
if code != executionErrorTargetMissing || message != "任务调用目标未注册" {
|
||||
t.Fatalf("unexpected target failure %s %s", code, message)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecJobRecordsMissingTargetAsFailure(t *testing.T) {
|
||||
db := jobExecutionTestDB(t)
|
||||
previousJobList := jobList
|
||||
jobList = map[string]JobExec{}
|
||||
t.Cleanup(func() { jobList = previousJobList })
|
||||
|
||||
job := &ExecJob{JobCore: JobCore{JobId: 10, Name: "未注册任务", InvokeTarget: "MissingTarget"}, DB: db}
|
||||
job.Run()
|
||||
|
||||
var record models.SysJobExecutionLog
|
||||
if err := db.First(&record).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record.Status != models.JobExecutionFailed || record.ErrorCode != executionErrorTargetMissing || record.ErrorMessage != "任务调用目标未注册" {
|
||||
t.Fatalf("missing target was not safely recorded: %+v", record)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunWithExecutionLogSafelyRecordsPanic(t *testing.T) {
|
||||
db := jobExecutionTestDB(t)
|
||||
err := runWithExecutionLog(db, JobCore{JobId: 11, Name: "异常任务", InvokeTarget: "Panic"}, func() error {
|
||||
panic("provider-secret")
|
||||
})
|
||||
if code, message := publicExecutionFailure(err); code != executionErrorPanicked || message != "任务执行异常中断" {
|
||||
t.Fatalf("panic was not returned as a safe failure: %s %s", code, message)
|
||||
}
|
||||
var record models.SysJobExecutionLog
|
||||
if err := db.First(&record).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record.Status != models.JobExecutionFailed || record.ErrorCode != executionErrorPanicked || record.ErrorMessage != "任务执行异常中断" {
|
||||
t.Fatalf("panic was not safely persisted: %+v", record)
|
||||
}
|
||||
}
|
||||
+33
-22
@@ -33,6 +33,7 @@ type JobCore struct {
|
||||
// HttpJob 任务类型 http
|
||||
type HttpJob struct {
|
||||
JobCore
|
||||
DB *gorm.DB
|
||||
}
|
||||
|
||||
type ExecJob struct {
|
||||
@@ -42,14 +43,16 @@ type ExecJob struct {
|
||||
|
||||
func (e *ExecJob) Run() {
|
||||
startTime := time.Now()
|
||||
var obj = jobList[e.InvokeTarget]
|
||||
if obj == nil {
|
||||
log.Warn("[Job] ExecJob Run job nil")
|
||||
return
|
||||
}
|
||||
err := CallExecWithDB(obj.(JobExec), e.DB, e.Args)
|
||||
err := runWithExecutionLog(e.DB, e.JobCore, func() error {
|
||||
obj := jobList[e.InvokeTarget]
|
||||
if obj == nil {
|
||||
return missingExecutionTarget(e.InvokeTarget)
|
||||
}
|
||||
return CallExecWithDB(obj, e.DB, e.Args)
|
||||
})
|
||||
if err != nil {
|
||||
log.Errorf("[Job] JobCore %s failed: %v", e.Name, err)
|
||||
code, _ := publicExecutionFailure(err)
|
||||
log.Errorf("[Job] JobCore %s failed error_code=%s", e.Name, code)
|
||||
return
|
||||
}
|
||||
// 结束时间
|
||||
@@ -68,22 +71,26 @@ func (e *ExecJob) Run() {
|
||||
func (h *HttpJob) Run() {
|
||||
|
||||
startTime := time.Now()
|
||||
var count = 0
|
||||
var err error
|
||||
var str string
|
||||
/* 循环 */
|
||||
LOOP:
|
||||
if count < retryCount {
|
||||
/* 跳过迭代 */
|
||||
str, err = pkg.Get(h.InvokeTarget)
|
||||
if err != nil {
|
||||
// 如果失败暂停一段时间重试
|
||||
log.Warnf("[Job] mission failed! %v", err)
|
||||
log.Warnf("[Job] Retry after the task fails %d seconds! %s \n", (count+1)*5, str)
|
||||
time.Sleep(time.Duration(count+1) * 5 * time.Second)
|
||||
count = count + 1
|
||||
goto LOOP
|
||||
err := runWithExecutionLog(h.DB, h.JobCore, func() error {
|
||||
var lastErr error
|
||||
for count := 0; count < retryCount; count++ {
|
||||
_, requestErr := pkg.Get(h.InvokeTarget)
|
||||
if requestErr == nil {
|
||||
return nil
|
||||
}
|
||||
lastErr = requestErr
|
||||
log.Warnf("[Job] HTTP mission failed attempt=%d", count+1)
|
||||
if count+1 < retryCount {
|
||||
log.Warnf("[Job] Retry after the task fails %d seconds!\n", (count+1)*5)
|
||||
time.Sleep(time.Duration(count+1) * 5 * time.Second)
|
||||
}
|
||||
}
|
||||
return newExecutionFailure(executionErrorHTTPFailed, "HTTP 任务请求失败", lastErr)
|
||||
})
|
||||
if err != nil {
|
||||
code, _ := publicExecutionFailure(err)
|
||||
log.Errorf("[Job] JobCore %s failed error_code=%s", h.Name, code)
|
||||
return
|
||||
}
|
||||
// 结束时间
|
||||
endTime := time.Now()
|
||||
@@ -102,6 +109,9 @@ func Setup(dbs map[string]*gorm.DB) {
|
||||
fmt.Println(time.Now().Format(timeFormat), " [INFO] JobCore Starting...")
|
||||
|
||||
for k, db := range dbs {
|
||||
if err := RecoverInterruptedExecutionLogs(db); err != nil {
|
||||
log.Errorf("[Job] recover interrupted execution logs failed: %v", err)
|
||||
}
|
||||
sdk.Runtime.SetCrontab(k, cronjob.NewWithSeconds())
|
||||
setup(k, db)
|
||||
}
|
||||
@@ -127,6 +137,7 @@ func setup(key string, db *gorm.DB) {
|
||||
for i := 0; i < len(jobList); i++ {
|
||||
if jobList[i].JobType == 1 {
|
||||
j := &HttpJob{}
|
||||
j.DB = db
|
||||
j.InvokeTarget = jobList[i].InvokeTarget
|
||||
j.CronExpression = jobList[i].CronExpression
|
||||
j.JobId = jobList[i].JobId
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
package models
|
||||
|
||||
import "time"
|
||||
|
||||
const (
|
||||
JobExecutionRunning = "running"
|
||||
JobExecutionSucceeded = "succeeded"
|
||||
JobExecutionFailed = "failed"
|
||||
JobExecutionInterrupted = "interrupted"
|
||||
JobTriggerScheduled = "scheduled"
|
||||
)
|
||||
|
||||
// SysJobExecutionLog stores one scheduler invocation. Job arguments and raw
|
||||
// provider responses are deliberately excluded from this audit record.
|
||||
type SysJobExecutionLog struct {
|
||||
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
|
||||
ExecutionID string `json:"executionId" gorm:"size:36;not null;uniqueIndex:ux_sys_job_execution_id"`
|
||||
JobID int `json:"jobId" gorm:"not null;index:idx_sys_job_execution_job_started,priority:1"`
|
||||
JobNameSnapshot string `json:"jobName" gorm:"size:255;not null"`
|
||||
InvokeTargetSnapshot string `json:"invokeTarget" gorm:"size:255;not null"`
|
||||
TriggerType string `json:"triggerType" gorm:"size:16;not null"`
|
||||
Status string `json:"status" gorm:"size:16;not null;index:idx_sys_job_execution_status_started,priority:1"`
|
||||
StartedAt time.Time `json:"startedAt" gorm:"not null;index:idx_sys_job_execution_job_started,priority:2;index:idx_sys_job_execution_status_started,priority:2"`
|
||||
FinishedAt *time.Time `json:"finishedAt,omitempty"`
|
||||
DurationMS int64 `json:"durationMs" gorm:"not null;default:0"`
|
||||
ErrorCode string `json:"errorCode,omitempty" gorm:"size:64;not null;default:''"`
|
||||
ErrorMessage string `json:"errorMessage,omitempty" gorm:"size:500;not null;default:''"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func (SysJobExecutionLog) TableName() string { return "sys_job_execution_log" }
|
||||
@@ -24,6 +24,8 @@ func registerSysJobRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddlew
|
||||
list := make([]models2.SysJob, 0)
|
||||
return &list
|
||||
}))
|
||||
jobAPI := apis.SysJob{}
|
||||
r.GET("/:id/execution-logs", actions.PermissionAction(), jobAPI.ListExecutionLogs)
|
||||
r.GET("/:id", actions.PermissionAction(), actions.ViewAction(new(dto2.SysJobById), func() interface{} {
|
||||
return &dto2.SysJobItem{}
|
||||
}))
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/jobs/models"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrExecutionLogInvalidRequest = errors.New("invalid execution log request")
|
||||
ErrExecutionLogJobNotFound = errors.New("scheduled job not found")
|
||||
)
|
||||
|
||||
type ExecutionLogListRequest struct {
|
||||
Page int
|
||||
PageSize int
|
||||
Status string
|
||||
StartedFrom *time.Time
|
||||
StartedTo *time.Time
|
||||
}
|
||||
|
||||
type ExecutionLogJob struct {
|
||||
JobID int `json:"jobId"`
|
||||
JobName string `json:"jobName"`
|
||||
InvokeTarget string `json:"invokeTarget"`
|
||||
Deleted bool `json:"deleted"`
|
||||
}
|
||||
|
||||
type ExecutionLogListResponse struct {
|
||||
Job ExecutionLogJob `json:"job"`
|
||||
Items []models.SysJobExecutionLog `json:"items"`
|
||||
Total int64 `json:"total"`
|
||||
Page int `json:"page"`
|
||||
PageSize int `json:"pageSize"`
|
||||
}
|
||||
|
||||
type ExecutionLogService struct{ db *gorm.DB }
|
||||
|
||||
func NewExecutionLogService(db *gorm.DB) *ExecutionLogService {
|
||||
return &ExecutionLogService{db: db}
|
||||
}
|
||||
|
||||
func (service *ExecutionLogService) List(ctx context.Context, jobID int, request ExecutionLogListRequest) (ExecutionLogListResponse, error) {
|
||||
if service.db == nil || jobID < 1 {
|
||||
return ExecutionLogListResponse{}, fmt.Errorf("%w: jobId 无效", ErrExecutionLogInvalidRequest)
|
||||
}
|
||||
if request.Page < 1 {
|
||||
request.Page = 1
|
||||
}
|
||||
if request.PageSize < 1 {
|
||||
request.PageSize = 20
|
||||
}
|
||||
if request.PageSize > 100 {
|
||||
return ExecutionLogListResponse{}, fmt.Errorf("%w: pageSize 必须是 1 到 100 的整数", ErrExecutionLogInvalidRequest)
|
||||
}
|
||||
if request.Status != "" && !validExecutionStatus(request.Status) {
|
||||
return ExecutionLogListResponse{}, fmt.Errorf("%w: status 无效", ErrExecutionLogInvalidRequest)
|
||||
}
|
||||
if request.StartedFrom != nil && request.StartedTo != nil && request.StartedFrom.After(*request.StartedTo) {
|
||||
return ExecutionLogListResponse{}, fmt.Errorf("%w: 开始时间范围无效", ErrExecutionLogInvalidRequest)
|
||||
}
|
||||
|
||||
var job models.SysJob
|
||||
if err := service.db.WithContext(ctx).Unscoped().First(&job, jobID).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ExecutionLogListResponse{}, ErrExecutionLogJobNotFound
|
||||
}
|
||||
return ExecutionLogListResponse{}, err
|
||||
}
|
||||
|
||||
query := service.db.WithContext(ctx).Model(&models.SysJobExecutionLog{}).Where("job_id = ?", jobID)
|
||||
if request.Status != "" {
|
||||
query = query.Where("status = ?", request.Status)
|
||||
}
|
||||
if request.StartedFrom != nil {
|
||||
query = query.Where("started_at >= ?", request.StartedFrom.UTC())
|
||||
}
|
||||
if request.StartedTo != nil {
|
||||
query = query.Where("started_at <= ?", request.StartedTo.UTC())
|
||||
}
|
||||
var total int64
|
||||
if err := query.Count(&total).Error; err != nil {
|
||||
return ExecutionLogListResponse{}, err
|
||||
}
|
||||
items := make([]models.SysJobExecutionLog, 0, request.PageSize)
|
||||
if err := query.Order("started_at DESC, id DESC").Offset((request.Page - 1) * request.PageSize).Limit(request.PageSize).Find(&items).Error; err != nil {
|
||||
return ExecutionLogListResponse{}, err
|
||||
}
|
||||
return ExecutionLogListResponse{
|
||||
Job: ExecutionLogJob{JobID: job.JobId, JobName: job.JobName, InvokeTarget: job.InvokeTarget, Deleted: job.DeletedAt.Valid},
|
||||
Items: items, Total: total, Page: request.Page, PageSize: request.PageSize,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func validExecutionStatus(status string) bool {
|
||||
switch status {
|
||||
case models.JobExecutionRunning, models.JobExecutionSucceeded, models.JobExecutionFailed, models.JobExecutionInterrupted:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/jobs/models"
|
||||
)
|
||||
|
||||
func executionLogTestDB(t *testing.T) *gorm.DB {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.AutoMigrate(&models.SysJob{}, &models.SysJobExecutionLog{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
func TestExecutionLogListFiltersPaginatesAndKeepsDeletedJob(t *testing.T) {
|
||||
db := executionLogTestDB(t)
|
||||
job := models.SysJob{JobName: "测试任务", InvokeTarget: "TestTarget"}
|
||||
if err := db.Create(&job).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
base := time.Date(2026, 9, 2, 10, 0, 0, 0, time.UTC)
|
||||
records := []models.SysJobExecutionLog{
|
||||
{ExecutionID: "00000000-0000-4000-8000-000000000001", JobID: job.JobId, JobNameSnapshot: job.JobName, InvokeTargetSnapshot: job.InvokeTarget, TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionSucceeded, StartedAt: base},
|
||||
{ExecutionID: "00000000-0000-4000-8000-000000000002", JobID: job.JobId, JobNameSnapshot: job.JobName, InvokeTargetSnapshot: job.InvokeTarget, TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionFailed, StartedAt: base.Add(time.Hour)},
|
||||
{ExecutionID: "00000000-0000-4000-8000-000000000003", JobID: job.JobId, JobNameSnapshot: job.JobName, InvokeTargetSnapshot: job.InvokeTarget, TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionFailed, StartedAt: base.Add(2 * time.Hour)},
|
||||
}
|
||||
if err := db.Create(&records).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
from, to := base.Add(30*time.Minute), base.Add(3*time.Hour)
|
||||
result, err := NewExecutionLogService(db).List(context.Background(), job.JobId, ExecutionLogListRequest{
|
||||
Page: 1, PageSize: 1, Status: models.JobExecutionFailed, StartedFrom: &from, StartedTo: &to,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if result.Total != 2 || len(result.Items) != 1 || result.Items[0].ExecutionID != records[2].ExecutionID {
|
||||
t.Fatalf("unexpected filtered page: %+v", result)
|
||||
}
|
||||
if err := db.Delete(&job).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
deleted, err := NewExecutionLogService(db).List(context.Background(), job.JobId, ExecutionLogListRequest{Page: 1, PageSize: 20})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !deleted.Job.Deleted || deleted.Job.JobName != job.JobName || deleted.Total != 3 {
|
||||
t.Fatalf("deleted job history unavailable: %+v", deleted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecutionLogListValidationAndNotFound(t *testing.T) {
|
||||
db := executionLogTestDB(t)
|
||||
service := NewExecutionLogService(db)
|
||||
if _, err := service.List(context.Background(), 0, ExecutionLogListRequest{}); !errors.Is(err, ErrExecutionLogInvalidRequest) {
|
||||
t.Fatalf("invalid job id error = %v", err)
|
||||
}
|
||||
if _, err := service.List(context.Background(), 1, ExecutionLogListRequest{PageSize: 101}); !errors.Is(err, ErrExecutionLogInvalidRequest) {
|
||||
t.Fatalf("invalid page size error = %v", err)
|
||||
}
|
||||
job := models.SysJob{JobName: "测试任务", InvokeTarget: "TestTarget"}
|
||||
if err := db.Create(&job).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := service.List(context.Background(), job.JobId, ExecutionLogListRequest{Status: "unknown"}); !errors.Is(err, ErrExecutionLogInvalidRequest) {
|
||||
t.Fatalf("invalid status error = %v", err)
|
||||
}
|
||||
from, to := time.Now(), time.Now().Add(-time.Hour)
|
||||
if _, err := service.List(context.Background(), job.JobId, ExecutionLogListRequest{StartedFrom: &from, StartedTo: &to}); !errors.Is(err, ErrExecutionLogInvalidRequest) {
|
||||
t.Fatalf("invalid range error = %v", err)
|
||||
}
|
||||
if _, err := service.List(context.Background(), 999, ExecutionLogListRequest{}); !errors.Is(err, ErrExecutionLogJobNotFound) {
|
||||
t.Fatalf("not found error = %v", err)
|
||||
}
|
||||
}
|
||||
@@ -62,6 +62,7 @@ func (e *SysJob) StartJob(c *dto.GeneralGetDto) error {
|
||||
|
||||
if data.JobType == 1 {
|
||||
var j = &jobs.HttpJob{}
|
||||
j.DB = e.Orm.WithContext(context.Background())
|
||||
j.InvokeTarget = data.InvokeTarget
|
||||
j.CronExpression = data.CronExpression
|
||||
j.JobId = data.JobId
|
||||
|
||||
@@ -54,6 +54,12 @@ var (
|
||||
}
|
||||
)
|
||||
|
||||
// The synchronous Admin AI endpoints allow a provider timeout of up to 600
|
||||
// seconds and the browser waits 610 seconds. Keep the HTTP server alive a
|
||||
// little longer so it can return the domain response instead of truncating
|
||||
// the connection and surfacing a proxy-level 502.
|
||||
const minimumAPIWriteTimeout = 620 * time.Second
|
||||
|
||||
var AppRouters = make([]func(), 0)
|
||||
|
||||
func init() {
|
||||
@@ -125,11 +131,15 @@ func run() error {
|
||||
)
|
||||
}
|
||||
|
||||
writeTimeout, err := validatedAPIWriteTimeout(config.ApplicationConfig.WriterTimeout)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
srv := &http.Server{
|
||||
Addr: fmt.Sprintf("%s:%d", config.ApplicationConfig.Host, config.ApplicationConfig.Port),
|
||||
Handler: sdk.Runtime.GetEngine(),
|
||||
ReadTimeout: time.Duration(config.ApplicationConfig.ReadTimeout) * time.Second,
|
||||
WriteTimeout: time.Duration(config.ApplicationConfig.WriterTimeout) * time.Second,
|
||||
WriteTimeout: writeTimeout,
|
||||
}
|
||||
|
||||
go func() {
|
||||
@@ -193,6 +203,14 @@ func run() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func validatedAPIWriteTimeout(seconds int) (time.Duration, error) {
|
||||
timeout := time.Duration(seconds) * time.Second
|
||||
if timeout < minimumAPIWriteTimeout {
|
||||
return 0, fmt.Errorf("application writetimeout must be at least %s for synchronous AI requests", minimumAPIWriteTimeout)
|
||||
}
|
||||
return timeout, nil
|
||||
}
|
||||
|
||||
type policyLoader interface {
|
||||
LoadPolicy() error
|
||||
}
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestValidatedAPIWriteTimeoutProtectsSynchronousAIRequests(t *testing.T) {
|
||||
if _, err := validatedAPIWriteTimeout(2); err == nil {
|
||||
t.Fatal("two-second write timeout must be rejected")
|
||||
}
|
||||
got, err := validatedAPIWriteTimeout(620)
|
||||
if err != nil {
|
||||
t.Fatalf("620-second write timeout should be accepted: %v", err)
|
||||
}
|
||||
if got != 620*time.Second {
|
||||
t.Fatalf("write timeout = %s, want 620s", got)
|
||||
}
|
||||
}
|
||||
@@ -38,11 +38,11 @@ func TestEnsurePurchaserRoleAndPolicies(t *testing.T) {
|
||||
if count != int64(len(access.PurchaserAPIs())) {
|
||||
t.Fatalf("got %d policies, want %d", count, len(access.PurchaserAPIs()))
|
||||
}
|
||||
var forbidden int64
|
||||
var allowed int64
|
||||
db.Model(&purchaserCasbinRule{}).
|
||||
Where("v0 = ? AND v1 = ? AND v2 = ?", access.RolePurchaser, "/api/admin/v1/syb-products/import", "POST").
|
||||
Count(&forbidden)
|
||||
if forbidden != 0 {
|
||||
t.Fatal("manual SYB import must remain administrator-only")
|
||||
Count(&allowed)
|
||||
if allowed != 1 {
|
||||
t.Fatal("manual SYB import must follow the current purchaser permission matrix")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
package version_local
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"runtime"
|
||||
|
||||
goautomigrations "go-admin/app/goauto/migrations"
|
||||
"go-admin/app/goauto/shopeeproduct"
|
||||
jobsmodels "go-admin/app/jobs/models"
|
||||
"go-admin/cmd/migrate/migration"
|
||||
common "go-admin/common/models"
|
||||
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func init() {
|
||||
_, fileName, _, _ := runtime.Caller(0)
|
||||
migration.Migrate.SetVersion(migration.GetFilename(fileName), migrateShopeeSpecAutoMatch)
|
||||
}
|
||||
|
||||
func migrateShopeeSpecAutoMatch(db *gorm.DB, version string) error {
|
||||
return db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := goautomigrations.Migrate(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ensureShopeeSpecAutoMatchJob(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
}
|
||||
|
||||
func ensureShopeeSpecAutoMatchJob(db *gorm.DB) error {
|
||||
var existing jobsmodels.SysJob
|
||||
err := db.Where("invoke_target = ?", shopeeproduct.SpecAutoMatchInvokeTarget).First(&existing).Error
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return err
|
||||
}
|
||||
return db.Create(&jobsmodels.SysJob{
|
||||
JobName: "蝦皮规格自动匹配", JobGroup: "GoAuto", JobType: 2,
|
||||
CronExpression: "0 15 * * * *", InvokeTarget: shopeeproduct.SpecAutoMatchInvokeTarget,
|
||||
Args: `{"batchLimit":20}`, MisfirePolicy: 1, Concurrent: 1, Status: 1,
|
||||
}).Error
|
||||
}
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
package version_local
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"go-admin/app/goauto/shopeeproduct"
|
||||
jobsmodels "go-admin/app/jobs/models"
|
||||
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func TestEnsureShopeeSpecAutoMatchJobIsDisabledIdempotentAndPreservesChanges(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open("file:shopee-spec-auto-match-job?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.AutoMigrate(&jobsmodels.SysJob{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ensureShopeeSpecAutoMatchJob(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var job jobsmodels.SysJob
|
||||
if err := db.Where("invoke_target = ?", shopeeproduct.SpecAutoMatchInvokeTarget).First(&job).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if job.Status != 1 || job.CronExpression != "0 15 * * * *" || job.Args != `{"batchLimit":20}` {
|
||||
t.Fatalf("unexpected seed: %+v", job)
|
||||
}
|
||||
if err := db.Model(&job).Updates(map[string]any{"status": 2, "cron_expression": "0 30 * * * *", "args": `{"batchLimit":5}`}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ensureShopeeSpecAutoMatchJob(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var count int64
|
||||
if err := db.Model(&jobsmodels.SysJob{}).Where("invoke_target = ?", shopeeproduct.SpecAutoMatchInvokeTarget).Count(&count).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Fatalf("count=%d", count)
|
||||
}
|
||||
if err := db.Where("invoke_target = ?", shopeeproduct.SpecAutoMatchInvokeTarget).First(&job).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if job.Status != 2 || job.CronExpression != "0 30 * * * *" || job.Args != `{"batchLimit":5}` {
|
||||
t.Fatalf("admin changes overwritten: %+v", job)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
package version_local
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"runtime"
|
||||
|
||||
goautomigrations "go-admin/app/goauto/migrations"
|
||||
"go-admin/app/goauto/sybimport"
|
||||
jobsmodels "go-admin/app/jobs/models"
|
||||
"go-admin/cmd/migrate/migration"
|
||||
common "go-admin/common/models"
|
||||
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func init() {
|
||||
_, fileName, _, _ := runtime.Caller(0)
|
||||
migration.Migrate.SetVersion(migration.GetFilename(fileName), migrateSYBSpecAIParse)
|
||||
}
|
||||
|
||||
func migrateSYBSpecAIParse(db *gorm.DB, version string) error {
|
||||
return db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := goautomigrations.Migrate(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ensureSYBSpecAIParseJob(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
}
|
||||
|
||||
func ensureSYBSpecAIParseJob(db *gorm.DB) error {
|
||||
var existing jobsmodels.SysJob
|
||||
err := db.Where("invoke_target = ?", sybimport.SpecAIParseInvokeTarget).First(&existing).Error
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return err
|
||||
}
|
||||
return db.Create(&jobsmodels.SysJob{
|
||||
JobName: "SYB 异常规格 AI 解析", JobGroup: "GoAuto", JobType: 2,
|
||||
CronExpression: "0 5 * * * *", InvokeTarget: sybimport.SpecAIParseInvokeTarget,
|
||||
Args: `{"batchLimit":20}`, MisfirePolicy: 1, Concurrent: 1, Status: 1,
|
||||
}).Error
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
package version_local
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"go-admin/app/goauto/sybimport"
|
||||
jobsmodels "go-admin/app/jobs/models"
|
||||
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func TestEnsureSYBSpecAIParseJobIsDisabledIdempotentAndPreservesChanges(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.AutoMigrate(&jobsmodels.SysJob{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ensureSYBSpecAIParseJob(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var job jobsmodels.SysJob
|
||||
if err := db.Where("invoke_target = ?", sybimport.SpecAIParseInvokeTarget).First(&job).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if job.Status != 1 || job.CronExpression != "0 5 * * * *" || job.Args != `{"batchLimit":20}` {
|
||||
t.Fatalf("unexpected initial job: %+v", job)
|
||||
}
|
||||
if err := db.Model(&job).Updates(map[string]any{"status": 2, "cron_expression": "0 7 * * * *", "args": `{"batchLimit":7}`}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ensureSYBSpecAIParseJob(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var count int64
|
||||
if err := db.Model(&jobsmodels.SysJob{}).Where("invoke_target = ?", sybimport.SpecAIParseInvokeTarget).Count(&count).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Fatalf("job count=%d, want 1", count)
|
||||
}
|
||||
if err := db.First(&job, job.JobId).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if job.Status != 2 || job.CronExpression != "0 7 * * * *" || job.Args != `{"batchLimit":7}` {
|
||||
t.Fatalf("existing administrator changes were overwritten: %+v", job)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package version_local
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"runtime"
|
||||
|
||||
jobsmodels "go-admin/app/jobs/models"
|
||||
"go-admin/cmd/migrate/migration"
|
||||
migrationmodels "go-admin/cmd/migrate/migration/models"
|
||||
common "go-admin/common/models"
|
||||
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
const jobExecutionLogsAPIPath = "/api/v1/sysjob/:id/execution-logs"
|
||||
|
||||
func init() {
|
||||
_, fileName, _, _ := runtime.Caller(0)
|
||||
migration.Migrate.SetVersion(migration.GetFilename(fileName), migrateSysJobExecutionLog)
|
||||
}
|
||||
|
||||
func migrateSysJobExecutionLog(db *gorm.DB, version string) error {
|
||||
return db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := ensureSysJobExecutionLog(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
}
|
||||
|
||||
func ensureSysJobExecutionLog(db *gorm.DB) error {
|
||||
if err := db.AutoMigrate(&jobsmodels.SysJobExecutionLog{}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var logMenu migrationmodels.SysMenu
|
||||
if err := db.Where("menu_name = ?", "JobLog").First(&logMenu).Error; err != nil {
|
||||
return fmt.Errorf("find JobLog menu: %w", err)
|
||||
}
|
||||
api := migrationmodels.SysApi{}
|
||||
if err := db.Where(migrationmodels.SysApi{Path: jobExecutionLogsAPIPath, Action: "GET"}).
|
||||
Attrs(migrationmodels.SysApi{Title: "查询定时任务执行日志", Type: "BUS"}).FirstOrCreate(&api).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if err := db.Model(&logMenu).Association("SysApi").Append(&api); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var roleIDs []int
|
||||
if err := db.Table("sys_role_menu").Where("menu_id = ?", logMenu.MenuId).Distinct().Pluck("role_id", &roleIDs).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(roleIDs) == 0 {
|
||||
return nil
|
||||
}
|
||||
var roles []migrationmodels.SysRole
|
||||
if err := db.Where("role_id IN ?", roleIDs).Find(&roles).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
for _, role := range roles {
|
||||
rule := purchaserCasbinRule{Ptype: "p", V0: role.RoleKey, V1: jobExecutionLogsAPIPath, V2: "GET"}
|
||||
if err := db.Where("ptype = ? AND v0 = ? AND v1 = ? AND v2 = ?", rule.Ptype, rule.V0, rule.V1, rule.V2).
|
||||
FirstOrCreate(&rule).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
package version_local
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
jobsmodels "go-admin/app/jobs/models"
|
||||
migrationmodels "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func TestEnsureSysJobExecutionLogIsIdempotentAndGrantsOnlyBoundRoles(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.AutoMigrate(
|
||||
&migrationmodels.SysMenu{}, &migrationmodels.SysApi{}, &migrationmodels.SysRole{}, &purchaserCasbinRule{},
|
||||
); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
menu := migrationmodels.SysMenu{MenuName: "JobLog", Title: "日志", Path: "/schedule/log", Component: "/schedule/log"}
|
||||
if err := db.Create(&menu).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
bound := migrationmodels.SysRole{RoleName: "日志管理员", RoleKey: "job_logger"}
|
||||
unbound := migrationmodels.SysRole{RoleName: "无日志权限", RoleKey: "no_job_logs"}
|
||||
if err := db.Create(&bound).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Create(&unbound).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Model(&bound).Association("SysMenu").Append(&menu); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
custom := purchaserCasbinRule{Ptype: "p", V0: bound.RoleKey, V1: "/custom", V2: "GET"}
|
||||
if err := db.Create(&custom).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if err := ensureSysJobExecutionLog(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ensureSysJobExecutionLog(db); err != nil {
|
||||
t.Fatalf("migration helper must be repeatable: %v", err)
|
||||
}
|
||||
if !db.Migrator().HasTable(&jobsmodels.SysJobExecutionLog{}) {
|
||||
t.Fatal("execution log table was not created")
|
||||
}
|
||||
var apiCount, menuAPI, boundPolicy, unboundPolicy, customPolicy int64
|
||||
db.Model(&migrationmodels.SysApi{}).Where("path = ? AND action = ?", jobExecutionLogsAPIPath, "GET").Count(&apiCount)
|
||||
var api migrationmodels.SysApi
|
||||
if err := db.Where("path = ? AND action = ?", jobExecutionLogsAPIPath, "GET").First(&api).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
db.Table("sys_menu_api_rule").Where("sys_menu_menu_id = ? AND sys_api_id = ?", menu.MenuId, api.Id).Count(&menuAPI)
|
||||
db.Model(&purchaserCasbinRule{}).Where("v0 = ? AND v1 = ? AND v2 = ?", bound.RoleKey, jobExecutionLogsAPIPath, "GET").Count(&boundPolicy)
|
||||
db.Model(&purchaserCasbinRule{}).Where("v0 = ? AND v1 = ? AND v2 = ?", unbound.RoleKey, jobExecutionLogsAPIPath, "GET").Count(&unboundPolicy)
|
||||
db.Model(&purchaserCasbinRule{}).Where("v0 = ? AND v1 = ?", bound.RoleKey, "/custom").Count(&customPolicy)
|
||||
if apiCount != 1 || menuAPI != 1 || boundPolicy != 1 || unboundPolicy != 0 || customPolicy != 1 {
|
||||
t.Fatalf("unexpected permission state api=%d menu=%d bound=%d unbound=%d custom=%d", apiCount, menuAPI, boundPolicy, unboundPolicy, customPolicy)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnsureSysJobExecutionLogRequiresExistingMenu(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.AutoMigrate(&migrationmodels.SysMenu{}, &migrationmodels.SysApi{}, &migrationmodels.SysRole{}, &purchaserCasbinRule{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ensureSysJobExecutionLog(db); err == nil {
|
||||
t.Fatal("missing JobLog menu must fail closed")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
package version_local
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"go-admin/app/goauto/clientkey"
|
||||
"go-admin/cmd/migrate/migration"
|
||||
migrationmodels "go-admin/cmd/migrate/migration/models"
|
||||
common "go-admin/common/models"
|
||||
"gorm.io/gorm"
|
||||
"runtime"
|
||||
)
|
||||
|
||||
func init() {
|
||||
_, file, _, _ := runtime.Caller(0)
|
||||
migration.Migrate.SetVersion(migration.GetFilename(file), migrateClientAPIKey)
|
||||
}
|
||||
func migrateClientAPIKey(db *gorm.DB, version string) error {
|
||||
if err := db.AutoMigrate(&clientkey.Key{}, &clientkey.Audit{}); err != nil {
|
||||
return err
|
||||
}
|
||||
return db.Transaction(func(tx *gorm.DB) error {
|
||||
var parent migrationmodels.SysMenu
|
||||
if err := tx.Where("menu_name = ?", "GoAutoCollectionManagement").First(&parent).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
child, _, err := upsertGoAutoMenu(tx, migrationmodels.SysMenu{MenuName: "GoAutoClientKeys", Title: "客户端密钥", Icon: "lock", Path: "/client-keys/index", MenuType: "C", Action: "无", ParentId: parent.MenuId, Component: "/goauto/client-keys/index", Sort: 6, Visible: "0", IsFrame: "1"})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err = tx.Model(&child).Update("paths", fmt.Sprintf("/0/%d/%d", parent.MenuId, child.MenuId)).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
var admin migrationmodels.SysRole
|
||||
if err = tx.Where("role_key = ?", "admin").First(&admin).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if err = tx.Model(&admin).Association("SysMenu").Append(&child); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
package version_local
|
||||
|
||||
import (
|
||||
"go-admin/app/goauto/clientkey"
|
||||
migrationmodels "go-admin/cmd/migrate/migration/models"
|
||||
common "go-admin/common/models"
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestClientKeyMigrationOnlyAdminMenu(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sql, _ := db.DB()
|
||||
defer sql.Close()
|
||||
if err = db.AutoMigrate(&migrationmodels.SysRole{}, &migrationmodels.SysMenu{}, &migrationmodels.SysApi{}, &common.Migration{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := migrationmodels.SysRole{RoleKey: "admin"}
|
||||
purchaser := migrationmodels.SysRole{RoleKey: "purchaser"}
|
||||
parent := migrationmodels.SysMenu{MenuName: "GoAutoCollectionManagement"}
|
||||
for _, v := range []any{&admin, &purchaser, &parent} {
|
||||
if err = db.Create(v).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err = migrateClientAPIKey(db, "client-key-test"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !db.Migrator().HasTable(&clientkey.Key{}) || !db.Migrator().HasTable(&clientkey.Audit{}) {
|
||||
t.Fatal("missing tables")
|
||||
}
|
||||
var child migrationmodels.SysMenu
|
||||
if err = db.Where("menu_name = ?", "GoAutoClientKeys").First(&child).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if child.ParentId != parent.MenuId {
|
||||
t.Fatal("wrong menu group")
|
||||
}
|
||||
assertRoleHasMenu(t, db, admin.RoleId, child.MenuId, true)
|
||||
assertRoleHasMenu(t, db, purchaser.RoleId, child.MenuId, false)
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
// Package clientprincipal carries an authenticated client identity, never an
|
||||
// administrator or purchaser JWT. Only the client gateway sets it.
|
||||
package clientprincipal
|
||||
|
||||
import "github.com/gin-gonic/gin"
|
||||
|
||||
const contextKey = "goauto.authenticatedClient"
|
||||
|
||||
type Identity struct {
|
||||
KeyID uint64
|
||||
RequestID string
|
||||
AuthorizedBy uint64
|
||||
}
|
||||
|
||||
func Set(c *gin.Context, id Identity) { c.Set(contextKey, id) }
|
||||
func Get(c *gin.Context) (Identity, bool) {
|
||||
v, ok := c.Get(contextKey)
|
||||
id, valid := v.(Identity)
|
||||
return id, ok && valid && id.KeyID > 0
|
||||
}
|
||||
@@ -24,6 +24,12 @@ import (
|
||||
// LoggerToFile 日志记录到文件
|
||||
func LoggerToFile() gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
// #237: client request/response bodies and one-time credentials must never
|
||||
// enter the legacy operation logger. The client gateway keeps metadata-only audit.
|
||||
if strings.HasPrefix(c.Request.URL.Path, "/api/client/") || strings.HasPrefix(c.Request.URL.Path, "/api/admin/v1/client-keys") {
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
log := api.GetRequestLogger(c)
|
||||
// 开始时间
|
||||
startTime := time.Now()
|
||||
|
||||
@@ -9,7 +9,7 @@ settings:
|
||||
# 端口号
|
||||
port: 8000 # 服务端口号
|
||||
readtimeout: 1
|
||||
writertimeout: 2
|
||||
writertimeout: 620
|
||||
# 数据权限功能开关
|
||||
enabledp: false
|
||||
ssl:
|
||||
|
||||
@@ -9,7 +9,7 @@ settings:
|
||||
# 端口号
|
||||
port: 8000 # 服务端口号
|
||||
readtimeout: 1
|
||||
writertimeout: 2
|
||||
writertimeout: 620
|
||||
# 数据权限功能开关
|
||||
enabledp: false
|
||||
logger:
|
||||
@@ -84,4 +84,4 @@ settings:
|
||||
# blockingTimeout: 5
|
||||
# reclaimInterval: 1
|
||||
locker:
|
||||
redis:
|
||||
redis:
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
import request from '@/utils/request'
|
||||
|
||||
const base = '/api/admin/v1/client-keys'
|
||||
export const listClientKeys = page => request({ url: base, method: 'get', params: { page }, suppressErrorMessage: true })
|
||||
export const clientKeyModules = () => request({ url: `${base}/modules`, method: 'get', suppressErrorMessage: true })
|
||||
export const createClientKey = data => request({ url: base, method: 'post', data, suppressErrorMessage: true })
|
||||
export const editClientKey = (id, data) => request({ url: `${base}/${id}/grants`, method: 'patch', data, suppressErrorMessage: true })
|
||||
export const disableClientKey = (id, version) => request({ url: `${base}/${id}/disable`, method: 'post', data: { version }, suppressErrorMessage: true })
|
||||
@@ -8,16 +8,16 @@ export function createShopeeProduct(data) {
|
||||
return request({ url: '/api/admin/v1/shopee-products', method: 'post', data })
|
||||
}
|
||||
|
||||
export function getShopeeProduct(productId) {
|
||||
return request({ url: `/api/admin/v1/shopee-products/${productId}`, method: 'get' })
|
||||
export function getShopeeProduct(productId, options = {}) {
|
||||
return request({ url: `/api/admin/v1/shopee-products/${productId}`, method: 'get', ...options })
|
||||
}
|
||||
|
||||
export function updateShopeeProduct(productId, data) {
|
||||
return request({ url: `/api/admin/v1/shopee-products/${productId}`, method: 'patch', data })
|
||||
}
|
||||
|
||||
export function linkShopeeProductPdd(productId, data) {
|
||||
return request({ url: `/api/admin/v1/shopee-products/${productId}/link-pdd`, method: 'post', data })
|
||||
export function linkShopeeProductPdd(productId, data, options = {}) {
|
||||
return request({ url: `/api/admin/v1/shopee-products/${productId}/link-pdd`, method: 'post', data, ...options })
|
||||
}
|
||||
|
||||
export function restoreShopeeProduct(productId, data) {
|
||||
@@ -66,8 +66,19 @@ export function suggestShopeeSizeMappings(productId, data) {
|
||||
return request({ url: `/api/admin/v1/shopee-products/${productId}/specs/mapping/suggest-sizes`, method: 'post', data, timeout: aiSuggestTimeoutMs })
|
||||
}
|
||||
|
||||
export function autoMatchShopeeSpecMappings(productId, data) {
|
||||
return request({ url: `/api/admin/v1/shopee-products/${productId}/specs/mapping/auto-match`, method: 'post', data, timeout: aiSuggestTimeoutMs })
|
||||
export function autoMatchShopeeSpecMappings(productId, data, timeoutSeconds) {
|
||||
if (!Number.isInteger(timeoutSeconds) || timeoutSeconds < 16 || timeoutSeconds > 1210) {
|
||||
return Promise.reject(new Error('匹配等待时间不可用,请刷新页面后重试'))
|
||||
}
|
||||
return request({ url: `/api/admin/v1/shopee-products/${productId}/specs/mapping/auto-match`, method: 'post', data, timeout: (timeoutSeconds + 10) * 1000, suppressErrorMessage: true })
|
||||
}
|
||||
|
||||
export function startShopeeSpecAutoMatchRun(data) {
|
||||
return request({ url: '/api/admin/v1/shopee-spec-auto-match/runs', method: 'post', data })
|
||||
}
|
||||
|
||||
export function getLatestShopeeSpecAutoMatchRun() {
|
||||
return request({ url: '/api/admin/v1/shopee-spec-auto-match/runs/latest', method: 'get' })
|
||||
}
|
||||
|
||||
export function batchDeleteShopeeProducts(data) {
|
||||
|
||||
@@ -17,6 +17,15 @@ export function getSysJob(jobId) {
|
||||
})
|
||||
}
|
||||
|
||||
// 查询指定定时任务的持久化执行历史
|
||||
export function listJobExecutionLogs(jobId, query) {
|
||||
return request({
|
||||
url: '/api/v1/sysjob/' + jobId + '/execution-logs',
|
||||
method: 'get',
|
||||
params: query
|
||||
})
|
||||
}
|
||||
|
||||
// 新增SysJob
|
||||
export function addSysJob(data) {
|
||||
return request({
|
||||
@@ -59,4 +68,3 @@ export function startJob(jobId) {
|
||||
method: 'get'
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
// Separate from the purchase-device preference; browser origin provides the
|
||||
// outer isolation and API environment + user identify the inner namespace.
|
||||
function key(userId) {
|
||||
const id = Number(userId)
|
||||
return Number.isSafeInteger(id) && id > 0 ? `goauto:quick-link-device:${encodeURIComponent(process.env.VUE_APP_BASE_API || '/')}:user:${id}` : null
|
||||
}
|
||||
|
||||
export function readQuickLinkDevice(userId) {
|
||||
try {
|
||||
const name = key(userId)
|
||||
if (!name) return null
|
||||
const id = JSON.parse(localStorage.getItem(name) || 'null')
|
||||
return Number.isSafeInteger(id) && id > 0 ? id : null
|
||||
} catch { return null }
|
||||
}
|
||||
|
||||
export function rememberQuickLinkDevice(userId, deviceId) {
|
||||
try {
|
||||
const name = key(userId)
|
||||
if (!name || !Number.isSafeInteger(deviceId) || deviceId <= 0) return false
|
||||
localStorage.setItem(name, JSON.stringify(deviceId))
|
||||
return true
|
||||
} catch { return false }
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
<template>
|
||||
<BasicLayout>
|
||||
<template #wrapper>
|
||||
<el-card shadow="never">
|
||||
<el-alert v-if="!isAdmin" title="只有管理员可以管理客户端密钥" type="warning" :closable="false" />
|
||||
<template v-else>
|
||||
<div class="heading"><div><h2>客户端密钥</h2><p>按菜单模块开放客户端访问。完整密钥仅在创建成功后显示一次。</p></div><el-button type="primary" :disabled="loading || !!loadError" @click="openEditor()">创建密钥</el-button></div>
|
||||
<el-alert v-if="loadError" :title="loadError" type="error" :closable="false"><el-button @click="load">重新加载</el-button></el-alert>
|
||||
<el-table v-loading="loading" :data="items" row-key="id" border empty-text="还没有客户端密钥,点击右上角创建">
|
||||
<el-table-column label="名称 / 标识" min-width="210"><template #default="{ row }"><strong>{{ row.name }}</strong><p class="muted">{{ row.prefix }}••••</p></template></el-table-column>
|
||||
<el-table-column label="授权模块" min-width="250"><template #default="{ row }"><div v-for="g in row.grants" :key="g.module">{{ moduleName(g.module) }} · {{ g.write ? '读写' : '只读' }}<span v-if="g.actions.length"> / {{ g.actions.map(actionName).join('、') }}</span></div></template></el-table-column>
|
||||
<el-table-column label="状态" width="100"><template #default="{ row }"><el-tag :type="row.enabled ? 'success' : 'info'">{{ row.enabled ? '启用中' : '已停用' }}</el-tag></template></el-table-column>
|
||||
<el-table-column label="最后使用" min-width="170"><template #default="{ row }">{{ row.lastUsedAt ? formatTime(row.lastUsedAt) : '尚未使用' }}</template></el-table-column>
|
||||
<el-table-column label="操作" width="255"><template #default="{ row }"><el-button link type="primary" @click="openEditor(row, true)">查看授权</el-button><el-button link type="primary" :disabled="!row.enabled || saving" @click="openEditor(row)">编辑授权</el-button><el-button link type="danger" :disabled="!row.enabled || saving" @click="disable(row)">停用</el-button></template></el-table-column>
|
||||
</el-table>
|
||||
<el-pagination v-if="total" v-model:current-page="page" :total="total" :page-size="20" layout="prev, pager, next, total" @current-change="load" />
|
||||
<p class="muted">客户端地址:/api/client/v1。支持 HTTP / HTTPS;HTTP 明文传输密钥和数据。不开放支付、账号权限管理及敏感密钥读取。</p>
|
||||
</template>
|
||||
</el-card>
|
||||
<el-dialog v-model="editorOpen" :title="readonly ? '查看授权' : editing ? '编辑授权' : '创建客户端密钥'" width="min(1080px, 95vw)" :close-on-click-modal="false" :close-on-press-escape="!saving" :show-close="!saving" :before-close="closeEditor">
|
||||
<el-alert v-if="editError" :title="editError" type="error" :closable="false" class="notice" />
|
||||
<el-form label-position="top" @submit.prevent="save">
|
||||
<el-form-item label="名称" required><el-input v-model.trim="name" maxlength="80" :disabled="readonly || editing || saving" placeholder="例如:商品同步工具" /></el-form-item>
|
||||
<p v-if="editing" class="muted">标识:{{ selected.prefix }}•••• · 保存不会更换 API Key,完整密钥不可再次查看。</p>
|
||||
<el-alert title="新选模块默认只读;同步、采集、采购、回写、删除等动作需单独勾选。新增菜单不会自动授权。" type="info" :closable="false" class="notice" />
|
||||
<div class="groups">
|
||||
<section v-for="group in groups" :key="group.title" class="module-group">
|
||||
<el-checkbox :model-value="group.modules.every(m => !!grants[m.key])" :indeterminate="group.modules.some(m => !!grants[m.key]) && !group.modules.every(m => !!grants[m.key])" :disabled="readonly || saving" @change="value => selectGroup(group, value)">{{ group.title }}</el-checkbox>
|
||||
<div v-for="m in group.modules" :key="m.key" class="module-row">
|
||||
<div class="module-main"><el-checkbox :model-value="!!grants[m.key]" :disabled="readonly || saving" @change="value => selectModule(m.key, value)">{{ m.title }}</el-checkbox><el-radio-group v-if="grants[m.key]" v-model="grants[m.key].write" :disabled="readonly || saving" size="small" :aria-label="`${m.title}访问方式`"><el-radio-button :value="false">只读</el-radio-button><el-radio-button :value="true" :disabled="!m.writable">读写</el-radio-button></el-radio-group></div>
|
||||
<div v-if="grants[m.key] && m.actions.length" class="actions"><span class="muted">独立动作:</span><el-checkbox-group v-model="grants[m.key].actions" :disabled="readonly || saving"><el-checkbox v-for="action in m.actions" :key="action" :value="action">{{ actionName(action) }}</el-checkbox></el-checkbox-group></div>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
<p v-if="editing" class="muted">{{ selected.enabled ? '保存成功后,后续请求按新授权校验;已执行操作不回滚。' : '密钥已停用,只允许查看授权。' }} 最近修改:管理员 #{{ selected.updatedBy }} · {{ formatTime(selected.updatedAt) }}</p>
|
||||
<p v-if="!Object.keys(grants).length" class="validation" role="alert">至少选择一个模块;要禁止全部访问,请使用停用。</p>
|
||||
</el-form>
|
||||
<template #footer><el-button :disabled="saving" @click="closeEditor()">{{ readonly ? '关闭' : '取消' }}</el-button><el-button v-if="!readonly" type="primary" :loading="saving" :disabled="!valid || !changed || conflicted" @click="save">{{ editing ? '保存授权' : '创建密钥' }}</el-button></template>
|
||||
</el-dialog>
|
||||
<el-dialog v-model="secretOpen" title="密钥已创建,请立即保存" width="min(650px, 95vw)" :close-on-click-modal="false" @closed="clearSecret">
|
||||
<el-alert title="完整密钥只显示这一次;关闭后不可再次查看。请勿放入 URL、日志或前端代码。" type="warning" :closable="false" />
|
||||
<pre class="secret">{{ secret }}</pre><el-button type="primary" @click="copySecret">复制密钥</el-button>
|
||||
<p>请求头:Authorization: Bearer <客户端密钥></p>
|
||||
<template #footer><el-button type="primary" @click="secretOpen = false">已保存,返回列表</el-button></template>
|
||||
</el-dialog>
|
||||
</template>
|
||||
</BasicLayout>
|
||||
</template>
|
||||
|
||||
<script>
|
||||
import { ElMessage, ElMessageBox } from 'element-plus'
|
||||
import { listClientKeys, clientKeyModules, createClientKey, editClientKey, disableClientKey } from '@/api/goauto/client-keys'
|
||||
|
||||
const actionLabels = { sync: '立即同步', import: '导入', match: '匹配 / 确认', collect: '创建 / 重新采集', purchase: '创建 / 重试采购', writeback: '回写', delete: '删除', reparse: '重新解析', activate: '设置当前规则' }
|
||||
const snapshot = grants => JSON.stringify(Object.values(grants).map(g => ({ ...g, actions: [...g.actions].sort() })).sort((a, b) => a.module.localeCompare(b.module)))
|
||||
const errorText = error => error.response?.data?.message || '请求失败,请检查网络后重试'
|
||||
|
||||
export default {
|
||||
name: 'GoAutoClientKeys',
|
||||
data() { return { active: true, items: [], modules: [], total: 0, page: 1, loading: false, loadError: '', saving: false, editorOpen: false, editing: false, readonly: false, selected: null, name: '', grants: {}, initial: '', editError: '', conflicted: false, secret: '', secretOpen: false } },
|
||||
computed: {
|
||||
isAdmin() { return (this.$store.getters.roles || []).includes('admin') },
|
||||
groups() { return [...new Set(this.modules.map(m => m.group))].map(title => ({ title, modules: this.modules.filter(m => m.group === title) })) },
|
||||
valid() { return !!this.name.trim() && Object.keys(this.grants).length > 0 },
|
||||
changed() { return !this.editing || snapshot(this.grants) !== this.initial }
|
||||
},
|
||||
mounted() { this.load() },
|
||||
activated() { this.active = true },
|
||||
deactivated() { this.active = false; this.clearSecret(); this.secretOpen = false },
|
||||
beforeUnmount() { this.active = false; this.clearSecret() },
|
||||
methods: {
|
||||
moduleName(key) { return this.modules.find(m => m.key === key)?.title || key },
|
||||
actionName(action) { return actionLabels[action] || action },
|
||||
formatTime(value) { return value ? new Date(value).toLocaleString() : '—' },
|
||||
async load() {
|
||||
if (!this.isAdmin || this.loading) return
|
||||
this.loading = true; this.loadError = ''
|
||||
try { const [rows, catalog] = await Promise.all([listClientKeys(this.page), clientKeyModules()]); this.items = rows.data.items; this.total = rows.data.total; this.modules = catalog.data } catch (error) { this.loadError = errorText(error) } finally { this.loading = false }
|
||||
},
|
||||
openEditor(row = null, readonly = false) {
|
||||
if (!this.isAdmin || (row && !row.enabled && !readonly)) return
|
||||
this.selected = row; this.editing = !!row; this.readonly = readonly; this.name = row?.name || ''; this.grants = {}
|
||||
for (const g of row?.grants || []) this.grants[g.module] = { ...g, actions: [...g.actions] }
|
||||
this.initial = snapshot(this.grants); this.editError = ''; this.conflicted = false; this.editorOpen = true
|
||||
},
|
||||
selectModule(key, value) { if (value) { if (!this.grants[key]) this.grants[key] = { module: key, write: false, actions: [] } } else delete this.grants[key] },
|
||||
selectGroup(group, value) { for (const m of group.modules) this.selectModule(m.key, value) },
|
||||
closeEditor(done) { if (this.saving) return; this.editorOpen = false; this.grants = {}; if (typeof done === 'function') done() },
|
||||
async save() {
|
||||
if (!this.valid || !this.changed || this.saving || this.readonly || this.conflicted) return
|
||||
this.saving = true; this.editError = ''
|
||||
try {
|
||||
if (this.editing) { await editClientKey(this.selected.id, { version: this.selected.version, grants: Object.values(this.grants) }); ElMessage.success('授权已更新,API Key 保持不变') } else { const result = await createClientKey({ name: this.name, grants: Object.values(this.grants) }); if (this.active) { this.secret = result.data.secret; this.secretOpen = true } }
|
||||
this.editorOpen = false; await this.load()
|
||||
} catch (error) { this.editError = errorText(error); if (!this.editing) this.editError += '。响应丢失时可能已创建,请先检查列表;丢失密钥需停用后重建。'; this.conflicted = error.response?.status === 409; if (this.conflicted) { this.editError += '。请取消并刷新列表后重新打开。'; await this.load() } } finally { this.saving = false }
|
||||
},
|
||||
async disable(row) {
|
||||
try { await ElMessageBox.confirm(`停用“${row.name}”后,新请求将被拒绝,已执行操作不回滚。首版不支持恢复。`, '停用客户端密钥', { confirmButtonText: '确认停用', cancelButtonText: '取消', type: 'warning' }) } catch { return }
|
||||
this.saving = true
|
||||
try { await disableClientKey(row.id, row.version); ElMessage.success('密钥已停用'); await this.load() } catch (error) { ElMessage.error(errorText(error)); await this.load() } finally { this.saving = false }
|
||||
},
|
||||
async copySecret() { try { await navigator.clipboard.writeText(this.secret); ElMessage.success('已复制,请妥善保存') } catch { ElMessage.warning('复制失败,请手动选择并复制密钥') } },
|
||||
clearSecret() { this.secret = '' }
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.heading { display: flex; align-items: center; justify-content: space-between; gap: 20px; margin-bottom: 24px; }
|
||||
h2 { margin: 0 0 12px; }
|
||||
.muted, .heading p { color: var(--el-text-color-secondary); font-size: 13px; line-height: 1.6; }
|
||||
.groups { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; }
|
||||
.module-group { padding: 16px; border: 1px solid var(--el-border-color); border-radius: 6px; min-width: 0; }
|
||||
.module-row { padding: 10px 0; border-top: 1px solid var(--el-border-color-lighter); }
|
||||
.module-main { display: flex; align-items: center; justify-content: space-between; gap: 8px; flex-wrap: wrap; }
|
||||
.actions { padding: 4px 0 0 22px; }
|
||||
.notice, .el-pagination { margin: 16px 0; }
|
||||
.validation { color: var(--el-color-danger); }
|
||||
.secret { white-space: pre-wrap; overflow-wrap: anywhere; padding: 16px; background: var(--el-fill-color-light); }
|
||||
@media (max-width: 760px) { .groups { grid-template-columns: 1fr; } .heading { align-items: flex-start; } }
|
||||
</style>
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user