Compare commits

..
120 changed files with 4068 additions and 2584 deletions
+23 -5
View File
@@ -1,11 +1,17 @@
# Agent 开发规则
本仓库采用精简的单人 DevHarness 工作流:Gitea 工单是单次任务需求、变化、实现、测试、提交和验收的唯一事实来源;Gitea Wiki 只维护长期产品、架构、契约、业务规则、安全边界和运行说明;Git 保存源码、迁移、测试、版本绑定资料和 Wiki 的本地镜像。`docs/` 中显式映射的 Markdown 是 Wiki 只读镜像;既有 Wiki 任务归档和 `docs/task/` 仅作历史兼容,只有用户明确要求专项快照时才创建或导出。
本仓库采用轻量治理的精简单人 DevHarness 工作流:需要工单的任务以 Gitea 工单作为单次需求、变化、实现、测试、提交和验收的事实来源;Gitea Wiki 只维护长期产品、架构、契约、业务规则、安全边界和运行说明;Git 保存源码、迁移、测试、版本绑定资料和 Wiki 的本地镜像。`docs/` 中显式映射的 Markdown 是 Wiki 只读镜像;既有 Wiki 任务归档和 `docs/task/` 仅作历史兼容,只有用户明确要求专项快照时才创建或导出。
当前文档规则参考 DevHarness 提交 `4bbacf4d7fb265984396bb5589c544105043fa0b`,但所有模板内容都必须按 GoAuto 事实改写。开始工作前阅读任务涉及目录中的 `AGENTS.md`。不同交付单元规则不同时,在 `server/`、`web/` 或 `android/` 下增加更具体的 `AGENTS.md`;目录越深的规则越具体,但不得削弱上级安全规则。
当前文档规则参考 DevHarness 提交 `ecab899`,但所有模板内容都必须按 GoAuto 事实改写。开始工作前阅读任务涉及目录中的 `AGENTS.md`。不同交付单元规则不同时,在 `server/`、`web/` 或 `android/` 下增加更具体的 `AGENTS.md`;目录越深的规则越具体,但不得削弱上级安全规则。
[项目档案](docs/00-project-profile.md) 按需阅读,不作为每次任务的固定前置。出现下列情况之一时必须读:需要环境、配置或凭据来源;需要确认目录边界;需要判断子项目与交付单元划分;需要 DevHarness 来源与基线;需要项目专用验收要求。只为查命令不必打开项目档案。涉及采集、采购或设备行为时另读 `docs/03-business-rules-and-glossary.md` 和当前工单。
## 语言与术语
- 用户可以使用中文、英文或合理的中英混合语言交流;默认使用中文分析、回复、编写工单和维护内部项目文档。
- 代码标识符、命令、参数、路径、文件名、API 名称、协议名、日志和错误原文保持原样;必要时补充简短中文解释。
- 用户明确要求某次回复或交付物使用其他语言时,按该次要求执行,不改写接口契约或影响搜索和执行的原文。
## 常用命令
所有命令默认从仓库根目录执行。
@@ -16,6 +22,7 @@
| 检查模板结构 | `python dev_scripts/harness.py check --strict` |
| 导出核心 Wiki 镜像 | `python dev_scripts/harness.py sync` |
| 检查核心 Wiki 镜像 | `python dev_scripts/harness.py sync --check` |
| 深度检查核心 Wiki 镜像 | `python dev_scripts/harness.py sync --deep-check` |
| 导出并完整校验 | `python dev_scripts/harness.py sync --verify` |
| 创建可选任务快照 | `python dev_scripts/harness.py archive 123 "修复登录超时"` |
| 增量导出已有快照 | `python dev_scripts/harness.py export` |
@@ -40,18 +47,28 @@
- 测试结果必须真实;未执行或无法覆盖的真机、多设备、云环境和高风险行为必须明确记录。
- 高风险修改必须停止并等待人工确认:创建订单、权限、安全、并发、数据库迁移、删除数据、发布和其他不可逆操作。
### 项目治理模式与明确授权后的执行
- GoAuto 默认采用轻量治理:文案、注释、格式、局部样式或布局、预期行为明确的小 Bug,以及不改变接口、数据结构、权限和安全边界的单模块低风险调整可以直接实施,无需为了留痕补建工单。完整独立需求、新页面、跨模块功能,以及涉及 API、数据结构、权限、安全、迁移或范围不明确的变化必须建单。
- 采购、创建订单、权限、安全、并发、数据库迁移、删除数据、发布和其他不可逆操作按高风险任务处理;未取得有效人工授权时必须停止。
- 当前聊天中用户给出的明确指令,或 Gitea 工单中能够归属于有权人工的明确授权,可以作为执行依据,不要求把同一授权重复复制到工单后再次确认。
- 授权必须能识别操作、对象和范围;Agent 自动生成的工单、草稿、摘要或对用户意图的转述不能单独构成人工授权。
- 获得有效授权后,只核对准确目标、授权范围和当前状态等最小必要前提,不得仅因操作不可逆而重复询问或拒绝。
- 授权不自动覆盖相邻对象或后续任务;环境、对象、范围或影响发生实质变化时必须重新确认。平台自身强制的审批、安全策略或权限限制继续有效。
## 2. 哪些改动需要工单
新功能、缺陷修复、重构,以及接口、数据库、权限、并发、状态机、安全或用户界面变化必须先有单元工单。
完整独立需求、新页面、跨模块功能、范围不明确的变化,以及接口、数据库、权限、并发、状态机或安全边界变化必须先有单元工单。采购、创建订单、数据库迁移、删除数据、发布和其他不可逆操作无论规模大小都必须建单。
以下小改动只有在范围明确、容易回退且不涉及上面的必须建单项时才可以直接提交:
以下低风险改动在范围明确、容易回退、不改变接口、数据结构、权限或安全边界,且不属于上述高风险事项时可以直接提交:
- 只改错别字、注释或文档措辞;
- 只做格式化、导入排序或不跨文件的内部变量改名;
- 补充类型标注或文档字符串且不改变行为;
- 补充不改变产品行为的测试;
- 删除已经确认无人使用的死代码;
- 修复单文件、低风险且只恢复已有明确行为的缺陷;
- 修复预期行为明确、影响局限于单个模块的低风险缺陷;
- 不改变对外行为、入口、配置和验证方式的局部内部重构;
- 只修改用户看到的界面显示文案,并且满足本文件「工单与设计证据双门禁」的全部豁免条件。
直接提交仍须保护无关改动、执行受影响范围的最小验证并写清提交说明。有任何不确定,或涉及接口、数据库、状态、权限、安全、并发、用户界面时,必须退回单元工单流程。代码组件名、类名、变量、国际化键、API 字段和数据库字段不是显示文案,不适用豁免。
@@ -184,6 +201,7 @@ Gitea 不可用时,输出完整工单草稿并说明阻塞。未经用户明
- 提交只包含当前工单相关文件,提交信息引用工单号。
- 优先运行项目档案记录的格式、单元、契约和集成测试。
- Windows 环境优先使用当前已配置的 PowerShell;可选择时优先 PowerShell 7 `pwsh.exe`,不得仅为设置编码重复启动一层 PowerShell。
- Windows 命令不得默认套用 Bash 语法。复杂正则优先使用变量或 `rg -e`;包含引号和换行的脚本正文优先使用单引号 PowerShell here-string;`foreach`、`if` 等语句块保持在同一个 PowerShell 解析上下文中;`rg` 使用真实目录配合 `-g/--glob`,不要把 Bash 风格通配路径作为目录参数。
- 文本文件读写在命令支持时显式指定 UTF-8;文件解码和控制台输出分别处理,只有出现真实乱码或已知宿主非 UTF-8 时才设置当前进程的输出编码或 Python UTF-8 环境变量。
- 不得默认使用 `-ExecutionPolicy Bypass`;只有可信 `.ps1` 确实被执行策略阻止且没有更小替代方案时,才对该次进程使用并在工单记录原因。
- 涉及创建订单、权限、安全、并发、迁移和删除数据属于高风险,真机或正式实施前必须再次等待人工确认。
+2 -2
View File
@@ -11,8 +11,8 @@ android {
applicationId = "cn.ilapage.goauto.agent"
minSdk = 23
targetSdk = 34
versionCode = 52
versionName = "0.9.39"
versionCode = 51
versionName = "0.9.38"
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
@@ -333,32 +333,31 @@ class GoAutoAccessibilityService : AccessibilityService(), UiDriver, PddCollecto
if (bounds.width() < 2 || bounds.height() < 2 || Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {
return FreshActionResult.FAILED
}
return if (dispatchCenterTap(bounds)) FreshActionResult.SUCCESS else FreshActionResult.FAILED
}
val path = Path().apply {
moveTo(bounds.centerX().toFloat(), bounds.centerY().toFloat())
}
val completed = AtomicBoolean(false)
val latch = CountDownLatch(1)
val queued = dispatchGesture(
GestureDescription.Builder().addStroke(GestureDescription.StrokeDescription(path, 0, 80)).build(),
object : GestureResultCallback() {
override fun onCompleted(gestureDescription: GestureDescription?) {
completed.set(true)
latch.countDown()
}
override fun tapSpecFresh(target: SnapshotNode): FreshActionResult {
val width = resources.displayMetrics.widthPixels
val height = resources.displayMetrics.heightPixels
if (!PurchaseSpecGesturePolicy.isAllowed(target, width, height)) return FreshActionResult.FAILED
val root = rootInActiveWindow ?: return FreshActionResult.NOT_FOUND
val candidates = mutableListOf<AccessibilityNodeInfo>()
walk(root) { node ->
val bounds = Rect().also(node::getBoundsInScreen)
if (node.isVisibleToUser && node.isEnabled &&
node.preferredOrDescendantLabel() == target.label &&
node.className?.toString() == target.className &&
kotlin.math.abs(bounds.centerX() - target.bounds.centerX) <= 32 &&
kotlin.math.abs(bounds.centerY() - target.bounds.centerY) <= 32
) candidates += node
override fun onCancelled(gestureDescription: GestureDescription?) {
latch.countDown()
}
},
null,
)
if (!queued) return FreshActionResult.FAILED
return if (latch.await(1_500, TimeUnit.MILLISECONDS) && completed.get()) {
FreshActionResult.SUCCESS
} else {
FreshActionResult.FAILED
}
if (candidates.isEmpty()) return FreshActionResult.NOT_FOUND
if (candidates.size != 1) return FreshActionResult.AMBIGUOUS
val bounds = Rect().also(candidates.single()::getBoundsInScreen)
val freshTarget = target.copy(bounds = NodeBounds(bounds.left, bounds.top, bounds.right, bounds.bottom))
if (!PurchaseSpecGesturePolicy.isAllowed(freshTarget, width, height)) {
return FreshActionResult.FAILED
}
return if (dispatchCenterTap(bounds)) FreshActionResult.SUCCESS else FreshActionResult.FAILED
}
override fun inputFresh(target: SnapshotNode, value: String): FreshActionResult {
@@ -408,7 +407,7 @@ class GoAutoAccessibilityService : AccessibilityService(), UiDriver, PddCollecto
) matches += node
}
if (matches.size != 1) return false
return swipeNode(matches.single(), direction, durationMs, preferScrollAction = true)
return swipeNode(matches.single(), direction, durationMs, preferScrollAction = false)
}
override fun backPurchase(): Boolean = performGlobalAction(GLOBAL_ACTION_BACK)
@@ -618,28 +617,6 @@ class GoAutoAccessibilityService : AccessibilityService(), UiDriver, PddCollecto
return latch.await(1500, TimeUnit.MILLISECONDS) && completed.get()
}
private fun dispatchCenterTap(bounds: Rect): Boolean {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N || bounds.width() < 2 || bounds.height() < 2) return false
val path = Path().apply { moveTo(bounds.centerX().toFloat(), bounds.centerY().toFloat()) }
val completed = AtomicBoolean(false)
val latch = CountDownLatch(1)
val queued = dispatchGesture(
GestureDescription.Builder().addStroke(GestureDescription.StrokeDescription(path, 0, 80)).build(),
object : GestureResultCallback() {
override fun onCompleted(gestureDescription: GestureDescription?) {
completed.set(true)
latch.countDown()
}
override fun onCancelled(gestureDescription: GestureDescription?) {
latch.countDown()
}
},
null,
)
return queued && latch.await(1_500, TimeUnit.MILLISECONDS) && completed.get()
}
private fun walk(node: AccessibilityNodeInfo, visit: (AccessibilityNodeInfo) -> Unit) {
visit(node)
for (index in 0 until node.childCount) node.getChild(index)?.let { walk(it, visit) }
@@ -123,9 +123,6 @@ data class ParsedPddScreen(
val hasSelectionSummary: Boolean,
val hasQuantityControls: Boolean,
val hasOrderSubmitAction: Boolean,
val explicitSpecEntryCount: Int,
val nestedSpecEntryCount: Int,
val bottomPurchaseEntryCount: Int,
val problem: PageProblem?,
val sourceNodes: List<SnapshotNode>,
) {
@@ -309,14 +306,6 @@ object PddScreenParser {
orderConfirmationEvidence -> SpecPanelType.ORDER_CONFIRMATION
panelScrollable != null && (hasSelectionSummary || hasSubmitHint || (hasPanelTitle && hasPanelAction)) -> SpecPanelType.NORMAL_SCROLLABLE
hasSelectionSummary && hasPanelTitle && hasPanelAction -> SpecPanelType.NON_SCROLLABLE_CONFIRMATION
// Some PDD builds expose the complete selector as non-scrollable
// before a value is selected, so an "已选" summary is absent. Two
// parsed dimensions with selectable values plus the unique quantity
// controls and lower-page order action remain required; generic
// product/review pages cannot satisfy this combined evidence.
panelScrollable == null && headings.size >= 2 && dimensions.size >= 2 &&
dimensions.sumOf { it.values.size } >= 2 &&
hasQuantityControls && hasOrderSubmitAction -> SpecPanelType.NON_SCROLLABLE_CONFIRMATION
else -> SpecPanelType.UNKNOWN
}
val panelOpen = specPanelType != SpecPanelType.UNKNOWN
@@ -325,30 +314,19 @@ object PddScreenParser {
.filter { it.bounds.top < firstHeadingTop }
.mapNotNull { node -> pricePattern.find(node.label)?.groupValues?.get(1)?.let(::priceCent) }
.firstOrNull()
val explicitSpecEntries = if (panelOpen) emptyList() else visible
val explicitSpecEntry = if (panelOpen) null else visible
.filter { it.clickable && isSpecEntry(it.label, config) && !hasReviewContext(it, visibleNodes, config) }
val explicitSpecEntry = explicitSpecEntries.maxByOrNull { it.bounds.top }
val nestedSpecEntries = if (panelOpen || explicitSpecEntry != null) emptyList() else
safeNestedSpecEntries(visibleNodes, visible, config)
// A nested selection row is accepted only when it is the single safe
// candidate. This covers PDD layouts that split “请选择” and the
// dimension name across child nodes of one clickable parent, without
// turning arbitrary page text into a click target.
val nestedSpecEntry = nestedSpecEntries.singleOrNull()
val bottomSpecEntries = if (panelOpen || explicitSpecEntry != null || nestedSpecEntry != null) emptyList() else
safeBottomSpecEntries(visibleNodes, visible, config)
val bottomSpecEntry = bottomSpecEntries.firstOrNull()
val candidateSpecEntry = explicitSpecEntry ?: nestedSpecEntry?.anchor ?: bottomSpecEntry?.anchor
val candidateClickTarget = explicitSpecEntry ?: nestedSpecEntry?.clickTarget ?: bottomSpecEntry?.clickTarget
.maxByOrNull { it.bounds.top }
val bottomSpecEntry = if (panelOpen || explicitSpecEntry != null) null else safeBottomSpecEntry(visibleNodes, visible, config)
val candidateSpecEntry = explicitSpecEntry ?: bottomSpecEntry?.anchor
val candidateClickTarget = explicitSpecEntry ?: bottomSpecEntry?.clickTarget
val reviewPageOpen = isReviewPage(visibleNodes, visible, screenHeight, candidateSpecEntry, config)
val specEntry = candidateSpecEntry.takeUnless { reviewPageOpen }
val specEntryClickTarget = candidateClickTarget.takeUnless { reviewPageOpen }
val specEntrySource = when {
reviewPageOpen -> null
explicitSpecEntry != null -> "explicit_selection"
nestedSpecEntry != null -> "nested_selection"
bottomSpecEntries.size == 1 -> "bottom_purchase"
bottomSpecEntry != null -> "bottom_purchase_rightmost"
bottomSpecEntry != null -> "bottom_purchase"
else -> null
}
val quickConfirmationEntry = if (quickConfirmationEvidence) quickConfirmationSpecEntry(visibleNodes, visible, config) else null
@@ -392,9 +370,6 @@ object PddScreenParser {
hasSelectionSummary = hasSelectionSummary,
hasQuantityControls = hasQuantityControls,
hasOrderSubmitAction = hasOrderSubmitAction,
explicitSpecEntryCount = explicitSpecEntries.size,
nestedSpecEntryCount = nestedSpecEntries.size,
bottomPurchaseEntryCount = bottomSpecEntries.size,
problem = problem,
sourceNodes = visibleNodes,
)
@@ -483,40 +458,10 @@ object PddScreenParser {
return hasSpecWord && config.textAliases.selection.specEntryPrefixes.any(compact::startsWith)
}
private fun safeNestedSpecEntries(source: List<SnapshotNode>, visible: List<SnapshotNode>, config: PddCollectorConfig): List<SafeSpecEntry> {
val screenHeight = source.maxOfOrNull { it.bounds.bottom } ?: return emptyList()
if (screenHeight <= 0) return emptyList()
return source.asSequence()
.filter { it.visible && it.enabled && it.clickable && it.bounds.width > 0 && it.bounds.height > 0 }
// The fixed purchase bar begins at the lower fifth of the screen.
// A specs row has no reason to be inside that action-only zone.
.filter { it.bounds.centerY.toDouble() < screenHeight * 0.8 }
.mapNotNull { candidate ->
val context = (listOf(candidate.label) + descendants(candidate, source).map(SnapshotNode::label))
.joinToString("") { it.replace(Regex("\\s+"), "") }
if (!isSpecEntryContext(context, config) ||
nonConfigurableClickDenylist.any(context::contains) ||
hasReviewContext(candidate, source, config)
) return@mapNotNull null
val anchor = visible.firstOrNull { it.path == candidate.path } ?: candidate
SafeSpecEntry(anchor, candidate)
}
.distinctBy { it.clickTarget.path }
.toList()
}
private fun isSpecEntryContext(compact: String, config: PddCollectorConfig): Boolean {
if (config.textAliases.review.entryAliases.any(compact::contains)) return false
val hasSpecWord = (config.colorAliases + config.sizeAliases +
config.textAliases.dimension.exactNames + config.textAliases.dimension.adaptiveAliases)
.any(compact::contains)
return hasSpecWord && config.textAliases.selection.specEntryPrefixes.any(compact::contains)
}
private fun safeBottomSpecEntries(source: List<SnapshotNode>, visible: List<SnapshotNode>, config: PddCollectorConfig): List<SafeSpecEntry> {
val screenWidth = source.maxOfOrNull { it.bounds.right } ?: return emptyList()
val screenHeight = source.maxOfOrNull { it.bounds.bottom } ?: return emptyList()
if (screenWidth <= 0 || screenHeight <= 0) return emptyList()
private fun safeBottomSpecEntry(source: List<SnapshotNode>, visible: List<SnapshotNode>, config: PddCollectorConfig): SafeSpecEntry? {
val screenWidth = source.maxOfOrNull { it.bounds.right } ?: return null
val screenHeight = source.maxOfOrNull { it.bounds.bottom } ?: return null
if (screenWidth <= 0 || screenHeight <= 0) return null
val byPath = source.associateBy(SnapshotNode::path)
val normalizedByPath = visible.associateBy(SnapshotNode::path)
val buyWords = config.textAliases.purchase.buyWords
@@ -540,10 +485,8 @@ object PddScreenParser {
val normalizedTarget = normalizedByPath[clickTarget.path] ?: return@mapNotNull null
SafeSpecEntry(normalizedAnchor, normalizedTarget) to clickTarget.bounds.width.toLong() * clickTarget.bounds.height
}
.sortedWith(compareByDescending<Pair<SafeSpecEntry, Long>> { it.first.clickTarget.bounds.centerX }.thenBy { it.second })
.map { it.first }
.distinctBy { it.clickTarget.path }
.toList()
.minWithOrNull(compareBy<Pair<SafeSpecEntry, Long>> { it.second }.thenByDescending { it.first.clickTarget.bounds.centerX })
?.first
}
private fun hasReviewContext(node: SnapshotNode, source: List<SnapshotNode>, config: PddCollectorConfig): Boolean {
@@ -117,12 +117,10 @@ class PurchaseLiveAutomation(
return null
}
val labels = linkedSetOf<String>()
var paymentBackAttempts = 0
var consecutivePaymentSamplesAfterBack = 0
var backedOutOfPayment = false
var backedOutOfChooser = false
var restoredFromWechat = false
var pddObservedAfterWechatRestore = false
var orderDetailEntryOpened = false
var wechatRestorePendingSamples = 0
var consecutiveEmptySnapshots = 0
repeat(ORDER_RESULT_MAX_SAMPLES) { index ->
@@ -164,58 +162,24 @@ class PurchaseLiveAutomation(
return unknown("PURCHASE_ORDER_UNEXPECTED_APP", "核单期间出现未授权应用")
}
if (restoredFromWechat) pddObservedAfterWechatRestore = true
val paymentVisible = isKnownPddPaymentActivity(snapshot) ||
currentLabels.any { label -> PAYMENT_MARKERS.any(label::contains) }
val orderContextVisible = currentLabels.any { label -> ORDER_CONTEXT_MARKERS.any(label::contains) }
val unpaidContextVisible = currentLabels.any { label -> UNPAID_MARKERS.any(label::contains) }
// PDD can reuse its payment Activity for a read-only unpaid order page.
// Visible order-result evidence takes precedence over the Activity name:
// it permits only bounded reading gestures below, never a payment click.
val paymentVisible = isKnownPddPaymentActivity(snapshot) && !orderContextVisible && !unpaidContextVisible
if (paymentVisible) {
if (paymentBackAttempts == 0) {
paymentBackAttempts++
if (!driver.backPurchase()) {
return unknown("PURCHASE_ORDER_PAYMENT_BACK_FAILED", "支付页无法安全返回订单详情")
}
pause(500)
} else {
consecutivePaymentSamplesAfterBack++
if (consecutivePaymentSamplesAfterBack >= ORDER_RESULT_PAYMENT_POST_BACK_MAX_SAMPLES) {
return unknown(
"PURCHASE_ORDER_PAYMENT_REPEATED",
"支付页安全返回后持续无订单证据,已停止自动核单" +
"[paymentBackAttempts=$paymentBackAttempts;" +
"consecutivePaymentSamplesAfterBack=$consecutivePaymentSamplesAfterBack]",
)
}
pause(ORDER_RESULT_SAMPLE_INTERVAL_MS)
}
return@repeat
}
consecutivePaymentSamplesAfterBack = 0
if (!orderContextVisible && !unpaidContextVisible) {
val entries = orderDetailEntryTargets(snapshot)
if (entries.size > 1) {
return unknown("PURCHASE_ORDER_DETAIL_ENTRY_AMBIGUOUS", "订单详情入口不唯一,已停止只读核单")
}
if (entries.size == 1) {
if (orderDetailEntryOpened) {
return unknown("PURCHASE_ORDER_DETAIL_ENTRY_TIMEOUT", "进入订单详情后页面未出现可验证证据")
}
when (driver.clickFresh(entries.single())) {
FreshActionResult.SUCCESS -> Unit
FreshActionResult.AMBIGUOUS -> return unknown("PURCHASE_ORDER_DETAIL_ENTRY_AMBIGUOUS", "订单详情入口不唯一,已停止只读核单")
else -> return unknown("PURCHASE_ORDER_DETAIL_ENTRY_FAILED", "订单详情入口点击失败,已停止只读核单")
}
orderDetailEntryOpened = true
pause(500)
return@repeat
}
if (!paymentVisible && !orderContextVisible) {
pause(ORDER_RESULT_SAMPLE_INTERVAL_MS)
return@repeat
}
currentLabels.forEach(labels::add)
parseOrderEvidence(labels)?.let { return it }
if (index > 0 && index % ORDER_RESULT_SCROLL_SAMPLE_INTERVAL == 0) {
if (paymentVisible && !backedOutOfPayment) {
backedOutOfPayment = true
if (!driver.backPurchase()) {
return unknown("PURCHASE_ORDER_PAYMENT_BACK_FAILED", "支付页无法安全返回订单详情")
}
pause(500)
} else if (paymentVisible) {
return unknown("PURCHASE_ORDER_PAYMENT_REPEATED", "支付页重复出现,已停止自动核单")
} else if (index > 0 && index % 15 == 0) {
driver.swipePurchase(SwipeDirection.UP, 400)
}
pause(ORDER_RESULT_SAMPLE_INTERVAL_MS)
@@ -407,14 +371,6 @@ class PurchaseLiveAutomation(
snapshot.nodes.filter { node -> node.visible && node.enabled && FINAL_SUBMIT_MARKERS.any { node.label == it || node.label.startsWith(it) } },
)
/** A post-submit navigation target is allowed only when it is one exact, non-payment PDD order-detail entry. */
private fun orderDetailEntryTargets(snapshot: UiSnapshot): List<SnapshotNode> = uniqueClickable(
snapshot,
snapshot.nodes.filter { node ->
node.visible && node.enabled && node.label in ORDER_DETAIL_ENTRY_MARKERS
},
)
private fun pageProblem(snapshot: UiSnapshot) {
val labels = snapshot.nodes.filter { it.visible }.map { it.label }
PddPageClassifier.classify(snapshot.packageName, snapshot.activityName, labels)?.let { fail(it.code, it.message) }
@@ -465,7 +421,6 @@ class PurchaseLiveAutomation(
val FINAL_SUBMIT_MARKERS = listOf("提交订单", "现在买,仅", "确认购买")
val PAYMENT_MARKERS = listOf("立即支付", "确认支付", "输入支付密码")
val UNPAID_MARKERS = listOf("待付款", "待支付", "去支付")
val ORDER_DETAIL_ENTRY_MARKERS = setOf("查看订单", "订单详情")
val ORDER_CONTEXT_MARKERS = listOf("订单编号", "订单号", "下单时间", "创建时间")
val ORDER_NO = Regex("(?:订单编号|订单号)\\s*[::]?\\s*([A-Za-z0-9-]{6,64})")
val ORDER_TIME = Regex("(?:下单时间|创建时间)\\s*[::]?\\s*(20[0-9]{2}[-/.年][0-9]{1,2}[-/.月][0-9]{1,2}日?\\s+[0-9]{1,2}:[0-9]{2}(?::[0-9]{2})?)")
@@ -480,8 +435,6 @@ class PurchaseLiveAutomation(
const val ORDER_RESULT_MAX_SAMPLES = 60
const val ORDER_RESULT_MAX_EMPTY_SAMPLES = 15
const val ORDER_RESULT_WECHAT_RESTORE_MAX_SAMPLES = 15
const val ORDER_RESULT_PAYMENT_POST_BACK_MAX_SAMPLES = 3
const val ORDER_RESULT_SCROLL_SAMPLE_INTERVAL = 15
const val ORDER_RESULT_SAMPLE_INTERVAL_MS = 200L
}
}
@@ -11,12 +11,6 @@ interface PurchaseUiDriver {
reason = FreshClickReason.UNKNOWN,
)
fun tapPurchaseFresh(target: SnapshotNode): FreshActionResult
/**
* Reacquires and center-taps a target that the purchase parser has already
* accepted as either a safe spec entry or one exact spec value. Callers
* must still verify the semantic postcondition from a fresh snapshot.
*/
fun tapSpecFresh(target: SnapshotNode): FreshActionResult = FreshActionResult.FAILED
fun inputFresh(target: SnapshotNode, value: String): FreshActionResult
fun swipePurchase(direction: SwipeDirection, durationMs: Long): Boolean
fun swipePurchaseIn(target: SnapshotNode, direction: SwipeDirection, durationMs: Long): Boolean
@@ -26,21 +20,6 @@ interface PurchaseUiDriver {
fun bringPddToForeground(): Boolean = false
}
internal object PurchaseSpecGesturePolicy {
private val forbiddenLabels = listOf(
"提交订单", "创建订单", "确认订单", "确认购买", "支付", "付款",
"收货地址", "修改地址", "保存地址",
)
fun isAllowed(target: SnapshotNode, screenWidth: Int, screenHeight: Int): Boolean {
val label = target.label.replace(" ", "")
return target.visible && target.enabled && label.isNotBlank() &&
forbiddenLabels.none(label::contains) &&
target.bounds.width >= 2 && target.bounds.height >= 2 &&
target.bounds.centerX in 0 until screenWidth && target.bounds.centerY in 0 until screenHeight
}
}
data class PurchaseExecutionInput(
val taskId: Long,
val executionMode: String,
@@ -72,7 +51,6 @@ class PurchaseRehearsalExecutor(
private val probeSpecs: () -> String?,
private val pause: (Long) -> Unit = Thread::sleep,
private val stepChanged: (String) -> Unit = {},
private val panelDiagnostic: (String) -> Unit = {},
private val beforeOrderSubmit: (FinalConfirmationEvidence) -> Unit = { throw PurchaseLiveException("PURCHASE_MODE_NOT_ALLOWED", "当前执行器没有正式采购授权") },
) {
fun execute(input: PurchaseExecutionInput, rule: PurchaseRule, supportedCapabilities: Set<String>): PurchaseExecutionOutcome {
@@ -145,7 +123,7 @@ class PurchaseRehearsalExecutor(
}
return failure
}
applyPostAction(input, action)?.let { return it }
applyPostAction(action)?.let { return it }
}
if (input.phase == "spec_probe") return failure("PURCHASE_RULE_INVALID", "规格探测任务缺少 probeSpecs 动作")
return if (input.executionMode == "rehearsal") PurchaseExecutionOutcome("rehearsal_completed", message = "商品、规格、数量和价格复核完成,已在下单前安全停止", actualUnitPriceCent = observedPrice)
@@ -245,9 +223,8 @@ class PurchaseRehearsalExecutor(
repeat(50) {
val snapshot = driver.capture()
pageProblem(snapshot)?.let { return it }
val screen = PddScreenParser.parse(snapshot, DEFAULT_COLLECTOR, input.goodsId, null)
if (screen.hasPurchaseProductEvidence()) {
return recoverSoldOut(input, screen)
if (snapshot.packageName == PDD_PACKAGE && snapshot.nodes.any { it.visible }) {
return recoverSoldOut(input, PddScreenParser.parse(snapshot, DEFAULT_COLLECTOR, input.goodsId, null))
}
pause(100)
}
@@ -288,145 +265,37 @@ class PurchaseRehearsalExecutor(
private fun openSpecPanel(input: PurchaseExecutionInput, action: PurchaseAction): PurchaseExecutionOutcome? {
var screen = currentScreen(input)
var entryReadyWaitPolls = 0
var target: SnapshotNode? = null
while (target == null) {
if (screen.reviewPageOpen) return leaveUnexpectedReviewPage(input)
screen.problem?.let { return failure(it.code, it.message) }
if (screen.specPanelOpen) return null
val safeCandidates = listOfNotNull(
screen.specEntry?.let { anchor -> anchor to (screen.specEntryClickTarget ?: anchor) },
screen.quickConfirmationEntry?.let { it to it },
).distinctBy { it.second.path }
// A rule alias may only narrow the semantic candidates already accepted
// by PddScreenParser. It must never turn into a raw-page text lookup:
// review cards and unrelated controls can share arbitrary labels.
val candidates = action.textAliases?.let { aliases ->
safeCandidates.filter { (anchor, _) -> specEntryMatchesAliases(screen, anchor, aliases) }
} ?: safeCandidates
if (candidates.size > 1) {
panelDiagnostic(specEntryEvidence(screen, candidates.size, entryReadyWaitPolls))
return failure(SPEC_ENTRY_TARGET_AMBIGUOUS, "规格入口候选不唯一 [${specEntryEvidence(screen, candidates.size, entryReadyWaitPolls)}]")
}
target = candidates.singleOrNull()?.second
if (target != null) continue
if (entryReadyWaitPolls >= SPEC_ENTRY_READY_WAIT_POLLS) {
panelDiagnostic(specEntryEvidence(screen, 0, entryReadyWaitPolls))
return failure(SPEC_ENTRY_NOT_FOUND, "没有找到安全的商品规格入口 [${specEntryEvidence(screen, 0, entryReadyWaitPolls)}]")
}
pause(SPEC_ENTRY_READY_POLL_MILLIS)
entryReadyWaitPolls++
screen = currentScreen(input)
}
val beforeSignature = specActionSignature(screen)
val click = driver.clickFreshDetailed(requireNotNull(target))
if (screen.reviewPageOpen) return leaveUnexpectedReviewPage(input)
screen.problem?.let { return failure(it.code, it.message) }
if (screen.specPanelOpen) return null
val safeCandidates = listOfNotNull(
screen.specEntry?.let { anchor -> anchor to (screen.specEntryClickTarget ?: anchor) },
screen.quickConfirmationEntry?.let { it to it },
).distinctBy { it.second.path }
// A rule alias may only narrow the semantic candidates already accepted
// by PddScreenParser. It must never turn into a raw-page text lookup:
// review cards and unrelated controls can share arbitrary labels.
val candidates = action.textAliases?.let { aliases ->
safeCandidates.filter { (anchor, _) -> specEntryMatchesAliases(screen, anchor, aliases) }
} ?: safeCandidates
if (candidates.size > 1) return failure(SPEC_ENTRY_TARGET_AMBIGUOUS, "规格入口候选不唯一")
val target = candidates.singleOrNull()?.second ?: return failure(SPEC_ENTRY_NOT_FOUND, "没有找到安全的商品规格入口")
val click = driver.clickFreshDetailed(target)
when (click.result) {
// The parser already narrowed to a single semantic candidate; the
// ambiguity here comes from the live tree matching that target more
// than once at click time, so the wording must not claim otherwise.
FreshActionResult.AMBIGUOUS -> return failure(
SPEC_ENTRY_TARGET_AMBIGUOUS,
"规格入口点击目标不唯一 [${specEntryEvidence(screen, 1, entryReadyWaitPolls)}]",
)
else -> Unit
}
var wait = waitForSpecPanel(input, beforeSignature)
wait.failure?.let { return it }
if (wait.opened) return null
if (wait.changed) {
return failure(
SPEC_PANEL_EVIDENCE_NOT_MATCHED,
"规格入口点击后页面已变化,但规格面板强证据不足 [${panelEvidence(wait.screen)}]",
)
}
when (driver.tapSpecFresh(requireNotNull(target))) {
FreshActionResult.AMBIGUOUS -> return failure(
SPEC_ENTRY_TARGET_AMBIGUOUS,
"规格入口手势目标不唯一 [${specEntryEvidence(screen, 1, entryReadyWaitPolls)}]",
)
FreshActionResult.AMBIGUOUS -> return failure(SPEC_ENTRY_TARGET_AMBIGUOUS, "规格入口候选不唯一")
FreshActionResult.SUCCESS -> Unit
else -> return failure(SPEC_ENTRY_CLICK_FAILED, click.reason.specEntrySubreason())
}
wait = waitForSpecPanel(input, specActionSignature(wait.screen))
wait.failure?.let { return it }
if (wait.opened) return null
if (wait.changed) {
return failure(
SPEC_PANEL_EVIDENCE_NOT_MATCHED,
"规格入口手势后页面已变化,但规格面板强证据不足 [${panelEvidence(wait.screen)}]",
)
repeat(30) {
screen = currentScreen(input)
if (screen.reviewPageOpen) return leaveUnexpectedReviewPage(input)
screen.problem?.let { return failure(it.code, it.message) }
if (screen.specPanelOpen) return null
pause(100)
}
return failure(
SPEC_ENTRY_CLICK_NO_EFFECT,
"规格入口无障碍点击和受控手势后页面均无变化 [${panelEvidence(wait.screen)}]",
)
return failure(SPEC_PANEL_NOT_OPENED, "点击后未识别到商品规格面板")
}
private data class SpecPanelWait(
val screen: ParsedPddScreen,
val opened: Boolean,
val changed: Boolean,
val failure: PurchaseExecutionOutcome? = null,
)
private data class SpecActionNodeSignature(
val path: String,
val className: String?,
val bounds: NodeBounds,
val clickable: Boolean,
val scrollable: Boolean,
val selected: Boolean,
val checked: Boolean,
val enabled: Boolean,
val visible: Boolean,
)
private fun waitForSpecPanel(input: PurchaseExecutionInput, beforeSignature: List<SpecActionNodeSignature>): SpecPanelWait {
var last = currentScreen(input)
var changed = false
repeat(SPEC_POST_CLICK_VERIFY_POLLS) {
last = currentScreen(input)
panelDiagnostic(panelEvidence(last))
if (last.reviewPageOpen) {
return SpecPanelWait(last, false, true, leaveUnexpectedReviewPage(input))
}
last.problem?.let { return SpecPanelWait(last, false, true, failure(it.code, it.message)) }
if (last.specPanelOpen) return SpecPanelWait(last, true, true)
changed = changed || specActionSignature(last) != beforeSignature
pause(SPEC_SELECTION_POLL_MILLIS)
}
return SpecPanelWait(last, false, changed)
}
/** In-memory only; the signature is never logged, persisted, or uploaded. */
private fun specActionSignature(screen: ParsedPddScreen): List<SpecActionNodeSignature> = screen.sourceNodes.map { node ->
SpecActionNodeSignature(
node.path,
node.className,
node.bounds,
node.clickable,
node.scrollable,
node.selected,
node.checked,
node.enabled,
node.visible,
)
}
private fun panelEvidence(screen: ParsedPddScreen): String =
"type=${screen.specPanelType};scrollables=${screen.panelScrollableCount};headings=${screen.panelHeadingCount};" +
"options=${screen.panelOptionCount};summary=${screen.hasSelectionSummary};quantity=${screen.hasQuantityControls};" +
"orderAction=${screen.hasOrderSubmitAction};pageEvidence=${screen.pageEvidenceMatched}"
private fun specEntryEvidence(screen: ParsedPddScreen, candidateCount: Int, entryReadyWaitPolls: Int = 0): String =
"specEntryCandidates=$candidateCount;explicit=${screen.explicitSpecEntryCount};" +
"nested=${screen.nestedSpecEntryCount};bottomPurchase=${screen.bottomPurchaseEntryCount};" +
"panelAlreadyOpen=${screen.specPanelOpen};reviewPage=${screen.reviewPageOpen};" +
"pageEvidence=${screen.pageEvidenceMatched};entryReadyWaitPolls=$entryReadyWaitPolls;" +
"entryReadyWaitMillis=${entryReadyWaitPolls * SPEC_ENTRY_READY_POLL_MILLIS}"
private fun specEntryMatchesAliases(screen: ParsedPddScreen, candidate: SnapshotNode, aliases: List<String>): Boolean {
val prefix = "${candidate.path}/"
return (sequenceOf(candidate) + screen.sourceNodes.asSequence().filter { it.path.startsWith(prefix) })
@@ -481,39 +350,44 @@ class PurchaseRehearsalExecutor(
target: String,
selectionProofs: MutableMap<String, ExactSpecSelectionProof>,
): PurchaseExecutionOutcome? {
val initial = currentScreen(input)
initial.problem?.let { return failure(it.code, it.message) }
if (rememberExactSelection(initial, dimension, target, selectionProofs)) return null
var lastClickReason = FreshClickReason.UNKNOWN
var clickSucceeded = false
repeat(SPEC_SELECTION_CLICK_ATTEMPTS) {
val lookup = locateExactSpec(input, dimension, target)
lookup.failure?.let { return it }
val outcome = driver.clickFreshDetailed(requireNotNull(lookup.node))
lastClickReason = outcome.reason
if (outcome.result == FreshActionResult.AMBIGUOUS) {
return failure(SPEC_TARGET_AMBIGUOUS, "精确规格匹配到多个控件")
}
clickSucceeded = clickSucceeded || outcome.result == FreshActionResult.SUCCESS
val lookup = locateExactSpec(input, dimension, target)
lookup.failure?.let { return it }
val outcome = driver.clickFreshDetailed(requireNotNull(lookup.node))
if (outcome.result == FreshActionResult.AMBIGUOUS) {
return failure(SPEC_TARGET_AMBIGUOUS, "精确规格匹配到多个控件")
// PDD rerenders the remaining options after a color selection. A
// stale node can report NOT_FOUND/FAILED even when the exact option
// was selected. Always verify the observable selected state before
// deciding whether to refind and retry the same exact target.
val verifyPolls = if (outcome.result == FreshActionResult.SUCCESS) {
SPEC_SELECTION_SUCCESS_VERIFY_POLLS
} else {
SPEC_SELECTION_FAILED_VERIFY_POLLS
}
repeat(verifyPolls) {
val refreshed = currentScreen(input)
refreshed.problem?.let { return failure(it.code, it.message) }
if (isExactSpecSelected(refreshed, dimension, target)) {
exactSpecSelectionProof(refreshed, dimension, target)?.let { proof ->
selectionProofs[dimension] = proof
}
return null
}
pause(SPEC_SELECTION_POLL_MILLIS)
}
}
var verification = waitForExactSelection(input, dimension, target, selectionProofs, SPEC_SELECTION_SUCCESS_VERIFY_POLLS)
verification.failure?.let { return it }
if (verification.confirmed) return null
// Reacquire the exact option after PDD has had a chance to rerender.
// The fallback is still bounded to one center gesture and the same
// unique, server-provided exact value.
val refreshedLookup = locateExactSpec(input, dimension, target)
refreshedLookup.failure?.let { return it }
val gesture = driver.tapSpecFresh(requireNotNull(refreshedLookup.node))
if (gesture == FreshActionResult.AMBIGUOUS) {
return failure(SPEC_TARGET_AMBIGUOUS, "精确规格手势目标不唯一")
}
if (gesture == FreshActionResult.SUCCESS) {
verification = waitForExactSelection(input, dimension, target, selectionProofs, SPEC_SELECTION_SUCCESS_VERIFY_POLLS)
verification.failure?.let { return it }
if (verification.confirmed) return null
return failure(SPEC_SELECTION_UNCONFIRMED, "无障碍点击和受控手势后未能确认精确选中状态")
}
if (outcome.result == FreshActionResult.SUCCESS) {
if (clickSucceeded) {
return failure(SPEC_SELECTION_UNCONFIRMED, "点击规格后未能确认精确选中状态")
}
val subreason = when (outcome.reason) {
val subreason = when (lastClickReason) {
FreshClickReason.ROOT_UNAVAILABLE -> SPEC_CLICK_ROOT_UNAVAILABLE
FreshClickReason.TARGET_NOT_FOUND -> SPEC_CLICK_TARGET_STALE
FreshClickReason.NO_CLICKABLE_ANCESTOR -> SPEC_CLICK_NO_CLICKABLE_ANCESTOR
@@ -523,38 +397,6 @@ class PurchaseRehearsalExecutor(
return failure(SPEC_CLICK_FAILED, subreason)
}
private data class ExactSelectionWait(
val confirmed: Boolean,
val failure: PurchaseExecutionOutcome? = null,
)
private fun waitForExactSelection(
input: PurchaseExecutionInput,
dimension: String,
target: String,
selectionProofs: MutableMap<String, ExactSpecSelectionProof>,
polls: Int,
): ExactSelectionWait {
repeat(polls) {
val refreshed = currentScreen(input)
refreshed.problem?.let { return ExactSelectionWait(false, failure(it.code, it.message)) }
if (rememberExactSelection(refreshed, dimension, target, selectionProofs)) return ExactSelectionWait(true)
pause(SPEC_SELECTION_POLL_MILLIS)
}
return ExactSelectionWait(false)
}
private fun rememberExactSelection(
screen: ParsedPddScreen,
dimension: String,
target: String,
selectionProofs: MutableMap<String, ExactSpecSelectionProof>,
): Boolean {
if (!isExactSpecSelected(screen, dimension, target)) return false
exactSpecSelectionProof(screen, dimension, target)?.let { proof -> selectionProofs[dimension] = proof }
return true
}
private fun isExactSpecSelected(screen: ParsedPddScreen, dimension: String, target: String): Boolean {
val candidates = screen.dimensions.filter { it.key == dimension }.flatMap { it.values }
if (candidates.any { it.text == target && (it.node.selected || it.node.checked) }) return true
@@ -649,40 +491,38 @@ class PurchaseRehearsalExecutor(
* that direction early.
*/
private fun locateExactSpec(input: PurchaseExecutionInput, dimension: String, target: String): SpecLookup {
data class Inspection(val lookup: SpecLookup?, val signature: String, val container: SnapshotNode?)
fun inspect(): Inspection {
fun inspect(): Pair<SpecLookup?, String> {
val screen = currentScreen(input)
screen.problem?.let { return Inspection(SpecLookup(failure = failure(it.code, it.message)), "", null) }
screen.problem?.let { return SpecLookup(failure = failure(it.code, it.message)) to "" }
if (!screen.specPanelOpen) {
return Inspection(SpecLookup(failure = failure("RULE_NOT_MATCHED", "商品规格面板已经关闭")), "", null)
return SpecLookup(failure = failure("RULE_NOT_MATCHED", "商品规格面板已经关闭")) to ""
}
val dimensionValues = screen.dimensions.filter { it.key == dimension }.flatMap { it.values }
val exact = dimensionValues.filter { it.text == target }
if (exact.size > 1) {
return Inspection(SpecLookup(failure = failure(SPEC_TARGET_AMBIGUOUS, "精确规格匹配到多个控件")), "", screen.specPanelContainer)
return SpecLookup(failure = failure(SPEC_TARGET_AMBIGUOUS, "精确规格匹配到多个控件")) to ""
}
if (exact.size == 1 && exact.single().available) return Inspection(SpecLookup(node = exact.single().node), "", screen.specPanelContainer)
if (exact.size == 1 && exact.single().available) return SpecLookup(node = exact.single().node) to ""
if (exact.size == 1) {
return Inspection(SpecLookup(failure = failure(SPEC_SAFE_TARGET_MISSING, "精确规格当前不可安全点击")), "", screen.specPanelContainer)
return SpecLookup(failure = failure(SPEC_SAFE_TARGET_MISSING, "精确规格当前不可安全点击")) to ""
}
val signature = screen.dimensions.joinToString("|") { item ->
"${item.key}:${item.values.joinToString(",") { value -> "${value.text}:${value.available}" }}"
}
return Inspection(null, signature, screen.specPanelContainer)
return null to signature
}
var inspected = inspect()
inspected.lookup?.let { return it }
var currentSignature = inspected.signature
inspected.first?.let { return it }
var currentSignature = inspected.second
for ((direction, limit) in listOf(SwipeDirection.DOWN to 3, SwipeDirection.UP to 6)) {
for (attempt in 0 until limit) {
val container = inspected.container ?: break
if (!driver.swipePurchaseIn(container, direction, 350)) break
if (!driver.swipePurchase(direction, 350)) break
pause(300)
inspected = inspect()
inspected.lookup?.let { return it }
if (inspected.signature == currentSignature) break
currentSignature = inspected.signature
inspected.first?.let { return it }
if (inspected.second == currentSignature) break
currentSignature = inspected.second
}
}
return SpecLookup(failure = failure(SPEC_TARGET_NOT_VISIBLE, "有界搜索后未找到精确规格"))
@@ -750,17 +590,9 @@ class PurchaseRehearsalExecutor(
return null
}
private fun applyPostAction(input: PurchaseExecutionInput, action: PurchaseAction): PurchaseExecutionOutcome? {
private fun applyPostAction(action: PurchaseAction): PurchaseExecutionOutcome? {
if (action.waitAfterMs > 0) pause(action.waitAfterMs)
action.swipeAfter?.let { swipe ->
// The stock purchase rule asks to reveal additional selector rows after
// opening the sheet. A fully-evidenced non-scrollable selector has no
// scroll target, and treating that absence as an action failure blocks
// an otherwise safe exact-spec flow. Keep all other configured swipes
// mandatory; this exception is limited to that confirmed panel state.
if (action.type == PurchaseActionType.OPEN_SPEC_PANEL &&
currentScreen(input).specPanelType == SpecPanelType.NON_SCROLLABLE_CONFIRMATION
) return null
repeat(swipe.count) { index ->
if (!driver.swipePurchase(swipe.direction, swipe.durationMs)) {
return failure("RULE_ACTION_FAILED", "规则要求的有限滑动失败")
@@ -812,9 +644,7 @@ class PurchaseRehearsalExecutor(
private const val OPEN_PRODUCT_POLL_LIMIT = 50
private const val OPEN_PRODUCT_RETRY_POLLS = 10
private const val OPEN_PRODUCT_POLL_MILLIS = 100L
private const val SPEC_ENTRY_READY_WAIT_POLLS = 20
private const val SPEC_ENTRY_READY_POLL_MILLIS = 100L
private const val SPEC_POST_CLICK_VERIFY_POLLS = 30
private const val SPEC_SELECTION_CLICK_ATTEMPTS = 3
private const val SPEC_SELECTION_SUCCESS_VERIFY_POLLS = 20
private const val SPEC_SELECTION_FAILED_VERIFY_POLLS = 5
private const val SPEC_SELECTION_POLL_MILLIS = 100L
@@ -823,8 +653,7 @@ class PurchaseRehearsalExecutor(
private const val SPEC_ENTRY_TARGET_AMBIGUOUS = "PURCHASE_SPEC_ENTRY_TARGET_AMBIGUOUS"
private const val SPEC_ENTRY_CLICK_FAILED = "PURCHASE_SPEC_ENTRY_CLICK_FAILED"
private const val SPEC_ENTRY_OPENED_REVIEW = "PURCHASE_SPEC_ENTRY_OPENED_REVIEW"
private const val SPEC_ENTRY_CLICK_NO_EFFECT = "PURCHASE_SPEC_ENTRY_CLICK_NO_EFFECT"
private const val SPEC_PANEL_EVIDENCE_NOT_MATCHED = "PURCHASE_SPEC_PANEL_EVIDENCE_NOT_MATCHED"
private const val SPEC_PANEL_NOT_OPENED = "PURCHASE_SPEC_PANEL_NOT_OPENED"
private const val SPEC_TARGET_NOT_VISIBLE = "PURCHASE_SPEC_TARGET_NOT_VISIBLE"
private const val SPEC_TARGET_AMBIGUOUS = "PURCHASE_SPEC_TARGET_AMBIGUOUS"
private const val SPEC_SAFE_TARGET_MISSING = "PURCHASE_SPEC_SAFE_TARGET_MISSING"
@@ -226,26 +226,6 @@ class AgentApiClient(private val serverUrl: String) {
)
}
fun recoverRegistration(info: DeviceInfo, token: String?, recoveryCode: String): RegistrationResult {
require(recoveryCode.isNotBlank()) { "请输入管理员恢复码" }
val payload = JSONObject()
.put("requestId", UUID.randomUUID().toString())
.put("installId", info.installId)
.put("name", info.name)
.put("manufacturer", info.manufacturer)
.put("model", info.model)
.put("androidVersion", info.androidVersion)
.put("agentVersion", info.agentVersion)
.put("pddVersion", info.pddVersion)
.put("capabilities", JSONArray(info.capabilities))
val data = post("/api/agent/v1/register", payload, token, recoveryCode).getJSONObject("data")
return RegistrationResult(
deviceId = data.getLong("deviceId"),
deviceToken = data.optString("deviceToken").takeIf { it.isNotBlank() },
heartbeatIntervalSeconds = data.optInt("heartbeatIntervalSeconds", 15),
)
}
fun heartbeat(token: String, currentTaskId: Long?, capabilities: List<String> = emptyList()): HeartbeatResult {
val payload = JSONObject()
.put("requestId", UUID.randomUUID().toString())
@@ -610,11 +590,11 @@ class AgentApiClient(private val serverUrl: String) {
leaseVersion = data.getLong("leaseVersion"),
)
private fun post(path: String, payload: JSONObject, token: String?, recoveryCode: String? = null): JSONObject {
return requireNotNull(request("POST", path, payload, token, recoveryCode))
private fun post(path: String, payload: JSONObject, token: String?): JSONObject {
return requireNotNull(request("POST", path, payload, token))
}
private fun request(method: String, path: String, payload: JSONObject?, token: String?, recoveryCode: String? = null): JSONObject? {
private fun request(method: String, path: String, payload: JSONObject?, token: String?): JSONObject? {
val connection = (URL(serverUrl + path).openConnection() as HttpURLConnection).apply {
requestMethod = method
connectTimeout = 10_000
@@ -625,7 +605,6 @@ class AgentApiClient(private val serverUrl: String) {
setRequestProperty("Accept", "application/json")
setRequestProperty("Cache-Control", "no-store")
if (!token.isNullOrBlank()) setRequestProperty("Authorization", "Bearer $token")
if (!recoveryCode.isNullOrBlank()) setRequestProperty("X-GoAuto-Device-Recovery-Code", recoveryCode)
}
try {
if (payload != null) {
@@ -187,12 +187,11 @@ class AgentForegroundService : Service() {
if (!registeredThisProcess.get()) {
val registration = api.register(deviceInfo(), credentials?.token)
if (registration.deviceToken != null) {
if (credentials == null) {
val issuedToken = registration.deviceToken
?: error("注册请求已处理,但未返回新 Token;请联系管理员重新签发")
identityStore.saveCredentials(registration.deviceId, issuedToken)
credentials = identityStore.credentials() ?: error("设备凭据保存失败")
} else if (credentials == null) {
error("注册请求已处理,但未返回新 Token;请联系管理员重新签发")
} else {
check(registration.deviceId == credentials.deviceId) { "服务端设备身份与本地不一致" }
}
@@ -473,7 +472,6 @@ class AgentForegroundService : Service() {
openLink = { PddLinkLauncher(this).open(it) },
probeSpecs = { collectPurchaseProbe(accessibility, task, parsedRule) },
stepChanged = { step -> purchaseStore.updateStep(task.taskId, task.taskAttemptId, step) },
panelDiagnostic = { evidence -> Log.i("GoAutoPurchasePanel", "task=${task.taskId};$evidence") },
beforeOrderSubmit = { evidence ->
val boundaryRequestId = UUID.randomUUID().toString()
val finalEvidence = JSONObject()
@@ -367,64 +367,6 @@ class PddProductDetailCollectorTest {
assertEquals("bottom_purchase", parsed.specEntrySource)
}
@Test
fun bottomPurchaseEntriesPreferRightmostCandidate() {
val snapshot = UiSnapshot(
PDD_PACKAGE,
ACTIVITY,
listOf(
node("content", "", 0, 0, 1080, 2200, resourceId = "android:id/content", className = "android.widget.FrameLayout"),
node("buy-left", "单独购买", 446, 2000, 685, 2160, clickable = true),
node("buy-right", "发起拼单", 685, 2000, 1080, 2160, clickable = true),
),
)
val parsed = PddScreenParser.parse(snapshot, config(), GOODS_ID, evidence())
assertEquals("buy-right", parsed.specEntry?.path)
assertEquals("buy-right", parsed.specEntryClickTarget?.path)
assertEquals("bottom_purchase_rightmost", parsed.specEntrySource)
assertEquals(2, parsed.bottomPurchaseEntryCount)
}
@Test
fun equallyRightmostBottomPurchaseEntriesPreferSmallerArea() {
val snapshot = UiSnapshot(
PDD_PACKAGE,
ACTIVITY,
listOf(
node("content", "", 0, 0, 1080, 2200, resourceId = "android:id/content", className = "android.widget.FrameLayout"),
node("buy-large", "发起拼单", 685, 2000, 1080, 2160, clickable = true),
node("buy-small", "立即购买", 785, 2000, 980, 2140, clickable = true),
),
)
val parsed = PddScreenParser.parse(snapshot, config(), GOODS_ID, evidence())
assertEquals("buy-small", parsed.specEntry?.path)
assertEquals("bottom_purchase_rightmost", parsed.specEntrySource)
}
@Test
fun bottomPurchaseCandidatesStillExcludeReviewOrderAndPaymentContexts() {
val snapshot = UiSnapshot(
PDD_PACKAGE,
ACTIVITY,
listOf(
node("content", "", 0, 0, 1080, 2200, resourceId = "android:id/content", className = "android.widget.FrameLayout"),
node("reviews", "商品评价", 446, 1900, 685, 2160, clickable = true),
node("reviews/buy", "购买", 480, 2000, 650, 2100, parentPath = "reviews"),
node("order", "购买并提交订单", 685, 2000, 880, 2160, clickable = true),
node("payment", "购买后立即支付", 880, 2000, 1080, 2160, clickable = true),
),
)
val parsed = PddScreenParser.parse(snapshot, config(), GOODS_ID, evidence())
assertEquals(null, parsed.specEntry)
assertEquals(0, parsed.bottomPurchaseEntryCount)
}
@Test
fun bottomPurchaseInsideReviewCardIsNeverSpecEntry() {
val snapshot = UiSnapshot(
@@ -629,74 +571,6 @@ class PddProductDetailCollectorTest {
assertTrue(parsed.dimensions.isEmpty())
}
@Test
fun nonScrollableSelectorWithTwoDimensionsDoesNotRequireInitialSelectedSummary() {
val snapshot = UiSnapshot(
PDD_PACKAGE,
ACTIVITY,
listOf(
node("content", "", 0, 0, 1080, 2200, resourceId = "android:id/content", className = "android.widget.FrameLayout"),
node("color-heading", "颜色", 20, 420, 300, 470),
node("color", "豹纹", 20, 490, 300, 550, clickable = true),
node("size-heading", "尺码", 20, 650, 300, 700),
node("size", "均码", 20, 720, 300, 780, clickable = true),
node("quantity", "1", 480, 1400, 600, 1480, className = "android.widget.EditText"),
node("decrease", "减少数量", 360, 1400, 470, 1480, clickable = true),
node("increase", "增加数量", 610, 1400, 720, 1480, clickable = true),
node("order", "提交订单", 20, 1900, 1060, 2100, clickable = true),
),
)
val parsed = PddScreenParser.parse(snapshot, config(), GOODS_ID, evidence())
assertTrue(parsed.specPanelOpen)
assertEquals(SpecPanelType.NON_SCROLLABLE_CONFIRMATION, parsed.specPanelType)
assertFalse(parsed.hasSelectionSummary)
assertEquals(2, parsed.dimensions.size)
assertEquals(2, parsed.panelOptionCount)
}
@Test
fun nestedSelectionRowCombinesPrefixAndDimensionBeforeChoosingClickableParent() {
val snapshot = UiSnapshot(
PDD_PACKAGE,
ACTIVITY,
listOf(
node("content", "", 0, 0, 1080, 2200, resourceId = "android:id/content", className = "android.widget.FrameLayout"),
node("selection-row", "", 20, 720, 1060, 860, clickable = true),
node("selection-row/prefix", "请选择", 48, 750, 220, 810, parentPath = "selection-row"),
node("selection-row/dimension", "颜色分类", 240, 750, 480, 810, parentPath = "selection-row"),
),
)
val parsed = PddScreenParser.parse(snapshot, config(), GOODS_ID, evidence())
assertEquals("selection-row", parsed.specEntry?.path)
assertEquals("selection-row", parsed.specEntryClickTarget?.path)
assertEquals("nested_selection", parsed.specEntrySource)
assertEquals(1, parsed.nestedSpecEntryCount)
}
@Test
fun nestedSelectionSemanticsNeverAcceptsOrderOrPaymentContainer() {
val snapshot = UiSnapshot(
PDD_PACKAGE,
ACTIVITY,
listOf(
node("content", "", 0, 0, 1080, 2200, resourceId = "android:id/content", className = "android.widget.FrameLayout"),
node("unsafe", "", 20, 720, 1060, 860, clickable = true),
node("unsafe/prefix", "请选择", 48, 750, 220, 810, parentPath = "unsafe"),
node("unsafe/dimension", "颜色", 240, 750, 480, 810, parentPath = "unsafe"),
node("unsafe/order", "提交订单", 700, 750, 1020, 810, parentPath = "unsafe"),
),
)
val parsed = PddScreenParser.parse(snapshot, config(), GOODS_ID, evidence())
assertEquals(null, parsed.specEntry)
assertEquals(0, parsed.nestedSpecEntryCount)
}
@Test
fun genericQuantityAndBuyControlsDoNotProveQuickConfirmation() {
val snapshot = UiSnapshot(
@@ -120,122 +120,6 @@ class PurchaseLiveAutomationTest {
assertFalse(driver.clicked.any { it.startsWith("微信") || it.contains("支付") })
}
@Test
fun `payment page returns to folded unpaid order detail and scrolls to read evidence`() {
val driver = LiveDriver(chooserAfterSubmit = true, orderEvidenceBelowFold = true)
val automation = PurchaseLiveAutomation(driver, pause = {})
val address = automation.updateShippingAddress("_cg54")
automation.finalConfirmation(input().copy(addressSuffix = "_cg54"), address)
automation.submitOrderOnce()
val order = automation.readOrderResult()
assertEquals("PDD-202608210001", order?.orderNo)
assertEquals(2, driver.postSubmitBackCount)
assertEquals(1, driver.genericSwipes)
assertFalse(driver.clicked.any { it.contains("支付") })
}
@Test
fun `successful order page enters the unique order detail before reading the result`() {
val driver = LiveDriver(orderDetailEntryAfterSubmit = true)
val automation = PurchaseLiveAutomation(driver, pause = {})
val address = automation.updateShippingAddress("_cg56")
automation.finalConfirmation(input().copy(addressSuffix = "_cg56"), address)
automation.submitOrderOnce()
val order = automation.readOrderResult()
assertEquals("PDD-202608210001", order?.orderNo)
assertEquals(1, driver.orderDetailEntryClicks)
assertEquals("order", driver.currentPage)
assertFalse(driver.clicked.any { it.contains("支付") })
}
@Test
fun `order detail entered from success page scrolls only to reveal folded result evidence`() {
val driver = LiveDriver(orderDetailEntryAfterSubmit = true, orderDetailEvidenceBelowFold = true)
val automation = PurchaseLiveAutomation(driver, pause = {})
val address = automation.updateShippingAddress("_cg58")
automation.finalConfirmation(input().copy(addressSuffix = "_cg58"), address)
automation.submitOrderOnce()
val order = automation.readOrderResult()
assertEquals("PDD-202608210001", order?.orderNo)
assertEquals(1, driver.orderDetailEntryClicks)
assertEquals(1, driver.genericSwipes)
assertFalse(driver.clicked.any { it.contains("支付") })
}
@Test
fun `unpaid order evidence on reused payment activity scrolls read only instead of backing out`() {
val driver = LiveDriver(postSubmitCaptureSequence = listOf("order-folded-payment-activity"))
val automation = PurchaseLiveAutomation(driver, pause = {})
val address = automation.updateShippingAddress("_cg59")
automation.finalConfirmation(input().copy(addressSuffix = "_cg59"), address)
automation.submitOrderOnce()
val order = automation.readOrderResult()
assertEquals("PDD-202608210001", order?.orderNo)
assertEquals(0, driver.postSubmitBackCount)
assertEquals(1, driver.genericSwipes)
assertFalse(driver.clicked.any { it.contains("支付") })
}
@Test
fun `ambiguous order detail entries stop without navigating`() {
val driver = LiveDriver(orderDetailEntryAfterSubmit = true, duplicateOrderDetailEntry = true)
val automation = PurchaseLiveAutomation(driver, pause = {})
val address = automation.updateShippingAddress("_cg57")
automation.finalConfirmation(input().copy(addressSuffix = "_cg57"), address)
automation.submitOrderOnce()
assertEquals(null, automation.readOrderResult())
assertEquals("PURCHASE_ORDER_DETAIL_ENTRY_AMBIGUOUS", automation.lastOrderReadFailure?.code)
assertEquals(0, driver.orderDetailEntryClicks)
assertFalse(driver.clicked.any { it.contains("支付") })
}
@Test
fun `payment transition frame after safe back reaches unpaid order evidence without payment clicks`() {
val driver = LiveDriver(postSubmitCaptureSequence = listOf("payment", "payment", "order"))
val automation = PurchaseLiveAutomation(driver, pause = {})
val address = automation.updateShippingAddress("_cg55")
automation.finalConfirmation(input().copy(addressSuffix = "_cg55"), address)
automation.submitOrderOnce()
val order = automation.readOrderResult()
assertEquals("PDD-202608210001", order?.orderNo)
assertEquals("2026-08-21T02:30:00Z", order?.submittedAt)
assertEquals(1, driver.postSubmitBackCount)
assertEquals(0, driver.genericSwipes)
assertFalse(driver.clicked.any { it.contains("支付") })
}
@Test
fun `continuous payment activity still stops at bounded post back samples without payment clicks`() {
val driver = LiveDriver(postSubmitCaptureSequence = List(4) { "payment" })
val automation = PurchaseLiveAutomation(driver, pause = {})
val address = automation.updateShippingAddress("_cg55")
automation.finalConfirmation(input().copy(addressSuffix = "_cg55"), address)
automation.submitOrderOnce()
assertEquals(null, automation.readOrderResult())
assertEquals("PURCHASE_ORDER_PAYMENT_REPEATED", automation.lastOrderReadFailure?.code)
assertEquals(
"支付页安全返回后持续无订单证据,已停止自动核单" +
"[paymentBackAttempts=1;consecutivePaymentSamplesAfterBack=3]",
automation.lastOrderReadFailure?.message,
)
assertEquals(4, driver.postSubmitCaptureCount)
assertEquals(1, driver.postSubmitBackCount)
assertEquals(0, driver.genericSwipes)
assertFalse(driver.clicked.any { it.contains("支付") })
}
@Test
fun `WeChat login is not touched and PDD is restored once before reading order detail`() {
val driver = LiveDriver(wechatLoginAfterSubmit = true)
@@ -426,10 +310,6 @@ class PurchaseLiveAutomationTest {
private val savedTransitionHidesSuffix: Boolean = false,
private val wechatLoginAfterSubmit: Boolean = false,
private val wechatRestoreStuck: Boolean = false,
private val orderEvidenceBelowFold: Boolean = false,
private val orderDetailEntryAfterSubmit: Boolean = false,
private val orderDetailEvidenceBelowFold: Boolean = false,
private val duplicateOrderDetailEntry: Boolean = false,
postSubmitCaptureSequence: List<String> = emptyList(),
) : PurchaseUiDriver {
private var page = "confirmation"
@@ -446,7 +326,6 @@ class PurchaseLiveAutomationTest {
var backCount = 0
var postSubmitBackCount = 0
var pddRestoreCount = 0
var orderDetailEntryClicks = 0
var postSubmitCaptureCount = 0
val currentPage: String get() = page
@@ -480,20 +359,7 @@ class PurchaseLiveAutomationTest {
node("address-summary", if (savedTransitionHidesSuffix) "已保存的收货信息" else address.substring(address.lastIndexOf("_cg"))),
))
"order" -> snapshot(listOf(node("status", "待付款"), node("order", "订单号:PDD-202608210001"), node("time", "下单时间:2026-08-21 10:30:00"), node("pay", "立即支付", clickable = true)))
"order-folded" -> snapshot(listOf(node("status", "待付款"), node("pay", "立即支付", clickable = true)))
"order-folded-payment-activity" -> UiSnapshot(PDD, "com.xunmeng.pinduoduo.app_pay.core.PayActivity", listOf(
node("status", "待付款"), node("pay", "立即支付", clickable = true),
))
"order-no-time" -> snapshot(listOf(node("status", "待付款"), node("order", "订单号:PDD-202608210001"), node("pay", "去支付", clickable = true)))
"success" -> snapshot(buildList {
add(node("success-title", "购买成功"))
add(node("detail-parent", "", clickable = true))
add(node("detail", "查看订单", parentPath = "detail-parent"))
if (duplicateOrderDetailEntry) {
add(node("detail-parent-2", "", clickable = true))
add(node("detail-2", "订单详情", parentPath = "detail-parent-2"))
}
})
"chooser" -> UiSnapshot(if (trustedChooser) "android" else "example.untrusted", "com.android.internal.app.ChooserActivity", listOf(
node("chooser-title", "选择要使用的应用"), node("wechat-1", "微信"), node("wechat-2", "微信分身"),
))
@@ -543,14 +409,9 @@ class PurchaseLiveAutomationTest {
page = when {
chooserAfterSubmit -> "chooser"
wechatLoginAfterSubmit -> "wechat-login"
orderDetailEntryAfterSubmit -> "success"
else -> "order"
}
}
"查看订单", "订单详情" -> {
orderDetailEntryClicks++
page = if (orderDetailEvidenceBelowFold) "order-folded" else "order"
}
}
return FreshActionResult.SUCCESS
}
@@ -567,11 +428,7 @@ class PurchaseLiveAutomationTest {
address = value
return FreshActionResult.SUCCESS
}
override fun swipePurchase(direction: SwipeDirection, durationMs: Long): Boolean {
genericSwipes++
if (page in setOf("order-folded", "order-folded-payment-activity") && direction == SwipeDirection.UP) page = "order"
return true
}
override fun swipePurchase(direction: SwipeDirection, durationMs: Long): Boolean { genericSwipes++; return true }
override fun swipePurchaseIn(target: SnapshotNode, direction: SwipeDirection, durationMs: Long): Boolean {
scopedSwipes++
if (target.path == "panel" && direction == SwipeDirection.DOWN) addressVisible = true
@@ -582,7 +439,7 @@ class PurchaseLiveAutomationTest {
if (page == "chooser" || page == "payment") postSubmitBackCount++
page = when (page) {
"chooser" -> "payment"
"payment" -> if (orderEvidenceBelowFold) "order-folded" else "order"
"payment" -> "order"
else -> "confirmation"
}
return true
@@ -8,7 +8,6 @@ import cn.ilapage.goauto.agent.automation.PurchaseAgentCapabilities
import cn.ilapage.goauto.agent.automation.PurchaseExecutionInput
import cn.ilapage.goauto.agent.automation.PurchaseRehearsalExecutor
import cn.ilapage.goauto.agent.automation.PurchaseRuleParser
import cn.ilapage.goauto.agent.automation.PurchaseSpecGesturePolicy
import cn.ilapage.goauto.agent.automation.PurchaseUiDriver
import cn.ilapage.goauto.agent.automation.RuleValidationException
import cn.ilapage.goauto.agent.automation.SnapshotNode
@@ -22,22 +21,6 @@ import org.junit.Assert.assertTrue
import org.junit.Test
class PurchaseRehearsalExecutorTest {
@Test
fun `spec gesture policy excludes irreversible and out of bounds targets`() {
fun target(label: String, bounds: NodeBounds = NodeBounds(20, 100, 300, 180)) = SnapshotNode(
"target", null, label, null, null, "android.widget.TextView", bounds,
clickable = true, scrollable = false, selected = false, checked = false, enabled = true, visible = true,
)
assertTrue(PurchaseSpecGesturePolicy.isAllowed(target("选择规格"), 1080, 2200))
assertTrue(PurchaseSpecGesturePolicy.isAllowed(target("XL"), 1080, 2200))
assertFalse(PurchaseSpecGesturePolicy.isAllowed(target("提交订单"), 1080, 2200))
assertFalse(PurchaseSpecGesturePolicy.isAllowed(target("确认购买"), 1080, 2200))
assertFalse(PurchaseSpecGesturePolicy.isAllowed(target("立即支付"), 1080, 2200))
assertFalse(PurchaseSpecGesturePolicy.isAllowed(target("修改地址"), 1080, 2200))
assertFalse(PurchaseSpecGesturePolicy.isAllowed(target("XL", NodeBounds(-20, 100, -2, 180)), 1080, 2200))
}
@Test
fun `shared current live rule matches Android contract`() {
val raw = requireNotNull(javaClass.classLoader?.getResource("purchase-rule-live-v1.json")) {
@@ -147,8 +130,6 @@ class PurchaseRehearsalExecutorTest {
assertEquals("rehearsal_completed", outcome.resultType)
assertTrue(driver.clicked.contains("XL"))
assertEquals(3, driver.upSwipeCount)
assertTrue(driver.swipeInPaths.isNotEmpty())
assertTrue(driver.swipeInPaths.all { it == "scroll" })
}
@Test
@@ -166,31 +147,18 @@ class PurchaseRehearsalExecutorTest {
}
@Test
fun `rerendered exact size falls back to one verified center gesture`() {
fun `rerendered exact size is refound and selected on the second attempt`() {
val driver = FakePurchaseDriver(
sizeClickResults = mutableListOf(FreshActionResult.NOT_FOUND),
specTapResult = FreshActionResult.SUCCESS,
sizeClickResults = mutableListOf(FreshActionResult.NOT_FOUND, FreshActionResult.SUCCESS),
)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("rehearsal_completed", outcome.resultType)
assertEquals(1, driver.sizeClickCount)
assertEquals(1, driver.specTapCount)
assertEquals(2, driver.sizeClickCount)
assertEquals("XL", driver.size)
}
@Test
fun `already selected exact size is not clicked again`() {
val driver = FakePurchaseDriver(initialSize = "XL")
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("rehearsal_completed", outcome.resultType)
assertEquals(0, driver.sizeClickCount)
assertEquals(0, driver.specTapCount)
}
@Test
fun `long exact size can be verified by one unambiguous short summary token`() {
val target = "XL【建议121-140斤】"
@@ -312,17 +280,16 @@ class PurchaseRehearsalExecutorTest {
}
@Test
fun `persistent exact size click failure stops after one action and one bounded gesture`() {
fun `persistent exact size click failure stops after three attempts`() {
val driver = FakePurchaseDriver(
sizeClickResults = mutableListOf(FreshActionResult.FAILED),
sizeClickResults = MutableList(3) { FreshActionResult.FAILED },
)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("PURCHASE_SPEC_CLICK_FAILED", outcome.errorCode)
assertEquals("action_click_false", outcome.message)
assertEquals(1, driver.sizeClickCount)
assertEquals(1, driver.specTapCount)
assertEquals(3, driver.sizeClickCount)
assertEquals(null, driver.size)
}
@@ -451,109 +418,6 @@ class PurchaseRehearsalExecutorTest {
assertEquals(50, pauses.size)
}
@Test
fun `open spec panel skips required follow-up swipe only for confirmed non-scrollable panel`() {
val driver = FakePurchaseDriver(nonScrollablePanel = true, purchaseSwipeSucceeds = false)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("rehearsal_completed", outcome.resultType)
assertEquals(0, driver.swipeCount)
}
@Test
fun `unrecognized opened panel returns only scalar panel evidence`() {
val driver = FakePurchaseDriver(unrecognizedPanel = true)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("PURCHASE_SPEC_PANEL_EVIDENCE_NOT_MATCHED", outcome.errorCode)
assertEquals(
"规格入口点击后页面已变化,但规格面板强证据不足 [type=UNKNOWN;scrollables=0;headings=0;options=0;summary=false;quantity=false;orderAction=false;pageEvidence=true]",
outcome.message,
)
assertEquals(0, driver.specTapCount)
assertFalse(outcome.message.orEmpty().contains("确认款式"))
}
@Test
fun `unchanged spec entry action uses one verified center gesture`() {
val driver = FakePurchaseDriver(
entryActionHasEffect = false,
specTapResult = FreshActionResult.SUCCESS,
)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("rehearsal_completed", outcome.resultType)
assertEquals(1, driver.specTapCount)
assertTrue(driver.panel)
}
@Test
fun `unchanged spec entry after action and gesture returns no effect`() {
val driver = FakePurchaseDriver(
entryActionHasEffect = false,
specTapResult = FreshActionResult.SUCCESS,
specTapHasEffect = false,
)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("PURCHASE_SPEC_ENTRY_CLICK_NO_EFFECT", outcome.errorCode)
assertEquals(1, driver.specTapCount)
assertFalse(driver.panel)
}
@Test
fun `missing spec entry emits scalar source counts without node text`() {
val diagnostics = mutableListOf<String>()
val pauses = mutableListOf<Long>()
val driver = FakePurchaseDriver(missingSpecEntry = true)
val outcome = PurchaseRehearsalExecutor(
driver,
{ driver.browser = true; true },
{ null },
pause = pauses::add,
panelDiagnostic = diagnostics::add,
).execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("PURCHASE_SPEC_ENTRY_NOT_FOUND", outcome.errorCode)
assertEquals(
"specEntryCandidates=0;explicit=0;nested=0;bottomPurchase=0;panelAlreadyOpen=false;reviewPage=false;pageEvidence=true;entryReadyWaitPolls=20;entryReadyWaitMillis=2000",
diagnostics.single(),
)
// One 100ms pause belongs to the existing open-product foreground poll;
// the diagnostic proves the entry-ready loop itself used exactly 20.
assertEquals(21, pauses.count { it == 100L })
assertFalse(diagnostics.single().contains("选择规格"))
}
@Test
fun `open spec panel waits for a late safe bottom purchase entry`() {
val pauses = mutableListOf<Long>()
val driver = FakePurchaseDriver(bottomPurchaseEntry = true, specEntryVisibleAfterPddCaptures = 4)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { "{\"dimensions\":[]}" }, pause = pauses::add)
.execute(input().copy(executionMode = "live", phase = "spec_probe"), PurchaseRuleParser.parse(liveRule()), PurchaseAgentCapabilities.supported)
assertEquals("spec_probe_completed", outcome.resultType)
assertTrue(driver.clickedPaths.contains("buy"))
// One 100ms pause belongs to the existing open-product foreground poll;
// two belong to the entry-ready wait before the bottom bar appears.
assertEquals(3, pauses.count { it == 100L })
}
@Test
fun `verify product waits for product evidence instead of a visible loading frame`() {
val pauses = mutableListOf<Long>()
val driver = FakePurchaseDriver(loadingPddCaptures = 1)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = pauses::add)
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("rehearsal_completed", outcome.resultType)
assertEquals(1, pauses.count { it == 100L })
}
@Test
fun `transient sold out page recovers before opening specs`() {
val driver = FakePurchaseDriver(soldOut = true, recoverSoldOutAfterPull = true)
@@ -618,22 +482,11 @@ class PurchaseRehearsalExecutorTest {
}
@Test
fun `live rule spec probe ignores historical mapping and returns before address or order actions`() {
fun `live rule spec probe returns before address or order actions`() {
val driver = FakePurchaseDriver()
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { "{\"dimensions\":[]}" }, pause = {})
.execute(
input().copy(
executionMode = "live",
phase = "spec_probe",
mappedColor = "历史颜色",
mappedSize = "历史尺码",
addressSuffix = "_cg42",
),
PurchaseRuleParser.parse(liveRule()),
PurchaseAgentCapabilities.supported,
)
.execute(input().copy(executionMode = "live", phase = "spec_probe", addressSuffix = "_cg42"), PurchaseRuleParser.parse(liveRule()), PurchaseAgentCapabilities.supported)
assertEquals("spec_probe_completed", outcome.resultType)
assertEquals("{\"dimensions\":[]}", outcome.probedSpecs)
assertFalse(driver.clicked.any { it.contains("订单") || it.contains("地址") || it.contains("支付") })
}
@@ -708,29 +561,6 @@ class PurchaseRehearsalExecutorTest {
assertFalse(driver.clicked.contains("商品评价"))
}
@Test
fun `spec entry failures include scalar diagnostics only`() {
val driver = FakePurchaseDriver(missingSpecEntry = true)
val outcome = PurchaseRehearsalExecutor(driver, { driver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("PURCHASE_SPEC_ENTRY_NOT_FOUND", outcome.errorCode)
assertEquals(
"没有找到安全的商品规格入口 [specEntryCandidates=0;explicit=0;nested=0;bottomPurchase=0;panelAlreadyOpen=false;reviewPage=false;pageEvidence=true;entryReadyWaitPolls=20;entryReadyWaitMillis=2000]",
outcome.message,
)
val ambiguousDriver = FakePurchaseDriver(forcedEntryClickReason = FreshClickReason.TARGET_AMBIGUOUS)
val ambiguous = PurchaseRehearsalExecutor(ambiguousDriver, { ambiguousDriver.browser = true; true }, { null }, pause = {})
.execute(input(), PurchaseRuleParser.parse(rule()), PurchaseAgentCapabilities.supported)
assertEquals("PURCHASE_SPEC_ENTRY_TARGET_AMBIGUOUS", ambiguous.errorCode)
assertEquals(
"规格入口点击目标不唯一 [specEntryCandidates=1;explicit=1;nested=0;bottomPurchase=0;panelAlreadyOpen=false;reviewPage=false;pageEvidence=true;entryReadyWaitPolls=0;entryReadyWaitMillis=0]",
ambiguous.message,
)
}
@Test
fun `parser keeps type only compatibility and rejects dangerous or unknown fields`() {
val legacy = """{"schemaVersion":1,"ruleType":"pddPurchase","requiredCapabilities":["purchase.rehearsal.v1"],"actions":[{"type":"openProduct"}]}"""
@@ -879,9 +709,6 @@ class PurchaseRehearsalExecutorTest {
private val priceCent: Long = 2_000,
private val duplicateOpen: Boolean = false,
private val bottomPurchaseEntry: Boolean = false,
private val missingSpecEntry: Boolean = false,
private val specEntryVisibleAfterPddCaptures: Int = 0,
private val loadingPddCaptures: Int = 0,
private val includeReviewEntry: Boolean = false,
private val openReviewOnBottomClick: Boolean = false,
private val reviewBackSucceeds: Boolean = true,
@@ -891,10 +718,6 @@ class PurchaseRehearsalExecutorTest {
private val sizeClickResults: MutableList<FreshActionResult> = mutableListOf(),
private val forcedSizeClickReason: FreshClickReason? = null,
private val forcedEntryClickReason: FreshClickReason? = null,
private val initialSize: String? = null,
private val entryActionHasEffect: Boolean = true,
private val specTapResult: FreshActionResult = FreshActionResult.FAILED,
private val specTapHasEffect: Boolean = true,
private val sizeSelectsOnFailedClick: Boolean = false,
private val selectedSizeSummaryOverride: String? = null,
private val hideSizeSelectedState: Boolean = false,
@@ -909,14 +732,11 @@ class PurchaseRehearsalExecutorTest {
private val loseEvidenceAfterPull: Boolean = false,
private val unavailableSizes: Set<String> = emptySet(),
allSpecsUnavailable: Boolean = false,
private val nonScrollablePanel: Boolean = false,
private val unrecognizedPanel: Boolean = false,
private val purchaseSwipeSucceeds: Boolean = true,
) : PurchaseUiDriver {
var browser = false
var panel = false
var color: String? = null
var size: String? = initialSize
var size: String? = null
var quantity = 1L
var swipeCount = 0
var upSwipeCount = 0
@@ -924,13 +744,10 @@ class PurchaseRehearsalExecutorTest {
var sizeClickCount = 0
var pullDownCount = 0
var backCount = 0
var specTapCount = 0
val swipeInPaths = mutableListOf<String>()
private var soldOut = soldOut
private var allSpecsUnavailable = allSpecsUnavailable
private var productEvidenceLost = false
private var reviewPage = false
private var pddCaptureCount = 0
val clicked = mutableListOf<String>()
val clickedPaths = mutableListOf<String>()
@@ -941,10 +758,6 @@ class PurchaseRehearsalExecutorTest {
openNodes += node("content", "", 0, 0, 1080, 2200)
return UiSnapshot("com.heytap.browser", "BrowserActivity", openNodes)
}
pddCaptureCount++
if (pddCaptureCount <= loadingPddCaptures) {
return UiSnapshot(PDD, ACTIVITY, listOf(node("content", "", 0, 0, 1080, 2200)))
}
if (!panel) {
if (reviewPage) {
return UiSnapshot(PDD, ACTIVITY, listOf(
@@ -960,32 +773,23 @@ class PurchaseRehearsalExecutorTest {
if (soldOut) {
return UiSnapshot(PDD, ACTIVITY, listOf(
node("content", "", 0, 0, 1080, 2200),
node("title", "测试商品标题文本", 20, 200, 900, 280, className = "android.widget.ViewPager"),
node("sold-out", "商品已售罄", 100, 300, 900, 380),
node("similar", "相似商品", 100, 500, 900, 580),
))
}
val nodes = mutableListOf(
node("content", "", 0, 0, 1080, 2200),
node("title", "测试商品标题文本", 20, 200, 900, 280, className = "android.widget.ViewPager"),
)
val specEntryReady = pddCaptureCount > specEntryVisibleAfterPddCaptures
if (bottomPurchaseEntry && specEntryReady) {
if (bottomPurchaseEntry) {
nodes += node("buy", "", 500, 1800, 1080, 2180, clickable = true)
nodes += node("buy/price", "¥20.00", 560, 1840, 760, 1910, parentPath = "buy")
nodes += node("buy/label", "免拼购买", 780, 1840, 1040, 1910, parentPath = "buy")
} else if (!missingSpecEntry && specEntryReady) {
} else {
nodes += node("spec", "选择规格", 20, 1000, 900, 1100, clickable = true)
}
if (includeReviewEntry) nodes += node("review", "商品评价", 20, 1200, 900, 1300, clickable = true)
return UiSnapshot(PDD, ACTIVITY, nodes)
}
if (unrecognizedPanel) {
return UiSnapshot(PDD, ACTIVITY, listOf(
node("content", "", 0, 0, 1080, 2200),
node("panel-title", "确认款式", 20, 396, 300, 430),
))
}
val hideColor = hideColorAfterQuantitySet && quantity == 2L
val hideSize = hideSizeAfterQuantitySet && quantity == 2L
val hideSummary = hideSelectedSummaryAfterQuantitySet && quantity == 2L
@@ -998,8 +802,8 @@ class PurchaseRehearsalExecutorTest {
node("content", "", 0, 0, 1080, 2200),
node("price", "¥${priceCent / 100}.${(priceCent % 100).toString().padStart(2, '0')}", 20, 300, 300, 360),
node("title", "确认款式", 20, 396, 300, 430),
node("scroll", "", 0, 400, 1080, 950, scrollable = true),
)
if (!nonScrollablePanel) nodes += node("scroll", "", 0, 400, 1080, 950, scrollable = true)
if (!hideSummary) {
nodes += node("selected", "已选 $displayedSummary", 20, 365, 700, 395)
}
@@ -1044,9 +848,7 @@ class PurchaseRehearsalExecutorTest {
clicked += target.label
clickedPaths += target.path
if (target.path == "buy") {
if (entryActionHasEffect) {
if (openReviewOnBottomClick) reviewPage = true else panel = true
}
if (openReviewOnBottomClick) reviewPage = true else panel = true
return FreshActionResult.SUCCESS
}
when (target.label) {
@@ -1056,7 +858,7 @@ class PurchaseRehearsalExecutorTest {
if (result == FreshActionResult.SUCCESS || openPddOnFailedClick) browser = false
return result
}
"选择规格", "免拼购买" -> if (entryActionHasEffect) panel = true
"选择规格", "免拼购买" -> panel = true
in sizes -> {
sizeClickCount++
val result = sizeClickResults.removeFirstOrNull() ?: FreshActionResult.SUCCESS
@@ -1102,17 +904,6 @@ class PurchaseRehearsalExecutorTest {
override fun tapPurchaseFresh(target: SnapshotNode): FreshActionResult = clickFresh(target)
override fun tapSpecFresh(target: SnapshotNode): FreshActionResult {
specTapCount++
if (specTapResult != FreshActionResult.SUCCESS || !specTapHasEffect) return specTapResult
when {
target.path in setOf("spec", "buy") -> panel = true
target.label in sizes -> size = target.label
target.label in colors -> color = target.label
}
return FreshActionResult.SUCCESS
}
override fun inputFresh(target: SnapshotNode, value: String): FreshActionResult {
quantity = value.toLong()
return FreshActionResult.SUCCESS
@@ -1121,13 +912,11 @@ class PurchaseRehearsalExecutorTest {
override fun swipePurchase(direction: SwipeDirection, durationMs: Long): Boolean {
swipeCount++
if (direction == SwipeDirection.UP) upSwipeCount++
return purchaseSwipeSucceeds
return true
}
override fun swipePurchaseIn(target: SnapshotNode, direction: SwipeDirection, durationMs: Long): Boolean {
swipeInPaths += target.path
return swipePurchase(direction, durationMs)
}
override fun swipePurchaseIn(target: SnapshotNode, direction: SwipeDirection, durationMs: Long): Boolean =
swipePurchase(direction, durationMs)
override fun pullDownGoodsPage(): Boolean {
pullDownCount++
+59 -20
View File
@@ -67,6 +67,7 @@ CORE_DOCUMENT_REQUIREMENTS = {
),
"docs/00-project-profile.md": (
"## 基本信息",
"## 项目治理模式",
"## 建设基线",
"## 交付单元",
"## 文档事实来源",
@@ -74,6 +75,7 @@ CORE_DOCUMENT_REQUIREMENTS = {
"## 当前阶段",
),
"docs/01-workflow.md": (
"## 语言与术语",
"## 事实来源",
"## 权威源与事实边界",
"## Gitea 交互与工单最小读取",
@@ -102,6 +104,7 @@ CORE_DOCUMENT_REQUIREMENTS = {
),
"docs/04-local-development-and-verification.md": (
"## Windows PowerShell 与 UTF-8",
"### PowerShell 语法与外部命令",
"## 通用检查",
"## 服务端验证",
"## Web 验证",
@@ -271,9 +274,14 @@ def check_agent_efficiency_rules(errors: list[str], root: Path = ROOT) -> None:
"#### 明确停止条件",
"单元任务是唯一正式实施单位",
"高风险修改必须停止",
"### 项目治理模式与明确授权后的执行",
"默认采用轻量治理",
"不能单独构成人工授权",
"不得仅因操作不可逆而重复询问或拒绝",
"平台自身强制的审批、安全策略或权限限制继续有效",
"用户没有明确验收通过前不得关闭",
"只有长期事实变化时才更新 Wiki",
"Gitea 工单是单次任务需求、变化、实现、测试、提交和验收的唯一事实来源",
"需要工单的任务以 Gitea 工单作为单次需求、变化、实现、测试、提交和验收的事实来源",
"Wiki 同步由长期事实变化触发,不由任务完成触发",
"标准流程不创建 Wiki 任务归档",
"工单默认只在开始实施、集中回写待验收、验收关闭三个节点更新",
@@ -289,6 +297,11 @@ def check_agent_efficiency_rules(errors: list[str], root: Path = ROOT) -> None:
"不得把模板自带的本地 `docs/` 当作新项目 Wiki 已初始化的证据",
"提交只包含当前工单相关文件",
"不得仅为设置编码重复启动一层 PowerShell",
"不得默认套用 Bash",
"复杂正则优先使用变量或 `rg -e`",
"单引号 PowerShell here-string",
"`foreach`、`if` 等语句块",
"使用真实目录配合 `-g/--glob`",
"文件解码和控制台输出分别处理",
"不得默认使用 `-ExecutionPolicy Bypass`",
"### 工单与设计证据双门禁",
@@ -316,6 +329,9 @@ def check_agent_efficiency_rules(errors: list[str], root: Path = ROOT) -> None:
"不得臆造用户原话",
"不复制完整聊天",
"Gitea 工单全文不导出到仓库",
"用户可以使用中文、英文或合理的中英混合语言交流",
"默认使用中文分析、回复、编写工单和维护内部项目文档",
"日志和错误原文保持原样",
)
for section in missing_sections(content, required):
errors.append(f"AGENTS.md 缺少:{section}")
@@ -562,32 +578,47 @@ def run_check(args: argparse.Namespace) -> int:
def run_sync(args: argparse.Namespace) -> int:
"""--verify 依次执行导出、结构检查和一致性校验,替代原来的三条命令。"""
"""同步核心镜像;--verify 复用本轮完整读取完成初始化验证。"""
if args.verify:
steps = (
("同步", lambda: run_sync(
argparse.Namespace(check=False, verify=False, config=args.config))),
("结构检查", lambda: run_check(argparse.Namespace(strict=True))),
("一致性校验", lambda: run_sync(
argparse.Namespace(check=True, verify=False, config=args.config))),
)
for name, step in steps:
code = step()
if code != 0:
print(f"错误:{name}未通过,已停止")
return code
try:
config = load_config(Path(args.config).resolve())
messages = sync_all(
config, WikiClient(config), check=False, deep_check=True
)
except WikiDocsError as exc:
print(f"错误:{exc}")
return 1
for message in messages:
print(message)
code = run_check(argparse.Namespace(strict=True))
if code != 0:
print("错误:结构检查未通过,已停止")
return code
print("Wiki 镜像初始化验证通过")
return 0
try:
config = load_config(Path(args.config).resolve())
messages = sync_all(config, WikiClient(config), check=args.check)
deep_check = getattr(args, "deep_check", False)
messages = sync_all(
config,
WikiClient(config),
check=args.check or deep_check,
deep_check=deep_check,
)
except WikiDocsError as exc:
print(f"错误:{exc}")
return 1
for message in messages:
print(message)
print("Wiki 镜像检查通过" if args.check else "Wiki 镜像同步完成")
print(
"Wiki 镜像深度检查通过"
if deep_check
else "Wiki 镜像检查通过"
if args.check
else "Wiki 镜像同步完成"
)
return 0
@@ -652,13 +683,21 @@ def main() -> int:
p_check.set_defaults(func=run_check)
p_sync = sub.add_parser("sync", help="从 Gitea Wiki 单向同步核心 docs 镜像")
p_sync.add_argument(
"--check", action="store_true", help="只检查 Wiki 与镜像是否一致,不写文件"
sync_mode = p_sync.add_mutually_exclusive_group()
sync_mode.add_argument(
"--check",
action="store_true",
help="按 revision 快速检查 Wiki 与镜像,不写文件",
)
p_sync.add_argument(
sync_mode.add_argument(
"--deep-check",
action="store_true",
help="下载全部 Wiki 正文并逐页检查镜像,不写文件",
)
sync_mode.add_argument(
"--verify",
action="store_true",
help="依次执行导出、check --strict 和一致性校验",
help="完整读取并导出 Wiki,再执行 check --strict 初始化验证",
)
p_sync.add_argument(
"--config", default=str(DEFAULT_CONFIG), help="Wiki 页面映射 JSON 文件"
+137 -14
View File
@@ -8,6 +8,7 @@ import os
import re
import subprocess
import tempfile
import time
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path, PurePosixPath
@@ -26,6 +27,7 @@ HEADER_PATTERN = re.compile(
rf"{re.escape(MIRROR_END)}\n\n(?P<body>.*)\Z",
re.DOTALL,
)
RATE_LIMIT_RETRY_DELAYS = (1.0, 2.0, 4.0)
class WikiDocsError(RuntimeError):
@@ -142,11 +144,33 @@ class WikiClient:
data = json.dumps(payload, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = "application/json"
request = Request(url, data=data, headers=headers, method=method)
try:
with urlopen(request, timeout=30) as response:
body = response.read()
except HTTPError as exc:
if self.token and method == "GET" and exc.code in {401, 403, 404}:
request_error: HTTPError | URLError | None = None
body = b""
for attempt in range(len(RATE_LIMIT_RETRY_DELAYS) + 1):
try:
with urlopen(request, timeout=30) as response:
body = response.read()
request_error = None
break
except HTTPError as exc:
if (
method == "GET"
and exc.code == 429
and attempt < len(RATE_LIMIT_RETRY_DELAYS)
):
exc.close()
time.sleep(RATE_LIMIT_RETRY_DELAYS[attempt])
continue
request_error = exc
break
except URLError as exc:
request_error = exc
break
else: # pragma: no cover - for 循环必定通过成功或异常分支退出
raise WikiDocsError(f"Gitea API {method} {api_path} 请求失败")
if isinstance(request_error, HTTPError):
if self.token and method == "GET" and request_error.code in {401, 403, 404}:
# 公共仓库可能可匿名读取,而当前 shell 中的通用令牌属于
# 另一个实例或已失效。只对只读请求安全降级为匿名访问。
anonymous_headers = {"Accept": "application/json"}
@@ -167,12 +191,13 @@ class WikiClient:
f"无法连接 Gitea:{anonymous_exc.reason}"
) from anonymous_exc
else:
detail = exc.read().decode("utf-8", errors="replace")
detail = request_error.read().decode("utf-8", errors="replace")
raise WikiDocsError(
f"Gitea API {method} {api_path} 返回 {exc.code}: {detail}"
) from exc
except URLError as exc:
raise WikiDocsError(f"无法连接 Gitea:{exc.reason}") from exc
f"Gitea API {method} {api_path} 返回 "
f"{request_error.code}: {detail}"
) from request_error
elif isinstance(request_error, URLError):
raise WikiDocsError(f"无法连接 Gitea:{request_error.reason}") from request_error
if not body:
return None
try:
@@ -381,8 +406,73 @@ def check_mirror(mapping: Mapping, page: WikiPage, path: Path) -> list[str]:
return errors
def sync_all(config: Config, client: WikiClient, *, check: bool = False) -> list[str]:
"""检查或写入所有显式映射;绝不处理映射外的文件。"""
def _metadata_revision(metadata: dict[str, Any]) -> str | None:
last_commit = metadata.get("last_commit")
revision = last_commit.get("sha") if isinstance(last_commit, dict) else None
return revision if isinstance(revision, str) and revision else None
def _find_page_metadata(
pages: list[dict[str, Any]], page_name: str
) -> dict[str, Any]:
metadata = next(
(
item
for item in pages
if item.get("title") == page_name or item.get("sub_url") == page_name
),
None,
)
if metadata is None:
raise WikiDocsError(
f"Wiki 页面不存在:{page_name};不会自动删除或重命名本地镜像"
)
return metadata
def _metadata_identity(
config: Config, metadata: dict[str, Any], fallback_title: str
) -> tuple[str, str] | None:
title = metadata.get("title")
sub_url = metadata.get("sub_url")
if not isinstance(title, str) or not title:
title = fallback_title
if not isinstance(sub_url, str) or not sub_url:
return None
url = (
f"{config.gitea_url}/{quote(config.owner, safe='')}/"
f"{quote(config.repository, safe='')}/wiki/{quote(sub_url, safe='%')}"
)
return title, url
def _local_revision(
path: Path, *, expected_title: str, expected_url: str
) -> str | None:
if not path.is_file():
return None
try:
metadata, _body = parse_mirror(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, WikiDocsError):
return None
if (
metadata.get("wiki_page") != expected_title
or metadata.get("wiki_url") != expected_url
or not metadata.get("synchronized_at")
):
return None
revision = metadata.get("wiki_revision")
return revision if revision else None
def sync_all(
config: Config,
client: WikiClient,
*,
check: bool = False,
deep_check: bool = False,
) -> list[str]:
"""检查或写入所有显式映射;写入前先完成全部远端读取。"""
if not check:
dirty = dirty_mirror_paths(config)
@@ -392,10 +482,43 @@ def sync_all(config: Config, client: WikiClient, *, check: bool = False) -> list
"已映射的本地镜像存在未提交改动,已停止以防覆盖:\n" + details
)
messages: list[str] = []
pages = client.list_pages()
resolved: list[tuple[Mapping, Path, WikiPage | None, str | None]] = []
for mapping in config.mappings:
page = client.get_page(mapping.page)
metadata = _find_page_metadata(pages, mapping.page)
target = ROOT / PurePosixPath(mapping.path)
remote_revision = _metadata_revision(metadata)
identity = _metadata_identity(config, metadata, mapping.page)
local_revision = (
_local_revision(
target, expected_title=identity[0], expected_url=identity[1]
)
if identity is not None
else None
)
if not deep_check and remote_revision and local_revision == remote_revision:
action = "一致" if check else "无变化"
resolved.append(
(
mapping,
target,
None,
f"{action}:{mapping.path} <- "
f"{mapping.page}@{remote_revision[:12]}",
)
)
continue
page = client.get_page_from_metadata(metadata, mapping.page)
resolved.append((mapping, target, page, None))
messages: list[str] = []
for mapping, target, page, skip_message in resolved:
if skip_message is not None:
messages.append(skip_message)
continue
if page is None: # pragma: no cover - resolved 元组由上面的单一路径构造
raise WikiDocsError(f"Wiki 页面未解析:{mapping.page}")
if check:
errors = check_mirror(mapping, page, target)
if errors:
+8 -4
View File
@@ -2,8 +2,8 @@
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Project-Profile
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Project-Profile.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:29:41Z
wiki_revision: 7468b9fbdd4d0bbbb9a73580c22ec868b3085753
synchronized_at: 2026-09-05T04:06:47Z
<!-- gitea-wiki-mirror:end -->
# 项目档案
@@ -21,16 +21,20 @@ synchronized_at: 2026-09-04T11:29:41Z
| 后续设计范围 | 从 SYB 创建采购任务、创建待付款订单、物流采集与自动回填 |
| 预计规模 | 20 台 Android;每天约 100 个采集任务、200 个采购任务 |
## 项目治理模式
GoAuto 默认采用轻量治理:文案、注释、格式、局部样式或布局、预期行为明确的小 Bug,以及不改变接口、数据结构、权限和安全边界的单模块低风险调整可以直接实施,无需为了留痕补建工单。完整独立需求、新页面、跨模块功能,以及涉及 API、数据结构、权限、安全、迁移或范围不明确的变化必须建立单元工单。采购、创建订单、真实个人或生产数据、权限、安全、并发、迁移、删除、发布和不可逆操作始终升级为高风险,必须具备对象和范围明确的人工授权;已有有效授权时不机械重复确认,范围或环境发生实质变化时重新确认。永久禁止付款以及设备、隐私、AI 和真机安全红线不可裁剪。
## 建设基线
| 基线 | 来源与版本 | 许可证 / 使用方式 | GoAuto 适配 |
|---|---|---|---|
| DevHarness | `D:\OPC\dev_harness`,目标提交 `4bbacf4d7fb265984396bb5589c544105043fa0b` | 开发流程与文档模板 | 2026-08-27 升级(#114):在 #76 已采用的单人工单事实源基础上,增量加入部署模板、Gitea MCP 与最小工单读取、线上原型默认审核与按需 HTML 导出、PowerShell UTF-8/ExecutionPolicy 边界;继续保留 GoAuto 专用安全与设计门禁 |
| DevHarness | `D:\OPC\dev_harness`,目标提交 `ecab899` | 开发流程与文档模板 | 2026-09-05 升级(#221):在既有单人工单事实源基础上,选择性加入轻量治理、明确授权边界、中文默认沟通、Windows PowerShell 安全规则,以及按 revision 增量同步与显式深度检查;继续保留 GoAuto 专用高风险与安全门禁 |
| 服务端 | `go-admin` v2.3.0 | 上游开源管理端基线;升级时复核许可证和安全公告 | 保留认证、菜单、配置和管理端基础能力,新增 GoAuto 业务模块 |
| 管理端 | `go-admin-ui` v3.0.0,`web/package.json` 标注 MIT | Vue 管理界面基线 | 保留应用外壳与通用组件,新增 GoAuto 页面 |
| Android | 原生 Kotlin Agent | 自研业务客户端 | 通过管理员明确配置的 HTTP 或 HTTPS Origin 直连服务端,不保留 Windows 桌面 Client/ADB 作为生产拓扑 |
升级必须比较当前记录的目标提交与新的明确提交,不能笼统复制“最新版”。本次上一基线为 `bfdf648962d11a8024f62768380d8571e1f45f68`,目标为 `4bbacf4d7fb265984396bb5589c544105043fa0b`,区间共 13 个提交;更早基线 `b1f500128d6eb100985792d4a715db8b6b5ae203` 的适配见 #47。模板内容一律按 GoAuto 事实改写:不复制 DevHarness 的项目事实、任务记录、占位部署参数或历史归档;`harness.py` 继续校验 GoAuto 实际核心页面与产品 README,GoAuto 更严格的付款、订单、设备、数据和真机门禁继续优先。
升级必须比较当前记录的目标提交与新的明确提交,不能笼统复制“最新版”。本次上一基线为 `4bbacf4d7fb265984396bb5589c544105043fa0b`,目标为 `ecab899`,选择性适配其后的 7 个提交;更早基线适配见 #47 与 #114。模板内容一律按 GoAuto 事实改写:不复制 DevHarness 的项目事实、任务记录、占位部署参数或历史归档;`harness.py` 继续校验 GoAuto 实际核心页面与产品 README,GoAuto 更严格的付款、订单、设备、数据和真机门禁继续优先。
## 交付单元
+22 -3
View File
@@ -2,12 +2,18 @@
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Development-Workflow
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Development-Workflow.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:29:46Z
wiki_revision: 62ddbe4469740c02ce4a6ca2fd1966a89a79322f
synchronized_at: 2026-09-05T04:06:53Z
<!-- gitea-wiki-mirror:end -->
# 开发工作流
## 语言与术语
- 用户可以使用中文、英文或合理的中英混合语言交流;默认使用中文分析、回复、编写工单和维护内部项目文档。
- 代码标识符、命令、参数、路径、文件名、API 名称、协议名、日志和错误原文保持原样;必要时补充简短中文解释。
- 用户明确要求某次回复或交付物使用其他语言时,按该次要求执行,不改写接口契约或影响搜索和执行的原文。
## 事实来源
| 信息 | 唯一事实来源 |
@@ -18,7 +24,7 @@ synchronized_at: 2026-09-04T11:29:46Z
| 源码、迁移、测试、版本绑定分析、本地原型和核心 Wiki 镜像 | Git |
| 可编辑交互设计 | QuantUX;App ID、版本、链接和确认状态记录在工单 |
核心页面通过 `wiki-docs.json` 显式映射,固定执行 Wiki → `docs/` 单向同步。既有 Wiki 任务归档和 `docs/task/` 只作历史兼容;标准任务不创建或导出,只有用户明确要求专项快照时才使用 `archive` / `export`。
核心页面通过 `wiki-docs.json` 显式映射,固定执行 Wiki → `docs/` 单向同步。日常 `sync` 与 `sync --check` 先比较页面 revision,revision 未变化时不重复下载正文;疑似镜像损坏或需要完整核对时显式使用 `sync --deep-check`。既有 Wiki 任务归档和 `docs/task/` 只作历史兼容;标准任务不创建或导出,只有用户明确要求专项快照时才使用 `archive` / `export`。
## 权威源与事实边界
@@ -53,6 +59,19 @@ synchronized_at: 2026-09-04T11:29:46Z
- 每个核心页面写入后必须在线回读并取得 revision。页面缺失、回读失败或没有 revision 时停止初始化。
- 产品编码前运行 `python dev_scripts/harness.py sync --verify`;全部成功才表示初始化完成。
## 项目治理模式与不可裁剪底线
GoAuto 默认采用轻量治理。文案、注释、格式、局部样式或布局、预期行为明确的小 Bug,以及不改变接口、数据结构、权限和安全边界的单模块低风险调整可以直接实施,无需为了留痕补建工单。完整独立需求、新页面、跨模块功能,以及 API、数据结构、权限、安全、迁移或范围不明确的变化必须建单。
采购、创建订单、真实个人或生产数据、权限、安全、并发、数据库迁移、删除数据、发布和其他不可逆操作始终升级为高风险。无论任务采用何种最小门禁,凭据保护、永久禁止付款、个人与生产数据最小化、设备互斥、Agent 禁止 OCR/VLM、控件树与整屏截图禁存、工作区保护、真实测试和人工验收均不可裁剪。
### 明确授权后的执行
- 当前聊天中用户给出的明确指令,或 Gitea 工单中能够归属于有权人工的明确授权,可以作为执行依据,不要求把同一授权重复复制到工单后再次确认。
- 授权必须能识别操作、对象和范围;Agent 自动生成的工单、草稿、摘要或对用户意图的转述不能单独构成人工授权。
- 获得有效授权后,只核对准确目标、授权范围和当前状态等最小必要前提,不得仅因操作不可逆而重复询问或拒绝。
- 授权不自动覆盖相邻对象或后续任务;环境、对象、范围或影响发生实质变化时重新确认。平台自身强制的审批、安全策略或权限限制继续有效。
## 工单与设计证据双门禁
正式实施前先判断是否需要工单,再判断需要什么设计证据。工单不能替代原型确认,原型也不能替代技术方案、安全检查和单元工单。
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Architecture-and-Code-Map
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Architecture-and-Code-Map.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:29:50Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 架构与代码地图
+12 -14
View File
@@ -2,8 +2,8 @@
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Business-Rules-and-Glossary
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Business-Rules-and-Glossary.-
wiki_revision: 069248c1466d54273900a4c87d1285436f7d50f9
synchronized_at: 2026-09-04T10:57:41Z
wiki_revision: f8a2e64acaf245066d55d4a0f1e928d672712270
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 业务规则与术语
@@ -144,7 +144,7 @@ synchronized_at: 2026-09-04T10:57:41Z
- SYB 商品页的 PDD 采集资格独立于采购处理阶段:只要已关联的 PDD 商品未停用、没有 `pending` / `running` 采集任务且存在可用采集规则,即可创建采集任务;因此已完成采集并进入“可创建采购”的商品也可以重新采集。相同 PDD 商品在批量创建前按商品去重,服务端创建时仍按最新状态复核。
- 批量创建读取管理员选定的当前采购规则并重新执行契约校验,每条任务保存不可变快照;当前规则缺失或无效时明确阻断,不回退到代码常量。设备默认人工指定,也可以留空由符合能力的空闲设备领取。
- 批量创建的价格保护来自 PDD 商品档案而不是 SYB/Shopee 的 TWD 售价:当前采购规则可用 `priceGuard.minRatio`(0.1~1.0)和 `maxRatio`(1.0~3.0)配置比例,缺省仍为 0.2 / 1.5;最低价向下取整、最高价向上取整到人民币分。已确认颜色映射时按该颜色计算,待规格探测时以全部可用颜色的最低/最高价计算,参考价取最高价;没有可用颜色价格时不能创建。
- 地址后缀为 `_cg{purchase_task.id}`。选好规格和数量后,Agent 仍停留在当前规格/下单面板;若收货地址被面板裁切,只允许在唯一、可见且占据主要宽度的纵向滚动容器内有限向下拉动来显示地址,禁止按页面最大滚动区域盲目滑动。Agent 只在唯一地址入口、唯一修改按钮和唯一详细地址输入框均成立时修改;脱敏手机号文字本身不可点击时,仍以该唯一文字节点的中心坐标执行一次精确手势点击,不沿用可能覆盖整个下单面板的可点击祖先。“修改”“保存”和“提交订单”等文字节点即使依赖可点击父节点,也必须保留原始文字节点作为每次重新定位的锚点,父节点只用于验证存在可点击路径。近乎完全重叠的无障碍重复节点按一个目标处理,仍有多个独立目标或页面切换超时则明确失败。按首个 `-` 或 `_` 截取地址主体后追加当前后缀,保存后必须回读完整新地址。修改或回读失败时禁止创建订单,地址全文和控件树不落库。
- 地址后缀为 `_cg{purchase_task.id}`。精确规格、数量、价格和摘要核验通过后,若仍处于已识别的规格面板,Agent 只允许点击唯一、可见、启用、可点击且文案精确命中“确定/确认”别名的规格确认控件;不得点击提交订单、确认购买、付款、支付、地址修改或地址保存控件,也不使用中心手势兜底。点击后必须重新读取并确认已出现订单确认页或地址入口强证据,否则明确失败。进入订单确认页后,若收货地址被面板裁切,只允许在唯一、可见且占据主要宽度的纵向滚动容器内有限向下拉动来显示地址,禁止按页面最大滚动区域盲目滑动。Agent 只在唯一地址入口、唯一修改按钮和唯一详细地址输入框均成立时修改;脱敏手机号文字本身不可点击时,仍以该唯一文字节点的中心坐标执行一次精确手势点击,不沿用可能覆盖整个下单面板的可点击祖先。“修改”“保存”和“提交订单”等文字节点即使依赖可点击父节点,也必须保留原始文字节点作为每次重新定位的锚点,父节点只用于验证存在可点击路径。近乎完全重叠的无障碍重复节点按一个目标处理,仍有多个独立目标或页面切换超时则明确失败。按首个 `-` 或 `_` 截取地址主体后追加当前后缀,保存后必须回读完整新地址。修改或回读失败时禁止创建订单,地址全文和控件树不落库。
- 地址编辑页可以同时存在收货人、手机号和详细地址等多个输入框;Agent 只选择与“详细地址”标签纵向重叠且位于其右侧的唯一输入框,不能用页面输入框总数或顺序猜测。
- 点击创建订单前,Agent 必须先在本地事务保存 `order_submit_started`、不可逆时间、稳定请求 ID 和不含地址全文的最终确认快照,再用同一请求 ID通知服务端;两侧成功后才允许精确点击唯一创建订单按钮一次。
- 进入不可逆边界后,进程重启、断网、点击结果不明或无法取得唯一订单号/下单时间时只允许只读核单并进入 `order_result_unknown`,禁止再次点击;任务与订单正式关联仍以完整 PDD 订单号为准。
@@ -256,16 +256,14 @@ synchronized_at: 2026-09-04T10:57:41Z
## Agent 受控重试采购
- 当前设备只可就地重跑自身最近 30 天内、服务端标记 `retryable=true` 的正式采购失败任务;列表和详情都只能发起单任务重试,不支持多选、批量或自动重试。
- 普通“重试采购”复用原 `purchase_task.id`,不新建任务,不改变商品、虾皮/SYB 身份、目标规格、已映射规格、价格保护、地址后缀与其他业务快照。重试只把任务恢复为 `pending`,清理错误、租约和运行守卫。
- 重试时刷新服务端当前采购规则及其类型、schema 和能力要求。#127 尚未实施前,当前规则仍为服务端内置规则;#127 完成后才切换为数据库单例设置,不能在本工单提前引入第二事实源。
- 新规则下发前必须重新校验原设备在线、空闲且具备全部能力。规则不可用或能力不匹配时拒绝重试,原任务保持失败状态。
- 已出现 `order_submit_started` 证据,或存在不可逆时间、订单提交请求、PDD 订单号、下单时间的任务一律拒绝就地重试,并提示走既有“授权重新采购”流程,防止重复下单。
- 每次成功受理就地重试都会为同一任务预建一个新的 `pending` attempt,保存本次规则哈希;正常 Start 复用并转为 `running`。因此尝试次数、每次规则哈希和失败原因均可追溯,且不限制人工重试次数。
- `requestId` 按“任务 + 请求”幂等;相同请求重放不重复修改任务或增加 attempt。
- Agent 详情显示已尝试次数与上次失败原因。确认界面必须说明同一任务使用最新规则重跑、可能创建待付款订单且系统不会支付。
- “继续采购”与普通“重试采购”语义不同:替代商品匹配完成后的“继续采购”继续调用既有 `AgentRetry → BatchRetry → Create`,保留旧任务并按当前替代商品档案创建新任务;Admin 批量重试同样继续创建新任务。二者均不得改成就地更新旧任务。
- 取消订单、修改既有订单和支付仍禁止;真机重跑可能进入创建待付款订单流程,执行前必须再次取得人工授权。
- 当前设备只可重试自身最近 30 天内、服务端标记 `retryable=true` 的正式采购失败任务;列表和详情都只能发起单任务重试,不支持多选、批量或自动重试。
- 普通“重试采购”调用既有 `AgentRetry → BatchRetry → Create`:原失败任务及其商品、目标规格、执行规格、价格和执行记录保持不可变;服务端根据当前 SYB、虾皮/PDD 档案、当前采购规则和当前设备创建不同 `purchase_task.id` 的新任务。
- 新 SYB 采购任务继续遵循 #215 的强制当次规格探测,首趟不得直接使用历史任务的规格决策;当前档案、规则、价格、设备或能力门禁不通过时拒绝创建,旧任务保持失败状态。
- 已出现 `order_submit_started` 证据,或存在不可逆时间、订单提交请求、PDD 订单号、下单时间的任务一律拒绝重试,并提示走既有“授权重新采购”流程,防止重复下单。
- `requestId` 按“来源任务 + 请求”幂等;相同请求重放返回同一新任务,不重复创建。
- Android 只在服务端 `retryable=true` 且状态为 `failed` 时显示普通“重试采购”。确认和成功反馈必须说明旧任务保留、新任务读取当前档案和规则、可能创建待付款订单且系统不会支付。
- 历史兼容的就地 `/reset` 服务端入口不得把“目标规格存在但对应执行规格为空”的任务恢复到正式采购阶段;此类异常快照必须拒绝,并提示创建新任务。Android 普通重试不再调用该入口。
- 替代商品匹配完成后的“继续采购”和 Admin 批量重试继续使用同一新任务语义;取消订单、修改既有订单和支付仍禁止。真机重试可能进入创建待付款订单流程,执行前必须再次取得人工授权。
## Agent 状态页手动检查任务
@@ -343,7 +341,7 @@ synchronized_at: 2026-09-04T10:57:41Z
## SYB 采购强制当次规格探测(#215)
- 每个新 SYB 采购任务固定执行“首趟只读探测 → 服务端确定性优先/必要时 AI → 固化任务级精确规格 → 第二趟正式采购”。已有长期映射只作商品档案事实,不直接进入任务执行规格。
- 每个新 SYB 采购任务固定执行“首趟只读探测 → 服务端确定性优先/必要时 AI → 固化任务级精确规格 → 第二趟正式采购”。首趟只打开一次浏览器商品链接;匹配期间当前设备保留给同一任务,不领取其他采购或采集任务;第二趟复用 PDD 当前页,不再次打开链接,也不严格核验标题、goodsId 或页面指纹,但仍要求 PDD 包名与商品/规格/订单页面结构安全证据。已有长期映射只作商品档案事实,不直接进入任务执行规格。
- 首趟候选与 `taskId`、`taskAttemptId`、`deviceId`、规则快照哈希和幂等结果哈希关联;第二趟失败不得回到首趟循环探测。备货 `stock/direct_select` 没有 SYB 目标规格,继续使用用户逐字选择的档案规格,不进入本规则。
- 候选和 Provider 结果仅保存颜色、尺码原始标签及结构化决策,不保存控件树、整屏截图、账号、地址、订单或支付数据;付款仍永久禁止。
+28 -2
View File
@@ -2,8 +2,8 @@
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Local-Development-and-Verification
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Local-Development-and-Verification.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:02Z
wiki_revision: 835494c4a63a1494601658561fde1ab76657be3d
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 本地开发与验证
@@ -24,6 +24,32 @@ T01 已建立可执行的三端骨架。建议从仓库根目录运行统一脚
- 不得仅为设置编码重复启动一层 PowerShell;嵌套进程会增加启动时间、转义复杂度和错误定位成本。
- 代码发现优先使用项目配置的代码图工具;检索字符串、配置和非代码文件,或图工具不足时使用 `rg`。
### PowerShell 语法与外部命令
- Windows 命令不得默认套用 Bash 语法;复杂正则优先先赋给变量或使用 `rg -e`,避免在多层引号中继续嵌套。
- 多行 Python 或 JSON 正文使用单引号 PowerShell here-string,避免 `$()`、反引号和变量被 PowerShell 提前展开:
```powershell
$script = @'
print("保持原文")
'@
$script | python -
```
- `foreach`、`if` 等语句块应保留在同一个 PowerShell 解析上下文中;需要收集表达式结果时使用数组表达式:
```powershell
$items = @(foreach ($path in $paths) {
if (Test-Path -LiteralPath $path) { Get-Item -LiteralPath $path }
})
```
- `rg` 使用真实目录配合 `-g/--glob`,不要把 Bash 风格通配路径作为目录参数:
```powershell
rg -n -g '*.md' 'sync --check' docs
```
### 文件编码与控制台输出
文件解码和控制台输出是两个边界。读取 UTF-8 文本时,在命令支持的情况下显式指定字面路径与编码:
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Common-Changes
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Common-Changes.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:12Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 常见修改指南
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Troubleshooting
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Troubleshooting
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:16Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 故障排查
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Product-Requirements-Overview
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Product-Requirements-Overview.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:20Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 产品需求总览与当前 MVP
+25 -27
View File
@@ -2,8 +2,8 @@
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Android-Agent-API-Contract
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Android-Agent-API-Contract.-
wiki_revision: a022259ba39db7588eaa3b6579c799e2eb6a6f9e
synchronized_at: 2026-09-04T10:58:12Z
wiki_revision: 08a65eb49d3d1091f0be30e6f6d1a9372f6e5b37
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# MVP 共享 API 契约
@@ -415,7 +415,7 @@ POST /api/agent/v1/tasks/{taskId}/fail
| 状态 | 含义 | 是否占用 SYB 活动槽 |
|---|---|---|
| `pending` | 待执行 | 是 |
| `spec_probe_pending` | 第一趟探测结束,待服务端固化规格并重新派发 | 是 |
| `spec_probe_pending` | 第一趟探测结束,服务端规格匹配中;当前设备保留同任务连续流程且不得领取其他任务 | 是 |
| `running` | Agent 执行中 | 是 |
| `rehearsal_completed` | 演练安全结束,未改地址、未创建订单 | 否 |
| `order_submit_started` | 不可逆标记已落库,只能核单,禁止再次点击 | 是 |
@@ -529,17 +529,17 @@ Admin 列表与详情由 #35 实现;#67 增加 `shopeeOrderNoSnapshot` 的列
| 方法 | 路径 | 说明 |
|---|---|---|
| `GET` | `/api/agent/v1/purchase-tasks/next` | 返回与设备能力兼容的指定任务或空闲任务 |
| `GET` | `/api/agent/v1/purchase-tasks/next` | 优先返回当前设备的运行任务;存在 `spec_probe_pending` 时返回同一等待任务以阻止其他任务插队,否则返回能力兼容的指定任务或空闲任务 |
| `POST` | `/api/agent/v1/purchase-tasks/{taskId}/claim` | `requestId` 原子领取,并建立设备/可选账号租约 |
| `POST` | `/api/agent/v1/purchase-tasks/{taskId}/start` | 创建不可变 `taskAttemptId` |
| `POST` | `/api/agent/v1/purchase-tasks/{taskId}/order-submit-started` | 创建订单前先落不可逆标记;演练任务和 `spec_probe` attempt 永远拒绝 |
| `POST` | `/api/agent/v1/purchase-tasks/{taskId}/result` | 请求体携带 `taskAttemptId` 和 `requestId`;幂等提交演练、规格探测、订单或失败结果 |
自 #215 起,新建 SYB 采购任务不再从 PDD 档案创建持久匹配工作项,也不在首次派发前调用外部 AI;部署前已存在的 `purchase_spec_match_work_item` 继续按原状态兼容处理。新任务首次 `start` 固定得到 `phase=spec_probe`,Android 通过既有结果字段回传当次候选;匹配成功后的第二次 `start` 才得到 `phase=purchase` 和服务端固化的精确 PDD 原始标签。Android 不接收 AI 配置或自由决策权限。
自 #215 起,新建 SYB 采购任务不再从 PDD 档案创建持久匹配工作项,也不在首次派发前调用外部 AI;部署前已存在的 `purchase_spec_match_work_item` 继续按原状态兼容处理。新任务首次 `start` 固定得到 `phase=spec_probe`,Android 通过浏览器打开任务链接一次并经既有结果字段回传当次候选;匹配成功后的第二次 `start` 才得到 `phase=purchase` 和服务端固化的精确 PDD 原始标签。第二阶段直接复用首趟保留的 PDD 页面,不再次打开浏览器链接,也不以标题、goodsId 或页面指纹做严格同页校验;仍必须通过 PDD 包名和商品/规格/订单页面结构安全证据。Android 不接收 AI 配置或自由决策权限。
结果提交至少关联 `taskId`、`taskAttemptId`、`deviceId`、规则快照哈希和结构化结果。相同 attempt 的相同结果重复提交返回同一事实;不同内容拒绝覆盖。每个新 SYB 采购任务的第一趟只读遍历当次 PDD 规格面板并提交颜色、尺码原始候选,随后释放设备与已知账号租约并进入 `spec_probe_pending`;服务端只以任务冻结的 SYB 目标和当次候选先做繁简、空白/全半角/大小写及公斤/斤的唯一确定性匹配,仍无唯一结果才调用 AI。AI 的颜色和尺码必须逐字属于当次对应候选,否则按无匹配失败。第二趟只会收到服务端固化的精确 PDD 原始标签;Agent 只在已打开的规格面板内做有限纵向滑动,每次重新读取节点并按完整规范化文字精确点击,连续没有新证据或达到上限即停止。尺码的任务目标与页面值在选择边界使用同一安全尾价规范化;不改写任务快照,规范化为空、仍含货币符号或多个原始候选折叠为同一值时安全失败。
结果提交至少关联 `taskId`、`taskAttemptId`、`deviceId`、规则快照哈希和结构化结果。相同 attempt 的相同结果重复提交返回同一事实;不同内容拒绝覆盖。每个新 SYB 采购任务的第一趟只读遍历当次 PDD 规格面板并提交颜色、尺码原始候选,随后释放数据库租约和已知账号运行守卫并进入 `spec_probe_pending`,但服务端调度与 Agent 必须把当前设备保留给同一采购流程:`next` 返回该等待任务,Agent 只轮询等待,不领取其他采购或采集任务。服务端只以任务冻结的 SYB 目标和当次候选先做繁简、空白/全半角/大小写及公斤/斤的唯一确定性匹配,仍无唯一结果才调用 AI。AI 的颜色和尺码必须逐字属于当次对应候选,否则按无匹配失败。第二趟只会收到服务端固化的精确 PDD 原始标签;Agent 复用首趟仍打开的页面,只在已打开的规格面板内做有限纵向滑动,每次重新读取节点并按完整规范化文字精确点击,连续没有新证据或达到上限即停止。尺码的任务目标与页面值在选择边界使用同一安全尾价规范化;不改写任务快照,规范化为空、仍含货币符号或多个原始候选折叠为同一值时安全失败。
任务 payload 的必传布尔字段 `specResolutionAllowed` 是 Android 是否可以提交规格探测的唯一资格事实。新建 `taskType=syb_order` 任务必须由声明 `purchase.spec-probe.v1` 的规则创建,初始 `SpecDecisionRequestID` 为空且 `specSource=unresolved`,首趟返回 `true`;当次决策固化后返回 `false`。`stock`、`direct_select`、已固化规格决策、能力缺失及其他组合均返回 `false`。普通就地重试保留 `SpecDecisionRequestID`、目标规格、映射规格和规格决策快照,不能恢复探测资格;第二趟选择失败必须提交真实失败,不能重新探测。Android 不得根据映射是否非空、错误文字或本地判断扩大资格。
任务 payload 的必传布尔字段 `specResolutionAllowed` 是 Android 是否可以提交规格探测的唯一资格事实。新建 `taskType=syb_order` 任务必须由声明 `purchase.spec-probe.v1` 的规则创建,初始 `SpecDecisionRequestID` 为空且 `specSource=unresolved`,首趟返回 `true`;当次决策固化后返回 `false`。`stock`、`direct_select`、已固化规格决策、能力缺失及其他组合均返回 `false`。历史兼容的就地 `/reset` 保留 `SpecDecisionRequestID`、目标规格、映射规格和规格决策快照,不能恢复探测资格;映射不完整时必须拒绝,不能进入正式采购阶段。普通 Agent 重试创建新任务并重新取得一次探测资格。Android 不得根据映射是否非空、错误文字或本地判断扩大资格。
Android 规格失败使用五个稳定阶段:`PURCHASE_SPEC_TARGET_NOT_VISIBLE`、`PURCHASE_SPEC_TARGET_AMBIGUOUS`、`PURCHASE_SPEC_SAFE_TARGET_MISSING`、`PURCHASE_SPEC_CLICK_FAILED` 和 `PURCHASE_SPEC_SELECTION_UNCONFIRMED`。`PURCHASE_SPEC_CLICK_FAILED` 的 `errorMessage` 只允许稳定子原因 `root_unavailable`、`target_stale`、`no_clickable_ancestor`、`action_click_false` 或 `unknown`;其他阶段的消息不得包含规格原文、坐标、控件树或截图。只有 `PURCHASE_SPEC_TARGET_NOT_VISIBLE && specResolutionAllowed=true` 可以提交规格探测,其他四态直接提交真实失败,服务端原样保留稳定阶段/子原因。旧 Agent 在资格已用尽后再次提交 `spec_probe_completed` 时,服务端以 `PURCHASE_SPEC_REPROBE_REJECTED` fail-closed,释放租约并保留第一次规格决策,不再冒充新的选择根因或再次派发。无匹配、候选不完整、歧义或 Provider 异常同样使任务失败。`order_result_unknown` 只允许管理员或采购员人工解除,永不自动重派。
@@ -617,12 +617,12 @@ Content-Type: application/json
- 响应返回 `taskId`、`attemptNumber`、`status` 和可选的 `replayed`,不返回规则快照、URL、Token、控件树或截图。
- 设备离线、任务非终态、设备忙、规则不可用或同商品存在活动任务时返回明确冲突,不支持离线排队。
## Agent 受控采购重试(#95、#157)
## Agent 受控采购重试(#95、#157、#217)
普通失败任务的“重试采购”改为就地重跑:
普通失败任务的“重试采购”和替代商品匹配完成后的“继续采购”统一调用新任务接口:
```http
POST /api/agent/v1/purchase-tasks/{taskId}/reset
POST /api/agent/v1/purchase-tasks/{taskId}/retry
Authorization: Bearer <device-token>
Content-Type: application/json
@@ -634,37 +634,35 @@ Content-Type: application/json
```json
{
"data": {
"taskId": 12,
"taskNo": "CG-12",
"attemptNumber": 2,
"status": "pending",
"sourceTaskId": 12,
"sourceTaskNo": "CG-12",
"taskId": 13,
"taskNo": "CG-13",
"replayed": false
}
}
```
- 原任务必须属于当前 Device Token、在最近 30 天内、状态为 `failed`,且是分配给该设备的正式 SYB 采购任务;跨设备或超期按任务不存在处理。
- 任务不得存在 `irreversibleAt`、`orderSubmitRequestId`、PDD 订单号或下单时间。已有任何不可逆证据时返回 `PURCHASE_RETRY_UNSAFE`,提示走“授权重新采购”,不得恢复为待执行。
- 服务端在同一事务锁定任务和设备,确认同一 SYB 商品没有更新任务、设备在线且空闲,然后读取当前服务端采购规则,重新校验 schema、动作安全边界与设备能力。
- 成功时复用原 `purchase_task.id`,只刷新 `ruleSnapshot`、`ruleType`、`ruleSchemaVersion`、`requiredCapabilities`,清除错误、租约和运行守卫并恢复 `pending`。商品、Target、Mapped、价格保护、数量、地址后缀及其他业务快照逐字段保持不变。
- 每次受理创建该任务的新 `pending` attempt 并记录规则哈希;Start 复用该 attempt 转为 `running`,不会重复创建执行记录。相同 `requestId` 重放返回相同 attempt 且 `replayed=true`;不同请求可在任务再次失败后继续重试,不限制次数。
- 采购详情新增 `attemptCount`、可选 `lastFailureCode` 和 `lastFailureMessage`,供 Agent 显示已尝试次数与上次失败原因;不返回规则快照、Token、地址、控件树或截图。
- Android 仍只在服务端 `retryable=true` 且状态为 `failed` 时显示普通“重试采购”。确认文案必须说明任务号不变、使用最新规则重跑、可能产生待付款订单且系统不会支付。
- 规则无效、设备离线/忙、能力不匹配、任务状态变化或同一 SYB 商品已有更新任务时,服务端明确拒绝且不得部分修改任务。
- 来源任务必须属于当前 Device Token、在最近 30 天内、状态为 `failed`,且是分配给该设备的正式 SYB 采购任务;跨设备或超期按任务不存在处理。
- 来源任务不得存在 `irreversibleAt`、`orderSubmitRequestId`、PDD 订单号或下单时间。已有任何不可逆证据时返回 `PURCHASE_RETRY_UNSAFE`,提示走“授权重新采购”,不得创建新任务。
- `AgentRetry → BatchRetry → Create` 保留来源失败任务并创建不同 `purchase_task.id` 的新任务;新任务重新读取当前 SYB、虾皮/PDD 档案、当前采购规则、价格保护和设备能力,重新生成地址后缀,不继承来源任务的旧规格决策。
- 新 SYB 任务按 #215 固定从 `spec_probe` 开始。相同 `requestId` 重放返回同一新任务且 `replayed=true`;不同 requestId 再次请求受同一 SYB 商品最新任务和设备并发门禁约束。
- Android 只在服务端 `retryable=true` 且状态为 `failed` 时显示普通“重试采购”;确认文案和成功反馈必须说明原任务保留、新任务使用当前档案与规则、可能产生待付款订单且系统不会支付。
- 规则无效、当前档案或价格不合格、设备离线/忙、能力不匹配、任务状态变化或同一 SYB 商品已有更新任务时,服务端明确拒绝且不得部分创建。
- Admin 批量重试继续使用相同的新任务语义;替代商品“继续采购”仍在 AgentRetry 前额外验证替换分项与继续采购资格。
既有新建任务接口保留原语义:
历史兼容的就地重置接口仍保留,但 Android 普通重试不再调用:
```http
POST /api/agent/v1/purchase-tasks/{taskId}/retry
POST /api/agent/v1/purchase-tasks/{taskId}/reset
Authorization: Bearer <device-token>
Content-Type: application/json
{"requestId":"<uuid>"}
```
- `/retry` 的 `AgentRetry → BatchRetry → Create` 行为不变:保留来源失败任务并创建新任务,返回 `sourceTaskId`、`sourceTaskNo`、新 `taskId`、新 `taskNo` 和 `replayed`。
- Android 普通“重试采购”不再调用 `/retry`;只有 #132 替代商品规格匹配完成后的“继续采购”继续调用它。Admin 批量重试行为也不变。
- “继续采购”会重新解析替代商品、规格映射与价格并生成新地址后缀;这与普通失败任务保持快照的就地重跑不可互换。
- `/reset` 只允许安全失败、无不可逆证据且不存在更新任务的原任务;它保留原业务快照并刷新当前规则。
- 目标颜色存在但映射颜色为空,或目标尺码存在但映射尺码为空时,必须返回 `PURCHASE_SPEC_MAPPING_REQUIRED`,不得创建 `purchase` attempt 或下发正式采购 payload。
- 两个入口都不执行支付。真机调用可能创建待付款订单,必须先取得人工授权。
## Agent 任务记录范围与同步(#99)
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Delivery-Issues
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Delivery-Issues.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:28Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 当前 MVP 交付工单索引
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: OnePlus-Real-Device-Acceptance
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/OnePlus-Real-Device-Acceptance.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:32Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 一加真机验收记录
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: PDD-Detail-Rule-Migration-Analysis
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/PDD-Detail-Rule-Migration-Analysis.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:37Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# PDD 商品详情采集规则迁移分析
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: SYB-ERP-Interface-Contract
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/SYB-ERP-Interface-Contract.-
wiki_revision: ad9adc22e69e48c9a8fd87d7121066eecca55fd6
synchronized_at: 2026-09-04T11:30:50Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 12 顺云宝(SYB)ERP 接口契约
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Deployment-and-Operations
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Deployment-and-Operations.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:08Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 部署与运维
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Home
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Home
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:29:34Z
synchronized_at: 2026-09-05T04:06:45Z
<!-- gitea-wiki-mirror:end -->
# GoAuto 文档中心
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Deployment-Template
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Deployment-Template.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:41Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
# 部署文档模板
+1 -1
View File
@@ -3,7 +3,7 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
wiki_page: Task-Archive-Template
wiki_url: https://git.ilapage.cn/OPC/goauto/wiki/Task-Archive-Template.-
wiki_revision: b1b1b343917e66288f4282bc6b3b90ea4ff3cca0
synchronized_at: 2026-09-04T11:30:46Z
synchronized_at: 2026-09-05T04:13:06Z
<!-- gitea-wiki-mirror:end -->
> 本模板只用于用户明确要求的专项历史快照或读取既有归档,不属于标准任务闭环。单次任务的唯一事实来源是 Gitea 工单;不要为了完成普通任务创建本页面,也不要自动导出到 `docs/task/`。
+2
View File
@@ -136,6 +136,8 @@ func moduleKeyForAPI(path string) string {
return ModulePDDProducts
case strings.HasPrefix(path, "/api/admin/v1/shopee-products"):
return ModuleShopeeProducts
case strings.HasPrefix(path, "/api/admin/v1/shopee-spec-auto-match"):
return ModuleShopeeProducts
case strings.HasPrefix(path, "/api/admin/v1/syb-products/sync-runs"):
return ModuleSYBSyncRuns
case strings.HasPrefix(path, "/api/admin/v1/syb-products"):
+2
View File
@@ -48,6 +48,8 @@ var AdminAPIs = []APIPermission{
{"AI 建议颜色映射", "/api/admin/v1/shopee-products/:productId/specs/mapping/suggest-colors", "POST", true},
{"AI 建议尺码映射", "/api/admin/v1/shopee-products/:productId/specs/mapping/suggest-sizes", "POST", true},
{"一键匹配并确认颜色尺码", "/api/admin/v1/shopee-products/:productId/specs/mapping/auto-match", "POST", true},
{"手动执行虾皮规格自动匹配", "/api/admin/v1/shopee-spec-auto-match/runs", "POST", false},
{"查看最近虾皮规格自动匹配", "/api/admin/v1/shopee-spec-auto-match/runs/latest", "GET", false},
{"查看 SYB 商品", "/api/admin/v1/syb-products", "GET", true},
{"查看 SYB 商品详情", "/api/admin/v1/syb-products/:productId", "GET", true},
+6 -4
View File
@@ -15,10 +15,12 @@ func TestPurchaserPermissionMatrixHasNoDuplicates(t *testing.T) {
func TestPurchaserExcludesAdministratorOperations(t *testing.T) {
denied := map[string]bool{
"POST /api/admin/v1/devices/:deviceId/disable": true,
"POST /api/admin/v1/syb-products/import": true,
"POST /api/admin/v1/collection-rules": true,
"PUT /api/admin/v1/ai-matching-settings": true,
"POST /api/admin/v1/devices/:deviceId/disable": true,
"POST /api/admin/v1/syb-products/import": true,
"POST /api/admin/v1/collection-rules": true,
"PUT /api/admin/v1/ai-matching-settings": true,
"POST /api/admin/v1/shopee-spec-auto-match/runs": true,
"GET /api/admin/v1/shopee-spec-auto-match/runs/latest": true,
}
for _, permission := range PurchaserAPIs() {
if denied[permission.Method+" "+permission.Path] {
+96 -2
View File
@@ -25,12 +25,28 @@ const (
defaultAutoConfirmMinConfidence = 0.9
)
// MaxProviderTimeout is also the total budget used by composite synchronous
// AI operations. This keeps their HTTP response inside the Admin and API
// transport windows even when an operation needs more than one provider call.
const MaxProviderTimeout = 600 * time.Second
// Service owns the internal AI Provider configuration. The API key exception
// is deliberately narrow: it is plain text only in the dedicated settings
// table and is returned only by the administrator settings handler.
type Service struct {
DB *gorm.DB
HTTPClient *http.Client
DB *gorm.DB
HTTPClient *http.Client
ProviderFailureLogger func(ProviderFailureDiagnostic)
}
// ProviderFailureDiagnostic deliberately contains no URL, model, prompt,
// candidates, response body or credential. It is safe for operational logs.
type ProviderFailureDiagnostic struct {
CallID string
Operation string
Kind string
StatusCode int
Duration time.Duration
}
func NewService(db *gorm.DB) *Service {
@@ -179,6 +195,58 @@ func (s *Service) Resolve(ctx context.Context, request MatchRequest) (MatchResul
return result, nil
}
// ResolveSYBSpec parses one SYB productSpec into the linked Shopee product's
// exact color/size labels. Unlike Resolve, this operation does not map to PDD:
// every non-empty answer must be an exact member of the supplied Shopee set.
func (s *Service) ResolveSYBSpec(ctx context.Context, request SYBSpecParseRequest) (SYBSpecParseResult, error) {
request.ProductSpec = strings.TrimSpace(request.ProductSpec)
request.Colors = usableCandidates(request.Colors)
request.Sizes = usableCandidates(request.Sizes)
if request.ProductSpec == "" || (len(request.Colors) == 0 && len(request.Sizes) == 0) {
return SYBSpecParseResult{}, fail(CodeNoMatch, "SYB 采购规格缺少可判断的原文或蝦皮候选")
}
setting, apiKey, err := s.activeSetting(ctx)
if err != nil {
return SYBSpecParseResult{}, err
}
payload := openAIChatRequest{Model: setting.Model, Temperature: 0, Messages: []openAIMessage{
{Role: "system", Content: "你只负责把一条 SYB 商品规格原文解析成给定蝦皮候选中的原始颜色和尺码。不得猜测、不得改写候选、不得返回候选外文本。只返回 JSON:{\"color\":\"颜色候选原文或空\",\"size\":\"尺码候选原文或空\",\"reason\":\"简短原因\",\"confidence\":0到1}。提供了某角色候选时必须唯一可靠地选择一个,否则对应字段留空。"},
{Role: "user", Content: sybSpecParsePrompt(request)},
}}
body, err := json.Marshal(payload)
if err != nil {
return SYBSpecParseResult{}, &Error{Code: CodeProviderUnavailable, Message: "SYB 规格 AI 解析请求生成失败", Cause: err}
}
ctx, cancel := context.WithTimeout(ctx, time.Duration(setting.TimeoutSeconds)*time.Second)
defer cancel()
httpRequest, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint(setting.BaseURL, "chat/completions"), bytes.NewReader(body))
if err != nil {
return SYBSpecParseResult{}, fail(CodeInvalidSetting, "AI 服务地址无效")
}
httpRequest.Header.Set("Authorization", "Bearer "+apiKey)
httpRequest.Header.Set("Content-Type", "application/json")
response, err := s.httpClient().Do(httpRequest)
if err != nil {
return SYBSpecParseResult{}, &Error{Code: CodeProviderUnavailable, Message: "SYB 规格 AI 解析服务暂时不可用", Cause: err}
}
defer response.Body.Close()
responseBody, readErr := io.ReadAll(io.LimitReader(response.Body, 1<<20))
if readErr != nil || response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
return SYBSpecParseResult{}, fail(CodeProviderUnavailable, "SYB 规格 AI 解析服务暂时不可用")
}
choice, err := parseProviderChoice(responseBody)
if err != nil || !validClosedChoice(choice.Color, request.Colors) || !validClosedChoice(choice.Size, request.Sizes) {
return SYBSpecParseResult{}, fail(CodeNoMatch, "AI 未能在蝦皮候选中唯一解析采购规格")
}
if choice.Confidence == nil || *choice.Confidence < 0 || *choice.Confidence > 1 || strings.TrimSpace(choice.Reason) == "" {
return SYBSpecParseResult{}, fail(CodeNoMatch, "AI 解析结果缺少有效置信度或理由")
}
return SYBSpecParseResult{
Color: choice.Color, Size: choice.Size, Provider: ProviderOpenAICompatible,
Model: setting.Model, Reason: safeReason(choice.Reason), Confidence: choice.Confidence,
}, nil
}
func (s *Service) activeSetting(ctx context.Context) (models.AIMatchingSetting, string, error) {
setting, err := s.setting(ctx)
if errors.Is(err, gorm.ErrRecordNotFound) || !setting.Enabled {
@@ -294,6 +362,22 @@ func validChoice(target, selected string, candidates []string) bool {
return false
}
func validClosedChoice(selected string, candidates []string) bool {
selected = strings.TrimSpace(selected)
if len(candidates) == 0 {
return selected == ""
}
if selected == "" {
return false
}
for _, candidate := range candidates {
if candidate == selected {
return true
}
}
return false
}
func safeReason(reason string) string {
reason = strings.TrimSpace(reason)
if reason == "" {
@@ -316,6 +400,16 @@ func matchPrompt(request MatchRequest) string {
return string(raw)
}
func sybSpecParsePrompt(request SYBSpecParseRequest) string {
payload := struct {
ProductSpec string `json:"productSpec"`
Colors []string `json:"shopeeColorCandidates,omitempty"`
Sizes []string `json:"shopeeSizeCandidates,omitempty"`
}{request.ProductSpec, request.Colors, request.Sizes}
raw, _ := json.Marshal(payload)
return string(raw)
}
type openAIMessage struct {
Role string `json:"role"`
Content string `json:"content"`
@@ -2,6 +2,7 @@ package aimatching
import (
"context"
"encoding/json"
"errors"
"io"
"net/http"
@@ -135,3 +136,61 @@ func TestAutoConfirmThresholdDefaultsPersistsAndValidates(t *testing.T) {
t.Fatalf("invalid threshold err=%v", err)
}
}
func TestResolveSYBSpecUsesOnlyProductSpecAndClosedShopeeCandidates(t *testing.T) {
service := matcherTestService(t)
if _, err := service.SaveSettings(context.Background(), SaveSettingsRequest{Enabled: true, BaseURL: "https://provider.example/v1", Model: "test-model", APIKey: "test-secret", TimeoutSeconds: 8}, 7); err != nil {
t.Fatal(err)
}
var sent map[string]any
service.HTTPClient = &http.Client{Transport: roundTripper(func(request *http.Request) (*http.Response, error) {
raw, err := io.ReadAll(request.Body)
if err != nil {
t.Fatal(err)
}
if err := json.Unmarshal(raw, &sent); err != nil {
t.Fatal(err)
}
body := `{"choices":[{"message":{"content":"{\"color\":\"黑色\",\"size\":\"XL\",\"reason\":\"原文对应唯一候选\",\"confidence\":0.95}"}}]}`
return &http.Response{StatusCode: http.StatusOK, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(body)), Request: request}, nil
})}
result, err := service.ResolveSYBSpec(context.Background(), SYBSpecParseRequest{ProductSpec: "黑色 XL【备注】", Colors: []string{"黑色", "白色"}, Sizes: []string{"L", "XL"}})
if err != nil || result.Color != "黑色" || result.Size != "XL" || result.Confidence == nil || *result.Confidence != 0.95 {
t.Fatalf("result=%+v err=%v", result, err)
}
encoded, _ := json.Marshal(sent)
for _, forbidden := range []string{"orderCode", "address", "rawJson", "price", "test-secret"} {
if strings.Contains(string(encoded), forbidden) {
t.Fatalf("provider payload leaked forbidden field %q: %s", forbidden, encoded)
}
}
for _, required := range []string{"productSpec", "shopeeColorCandidates", "shopeeSizeCandidates"} {
if !strings.Contains(string(encoded), required) {
t.Fatalf("provider payload missing %q: %s", required, encoded)
}
}
}
func TestResolveSYBSpecRejectsCandidateOutsideClosedSetAndMissingConfidence(t *testing.T) {
service := matcherTestService(t)
if _, err := service.SaveSettings(context.Background(), SaveSettingsRequest{Enabled: true, BaseURL: "https://provider.example/v1", Model: "test-model", APIKey: "test-secret", TimeoutSeconds: 8}, 7); err != nil {
t.Fatal(err)
}
responses := []string{
`{"choices":[{"message":{"content":"{\"color\":\"灰色\",\"size\":\"XL\",\"reason\":\"猜测\",\"confidence\":0.99}"}}]}`,
`{"choices":[{"message":{"content":"{\"color\":\"黑色\",\"size\":\"XL\",\"reason\":\"候选\"}"}}]}`,
}
service.HTTPClient = &http.Client{Transport: roundTripper(func(request *http.Request) (*http.Response, error) {
body := responses[0]
responses = responses[1:]
return &http.Response{StatusCode: http.StatusOK, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(body)), Request: request}, nil
})}
request := SYBSpecParseRequest{ProductSpec: "黑 XL", Colors: []string{"黑色"}, Sizes: []string{"XL"}}
for i := 0; i < 2; i++ {
_, err := service.ResolveSYBSpec(context.Background(), request)
var target *Error
if !errors.As(err, &target) || target.Code != CodeNoMatch {
t.Fatalf("attempt %d err=%v", i, err)
}
}
}
+28 -1
View File
@@ -10,6 +10,9 @@ import (
"net/http"
"strings"
"time"
log "github.com/go-admin-team/go-admin-core/logger"
"github.com/google/uuid"
)
// Suggestion limits are enforced defensively here too, even though callers
@@ -95,18 +98,26 @@ func (s *Service) SuggestBatch(ctx context.Context, request SuggestRequest) (Sug
}
httpRequest.Header.Set("Authorization", "Bearer "+apiKey)
httpRequest.Header.Set("Content-Type", "application/json")
callID, startedAt := uuid.NewString(), time.Now()
response, err := s.httpClient().Do(httpRequest)
if err != nil {
s.logProviderFailure(ProviderFailureDiagnostic{CallID: callID, Operation: "suggest_batch", Kind: providerNetworkErrorKind(err), Duration: time.Since(startedAt)})
return SuggestResult{}, &Error{Code: CodeProviderUnavailable, Message: "AI 建议服务暂时不可用", Cause: err}
}
defer response.Body.Close()
limited := io.LimitReader(response.Body, 1<<20)
responseBody, readErr := io.ReadAll(limited)
if readErr != nil || response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
if readErr != nil {
s.logProviderFailure(ProviderFailureDiagnostic{CallID: callID, Operation: "suggest_batch", Kind: "read_error", StatusCode: response.StatusCode, Duration: time.Since(startedAt)})
return SuggestResult{}, fail(CodeProviderUnavailable, "AI 建议服务暂时不可用")
}
if response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
s.logProviderFailure(ProviderFailureDiagnostic{CallID: callID, Operation: "suggest_batch", Kind: "http_status", StatusCode: response.StatusCode, Duration: time.Since(startedAt)})
return SuggestResult{}, fail(CodeProviderUnavailable, "AI 建议服务暂时不可用")
}
raw, err := parseSuggestChoices(responseBody)
if err != nil {
s.logProviderFailure(ProviderFailureDiagnostic{CallID: callID, Operation: "suggest_batch", Kind: "invalid_response", StatusCode: response.StatusCode, Duration: time.Since(startedAt)})
return SuggestResult{}, fail(CodeProviderUnavailable, "AI 建议响应无效")
}
@@ -145,6 +156,22 @@ func (s *Service) SuggestBatch(ctx context.Context, request SuggestRequest) (Sug
return SuggestResult{Decisions: decisions, Provider: ProviderOpenAICompatible, Model: setting.Model}, nil
}
func (s *Service) logProviderFailure(diagnostic ProviderFailureDiagnostic) {
if s.ProviderFailureLogger != nil {
s.ProviderFailureLogger(diagnostic)
return
}
log.Warnf("AI provider call failed: call_id=%s operation=%s kind=%s status=%d duration_ms=%d",
diagnostic.CallID, diagnostic.Operation, diagnostic.Kind, diagnostic.StatusCode, diagnostic.Duration.Milliseconds())
}
func providerNetworkErrorKind(err error) string {
if errors.Is(err, context.DeadlineExceeded) {
return "timeout"
}
return "network_error"
}
func suggestSystemPrompt(dimension string) string {
noun := "颜色或尺码"
switch dimension {
@@ -6,7 +6,9 @@ import (
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"go-admin/app/goauto/models"
@@ -134,3 +136,93 @@ func TestSuggestBatchRequiresConfiguredProvider(t *testing.T) {
t.Fatalf("expected CodeNotConfigured, got %v", target.Code)
}
}
func TestSuggestBatchLogsSafeDiagnosticForProvider502(t *testing.T) {
const sensitiveBody = "api-key-and-provider-body-must-not-be-logged"
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
http.Error(w, sensitiveBody, http.StatusBadGateway)
}))
defer server.Close()
db := openSuggestTestDB(t)
seedEnabledSetting(t, db, server.URL)
var diagnostic ProviderFailureDiagnostic
service := NewService(db)
service.ProviderFailureLogger = func(value ProviderFailureDiagnostic) { diagnostic = value }
_, err := service.SuggestBatch(context.Background(), SuggestRequest{
Sources: []SuggestSource{{ID: "s1", Label: "sensitive-source"}},
Candidates: []SuggestCandidate{{ID: "c1", Label: "sensitive-candidate"}},
})
if err == nil {
t.Fatal("expected provider failure")
}
if diagnostic.Operation != "suggest_batch" || diagnostic.Kind != "http_status" || diagnostic.StatusCode != http.StatusBadGateway || diagnostic.CallID == "" {
t.Fatalf("unexpected diagnostic: %+v", diagnostic)
}
printed := fmt.Sprintf("%+v", diagnostic)
for _, secret := range []string{sensitiveBody, "sensitive-source", "sensitive-candidate", "test-key", server.URL} {
if strings.Contains(printed, secret) {
t.Fatalf("diagnostic leaked %q: %s", secret, printed)
}
}
}
func TestSuggestBatchClassifiesProviderTimeout(t *testing.T) {
db := openSuggestTestDB(t)
seedEnabledSetting(t, db, "http://provider.invalid")
var diagnostic ProviderFailureDiagnostic
service := NewService(db)
service.HTTPClient = &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
return nil, context.DeadlineExceeded
})}
service.ProviderFailureLogger = func(value ProviderFailureDiagnostic) { diagnostic = value }
_, err := service.SuggestBatch(context.Background(), SuggestRequest{
Sources: []SuggestSource{{ID: "s1", Label: "黑色"}}, Candidates: []SuggestCandidate{{ID: "c1", Label: "黑色"}},
})
if err == nil || diagnostic.Kind != "timeout" || diagnostic.StatusCode != 0 {
t.Fatalf("timeout was not safely classified: diagnostic=%+v err=%v", diagnostic, err)
}
}
func TestSuggestBatchClassifiesInvalidResponseWithoutLoggingBody(t *testing.T) {
const sensitiveBody = "not-json-with-sensitive-provider-details"
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(sensitiveBody))
}))
defer server.Close()
db := openSuggestTestDB(t)
seedEnabledSetting(t, db, server.URL)
var diagnostic ProviderFailureDiagnostic
service := NewService(db)
service.ProviderFailureLogger = func(value ProviderFailureDiagnostic) { diagnostic = value }
_, err := service.SuggestBatch(context.Background(), SuggestRequest{
Sources: []SuggestSource{{ID: "s1", Label: "黑色"}}, Candidates: []SuggestCandidate{{ID: "c1", Label: "黑色"}},
})
if err == nil || diagnostic.Kind != "invalid_response" || strings.Contains(fmt.Sprintf("%+v", diagnostic), sensitiveBody) {
t.Fatalf("invalid response diagnostic is unsafe or missing: diagnostic=%+v err=%v", diagnostic, err)
}
}
func TestSuggestBatchAllowsProviderResponseAfterTwoSeconds(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
time.Sleep(2100 * time.Millisecond)
chatCompletionResponder(`{"suggestions":[{"sourceId":"s1","candidateId":"c1","confidence":0.95,"reason":"match"}]}`)(w, r)
}))
defer server.Close()
db := openSuggestTestDB(t)
seedEnabledSetting(t, db, server.URL)
result, err := NewService(db).SuggestBatch(context.Background(), SuggestRequest{
Sources: []SuggestSource{{ID: "s1", Label: "黑色"}}, Candidates: []SuggestCandidate{{ID: "c1", Label: "黑色"}},
})
if err != nil || result.Decisions["s1"].CandidateID != "c1" {
t.Fatalf("delayed provider response failed: result=%+v err=%v", result, err)
}
}
type roundTripFunc func(*http.Request) (*http.Response, error)
func (fn roundTripFunc) RoundTrip(request *http.Request) (*http.Response, error) { return fn(request) }
+18
View File
@@ -29,6 +29,24 @@ type MatchRequest struct {
Sizes []string
}
// SYBSpecParseRequest contains the only source text and closed Shopee
// candidate sets that may leave GoAuto for an AI-assisted SYB parse. It must
// never contain the shipment/order, account, address, price or full raw JSON.
type SYBSpecParseRequest struct {
ProductSpec string
Colors []string
Sizes []string
}
type SYBSpecParseResult struct {
Color string
Size string
Provider string
Model string
Reason string
Confidence *float64
}
type CandidateSnapshot struct {
Colors []string `json:"colors,omitempty"`
Sizes []string `json:"sizes,omitempty"`
+1 -21
View File
@@ -48,7 +48,7 @@ func (handler Handler) Register(context *gin.Context) {
writeError(context, internalError(err))
return
}
response, err := NewService(db).Register(context.Request.Context(), request, bearerToken(context.GetHeader("Authorization")), strings.TrimSpace(context.GetHeader("X-GoAuto-Device-Recovery-Code")))
response, err := NewService(db).Register(context.Request.Context(), request, bearerToken(context.GetHeader("Authorization")))
if err != nil {
writeError(context, err)
return
@@ -84,26 +84,6 @@ func (handler Handler) RevokeToken(context *gin.Context) {
handler.adminAction(context, (*Service).RevokeToken)
}
func (handler Handler) ResetIdentity(context *gin.Context) {
deviceID, err := strconv.ParseUint(context.Param("deviceId"), 10, 64)
if err != nil || deviceID == 0 {
writeError(context, invalidRequest("deviceId 无效"))
return
}
db, err := handler.database(context)
if err != nil {
writeError(context, internalError(err))
return
}
response, err := NewService(db).ResetIdentity(context.Request.Context(), deviceID)
if err != nil {
writeError(context, err)
return
}
context.Header("Cache-Control", "no-store")
context.JSON(http.StatusOK, gin.H{"code": http.StatusOK, "data": response})
}
func (handler Handler) adminAction(context *gin.Context, action func(*Service, stdcontext.Context, uint64) error) {
deviceID, err := strconv.ParseUint(context.Param("deviceId"), 10, 64)
if err != nil || deviceID == 0 {
-1
View File
@@ -22,6 +22,5 @@ func InitRouter(engine *gin.Engine, authMiddleware *jwt.GinJWTMiddleware) {
admin := engine.Group("/api/admin/v1/devices").Use(authMiddleware.MiddlewareFunc()).Use(middleware.AuthCheckRole())
admin.GET("", handler.List)
admin.POST("/:deviceId/disable", middleware.RequireRoleKey("admin"), handler.Disable)
admin.POST("/:deviceId/identity-reset", middleware.RequireRoleKey("admin"), handler.ResetIdentity)
admin.POST("/:deviceId/token/revoke", middleware.RequireRoleKey("admin"), handler.RevokeToken)
}
+4 -77
View File
@@ -28,12 +28,8 @@ const (
CodeDeviceTaskMismatch = "DEVICE_TASK_MISMATCH"
CodeDeviceNotFound = "DEVICE_NOT_FOUND"
CodeInternal = "INTERNAL_ERROR"
CodeRecoveryInvalid = "DEVICE_RECOVERY_INVALID"
CodeRecoveryExpired = "DEVICE_RECOVERY_EXPIRED"
)
const deviceRecoveryLifetime = 10 * time.Minute
type ServiceError struct {
Code string
Message string
@@ -71,11 +67,6 @@ type RegisterResponse struct {
Replayed bool `json:"replayed,omitempty"`
}
type ResetIdentityResponse struct {
DeviceID uint64 `json:"deviceId"`
ExpiresAt time.Time `json:"expiresAt"`
}
type Service struct {
DB *gorm.DB
Now func() time.Time
@@ -117,10 +108,6 @@ func tokenMatches(token, digest string) bool {
return subtle.ConstantTimeCompare(got[:], want) == 1
}
func digestMatches(value string, digest *string) bool {
return digest != nil && tokenMatches(value, *digest)
}
// Authenticate returns the active device represented by a bearer token.
// Agent feature packages use this method so token verification stays in one
// place and raw tokens never leave request memory.
@@ -144,11 +131,7 @@ func (service *Service) Authenticate(ctx context.Context, token string) (models.
return result, nil
}
func (service *Service) Register(ctx context.Context, request RegisterRequest, presentedToken string, recoveryCodes ...string) (RegisterResponse, error) {
recoveryCode := ""
if len(recoveryCodes) > 0 {
recoveryCode = recoveryCodes[0]
}
func (service *Service) Register(ctx context.Context, request RegisterRequest, presentedToken string) (RegisterResponse, error) {
request = normalizeRegisterRequest(request)
if err := validateRegisterRequest(request); err != nil {
return RegisterResponse{}, err
@@ -179,15 +162,9 @@ func (service *Service) Register(ctx context.Context, request RegisterRequest, p
if existing.Status == models.DeviceStatusDisabled {
return RegisterResponse{}, &ServiceError{Code: CodeDeviceDisabled, Message: "设备已停用", Retryable: false}
}
usingRecovery := !tokenMatches(presentedToken, existing.TokenDigest)
if usingRecovery {
validCode := recoveryCode != "" && digestMatches(recoveryCode, existing.RecoveryCodeDigest)
autoRecovery := recoveryCode == "" && existing.RecoveryCodeDigest == nil && existing.RecoveryExpiresAt != nil && existing.RecoveryUsedAt == nil
if !validCode && !autoRecovery {
return RegisterResponse{}, &ServiceError{Code: CodeInstallIDConflict, Message: "installId 已注册,需要该设备的有效 Token", Retryable: false}
}
if existing.RecoveryExpiresAt == nil || !service.Now().Before(*existing.RecoveryExpiresAt) {
return RegisterResponse{}, &ServiceError{Code: CodeRecoveryExpired, Message: "设备身份重置窗口已过期,请在后台重新操作", Retryable: false}
if !tokenMatches(presentedToken, existing.TokenDigest) {
return RegisterResponse{}, &ServiceError{
Code: CodeInstallIDConflict, Message: "installId 已注册,需要该设备的有效 Token", Retryable: false,
}
}
updates := map[string]any{
@@ -198,20 +175,6 @@ func (service *Service) Register(ctx context.Context, request RegisterRequest, p
if request.Capabilities != nil {
updates["capabilities_json"] = encodeCapabilities(request.Capabilities)
}
if usingRecovery {
newToken, generateErr := service.GenerateToken()
if generateErr != nil {
return RegisterResponse{}, internalError(generateErr)
}
now := service.Now()
updates["token_digest"] = tokenDigest(newToken)
updates["token_issued_at"] = now
updates["recovery_code_digest"] = nil
updates["recovery_expires_at"] = nil
updates["recovery_used_at"] = now
updates["status"] = models.DeviceStatusOnline
response.DeviceToken = newToken
}
result := db.Model(&models.AgentDevice{}).
Where("id = ? AND token_digest = ? AND token_revoked_at IS NULL", existing.ID, existing.TokenDigest).
Updates(updates)
@@ -259,42 +222,6 @@ func (service *Service) Register(ctx context.Context, request RegisterRequest, p
return response, nil
}
// ResetIdentity invalidates the current token and opens a short-lived automatic
// re-registration window for the same installId. It retains the device row and
// task bindings, so no recovery code needs to leave the Admin workflow.
func (service *Service) ResetIdentity(ctx context.Context, deviceID uint64) (ResetIdentityResponse, error) {
if service.DB == nil {
return ResetIdentityResponse{}, internalError(errors.New("database is nil"))
}
now := service.Now()
expiresAt := now.Add(deviceRecoveryLifetime)
response := ResetIdentityResponse{DeviceID: deviceID, ExpiresAt: expiresAt}
err := service.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
var device models.AgentDevice
if err := tx.First(&device, deviceID).Error; errors.Is(err, gorm.ErrRecordNotFound) {
return &ServiceError{Code: CodeDeviceNotFound, Message: "设备不存在", Retryable: false}
} else if err != nil {
return internalError(err)
}
if device.Status == models.DeviceStatusDisabled {
return &ServiceError{Code: CodeDeviceDisabled, Message: "设备已停用", Retryable: false}
}
invalidatedToken, generateErr := service.GenerateToken()
if generateErr != nil {
return internalError(generateErr)
}
return tx.Model(&models.AgentDevice{}).Where("id = ?", deviceID).Updates(map[string]any{
"token_digest": tokenDigest(invalidatedToken), "token_issued_at": now,
"recovery_code_digest": nil, "recovery_expires_at": expiresAt,
"recovery_used_at": nil, "status": models.DeviceStatusOffline,
}).Error
})
if err != nil {
return ResetIdentityResponse{}, err
}
return response, nil
}
func (service *Service) Disable(ctx context.Context, deviceID uint64) error {
return service.deactivate(ctx, deviceID, false)
}
-66
View File
@@ -185,72 +185,6 @@ func TestDisableAndRevokePreventFurtherAuthentication(t *testing.T) {
}
}
func TestResetIdentityRecoversSameDeviceAndRotatesToken(t *testing.T) {
db := openTestDatabase(t)
service := newTestService(t, db)
tokens := []string{"original-token", "invalidated-token", "replacement-token"}
service.GenerateToken = func() (string, error) {
if len(tokens) == 0 {
t.Fatal("unexpected token generation")
}
value := tokens[0]
tokens = tokens[1:]
return value, nil
}
request := validRegisterRequest()
registered, err := service.Register(context.Background(), request, "")
if err != nil {
t.Fatalf("first registration: %v", err)
}
recovery, err := service.ResetIdentity(context.Background(), registered.DeviceID)
if err != nil {
t.Fatalf("reset identity: %v", err)
}
if !recovery.ExpiresAt.After(service.Now()) {
t.Fatalf("unexpected recovery response: %+v", recovery)
}
request.RequestID = uuid.NewString()
recovered, err := service.Register(context.Background(), request, "original-token")
if err != nil {
t.Fatalf("recover registration: %v", err)
}
if recovered.DeviceID != registered.DeviceID || recovered.DeviceToken != "replacement-token" {
t.Fatalf("unexpected recovery result: %+v", recovered)
}
var stored models.AgentDevice
if err := db.First(&stored, registered.DeviceID).Error; err != nil {
t.Fatalf("load recovered device: %v", err)
}
if stored.InstallID != strings.ToLower(request.InstallID) || stored.Status != models.DeviceStatusOnline || stored.RecoveryCodeDigest != nil || stored.RecoveryUsedAt == nil {
t.Fatalf("recovery did not retain device safely: %+v", stored)
}
request.RequestID = uuid.NewString()
if _, err := service.Register(context.Background(), request, "original-token"); serviceErrorCode(t, err) != CodeInstallIDConflict {
t.Fatal("automatic recovery window was reusable")
}
}
func TestResetIdentityRejectsExpiredRecoveryCode(t *testing.T) {
db := openTestDatabase(t)
service := newTestService(t, db)
tokens := []string{"original-token", "invalidated-token"}
service.GenerateToken = func() (string, error) { value := tokens[0]; tokens = tokens[1:]; return value, nil }
request := validRegisterRequest()
registered, err := service.Register(context.Background(), request, "")
if err != nil {
t.Fatal(err)
}
recovery, err := service.ResetIdentity(context.Background(), registered.DeviceID)
if err != nil {
t.Fatal(err)
}
service.Now = func() time.Time { return recovery.ExpiresAt.Add(time.Second) }
request.RequestID = uuid.NewString()
if _, err := service.Register(context.Background(), request, ""); serviceErrorCode(t, err) != CodeRecoveryExpired {
t.Fatalf("expected expired recovery code, got %v", err)
}
}
func TestRegistrationValidationRejectsInvalidUUIDs(t *testing.T) {
service := newTestService(t, openTestDatabase(t))
request := validRegisterRequest()
+4
View File
@@ -34,7 +34,11 @@ func MigratedModels() []any {
&models.PDDProduct{},
&models.AIMatchingSetting{},
&models.ShopeeProduct{},
&models.ShopeeSpecAutoMatchRun{},
&models.ShopeeSpecAutoMatchWorkItem{},
&models.SYBProduct{},
&models.SYBSpecAIParseRun{},
&models.SYBSpecAIParseWorkItem{},
&models.SYBSession{},
&models.SYBShop{},
&models.SYBSyncRun{},
+9 -3
View File
@@ -39,9 +39,6 @@ type AgentDevice struct {
TokenDigest string `json:"-" gorm:"size:64;not null;uniqueIndex:ux_agent_device_token_digest"`
TokenIssuedAt time.Time `json:"tokenIssuedAt" gorm:"not null"`
TokenRevokedAt *time.Time `json:"tokenRevokedAt" gorm:"index"`
RecoveryCodeDigest *string `json:"-" gorm:"size:64;index"`
RecoveryExpiresAt *time.Time `json:"-" gorm:"index"`
RecoveryUsedAt *time.Time `json:"-"`
LastRegisterRequestID *string `json:"-" gorm:"size:36;uniqueIndex:ux_agent_device_last_register_request_id"`
LastHeartbeatRequestID *string `json:"-" gorm:"size:36;uniqueIndex:ux_agent_device_last_heartbeat_request_id"`
LastHeartbeatAt *time.Time `json:"lastHeartbeatAt" gorm:"index"`
@@ -536,6 +533,15 @@ type SYBProduct struct {
// recording the parser's own last output for audit even after a manual
// correction; it is not overwritten by the correction itself.
ManuallyConfirmed bool `json:"manuallyConfirmed" gorm:"not null;default:false"`
// AIConfirmed is independent of ParseStatus: ParseStatus remains the
// deterministic parser's audit result, while these fields record a closed-
// candidate, high-confidence AI decision. Human correction always clears
// and supersedes this decision.
AIConfirmed bool `json:"aiConfirmed" gorm:"not null;default:false;index"`
AIConfidence *float64 `json:"aiConfidence,omitempty"`
AIReason string `json:"aiReason,omitempty" gorm:"size:500;not null;default:''"`
AIConfirmedAt *time.Time `json:"aiConfirmedAt,omitempty"`
AIInputFingerprint string `json:"-" gorm:"size:64;not null;default:'';index"`
// RawJSON is the untouched `details[]` element as SYB returned it. It is
// what reparse (#41: "适用于解析规则更新后批量重跑,只读取已保存的原始
@@ -0,0 +1,56 @@
package models
import "time"
// ShopeeSpecAutoMatchRun is one scheduled or administrator-triggered batch.
// ActiveSlot is 1 only while running; its nullable unique index is the
// database-level cross-process mutex shared by both trigger paths.
type ShopeeSpecAutoMatchRun struct {
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
RequestID string `json:"requestId" gorm:"size:36;not null;uniqueIndex:ux_shopee_spec_auto_match_run_request"`
Trigger string `json:"trigger" gorm:"size:16;not null;index"`
Status string `json:"status" gorm:"size:24;not null;index"`
ActiveSlot *uint8 `json:"-" gorm:"uniqueIndex:ux_shopee_spec_auto_match_run_active"`
LeaseOwner string `json:"-" gorm:"size:64;not null;default:''"`
LeaseExpiresAt *time.Time `json:"-" gorm:"index"`
RequestedBy *uint64 `json:"requestedBy,omitempty"`
BatchLimit int `json:"batchLimit" gorm:"not null;default:20"`
ScannedCount int `json:"scannedCount" gorm:"not null;default:0"`
EligibleCount int `json:"eligibleCount" gorm:"not null;default:0"`
ProcessedCount int `json:"processedCount" gorm:"not null;default:0"`
ConfirmedCount int `json:"confirmedCount" gorm:"not null;default:0"`
UnmatchedCount int `json:"unmatchedCount" gorm:"not null;default:0"`
FailedCount int `json:"failedCount" gorm:"not null;default:0"`
ErrorSummary string `json:"errorSummary,omitempty" gorm:"size:500;not null;default:''"`
StartedAt time.Time `json:"startedAt" gorm:"not null"`
FinishedAt *time.Time `json:"finishedAt,omitempty"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (ShopeeSpecAutoMatchRun) TableName() string { return "shopee_spec_auto_match_run" }
// ShopeeSpecAutoMatchWorkItem remembers the last input fingerprint and retry
// state for each product, preventing unchanged low-confidence inputs from
// repeatedly spending AI calls.
type ShopeeSpecAutoMatchWorkItem struct {
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
ShopeeProductID uint64 `json:"shopeeProductId" gorm:"not null;uniqueIndex:ux_shopee_spec_auto_match_work_product"`
RunID *uint64 `json:"runId,omitempty" gorm:"index"`
InputFingerprint string `json:"inputFingerprint" gorm:"size:128;not null;default:'';index"`
Status string `json:"status" gorm:"size:24;not null;index"`
AttemptCount int `json:"attemptCount" gorm:"not null;default:0"`
NextAttemptAt *time.Time `json:"nextAttemptAt,omitempty" gorm:"index"`
LeaseOwner string `json:"-" gorm:"size:64;not null;default:''"`
LeaseExpiresAt *time.Time `json:"-" gorm:"index"`
ConfirmedCount int `json:"confirmedCount" gorm:"not null;default:0"`
UnmatchedCount int `json:"unmatchedCount" gorm:"not null;default:0"`
LastErrorCode string `json:"lastErrorCode,omitempty" gorm:"size:64;not null;default:''"`
LastError string `json:"lastError,omitempty" gorm:"size:500;not null;default:''"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (ShopeeSpecAutoMatchWorkItem) TableName() string {
return "shopee_spec_auto_match_work_item"
}
@@ -0,0 +1,48 @@
package models
import "time"
// SYBSpecAIParseRun is one globally serialized scheduled batch.
type SYBSpecAIParseRun struct {
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
RequestID string `json:"requestId" gorm:"size:36;not null;uniqueIndex:ux_syb_spec_ai_parse_run_request"`
Trigger string `json:"trigger" gorm:"size:16;not null;index"`
Status string `json:"status" gorm:"size:24;not null;index"`
ActiveSlot *uint8 `json:"-" gorm:"uniqueIndex:ux_syb_spec_ai_parse_run_active"`
LeaseOwner string `json:"-" gorm:"size:64;not null;default:''"`
LeaseExpiresAt *time.Time `json:"-" gorm:"index"`
BatchLimit int `json:"batchLimit" gorm:"not null;default:20"`
ScannedCount int `json:"scannedCount" gorm:"not null;default:0"`
EligibleCount int `json:"eligibleCount" gorm:"not null;default:0"`
ProcessedCount int `json:"processedCount" gorm:"not null;default:0"`
ConfirmedCount int `json:"confirmedCount" gorm:"not null;default:0"`
UnmatchedCount int `json:"unmatchedCount" gorm:"not null;default:0"`
FailedCount int `json:"failedCount" gorm:"not null;default:0"`
ErrorSummary string `json:"errorSummary,omitempty" gorm:"size:500;not null;default:''"`
StartedAt time.Time `json:"startedAt" gorm:"not null"`
FinishedAt *time.Time `json:"finishedAt,omitempty"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (SYBSpecAIParseRun) TableName() string { return "syb_spec_ai_parse_run" }
// SYBSpecAIParseWorkItem prevents unchanged ambiguous input from repeatedly
// spending provider calls and owns the per-row recovery lease.
type SYBSpecAIParseWorkItem struct {
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
SYBProductID uint64 `json:"sybProductId" gorm:"not null;uniqueIndex:ux_syb_spec_ai_parse_work_product"`
RunID *uint64 `json:"runId,omitempty" gorm:"index"`
InputFingerprint string `json:"inputFingerprint" gorm:"size:64;not null;default:'';index"`
Status string `json:"status" gorm:"size:24;not null;index"`
AttemptCount int `json:"attemptCount" gorm:"not null;default:0"`
NextAttemptAt *time.Time `json:"nextAttemptAt,omitempty" gorm:"index"`
LeaseOwner string `json:"-" gorm:"size:64;not null;default:''"`
LeaseExpiresAt *time.Time `json:"-" gorm:"index"`
LastErrorCode string `json:"lastErrorCode,omitempty" gorm:"size:64;not null;default:''"`
LastError string `json:"lastError,omitempty" gorm:"size:500;not null;default:''"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (SYBSpecAIParseWorkItem) TableName() string { return "syb_spec_ai_parse_work_item" }
@@ -2,6 +2,7 @@ package purchase
import (
"context"
"strings"
"testing"
"time"
@@ -157,7 +158,7 @@ func TestMatchedReplacementExposesContinueAndReusesAgentRetry(t *testing.T) {
}
}
func TestMatchedReplacementAllowsRetryWithoutPersistedMappingAndDoesNotCallAI(t *testing.T) {
func TestMatchedReplacementRejectsClearedMappingWithoutCallingAI(t *testing.T) {
db := testDB(t)
f := seed(t, db, liveCaps(), true)
setCollectedPDDPrice(t, db, f.pdd.ID)
@@ -171,11 +172,11 @@ func TestMatchedReplacementAllowsRetryWithoutPersistedMappingAndDoesNotCallAI(t
service.Matcher = matcher
detail, err := service.AgentHistoryDetail(context.Background(), failed.ID, f.token)
if err != nil || !detail.ContinuePurchaseEligible || detail.ContinuePurchaseDisabledReason != "" || matcher.calls != 0 {
if err != nil || detail.ContinuePurchaseEligible || !strings.Contains(detail.ContinuePurchaseDisabledReason, "规格匹配已失效") || matcher.calls != 0 {
t.Fatalf("cleared mapping qualification mismatch: detail=%+v calls=%d error=%v", detail, matcher.calls, err)
}
created, err := service.AgentRetry(context.Background(), failed.ID, AgentRetryRequest{RequestID: uuid.NewString()}, f.token)
if err != nil || created.TaskID == 0 || matcher.calls != 0 {
t.Fatalf("cleared mapping retry did not create probe task: result=%+v calls=%d error=%v", created, matcher.calls, err)
_, err = service.AgentRetry(context.Background(), failed.ID, AgentRetryRequest{RequestID: uuid.NewString()}, f.token)
if code(err) != CodeMappingRequired || matcher.calls != 0 {
t.Fatalf("cleared mapping reached AI or retry: code=%s calls=%d error=%v", code(err), matcher.calls, err)
}
}
@@ -32,10 +32,41 @@ type skuCombinationRow struct {
}
func sybSpecsTrusted(syb models.SYBProduct) bool {
if syb.ParseStatus == models.SYBParseStatusFailed {
return syb.ParseStatus == models.SYBParseStatusSuccess || syb.ManuallyConfirmed || syb.AIConfirmed
}
func aiConfirmedSpecsCurrent(syb models.SYBProduct, shopee models.ShopeeProduct) bool {
if !syb.AIConfirmed {
return true
}
specs, err := shopeeproduct.Unmarshal(shopee.SpecsJSON)
if err != nil {
return false
}
return strings.TrimSpace(syb.TargetColor) != "" || strings.TrimSpace(syb.TargetSize) != ""
wanted := map[string]string{shopeeproduct.RoleColor: strings.TrimSpace(syb.TargetColor), shopeeproduct.RoleSize: strings.TrimSpace(syb.TargetSize)}
foundAny := false
for role, target := range wanted {
if target == "" {
continue
}
foundAny = true
found := false
for _, dimension := range specs {
if dimension.Role != role {
continue
}
for _, value := range dimension.Values {
if value.Name == target {
found = true
break
}
}
}
if !found {
return false
}
}
return foundAny
}
func (s *Service) loadLatestSKUCombinations(ctx context.Context, pddIDs []uint64, dataset *batchPreviewDataset) error {
@@ -143,10 +174,7 @@ func aiMatchQualificationForDataset(id uint64, dataset batchPreviewDataset) aiMa
if request.TargetSize != "" {
size = matched.MappedSize
}
// 人工录入或外部导入的 PDD 档案可以没有采集任务 SKU 证据。没有
// 证据时只确认候选值;一旦有证据,仍必须命中同一个可售组合。
combinations := dataset.skuCombinationsByPDD[pdd.ID]
if len(combinations) == 0 || validSKUCombination(combinations, syb.TargetColor, syb.TargetSize, color, size) {
if validSKUCombination(dataset.skuCombinationsByPDD[pdd.ID], syb.TargetColor, syb.TargetSize, color, size) {
return aiMatchQualification{Eligible: true, Request: request, MappedColor: mappedColor, MappedSize: mappedSize, Deterministic: &matched}
}
}
+78 -5
View File
@@ -105,7 +105,8 @@ func (s *Service) BatchPreview(ctx context.Context, req BatchPreviewRequest) (Ba
items := make([]BatchPreviewItem, 0, len(ids))
eligible := 0
for _, id := range ids {
item := s.previewFromDataset(id, dataset, guard)
item := s.previewFromDataset(ctx, id, dataset, false, guard)
item.enforcePersistedMatch(id, dataset)
item.applyCollectionEligibility(id, dataset)
qualification := aiMatchQualificationForDataset(id, dataset)
item.AIMatchEligible = qualification.Eligible
@@ -330,6 +331,22 @@ func (s *Service) validateBatchDevice(ctx context.Context, deviceID *uint64, rul
return ensureCapabilities(record, purchasecontract.RequiredCapabilities(rule))
}
func (s *Service) previewOne(ctx context.Context, id uint64) BatchPreviewItem {
_, _, currentRule, ruleErr := purchaserule.CurrentRule(ctx, s.DB, models.PurchaseExecutionModeLive)
if ruleErr != nil {
item := BatchPreviewItem{SYBProductID: id}
item.ReasonCode, item.Reason, item.NextAction = serviceErrorFields(ruleErr)
return item
}
dataset, err := s.loadBatchPreviewDataset(ctx, []uint64{id})
if err != nil {
item := BatchPreviewItem{SYBProductID: id}
item.ReasonCode, item.Reason, item.NextAction = CodeInternal, "服务端处理失败", "retry"
return item
}
return s.previewFromDataset(ctx, id, dataset, true, purchasecontract.EffectivePriceGuard(currentRule))
}
func (s *Service) previewOneDeterministic(ctx context.Context, id uint64, guard purchasecontract.PriceLimits) BatchPreviewItem {
dataset, err := s.loadBatchPreviewDataset(ctx, []uint64{id})
if err != nil {
@@ -337,7 +354,8 @@ func (s *Service) previewOneDeterministic(ctx context.Context, id uint64, guard
item.ReasonCode, item.Reason, item.NextAction = CodeInternal, "服务端处理失败", "retry"
return item
}
item := s.previewFromDataset(id, dataset, guard)
item := s.previewFromDataset(ctx, id, dataset, false, guard)
item.enforcePersistedMatch(id, dataset)
return item
}
@@ -361,7 +379,7 @@ func taskPointerValue(value *uint64) uint64 {
return *value
}
func (s *Service) previewFromDataset(id uint64, dataset batchPreviewDataset, guard purchasecontract.PriceLimits) BatchPreviewItem {
func (s *Service) previewFromDataset(ctx context.Context, id uint64, dataset batchPreviewDataset, allowAI bool, guard purchasecontract.PriceLimits) BatchPreviewItem {
item := BatchPreviewItem{SYBProductID: id}
syb, found := dataset.sybByID[id]
if !found {
@@ -388,6 +406,10 @@ func (s *Service) previewFromDataset(id uint64, dataset batchPreviewDataset, gua
item.ReasonCode, item.Reason, item.NextAction = "SHOPEE_NOT_FOUND", "关联的蝦皮商品不存在,请先处理商品档案", "open_shopee"
return item
}
if !aiConfirmedSpecsCurrent(syb, shopee) {
item.ReasonCode, item.Reason, item.NextAction = "SYB_PARSE_FAILED", "AI 解析依据已变化,请等待重新解析或人工修正", "reparse"
return item
}
if shopee.PDDProductID == nil {
item.ReasonCode, item.Reason, item.NextAction = "PDD_NOT_LINKED", "尚未关联 PDD 商品,请先关联", processActionOpenPDDLink
return item
@@ -405,15 +427,26 @@ func (s *Service) previewFromDataset(id uint64, dataset batchPreviewDataset, gua
}
mappedColor, mappedSize, source := confirmedMappings(shopee.SpecsJSON, syb.TargetColor, syb.TargetSize)
item.MappedColor, item.MappedSize = mappedColor, mappedSize
candidates, _ := archiveCandidates(pdd.SpecsJSON, syb.TargetColor, syb.TargetSize)
candidates, archiveUsable := archiveCandidates(pdd.SpecsJSON, syb.TargetColor, syb.TargetSize)
// #190:失效的已保存映射不再拦截,降级为 unresolved,与创建路径保持一致。
if source != "unresolved" && !mappingTargetsValid(candidates, syb.TargetColor, syb.TargetSize, mappedColor, mappedSize) {
mappedColor, mappedSize, source = "", "", "unresolved"
item.MappedColor, item.MappedSize = "", ""
}
if source == "unresolved" {
if !archiveUsable {
item.ReasonCode, item.Reason, item.NextAction = CodeMappingRequired, "关联的 PDD 商品没有完整可选规格", "open_pdd"
return item
}
request := aimatching.MatchRequest{TargetColor: syb.TargetColor, TargetSize: syb.TargetSize, Colors: candidates.Colors, Sizes: candidates.Sizes}
if matched, ok := aimatching.DeterministicMatch(request); ok {
if allowAI {
matched, matchErr := s.matcher().Resolve(ctx, request)
if matchErr != nil {
item.ReasonCode, item.Reason, item.NextAction = serviceErrorFields(purchaseMatchError(matchErr))
return item
}
item.MappedColor, item.MappedSize = matched.MappedColor, matched.MappedSize
} else if matched, ok := aimatching.DeterministicMatch(request); ok {
item.MappedColor, item.MappedSize = matched.MappedColor, matched.MappedSize
}
// #190:映射不完整不再拦截,任务以 unresolved 建立并交由规格探测解析。
@@ -447,6 +480,46 @@ func (s *Service) previewFromDataset(id uint64, dataset batchPreviewDataset, gua
return item
}
// enforcePersistedMatch is the #188 gate for the SYB list and batch-create
// workflow. A transient deterministic result is not purchase readiness: the
// color/size mapping must already be confirmed on the Shopee product and must
// still identify one complete, available SKU combination from the latest
// successful collection.
func (item *BatchPreviewItem) enforcePersistedMatch(id uint64, dataset batchPreviewDataset) {
if item.ReasonCode != "" && item.ReasonCode != CodeMappingRequired {
return
}
syb, ok := dataset.sybByID[id]
if !ok || syb.ShopeeProductID == nil {
return
}
shopee, ok := dataset.shopeeByID[*syb.ShopeeProductID]
if !ok || shopee.PDDProductID == nil {
return
}
pdd, ok := dataset.pddByID[*shopee.PDDProductID]
if !ok || pdd.Status != "active" {
return
}
mappedColor, mappedSize, source := confirmedMappings(shopee.SpecsJSON, syb.TargetColor, syb.TargetSize)
item.MappedColor, item.MappedSize = mappedColor, mappedSize
if source == "unresolved" {
item.Eligible = false
item.ReasonCode, item.Reason, item.NextAction = CodeMappingRequired, "规格尚未匹配并保存,请先完成 AI 匹配或人工确认", "open_mapping"
return
}
combinations := dataset.skuCombinationsByPDD[pdd.ID]
if len(combinations) == 0 {
item.Eligible = false
item.ReasonCode, item.Reason, item.NextAction = CodeMappingRequired, "缺少当前 PDD 商品的完整可售 SKU 组合,请先重新采集", "open_pdd"
return
}
if !validSKUCombination(combinations, syb.TargetColor, syb.TargetSize, mappedColor, mappedSize) {
item.Eligible = false
item.ReasonCode, item.Reason, item.NextAction = CodeMappingRequired, "已保存规格映射不属于当前可售的 PDD 规格组合,请重新匹配", "open_mapping"
}
}
func purchasePriceRange(raw, mappedColor string, guard purchasecontract.PriceLimits) (reference, minPrice, maxPrice int64, err error) {
var dimensions []productspec.Dimension
if json.Unmarshal([]byte(raw), &dimensions) != nil {
@@ -50,6 +50,11 @@ func (s *Service) BatchSpecMatch(ctx context.Context, req BatchSpecMatchRequest)
if err != nil {
return BatchSpecMatchResponse{}, internal(err)
}
settings, err := aimatching.NewService(s.DB).Settings(ctx)
if err != nil {
return BatchSpecMatchResponse{}, internal(err)
}
response := BatchSpecMatchResponse{Items: make([]BatchSpecMatchItem, 0, len(ids))}
for _, id := range ids {
item := BatchSpecMatchItem{SYBProductID: id, Status: BatchSpecMatchFailed}
@@ -82,11 +87,9 @@ func (s *Service) BatchSpecMatch(ctx context.Context, req BatchSpecMatchRequest)
}
}
item.Source, item.Confidence = matched.Source, matched.Decision.Confidence
// #200:在 SYB 批量入口,AI 只要返回了可保存的规格结果,就由后续的
// 候选与可售 SKU 组合校验决定是否放行;置信度仅保留为审计信息。
autoConfirm := qualification.Deterministic != nil || (matched.Source == aimatching.SourceAI && strings.TrimSpace(matched.Decision.Reason) != "")
autoConfirm := qualification.Deterministic != nil || (matched.Source == aimatching.SourceAI && matched.Decision.Confidence != nil && *matched.Decision.Confidence >= settings.AutoConfirmMinConfidence && strings.TrimSpace(matched.Decision.Reason) != "")
if !autoConfirm {
item.Status, item.Reason = BatchSpecMatchPending, "AI 未返回可用规格结果,请人工确认"
item.Status, item.Reason = BatchSpecMatchPending, "匹配结果未达到自动确认阈值,请人工确认"
if strings.TrimSpace(matched.Decision.Reason) != "" {
item.Reason += ":" + strings.TrimSpace(matched.Decision.Reason)
}
@@ -101,8 +104,7 @@ func (s *Service) BatchSpecMatch(ctx context.Context, req BatchSpecMatchRequest)
if request.TargetSize != "" {
mappedSize = matched.MappedSize
}
combinations := dataset.skuCombinationsByPDD[pdd.ID]
if !mappingTargetsValid(specCandidates{Colors: request.Colors, Sizes: request.Sizes}, syb.TargetColor, syb.TargetSize, mappedColor, mappedSize) || (len(combinations) > 0 && !validSKUCombination(combinations, syb.TargetColor, syb.TargetSize, mappedColor, mappedSize)) {
if !mappingTargetsValid(specCandidates{Colors: request.Colors, Sizes: request.Sizes}, syb.TargetColor, syb.TargetSize, mappedColor, mappedSize) || !validSKUCombination(dataset.skuCombinationsByPDD[pdd.ID], syb.TargetColor, syb.TargetSize, mappedColor, mappedSize) {
item.Status, item.Reason = BatchSpecMatchPending, "AI 结果不是当前可售的 PDD 规格组合,请人工确认"
response.PendingCount++
response.Items = append(response.Items, item)
@@ -115,7 +117,7 @@ func (s *Service) BatchSpecMatch(ctx context.Context, req BatchSpecMatchRequest)
response.Items = append(response.Items, item)
continue
}
if _, err := shopeeproduct.NewService(s.DB).ApplyResolvedMappings(ctx, shopee.ID, uuid.NewString(), writes); err != nil {
if _, err := shopeeproduct.NewService(s.DB).ApplyResolvedMappings(ctx, shopee.ID, uuid.NewString(), settings.AutoConfirmMinConfidence, writes); err != nil {
item.Reason = "规格映射保存失败,请刷新后重试"
response.FailedCount++
response.Items = append(response.Items, item)
@@ -9,6 +9,7 @@ import (
"go-admin/app/goauto/models"
"go-admin/app/goauto/shopeeproduct"
"github.com/google/uuid"
"gorm.io/gorm"
)
@@ -136,14 +137,18 @@ func exactBatchSpecFixture(t *testing.T) (*Service, fixture) {
return testService(db), f
}
func TestBatchSpecMatchPersistsExactMatchButPurchaseDoesNotWaitForIt(t *testing.T) {
func TestBatchSpecMatchPersistsExactMatchBeforePurchaseCreation(t *testing.T) {
service, f := exactBatchSpecFixture(t)
matcher := &batchSpecMatcher{}
service.Matcher = matcher
before, err := service.BatchPreview(context.Background(), BatchPreviewRequest{SYBProductIDs: []uint64{f.syb.ID}})
if err != nil || len(before.Items) != 1 || !before.Items[0].Eligible || !before.Items[0].AIMatchEligible || before.Items[0].ProcessStage != ProcessStagePurchaseReady {
t.Fatalf("live-probe purchase was incorrectly gated by persisted mapping: %+v err=%v", before, err)
if err != nil || len(before.Items) != 1 || before.Items[0].Eligible || !before.Items[0].AIMatchEligible || before.Items[0].ProcessStage != ProcessStageColorMapping {
t.Fatalf("unpersisted exact match was not gated: %+v err=%v", before, err)
}
blocked, err := service.BatchCreate(context.Background(), BatchCreateRequest{RequestID: uuid.NewString(), SYBProductIDs: []uint64{f.syb.ID}, DeviceID: &f.device.ID})
if err != nil || blocked.CreatedCount != 0 || blocked.FailedCount != 1 || blocked.Items[0].ReasonCode != CodeMappingRequired {
t.Fatalf("batch create bypassed persisted mapping gate: %+v err=%v", blocked, err)
}
matched, err := service.BatchSpecMatch(context.Background(), BatchSpecMatchRequest{SYBProductIDs: []uint64{f.syb.ID}})
@@ -170,10 +175,10 @@ func TestBatchSpecMatchPersistsExactMatchButPurchaseDoesNotWaitForIt(t *testing.
}
}
func TestExtractedUncertainSpecsCanBeMatchedBeforePurchase(t *testing.T) {
func TestManuallyConfirmedUncertainSpecsCanBeMatchedBeforePurchase(t *testing.T) {
service, f := exactBatchSpecFixture(t)
if err := service.DB.Model(&models.SYBProduct{}).Where("id = ?", f.syb.ID).Updates(map[string]any{
"parse_status": models.SYBParseStatusUncertain, "manually_confirmed": false,
"parse_status": models.SYBParseStatusUncertain, "manually_confirmed": true,
}).Error; err != nil {
t.Fatal(err)
}
@@ -181,16 +186,16 @@ func TestExtractedUncertainSpecsCanBeMatchedBeforePurchase(t *testing.T) {
service.Matcher = matcher
before, err := service.BatchPreview(context.Background(), BatchPreviewRequest{SYBProductIDs: []uint64{f.syb.ID}})
if err != nil || len(before.Items) != 1 || !before.Items[0].Eligible || !before.Items[0].AIMatchEligible || before.Items[0].ProcessStage != ProcessStagePurchaseReady {
t.Fatalf("extracted uncertain specs were not purchase-ready for live probing: %+v err=%v", before, err)
if err != nil || len(before.Items) != 1 || before.Items[0].Eligible || !before.Items[0].AIMatchEligible || before.Items[0].ProcessStage != ProcessStageColorMapping {
t.Fatalf("manually confirmed specs did not enter matching: %+v err=%v", before, err)
}
matched, err := service.BatchSpecMatch(context.Background(), BatchSpecMatchRequest{SYBProductIDs: []uint64{f.syb.ID}})
if err != nil || matched.AutoConfirmedCount != 1 || matcher.calls != 0 {
t.Fatalf("extracted uncertain specs did not allow exact match: %+v calls=%d err=%v", matched, matcher.calls, err)
t.Fatalf("manual correction did not allow exact match: %+v calls=%d err=%v", matched, matcher.calls, err)
}
after, err := service.BatchPreview(context.Background(), BatchPreviewRequest{SYBProductIDs: []uint64{f.syb.ID}})
if err != nil || !after.Items[0].Eligible || after.Items[0].ProcessStage != ProcessStagePurchaseReady {
t.Fatalf("saved mapping did not unlock extracted uncertain specs: %+v err=%v", after, err)
t.Fatalf("saved mapping did not unlock manual correction: %+v err=%v", after, err)
}
}
@@ -231,7 +236,7 @@ func TestBatchPreviewExposesExplicitAIMatchEligibility(t *testing.T) {
t.Fatalf("preview failed: %+v err=%v", preview, err)
}
item := preview.Items[0]
if !item.AIMatchEligible || item.AIMatchDisabledReason != "" || !item.Eligible || item.ProcessStage != ProcessStagePurchaseReady {
if !item.AIMatchEligible || item.AIMatchDisabledReason != "" || item.Eligible || item.ProcessStage != ProcessStageColorMapping {
t.Fatalf("explicit AI eligibility mismatch: %+v", item)
}
}
@@ -260,38 +265,21 @@ func TestBatchPreviewAllowsUncertainParseAndMissingSKUCombination(t *testing.T)
}
}
func TestBatchSpecMatchAndPreviewAllowMissingSKUCombinationEvidence(t *testing.T) {
service, f := unresolvedBatchSpecFixture(t)
if err := service.DB.Where("pdd_product_id = ?", f.pdd.ID).Delete(&models.CollectionTask{}).Error; err != nil {
t.Fatal(err)
}
service.Matcher = &batchSpecMatcher{results: []aimatching.MatchResult{aiBatchResult(0.6)}}
matched, err := service.BatchSpecMatch(context.Background(), BatchSpecMatchRequest{SYBProductIDs: []uint64{f.syb.ID}})
if err != nil || matched.AutoConfirmedCount != 1 || matched.PendingCount != 0 {
t.Fatalf("AI result without SKU evidence was not confirmed: %+v err=%v", matched, err)
}
preview, err := service.BatchPreview(context.Background(), BatchPreviewRequest{SYBProductIDs: []uint64{f.syb.ID}})
if err != nil || len(preview.Items) != 1 || !preview.Items[0].Eligible || preview.Items[0].ProcessStage != ProcessStagePurchaseReady {
t.Fatalf("missing SKU evidence unexpectedly blocked purchase: %+v err=%v", preview, err)
}
}
func TestBatchSpecMatchAutoConfirmsReturnedAIMatchRegardlessOfConfidence(t *testing.T) {
func TestBatchSpecMatchLeavesLowConfidenceForManualHandling(t *testing.T) {
service, f := unresolvedBatchSpecFixture(t)
service.Matcher = &batchSpecMatcher{results: []aimatching.MatchResult{aiBatchResult(0.6)}}
response, err := service.BatchSpecMatch(context.Background(), BatchSpecMatchRequest{SYBProductIDs: []uint64{f.syb.ID}})
if err != nil || response.PendingCount != 0 || response.AutoConfirmedCount != 1 {
t.Fatalf("low-confidence AI result was not auto-confirmed: %+v err=%v", response, err)
if err != nil || response.PendingCount != 1 || response.AutoConfirmedCount != 0 {
t.Fatalf("unexpected low-confidence result: %+v err=%v", response, err)
}
mapping := savedColorMapping(t, service, f.shopee.ID)
if mapping == nil || mapping.PDDValue != "米白色" || mapping.Status != shopeeproduct.MappingStatusConfirmed || mapping.Confidence == nil || *mapping.Confidence != 0.6 {
t.Fatalf("AI result was not saved as confirmed: %+v", mapping)
if mapping == nil || mapping.PDDValue != "旧白色" || mapping.Status != shopeeproduct.MappingStatusConfirmed {
t.Fatalf("low-confidence result changed the saved mapping: %+v", mapping)
}
preview, err := service.BatchPreview(context.Background(), BatchPreviewRequest{SYBProductIDs: []uint64{f.syb.ID}})
if err != nil || !preview.Items[0].Eligible || preview.Items[0].ProcessStage != ProcessStagePurchaseReady {
t.Fatalf("saved AI result did not become purchase-ready: %+v err=%v", preview, err)
if err != nil || preview.Items[0].Eligible || preview.Items[0].ProcessStage != ProcessStageColorMapping {
t.Fatalf("pending mapping unexpectedly became purchase-ready: %+v err=%v", preview, err)
}
}
+20 -35
View File
@@ -44,24 +44,25 @@ func TestBatchPreviewUsesPDDPriceAndExplainsIneligibleRows(t *testing.T) {
}
}
func TestSybSpecsTrustedOnlyBlocksFailedOrEmptyExtraction(t *testing.T) {
tests := []struct {
name string
syb models.SYBProduct
trust bool
}{
{"success", models.SYBProduct{ParseStatus: models.SYBParseStatusSuccess, TargetColor: "黑色", TargetSize: "XL"}, true},
{"uncertain with color", models.SYBProduct{ParseStatus: models.SYBParseStatusUncertain, TargetColor: "套装"}, true},
{"uncertain with size", models.SYBProduct{ParseStatus: models.SYBParseStatusUncertain, TargetSize: "均码"}, true},
{"failed with values", models.SYBProduct{ParseStatus: models.SYBParseStatusFailed, TargetColor: "黑色"}, false},
{"uncertain without values", models.SYBProduct{ParseStatus: models.SYBParseStatusUncertain}, false},
func TestSYBSpecsTrustedAcceptsAIWithoutCallingItManual(t *testing.T) {
row := models.SYBProduct{ParseStatus: models.SYBParseStatusUncertain, AIConfirmed: true}
if !sybSpecsTrusted(row) {
t.Fatal("a valid AI-confirmed parse must pass the purchase parse gate")
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := sybSpecsTrusted(tt.syb); got != tt.trust {
t.Fatalf("sybSpecsTrusted=%v, want %v", got, tt.trust)
}
})
if row.ManuallyConfirmed {
t.Fatal("AI confirmation must not be represented as human confirmation")
}
}
func TestAIConfirmedSpecsBecomeUntrustedWhenShopeeCandidateDisappears(t *testing.T) {
row := models.SYBProduct{TargetColor: "黑色", TargetSize: "XL", ParseStatus: models.SYBParseStatusUncertain, AIConfirmed: true}
product := models.ShopeeProduct{SpecsJSON: `[{"name":"颜色","role":"color","values":[{"name":"白色","source":"import"}]},{"name":"尺码","role":"size","values":[{"name":"XL","source":"import"}]}]`}
if aiConfirmedSpecsCurrent(row, product) {
t.Fatal("removed Shopee candidate must invalidate the AI parse gate")
}
product.SpecsJSON = `[{"name":"颜色","role":"color","values":[{"name":"黑色","source":"import"}]},{"name":"尺码","role":"size","values":[{"name":"XL","source":"import"}]}]`
if !aiConfirmedSpecsCurrent(row, product) {
t.Fatal("unchanged exact Shopee candidates should keep AI parse valid")
}
}
@@ -288,24 +289,8 @@ func TestBatchPreviewBulkLoadsAndNeverCallsAIMatcher(t *testing.T) {
if queries != 7 {
t.Fatalf("batch preview used %d queries, want 7 bounded queries including collection eligibility and current purchase rule", queries)
}
if len(response.Items) != 2 || !response.Items[0].Eligible || !response.Items[1].Eligible || response.EligibleCount != 2 {
t.Fatalf("unresolved rows must be ready for mandatory live probing: %+v", response)
}
}
func TestBatchPreviewAllowsIncompleteArchivedCandidatesForMandatoryLiveProbe(t *testing.T) {
db := testDB(t)
fixture := seed(t, db, liveCaps(), false)
if err := db.Model(&models.PDDProduct{}).Where("id = ?", fixture.pdd.ID).Update(
"specs_json",
`[{"name":"颜色","role":"color","values":[{"name":"黑色","selectable":true,"priceCent":2000}]}]`,
).Error; err != nil {
t.Fatal(err)
}
response, err := testService(db).BatchPreview(context.Background(), BatchPreviewRequest{SYBProductIDs: []uint64{fixture.syb.ID}})
if err != nil || len(response.Items) != 1 || !response.Items[0].Eligible || response.Items[0].ProcessStage != ProcessStagePurchaseReady {
t.Fatalf("incomplete archive blocked live probe task creation: %+v err=%v", response, err)
if len(response.Items) != 2 || response.Items[0].Eligible || response.Items[1].Eligible || response.Items[0].ReasonCode != CodeMappingRequired {
t.Fatalf("unresolved rows must not be reported as purchase-ready: %+v", response)
}
}
+12 -25
View File
@@ -266,9 +266,6 @@ func (s *Service) MarkOrderSubmitStarted(ctx context.Context, taskID uint64, req
if t.ExecutionMode != models.PurchaseExecutionModeLive {
return TaskPayload{}, fail(CodeStateConflict, "演练任务不能创建订单")
}
if a.Phase != models.PurchaseAttemptPhasePurchase {
return TaskPayload{}, fail(CodeStateConflict, "规格探测阶段不能进入创建订单边界")
}
if _, e := uuid.Parse(req.RequestID); e != nil {
return TaskPayload{}, fail(CodeInvalidRequest, "requestId 无效")
}
@@ -539,29 +536,19 @@ func (s *Service) resolveProbedSpecs(ctx context.Context, taskID uint64, attempt
}
decision.NoMatch, decision.Decision = true, snapshot
decision.FailureCode, decision.FailureMessage = "PURCHASE_SPEC_NOT_MATCHED", "没有找到可采购的 PDD 颜色或尺码"
} else {
matched, matchErr := s.matcher().Resolve(ctx, request)
valid := matchErr == nil && (matched.Source == aimatching.SourceExact || matched.Source == aimatching.SourceAI) &&
matchCandidateValid(request.TargetColor, matched.MappedColor, request.Colors) &&
matchCandidateValid(request.TargetSize, matched.MappedSize, request.Sizes)
if valid {
snapshot, marshalErr := json.Marshal(matched.Decision)
if marshalErr != nil {
return TaskPayload{}, internal(marshalErr)
}
decision.MappedColor, decision.MappedSize, decision.Source, decision.Decision = matched.MappedColor, matched.MappedSize, matched.Source, snapshot
} else {
reason := purchaseMatchReason(matchErr)
if matchErr == nil {
reason = "AI 规格匹配结果不属于当次 PDD 候选"
}
snapshot, marshalErr := json.Marshal(aimatching.NoMatchDecision(request, aimatching.SourceAI, reason))
if marshalErr != nil {
return TaskPayload{}, internal(marshalErr)
}
decision.NoMatch, decision.Decision = true, snapshot
decision.FailureCode, decision.FailureMessage = "PURCHASE_SPEC_NOT_MATCHED", reason
} else if matched, matchErr := s.matcher().Resolve(ctx, request); matchErr == nil {
snapshot, marshalErr := json.Marshal(matched.Decision)
if marshalErr != nil {
return TaskPayload{}, internal(marshalErr)
}
decision.MappedColor, decision.MappedSize, decision.Source, decision.Decision = matched.MappedColor, matched.MappedSize, matched.Source, snapshot
} else {
snapshot, marshalErr := json.Marshal(aimatching.NoMatchDecision(request, aimatching.SourceAI, purchaseMatchReason(matchErr)))
if marshalErr != nil {
return TaskPayload{}, internal(marshalErr)
}
decision.NoMatch, decision.Decision = true, snapshot
decision.FailureCode, decision.FailureMessage = "PURCHASE_SPEC_NOT_MATCHED", purchaseMatchReason(matchErr)
}
updated, _, err := s.applySpecDecision(ctx, taskID, decision, nil)
if err != nil {
@@ -40,19 +40,6 @@ func queuedMatchTask(t *testing.T) (*gorm.DB, fixture, models.PurchaseTask) {
if err != nil {
t.Fatal(err)
}
fingerprint, snapshot, err := currentPurchaseMatchInput(db, task)
if err != nil {
t.Fatal(err)
}
work := models.PurchaseSpecMatchWorkItem{
PurchaseTaskID: task.ID,
Status: models.PurchaseMatchPending,
InputFingerprint: fingerprint,
InputSnapshotJSON: snapshot,
}
if err := db.Create(&work).Error; err != nil {
t.Fatal(err)
}
return db, f, task
}
+3 -2
View File
@@ -69,7 +69,8 @@ func (s *Service) ProcessStages(ctx context.Context, ids []uint64) (map[uint64]P
return nil, err
}
for _, id := range ids {
preview := s.previewFromDataset(id, dataset, guard)
preview := s.previewFromDataset(ctx, id, dataset, false, guard)
preview.enforcePersistedMatch(id, dataset)
qualification := aiMatchQualificationForDataset(id, dataset)
preview.AIMatchEligible = qualification.Eligible
preview.AIMatchDisabledReason = qualification.DisabledReason
@@ -136,7 +137,7 @@ func processStageFromDataset(id uint64, dataset batchPreviewDataset, preview Bat
if !ok {
return stage(ProcessStageManualAction, "SYB 商品不存在或已删除", "refresh")
}
if syb.ParseStatus == models.SYBParseStatusFailed {
if syb.ParseStatus == models.SYBParseStatusFailed && !syb.ManuallyConfirmed && !syb.AIConfirmed {
return stage(ProcessStageManualAction, "解析失败,请先处理", "reparse")
}
if syb.ShopeeProductID == nil {
+15 -14
View File
@@ -218,12 +218,7 @@ func (s *Service) retryEligibility(ctx context.Context, task models.PurchaseTask
return decision
}
_, _, currentRule, ruleErr := purchaserule.CurrentRule(ctx, s.DB, models.PurchaseExecutionModeLive)
if ruleErr != nil {
code, message, _ := serviceErrorFields(ruleErr)
return retryDecision{ReasonCode: code, Reason: message}
}
preview := s.previewOneDeterministic(ctx, *task.SYBProductID, purchasecontract.EffectivePriceGuard(currentRule))
preview := s.previewOne(ctx, *task.SYBProductID)
if !preview.Eligible {
return retryDecision{ReasonCode: preview.ReasonCode, Reason: preview.Reason}
}
@@ -235,9 +230,10 @@ func (s *Service) retryEligibility(ctx context.Context, task models.PurchaseTask
}
// retryQueryEligibility is deliberately limited to local task and device
// safety checks. The batch-retry command performs deterministic archive and
// price validation, but specification resolution belongs to the new task's
// mandatory live probe and never calls AI before task creation.
// safety checks. Admin list/detail queries must never call previewOne because
// previewOne can fall back to the external AI provider for unresolved specs.
// The batch-retry command runs retryEligibility and performs that complete,
// current-archive validation only after the operator submits the retry.
func (s *Service) retryQueryEligibility(ctx context.Context, task models.PurchaseTask, checkDeviceBusy bool) retryDecision {
decision := s.retryStateEligibility(ctx, task)
if !decision.Allowed {
@@ -267,20 +263,25 @@ func (s *Service) continuePurchaseEligibility(ctx context.Context, task models.P
guard := purchasecontract.EffectivePriceGuard(currentRule)
syb, found := dataset.sybByID[*task.SYBProductID]
if !found || syb.ShopeeProductID == nil {
preview := s.previewFromDataset(*task.SYBProductID, dataset, guard)
preview := s.previewFromDataset(ctx, *task.SYBProductID, dataset, false, guard)
return retryDecision{ReasonCode: preview.ReasonCode, Reason: preview.Reason}
}
shopee, found := dataset.shopeeByID[*syb.ShopeeProductID]
if !found || shopee.PDDProductID == nil {
preview := s.previewFromDataset(*task.SYBProductID, dataset, guard)
preview := s.previewFromDataset(ctx, *task.SYBProductID, dataset, false, guard)
return retryDecision{ReasonCode: preview.ReasonCode, Reason: preview.Reason}
}
_, found = dataset.pddByID[*shopee.PDDProductID]
pdd, found := dataset.pddByID[*shopee.PDDProductID]
if !found {
preview := s.previewFromDataset(*task.SYBProductID, dataset, guard)
preview := s.previewFromDataset(ctx, *task.SYBProductID, dataset, false, guard)
return retryDecision{ReasonCode: preview.ReasonCode, Reason: preview.Reason}
}
preview := s.previewFromDataset(*task.SYBProductID, dataset, guard)
mappedColor, mappedSize, source := confirmedMappings(shopee.SpecsJSON, syb.TargetColor, syb.TargetSize)
candidates, _ := archiveCandidates(pdd.SpecsJSON, syb.TargetColor, syb.TargetSize)
if source == "unresolved" || !mappingTargetsValid(candidates, syb.TargetColor, syb.TargetSize, mappedColor, mappedSize) {
return retryDecision{ReasonCode: CodeMappingRequired, Reason: "规格匹配已失效,请在 Admin 重新确认"}
}
preview := s.previewFromDataset(ctx, *task.SYBProductID, dataset, false, guard)
if !preview.Eligible {
return retryDecision{ReasonCode: preview.ReasonCode, Reason: preview.Reason}
}
+3 -3
View File
@@ -223,7 +223,7 @@ func TestAdminListRetryEligibilityRejectsBusyOriginalDevice(t *testing.T) {
}
}
func TestAdminQueriesAndBatchRetryDoNotResolveArchivedSpecs(t *testing.T) {
func TestAdminQueriesDoNotResolveSpecsButBatchRetryStillDoes(t *testing.T) {
db := testDB(t)
f := seed(t, db, liveCaps(), false)
service := testService(db)
@@ -253,8 +253,8 @@ func TestAdminQueriesAndBatchRetryDoNotResolveArchivedSpecs(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if matcher.calls != 0 || retry.CreatedCount != 1 || retry.FailedCount != 0 {
t.Fatalf("batch retry did not create a mandatory probe task: result=%+v matcherCalls=%d", retry, matcher.calls)
if matcher.calls != 1 || retry.CreatedCount != 0 || retry.FailedCount != 1 || retry.Items[0].ReasonCode != CodeMappingRequired {
t.Fatalf("batch retry skipped full matching: result=%+v matcherCalls=%d", retry, matcher.calls)
}
}
+75 -8
View File
@@ -54,6 +54,11 @@ type SpecMatcher interface {
Resolve(context.Context, aimatching.MatchRequest) (aimatching.MatchResult, error)
}
type externalMatchPlan struct {
fingerprint string
snapshot string
}
type matchFingerprint struct {
SYBProductID uint64 `json:"sybProductId"`
TargetColor string `json:"targetColor"`
@@ -91,6 +96,51 @@ func (s *Service) Create(ctx context.Context, req CreateRequest) (models.Purchas
return call.task, call.replayed, call.err
}
func (s *Service) prepareExternalMatch(ctx context.Context, req CreateRequest) (*externalMatchPlan, error) {
if req.ExecutionMode != models.PurchaseExecutionModeLive || req.SYBProductID == nil {
return nil, nil
}
var syb models.SYBProduct
if err := s.DB.WithContext(ctx).First(&syb, *req.SYBProductID).Error; err != nil {
return nil, notFound(err, "顺云宝商品不存在")
}
if syb.ShopeeProductID == nil {
return nil, fail(CodeInvalidRequest, "该商品尚未关联蝦皮商品")
}
var shopee models.ShopeeProduct
if err := s.DB.WithContext(ctx).First(&shopee, *syb.ShopeeProductID).Error; err != nil {
return nil, notFound(err, "蝦皮商品不存在")
}
if shopee.PDDProductID == nil {
return nil, fail(CodeInvalidRequest, "该蝦皮商品尚未关联拼多多商品")
}
var pdd models.PDDProduct
if err := s.DB.WithContext(ctx).First(&pdd, *shopee.PDDProductID).Error; err != nil {
return nil, notFound(err, "拼多多商品不存在")
}
targetColor, targetSize := strings.TrimSpace(req.TargetColor), strings.TrimSpace(req.TargetSize)
if targetColor == "" {
targetColor = syb.TargetColor
}
if targetSize == "" {
targetSize = syb.TargetSize
}
_, _, source := confirmedMappings(shopee.SpecsJSON, targetColor, targetSize)
candidates, usable := archiveCandidates(pdd.SpecsJSON, targetColor, targetSize)
if pdd.Status != "active" || source != "unresolved" || !usable {
return nil, nil
}
request := aimatching.MatchRequest{TargetColor: targetColor, TargetSize: targetSize, Colors: candidates.Colors, Sizes: candidates.Sizes}
if _, deterministic := aimatching.DeterministicMatch(request); deterministic {
return nil, nil
}
fingerprint, snapshot, err := creationMatchInput(syb, shopee, pdd, request)
if err != nil {
return nil, internal(err)
}
return &externalMatchPlan{fingerprint: fingerprint, snapshot: snapshot}, nil
}
func creationMatchInput(syb models.SYBProduct, shopee models.ShopeeProduct, pdd models.PDDProduct, request aimatching.MatchRequest) (string, string, error) {
canonical := func(raw string) (json.RawMessage, error) {
var value any
@@ -144,6 +194,10 @@ func (s *Service) create(ctx context.Context, req CreateRequest) (models.Purchas
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
return models.PurchaseTask{}, false, internal(err)
}
externalPlan, err := s.prepareExternalMatch(ctx, req)
if err != nil {
return models.PurchaseTask{}, false, err
}
replayed := false
err = s.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
if err := tx.Where("create_request_id = ?", req.RequestID).First(&out).Error; err == nil {
@@ -280,20 +334,22 @@ func (s *Service) create(ctx context.Context, req CreateRequest) (models.Purchas
if currency == "" {
currency = shopee.Currency
}
// Every new SYB purchase must use the candidates observed on the
// current PDD page. Persisted mappings remain product-level history,
// but they cannot skip this task's read-only probe phase.
mappedColor, mappedSize, specSource = "", "", "unresolved"
mappedColor, mappedSize, specSource = confirmedMappings(shopee.SpecsJSON, targetColor, targetSize)
}
candidates, archiveUsable := archiveCandidates(pdd.SpecsJSON, targetColor, targetSize)
matchRequest := aimatching.MatchRequest{TargetColor: targetColor, TargetSize: targetSize, Colors: candidates.Colors, Sizes: candidates.Sizes}
if externalPlan != nil {
fingerprint, fingerprintErr := creationMatchFingerprint(syb, shopee, pdd, matchRequest)
if fingerprintErr != nil {
return internal(fingerprintErr)
}
if fingerprint != externalPlan.fingerprint {
return fail(CodeMappingRequired, "规格匹配输入已变化,请重新创建采购任务")
}
}
if taskType == models.PurchaseTaskTypeStock {
// Direct stock selection was already validated above. It never enters
// mapping, deterministic matching, or the external AI fallback.
} else if req.ExecutionMode == models.PurchaseExecutionModeLive {
// The first attempt is always spec_probe for an SYB purchase. The
// exact task-level decision is frozen only from that probe result.
mappedColor, mappedSize, specSource, decisionSnapshot = "", "", "unresolved", "{}"
} else if pdd.Status != "active" {
mappedColor, mappedSize, specSource = "", "", "unresolved"
} else if specSource == "manual_mapping" || specSource == "exact_match" || specSource == "ai_match" {
@@ -309,6 +365,8 @@ func (s *Service) create(ctx context.Context, req CreateRequest) (models.Purchas
}
} else if !archiveUsable {
mappedColor, mappedSize, specSource = "", "", "unresolved"
} else if externalPlan != nil {
mappedColor, mappedSize, specSource = "", "", "unresolved"
} else if match, matched := aimatching.DeterministicMatch(matchRequest); !matched {
// #190:确定性匹配失败不再拒绝,降级为 unresolved 交给规格探测。
mappedColor, mappedSize, specSource = "", "", "unresolved"
@@ -346,8 +404,17 @@ func (s *Service) create(ctx context.Context, req CreateRequest) (models.Purchas
if err := tx.Model(&out).Update("address_suffix", out.AddressSuffix).Error; err != nil {
return internal(err)
}
if externalPlan != nil {
work := models.PurchaseSpecMatchWorkItem{PurchaseTaskID: out.ID, Status: models.PurchaseMatchPending, InputFingerprint: externalPlan.fingerprint, InputSnapshotJSON: externalPlan.snapshot}
if err := tx.Create(&work).Error; err != nil {
return internal(err)
}
}
return nil
})
if err == nil && !replayed && externalPlan != nil && s.StartMatching != nil {
s.StartMatching(s.DB)
}
return out, replayed, err
}
+25 -148
View File
@@ -9,7 +9,6 @@ import (
"testing"
"time"
"go-admin/app/goauto/aimatching"
"go-admin/app/goauto/device"
"go-admin/app/goauto/migrations"
"go-admin/app/goauto/models"
@@ -30,17 +29,6 @@ type fixture struct {
token string
}
type liveProbeMatcher struct {
result aimatching.MatchResult
err error
calls int
}
func (matcher *liveProbeMatcher) Resolve(context.Context, aimatching.MatchRequest) (aimatching.MatchResult, error) {
matcher.calls++
return matcher.result, matcher.err
}
func testDB(t *testing.T) *gorm.DB {
t.Helper()
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared&_foreign_keys=on", strings.ReplaceAll(t.Name(), "/", "_"))
@@ -119,30 +107,6 @@ func createLive(t *testing.T, s *Service, f fixture) (models.PurchaseTask, error
r, _, err := s.Create(context.Background(), CreateRequest{RequestID: uuid.NewString(), ExecutionMode: models.PurchaseExecutionModeLive, SYBProductID: &f.syb.ID, DeviceID: &f.device.ID, MinUnitPriceCent: 400, MaxUnitPriceCent: 3000, RuleSnapshot: liveRule(true)})
return r, err
}
func startLivePurchaseAfterProbe(t *testing.T, s *Service, f fixture, task models.PurchaseTask) TaskPayload {
t.Helper()
if _, err := s.Claim(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
probeAttempt, err := s.Start(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
if err != nil || probeAttempt.Phase != models.PurchaseAttemptPhaseSpecProbe {
t.Fatalf("probe start: %+v %v", probeAttempt, err)
}
probe := ResultRequest{RequestID: uuid.NewString(), TaskAttemptID: probeAttempt.TaskAttemptID, ResultType: "spec_probe_completed", ProbedSpecs: []byte(`{"dimensions":[{"key":"color","values":["黑色"]},{"key":"size","values":["XL"]}]}`)}
if _, err = s.SubmitResult(context.Background(), task.ID, probe, f.token); err != nil {
t.Fatal(err)
}
if _, err = s.Claim(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
purchaseAttempt, err := s.Start(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
if err != nil || purchaseAttempt.Phase != models.PurchaseAttemptPhasePurchase {
t.Fatalf("purchase start: %+v %v", purchaseAttempt, err)
}
return purchaseAttempt
}
func code(err error) string {
var e *ServiceError
if errors.As(err, &e) {
@@ -260,23 +224,9 @@ func TestCreateAndLifecycleValidateCapabilitiesAndIdempotentResult(t *testing.T)
if err != nil {
t.Fatal(err)
}
if start.Phase != models.PurchaseAttemptPhaseSpecProbe {
if start.Phase != models.PurchaseAttemptPhasePurchase {
t.Fatalf("unexpected phase %s", start.Phase)
}
if _, err = s.MarkOrderSubmitStarted(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); code(err) != CodeStateConflict {
t.Fatalf("probe entered order boundary: %v", err)
}
probe := ResultRequest{RequestID: uuid.NewString(), TaskAttemptID: start.TaskAttemptID, ResultType: "spec_probe_completed", ProbedSpecs: []byte(`{"dimensions":[{"key":"color","values":["黑色"]},{"key":"size","values":["XL"]}]}`)}
if _, err = s.SubmitResult(context.Background(), task.ID, probe, f.token); err != nil {
t.Fatal(err)
}
if _, err = s.Claim(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
start, err = s.Start(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
if err != nil || start.Phase != models.PurchaseAttemptPhasePurchase {
t.Fatalf("purchase start: %+v %v", start, err)
}
if _, err = s.MarkOrderSubmitStarted(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
@@ -293,7 +243,7 @@ func TestCreateAndLifecycleValidateCapabilitiesAndIdempotentResult(t *testing.T)
}
var count int64
db.Model(&models.PurchaseTaskAttempt{}).Where("task_id = ?", task.ID).Count(&count)
if count != 2 || first.Status != models.PurchaseTaskStatusOrderCreated {
if count != 1 || first.Status != models.PurchaseTaskStatusOrderCreated {
t.Fatalf("duplicate attempt or wrong status: %d %+v", count, first)
}
var stored models.PurchaseTask
@@ -394,63 +344,6 @@ func TestSlowPathUsesTwoAttemptsAndFreezesDecision(t *testing.T) {
}
}
func TestLiveProbeCallsAIOnlyAfterCandidatesAreReturned(t *testing.T) {
db := testDB(t)
f := seed(t, db, liveCaps(), false)
if err := db.Model(&models.SYBProduct{}).Where("id = ?", f.syb.ID).Update("target_color", "象牙白").Error; err != nil {
t.Fatal(err)
}
request := aimatching.MatchRequest{TargetColor: "象牙白", TargetSize: "XL", Colors: []string{"米白色", "白色"}, Sizes: []string{"XL"}}
matcher := &liveProbeMatcher{result: aimatching.RecordedMatch(request, aimatching.SourceAI, "米白色", "XL", "在当次候选中唯一匹配")}
s := testService(db)
s.Matcher = matcher
task, err := createLive(t, s, f)
if err != nil || matcher.calls != 0 || task.SpecSource != "unresolved" {
t.Fatalf("task creation called matcher or skipped probe: task=%+v calls=%d err=%v", task, matcher.calls, err)
}
if _, err = s.Claim(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
first, err := s.Start(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
if err != nil || first.Phase != models.PurchaseAttemptPhaseSpecProbe {
t.Fatalf("probe start: %+v %v", first, err)
}
probe := ResultRequest{RequestID: uuid.NewString(), TaskAttemptID: first.TaskAttemptID, ResultType: "spec_probe_completed", ProbedSpecs: []byte(`{"dimensions":[{"key":"color","values":["米白色","白色"]},{"key":"size","values":["XL"]}]}`)}
resolved, err := s.SubmitResult(context.Background(), task.ID, probe, f.token)
if err != nil || matcher.calls != 1 || resolved.Status != models.PurchaseTaskStatusPending || resolved.MappedColor != "米白色" || resolved.MappedSize != "XL" {
t.Fatalf("probe AI result: %+v calls=%d err=%v", resolved, matcher.calls, err)
}
}
func TestLiveProbeRejectsMatcherValueOutsideCurrentCandidates(t *testing.T) {
db := testDB(t)
f := seed(t, db, liveCaps(), false)
request := aimatching.MatchRequest{TargetColor: "黑色", TargetSize: "XL", Colors: []string{"黑色"}, Sizes: []string{"XL"}}
matcher := &liveProbeMatcher{result: aimatching.RecordedMatch(request, aimatching.SourceAI, "候选外颜色", "XL", "无效返回")}
s := testService(db)
s.Matcher = matcher
task, err := createLive(t, s, f)
if err != nil {
t.Fatal(err)
}
if _, err = s.Claim(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
first, err := s.Start(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
if err != nil {
t.Fatal(err)
}
probe := ResultRequest{RequestID: uuid.NewString(), TaskAttemptID: first.TaskAttemptID, ResultType: "spec_probe_completed", ProbedSpecs: []byte(`{"dimensions":[{"key":"color","values":["黑色"]},{"key":"size","values":["XL"]}]}`)}
resolved, err := s.SubmitResult(context.Background(), task.ID, probe, f.token)
if err != nil || resolved.Status != models.PurchaseTaskStatusFailed || resolved.MappedColor != "" || resolved.MappedSize != "" {
t.Fatalf("out-of-candidate result was accepted: %+v err=%v", resolved, err)
}
var saved models.PurchaseTask
if err = db.First(&saved, task.ID).Error; err != nil || saved.ErrorCode == nil || *saved.ErrorCode != "PURCHASE_SPEC_NOT_MATCHED" {
t.Fatalf("outside-candidate failure was not persisted: %+v err=%v", saved, err)
}
}
func TestSecondSpecProbeFailsClosedWithoutClearingDecision(t *testing.T) {
db := testDB(t)
f := seed(t, db, liveCaps(), false)
@@ -535,7 +428,7 @@ func TestSpecResolutionAllowedMatrix(t *testing.T) {
}
}
func TestLiveProbeUsesDeterministicTraditionalAndWeightMatching(t *testing.T) {
func TestCreateUsesDeterministicTraditionalAndWeightMatching(t *testing.T) {
db := testDB(t)
f := seed(t, db, liveCaps(), false)
if err := db.Model(&models.PDDProduct{}).Where("id = ?", f.pdd.ID).Update("specs_json", `[{"name":"颜色","role":"color","values":[{"name":"浅蓝色","selectable":true,"priceCent":2000}]},{"name":"尺码","role":"size","values":[{"name":"4XL 160-170斤","selectable":true}]}]`).Error; err != nil {
@@ -544,26 +437,12 @@ func TestLiveProbeUsesDeterministicTraditionalAndWeightMatching(t *testing.T) {
if err := db.Model(&models.SYBProduct{}).Where("id = ?", f.syb.ID).Updates(map[string]any{"target_color": "淺藍色", "target_size": "4XL 80-85公斤"}).Error; err != nil {
t.Fatal(err)
}
s := testService(db)
task, err := createLive(t, s, f)
if err != nil || task.SpecSource != "unresolved" || task.MappedColorSnapshot != "" || task.MappedSizeSnapshot != "" {
t.Fatalf("live task skipped probe: %+v %v", task, err)
task, err := createLive(t, testService(db), f)
if err != nil || task.SpecSource != "exact_match" || task.MappedColorSnapshot != "浅蓝色" || task.MappedSizeSnapshot != "4XL 160-170斤" {
t.Fatalf("deterministic match failed: %+v %v", task, err)
}
if _, err = s.Claim(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
first, err := s.Start(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
if err != nil || first.Phase != models.PurchaseAttemptPhaseSpecProbe {
t.Fatalf("probe start: %+v %v", first, err)
}
probe := ResultRequest{RequestID: uuid.NewString(), TaskAttemptID: first.TaskAttemptID, ResultType: "spec_probe_completed", ProbedSpecs: []byte(`{"dimensions":[{"key":"color","values":["浅蓝色"]},{"key":"size","values":["4XL 160-170斤"]}]}`)}
resolved, err := s.SubmitResult(context.Background(), task.ID, probe, f.token)
if err != nil || resolved.MappedColor != "浅蓝色" || resolved.MappedSize != "4XL 160-170斤" {
t.Fatalf("deterministic probe match failed: %+v %v", resolved, err)
}
var saved models.PurchaseTask
if err = db.First(&saved, task.ID).Error; err != nil || strings.Contains(saved.SpecDecisionSnapshot, "apiKey") || !strings.Contains(saved.SpecDecisionSnapshot, "exact_match") {
t.Fatalf("task decision snapshot is unsafe or incomplete: %s err=%v", saved.SpecDecisionSnapshot, err)
if strings.Contains(task.SpecDecisionSnapshot, "apiKey") || !strings.Contains(task.SpecDecisionSnapshot, "exact_match") {
t.Fatalf("task decision snapshot is unsafe or incomplete: %s", task.SpecDecisionSnapshot)
}
}
@@ -578,12 +457,12 @@ func TestCreateDoesNotTrustMappedLabelsFromRequest(t *testing.T) {
SYBProductID: &f.syb.ID, DeviceID: &f.device.ID, MappedColor: "白色", MappedSize: "XL",
MinUnitPriceCent: 400, MaxUnitPriceCent: 3000, RuleSnapshot: liveRule(true),
})
if err != nil || task.SpecSource != "unresolved" || task.MappedColorSnapshot != "" || task.MappedSizeSnapshot != "" {
t.Fatalf("request-provided mapping must not skip live probe: %+v %v", task, err)
if err != nil || task.SpecSource != "exact_match" || task.MappedColorSnapshot != "黑色" || task.MappedSizeSnapshot != "XL" {
t.Fatalf("request-provided mapping must not override server match: %+v %v", task, err)
}
}
func TestCreateDoesNotQueueArchiveMatchingBeforeLiveProbe(t *testing.T) {
func TestCreateQueuesUsableArchiveWhenDeterministicMatchFails(t *testing.T) {
db := testDB(t)
f := seed(t, db, liveCaps(), false)
if err := db.Model(&models.PDDProduct{}).Where("id = ?", f.pdd.ID).Update("specs_json", `[{"name":"颜色","role":"color","values":[{"name":"白色","selectable":true,"priceCent":2000}]},{"name":"尺码","role":"size","values":[{"name":"L","selectable":true}]}]`).Error; err != nil {
@@ -596,12 +475,12 @@ func TestCreateDoesNotQueueArchiveMatchingBeforeLiveProbe(t *testing.T) {
t.Fatalf("task=%+v err=%v", task, err)
}
var count int64
if err := db.Model(&models.PurchaseSpecMatchWorkItem{}).Where("purchase_task_id = ?", task.ID).Count(&count).Error; err != nil || count != 0 {
if err := db.Model(&models.PurchaseSpecMatchWorkItem{}).Where("purchase_task_id = ?", task.ID).Count(&count).Error; err != nil || count != 1 {
t.Fatalf("work count=%d err=%v", count, err)
}
}
func TestCreateDispatchesProbeInsteadOfExternalArchiveMatching(t *testing.T) {
func TestCreateQueuesExternalMatchingWithoutCallingProvider(t *testing.T) {
db := testDB(t)
f := seed(t, db, liveCaps(), false)
if err := db.Model(&models.PDDProduct{}).Where("id = ?", f.pdd.ID).Update("specs_json", `[{"name":"颜色","role":"color","values":[{"name":"白色","selectable":true,"priceCent":2000},{"name":"米白色","selectable":true,"priceCent":2000}]},{"name":"尺码","role":"size","values":[{"name":"XL","selectable":true}]}]`).Error; err != nil {
@@ -616,12 +495,12 @@ func TestCreateDispatchesProbeInsteadOfExternalArchiveMatching(t *testing.T) {
if err != nil || task.ID == 0 || task.SpecSource != "unresolved" {
t.Fatalf("queued task=%+v err=%v", task, err)
}
var count int64
if err := db.Model(&models.PurchaseSpecMatchWorkItem{}).Where("purchase_task_id = ?", task.ID).Count(&count).Error; err != nil || count != 0 {
t.Fatalf("unexpected archive match work count=%d err=%v", count, err)
var work models.PurchaseSpecMatchWorkItem
if err := db.Where("purchase_task_id = ?", task.ID).First(&work).Error; err != nil || work.Status != models.PurchaseMatchPending || work.InputFingerprint == "" {
t.Fatalf("work=%+v err=%v", work, err)
}
if next, err := service.Next(context.Background(), f.token); err != nil || next == nil || next.MappedColor != "" || next.MappedSize != "" || !next.SpecResolutionAllowed {
t.Fatalf("probe task was not dispatchable: %+v %v", next, err)
if next, err := service.Next(context.Background(), f.token); err != nil || next != nil {
t.Fatalf("matching task dispatched: %+v %v", next, err)
}
}
@@ -630,10 +509,9 @@ func TestOrderUnknownIsNotAutomaticallyRedispatched(t *testing.T) {
f := seed(t, db, liveCaps(), true)
s := testService(db)
task, _ := createLive(t, s, f)
started := startLivePurchaseAfterProbe(t, s, f, task)
if _, err := s.MarkOrderSubmitStarted(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
s.Claim(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
started, _ := s.Start(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
s.MarkOrderSubmitStarted(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
actual := int64(1999)
if _, err := s.SubmitResult(context.Background(), task.ID, ResultRequest{
RequestID: uuid.NewString(), TaskAttemptID: started.TaskAttemptID, ResultType: "order_result_unknown",
@@ -649,7 +527,7 @@ func TestOrderUnknownIsNotAutomaticallyRedispatched(t *testing.T) {
t.Fatalf("unknown task evidence not preserved: %+v", saved)
}
var attempt models.PurchaseTaskAttempt
if err := db.Where("task_id = ?", task.ID).Order("attempt_number DESC").First(&attempt).Error; err != nil {
if err := db.Where("task_id = ?", task.ID).First(&attempt).Error; err != nil {
t.Fatal(err)
}
if attempt.ErrorCode == nil || *attempt.ErrorCode != CodeOrderTimeMissing || attempt.ErrorMessage == nil {
@@ -666,10 +544,9 @@ func TestOrderUnknownRejectsOrderFieldsAndUnapprovedFailureCode(t *testing.T) {
f := seed(t, db, liveCaps(), true)
s := testService(db)
task, _ := createLive(t, s, f)
started := startLivePurchaseAfterProbe(t, s, f, task)
if _, err := s.MarkOrderSubmitStarted(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token); err != nil {
t.Fatal(err)
}
s.Claim(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
started, _ := s.Start(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
s.MarkOrderSubmitStarted(context.Background(), task.ID, ActionRequest{RequestID: uuid.NewString()}, f.token)
if _, err := s.SubmitResult(context.Background(), task.ID, ResultRequest{
RequestID: uuid.NewString(), TaskAttemptID: started.TaskAttemptID, ResultType: "order_result_unknown",
@@ -48,6 +48,9 @@ type autoMatchRef struct {
// before the transaction; the transaction rechecks the complete spec context
// and PDD candidate set so a stale decision can never be written.
func (service *Service) AutoMatchMappings(ctx context.Context, id uint64, request AutoMatchRequest) (AutoMatchResponse, error) {
ctx, cancel := context.WithTimeout(ctx, aimatching.MaxProviderTimeout)
defer cancel()
requestID := strings.TrimSpace(request.RequestID)
if _, err := uuid.Parse(requestID); err != nil {
return AutoMatchResponse{}, invalidRequest("requestId 必须是 UUID")
@@ -0,0 +1,360 @@
package shopeeproduct
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"go-admin/app/goauto/models"
"github.com/google/uuid"
"gorm.io/gorm"
)
const (
SpecAutoMatchInvokeTarget = "GoAutoShopeeSpecAutoMatch"
defaultAutoMatchBatchLimit = 20
autoMatchLeaseDuration = 30 * time.Minute
autoMatchRetryDelay = time.Hour
maxAutoMatchAttempts = 3
)
type AutoMatchRunView struct {
models.ShopeeSpecAutoMatchRun
AlreadyRunning bool `json:"alreadyRunning,omitempty"`
Replayed bool `json:"replayed,omitempty"`
}
// StartAutoMatchRun acquires the single database-backed activity slot. A
// repeated requestId is idempotent; a concurrent trigger receives the current
// run instead of starting a second batch.
func (service *Service) StartAutoMatchRun(ctx context.Context, trigger, requestID string, requestedBy *uint64, batchLimit int) (AutoMatchRunView, bool, error) {
if _, err := uuid.Parse(strings.TrimSpace(requestID)); err != nil {
return AutoMatchRunView{}, false, invalidRequest("requestId 必须是 UUID")
}
if trigger != "manual" && trigger != "scheduled" {
return AutoMatchRunView{}, false, invalidRequest("trigger 无效")
}
if batchLimit <= 0 {
batchLimit = defaultAutoMatchBatchLimit
}
if batchLimit > 100 {
return AutoMatchRunView{}, false, invalidRequest("batchLimit 不能超过 100")
}
now := time.Now().UTC()
lease := now.Add(autoMatchLeaseDuration)
owner := uuid.NewString()
one := uint8(1)
var result models.ShopeeSpecAutoMatchRun
created := false
err := service.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
if err := tx.Model(&models.ShopeeSpecAutoMatchRun{}).
Where("status = ? AND active_slot = ? AND lease_expires_at < ?", "running", 1, now).
Updates(map[string]any{"status": "failed", "active_slot": nil, "lease_owner": "", "lease_expires_at": nil, "error_summary": "上次运行租约过期,已安全释放", "finished_at": now}).Error; err != nil {
return err
}
if err := tx.Where("request_id = ?", requestID).First(&result).Error; err == nil {
return nil
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
return err
}
if err := tx.Where("status = ? AND active_slot = ?", "running", 1).First(&result).Error; err == nil {
result.ActiveSlot = &one
return nil
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
return err
}
result = models.ShopeeSpecAutoMatchRun{RequestID: requestID, Trigger: trigger, Status: "running", ActiveSlot: &one, LeaseOwner: owner, LeaseExpiresAt: &lease, RequestedBy: requestedBy, BatchLimit: batchLimit, StartedAt: now}
if err := tx.Create(&result).Error; err != nil {
return err
}
created = true
return nil
})
if err != nil {
// A unique-slot race means another instance won after our read. Return
// its run as the stable, non-error result.
if findErr := service.DB.WithContext(ctx).Where("status = ? AND active_slot = ?", "running", 1).First(&result).Error; findErr == nil {
return AutoMatchRunView{ShopeeSpecAutoMatchRun: result, AlreadyRunning: true}, false, nil
}
return AutoMatchRunView{}, false, internalError(err)
}
view := AutoMatchRunView{ShopeeSpecAutoMatchRun: result}
if !created {
view.AlreadyRunning = result.RequestID != requestID
view.Replayed = result.RequestID == requestID
}
return view, created, nil
}
func (service *Service) LatestAutoMatchRun(ctx context.Context) (*AutoMatchRunView, error) {
var run models.ShopeeSpecAutoMatchRun
err := service.DB.WithContext(ctx).Order("id DESC").First(&run).Error
if errors.Is(err, gorm.ErrRecordNotFound) {
return nil, nil
}
if err != nil {
return nil, internalError(err)
}
return &AutoMatchRunView{ShopeeSpecAutoMatchRun: run}, nil
}
// ProcessAutoMatchRun performs a bounded batch. It is safe to call from an
// HTTP-launched goroutine or the scheduler because only the run owning the
// active slot may update and finish itself.
func (service *Service) ProcessAutoMatchRun(ctx context.Context, runID uint64) error {
var run models.ShopeeSpecAutoMatchRun
if err := service.DB.WithContext(ctx).First(&run, runID).Error; err != nil {
return err
}
if run.Status != "running" || run.ActiveSlot == nil || *run.ActiveSlot != 1 {
return nil
}
limit := run.BatchLimit
if limit <= 0 || limit > 100 {
limit = defaultAutoMatchBatchLimit
}
var candidates []models.ShopeeProduct
queryLimit := limit * 25
if queryLimit < 100 {
queryLimit = 100
}
if queryLimit > 1000 {
queryLimit = 1000
}
if err := service.DB.WithContext(ctx).
Joins("JOIN pdd_product ON pdd_product.id = shopee_product.pdd_product_id AND pdd_product.status = ?", "active").
Where("shopee_product.pdd_product_id IS NOT NULL").
Order("shopee_product.updated_at ASC, shopee_product.id ASC").Limit(queryLimit).Find(&candidates).Error; err != nil {
service.finishAutoMatchRun(run, "failed", 0, 0, 0, 0, 0, 1, "扫描符合条件的商品失败")
return err
}
eligible, processed, confirmed, unmatched, failed := 0, 0, 0, 0, 0
firstError := ""
for _, product := range candidates {
if processed >= limit {
break
}
fingerprint, ok, err := service.autoMatchEligibility(ctx, product)
if err != nil {
failed++
if firstError == "" {
firstError = safeBatchError(err)
}
continue
}
if !ok {
continue
}
eligible++
work, claimed, err := service.claimAutoMatchWork(ctx, run, product.ID, fingerprint)
if err != nil {
failed++
if firstError == "" {
firstError = safeBatchError(err)
}
continue
}
if !claimed {
continue
}
processed++
service.renewAutoMatchRun(run)
response, matchErr := service.AutoMatchMappings(ctx, product.ID, AutoMatchRequest{RequestID: uuid.NewString(), SpecContextVersion: fingerprint[:64]})
// fingerprint begins with the 64-character context version.
postFingerprint := fingerprint
if next, _, nextErr := service.autoMatchEligibility(ctx, product); nextErr == nil && next != "" {
postFingerprint = next
}
if matchErr != nil {
failed++
if firstError == "" {
firstError = safeBatchError(matchErr)
}
service.completeAutoMatchWork(work, postFingerprint, 0, 0, matchErr)
continue
}
confirmed += response.ConfirmedCount
unmatched += response.UnmatchedCount
service.completeAutoMatchWork(work, postFingerprint, response.ConfirmedCount, response.UnmatchedCount, nil)
}
status := "completed"
if failed > 0 {
status = "completed_partial"
}
return service.finishAutoMatchRun(run, status, len(candidates), eligible, processed, confirmed, unmatched, failed, firstError)
}
func (service *Service) autoMatchEligibility(ctx context.Context, product models.ShopeeProduct) (string, bool, error) {
if product.PDDProductID == nil {
return "", false, nil
}
var pdd models.PDDProduct
if err := service.DB.WithContext(ctx).First(&pdd, *product.PDDProductID).Error; err != nil {
return "", false, err
}
if pdd.Status != "active" {
return "", false, nil
}
shopeeSpecs, err := Unmarshal(product.SpecsJSON)
if err != nil {
return "", false, err
}
shared, needsMatch := false, false
for _, role := range []string{RoleColor, RoleSize} {
pddValues, err := selectablePDDValues(pdd.SpecsJSON, role)
if err != nil {
return "", false, err
}
if len(pddValues) == 0 {
continue
}
for _, dimension := range shopeeSpecs {
if dimension.Role != role || len(dimension.Values) == 0 {
continue
}
shared = true
for _, value := range dimension.Values {
if value.Mapping == nil || value.Mapping.Status != MappingStatusConfirmed || !pddValues[value.Mapping.PDDValue] {
needsMatch = true
}
}
}
}
if !shared || !needsMatch {
return "", false, nil
}
contextVersion := computeSpecContextVersion(product.PDDProductID, product.SpecsJSON, pdd.SpecsJSON)
var setting struct{ UpdatedAt time.Time }
_ = service.DB.WithContext(ctx).Table((models.AIMatchingSetting{}).TableName()).Select("updated_at").Where("id = ?", 1).Scan(&setting).Error
h := sha256.Sum256([]byte(contextVersion + "\x00" + setting.UpdatedAt.UTC().Format(time.RFC3339Nano)))
// Keeping the context version as a prefix lets ProcessAutoMatchRun pass the
// exact version to #194 without re-reading a potentially drifting input.
return contextVersion + hex.EncodeToString(h[:]), true, nil
}
func (service *Service) claimAutoMatchWork(ctx context.Context, run models.ShopeeSpecAutoMatchRun, productID uint64, fingerprint string) (models.ShopeeSpecAutoMatchWorkItem, bool, error) {
now := time.Now().UTC()
lease := now.Add(autoMatchLeaseDuration)
var work models.ShopeeSpecAutoMatchWorkItem
err := service.DB.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
err := tx.Where("shopee_product_id = ?", productID).First(&work).Error
if errors.Is(err, gorm.ErrRecordNotFound) {
work = models.ShopeeSpecAutoMatchWorkItem{ShopeeProductID: productID, RunID: &run.ID, InputFingerprint: fingerprint, Status: "running", AttemptCount: 1, LeaseOwner: run.LeaseOwner, LeaseExpiresAt: &lease}
return tx.Create(&work).Error
}
if err != nil {
return err
}
if work.InputFingerprint == fingerprint {
if work.Status == "completed" || work.Status == "unmatched" || work.AttemptCount >= maxAutoMatchAttempts || (work.NextAttemptAt != nil && work.NextAttemptAt.After(now)) || (work.Status == "running" && work.LeaseExpiresAt != nil && work.LeaseExpiresAt.After(now)) {
return errWorkNotClaimed
}
} else {
work.AttemptCount = 0
}
updates := map[string]any{"run_id": run.ID, "input_fingerprint": fingerprint, "status": "running", "attempt_count": work.AttemptCount + 1, "next_attempt_at": nil, "lease_owner": run.LeaseOwner, "lease_expires_at": lease, "last_error_code": "", "last_error": ""}
if err := tx.Model(&models.ShopeeSpecAutoMatchWorkItem{}).Where("id = ?", work.ID).Updates(updates).Error; err != nil {
return err
}
return tx.First(&work, work.ID).Error
})
if errors.Is(err, errWorkNotClaimed) {
return work, false, nil
}
return work, err == nil, err
}
var errWorkNotClaimed = errors.New("auto match work not claimed")
func (service *Service) completeAutoMatchWork(work models.ShopeeSpecAutoMatchWorkItem, fingerprint string, confirmed, unmatched int, matchErr error) {
now := time.Now().UTC()
updates := map[string]any{"input_fingerprint": fingerprint, "lease_owner": "", "lease_expires_at": nil, "confirmed_count": confirmed, "unmatched_count": unmatched}
if matchErr == nil {
if unmatched > 0 {
updates["status"] = "unmatched"
} else {
updates["status"] = "completed"
}
updates["next_attempt_at"], updates["last_error_code"], updates["last_error"] = nil, "", ""
} else {
code := batchErrorCode(matchErr)
updates["status"], updates["last_error_code"], updates["last_error"] = "failed", code, safeBatchError(matchErr)
if code == CodeAIUnavailable && work.AttemptCount < maxAutoMatchAttempts {
next := now.Add(autoMatchRetryDelay)
updates["next_attempt_at"] = next
} else {
updates["next_attempt_at"] = nil
}
}
_ = service.DB.Model(&models.ShopeeSpecAutoMatchWorkItem{}).Where("id = ?", work.ID).Updates(updates).Error
}
func (service *Service) renewAutoMatchRun(run models.ShopeeSpecAutoMatchRun) {
lease := time.Now().UTC().Add(autoMatchLeaseDuration)
_ = service.DB.Model(&models.ShopeeSpecAutoMatchRun{}).Where("id = ? AND status = ? AND lease_owner = ?", run.ID, "running", run.LeaseOwner).Update("lease_expires_at", lease).Error
}
func (service *Service) finishAutoMatchRun(run models.ShopeeSpecAutoMatchRun, status string, scanned, eligible, processed, confirmed, unmatched, failed int, summary string) error {
now := time.Now().UTC()
updates := map[string]any{"status": status, "active_slot": nil, "lease_owner": "", "lease_expires_at": nil, "scanned_count": scanned, "eligible_count": eligible, "processed_count": processed, "confirmed_count": confirmed, "unmatched_count": unmatched, "failed_count": failed, "error_summary": truncateBatchText(summary), "finished_at": now}
return service.DB.Model(&models.ShopeeSpecAutoMatchRun{}).Where("id = ? AND status = ? AND lease_owner = ?", run.ID, "running", run.LeaseOwner).Updates(updates).Error
}
func batchErrorCode(err error) string {
var serviceErr *ServiceError
if errors.As(err, &serviceErr) {
return serviceErr.Code
}
return CodeInternal
}
func safeBatchError(err error) string {
var serviceErr *ServiceError
if errors.As(err, &serviceErr) {
return truncateBatchText(serviceErr.Message)
}
return "服务端处理失败"
}
func truncateBatchText(value string) string {
runes := []rune(strings.TrimSpace(value))
if len(runes) > 500 {
runes = runes[:500]
}
return string(runes)
}
type scheduledAutoMatchArgs struct {
BatchLimit int `json:"batchLimit"`
}
type ScheduledAutoMatchJob struct{}
func (ScheduledAutoMatchJob) Exec(_ interface{}) error {
return errors.New("规格自动匹配定时任务缺少数据库连接")
}
func (ScheduledAutoMatchJob) ExecWithDB(db *gorm.DB, arg interface{}) error {
args := scheduledAutoMatchArgs{BatchLimit: defaultAutoMatchBatchLimit}
if raw, ok := arg.(string); ok && strings.TrimSpace(raw) != "" {
if err := json.Unmarshal([]byte(raw), &args); err != nil {
return fmt.Errorf("规格自动匹配参数不是合法 JSON: %w", err)
}
}
if args.BatchLimit < 1 || args.BatchLimit > 100 {
return errors.New("规格自动匹配 batchLimit 必须在 1 到 100 之间")
}
service := NewService(db)
run, created, err := service.StartAutoMatchRun(context.Background(), "scheduled", uuid.NewString(), nil, args.BatchLimit)
if err != nil || !created {
return err
}
return service.ProcessAutoMatchRun(context.Background(), run.ID)
}
@@ -0,0 +1,86 @@
package shopeeproduct
import (
"context"
"testing"
"go-admin/app/goauto/models"
"github.com/google/uuid"
)
func TestAutoMatchRunIsIdempotentAndGloballySerialized(t *testing.T) {
db := openTestDB(t)
service := NewService(db)
requestID := uuid.NewString()
first, created, err := service.StartAutoMatchRun(context.Background(), "manual", requestID, nil, 20)
if err != nil || !created {
t.Fatalf("first=%+v created=%v err=%v", first, created, err)
}
replay, created, err := service.StartAutoMatchRun(context.Background(), "manual", requestID, nil, 20)
if err != nil || created || !replay.Replayed || replay.ID != first.ID {
t.Fatalf("replay=%+v created=%v err=%v", replay, created, err)
}
concurrent, created, err := service.StartAutoMatchRun(context.Background(), "scheduled", uuid.NewString(), nil, 20)
if err != nil || created || !concurrent.AlreadyRunning || concurrent.ID != first.ID {
t.Fatalf("concurrent=%+v created=%v err=%v", concurrent, created, err)
}
}
func TestProcessAutoMatchRunConfirmsExactSizeAndFinishes(t *testing.T) {
db := openTestDB(t)
pdd := seedPDDProduct(t, db, "active")
service := NewService(db)
createdProduct, err := service.Create(context.Background(), CreateRequest{
RequestID: uuid.NewString(), ShopeeItemID: "SP-BATCH-EXACT", PDDProductID: &pdd.ID,
Specs: []SpecDimension{{Name: "尺码", Role: RoleSize, Values: []SpecValue{{Name: " xl ", Source: ValueSourceImport}}}},
})
if err != nil {
t.Fatal(err)
}
run, started, err := service.StartAutoMatchRun(context.Background(), "manual", uuid.NewString(), nil, 20)
if err != nil || !started {
t.Fatalf("run=%+v started=%v err=%v", run, started, err)
}
if err := service.ProcessAutoMatchRun(context.Background(), run.ID); err != nil {
t.Fatal(err)
}
latest, err := service.LatestAutoMatchRun(context.Background())
if err != nil {
t.Fatal(err)
}
if latest == nil || latest.Status != "completed" || latest.ProcessedCount != 1 || latest.ConfirmedCount != 1 || latest.ActiveSlot != nil {
t.Fatalf("latest=%+v", latest)
}
detail, err := service.Detail(context.Background(), createdProduct.Product.ID)
if err != nil {
t.Fatal(err)
}
mapping := detail.Product.Specs[0].Values[0].Mapping
if mapping == nil || mapping.Status != MappingStatusConfirmed || mapping.PDDValue != "XL" {
t.Fatalf("mapping=%+v", mapping)
}
}
func TestUnchangedUnmatchedWorkIsNotClaimedAgain(t *testing.T) {
db := openTestDB(t)
service := NewService(db)
one := uint8(1)
run := models.ShopeeSpecAutoMatchRun{RequestID: uuid.NewString(), Trigger: "manual", Status: "running", ActiveSlot: &one, LeaseOwner: uuid.NewString(), BatchLimit: 20}
if err := db.Create(&run).Error; err != nil {
t.Fatal(err)
}
work, claimed, err := service.claimAutoMatchWork(context.Background(), run, 99, "fingerprint")
if err != nil || !claimed {
t.Fatalf("work=%+v claimed=%v err=%v", work, claimed, err)
}
service.completeAutoMatchWork(work, "fingerprint", 0, 1, nil)
_, claimed, err = service.claimAutoMatchWork(context.Background(), run, 99, "fingerprint")
if err != nil || claimed {
t.Fatalf("unchanged unmatched claimed=%v err=%v", claimed, err)
}
_, claimed, err = service.claimAutoMatchWork(context.Background(), run, 99, "changed")
if err != nil || !claimed {
t.Fatalf("changed input claimed=%v err=%v", claimed, err)
}
}
@@ -5,11 +5,13 @@ import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"go-admin/app/goauto/models"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
)
@@ -116,6 +118,20 @@ func TestAutoMatchMappingsProviderFailureDoesNotWrite(t *testing.T) {
}
}
func TestWriteErrorMapsAIUnavailableToStructured503(t *testing.T) {
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
context, _ := gin.CreateTestContext(recorder)
writeError(context, aiUnavailable("AI 匹配服务暂时不可用,请稍后重试"))
if recorder.Code != http.StatusServiceUnavailable {
t.Fatalf("status = %d, body = %s", recorder.Code, recorder.Body.String())
}
if !strings.Contains(recorder.Body.String(), `"code":"AI_MATCHING_UNAVAILABLE"`) {
t.Fatalf("response is not structured: %s", recorder.Body.String())
}
}
func TestAutoMatchMappingsRejectsContextDriftBeforeAtomicWrite(t *testing.T) {
db := openTestDB(t)
pdd := seedPDDProduct(t, db, "active")
@@ -7,8 +7,7 @@ import (
// ResolvedMappingItem is one mapping produced by the narrowly scoped SYB
// batch-match entry point. Confirmed writes are restricted to auditable exact
// matches and AI decisions with a returned match reason; SetMapping remains
// pending-first.
// matches and high-confidence AI decisions; SetMapping remains pending-first.
type ResolvedMappingItem struct {
Dimension string
ValueName string
@@ -21,11 +20,14 @@ type ResolvedMappingItem struct {
// ApplyResolvedMappings atomically applies the color/size mappings needed by
// one SYB detail row. It is independent from SetMapping so #188's explicit
// confirmed-match exception cannot change existing callers' pending semantics.
func (service *Service) ApplyResolvedMappings(ctx context.Context, id uint64, requestID string, items []ResolvedMappingItem) (SaveResponse, error) {
// high-confidence exception cannot change existing callers' pending semantics.
func (service *Service) ApplyResolvedMappings(ctx context.Context, id uint64, requestID string, minimumConfidence float64, items []ResolvedMappingItem) (SaveResponse, error) {
if len(items) == 0 || len(items) > 2 {
return SaveResponse{}, invalidRequest("必须包含 1 至 2 个待写入规格映射")
}
if minimumConfidence < 0 || minimumConfidence > 1 {
return SaveResponse{}, invalidRequest("自动确认阈值无效")
}
seen := make(map[string]bool, len(items))
for _, item := range items {
key := strings.TrimSpace(item.Dimension) + "\x00" + strings.TrimSpace(item.ValueName)
@@ -39,6 +41,9 @@ func (service *Service) ApplyResolvedMappings(ctx context.Context, id uint64, re
if item.Status != MappingStatusConfirmed || strings.TrimSpace(item.Reason) == "" {
return SaveResponse{}, invalidRequest("自动确认映射必须包含确认状态和匹配理由")
}
if item.Source == MappingSourceAIMatch && (item.Confidence == nil || *item.Confidence < minimumConfidence) {
return SaveResponse{}, invalidRequest("AI 自动确认必须达到置信度阈值")
}
if err := service.validatePDDMappingTarget(ctx, id, item.Dimension, item.ValueName, strings.TrimSpace(item.PDDValue)); err != nil {
return SaveResponse{}, err
}
@@ -1,6 +1,7 @@
package shopeeproduct
import (
"context"
"encoding/json"
"errors"
"io"
@@ -300,6 +301,45 @@ func (handler Handler) AutoMatchMappings(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"code": 200, "data": response})
}
func (handler Handler) StartAutoMatchRun(c *gin.Context) {
var request struct {
RequestID string `json:"requestId"`
}
if err := decodeJSON(c, &request); err != nil {
writeError(c, invalidRequest("请求 JSON 无效"))
return
}
service, ok := handler.service(c)
if !ok {
return
}
operator := currentUserID(c)
run, created, err := service.StartAutoMatchRun(c.Request.Context(), "manual", request.RequestID, &operator, defaultAutoMatchBatchLimit)
if err != nil {
writeError(c, err)
return
}
if created {
go func(runID uint64, db *gorm.DB) {
_ = NewService(db).ProcessAutoMatchRun(context.Background(), runID)
}(run.ID, service.DB)
}
c.JSON(http.StatusAccepted, gin.H{"code": 200, "data": gin.H{"run": run}})
}
func (handler Handler) LatestAutoMatchRun(c *gin.Context) {
service, ok := handler.service(c)
if !ok {
return
}
run, err := service.LatestAutoMatchRun(c.Request.Context())
if err != nil {
writeError(c, err)
return
}
c.JSON(http.StatusOK, gin.H{"code": 200, "data": gin.H{"run": run}})
}
func (handler Handler) BatchDelete(c *gin.Context) {
var request BatchDeleteRequest
if err := decodeJSON(c, &request); err != nil {
@@ -10,6 +10,9 @@ import (
func InitRouter(engine *gin.Engine, auth *jwt.GinJWTMiddleware) {
handler := Handler{}
admin := engine.Group("/api/admin/v1/shopee-products").Use(auth.MiddlewareFunc()).Use(middleware.AuthCheckRole())
adminOnlyRuns := engine.Group("/api/admin/v1/shopee-spec-auto-match/runs").Use(auth.MiddlewareFunc()).Use(middleware.AuthCheckRole()).Use(middleware.RequireRoleKey("admin"))
adminOnlyRuns.POST("", handler.StartAutoMatchRun)
adminOnlyRuns.GET("/latest", handler.LatestAutoMatchRun)
admin.GET("", handler.List)
admin.POST("", handler.Create)
admin.POST("/batch-delete", handler.BatchDelete)
@@ -0,0 +1,489 @@
package sybimport
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"go-admin/app/goauto/aimatching"
"go-admin/app/goauto/models"
"go-admin/app/goauto/shopeeproduct"
"github.com/google/uuid"
"gorm.io/gorm"
)
const (
SpecAIParseInvokeTarget = "GoAutoSYBSpecAIParse"
defaultAIParseBatchLimit = 20
aiParseLeaseDuration = 30 * time.Minute
aiParseRetryDelay = time.Hour
maxAIParseAttempts = 3
)
var (
errAIParseWorkNotClaimed = errors.New("syb spec ai parse work not claimed")
errAIParseInputChanged = errors.New("syb spec ai parse input changed")
)
type aiParseInput struct {
ProductSpec string
Colors []string
Sizes []string
Fingerprint string
}
func StartSpecAIParseRun(ctx context.Context, db *gorm.DB, requestID string, batchLimit int) (models.SYBSpecAIParseRun, bool, error) {
if _, err := uuid.Parse(strings.TrimSpace(requestID)); err != nil {
return models.SYBSpecAIParseRun{}, false, fmt.Errorf("requestId 必须是 UUID")
}
if batchLimit <= 0 {
batchLimit = defaultAIParseBatchLimit
}
if batchLimit > 100 {
return models.SYBSpecAIParseRun{}, false, fmt.Errorf("batchLimit 不能超过 100")
}
now := time.Now().UTC()
lease := now.Add(aiParseLeaseDuration)
one := uint8(1)
owner := uuid.NewString()
var run models.SYBSpecAIParseRun
created := false
err := db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
if err := tx.Model(&models.SYBSpecAIParseRun{}).
Where("status = ? AND active_slot = ? AND lease_expires_at < ?", "running", 1, now).
Updates(map[string]any{"status": "failed", "active_slot": nil, "lease_owner": "", "lease_expires_at": nil, "error_summary": "上次运行租约过期,已安全释放", "finished_at": now}).Error; err != nil {
return err
}
if err := tx.Where("request_id = ?", requestID).First(&run).Error; err == nil {
return nil
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
return err
}
if err := tx.Where("status = ? AND active_slot = ?", "running", 1).First(&run).Error; err == nil {
return nil
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
return err
}
run = models.SYBSpecAIParseRun{
RequestID: requestID, Trigger: "scheduled", Status: "running", ActiveSlot: &one,
LeaseOwner: owner, LeaseExpiresAt: &lease, BatchLimit: batchLimit, StartedAt: now,
}
if err := tx.Create(&run).Error; err != nil {
return err
}
created = true
return nil
})
if err != nil {
if findErr := db.WithContext(ctx).Where("status = ? AND active_slot = ?", "running", 1).First(&run).Error; findErr == nil {
return run, false, nil
}
return models.SYBSpecAIParseRun{}, false, err
}
return run, created, nil
}
func ProcessSpecAIParseRun(ctx context.Context, db *gorm.DB, runID uint64) error {
var run models.SYBSpecAIParseRun
if err := db.WithContext(ctx).First(&run, runID).Error; err != nil {
return err
}
if run.Status != "running" || run.ActiveSlot == nil || *run.ActiveSlot != 1 {
return nil
}
limit := run.BatchLimit
if limit <= 0 || limit > 100 {
limit = defaultAIParseBatchLimit
}
queryLimit := limit * 25
if queryLimit < 100 {
queryLimit = 100
}
if queryLimit > 1000 {
queryLimit = 1000
}
var candidates []models.SYBProduct
if err := db.WithContext(ctx).
Where("manually_confirmed = ?", false).
Where("parse_status IN ?", []string{models.SYBParseStatusUncertain, models.SYBParseStatusFailed}).
Order("updated_at ASC, id ASC").Limit(queryLimit).Find(&candidates).Error; err != nil {
finishSpecAIParseRun(db, run, "failed", 0, 0, 0, 0, 0, 1, "扫描异常规格失败")
return err
}
eligible, processed, confirmed, unmatched, failed := 0, 0, 0, 0, 0
firstError := ""
for _, candidate := range candidates {
if processed >= limit {
break
}
if candidate.AIConfirmed {
current, currentErr := aiConfirmationTargetsCurrent(ctx, db, candidate)
if currentErr != nil {
failed++
continue
}
if current {
continue
}
if err := db.WithContext(ctx).Model(&models.SYBProduct{}).
Where("id = ? AND manually_confirmed = ?", candidate.ID, false).
Updates(map[string]any{"ai_confirmed": false, "ai_confidence": nil, "ai_reason": "", "ai_confirmed_at": nil, "ai_input_fingerprint": ""}).Error; err != nil {
failed++
continue
}
candidate.AIConfirmed = false
}
outcome, err := Reparse(ctx, db, candidate.ID, false)
if err != nil {
failed++
if firstError == "" {
firstError = "确定性重新解析失败"
}
continue
}
if outcome.NewStatus == models.SYBParseStatusSuccess {
processed++
confirmed++
continue
}
if err := db.WithContext(ctx).First(&candidate, candidate.ID).Error; err != nil {
failed++
continue
}
input, ok, err := buildAIParseInput(ctx, db, candidate)
if err != nil {
failed++
if firstError == "" {
firstError = "读取 AI 解析上下文失败"
}
continue
}
if !ok {
continue
}
eligible++
work, claimed, err := claimSpecAIParseWork(ctx, db, run, candidate.ID, input.Fingerprint)
if err != nil {
failed++
continue
}
if !claimed {
continue
}
processed++
renewSpecAIParseRun(db, run)
matcher := aimatching.NewService(db)
result, matchErr := matcher.ResolveSYBSpec(ctx, aimatching.SYBSpecParseRequest{
ProductSpec: input.ProductSpec, Colors: input.Colors, Sizes: input.Sizes,
})
if matchErr == nil {
settings, settingsErr := matcher.Settings(ctx)
if settingsErr != nil {
matchErr = settingsErr
} else if result.Confidence == nil || *result.Confidence < settings.AutoConfirmMinConfidence || strings.TrimSpace(result.Reason) == "" {
unmatched++
completeSpecAIParseWork(db, work, input.Fingerprint, false, nil)
continue
} else {
matchErr = applyAIParseResult(ctx, db, candidate.ID, input.Fingerprint, result)
}
}
if matchErr != nil {
if isNoAIParseMatch(matchErr) || errors.Is(matchErr, errAIParseInputChanged) {
unmatched++
completeSpecAIParseWork(db, work, input.Fingerprint, false, nil)
continue
}
failed++
if firstError == "" {
firstError = safeAIParseError(matchErr)
}
completeSpecAIParseWork(db, work, input.Fingerprint, false, matchErr)
continue
}
confirmed++
completeSpecAIParseWork(db, work, input.Fingerprint, true, nil)
}
status := "completed"
if failed > 0 {
status = "completed_partial"
}
return finishSpecAIParseRun(db, run, status, len(candidates), eligible, processed, confirmed, unmatched, failed, firstError)
}
func buildAIParseInput(ctx context.Context, db *gorm.DB, record models.SYBProduct) (aiParseInput, bool, error) {
if record.ManuallyConfirmed || (record.ParseStatus != models.SYBParseStatusUncertain && record.ParseStatus != models.SYBParseStatusFailed) || record.ShopeeProductID == nil {
return aiParseInput{}, false, nil
}
var raw rawDetailSpec
if err := json.Unmarshal([]byte(record.RawJSON), &raw); err != nil {
return aiParseInput{}, false, err
}
raw.ProductSpec = strings.TrimSpace(raw.ProductSpec)
if raw.ProductSpec == "" {
return aiParseInput{}, false, nil
}
var product models.ShopeeProduct
if err := db.WithContext(ctx).First(&product, *record.ShopeeProductID).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return aiParseInput{}, false, nil
}
return aiParseInput{}, false, err
}
specs, err := shopeeproduct.Unmarshal(product.SpecsJSON)
if err != nil {
return aiParseInput{}, false, err
}
colors, sizes, ambiguous := closedShopeeCandidates(specs)
if ambiguous || (len(colors) == 0 && len(sizes) == 0) {
return aiParseInput{}, false, nil
}
var setting models.AIMatchingSetting
if err := db.WithContext(ctx).First(&setting, 1).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return aiParseInput{}, false, nil
}
return aiParseInput{}, false, err
}
if !setting.Enabled || strings.TrimSpace(setting.APIKey) == "" {
return aiParseInput{}, false, nil
}
fingerprintPayload := struct {
ProductSpec string
ShopeeProductID uint64
ShopeeSpecsJSON string
SettingUpdatedAt string
}{raw.ProductSpec, product.ID, product.SpecsJSON, setting.UpdatedAt.UTC().Format(time.RFC3339Nano)}
encoded, _ := json.Marshal(fingerprintPayload)
hash := sha256.Sum256(encoded)
return aiParseInput{ProductSpec: raw.ProductSpec, Colors: colors, Sizes: sizes, Fingerprint: hex.EncodeToString(hash[:])}, true, nil
}
func aiConfirmationTargetsCurrent(ctx context.Context, db *gorm.DB, record models.SYBProduct) (bool, error) {
if !record.AIConfirmed || record.ShopeeProductID == nil {
return false, nil
}
var product models.ShopeeProduct
if err := db.WithContext(ctx).First(&product, *record.ShopeeProductID).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return false, nil
}
return false, err
}
specs, err := shopeeproduct.Unmarshal(product.SpecsJSON)
if err != nil {
return false, err
}
colors, sizes, ambiguous := closedShopeeCandidates(specs)
if ambiguous {
return false, nil
}
return closedCandidateContains(record.TargetColor, colors) && closedCandidateContains(record.TargetSize, sizes) &&
(strings.TrimSpace(record.TargetColor) != "" || strings.TrimSpace(record.TargetSize) != ""), nil
}
func closedCandidateContains(value string, candidates []string) bool {
value = strings.TrimSpace(value)
if len(candidates) == 0 {
return value == ""
}
for _, candidate := range candidates {
if candidate == value {
return true
}
}
return false
}
func closedShopeeCandidates(specs []shopeeproduct.SpecDimension) (colors, sizes []string, ambiguous bool) {
roleDimensions := map[string]int{}
for _, dimension := range specs {
if dimension.Role != shopeeproduct.RoleColor && dimension.Role != shopeeproduct.RoleSize {
continue
}
values := make([]string, 0, len(dimension.Values))
seen := map[string]bool{}
for _, value := range dimension.Values {
name := strings.TrimSpace(value.Name)
if name != "" && !seen[name] {
seen[name] = true
values = append(values, name)
}
}
if len(values) == 0 {
continue
}
roleDimensions[dimension.Role]++
if roleDimensions[dimension.Role] > 1 {
return nil, nil, true
}
if dimension.Role == shopeeproduct.RoleColor {
colors = values
} else {
sizes = values
}
}
return colors, sizes, false
}
func applyAIParseResult(ctx context.Context, db *gorm.DB, id uint64, fingerprint string, result aimatching.SYBSpecParseResult) error {
now := time.Now().UTC()
return db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
var record models.SYBProduct
if err := tx.First(&record, id).Error; err != nil {
return err
}
input, ok, err := buildAIParseInput(ctx, tx, record)
if err != nil {
return err
}
if !ok || input.Fingerprint != fingerprint {
return errAIParseInputChanged
}
if result.Confidence == nil || strings.TrimSpace(result.Reason) == "" {
return errAIParseInputChanged
}
write := tx.Model(&models.SYBProduct{}).Where("id = ? AND manually_confirmed = ? AND ai_confirmed = ?", id, false, false).Updates(map[string]any{
"target_color": result.Color, "target_size": result.Size,
"ai_confirmed": true, "ai_confidence": *result.Confidence, "ai_reason": truncateAIParseText(result.Reason),
"ai_confirmed_at": now, "ai_input_fingerprint": fingerprint,
})
if write.Error != nil {
return write.Error
}
if write.RowsAffected != 1 {
return errAIParseInputChanged
}
return mergeParsedSpec(tx, *record.ShopeeProductID, ParseResult{Color: result.Color, Size: result.Size, Status: models.SYBParseStatusSuccess})
})
}
func claimSpecAIParseWork(ctx context.Context, db *gorm.DB, run models.SYBSpecAIParseRun, productID uint64, fingerprint string) (models.SYBSpecAIParseWorkItem, bool, error) {
now := time.Now().UTC()
lease := now.Add(aiParseLeaseDuration)
var work models.SYBSpecAIParseWorkItem
err := db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
err := tx.Where("syb_product_id = ?", productID).First(&work).Error
if errors.Is(err, gorm.ErrRecordNotFound) {
work = models.SYBSpecAIParseWorkItem{SYBProductID: productID, RunID: &run.ID, InputFingerprint: fingerprint, Status: "running", AttemptCount: 1, LeaseOwner: run.LeaseOwner, LeaseExpiresAt: &lease}
return tx.Create(&work).Error
}
if err != nil {
return err
}
if work.InputFingerprint == fingerprint {
if work.Status == "completed" || work.Status == "unmatched" || work.AttemptCount >= maxAIParseAttempts || (work.NextAttemptAt != nil && work.NextAttemptAt.After(now)) || (work.Status == "running" && work.LeaseExpiresAt != nil && work.LeaseExpiresAt.After(now)) {
return errAIParseWorkNotClaimed
}
} else {
work.AttemptCount = 0
}
updates := map[string]any{"run_id": run.ID, "input_fingerprint": fingerprint, "status": "running", "attempt_count": work.AttemptCount + 1, "next_attempt_at": nil, "lease_owner": run.LeaseOwner, "lease_expires_at": lease, "last_error_code": "", "last_error": ""}
if err := tx.Model(&models.SYBSpecAIParseWorkItem{}).Where("id = ?", work.ID).Updates(updates).Error; err != nil {
return err
}
return tx.First(&work, work.ID).Error
})
if errors.Is(err, errAIParseWorkNotClaimed) {
return work, false, nil
}
return work, err == nil, err
}
func completeSpecAIParseWork(db *gorm.DB, work models.SYBSpecAIParseWorkItem, fingerprint string, confirmed bool, parseErr error) {
now := time.Now().UTC()
updates := map[string]any{"input_fingerprint": fingerprint, "lease_owner": "", "lease_expires_at": nil}
if parseErr == nil {
if confirmed {
updates["status"] = "completed"
} else {
updates["status"] = "unmatched"
}
updates["next_attempt_at"], updates["last_error_code"], updates["last_error"] = nil, "", ""
} else {
code := aiParseErrorCode(parseErr)
updates["status"], updates["last_error_code"], updates["last_error"] = "failed", code, safeAIParseError(parseErr)
if code == aimatching.CodeProviderUnavailable && work.AttemptCount < maxAIParseAttempts {
next := now.Add(aiParseRetryDelay)
updates["next_attempt_at"] = next
} else {
updates["next_attempt_at"] = nil
}
}
_ = db.Model(&models.SYBSpecAIParseWorkItem{}).Where("id = ?", work.ID).Updates(updates).Error
}
func renewSpecAIParseRun(db *gorm.DB, run models.SYBSpecAIParseRun) {
lease := time.Now().UTC().Add(aiParseLeaseDuration)
_ = db.Model(&models.SYBSpecAIParseRun{}).Where("id = ? AND status = ? AND lease_owner = ?", run.ID, "running", run.LeaseOwner).Update("lease_expires_at", lease).Error
}
func finishSpecAIParseRun(db *gorm.DB, run models.SYBSpecAIParseRun, status string, scanned, eligible, processed, confirmed, unmatched, failed int, summary string) error {
now := time.Now().UTC()
return db.Model(&models.SYBSpecAIParseRun{}).Where("id = ? AND status = ? AND lease_owner = ?", run.ID, "running", run.LeaseOwner).Updates(map[string]any{
"status": status, "active_slot": nil, "lease_owner": "", "lease_expires_at": nil,
"scanned_count": scanned, "eligible_count": eligible, "processed_count": processed,
"confirmed_count": confirmed, "unmatched_count": unmatched, "failed_count": failed,
"error_summary": truncateAIParseText(summary), "finished_at": now,
}).Error
}
func isNoAIParseMatch(err error) bool { return aiParseErrorCode(err) == aimatching.CodeNoMatch }
func aiParseErrorCode(err error) string {
var target *aimatching.Error
if errors.As(err, &target) {
return target.Code
}
return CodeInternal
}
func safeAIParseError(err error) string {
var target *aimatching.Error
if errors.As(err, &target) {
return truncateAIParseText(target.Message)
}
return "服务端处理失败"
}
func truncateAIParseText(value string) string {
runes := []rune(strings.TrimSpace(value))
if len(runes) > 500 {
runes = runes[:500]
}
return string(runes)
}
type scheduledSpecAIParseArgs struct {
BatchLimit int `json:"batchLimit"`
}
type ScheduledSpecAIParseJob struct{}
func (ScheduledSpecAIParseJob) Exec(_ interface{}) error {
return errors.New("SYB 规格 AI 解析定时任务缺少数据库连接")
}
func (ScheduledSpecAIParseJob) ExecWithDB(db *gorm.DB, arg interface{}) error {
args := scheduledSpecAIParseArgs{BatchLimit: defaultAIParseBatchLimit}
if raw, ok := arg.(string); ok && strings.TrimSpace(raw) != "" {
if err := json.Unmarshal([]byte(raw), &args); err != nil {
return fmt.Errorf("SYB 规格 AI 解析参数不是合法 JSON: %w", err)
}
}
if args.BatchLimit < 1 || args.BatchLimit > 100 {
return errors.New("SYB 规格 AI 解析 batchLimit 必须在 1 到 100 之间")
}
run, created, err := StartSpecAIParseRun(context.Background(), db, uuid.NewString(), args.BatchLimit)
if err != nil || !created {
return err
}
return ProcessSpecAIParseRun(context.Background(), db, run.ID)
}
@@ -0,0 +1,204 @@
package sybimport_test
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
"go-admin/app/goauto/models"
"go-admin/app/goauto/shopeeproduct"
"go-admin/app/goauto/sybimport"
"github.com/google/uuid"
"gorm.io/gorm"
)
func seedAIParseCandidate(t *testing.T, db *gorm.DB, serverURL string, detailID uint64, productSpec string) models.SYBProduct {
t.Helper()
detail := realDetailA()
detail.ID = detailID
detail.ProductSpec = productSpec
detail.Raw = []byte(fmt.Sprintf(`{"id":%d,"productSpec":%q}`, detail.ID, productSpec))
applied, err := sybimport.ApplyDetail(context.Background(), db, realOrder(), detail)
if err != nil {
t.Fatal(err)
}
specs, err := shopeeproduct.Marshal([]shopeeproduct.SpecDimension{
{Name: "颜色", Role: shopeeproduct.RoleColor, Values: []shopeeproduct.SpecValue{{Name: "黑色", Source: shopeeproduct.ValueSourceImport}, {Name: "白色", Source: shopeeproduct.ValueSourceImport}}},
{Name: "尺码", Role: shopeeproduct.RoleSize, Values: []shopeeproduct.SpecValue{{Name: "L", Source: shopeeproduct.ValueSourceImport}, {Name: "XL", Source: shopeeproduct.ValueSourceImport}}},
})
if err != nil {
t.Fatal(err)
}
if err := db.Model(&models.ShopeeProduct{}).Where("id = ?", *applied.SYBProduct.ShopeeProductID).Update("specs_json", specs).Error; err != nil {
t.Fatal(err)
}
setting := models.AIMatchingSetting{ID: 1, Enabled: true, Provider: "openai_compatible", BaseURL: serverURL, Model: "test-model", APIKey: "test-key", TimeoutSeconds: 5, AutoConfirmMinConfidence: 0.9}
if err := db.Save(&setting).Error; err != nil {
t.Fatal(err)
}
return applied.SYBProduct
}
func TestScheduledAIParseConfirmsClosedCandidatesAndDoesNotRepeat(t *testing.T) {
var calls atomic.Int32
provider := httptest.NewServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
calls.Add(1)
response.Header().Set("Content-Type", "application/json")
_, _ = response.Write([]byte(`{"choices":[{"message":{"content":"{\"color\":\"黑色\",\"size\":\"XL\",\"reason\":\"原文对应唯一候选\",\"confidence\":0.95}"}}]}`))
}))
defer provider.Close()
db := openTestDB(t)
record := seedAIParseCandidate(t, db, provider.URL, 19801, "黑色 XL")
rawBefore := record.RawJSON
run, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
if err != nil || !created {
t.Fatalf("start run: created=%v err=%v", created, err)
}
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, run.ID); err != nil {
t.Fatal(err)
}
if err := db.First(&record, record.ID).Error; err != nil {
t.Fatal(err)
}
if !record.AIConfirmed || record.ManuallyConfirmed || record.ParseStatus != models.SYBParseStatusUncertain || record.TargetColor != "黑色" || record.TargetSize != "XL" || record.AIConfidence == nil || *record.AIConfidence != 0.95 || record.AIReason == "" || record.AIInputFingerprint == "" {
t.Fatalf("unexpected confirmed record: %+v", record)
}
if record.RawJSON != rawBefore {
t.Fatal("AI confirmation must not rewrite RawJSON")
}
var finished models.SYBSpecAIParseRun
if err := db.First(&finished, run.ID).Error; err != nil {
t.Fatal(err)
}
if finished.Status != "completed" || finished.ConfirmedCount != 1 || finished.ProcessedCount != 1 {
t.Fatalf("unexpected run: %+v", finished)
}
second, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
if err != nil || !created {
t.Fatalf("second start: created=%v err=%v", created, err)
}
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, second.ID); err != nil {
t.Fatal(err)
}
if calls.Load() != 1 {
t.Fatalf("unchanged confirmed input called provider %d times", calls.Load())
}
}
func TestScheduledAIParseLeavesLowConfidenceUnmatchedForSameFingerprint(t *testing.T) {
var calls atomic.Int32
provider := httptest.NewServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
calls.Add(1)
_, _ = response.Write([]byte(`{"choices":[{"message":{"content":"{\"color\":\"黑色\",\"size\":\"XL\",\"reason\":\"仍有歧义\",\"confidence\":0.4}"}}]}`))
}))
defer provider.Close()
db := openTestDB(t)
record := seedAIParseCandidate(t, db, provider.URL, 19802, "黑色 XL")
for i := 0; i < 2; i++ {
run, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
if err != nil || !created {
t.Fatalf("start %d: created=%v err=%v", i, created, err)
}
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, run.ID); err != nil {
t.Fatal(err)
}
}
if err := db.First(&record, record.ID).Error; err != nil {
t.Fatal(err)
}
if record.AIConfirmed || calls.Load() != 1 {
t.Fatalf("low confidence must remain unconfirmed and not repeat: confirmed=%v calls=%d", record.AIConfirmed, calls.Load())
}
var work models.SYBSpecAIParseWorkItem
if err := db.Where("syb_product_id = ?", record.ID).First(&work).Error; err != nil {
t.Fatal(err)
}
if work.Status != "unmatched" {
t.Fatalf("work status=%s", work.Status)
}
}
func TestScheduledAIParseSkipsEmptySourceAndManualConfirmation(t *testing.T) {
var calls atomic.Int32
provider := httptest.NewServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
calls.Add(1)
response.WriteHeader(http.StatusInternalServerError)
}))
defer provider.Close()
db := openTestDB(t)
empty := seedAIParseCandidate(t, db, provider.URL, 19803, "")
manual := seedAIParseCandidate(t, db, provider.URL, 19804, "黑色 XL")
if _, err := sybimport.ManualCorrect(context.Background(), db, manual.ID, "黑色", "XL"); err != nil {
t.Fatal(err)
}
run, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
if err != nil || !created {
t.Fatalf("start: created=%v err=%v", created, err)
}
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, run.ID); err != nil {
t.Fatal(err)
}
if calls.Load() != 0 {
t.Fatalf("ineligible rows called provider %d times", calls.Load())
}
var workCount int64
if err := db.Model(&models.SYBSpecAIParseWorkItem{}).Where("syb_product_id IN ?", []uint64{empty.ID, manual.ID}).Count(&workCount).Error; err != nil {
t.Fatal(err)
}
if workCount != 0 {
t.Fatalf("ineligible rows created %d work items", workCount)
}
}
func TestSpecAIParseRunHasSingleGlobalActiveSlot(t *testing.T) {
db := openTestDB(t)
first, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
if err != nil || !created {
t.Fatalf("first: created=%v err=%v", created, err)
}
second, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
if err != nil || created || second.ID != first.ID {
t.Fatalf("second must reuse active run: first=%d second=%d created=%v err=%v", first.ID, second.ID, created, err)
}
}
func TestScheduledAIParseRetriesProviderFailureAtMostThreeTimes(t *testing.T) {
var calls atomic.Int32
provider := httptest.NewServer(http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
calls.Add(1)
response.WriteHeader(http.StatusBadGateway)
}))
defer provider.Close()
db := openTestDB(t)
record := seedAIParseCandidate(t, db, provider.URL, 19805, "黑色 XL")
for attempt := 1; attempt <= 4; attempt++ {
run, created, err := sybimport.StartSpecAIParseRun(context.Background(), db, uuid.NewString(), 20)
if err != nil || !created {
t.Fatalf("start %d: created=%v err=%v", attempt, created, err)
}
if err := sybimport.ProcessSpecAIParseRun(context.Background(), db, run.ID); err != nil {
t.Fatal(err)
}
if attempt < 3 {
past := time.Now().UTC().Add(-time.Minute)
if err := db.Model(&models.SYBSpecAIParseWorkItem{}).Where("syb_product_id = ?", record.ID).Update("next_attempt_at", past).Error; err != nil {
t.Fatal(err)
}
}
}
if calls.Load() != 3 {
t.Fatalf("provider calls=%d, want 3", calls.Load())
}
var work models.SYBSpecAIParseWorkItem
if err := db.Where("syb_product_id = ?", record.ID).First(&work).Error; err != nil {
t.Fatal(err)
}
if work.AttemptCount != 3 || work.NextAttemptAt != nil {
t.Fatalf("retry state=%+v", work)
}
}
+33 -2
View File
@@ -130,13 +130,37 @@ func ApplyDetail(ctx context.Context, db *gorm.DB, order OrderInput, detail Deta
result.Outcome = OutcomeCreated
case err == nil:
record.ID = existing.ID
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", existing.ID).Updates(map[string]any{
// Human-confirmed target values are authoritative and survive every
// source re-import. ParseStatus/ParseNote below still record what the
// current deterministic parser observed for audit.
if existing.ManuallyConfirmed {
record.TargetColor, record.TargetSize = existing.TargetColor, existing.TargetSize
record.ManuallyConfirmed = true
}
updates := map[string]any{
"stock_id": record.StockID, "shop_name": record.ShopName, "shopee_item_id": record.ShopeeItemID,
"shopee_product_id": record.ShopeeProductID, "product_title": record.ProductTitle,
"target_color": record.TargetColor, "target_size": record.TargetSize,
"quantity": record.Quantity, "unit_price_cent": record.UnitPriceCent, "image_url": record.ImageURL,
"parse_status": record.ParseStatus, "parse_note": record.ParseNote, "raw_json": record.RawJSON,
}).Error; err != nil {
}
// An identical re-import keeps a valid AI decision. Changed source,
// link, or a newly deterministic parse invalidates it atomically.
preserveAI := existing.AIConfirmed && !existing.ManuallyConfirmed && parsed.Status != models.SYBParseStatusSuccess &&
existing.RawJSON == record.RawJSON && sameOptionalID(existing.ShopeeProductID, record.ShopeeProductID)
if preserveAI {
record.TargetColor, record.TargetSize = existing.TargetColor, existing.TargetSize
updates["target_color"], updates["target_size"] = existing.TargetColor, existing.TargetSize
record.AIConfirmed, record.AIConfidence, record.AIReason = true, existing.AIConfidence, existing.AIReason
record.AIConfirmedAt, record.AIInputFingerprint = existing.AIConfirmedAt, existing.AIInputFingerprint
} else {
updates["ai_confirmed"], updates["ai_confidence"], updates["ai_reason"] = false, nil, ""
updates["ai_confirmed_at"], updates["ai_input_fingerprint"] = nil, ""
}
if existing.ManuallyConfirmed {
updates["target_color"], updates["target_size"] = existing.TargetColor, existing.TargetSize
}
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", existing.ID).Updates(updates).Error; err != nil {
return err
}
result.Outcome = OutcomeUpdated
@@ -161,6 +185,13 @@ func ApplyDetail(ctx context.Context, db *gorm.DB, order OrderInput, detail Deta
return result, nil
}
func sameOptionalID(left, right *uint64) bool {
if left == nil || right == nil {
return left == nil && right == nil
}
return *left == *right
}
// findOrCreateShopeeProduct implements #40's revival rule: a live match wins,
// a soft-deleted match is revived (keeping its prior mapping), and only when
// neither exists does the import create a minimal archive. On an existing
+4 -13
View File
@@ -85,8 +85,7 @@ func StartImport(ctx context.Context, db *gorm.DB, request ImportRequest, actor
}
run, err := NewSyncRunService(db).Create(ctx, CreateSyncRunInput{
DateFrom: request.DateFrom, DateTo: request.DateTo, ShopFilterHash: enabledShopHash(enabled),
ShopFilterSnapshot: cloneShopSnapshot(enabled),
OperatorID: actor.ID, OperatorName: strings.TrimSpace(actor.Name),
OperatorID: actor.ID, OperatorName: strings.TrimSpace(actor.Name),
})
if err != nil {
importGate.Unlock()
@@ -98,7 +97,7 @@ func StartImport(ctx context.Context, db *gorm.DB, request ImportRequest, actor
importGate.running = true
importGate.Unlock()
go runImport(db, run.ID, request, settings, cloneShopSnapshot(enabled))
go runImport(db, run.ID, request, settings)
return StartImportResult{RunID: run.ID, Status: run.Status}, nil
}
@@ -146,7 +145,7 @@ func (handler Handler) Import(c *gin.Context) {
c.JSON(http.StatusAccepted, gin.H{"code": 200, "data": gin.H{"runId": result.RunID, "status": result.Status}})
}
func runImport(db *gorm.DB, runID uint64, request ImportRequest, settings config.SYB, allowed map[string]string) {
func runImport(db *gorm.DB, runID uint64, request ImportRequest, settings config.SYB) {
defer func() { importGate.Lock(); importGate.running = false; importGate.Unlock() }()
ctx, cancel := context.WithTimeout(context.Background(), importTimeout)
defer cancel()
@@ -157,7 +156,7 @@ func runImport(db *gorm.DB, runID uint64, request ImportRequest, settings config
OcrURL: settings.OcrURL, OcrMaxAttempts: settings.OcrMaxAttempts,
})
if err == nil {
report, err = SyncWithShopSnapshot(ctx, db, client, SyncConfig{PageSize: settings.PageSize, MaxMatches: settings.MaxMatches}, request.DateFrom, request.DateTo, allowed,
report, err = SyncWithProgress(ctx, db, client, SyncConfig{PageSize: settings.PageSize, MaxMatches: settings.MaxMatches}, request.DateFrom, request.DateTo,
func(progress SyncProgress) error { return runs.UpdateProgress(ctx, runID, progress) })
}
status := SyncRunSucceeded
@@ -169,14 +168,6 @@ func runImport(db *gorm.DB, runID uint64, request ImportRequest, settings config
_ = runs.Finish(finishCtx, runID, status, report, err)
}
func cloneShopSnapshot(source map[string]string) map[string]string {
cloned := make(map[string]string, len(source))
for normalized, display := range source {
cloned[normalized] = display
}
return cloned
}
func claimUint64(value any) uint64 {
switch typed := value.(type) {
case float64:
+7 -26
View File
@@ -13,13 +13,6 @@ import (
var bracketPattern = regexp.MustCompile(`【[^】]*】`)
// explicitSizePattern recognizes only values whose spelling carries a strong
// size signal. SYB has now been observed returning both "color,size" and
// "size,color". A color dictionary would inevitably guess at product-specific
// labels, so role reversal is allowed only when exactly one side matches this
// deliberately narrow pattern.
var explicitSizePattern = regexp.MustCompile(`(?i)^(?:均(?:码|碼|号|號)|one\s*size|free\s*size|x{0,4}[sml]|[2-9]xl|(?:加大|大|中|小)(?:码|碼|号|號)|\d+(?:\.\d+)?(?:cm|mm|m|码|碼|号|號|公分)|\d+(?:\.\d+)?(?:[-~~至到]\d+(?:\.\d+)?)?(?:斤|公斤|千克|kg))$`)
// ambiguousPattern flags leftover separators or multi-token noise after
// bracket stripping — the signal that a "clean" split still isn't reliable.
// Observed in the real SYB sample: "黑色+白色【純棉兩件裝】 簡約親膚" strips
@@ -36,8 +29,8 @@ type ParseResult struct {
Note string
}
// Parse implements the #41/#216 rule: split on the last comma, strip 【...】
// annotations, and classify the result. It never guesses a
// Parse implements the #41 rule: split on the last comma into color/size,
// strip 【...】 annotations, and classify the result. It never guesses a
// missing value and never invents a color or size that is not literally
// present in the input.
//
@@ -48,13 +41,9 @@ type ParseResult struct {
// brackets stripped, becomes the size candidate; color stays empty. A
// single token with no separator cannot be split into two dimensions
// without guessing which one it is.
// - comma present and exactly one side has an explicit size signal -> that
// side is size and the other side is color. This supports both observed
// SYB orders without allowing AI or a fuzzy color dictionary to swap roles.
// - comma present and neither side has an explicit size signal -> retain the
// established SYB color,size contract for backward compatibility.
// - comma present and both sides have explicit size signals -> uncertain;
// there is no safe color decision.
// - comma present, last segment (size) and first segment(s) (color) both
// non-empty after stripping brackets, and the color candidate has no
// leftover separator/whitespace -> success.
// - comma present but either side is empty after stripping, or the color
// candidate still carries a leftover '+' or internal whitespace -> the
// split happened but is not trustworthy -> uncertain.
@@ -76,20 +65,12 @@ func Parse(raw string) ParseResult {
return ParseResult{Size: size, Status: models.SYBParseStatusUncertain, Note: "无分隔符,仅识别出单一规格值"}
}
firstPart := stripBrackets(trimmed[:lastComma])
secondPart := stripBrackets(trimmed[lastComma+1:])
colorPart, sizePart := firstPart, secondPart
colorPart := stripBrackets(trimmed[:lastComma])
sizePart := stripBrackets(trimmed[lastComma+1:])
if colorPart == "" || sizePart == "" {
return ParseResult{Color: colorPart, Size: sizePart, Status: models.SYBParseStatusUncertain, Note: "按逗号拆分后有一侧为空"}
}
firstIsSize, secondIsSize := explicitSizePattern.MatchString(firstPart), explicitSizePattern.MatchString(secondPart)
if firstIsSize && secondIsSize {
return ParseResult{Color: firstPart, Size: secondPart, Status: models.SYBParseStatusUncertain, Note: "逗号两侧均具有尺码特征,无法安全识别颜色"}
}
if firstIsSize {
colorPart, sizePart = secondPart, firstPart
}
if ambiguousPattern.MatchString(colorPart) {
return ParseResult{Color: colorPart, Size: sizePart, Status: models.SYBParseStatusUncertain, Note: "颜色部分含备注文本或多个分隔符,拆分结果可能不准确"}
}
-21
View File
@@ -19,27 +19,6 @@ func TestParseRealSample_CleanColorAndSize(t *testing.T) {
}
}
func TestParseReversedSizeAndColorWhenOnlyFirstHasExplicitSizeEvidence(t *testing.T) {
result := sybimport.Parse("均碼,黑色")
if result.Color != "黑色" || result.Size != "均碼" || result.Status != models.SYBParseStatusSuccess {
t.Fatalf("expected reversed role parse 黑色/均碼 success, got %+v", result)
}
}
func TestParseKeepsTraditionalColorAndSizeOrderWhenSecondHasSizeEvidence(t *testing.T) {
result := sybimport.Parse("黑色,均码")
if result.Color != "黑色" || result.Size != "均码" || result.Status != models.SYBParseStatusSuccess {
t.Fatalf("expected normal role parse 黑色/均码 success, got %+v", result)
}
}
func TestParseTwoExplicitSizesIsUncertainInsteadOfSwapping(t *testing.T) {
result := sybimport.Parse("M,L")
if result.Status != models.SYBParseStatusUncertain || result.Color != "M" || result.Size != "L" {
t.Fatalf("expected conflicting size evidence to stay uncertain, got %+v", result)
}
}
func TestParseRealSample_AmbiguousColorWithAnnotationAndSeparator(t *testing.T) {
result := sybimport.Parse("黑色+白色【純棉兩件裝】 簡約親膚,L【建議52.5-60公斤】")
if result.Status != models.SYBParseStatusUncertain {
+18 -3
View File
@@ -28,7 +28,7 @@ type rawDetailSpec struct {
// result page's per-line feedback.
type ReparseOutcome struct {
SYBProductID uint64 `json:"sybProductId"`
Outcome string `json:"outcome"` // reparsed | skipped_manual | unchanged
Outcome string `json:"outcome"` // reparsed | skipped_manual | skipped_ai | unchanged
OldStatus string `json:"oldStatus"`
NewStatus string `json:"newStatus"`
}
@@ -36,6 +36,7 @@ type ReparseOutcome struct {
const (
ReparseOutcomeReparsed = "reparsed"
ReparseOutcomeSkippedManual = "skipped_manual"
ReparseOutcomeSkippedAI = "skipped_ai"
ReparseOutcomeUnchanged = "unchanged"
)
@@ -65,6 +66,11 @@ func Reparse(ctx context.Context, db *gorm.DB, sybProductID uint64, force bool)
outcome.NewStatus = record.ParseStatus
return nil
}
if record.AIConfirmed && !force {
outcome.Outcome = ReparseOutcomeSkippedAI
outcome.NewStatus = record.ParseStatus
return nil
}
var raw rawDetailSpec
if err := json.Unmarshal([]byte(record.RawJSON), &raw); err != nil {
@@ -76,8 +82,11 @@ func Reparse(ctx context.Context, db *gorm.DB, sybProductID uint64, force bool)
if parsed.Color == record.TargetColor && parsed.Size == record.TargetSize && parsed.Status == record.ParseStatus {
outcome.Outcome = ReparseOutcomeUnchanged
if force {
record.ManuallyConfirmed = false
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", record.ID).Update("manually_confirmed", false).Error; err != nil {
record.ManuallyConfirmed, record.AIConfirmed = false, false
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", record.ID).Updates(map[string]any{
"manually_confirmed": false, "ai_confirmed": false, "ai_confidence": nil,
"ai_reason": "", "ai_confirmed_at": nil, "ai_input_fingerprint": "",
}).Error; err != nil {
return err
}
}
@@ -87,6 +96,8 @@ func Reparse(ctx context.Context, db *gorm.DB, sybProductID uint64, force bool)
updates := map[string]any{
"target_color": parsed.Color, "target_size": parsed.Size,
"parse_status": parsed.Status, "parse_note": parsed.Note, "manually_confirmed": false,
"ai_confirmed": false, "ai_confidence": nil, "ai_reason": "",
"ai_confirmed_at": nil, "ai_input_fingerprint": "",
}
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", record.ID).Updates(updates).Error; err != nil {
return err
@@ -140,8 +151,12 @@ func ManualCorrect(ctx context.Context, db *gorm.DB, sybProductID uint64, color,
return err
}
record.TargetColor, record.TargetSize, record.ManuallyConfirmed = color, size, true
record.AIConfirmed, record.AIConfidence, record.AIReason = false, nil, ""
record.AIConfirmedAt, record.AIInputFingerprint = nil, ""
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", sybProductID).Updates(map[string]any{
"target_color": color, "target_size": size, "manually_confirmed": true,
"ai_confirmed": false, "ai_confidence": nil, "ai_reason": "",
"ai_confirmed_at": nil, "ai_input_fingerprint": "",
}).Error; err != nil {
return err
}
+106
View File
@@ -60,6 +60,35 @@ func TestReparseSkipsManuallyConfirmedRowByDefault(t *testing.T) {
}
}
func TestReparseSkipsAIConfirmationUnlessForced(t *testing.T) {
db := openTestDB(t)
applied, err := sybimport.ApplyDetail(context.Background(), db, realOrder(), realDetailB())
if err != nil {
t.Fatal(err)
}
confidence := 0.95
if err := db.Model(&models.SYBProduct{}).Where("id = ?", applied.SYBProduct.ID).Updates(map[string]any{
"target_color": "AI颜色", "target_size": "AI尺码", "ai_confirmed": true,
"ai_confidence": confidence, "ai_reason": "AI 结果", "ai_input_fingerprint": strings.Repeat("c", 64),
}).Error; err != nil {
t.Fatal(err)
}
outcome, err := sybimport.Reparse(context.Background(), db, applied.SYBProduct.ID, false)
if err != nil || outcome.Outcome != sybimport.ReparseOutcomeSkippedAI {
t.Fatalf("unforced outcome=%+v err=%v", outcome, err)
}
if _, err := sybimport.Reparse(context.Background(), db, applied.SYBProduct.ID, true); err != nil {
t.Fatal(err)
}
var record models.SYBProduct
if err := db.First(&record, applied.SYBProduct.ID).Error; err != nil {
t.Fatal(err)
}
if record.AIConfirmed || record.AIConfidence != nil || record.AIReason != "" || record.AIInputFingerprint != "" {
t.Fatalf("forced reparse retained AI state: %+v", record)
}
}
// 可勾选强制覆盖.
func TestReparseWithForceOverridesManualCorrection(t *testing.T) {
db := openTestDB(t)
@@ -151,6 +180,83 @@ func TestManualCorrectMergesIntoArchiveLikeASuccessfulParse(t *testing.T) {
}
}
func TestManualCorrectSupersedesAIConfirmation(t *testing.T) {
db := openTestDB(t)
applied, err := sybimport.ApplyDetail(context.Background(), db, realOrder(), realDetailB())
if err != nil {
t.Fatal(err)
}
confidence := 0.96
if err := db.Model(&models.SYBProduct{}).Where("id = ?", applied.SYBProduct.ID).Updates(map[string]any{
"ai_confirmed": true, "ai_confidence": confidence, "ai_reason": "旧 AI 结果", "ai_input_fingerprint": strings.Repeat("a", 64),
}).Error; err != nil {
t.Fatal(err)
}
corrected, err := sybimport.ManualCorrect(context.Background(), db, applied.SYBProduct.ID, "人工颜色", "人工尺码")
if err != nil {
t.Fatal(err)
}
if !corrected.ManuallyConfirmed || corrected.AIConfirmed || corrected.AIConfidence != nil || corrected.AIReason != "" || corrected.AIInputFingerprint != "" {
t.Fatalf("human correction did not supersede AI state: %+v", corrected)
}
}
func TestReimportPreservesIdenticalAIInputAndInvalidatesChangedSource(t *testing.T) {
db := openTestDB(t)
order, detail := realOrder(), realDetailB()
first, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
if err != nil {
t.Fatal(err)
}
confidence := 0.95
if err := db.Model(&models.SYBProduct{}).Where("id = ?", first.SYBProduct.ID).Updates(map[string]any{
"target_color": "AI颜色", "target_size": "AI尺码", "ai_confirmed": true,
"ai_confidence": confidence, "ai_reason": "已确认", "ai_input_fingerprint": strings.Repeat("b", 64),
}).Error; err != nil {
t.Fatal(err)
}
same, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
if err != nil {
t.Fatal(err)
}
if !same.SYBProduct.AIConfirmed {
t.Fatal("identical re-import must preserve AI confirmation")
}
if same.SYBProduct.TargetColor != "AI颜色" || same.SYBProduct.TargetSize != "AI尺码" {
t.Fatal("identical re-import must preserve AI-confirmed target values")
}
detail.ProductSpec += " 新备注"
detail.Raw = []byte(`{"productSpec":"changed"}`)
changed, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
if err != nil {
t.Fatal(err)
}
if changed.SYBProduct.AIConfirmed || changed.SYBProduct.AIConfidence != nil || changed.SYBProduct.AIReason != "" {
t.Fatalf("changed source retained stale AI state: %+v", changed.SYBProduct)
}
}
func TestReimportNeverOverwritesManualTargetValues(t *testing.T) {
db := openTestDB(t)
order, detail := realOrder(), realDetailB()
first, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
if err != nil {
t.Fatal(err)
}
if _, err := sybimport.ManualCorrect(context.Background(), db, first.SYBProduct.ID, "人工颜色", "人工尺码"); err != nil {
t.Fatal(err)
}
detail.ProductSpec = "来源新颜色,来源新尺码"
detail.Raw = []byte(`{"productSpec":"来源新颜色,来源新尺码"}`)
updated, err := sybimport.ApplyDetail(context.Background(), db, order, detail)
if err != nil {
t.Fatal(err)
}
if !updated.SYBProduct.ManuallyConfirmed || updated.SYBProduct.TargetColor != "人工颜色" || updated.SYBProduct.TargetSize != "人工尺码" {
t.Fatalf("re-import overwrote human decision: %+v", updated.SYBProduct)
}
}
// Regression test: ReparseOutcome originally had no json tags at all, so Go's
// default marshaling produced PascalCase keys ("SYBProductID", "OldStatus")
// instead of the camelCase the rest of this API and the admin frontend use.
-164
View File
@@ -1,164 +0,0 @@
package sybimport
import (
"context"
"encoding/json"
"sort"
"go-admin/app/goauto/models"
"go-admin/app/goauto/shopeeproduct"
"gorm.io/gorm"
)
// SpecRoleRepairReport contains only aggregate migration evidence. It does not
// expose order numbers, raw source payloads, or specification text.
type SpecRoleRepairReport struct {
SYBRows int
ShopeeProducts int
}
type specRoleChange struct {
OldColor string
OldSize string
}
// RepairReversedSpecRoles corrects only the historical shape that #216 can
// prove was role-reversed: the newly parsed color/size are exactly the old
// size/color. Human-confirmed rows are never overwritten. RawJSON is read for
// reparsing and deliberately excluded from every update.
func RepairReversedSpecRoles(ctx context.Context, db *gorm.DB) (SpecRoleRepairReport, error) {
report := SpecRoleRepairReport{}
err := db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
var records []models.SYBProduct
if err := tx.Where("manually_confirmed = ?", false).Find(&records).Error; err != nil {
return err
}
changesByProduct := map[uint64][]specRoleChange{}
for _, record := range records {
var raw rawDetailSpec
if json.Unmarshal([]byte(record.RawJSON), &raw) != nil {
continue
}
parsed := Parse(raw.ProductSpec)
if parsed.Status == models.SYBParseStatusFailed || parsed.Color == record.TargetColor ||
parsed.Color != record.TargetSize || parsed.Size != record.TargetColor {
continue
}
if err := tx.Model(&models.SYBProduct{}).Where("id = ?", record.ID).Updates(map[string]any{
"target_color": parsed.Color,
"target_size": parsed.Size,
"parse_status": parsed.Status,
"parse_note": parsed.Note,
}).Error; err != nil {
return err
}
report.SYBRows++
if record.ShopeeProductID != nil {
productID := *record.ShopeeProductID
changesByProduct[productID] = append(changesByProduct[productID], specRoleChange{
OldColor: record.TargetColor,
OldSize: record.TargetSize,
})
}
}
for productID, changes := range changesByProduct {
changed, err := reconcileRepairedShopeeSpecs(tx, productID, changes)
if err != nil {
return err
}
if changed {
report.ShopeeProducts++
}
}
return nil
})
return report, err
}
func reconcileRepairedShopeeSpecs(tx *gorm.DB, productID uint64, changes []specRoleChange) (bool, error) {
var product models.ShopeeProduct
if err := tx.First(&product, productID).Error; err != nil {
return false, err
}
existing, err := shopeeproduct.Unmarshal(product.SpecsJSON)
if err != nil {
return false, err
}
var rows []models.SYBProduct
if err := tx.Where("shopee_product_id = ? AND (parse_status = ? OR manually_confirmed = ?)",
productID, models.SYBParseStatusSuccess, true).Find(&rows).Error; err != nil {
return false, err
}
usedColors, usedSizes := map[string]bool{}, map[string]bool{}
for _, row := range rows {
if row.TargetColor != "" {
usedColors[row.TargetColor] = true
}
if row.TargetSize != "" {
usedSizes[row.TargetSize] = true
}
}
oldColors, oldSizes := map[string]bool{}, map[string]bool{}
for _, change := range changes {
oldColors[change.OldColor] = true
oldSizes[change.OldSize] = true
}
cleaned := make([]shopeeproduct.SpecDimension, 0, len(existing))
for _, dimension := range existing {
values := make([]shopeeproduct.SpecValue, 0, len(dimension.Values))
for _, value := range dimension.Values {
remove := value.Source == shopeeproduct.ValueSourceImport && value.Mapping == nil
switch dimension.Role {
case shopeeproduct.RoleColor:
remove = remove && oldColors[value.Name] && !usedColors[value.Name]
case shopeeproduct.RoleSize:
remove = remove && oldSizes[value.Name] && !usedSizes[value.Name]
default:
remove = false
}
if !remove {
values = append(values, value)
}
}
dimension.Values = values
cleaned = append(cleaned, dimension)
}
incoming := make([]shopeeproduct.SpecDimension, 0, 2)
if values := sortedImportValues(usedColors); len(values) > 0 {
incoming = append(incoming, shopeeproduct.SpecDimension{Name: "颜色", Role: shopeeproduct.RoleColor, Values: values})
}
if values := sortedImportValues(usedSizes); len(values) > 0 {
incoming = append(incoming, shopeeproduct.SpecDimension{Name: "尺码", Role: shopeeproduct.RoleSize, Values: values})
}
reconciled := shopeeproduct.Merge(cleaned, incoming)
if err := shopeeproduct.Validate(reconciled); err != nil {
return false, err
}
encoded, err := shopeeproduct.Marshal(reconciled)
if err != nil {
return false, err
}
if encoded == product.SpecsJSON {
return false, nil
}
return true, tx.Model(&models.ShopeeProduct{}).Where("id = ?", productID).Update("specs_json", encoded).Error
}
func sortedImportValues(set map[string]bool) []shopeeproduct.SpecValue {
names := make([]string, 0, len(set))
for name := range set {
names = append(names, name)
}
sort.Strings(names)
values := make([]shopeeproduct.SpecValue, 0, len(names))
for _, name := range names {
values = append(values, shopeeproduct.SpecValue{Name: name, Source: shopeeproduct.ValueSourceImport})
}
return values
}
@@ -1,126 +0,0 @@
package sybimport
import (
"context"
"testing"
"go-admin/app/goauto/models"
"go-admin/app/goauto/shopeeproduct"
)
func TestRepairReversedSpecRolesIsSafeAndIdempotent(t *testing.T) {
db := newSyncTestDB(t)
manualMapping := &shopeeproduct.Mapping{PDDValue: "藏青", Source: shopeeproduct.MappingSourceManual, Status: shopeeproduct.MappingStatusConfirmed}
specs, err := shopeeproduct.Marshal([]shopeeproduct.SpecDimension{
{Name: "颜色", Role: shopeeproduct.RoleColor, Values: []shopeeproduct.SpecValue{
{Name: "均碼", Source: shopeeproduct.ValueSourceImport},
{Name: "手工蓝", Source: shopeeproduct.ValueSourceManual, Mapping: manualMapping},
}},
{Name: "尺码", Role: shopeeproduct.RoleSize, Values: []shopeeproduct.SpecValue{{Name: "黑色", Source: shopeeproduct.ValueSourceImport}}},
})
if err != nil {
t.Fatal(err)
}
product := models.ShopeeProduct{ShopeeItemID: "ROLE-REPAIR", Title: "测试", ShopName: "测试店铺", Currency: "TWD", SpecsJSON: specs}
if err := db.Create(&product).Error; err != nil {
t.Fatal(err)
}
raw := `{"productSpec":"均碼,黑色"}`
row := models.SYBProduct{OrderCode: "ROLE-ORDER", DetailID: 1, StockID: 1, ShopeeItemID: product.ShopeeItemID,
ShopeeProductID: &product.ID, ProductTitle: "测试", TargetColor: "均碼", TargetSize: "黑色", Quantity: 1,
ParseStatus: models.SYBParseStatusSuccess, RawJSON: raw}
if err := db.Create(&row).Error; err != nil {
t.Fatal(err)
}
manual := models.SYBProduct{OrderCode: "MANUAL-ORDER", DetailID: 2, StockID: 2, ShopeeItemID: product.ShopeeItemID,
ShopeeProductID: &product.ID, ProductTitle: "测试", TargetColor: "人工颜色", TargetSize: "人工尺码", Quantity: 1,
ParseStatus: models.SYBParseStatusSuccess, ManuallyConfirmed: true, RawJSON: raw}
if err := db.Create(&manual).Error; err != nil {
t.Fatal(err)
}
report, err := RepairReversedSpecRoles(context.Background(), db)
if err != nil || report.SYBRows != 1 || report.ShopeeProducts != 1 {
t.Fatalf("unexpected repair report=%+v err=%v", report, err)
}
var repaired, untouched models.SYBProduct
db.First(&repaired, row.ID)
db.First(&untouched, manual.ID)
if repaired.TargetColor != "黑色" || repaired.TargetSize != "均碼" || repaired.RawJSON != raw {
t.Fatalf("repaired row is invalid: %+v", repaired)
}
if untouched.TargetColor != "人工颜色" || untouched.TargetSize != "人工尺码" {
t.Fatalf("manual row was overwritten: %+v", untouched)
}
db.First(&product, product.ID)
reconciled, err := shopeeproduct.Unmarshal(product.SpecsJSON)
if err != nil {
t.Fatal(err)
}
assertSpecValue(t, reconciled, shopeeproduct.RoleColor, "均碼", false)
assertSpecValue(t, reconciled, shopeeproduct.RoleSize, "黑色", false)
assertSpecValue(t, reconciled, shopeeproduct.RoleColor, "黑色", true)
assertSpecValue(t, reconciled, shopeeproduct.RoleSize, "均碼", true)
assertSpecValue(t, reconciled, shopeeproduct.RoleColor, "手工蓝", true)
replay, err := RepairReversedSpecRoles(context.Background(), db)
if err != nil || replay.SYBRows != 0 || replay.ShopeeProducts != 0 {
t.Fatalf("repair was not idempotent: report=%+v err=%v", replay, err)
}
}
func TestRepairReversedSpecRolesKeepsAmbiguousColorUncertain(t *testing.T) {
db := newSyncTestDB(t)
specs, err := shopeeproduct.Marshal([]shopeeproduct.SpecDimension{
{Name: "颜色", Role: shopeeproduct.RoleColor, Values: []shopeeproduct.SpecValue{{Name: "均碼", Source: shopeeproduct.ValueSourceImport}}},
{Name: "尺码", Role: shopeeproduct.RoleSize, Values: []shopeeproduct.SpecValue{{Name: "黑色+白色", Source: shopeeproduct.ValueSourceImport}}},
})
if err != nil {
t.Fatal(err)
}
product := models.ShopeeProduct{ShopeeItemID: "ROLE-UNCERTAIN", Title: "测试", ShopName: "测试店铺", Currency: "TWD", SpecsJSON: specs}
if err := db.Create(&product).Error; err != nil {
t.Fatal(err)
}
row := models.SYBProduct{OrderCode: "UNCERTAIN-ORDER", DetailID: 1, StockID: 1, ShopeeItemID: product.ShopeeItemID,
ShopeeProductID: &product.ID, ProductTitle: "测试", TargetColor: "均碼", TargetSize: "黑色+白色", Quantity: 1,
ParseStatus: models.SYBParseStatusSuccess, RawJSON: `{"productSpec":"均碼,黑色+白色"}`}
if err := db.Create(&row).Error; err != nil {
t.Fatal(err)
}
report, err := RepairReversedSpecRoles(context.Background(), db)
if err != nil || report.SYBRows != 1 || report.ShopeeProducts != 1 {
t.Fatalf("unexpected repair report=%+v err=%v", report, err)
}
db.First(&row, row.ID)
if row.TargetColor != "黑色+白色" || row.TargetSize != "均碼" || row.ParseStatus != models.SYBParseStatusUncertain {
t.Fatalf("ambiguous color role/status was not preserved safely: %+v", row)
}
db.First(&product, product.ID)
reconciled, err := shopeeproduct.Unmarshal(product.SpecsJSON)
if err != nil {
t.Fatal(err)
}
assertSpecValue(t, reconciled, shopeeproduct.RoleColor, "均碼", false)
assertSpecValue(t, reconciled, shopeeproduct.RoleSize, "黑色+白色", false)
}
func assertSpecValue(t *testing.T, specs []shopeeproduct.SpecDimension, role, name string, want bool) {
t.Helper()
found := false
for _, dimension := range specs {
if dimension.Role != role {
continue
}
for _, value := range dimension.Values {
if value.Name == name {
found = true
}
}
}
if found != want {
t.Fatalf("role=%s value=%q found=%v want=%v specs=%+v", role, name, found, want, specs)
}
}
+13 -31
View File
@@ -59,16 +59,12 @@ type SyncReport struct {
// it a renamed or newly opened shop is never noticed (#50).
ShopBreakdown map[string]ShopBreakdown `json:"shopBreakdown"`
ShopFilterHash string `json:"shopFilterHash"`
// ShopFilterSnapshot is the immutable enabled-shop snapshot supplied when
// the run was created. It is intentionally a display-name list for audit;
// matching continues to use the normalized map passed to the sync.
ShopFilterSnapshot []string `json:"shopFilterSnapshot"`
OrderCount int `json:"orderCount"`
DetailCount int `json:"detailCount"`
Created int `json:"created"`
Updated int `json:"updated"`
StartedAt time.Time `json:"startedAt"`
FinishedAt time.Time `json:"finishedAt"`
OrderCount int `json:"orderCount"`
DetailCount int `json:"detailCount"`
Created int `json:"created"`
Updated int `json:"updated"`
StartedAt time.Time `json:"startedAt"`
FinishedAt time.Time `json:"finishedAt"`
}
type ShopBreakdown struct {
@@ -124,24 +120,19 @@ func Sync(ctx context.Context, db *gorm.DB, client *sybclient.Client, cfg SyncCo
}
func SyncWithProgress(ctx context.Context, db *gorm.DB, client *sybclient.Client, cfg SyncConfig, dateFrom, dateTo string, progress ProgressFunc) (SyncReport, error) {
allowed, err := sybshop.EnabledNames(ctx, db)
if err != nil {
return SyncReport{From: dateFrom, To: dateTo, StartedAt: time.Now().UTC(), ShopBreakdown: map[string]ShopBreakdown{}}, err
}
return SyncWithShopSnapshot(ctx, db, client, cfg, dateFrom, dateTo, allowed, progress)
}
// SyncWithShopSnapshot runs against the immutable enabled-shop snapshot taken
// by StartImport. It must not query syb_shop again: doing so would make the
// run's effective allow-list differ from the snapshot recorded on its row.
func SyncWithShopSnapshot(ctx context.Context, db *gorm.DB, client *sybclient.Client, cfg SyncConfig, dateFrom, dateTo string, allowed map[string]string, progress ProgressFunc) (SyncReport, error) {
report := SyncReport{From: dateFrom, To: dateTo, StartedAt: time.Now().UTC(),
ShopBreakdown: map[string]ShopBreakdown{}, ShopFilterSnapshot: shopSnapshotNames(allowed)}
ShopBreakdown: map[string]ShopBreakdown{}}
dates, err := splitDateRange(dateFrom, dateTo)
if err != nil {
return report, err
}
// `[必须]` The allow-list is read once and used for the whole run, so a shop
// enabled midway cannot make one day behave differently from the next.
allowed, err := sybshop.EnabledNames(ctx, db)
if err != nil {
return report, err
}
if len(allowed) == 0 {
return report, ErrNoEnabledShop
}
@@ -292,15 +283,6 @@ func SyncWithShopSnapshot(ctx context.Context, db *gorm.DB, client *sybclient.Cl
return report, nil
}
func shopSnapshotNames(allowed map[string]string) []string {
names := make([]string, 0, len(allowed))
for _, display := range allowed {
names = append(names, display)
}
sort.Strings(names)
return names
}
func enabledShopHash(allowed map[string]string) string {
names := make([]string, 0, len(allowed))
for normalized := range allowed {
+10 -57
View File
@@ -28,7 +28,6 @@ func NewSyncRunService(db *gorm.DB) *SyncRunService { return &SyncRunService{db:
type CreateSyncRunInput struct {
DateFrom, DateTo, ShopFilterHash string
ShopFilterSnapshot map[string]string
OperatorID uint64
OperatorName string
}
@@ -39,14 +38,10 @@ func (s *SyncRunService) Create(ctx context.Context, input CreateSyncRunInput) (
return nil, invalidRequest(err.Error())
}
slot := uint8(1)
payload, err := marshalSyncRunPayload(nil, shopSnapshotNames(input.ShopFilterSnapshot))
if err != nil {
return nil, &ServiceError{Code: CodeSyncRunCreateFailed, Message: "服务端处理失败", Cause: err}
}
run := &models.SYBSyncRun{
DateFrom: input.DateFrom, DateTo: input.DateTo, Status: SyncRunRunning,
ActiveSlot: &slot, DaysTotal: len(dates), ShopFilterHash: input.ShopFilterHash,
ShopBreakdownJSON: payload, OperatorID: input.OperatorID,
ShopBreakdownJSON: "{}", OperatorID: input.OperatorID,
OperatorName: strings.TrimSpace(input.OperatorName), StartedAt: time.Now().UTC(),
}
if err := s.db.WithContext(ctx).Create(run).Error; err != nil {
@@ -60,7 +55,7 @@ func (s *SyncRunService) Create(ctx context.Context, input CreateSyncRunInput) (
}
func (s *SyncRunService) UpdateProgress(ctx context.Context, id uint64, progress SyncProgress) error {
payload, err := marshalSyncRunPayload(progress.Report.ShopBreakdown, progress.Report.ShopFilterSnapshot)
breakdown, err := json.Marshal(progress.Report.ShopBreakdown)
if err != nil {
return err
}
@@ -70,7 +65,7 @@ func (s *SyncRunService) UpdateProgress(ctx context.Context, id uint64, progress
"order_count": progress.Report.OrderCount, "detail_count": progress.Report.DetailCount,
"accepted_count": progress.Report.AcceptedCount, "shop_skipped": progress.Report.ShopSkipped,
"created_count": progress.Report.Created, "updated_count": progress.Report.Updated,
"shop_filter_hash": progress.Report.ShopFilterHash, "shop_breakdown_json": payload,
"shop_filter_hash": progress.Report.ShopFilterHash, "shop_breakdown_json": string(breakdown),
}).Error
}
@@ -78,7 +73,7 @@ func (s *SyncRunService) Finish(ctx context.Context, id uint64, status string, r
if status != SyncRunSucceeded && status != SyncRunFailed {
return fmt.Errorf("invalid terminal sync status %q", status)
}
payload, err := marshalSyncRunPayload(report.ShopBreakdown, report.ShopFilterSnapshot)
breakdown, err := json.Marshal(report.ShopBreakdown)
if err != nil {
return err
}
@@ -92,7 +87,7 @@ func (s *SyncRunService) Finish(ctx context.Context, id uint64, status string, r
"order_count": report.OrderCount, "detail_count": report.DetailCount,
"accepted_count": report.AcceptedCount, "shop_skipped": report.ShopSkipped,
"created_count": report.Created, "updated_count": report.Updated,
"shop_breakdown_json": payload,
"shop_breakdown_json": string(breakdown),
}
if report.ShopFilterHash != "" {
updates["shop_filter_hash"] = report.ShopFilterHash
@@ -151,10 +146,8 @@ type ShopBreakdownItem struct {
}
type SyncRunDTO struct {
models.SYBSyncRun
ShopBreakdown []ShopBreakdownItem `json:"shopBreakdown"`
ShopFilterSnapshot []string `json:"shopFilterSnapshot"`
ShopFilterSnapshotPresent bool `json:"shopFilterSnapshotPresent"`
ProgressPercent int `json:"progressPercent"`
ShopBreakdown []ShopBreakdownItem `json:"shopBreakdown"`
ProgressPercent int `json:"progressPercent"`
}
func (s *SyncRunService) List(ctx context.Context, request SyncRunListRequest) (SyncRunListResponse, error) {
@@ -195,7 +188,8 @@ func (s *SyncRunService) Detail(ctx context.Context, id uint64) (SyncRunDTO, err
}
func syncRunDTO(row models.SYBSyncRun) SyncRunDTO {
values, snapshot, snapshotPresent := parseSyncRunPayload(row.ShopBreakdownJSON)
var values map[string]ShopBreakdown
_ = json.Unmarshal([]byte(row.ShopBreakdownJSON), &values)
names := make([]string, 0, len(values))
for name := range values {
names = append(names, name)
@@ -216,48 +210,7 @@ func syncRunDTO(row models.SYBSyncRun) SyncRunDTO {
if row.Status == SyncRunSucceeded {
percent = 100
}
return SyncRunDTO{SYBSyncRun: row, ShopBreakdown: items, ShopFilterSnapshot: snapshot,
ShopFilterSnapshotPresent: snapshotPresent, ProgressPercent: percent}
}
const syncRunPayloadVersion = 1
type syncRunPayload struct {
Version int `json:"version"`
ShopBreakdown map[string]ShopBreakdown `json:"shopBreakdown"`
ShopFilterSnapshot []string `json:"shopFilterSnapshot"`
}
func marshalSyncRunPayload(breakdown map[string]ShopBreakdown, snapshot []string) (string, error) {
if breakdown == nil {
breakdown = map[string]ShopBreakdown{}
}
if snapshot == nil {
snapshot = []string{}
}
encoded, err := json.Marshal(syncRunPayload{
Version: syncRunPayloadVersion, ShopBreakdown: breakdown, ShopFilterSnapshot: snapshot,
})
if err != nil {
return "", err
}
return string(encoded), nil
}
func parseSyncRunPayload(raw string) (map[string]ShopBreakdown, []string, bool) {
var envelope syncRunPayload
if err := json.Unmarshal([]byte(raw), &envelope); err == nil && envelope.Version == syncRunPayloadVersion {
if envelope.ShopBreakdown == nil {
envelope.ShopBreakdown = map[string]ShopBreakdown{}
}
sort.Strings(envelope.ShopFilterSnapshot)
return envelope.ShopBreakdown, envelope.ShopFilterSnapshot, true
}
// Records written before #212 are a bare breakdown map. Preserve their
// existing statistics and state explicitly that no immutable snapshot exists.
values := map[string]ShopBreakdown{}
_ = json.Unmarshal([]byte(raw), &values)
return values, nil, false
return SyncRunDTO{SYBSyncRun: row, ShopBreakdown: items, ProgressPercent: percent}
}
func truncateRunError(value string, limit int) string {
@@ -4,8 +4,6 @@ import (
"context"
"strings"
"testing"
"go-admin/app/goauto/models"
)
func TestSyncRunSingleFlightAndTerminalRelease(t *testing.T) {
@@ -59,32 +57,3 @@ func TestSyncRunProgressDetailAndRecovery(t *testing.T) {
t.Fatalf("启动恢复状态不正确: %+v", detail)
}
}
func TestSyncRunPersistsImmutableShopSnapshotAndReadsLegacyBreakdown(t *testing.T) {
db := newSyncTestDB(t)
service := NewSyncRunService(db)
run, err := service.Create(context.Background(), CreateSyncRunInput{
DateFrom: "2026-08-01", DateTo: "2026-08-01", ShopFilterHash: "snapshot-hash",
ShopFilterSnapshot: map[string]string{"alpha": "Alpha 店", "beta": "Beta 店"},
})
if err != nil {
t.Fatal(err)
}
detail, err := service.Detail(context.Background(), run.ID)
if err != nil {
t.Fatal(err)
}
if !detail.ShopFilterSnapshotPresent || strings.Join(detail.ShopFilterSnapshot, ",") != "Alpha 店,Beta 店" {
t.Fatalf("新任务必须返回冻结快照: %+v", detail)
}
if err := db.Model(&models.SYBSyncRun{}).Where("id = ?", run.ID).Update("shop_breakdown_json", `{"旧店":{"accepted":2,"skipped":1}}`).Error; err != nil {
t.Fatal(err)
}
legacy, err := service.Detail(context.Background(), run.ID)
if err != nil {
t.Fatal(err)
}
if legacy.ShopFilterSnapshotPresent || len(legacy.ShopFilterSnapshot) != 0 || len(legacy.ShopBreakdown) != 1 || legacy.ShopBreakdown[0].Accepted != 2 {
t.Fatalf("旧记录必须保留统计且明确没有快照: %+v", legacy)
}
}
-20
View File
@@ -429,26 +429,6 @@ func TestSyncMatchesShopNamesThroughNormalization(t *testing.T) {
}
}
func TestSyncWithShopSnapshotDoesNotRereadChangedShopState(t *testing.T) {
db := newSyncTestDB(t)
allowed, err := sybshop.EnabledNames(context.Background(), db)
if err != nil {
t.Fatal(err)
}
if err := db.Model(&models.SYBShop{}).Where("normalized_name = ?", "测试店铺").Update("enabled", false).Error; err != nil {
t.Fatal(err)
}
f := &fakeSYB{perDay: map[string]int{"2026-08-01": 2}, shopNames: []string{"测试店铺"}}
report, err := SyncWithShopSnapshot(context.Background(), db, newSyncClient(t, f),
SyncConfig{PageSize: 10, MaxMatches: 1000}, "2026-08-01", "2026-08-01", allowed, nil)
if err != nil {
t.Fatalf("冻结快照同步失败: %v", err)
}
if report.AcceptedCount != 2 || len(report.ShopFilterSnapshot) != 1 || report.ShopFilterSnapshot[0] != "测试店铺" {
t.Fatalf("后台不得重读已变化的店铺状态: %+v", report)
}
}
// 列表说属于启用店铺、明细却是另一家店时必须拦住。
func TestSyncRechecksShopOnDetailResponse(t *testing.T) {
db := newSyncTestDB(t)
-91
View File
@@ -1,91 +0,0 @@
package sybshop
import (
"context"
"fmt"
"go-admin/app/goauto/models"
"gorm.io/gorm"
)
// ReconcileNormalizedNames repairs the matching keys of live SYB shops created
// before Normalize became the single write-path rule. It changes neither the
// display name nor the enabled state. The operation is safe to repeat.
//
// A normalised-name collision is a data ambiguity. It must be resolved by an
// operator, rather than silently merging shops or choosing one arbitrarily.
func ReconcileNormalizedNames(ctx context.Context, db *gorm.DB) (int, error) {
if db == nil {
return 0, fmt.Errorf("syb shop normalized-name repair: database is nil")
}
updated := 0
err := db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
var shops []models.SYBShop
if err := tx.Order("id ASC").Find(&shops).Error; err != nil {
return err
}
desiredOwners := make(map[string]uint64, len(shops))
currentKeys := make(map[string]struct{}, len(shops))
type repair struct {
id uint64
desired string
temp string
}
repairs := make([]repair, 0)
for _, shop := range shops {
desired := Normalize(shop.DisplayName)
if desired == "" {
return fmt.Errorf("syb shop normalized-name repair: shop %d has blank display name", shop.ID)
}
if owner, exists := desiredOwners[desired]; exists && owner != shop.ID {
return fmt.Errorf("syb shop normalized-name repair: normalized key conflict between shops %d and %d", owner, shop.ID)
}
desiredOwners[desired] = shop.ID
currentKeys[shop.NormalizedName] = struct{}{}
if shop.NormalizedName != desired {
repairs = append(repairs, repair{id: shop.ID, desired: desired})
}
}
// Keys can be swapped (for example stale "a"/"b" values). Stage every
// repair through a unique temporary key so the unique index is preserved
// throughout the transaction.
reserved := make(map[string]struct{}, len(currentKeys)+len(desiredOwners))
for key := range currentKeys {
reserved[key] = struct{}{}
}
for key := range desiredOwners {
reserved[key] = struct{}{}
}
for index := range repairs {
base := fmt.Sprintf("__goauto_syb_shop_normalize_repair_%d__", repairs[index].id)
temp := base
for suffix := 1; ; suffix++ {
if _, exists := reserved[temp]; !exists {
break
}
temp = fmt.Sprintf("%s%d", base, suffix)
}
repairs[index].temp = temp
reserved[temp] = struct{}{}
}
for _, repair := range repairs {
if err := tx.Model(&models.SYBShop{}).Where("id = ?", repair.id).
UpdateColumn("normalized_name", repair.temp).Error; err != nil {
return err
}
}
for _, repair := range repairs {
if err := tx.Model(&models.SYBShop{}).Where("id = ?", repair.id).
UpdateColumn("normalized_name", repair.desired).Error; err != nil {
return err
}
}
updated = len(repairs)
return nil
})
return updated, err
}
@@ -1,93 +0,0 @@
package sybshop_test
import (
"context"
"strings"
"testing"
"go-admin/app/goauto/models"
"go-admin/app/goauto/sybshop"
)
func TestReconcileNormalizedNamesRepairsLegacyKey(t *testing.T) {
db := openDB(t)
shop := models.SYBShop{DisplayName: "樂齡樂活美學館|銀髮居家熟齡悅己壯世代優雅生活", NormalizedName: "legacy-key", Enabled: true}
if err := db.Create(&shop).Error; err != nil {
t.Fatalf("创建历史店铺失败: %v", err)
}
updated, err := sybshop.ReconcileNormalizedNames(context.Background(), db)
if err != nil {
t.Fatalf("回填失败: %v", err)
}
if updated != 1 {
t.Fatalf("应修复 1 条,实际 %d", updated)
}
var reloaded models.SYBShop
if err := db.First(&reloaded, shop.ID).Error; err != nil {
t.Fatalf("读取修复结果失败: %v", err)
}
if reloaded.NormalizedName != sybshop.Normalize(shop.DisplayName) {
t.Fatalf("规范化键不正确: %q", reloaded.NormalizedName)
}
if reloaded.DisplayName != shop.DisplayName || !reloaded.Enabled {
t.Fatalf("回填不应修改展示名或启用状态: %+v", reloaded)
}
names, err := sybshop.EnabledNames(context.Background(), db)
if err != nil || names[sybshop.Normalize(shop.DisplayName)] != shop.DisplayName {
t.Fatalf("修复后启用快照未命中: names=%v err=%v", names, err)
}
updated, err = sybshop.ReconcileNormalizedNames(context.Background(), db)
if err != nil || updated != 0 {
t.Fatalf("重复回填应无变化: updated=%d err=%v", updated, err)
}
}
func TestReconcileNormalizedNamesRejectsCollisionWithoutWriting(t *testing.T) {
db := openDB(t)
first := models.SYBShop{DisplayName: "A店", NormalizedName: "legacy-one", Enabled: true}
second := models.SYBShop{DisplayName: "A店", NormalizedName: "legacy-two", Enabled: true}
if err := db.Create(&first).Error; err != nil {
t.Fatalf("创建第一条历史店铺失败: %v", err)
}
if err := db.Create(&second).Error; err != nil {
t.Fatalf("创建第二条历史店铺失败: %v", err)
}
updated, err := sybshop.ReconcileNormalizedNames(context.Background(), db)
if updated != 0 || err == nil || !strings.Contains(err.Error(), "normalized key conflict") {
t.Fatalf("冲突应失败且不报告更新: updated=%d err=%v", updated, err)
}
var rows []models.SYBShop
if err := db.Order("id ASC").Find(&rows).Error; err != nil {
t.Fatalf("读取冲突结果失败: %v", err)
}
if len(rows) != 2 || rows[0].NormalizedName != "legacy-one" || rows[1].NormalizedName != "legacy-two" {
t.Fatalf("冲突时不应修改任何键: %+v", rows)
}
}
func TestReconcileNormalizedNamesHandlesSwappedLegacyKeys(t *testing.T) {
db := openDB(t)
first := models.SYBShop{DisplayName: "A", NormalizedName: "b", Enabled: true}
second := models.SYBShop{DisplayName: "B", NormalizedName: "a", Enabled: true}
if err := db.Create(&first).Error; err != nil {
t.Fatalf("创建第一条历史店铺失败: %v", err)
}
if err := db.Create(&second).Error; err != nil {
t.Fatalf("创建第二条历史店铺失败: %v", err)
}
updated, err := sybshop.ReconcileNormalizedNames(context.Background(), db)
if err != nil || updated != 2 {
t.Fatalf("交换键回填失败: updated=%d err=%v", updated, err)
}
var rows []models.SYBShop
if err := db.Order("id ASC").Find(&rows).Error; err != nil {
t.Fatalf("读取回填结果失败: %v", err)
}
if rows[0].NormalizedName != "a" || rows[1].NormalizedName != "b" {
t.Fatalf("交换键未正确回填: %+v", rows)
}
}
+87
View File
@@ -0,0 +1,87 @@
package apis
import (
"errors"
"net/http"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
jobservice "go-admin/app/jobs/service"
)
func (e SysJob) ListExecutionLogs(c *gin.Context) {
jobID, err := strconv.Atoi(c.Param("id"))
if err != nil || jobID < 1 {
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "jobId 无效"})
return
}
page, err := positiveQueryInt(c.Query("pageIndex"), 1)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "pageIndex 必须是正整数"})
return
}
pageSize, err := positiveQueryInt(c.Query("pageSize"), 20)
if err != nil || pageSize > 100 {
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "pageSize 必须是 1 到 100 的整数"})
return
}
startedFrom, err := optionalTime(c.Query("startedFrom"))
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "startedFrom 必须是 RFC3339 时间"})
return
}
startedTo, err := optionalTime(c.Query("startedTo"))
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": "startedTo 必须是 RFC3339 时间"})
return
}
e.MakeContext(c)
db, err := e.GetOrm()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"code": 500, "msg": "服务端处理失败"})
return
}
response, err := jobservice.NewExecutionLogService(db).List(c.Request.Context(), jobID, jobservice.ExecutionLogListRequest{
Page: page, PageSize: pageSize, Status: strings.TrimSpace(c.Query("status")),
StartedFrom: startedFrom, StartedTo: startedTo,
})
if err != nil {
switch {
case errors.Is(err, jobservice.ErrExecutionLogInvalidRequest):
c.JSON(http.StatusUnprocessableEntity, gin.H{"code": 400, "msg": err.Error()})
case errors.Is(err, jobservice.ErrExecutionLogJobNotFound):
c.JSON(http.StatusNotFound, gin.H{"code": 404, "msg": "定时任务不存在"})
default:
e.GetLogger().Errorf("list scheduled job execution logs failed job_id=%d: %v", jobID, err)
c.JSON(http.StatusInternalServerError, gin.H{"code": 500, "msg": "服务端处理失败"})
}
return
}
c.JSON(http.StatusOK, gin.H{"code": 200, "data": response})
}
func positiveQueryInt(value string, fallback int) (int, error) {
if strings.TrimSpace(value) == "" {
return fallback, nil
}
parsed, err := strconv.Atoi(value)
if err != nil || parsed < 1 {
return 0, errors.New("invalid positive integer")
}
return parsed, nil
}
func optionalTime(value string) (*time.Time, error) {
if strings.TrimSpace(value) == "" {
return nil, nil
}
parsed, err := time.Parse(time.RFC3339, value)
if err != nil {
return nil, err
}
return &parsed, nil
}
+5 -2
View File
@@ -4,6 +4,7 @@ import (
"fmt"
"time"
"go-admin/app/goauto/shopeeproduct"
"go-admin/app/goauto/sybimport"
)
@@ -12,8 +13,10 @@ import (
// 字典 key 可以配置到 自动任务 调用目标 中;
func InitJob() {
jobList = map[string]JobExec{
"ExamplesOne": ExamplesOne{},
sybimport.HourlySyncInvokeTarget: sybimport.HourlySyncJob{},
"ExamplesOne": ExamplesOne{},
sybimport.HourlySyncInvokeTarget: sybimport.HourlySyncJob{},
sybimport.SpecAIParseInvokeTarget: sybimport.ScheduledSpecAIParseJob{},
shopeeproduct.SpecAutoMatchInvokeTarget: shopeeproduct.ScheduledAutoMatchJob{},
// ...
}
}
+132
View File
@@ -0,0 +1,132 @@
package jobs
import (
"context"
"errors"
"fmt"
"time"
log "github.com/go-admin-team/go-admin-core/logger"
"github.com/google/uuid"
"gorm.io/gorm"
"go-admin/app/jobs/models"
)
const (
executionErrorTargetMissing = "JOB_TARGET_NOT_FOUND"
executionErrorExecFailed = "JOB_EXECUTION_FAILED"
executionErrorHTTPFailed = "JOB_HTTP_FAILED"
executionErrorPanicked = "JOB_EXECUTION_PANICKED"
executionErrorInterrupted = "JOB_INTERRUPTED"
)
type executionFailure struct {
code string
message string
cause error
}
func (failure *executionFailure) Error() string {
if failure.cause != nil {
return failure.cause.Error()
}
return failure.message
}
func newExecutionFailure(code, message string, cause error) error {
return &executionFailure{code: code, message: message, cause: cause}
}
func runWithExecutionLog(db *gorm.DB, core JobCore, execute func() error) (executionErr error) {
startedAt := time.Now().UTC()
record := &models.SysJobExecutionLog{
ExecutionID: uuid.NewString(), JobID: core.JobId,
JobNameSnapshot: core.Name, InvokeTargetSnapshot: core.InvokeTarget,
TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionRunning,
StartedAt: startedAt,
}
created := false
if db != nil {
if err := db.WithContext(context.Background()).Create(record).Error; err != nil {
log.Errorf("[Job] execution log create failed job_id=%d: %v", core.JobId, err)
} else {
created = true
}
}
defer func() {
if recovered := recover(); recovered != nil {
executionErr = newExecutionFailure(executionErrorPanicked, "任务执行异常中断", nil)
if created {
finishExecutionLog(db, core.JobId, record, startedAt, executionErr)
}
return
}
if created {
finishExecutionLog(db, core.JobId, record, startedAt, executionErr)
}
}()
executionErr = execute()
return executionErr
}
func finishExecutionLog(db *gorm.DB, jobID int, record *models.SysJobExecutionLog, startedAt time.Time, executionErr error) {
finishedAt := time.Now().UTC()
updates := map[string]any{
"status": models.JobExecutionSucceeded, "finished_at": finishedAt,
"duration_ms": finishedAt.Sub(startedAt).Milliseconds(), "error_code": "", "error_message": "",
}
if executionErr != nil {
code, message := publicExecutionFailure(executionErr)
updates["status"] = models.JobExecutionFailed
updates["error_code"] = code
updates["error_message"] = message
}
if err := db.WithContext(context.Background()).Model(&models.SysJobExecutionLog{}).
Where("id = ? AND status = ?", record.ID, models.JobExecutionRunning).Updates(updates).Error; err != nil {
log.Errorf("[Job] execution log finish failed job_id=%d execution_id=%s: %v", jobID, record.ExecutionID, err)
}
}
func publicExecutionFailure(err error) (string, string) {
var failure *executionFailure
if errors.As(err, &failure) {
return failure.code, failure.message
}
return executionErrorExecFailed, "任务执行失败,请查看受控服务日志"
}
// RecoverInterruptedExecutionLogs closes invocations left running by the
// previous process. Production currently runs one scheduler per database.
func RecoverInterruptedExecutionLogs(db *gorm.DB) error {
if db == nil {
return nil
}
now := time.Now().UTC()
var records []models.SysJobExecutionLog
if err := db.WithContext(context.Background()).Where("status = ?", models.JobExecutionRunning).Find(&records).Error; err != nil {
return err
}
return db.WithContext(context.Background()).Transaction(func(tx *gorm.DB) error {
for _, record := range records {
duration := now.Sub(record.StartedAt).Milliseconds()
if duration < 0 {
duration = 0
}
if err := tx.Model(&models.SysJobExecutionLog{}).
Where("id = ? AND status = ?", record.ID, models.JobExecutionRunning).Updates(map[string]any{
"status": models.JobExecutionInterrupted, "finished_at": now,
"duration_ms": duration, "error_code": executionErrorInterrupted,
"error_message": "服务重启前任务未完成",
}).Error; err != nil {
return err
}
}
return nil
})
}
func missingExecutionTarget(target string) error {
return newExecutionFailure(executionErrorTargetMissing, "任务调用目标未注册", fmt.Errorf("job target %q is not registered", target))
}
+113
View File
@@ -0,0 +1,113 @@
package jobs
import (
"errors"
"testing"
"time"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"go-admin/app/jobs/models"
)
func jobExecutionTestDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(&models.SysJobExecutionLog{}); err != nil {
t.Fatal(err)
}
return db
}
func TestRunWithExecutionLogRecordsSuccessAndSanitizedFailure(t *testing.T) {
db := jobExecutionTestDB(t)
core := JobCore{JobId: 7, Name: "测试任务", InvokeTarget: "TestTarget"}
if err := runWithExecutionLog(db, core, func() error { return nil }); err != nil {
t.Fatal(err)
}
rawSecret := "token=secret-value productSpec=private"
if err := runWithExecutionLog(db, core, func() error { return errors.New(rawSecret) }); err == nil {
t.Fatal("failed execution must return its error")
}
var records []models.SysJobExecutionLog
if err := db.Order("id asc").Find(&records).Error; err != nil {
t.Fatal(err)
}
if len(records) != 2 || records[0].Status != models.JobExecutionSucceeded || records[1].Status != models.JobExecutionFailed {
t.Fatalf("unexpected records: %+v", records)
}
if records[1].ErrorCode != executionErrorExecFailed || records[1].ErrorMessage == rawSecret || records[1].ErrorMessage == "" {
t.Fatalf("failure was not safely summarized: %+v", records[1])
}
if records[0].FinishedAt == nil || records[1].FinishedAt == nil || records[0].ExecutionID == records[1].ExecutionID {
t.Fatal("execution lifecycle or unique IDs were not recorded")
}
}
func TestRecoverInterruptedExecutionLogs(t *testing.T) {
db := jobExecutionTestDB(t)
started := time.Now().UTC().Add(-2 * time.Second)
record := models.SysJobExecutionLog{
ExecutionID: "00000000-0000-4000-8000-000000000010", JobID: 9,
JobNameSnapshot: "中断任务", InvokeTargetSnapshot: "Interrupted",
TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionRunning, StartedAt: started,
}
if err := db.Create(&record).Error; err != nil {
t.Fatal(err)
}
if err := RecoverInterruptedExecutionLogs(db); err != nil {
t.Fatal(err)
}
if err := db.First(&record, record.ID).Error; err != nil {
t.Fatal(err)
}
if record.Status != models.JobExecutionInterrupted || record.FinishedAt == nil || record.ErrorCode != executionErrorInterrupted || record.DurationMS < 1000 {
t.Fatalf("record was not safely interrupted: %+v", record)
}
}
func TestMissingExecutionTargetHasPublicSafeMessage(t *testing.T) {
code, message := publicExecutionFailure(missingExecutionTarget("SecretTarget"))
if code != executionErrorTargetMissing || message != "任务调用目标未注册" {
t.Fatalf("unexpected target failure %s %s", code, message)
}
}
func TestExecJobRecordsMissingTargetAsFailure(t *testing.T) {
db := jobExecutionTestDB(t)
previousJobList := jobList
jobList = map[string]JobExec{}
t.Cleanup(func() { jobList = previousJobList })
job := &ExecJob{JobCore: JobCore{JobId: 10, Name: "未注册任务", InvokeTarget: "MissingTarget"}, DB: db}
job.Run()
var record models.SysJobExecutionLog
if err := db.First(&record).Error; err != nil {
t.Fatal(err)
}
if record.Status != models.JobExecutionFailed || record.ErrorCode != executionErrorTargetMissing || record.ErrorMessage != "任务调用目标未注册" {
t.Fatalf("missing target was not safely recorded: %+v", record)
}
}
func TestRunWithExecutionLogSafelyRecordsPanic(t *testing.T) {
db := jobExecutionTestDB(t)
err := runWithExecutionLog(db, JobCore{JobId: 11, Name: "异常任务", InvokeTarget: "Panic"}, func() error {
panic("provider-secret")
})
if code, message := publicExecutionFailure(err); code != executionErrorPanicked || message != "任务执行异常中断" {
t.Fatalf("panic was not returned as a safe failure: %s %s", code, message)
}
var record models.SysJobExecutionLog
if err := db.First(&record).Error; err != nil {
t.Fatal(err)
}
if record.Status != models.JobExecutionFailed || record.ErrorCode != executionErrorPanicked || record.ErrorMessage != "任务执行异常中断" {
t.Fatalf("panic was not safely persisted: %+v", record)
}
}
+33 -22
View File
@@ -33,6 +33,7 @@ type JobCore struct {
// HttpJob 任务类型 http
type HttpJob struct {
JobCore
DB *gorm.DB
}
type ExecJob struct {
@@ -42,14 +43,16 @@ type ExecJob struct {
func (e *ExecJob) Run() {
startTime := time.Now()
var obj = jobList[e.InvokeTarget]
if obj == nil {
log.Warn("[Job] ExecJob Run job nil")
return
}
err := CallExecWithDB(obj.(JobExec), e.DB, e.Args)
err := runWithExecutionLog(e.DB, e.JobCore, func() error {
obj := jobList[e.InvokeTarget]
if obj == nil {
return missingExecutionTarget(e.InvokeTarget)
}
return CallExecWithDB(obj, e.DB, e.Args)
})
if err != nil {
log.Errorf("[Job] JobCore %s failed: %v", e.Name, err)
code, _ := publicExecutionFailure(err)
log.Errorf("[Job] JobCore %s failed error_code=%s", e.Name, code)
return
}
// 结束时间
@@ -68,22 +71,26 @@ func (e *ExecJob) Run() {
func (h *HttpJob) Run() {
startTime := time.Now()
var count = 0
var err error
var str string
/* 循环 */
LOOP:
if count < retryCount {
/* 跳过迭代 */
str, err = pkg.Get(h.InvokeTarget)
if err != nil {
// 如果失败暂停一段时间重试
log.Warnf("[Job] mission failed! %v", err)
log.Warnf("[Job] Retry after the task fails %d seconds! %s \n", (count+1)*5, str)
time.Sleep(time.Duration(count+1) * 5 * time.Second)
count = count + 1
goto LOOP
err := runWithExecutionLog(h.DB, h.JobCore, func() error {
var lastErr error
for count := 0; count < retryCount; count++ {
_, requestErr := pkg.Get(h.InvokeTarget)
if requestErr == nil {
return nil
}
lastErr = requestErr
log.Warnf("[Job] HTTP mission failed attempt=%d", count+1)
if count+1 < retryCount {
log.Warnf("[Job] Retry after the task fails %d seconds!\n", (count+1)*5)
time.Sleep(time.Duration(count+1) * 5 * time.Second)
}
}
return newExecutionFailure(executionErrorHTTPFailed, "HTTP 任务请求失败", lastErr)
})
if err != nil {
code, _ := publicExecutionFailure(err)
log.Errorf("[Job] JobCore %s failed error_code=%s", h.Name, code)
return
}
// 结束时间
endTime := time.Now()
@@ -102,6 +109,9 @@ func Setup(dbs map[string]*gorm.DB) {
fmt.Println(time.Now().Format(timeFormat), " [INFO] JobCore Starting...")
for k, db := range dbs {
if err := RecoverInterruptedExecutionLogs(db); err != nil {
log.Errorf("[Job] recover interrupted execution logs failed: %v", err)
}
sdk.Runtime.SetCrontab(k, cronjob.NewWithSeconds())
setup(k, db)
}
@@ -127,6 +137,7 @@ func setup(key string, db *gorm.DB) {
for i := 0; i < len(jobList); i++ {
if jobList[i].JobType == 1 {
j := &HttpJob{}
j.DB = db
j.InvokeTarget = jobList[i].InvokeTarget
j.CronExpression = jobList[i].CronExpression
j.JobId = jobList[i].JobId
@@ -0,0 +1,32 @@
package models
import "time"
const (
JobExecutionRunning = "running"
JobExecutionSucceeded = "succeeded"
JobExecutionFailed = "failed"
JobExecutionInterrupted = "interrupted"
JobTriggerScheduled = "scheduled"
)
// SysJobExecutionLog stores one scheduler invocation. Job arguments and raw
// provider responses are deliberately excluded from this audit record.
type SysJobExecutionLog struct {
ID uint64 `json:"id" gorm:"primaryKey;autoIncrement"`
ExecutionID string `json:"executionId" gorm:"size:36;not null;uniqueIndex:ux_sys_job_execution_id"`
JobID int `json:"jobId" gorm:"not null;index:idx_sys_job_execution_job_started,priority:1"`
JobNameSnapshot string `json:"jobName" gorm:"size:255;not null"`
InvokeTargetSnapshot string `json:"invokeTarget" gorm:"size:255;not null"`
TriggerType string `json:"triggerType" gorm:"size:16;not null"`
Status string `json:"status" gorm:"size:16;not null;index:idx_sys_job_execution_status_started,priority:1"`
StartedAt time.Time `json:"startedAt" gorm:"not null;index:idx_sys_job_execution_job_started,priority:2;index:idx_sys_job_execution_status_started,priority:2"`
FinishedAt *time.Time `json:"finishedAt,omitempty"`
DurationMS int64 `json:"durationMs" gorm:"not null;default:0"`
ErrorCode string `json:"errorCode,omitempty" gorm:"size:64;not null;default:''"`
ErrorMessage string `json:"errorMessage,omitempty" gorm:"size:500;not null;default:''"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (SysJobExecutionLog) TableName() string { return "sys_job_execution_log" }
+2
View File
@@ -24,6 +24,8 @@ func registerSysJobRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddlew
list := make([]models2.SysJob, 0)
return &list
}))
jobAPI := apis.SysJob{}
r.GET("/:id/execution-logs", actions.PermissionAction(), jobAPI.ListExecutionLogs)
r.GET("/:id", actions.PermissionAction(), actions.ViewAction(new(dto2.SysJobById), func() interface{} {
return &dto2.SysJobItem{}
}))
+107
View File
@@ -0,0 +1,107 @@
package service
import (
"context"
"errors"
"fmt"
"time"
"gorm.io/gorm"
"go-admin/app/jobs/models"
)
var (
ErrExecutionLogInvalidRequest = errors.New("invalid execution log request")
ErrExecutionLogJobNotFound = errors.New("scheduled job not found")
)
type ExecutionLogListRequest struct {
Page int
PageSize int
Status string
StartedFrom *time.Time
StartedTo *time.Time
}
type ExecutionLogJob struct {
JobID int `json:"jobId"`
JobName string `json:"jobName"`
InvokeTarget string `json:"invokeTarget"`
Deleted bool `json:"deleted"`
}
type ExecutionLogListResponse struct {
Job ExecutionLogJob `json:"job"`
Items []models.SysJobExecutionLog `json:"items"`
Total int64 `json:"total"`
Page int `json:"page"`
PageSize int `json:"pageSize"`
}
type ExecutionLogService struct{ db *gorm.DB }
func NewExecutionLogService(db *gorm.DB) *ExecutionLogService {
return &ExecutionLogService{db: db}
}
func (service *ExecutionLogService) List(ctx context.Context, jobID int, request ExecutionLogListRequest) (ExecutionLogListResponse, error) {
if service.db == nil || jobID < 1 {
return ExecutionLogListResponse{}, fmt.Errorf("%w: jobId 无效", ErrExecutionLogInvalidRequest)
}
if request.Page < 1 {
request.Page = 1
}
if request.PageSize < 1 {
request.PageSize = 20
}
if request.PageSize > 100 {
return ExecutionLogListResponse{}, fmt.Errorf("%w: pageSize 必须是 1 到 100 的整数", ErrExecutionLogInvalidRequest)
}
if request.Status != "" && !validExecutionStatus(request.Status) {
return ExecutionLogListResponse{}, fmt.Errorf("%w: status 无效", ErrExecutionLogInvalidRequest)
}
if request.StartedFrom != nil && request.StartedTo != nil && request.StartedFrom.After(*request.StartedTo) {
return ExecutionLogListResponse{}, fmt.Errorf("%w: 开始时间范围无效", ErrExecutionLogInvalidRequest)
}
var job models.SysJob
if err := service.db.WithContext(ctx).Unscoped().First(&job, jobID).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return ExecutionLogListResponse{}, ErrExecutionLogJobNotFound
}
return ExecutionLogListResponse{}, err
}
query := service.db.WithContext(ctx).Model(&models.SysJobExecutionLog{}).Where("job_id = ?", jobID)
if request.Status != "" {
query = query.Where("status = ?", request.Status)
}
if request.StartedFrom != nil {
query = query.Where("started_at >= ?", request.StartedFrom.UTC())
}
if request.StartedTo != nil {
query = query.Where("started_at <= ?", request.StartedTo.UTC())
}
var total int64
if err := query.Count(&total).Error; err != nil {
return ExecutionLogListResponse{}, err
}
items := make([]models.SysJobExecutionLog, 0, request.PageSize)
if err := query.Order("started_at DESC, id DESC").Offset((request.Page - 1) * request.PageSize).Limit(request.PageSize).Find(&items).Error; err != nil {
return ExecutionLogListResponse{}, err
}
return ExecutionLogListResponse{
Job: ExecutionLogJob{JobID: job.JobId, JobName: job.JobName, InvokeTarget: job.InvokeTarget, Deleted: job.DeletedAt.Valid},
Items: items, Total: total, Page: request.Page, PageSize: request.PageSize,
}, nil
}
func validExecutionStatus(status string) bool {
switch status {
case models.JobExecutionRunning, models.JobExecutionSucceeded, models.JobExecutionFailed, models.JobExecutionInterrupted:
return true
default:
return false
}
}
@@ -0,0 +1,87 @@
package service
import (
"context"
"errors"
"testing"
"time"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"go-admin/app/jobs/models"
)
func executionLogTestDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(&models.SysJob{}, &models.SysJobExecutionLog{}); err != nil {
t.Fatal(err)
}
return db
}
func TestExecutionLogListFiltersPaginatesAndKeepsDeletedJob(t *testing.T) {
db := executionLogTestDB(t)
job := models.SysJob{JobName: "测试任务", InvokeTarget: "TestTarget"}
if err := db.Create(&job).Error; err != nil {
t.Fatal(err)
}
base := time.Date(2026, 9, 2, 10, 0, 0, 0, time.UTC)
records := []models.SysJobExecutionLog{
{ExecutionID: "00000000-0000-4000-8000-000000000001", JobID: job.JobId, JobNameSnapshot: job.JobName, InvokeTargetSnapshot: job.InvokeTarget, TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionSucceeded, StartedAt: base},
{ExecutionID: "00000000-0000-4000-8000-000000000002", JobID: job.JobId, JobNameSnapshot: job.JobName, InvokeTargetSnapshot: job.InvokeTarget, TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionFailed, StartedAt: base.Add(time.Hour)},
{ExecutionID: "00000000-0000-4000-8000-000000000003", JobID: job.JobId, JobNameSnapshot: job.JobName, InvokeTargetSnapshot: job.InvokeTarget, TriggerType: models.JobTriggerScheduled, Status: models.JobExecutionFailed, StartedAt: base.Add(2 * time.Hour)},
}
if err := db.Create(&records).Error; err != nil {
t.Fatal(err)
}
from, to := base.Add(30*time.Minute), base.Add(3*time.Hour)
result, err := NewExecutionLogService(db).List(context.Background(), job.JobId, ExecutionLogListRequest{
Page: 1, PageSize: 1, Status: models.JobExecutionFailed, StartedFrom: &from, StartedTo: &to,
})
if err != nil {
t.Fatal(err)
}
if result.Total != 2 || len(result.Items) != 1 || result.Items[0].ExecutionID != records[2].ExecutionID {
t.Fatalf("unexpected filtered page: %+v", result)
}
if err := db.Delete(&job).Error; err != nil {
t.Fatal(err)
}
deleted, err := NewExecutionLogService(db).List(context.Background(), job.JobId, ExecutionLogListRequest{Page: 1, PageSize: 20})
if err != nil {
t.Fatal(err)
}
if !deleted.Job.Deleted || deleted.Job.JobName != job.JobName || deleted.Total != 3 {
t.Fatalf("deleted job history unavailable: %+v", deleted)
}
}
func TestExecutionLogListValidationAndNotFound(t *testing.T) {
db := executionLogTestDB(t)
service := NewExecutionLogService(db)
if _, err := service.List(context.Background(), 0, ExecutionLogListRequest{}); !errors.Is(err, ErrExecutionLogInvalidRequest) {
t.Fatalf("invalid job id error = %v", err)
}
if _, err := service.List(context.Background(), 1, ExecutionLogListRequest{PageSize: 101}); !errors.Is(err, ErrExecutionLogInvalidRequest) {
t.Fatalf("invalid page size error = %v", err)
}
job := models.SysJob{JobName: "测试任务", InvokeTarget: "TestTarget"}
if err := db.Create(&job).Error; err != nil {
t.Fatal(err)
}
if _, err := service.List(context.Background(), job.JobId, ExecutionLogListRequest{Status: "unknown"}); !errors.Is(err, ErrExecutionLogInvalidRequest) {
t.Fatalf("invalid status error = %v", err)
}
from, to := time.Now(), time.Now().Add(-time.Hour)
if _, err := service.List(context.Background(), job.JobId, ExecutionLogListRequest{StartedFrom: &from, StartedTo: &to}); !errors.Is(err, ErrExecutionLogInvalidRequest) {
t.Fatalf("invalid range error = %v", err)
}
if _, err := service.List(context.Background(), 999, ExecutionLogListRequest{}); !errors.Is(err, ErrExecutionLogJobNotFound) {
t.Fatalf("not found error = %v", err)
}
}
+1
View File
@@ -62,6 +62,7 @@ func (e *SysJob) StartJob(c *dto.GeneralGetDto) error {
if data.JobType == 1 {
var j = &jobs.HttpJob{}
j.DB = e.Orm.WithContext(context.Background())
j.InvokeTarget = data.InvokeTarget
j.CronExpression = data.CronExpression
j.JobId = data.JobId
+19 -1
View File
@@ -54,6 +54,12 @@ var (
}
)
// The synchronous Admin AI endpoints allow a provider timeout of up to 600
// seconds and the browser waits 610 seconds. Keep the HTTP server alive a
// little longer so it can return the domain response instead of truncating
// the connection and surfacing a proxy-level 502.
const minimumAPIWriteTimeout = 620 * time.Second
var AppRouters = make([]func(), 0)
func init() {
@@ -125,11 +131,15 @@ func run() error {
)
}
writeTimeout, err := validatedAPIWriteTimeout(config.ApplicationConfig.WriterTimeout)
if err != nil {
return err
}
srv := &http.Server{
Addr: fmt.Sprintf("%s:%d", config.ApplicationConfig.Host, config.ApplicationConfig.Port),
Handler: sdk.Runtime.GetEngine(),
ReadTimeout: time.Duration(config.ApplicationConfig.ReadTimeout) * time.Second,
WriteTimeout: time.Duration(config.ApplicationConfig.WriterTimeout) * time.Second,
WriteTimeout: writeTimeout,
}
go func() {
@@ -193,6 +203,14 @@ func run() error {
return nil
}
func validatedAPIWriteTimeout(seconds int) (time.Duration, error) {
timeout := time.Duration(seconds) * time.Second
if timeout < minimumAPIWriteTimeout {
return 0, fmt.Errorf("application writetimeout must be at least %s for synchronous AI requests", minimumAPIWriteTimeout)
}
return timeout, nil
}
type policyLoader interface {
LoadPolicy() error
}
+19
View File
@@ -0,0 +1,19 @@
package api
import (
"testing"
"time"
)
func TestValidatedAPIWriteTimeoutProtectsSynchronousAIRequests(t *testing.T) {
if _, err := validatedAPIWriteTimeout(2); err == nil {
t.Fatal("two-second write timeout must be rejected")
}
got, err := validatedAPIWriteTimeout(620)
if err != nil {
t.Fatalf("620-second write timeout should be accepted: %v", err)
}
if got != 620*time.Second {
t.Fatalf("write timeout = %s, want 620s", got)
}
}
@@ -1,25 +0,0 @@
package version_local
import (
"runtime"
goautomigrations "go-admin/app/goauto/migrations"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
"gorm.io/gorm"
)
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), migrateDeviceIdentityRecovery)
}
func migrateDeviceIdentityRecovery(db *gorm.DB, version string) error {
return db.Transaction(func(tx *gorm.DB) error {
if err := goautomigrations.Migrate(tx); err != nil {
return err
}
return tx.Create(&common.Migration{Version: version}).Error
})
}
@@ -1,32 +0,0 @@
package version_local
import (
"context"
"runtime"
goautomigrations "go-admin/app/goauto/migrations"
"go-admin/app/goauto/sybshop"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
"gorm.io/gorm"
)
// This migration repairs historical syb_shop.normalized_name values written
// before all shop-name write paths used sybshop.Normalize (#213).
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), migrateSYBShopNormalizedNameBackfill)
}
func migrateSYBShopNormalizedNameBackfill(db *gorm.DB, version string) error {
return db.Transaction(func(tx *gorm.DB) error {
if err := goautomigrations.Migrate(tx); err != nil {
return err
}
if _, err := sybshop.ReconcileNormalizedNames(context.Background(), tx); err != nil {
return err
}
return tx.Create(&common.Migration{Version: version}).Error
})
}
@@ -1,51 +0,0 @@
package version_local
import (
"testing"
goautomigrations "go-admin/app/goauto/migrations"
"go-admin/app/goauto/models"
common "go-admin/common/models"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func TestMigrateSYBShopNormalizedNameBackfillRepairsLegacyRows(t *testing.T) {
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
t.Fatalf("打开测试数据库失败: %v", err)
}
if err := db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatalf("创建迁移记录表失败: %v", err)
}
if err := goautomigrations.Migrate(db); err != nil {
t.Fatalf("创建 GoAuto 表失败: %v", err)
}
shop := models.SYBShop{DisplayName: "ABC店", NormalizedName: "ABC店", Enabled: true}
if err := db.Create(&shop).Error; err != nil {
t.Fatalf("创建历史店铺失败: %v", err)
}
const version = "1787983900000"
if err := migrateSYBShopNormalizedNameBackfill(db, version); err != nil {
t.Fatalf("执行回填迁移失败: %v", err)
}
var reloaded models.SYBShop
if err := db.First(&reloaded, shop.ID).Error; err != nil {
t.Fatalf("读取回填结果失败: %v", err)
}
if reloaded.NormalizedName != "abc店" {
t.Fatalf("迁移未回填规范化键: %q", reloaded.NormalizedName)
}
var records int64
if err := db.Model(&common.Migration{}).Where("version = ?", version).Count(&records).Error; err != nil {
t.Fatalf("读取迁移记录失败: %v", err)
}
if records != 1 {
t.Fatalf("迁移应留下一个版本记录,实际 %d", records)
}
}
@@ -1,34 +0,0 @@
package version_local
import (
"context"
"runtime"
goautomigrations "go-admin/app/goauto/migrations"
"go-admin/app/goauto/sybimport"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
"gorm.io/gorm"
)
// This migration repairs SYB rows whose two specification roles were stored
// in reverse before #216 recognized both observed source orders. The domain
// repair also removes only now-unreferenced, unmapped import values from the
// corresponding Shopee archive; human facts and raw source JSON are retained.
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), migrateSYBSpecRoleOrderRepair)
}
func migrateSYBSpecRoleOrderRepair(db *gorm.DB, version string) error {
return db.Transaction(func(tx *gorm.DB) error {
if err := goautomigrations.Migrate(tx); err != nil {
return err
}
if _, err := sybimport.RepairReversedSpecRoles(context.Background(), tx); err != nil {
return err
}
return tx.Create(&common.Migration{Version: version}).Error
})
}
@@ -1,62 +0,0 @@
package version_local
import (
"testing"
goautomigrations "go-admin/app/goauto/migrations"
"go-admin/app/goauto/models"
"go-admin/app/goauto/shopeeproduct"
common "go-admin/common/models"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func TestMigrateSYBSpecRoleOrderRepair(t *testing.T) {
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
t.Fatal(err)
}
if err := db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatal(err)
}
if err := goautomigrations.Migrate(db); err != nil {
t.Fatal(err)
}
specs, err := shopeeproduct.Marshal([]shopeeproduct.SpecDimension{
{Name: "颜色", Role: shopeeproduct.RoleColor, Values: []shopeeproduct.SpecValue{{Name: "均碼", Source: shopeeproduct.ValueSourceImport}}},
{Name: "尺码", Role: shopeeproduct.RoleSize, Values: []shopeeproduct.SpecValue{{Name: "黑色", Source: shopeeproduct.ValueSourceImport}}},
})
if err != nil {
t.Fatal(err)
}
product := models.ShopeeProduct{ShopeeItemID: "MIGRATION-ROLE", Title: "测试", ShopName: "测试店铺", Currency: "TWD", SpecsJSON: specs}
if err := db.Create(&product).Error; err != nil {
t.Fatal(err)
}
row := models.SYBProduct{OrderCode: "MIGRATION-ORDER", DetailID: 1, StockID: 1, ShopeeItemID: product.ShopeeItemID,
ShopeeProductID: &product.ID, ProductTitle: "测试", TargetColor: "均碼", TargetSize: "黑色", Quantity: 1,
ParseStatus: models.SYBParseStatusSuccess, RawJSON: `{"productSpec":"均碼,黑色"}`}
if err := db.Create(&row).Error; err != nil {
t.Fatal(err)
}
const version = "1787984000000"
if err := migrateSYBSpecRoleOrderRepair(db, version); err != nil {
t.Fatalf("执行规格角色修复迁移失败: %v", err)
}
if err := db.First(&row, row.ID).Error; err != nil {
t.Fatal(err)
}
if row.TargetColor != "黑色" || row.TargetSize != "均碼" {
t.Fatalf("迁移未修正规格角色: %q/%q", row.TargetColor, row.TargetSize)
}
var records int64
if err := db.Model(&common.Migration{}).Where("version = ?", version).Count(&records).Error; err != nil {
t.Fatal(err)
}
if records != 1 {
t.Fatalf("迁移应记录一次版本,实际 %d", records)
}
}
@@ -1,30 +0,0 @@
package version_local
import (
"context"
"runtime"
"go-admin/app/goauto/sybimport"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
"gorm.io/gorm"
)
// The first #216 local migration was exercised against live data before this
// follow-up case was known. Rerunning the idempotent repair lets already-
// migrated installations correct proven role reversals whose color remains
// uncertain, while fresh installations simply execute an empty second pass.
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), migrateSYBSpecRoleOrderUncertainRepair)
}
func migrateSYBSpecRoleOrderUncertainRepair(db *gorm.DB, version string) error {
return db.Transaction(func(tx *gorm.DB) error {
if _, err := sybimport.RepairReversedSpecRoles(context.Background(), tx); err != nil {
return err
}
return tx.Create(&common.Migration{Version: version}).Error
})
}

Some files were not shown because too many files have changed in this diff Show More