Compare commits
18
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 ila
|
1d19b2b92f | ||
|
ila
|
35ed692c4b | ||
|
ila
|
65b50d5038 | ||
|
ila
|
fb256d4916 | ||
|
ila
|
61d3f63c69 | ||
|
ila
|
cd2b893bba | ||
|
ila
|
85b8e24429 | ||
|
ila
|
325849816e | ||
|
ila
|
ec5ec2db35 | ||
|
ila
|
0328505b77 | ||
|
ila
|
527d8af0c4 | ||
|
ila
|
a5a8c3c72d | ||
|
ila
|
8c0946af9f | ||
|
ila
|
33182e584b | ||
|
ila
|
ad31ea4225 | ||
|
ila
|
8310438bee | ||
|
ila
|
b18f9cc4a5 | ||
|
ila
|
a55708cd37 |
@@ -280,3 +280,8 @@ MVP 内所有单元任务通过后才能做 MVP 集成验收；MVP 通过后才
|
||||
- #18 登录日志与操作审计已通过用户验收：schema v2 显式迁移；日志只保存白名单字段，禁止保存凭据、请求/响应正文及私人学习内容。仅管理员查询，默认保留 90 天；启动/每小时及 `python scripts/server.py audit-cleanup` 仅清理两张审计表的过期记录。
|
||||
- #3 独立小样位于 `spikes/english/`，使用 `.local/nlp-venv/Scripts/python.exe`（3.12.12）运行；固定 spaCy 3.8.7、英语模型 3.8.0、NLTK 3.9.2、WordNet 3.0。资源仅显式准备时下载，摘要见 resources.json。不得把本机无账号的实验接口用于正式学习端；后续集成仍需 Go 授权、数据归属和任务设计。原文不归一化，位置区分 cp/UTF-8/UTF-16，lemma 不自动合并学习状态。
|
||||
- #4 独立阅读选择小样位于 `spikes/selection/`，`python spikes/selection/serve.py` 默认仅本机 5184。桌面鼠标/键盘与 11 项测试已验证，#4 已获用户验收并关闭；真实手机长按/手柄/滚动详细证据仍未提供；禁止把窄屏桌面当作真机验收。Intl.Segmenter 只用于 UI 范围验证，不替代 #3 NLP；释义保存只在内存。固定 LinguaCafe 源码对照和与 v1 的差异记录见架构 Wiki。
|
||||
- 2026-09-11 用户确认正式 NLP/词典采用全 Go。#5 已验收并合入 main；#6 使用 Go WordNet 解析和词形候选、Go Unicode 原文分片、schema v4 共享词典资源表，不调用 Python NLP。WordNet 3.0 ZIP 来源与摘要见 `server/wordnet-resource.json`，许可保留在 `server/WORDNET-LICENSE.txt`。词形候选不等于上下文消歧，不自动合并个人学习状态；#3 Python 小样只保留历史验证。当前词典仅英语释义，个人释义输入为临时草稿，持久化归 #7。
|
||||
- 2026-09-11 用户确认 #7 个人词条口径（三项由 Agent 定案）：身份为「学习者＋语言＋规范化词形」，大小写合并但**不按 lemma/候选合并**（`dog` 与 `dogs` 是两条记录）；首次保存默认「新词」；状态为 新词/学习中/已知/忽略，只有「学习中」带 1～7 级，对应原版 stage 2/1/0/-1～-7；例句只保存手输内容，不自动关联原文句子。schema v5 新增 `lexgo_terms`（唯一键加状态/等级检查约束），个人释义与共享词典分离且不进入审计日志；等级编辑 UI 归 #8/#12。
|
||||
- 2026-09-11 用户确认 #10 编辑与删除口径：可改名、改章节标题、编辑章节正文；**只有正文变化才重新处理**，重复保存或改回原内容不新建任务，只改标题不改状态。版本键是 `content_sha256`：任务只在与章节版本一致时才能影响章节，过期版本任务标为 `error_reason=superseded` 且**完全不触碰章节**（认领、恢复扫描、重试都按版本裁决）；存储文本重算 SHA 与存储 SHA 不一致时按 `content_changed` 失败。删除为事务内硬删除 + 外键级联，删章后重排序号；个人词条、复习排期与作答记录**不随删除清理**。回收站/撤销、批量操作、章节跨书移动与语言变更不在范围内。编辑器行尾归一为 LF 是已知边界。
|
||||
- 2026-09-11 用户确认 #9 TXT 导入口径：只接受 UTF-8（允许可选 BOM，解码时剥离且不进入原文），非法字节整体拒绝、不使用替换字符；UTF-16 按 BOM 识别后明确拒绝，GB18030 等按非法 UTF-8 拒绝。文件字节上限 2 MiB，之后仍套用单章 100000 码点上限；换行与空白不归一化。文件只在内存中解码、不创建临时文件，客户端文件名不参与任何路径也不入库。解码后交给既有 `PasteBook`/`PasteChapter`，分章（一次提交一章）、`requestId` 幂等与任务恢复与粘贴一致；不改 schema。EPUB/PDF/字幕、UTF-16 转码、按空行自动分章与断点续传不在范围内。
|
||||
- 2026-09-11 用户确认 #8 到期单词复习决策表：固定间隔表 1/2/4/7/15/30/60 天，答对升级封顶 7、答错降级最低 1、再学一次不改等级，答错与再学立即回队；已知/忽略不入队，新保存的词立即到期，显式「学习中 level N」排 now+间隔[N]；只有新建或状态/等级实际变化才移动复习时间，编辑释义或例句保留原排期，保存未提及等级时保留已获得等级。到期判定用 UTC 绝对时刻（`due_at ≤ now`），不引入本地日边界。作答按 `answerId` 去重并以 `expectedDueAt` 判定过期标签页，重复提交、网络重发与双标签页都不得重复更新次数与间隔（作答响应 `result` 只取 applied/stale，重放另用 `duplicate` 标记并返回首次结果）；`correct_count` 只计答对，`wrong_count` 计答错与再学。短语复习归 #11，进度统计归 #13，不做策略配置 UI（X11）、练习模式（X08）与 FSRS。
|
||||
|
||||
@@ -2,13 +2,13 @@
|
||||
|
||||
面向自托管场景的阅读式语言学习项目，规划提供内容导入、阅读查词、词汇与短语、复习和实例管理。
|
||||
|
||||
已确认：**DevHarness 轻量模式、MySQL 8、go-admin 管理端**。工程基础 #2 已通过验收：两端用户名登录、学习账号管理、可撤销会话和本人英语空空间。管理端基于指定 go-admin/go-admin-ui 选用模块，学习端为独立 Vue 3 + TypeScript + Vite 工程，共用 Go 后端和 MySQL 8.4.3。#18 登录日志与操作审计已通过用户验收，支持管理员查询和 90 天保留清理。阅读、导入、词典与复习尚未实现。MVP 定位为“支持多账号、数据独立的自托管学习工具”，先邀请少量用户使用；F01～F12 已确认，X 系列后置。
|
||||
已确认：**DevHarness 轻量模式、MySQL 8、go-admin 管理端**。工程基础 #2 已通过验收：两端用户名登录、学习账号管理、可撤销会话和本人英语空空间。管理端基于指定 go-admin/go-admin-ui 选用模块，学习端为独立 Vue 3 + TypeScript + Vite 工程，共用 Go 后端和 MySQL 8.4.3。#18 登录日志与操作审计已通过用户验收，支持管理员查询和 90 天保留清理。#5 粘贴导入与章节原文阅读已验收并合入 main；#6 全 Go 英语词典与点词查义已通过用户验收，PR #25 已合入 main；#7 个人词条与学习状态已通过用户验收，PR #26 已合入 main；#8 到期单词复习已通过用户验收，PR #27 已合入 main；#9 上传 TXT、校验编码后导入本人书库已通过用户验收，PR #28 已合入 main。短语、进度与词汇库仍未实现。MVP 定位为“支持多账号、数据独立的自托管学习工具”，先邀请少量用户使用；F01～F12 已确认，X 系列后置。
|
||||
|
||||
- [文档入口](docs/README.md) · [线上 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Home)
|
||||
- [英语分词与离线词典验证小样](spikes/english/README.md)（#3 已验收，独立本机入口）
|
||||
- [阅读选择验证小样](spikes/selection/README.md)（#4 已验收，真机详细测试证据缺口保留）
|
||||
- [项目档案](docs/00-project-profile.md) · [需求总览](docs/09-product-requirements-overview.md)
|
||||
- [工作量估算](docs/10-workload-estimate.md)：#2、#3、#4、#18 已验收，剩余 #5～#15 与新增 #21 规划参考 44～71 人日；后续结合集成结果重估，旧全量研究仅供参考。
|
||||
- [工作量估算](docs/10-workload-estimate.md)：#2、#3、#4、#5、#6、#7、#8、#9、#18 已验收，原规划中的 #5 已完成；剩余 #10～#15 与新增 #21、#24 按工单复核；后续结合集成结果重估，旧全量研究仅供参考。
|
||||
- [四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16)：14 张单元工单，工程基础 → 技术验证 → 首条学习闭环 → 补齐 MVP；原型 v1 已获用户验收。两端使用账号（用户名）＋密码登录，不要求邮箱。
|
||||
- [原型工单 #1](https://git.ilapage.cn/OPC/lexgo/issues/1)：Quant-UX 桌面/手机原型 v1，预览入口与审核记录见工单及需求总览。
|
||||
- [工作流](docs/01-workflow.md) · [开发与验证](docs/04-local-development-and-verification.md)
|
||||
@@ -19,7 +19,15 @@
|
||||
|
||||
本地入口：学习端 http://127.0.0.1:5173，管理端 http://127.0.0.1:5174。完整安装与测试命令见[开发与验证](docs/04-local-development-and-verification.md)。账号使用用户名，无需邮箱；没有随代码交付的默认密码。
|
||||
|
||||
本机三个服务已由 `D:/supervisord/supervisord.conf` 中的 lexgo-learner、lexgo-admin、lexgo-api 托管，使用时不要重复手动启动同端口。当前 schema v2；从 #2 升级时停止 API，执行 build、migrate 后再启动。审计过期清理可执行 `python scripts/server.py audit-cleanup`，只影响超过 90 天的日志。
|
||||
本机三个服务已由 `D:/supervisord/supervisord.conf` 中的 lexgo-learner、lexgo-admin、lexgo-api 托管，使用时不要重复手动启动同端口。当前 schema v4；从 #2/#18 升级时停止 API，执行 build、migrate 后再启动。审计过期清理可执行 `python scripts/server.py audit-cleanup`，只影响超过 90 天的日志。
|
||||
|
||||
## 英语词典与点词查义（#6）
|
||||
|
||||
正式后端为纯 Go，`lexgo.exe serve` 不启动 Python NLP。开发辅助脚本与历史小样保留。
|
||||
|
||||
管理员在“英语词典”页下载并导入指定 WordNet 3.0 ZIP；来源、固定摘要及许可见 [资源清单](server/wordnet-resource.json)。词典提供英语释义，保存在 MySQL 中；导入失败保留当前资源，重复导入不新增资源并重新启用。也可在管理页停用。
|
||||
|
||||
学习者打开本人就绪章节，点击词语或聚焦后按 Enter/空格查询；Escape/关闭返回阅读。精确词形优先，未命中再查规则候选，例如 `went → go`。个人释义当前是未保存的临时草稿，保存功能在 #7 实现。真机测试证据尚未补齐。
|
||||
|
||||
## 文档与治理
|
||||
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
export const MAX_DICTIONARY_BYTES = 32 * 1024 * 1024
|
||||
|
||||
export function createDictionaryLoader(session, state) {
|
||||
let revision = 0
|
||||
function invalidate() {
|
||||
revision++
|
||||
Object.assign(state, { items: [], supported: null, loading: false, saving: false, error: '', notice: '' })
|
||||
}
|
||||
async function perform(operation, saving, apply) {
|
||||
const current = ++revision
|
||||
const generation = session.state.generation
|
||||
const stale = () => current !== revision || generation !== session.state.generation
|
||||
Object.assign(state, { loading: !saving, saving, error: '', notice: '' })
|
||||
try {
|
||||
const result = await operation()
|
||||
if (stale()) return false
|
||||
apply(result)
|
||||
return true
|
||||
} catch (error) {
|
||||
if (!stale()) state.error = error instanceof Error ? error.message : '词典操作失败，请重试'
|
||||
return false
|
||||
} finally {
|
||||
if (!stale()) { state.loading = false; state.saving = false }
|
||||
}
|
||||
}
|
||||
return {
|
||||
invalidate,
|
||||
load() {
|
||||
return perform(() => session.listDictionaries(), false, result => {
|
||||
state.items = result.items
|
||||
state.supported = result.supported
|
||||
})
|
||||
},
|
||||
import(form) {
|
||||
return perform(() => session.importDictionary(form), true, result => {
|
||||
state.items = [result.resource]
|
||||
state.notice = result.duplicate ? '该词典已存在，已启用，未重复导入。' : '词典导入成功。'
|
||||
})
|
||||
},
|
||||
toggle(id, enabled) {
|
||||
return perform(() => session.setDictionaryEnabled(id, enabled), true, result => {
|
||||
state.items = state.items.map(item => item.id === result.resource.id ? result.resource : item)
|
||||
state.notice = result.resource.enabled ? '词典已启用。' : '词典已停用。'
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
<div class="brand">{{ sidebar.opened ? 'LexGo 管理' : 'LG' }}</div>
|
||||
<el-menu :default-active="$route.path" :collapse="!sidebar.opened" router>
|
||||
<el-menu-item index="/accounts"><span>账号管理</span></el-menu-item>
|
||||
<el-menu-item index="/dictionaries"><span>英语词典</span></el-menu-item>
|
||||
<el-menu-item index="/login-logs"><span>登录日志</span></el-menu-item>
|
||||
<el-menu-item index="/operation-logs"><span>操作日志</span></el-menu-item>
|
||||
</el-menu>
|
||||
|
||||
@@ -4,11 +4,13 @@ import Layout from '../layout/index.vue'
|
||||
import Login from '../views/Login.vue'
|
||||
import Accounts from '../views/Accounts.vue'
|
||||
import AuditLogs from '../views/AuditLogs.vue'
|
||||
import Dictionaries from '../views/Dictionaries.vue'
|
||||
const router = createRouter({ history: createWebHashHistory(), routes: [
|
||||
{ path: '/login', component: Login },
|
||||
{ path: '/', component: Layout, children: [
|
||||
{ path: '', redirect: '/accounts' },
|
||||
{ path: 'accounts', component: Accounts, meta: { title: '账号管理' } },
|
||||
{ path: 'dictionaries', component: Dictionaries, meta: { title: '英语词典' } },
|
||||
{ path: 'login-logs', component: AuditLogs, props: { kind: 'login' }, meta: { title: '登录日志' } },
|
||||
{ path: 'operation-logs', component: AuditLogs, props: { kind: 'operation' }, meta: { title: '操作日志' } }
|
||||
] },
|
||||
|
||||
+17
-2
@@ -20,10 +20,11 @@ export function createSession({ fetch, storage, changed = () => {} }) {
|
||||
if (generation !== state.generation) throw new Error('会话已变化，请重新操作')
|
||||
}
|
||||
async function request(path, method = 'GET', body, token = state.token, generation = state.generation) {
|
||||
const multipart = typeof FormData !== 'undefined' && body instanceof FormData
|
||||
const result = await fetch('/api/v1' + path, {
|
||||
method,
|
||||
headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: 'Bearer ' + token } : {}) },
|
||||
...(body === undefined ? {} : { body: JSON.stringify(body) }),
|
||||
headers: { ...(!multipart ? { 'Content-Type': 'application/json' } : {}), ...(token ? { Authorization: 'Bearer ' + token } : {}) },
|
||||
...(body === undefined ? {} : { body: multipart ? body : JSON.stringify(body) }),
|
||||
cache: 'no-store'
|
||||
})
|
||||
const payload = await result.json()
|
||||
@@ -84,6 +85,20 @@ export function createSession({ fetch, storage, changed = () => {} }) {
|
||||
} catch (error) { if (generation === state.generation) clear(); throw error }
|
||||
},
|
||||
async logout() { const token = state.token; clear(); await revoke(token) },
|
||||
async listDictionaries() {
|
||||
authorized()
|
||||
return request('/dictionaries')
|
||||
},
|
||||
async importDictionary(form) {
|
||||
authorized()
|
||||
if (!(form instanceof FormData)) throw new Error('请选择词典文件')
|
||||
return request('/dictionaries/import', 'POST', form)
|
||||
},
|
||||
async setDictionaryEnabled(id, enabled) {
|
||||
authorized()
|
||||
if (!Number.isSafeInteger(id) || id <= 0 || typeof enabled !== 'boolean') throw new Error('词典参数无效')
|
||||
return request('/dictionaries/' + id, 'PATCH', { enabled })
|
||||
},
|
||||
async queryAuditLogs(kind, filters) {
|
||||
authorized()
|
||||
const generation = state.generation
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
<template>
|
||||
<basic-layout><template #wrapper>
|
||||
<el-card>
|
||||
<div class="toolbar"><div><h1>英语词典</h1><p class="subtle">供所有学习账号查询</p></div><el-button :disabled="busy" @click="reload">刷新</el-button></div>
|
||||
<el-alert v-if="resources.error" :title="resources.error" type="error" :closable="false" show-icon />
|
||||
<el-alert v-if="resources.notice" :title="resources.notice" type="success" :closable="false" show-icon />
|
||||
<el-table v-loading="resources.loading" :data="resources.items" border :empty-text="resources.error ? '加载失败，请重试' : '尚未导入词典'">
|
||||
<el-table-column label="名称" prop="name" min-width="175" />
|
||||
<el-table-column label="语言" width="80"><template #default>英语</template></el-table-column>
|
||||
<el-table-column label="版本" prop="version" width="80" />
|
||||
<el-table-column label="来源" prop="source" min-width="160" show-overflow-tooltip />
|
||||
<el-table-column label="格式" prop="format" min-width="160" />
|
||||
<el-table-column label="状态" width="95"><template #default="scope"><el-tag :type="scope.row.status === 'ready' ? 'success' : 'info'">{{ statusLabel(scope.row.status) }}</el-tag></template></el-table-column>
|
||||
<el-table-column label="词条数" prop="entryCount" min-width="95" />
|
||||
<el-table-column label="操作" width="100"><template #default="scope"><el-button :disabled="busy" link type="primary" @click="toggle(scope.row)">{{ scope.row.enabled ? '停用' : '启用' }}</el-button></template></el-table-column>
|
||||
</el-table>
|
||||
</el-card>
|
||||
<el-card class="import-card">
|
||||
<h2>导入词典</h2>
|
||||
<el-form class="dictionary-form" label-position="top" @submit.prevent="upload">
|
||||
<el-form-item label="名称"><el-input v-model="form.name" maxlength="120" :disabled="busy" /></el-form-item>
|
||||
<div class="form-row">
|
||||
<el-form-item label="语言"><el-input model-value="英语" disabled /></el-form-item>
|
||||
<el-form-item label="版本"><el-input v-model="form.version" readonly /></el-form-item>
|
||||
</div>
|
||||
<el-form-item label="来源"><el-input v-model="form.source" readonly /></el-form-item>
|
||||
<el-form-item label="格式"><el-input v-model="form.format" readonly /></el-form-item>
|
||||
<el-form-item label="词典文件">
|
||||
<input id="dictionary-file" ref="fileInput" type="file" accept=".zip,application/zip" aria-label="词典文件" :disabled="busy" @change="selectFile">
|
||||
</el-form-item>
|
||||
<p class="subtle">WordNet 3.0 ZIP · 上限 32 MiB · 英语释义</p>
|
||||
<p v-if="resources.supported?.source"><a :href="downloadURL" target="_blank" rel="noopener noreferrer">下载支持的词典文件</a></p>
|
||||
<p class="subtle">导入失败时保留当前词典。</p>
|
||||
<p v-if="fileError" role="alert" class="file-error">{{ fileError }}</p>
|
||||
<el-button type="primary" native-type="submit" :loading="resources.saving" :disabled="resources.loading || !file">导入并启用</el-button>
|
||||
</el-form>
|
||||
</el-card>
|
||||
</template></basic-layout>
|
||||
</template>
|
||||
<script>
|
||||
import BasicLayout from '../layout/BasicLayout.vue'
|
||||
import { session } from '../store'
|
||||
import { createDictionaryLoader, MAX_DICTIONARY_BYTES } from '../dictionaries.mjs'
|
||||
const downloadURL = 'https://raw.githubusercontent.com/nltk/nltk_data/96f9b3252457a2b97e52aec64c3dfceeb5c312d5/packages/corpora/wordnet.zip'
|
||||
export default {
|
||||
name: 'DictionariesView',
|
||||
components: { BasicLayout },
|
||||
data: () => ({
|
||||
resources: { items: [], supported: null, loading: false, saving: false, error: '', notice: '' },
|
||||
form: { name: 'Princeton WordNet', language: 'en', version: '3.0', source: downloadURL, format: 'wordnet-3.0-zip' },
|
||||
file: null, fileError: '', downloadURL
|
||||
}),
|
||||
computed: { busy() { return this.resources.loading || this.resources.saving } },
|
||||
watch: { '$store.state.generation': { flush: 'sync', handler() { this.loader.invalidate(); this.clearFile() } } },
|
||||
created() { this.loader = createDictionaryLoader(session, this.resources) },
|
||||
mounted() { this.reload() },
|
||||
beforeUnmount() { this.loader.invalidate(); this.file = null },
|
||||
methods: {
|
||||
reload() { return this.loader.load() },
|
||||
statusLabel(status) { return { ready: '可用', disabled: '已停用', unavailable: '不可用' }[status] || '不可用' },
|
||||
clearFile() { this.file = null; this.fileError = ''; if (this.$refs.fileInput) this.$refs.fileInput.value = '' },
|
||||
selectFile(event) {
|
||||
this.fileError = ''
|
||||
const selected = event.target.files?.[0]
|
||||
this.file = null
|
||||
if (!selected) return
|
||||
if (!selected.name.toLowerCase().endsWith('.zip') || selected.size === 0 || selected.size > MAX_DICTIONARY_BYTES) {
|
||||
this.fileError = '请选择不超过 32 MiB 的 ZIP 文件。'
|
||||
event.target.value = ''
|
||||
return
|
||||
}
|
||||
this.file = selected
|
||||
},
|
||||
async upload() {
|
||||
if (this.busy || !this.file) return
|
||||
this.fileError = ''
|
||||
if (!this.form.name.trim() || !this.form.source.trim()) { this.fileError = '请填写名称和来源。'; return }
|
||||
const data = new FormData()
|
||||
for (const [key, value] of Object.entries(this.form)) data.append(key, value.trim())
|
||||
data.append('file', this.file)
|
||||
if (await this.loader.import(data)) this.clearFile()
|
||||
},
|
||||
toggle(resource) { return this.loader.toggle(resource.id, !resource.enabled) }
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<style scoped>
|
||||
.toolbar { display:flex; align-items:center; justify-content:space-between; gap:16px; margin-bottom:20px; }
|
||||
h1 { font-size:20px; margin:0 0 8px; }
|
||||
h2 { font-size:18px; margin:0 0 24px; }
|
||||
.subtle { color:#606266; font-size:13px; line-height:1.6; }
|
||||
.toolbar p { margin:0; }
|
||||
.import-card { margin-top:20px; }
|
||||
.dictionary-form { max-width:600px; }
|
||||
.form-row { display:flex; gap:16px; }
|
||||
.form-row .el-form-item { flex:1; min-width:0; }
|
||||
.file-error { color:#b42318; }
|
||||
.el-alert { margin-bottom:16px; }
|
||||
input[type=file] { max-width:100%; }
|
||||
a { color:#176b63; }
|
||||
</style>
|
||||
@@ -0,0 +1,55 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import { createSession } from '../src/session.mjs'
|
||||
|
||||
const response = (data, status = 200) => ({ ok: status < 400, status, json: async () => ({ data, msg: '导入失败' }) })
|
||||
function setup() {
|
||||
const pending = []
|
||||
const session = createSession({ fetch: (url, options) => new Promise(resolve => pending.push({ url, options, resolve })), storage: { getItem() {}, setItem() {}, removeItem() {} } })
|
||||
session.state.user = { id: 1, username: 'fixture.admin', role: 'admin' }
|
||||
session.state.token = 'fictional-test-token'
|
||||
return { session, pending }
|
||||
}
|
||||
|
||||
test('dictionary upload sends multipart with authorization and lets browser set boundary', async () => {
|
||||
const { session, pending } = setup()
|
||||
const form = new FormData()
|
||||
form.append('name', 'Princeton WordNet')
|
||||
form.append('file', new Blob(['fictional-archive']), 'wordnet.zip')
|
||||
assert.equal(typeof session.importDictionary, 'function')
|
||||
const request = session.importDictionary(form)
|
||||
assert.equal(pending[0].url, '/api/v1/dictionaries/import')
|
||||
assert.equal(pending[0].options.method, 'POST')
|
||||
assert.equal(pending[0].options.body, form)
|
||||
assert.equal(pending[0].options.headers['Content-Type'], undefined)
|
||||
assert.equal(pending[0].options.headers.Authorization, 'Bearer fictional-test-token')
|
||||
pending[0].resolve(response({ resource: { id: 1 }, duplicate: false }))
|
||||
assert.deepEqual(await request, { resource: { id: 1 }, duplicate: false })
|
||||
})
|
||||
|
||||
test('dictionary read and toggle follow API contract, ordinary accounts cannot mutate', async () => {
|
||||
const { session, pending } = setup()
|
||||
assert.equal(typeof session.listDictionaries, 'function')
|
||||
const read = session.listDictionaries()
|
||||
assert.equal(pending[0].url, '/api/v1/dictionaries')
|
||||
pending[0].resolve(response({ items: [] }))
|
||||
await read
|
||||
const toggle = session.setDictionaryEnabled(1, false)
|
||||
assert.equal(pending[1].url, '/api/v1/dictionaries/1')
|
||||
assert.deepEqual(JSON.parse(pending[1].options.body), { enabled: false })
|
||||
pending[1].resolve(response({ resource: { id: 1, enabled: false } }))
|
||||
await toggle
|
||||
session.state.user.role = 'learner'
|
||||
await assert.rejects(session.importDictionary(new FormData()), /管理员/)
|
||||
await assert.rejects(session.setDictionaryEnabled(1, true), /管理员/)
|
||||
assert.equal(pending.length, 2)
|
||||
})
|
||||
|
||||
test('dictionary response cannot cross a logout or account switch', async () => {
|
||||
const { session, pending } = setup()
|
||||
assert.equal(typeof session.importDictionary, 'function')
|
||||
const request = session.importDictionary(new FormData())
|
||||
session.clear()
|
||||
pending[0].resolve(response({ resource: { id: 1 }, duplicate: false }))
|
||||
await assert.rejects(request, /会话已变化/)
|
||||
})
|
||||
@@ -0,0 +1,57 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
|
||||
async function fixture() {
|
||||
const { createDictionaryLoader } = await import('../src/dictionaries.mjs')
|
||||
const pending = []
|
||||
const later = () => new Promise((resolve, reject) => pending.push({ resolve, reject }))
|
||||
const session = { state: { generation: 1 }, listDictionaries: later, importDictionary: later, setDictionaryEnabled: later }
|
||||
const state = { items: [{ id: 1, name: 'Existing', enabled: true }], supported: null, loading: false, saving: false, error: '', notice: '' }
|
||||
return { loader: createDictionaryLoader(session, state), state, session, pending }
|
||||
}
|
||||
|
||||
test('failed upload preserves the existing resource and exposes the error', async () => {
|
||||
const { loader, state, pending } = await fixture()
|
||||
const action = loader.import(new FormData())
|
||||
assert.equal(state.saving, true)
|
||||
pending[0].reject(new Error('文件摘要不匹配'))
|
||||
assert.equal(await action, false)
|
||||
assert.equal(state.items[0].name, 'Existing')
|
||||
assert.equal(state.error, '文件摘要不匹配')
|
||||
assert.equal(state.saving, false)
|
||||
})
|
||||
|
||||
test('accepted mutation replaces old state directly, without a second refresh dependency', async () => {
|
||||
const { loader, state, pending } = await fixture()
|
||||
const action = loader.toggle(1, false)
|
||||
pending[0].resolve({ resource: { id: 1, name: 'Existing', enabled: false, status: 'disabled' } })
|
||||
assert.equal(await action, true)
|
||||
assert.equal(state.items[0].enabled, false)
|
||||
assert.equal(pending.length, 1)
|
||||
})
|
||||
|
||||
test('an old list cannot overwrite a newer import and duplicate import is explicit', async () => {
|
||||
const { loader, state, pending } = await fixture()
|
||||
const read = loader.load()
|
||||
const action = loader.import(new FormData())
|
||||
pending[1].resolve({ resource: { id: 1, name: 'Imported', enabled: true }, duplicate: true })
|
||||
await action
|
||||
pending[0].resolve({ items: [{ id: 1, name: 'Old' }] })
|
||||
await read
|
||||
assert.equal(state.items[0].name, 'Imported')
|
||||
assert.match(state.notice, /已存在/)
|
||||
})
|
||||
|
||||
test('page exit or session change discards late responses', async () => {
|
||||
const { loader, state, pending, session } = await fixture()
|
||||
const action = loader.import(new FormData())
|
||||
loader.invalidate()
|
||||
pending[0].resolve({ resource: { id: 1, name: 'Late' } })
|
||||
assert.equal(await action, false)
|
||||
assert.deepEqual(state.items, [])
|
||||
const read = loader.load()
|
||||
session.state.generation++
|
||||
pending[1].resolve({ items: [{ id: 1, name: 'Other session' }] })
|
||||
await read
|
||||
assert.deepEqual(state.items, [])
|
||||
})
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Project-Profile
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Project-Profile.-
|
||||
wiki_revision: 40ae5e3a0521195da4f1a9cd9f42beec15da20d4
|
||||
synchronized_at: 2026-09-10T12:43:19Z
|
||||
wiki_revision: b85c7f4650aae95b1429c0b4f7d1412c440731ce
|
||||
synchronized_at: 2026-09-13T14:36:36Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# LexGo 项目档案
|
||||
@@ -37,7 +37,7 @@ LinguaCafe 参考基线由现有调研记录为 `c1ea298ce40c65b9dd33e9b26fd2e52
|
||||
|---|---|---|---|
|
||||
| D01 | 管理端底座 | **已确认使用 `D:/github_project/goadmin` 的 go-admin + go-admin-ui**，依据用户指令“管理端用D:\github_project\goadmin”；替代 gin-vue-admin 建议 | M0 验证指定本地版本配套、MySQL 8 及数据隔离，不再比较管理底座 |
|
||||
| D02 | 数据库 | **已确认 MySQL 8**；依据 2026-09-10 用户原话“使用mysql8”。覆盖两份分析的数据库分歧，具体小版本待基线验证锁定 | 按单一 MySQL 8 数据层估算，不做 PostgreSQL 双库兼容 |
|
||||
| D03 | NLP | 保留 Python 为建议，纯 Go 是否硬约束未确认 | 首发语言质量与开发量 |
|
||||
| D03 | NLP | **用户于2026-09-11确认正式产品全 Go**；Python小样仅历史验证 | 使用词典词形候选，不承诺上下文消歧 |
|
||||
| D04 | 默认学习语言 | 用户已确认英语；中文/日语后续需独立范围与验收 | 分词、读音和 UI |
|
||||
| D05 | 用户范围 | **已确认：支持多账号、数据独立的自托管学习工具，先邀请少量用户使用** | 首版按多用户归属和隔离设计；不自动加入公开注册、邀请链接或组织租户 |
|
||||
| D06 | 旧数据 | 全量迁移是否需要未确认，CSV 与完整迁移不同 | 迁移另估 |
|
||||
@@ -136,3 +136,25 @@ server 是 go-admin 的选用模块接入：原样保留 SysUser、SysDept、必
|
||||
## 日志审计基线（#18）
|
||||
|
||||
管理端新增两个审计列表，后端为 LexGo 自有日志模型与接口，参照 go-admin 模块布局但不复制其原始参数/响应持久化逻辑。schema v2 新增 lexgo_login_logs、lexgo_operation_logs；保留 90 天，启动和每小时分批清理，也可显式 audit-cleanup。新建/重置密码均为 6～72 UTF-8 字节；初始 bootstrap 保留 10 字节下限。#2 已验收，#18 已验收；默认模块中的其他候选未纳入。
|
||||
|
||||
|
||||
## 全 Go 正式架构决定（2026-09-11，#6）
|
||||
|
||||
用户已明确选择全 Go：正式英语分词、原文位置映射、本地词典解析和词形候选查询由 Go 后端完成，不运行 Python NLP 服务。此前“Python 建议/全 Go 未决”仅为历史决策记录，由本决定覆盖；spikes/english 保留历史验证，不接入产品。#6 按该方向实施，当前方案见工单最新启动评论；WordNet 3.0 仍为首个资源（英语释义），词形规则候选不等同于 spaCy 上下文消歧，原文及个人学习状态不按候选合并。
|
||||
|
||||
|
||||
## #6 当前工程状态（2026-09-11）
|
||||
|
||||
#5已验收且相关前置PR均已合入main。#6正式词典/点词查义已按全Go实现，已于2026-09-11通过用户验收，PR #25已合入main；server schema v4，管理端新增英语词典页，学习端加入Go分片及查词面板。正式Go进程不依赖Python NLP，WordNet包随数据库持久化。#3小样仍是历史验证；保存个人释义和状态归#7，音频封面#21、列表优化#24尚未实施。
|
||||
|
||||
## #7 当前工程状态（2026-09-11）
|
||||
|
||||
#6已通过用户验收并合入main。#7个人词条、学习状态与阅读器高亮已按用户确认口径实现，并于2026-09-11通过用户验收，PR #26已fast-forward-only合入main；server schema v5新增lexgo_terms，学习端面板可保存释义、例句与状态并按状态高亮，同一词形在本人其他章节显示一致。个人释义与共享词典分离，且不进入审计日志。词汇库#12、短语#11、到期复习#8、进度#13、音频封面#21与列表优化#24尚未实施。
|
||||
|
||||
## #8 当前工程状态（2026-09-11）
|
||||
|
||||
#7已通过用户验收并合入main。#8到期单词复习已按用户确认的决策表实现，并于2026-09-11通过用户验收，PR #27已fast-forward-only合入main；server schema v6新增lexgo_term_reviews（排期与计数）与lexgo_review_answers（作答记录），固定间隔1/2/4/7/15/30/60天，答对升级封顶7、答错降级最低1、再学一次不改等级，答错与再学立即回队，已知与忽略不入队。作答按answerId去重并以expectedDueAt判定过期标签页，重复提交、网络重发与双标签页都只记一次；到期判定用UTC绝对时刻，不引入本地日边界。独立审核（Claude Code）指出的并发同键500、编辑文本重排复习、面板保存重置等级三项已整改并复测。第3阶段「首条学习闭环」（#5～#8）至此全部验收；剩余#9～#15与#21、#24尚未实施。
|
||||
|
||||
## #9 当前工程状态（2026-09-13）
|
||||
|
||||
#8已通过用户验收并合入main。#9 TXT文件导入已按用户确认的契约实现，并于2026-09-13通过用户验收，PR #28已fast-forward-only合入main；本单没有数据库结构变化，schema仍为v6。解码只接受UTF-8（可选BOM剥离且不进原文），非法字节整体拒绝、不使用替换字符，UTF-16按BOM识别后明确拒绝；文件字节上限2MiB并仍受单章100000码点约束；换行与空白不归一化。文件只在内存中解码，不创建临时文件，客户端文件名不参与任何路径也不入库。解码后交给既有PasteBook/PasteChapter，分章、任务幂等与恢复与粘贴一致。第4阶段已完成#9；剩余#10～#15与#21、#24尚未实施。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Architecture-and-Code-Map
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Architecture-and-Code-Map.-
|
||||
wiki_revision: 5bc7ccf33d1cc1cc28598c90e27a2ab6af4146bd
|
||||
synchronized_at: 2026-09-10T14:33:23Z
|
||||
wiki_revision: ce95236e4c8de27c1063448571cfc528c895f937
|
||||
synchronized_at: 2026-09-13T15:20:16Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 架构与代码地图
|
||||
@@ -175,3 +175,133 @@ WordNet 使用 ZIP 内原始 index/data/exception 文件，不使用 SysDict 或
|
||||
- [resources/js/components/Review/ReviewHotkeyInformationDialog.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Review/ReviewHotkeyInformationDialog.vue)
|
||||
|
||||
上游 LICENSE 为 GPL v3，本单只核对并描述行为，没有移植源码。#1 评论 7497 的四项对照登记由本节补充；原型 v1 的保存/关闭主要流程保持，不更改其历史验收记录，不声称已经在 Quant-UX 新建修订版。范围调整与原生手柄作为可运行小样验证，若真机结果导致主要流程变化，应先更新关键原型状态并由用户确认。
|
||||
|
||||
## 粘贴导入、章节与阅读（#5，schema v3）
|
||||
|
||||
#5 实现了目标路径一的第一段可运行链路：粘贴英语文本 → 持久导入任务 → 固定分章 → 处理中/就绪 → 本人阅读原文。Go 单进程同时承担 API 与后台任务。本单不接入 Python NLP，token、lemma 与词典索引仍待 #6，路线未决边界见业务规则页。
|
||||
|
||||
| 路径 | 职责 |
|
||||
|---|---|
|
||||
| server/app/lexgo/database.go | schema v3 显式迁移：lexgo_books、lexgo_chapters、lexgo_ingest_jobs；按版本累加语句，版本行只在全部语句成功后推进 |
|
||||
| server/app/lexgo/library.go | 粘贴校验与固定分章、书籍/章节/任务写入、本人归属查询、重试与请求幂等 |
|
||||
| server/app/lexgo/ingest.go | 任务声明 claim、处理完成、固定失败原因、启动恢复 |
|
||||
| server/app/lexgo/router.go | 新增书籍/章节/任务路由；粘贴请求使用独立的 4 MiB 体积上限 |
|
||||
| server/cmd/lexgo/main.go | serve 启动时恢复遗留任务，并按秒轮询处理待处理任务 |
|
||||
| learner/src/stores/library.ts、views/ImportView.vue、BookView.vue、ReaderView.vue | 粘贴导入、书库与章节状态、失败重试、原文阅读 |
|
||||
|
||||
任务状态为 pending → processing → ready/failed，章节与任务共用同一套状态词。声明与完成分属两个事务：声明一经提交，即使进程随即退出，也只会留下可被启动恢复重新入队的 processing 记录。
|
||||
|
||||
### 粘贴导入 API v1（#5）
|
||||
|
||||
| 方法与路径 | 行为和权限 |
|
||||
|---|---|
|
||||
| POST /api/v1/books | {requestId,title,text,language?}；创建书籍＋首个章节＋导入任务；重复 requestId 返回首次结果（HTTP 200，duplicate=true） |
|
||||
| POST /api/v1/books/:id/chapters | 向本人书籍追加一个章节 |
|
||||
| GET /api/v1/books | 本人书库与章节状态计数；拒绝查询参数，避免用参数替换认证身份 |
|
||||
| GET /api/v1/books/:id | 本人书籍与章节列表，含 jobId 与可读失败原因 |
|
||||
| GET /api/v1/chapters/:id | 本人章节详情；仅 ready 时返回 originalText，并附带前后章节编号 |
|
||||
| GET /api/v1/jobs/:id | 本人任务状态、尝试次数与失败原因 |
|
||||
| POST /api/v1/jobs/:id/retry | 仅失败任务可重试；复用同一章节，不新建章节 |
|
||||
|
||||
所有接口按认证身份过滤 owner_id；他人书籍、章节或任务编号统一返回 404，管理员角色也不能解除学习数据的本人归属过滤。后台任务只使用任务行内的 owner_id，不接受客户端用户编号；请求体含未知字段（例如 ownerId）直接返回 400。
|
||||
|
||||
### schema v3
|
||||
|
||||
lexgo_books(owner_id, title, language)、lexgo_chapters(book_id, owner_id, ordinal, title, original_text MEDIUMTEXT, char_count, content_sha256, status, error_reason) 与 lexgo_ingest_jobs(owner_id, book_id, chapter_id, request_key, content_sha256, status, attempts, error_reason, finished_at)。owner_id 在章节与任务上冗余存放，使任何查询都能直接按认证身份过滤而不依赖连接；UNIQUE(book_id, ordinal) 与 UNIQUE(owner_id, request_key) 分别阻止重复章节与重复提交。启动检查要求版本 3，服务不自动迁移。
|
||||
|
||||
并发重复提交：请求命中 request_key 唯一键冲突后，用加锁读读取已提交结果，因为该请求事务的快照早于并发提交；因此两个并发相同提交只会产生一个章节，另一个得到 duplicate=true 的首次结果。
|
||||
|
||||
## #5 审核整改（R1～R4，2026-09-11）
|
||||
|
||||
提交见工单 #5 的整改评论；本条记录实现与验证方式。
|
||||
|
||||
- 追加契约（R1）：学习端把新建与追加拆成两个请求类型，追加不发送 language；后端保持严格解码，并新增回归测试断言“追加带 language 返回 400、不带则 201”，学习端单测断言追加请求体只有 requestId/title/text。
|
||||
- 运行期任务恢复（R2）：`server/app/lexgo/ingest.go` 的恢复逻辑合并为一处——启动恢复使用阈值 0，运行期每轮清扫使用 15 秒阈值并把超过 5 次尝试的任务置为 failed（原因码 attempts_exhausted）；`cmd/lexgo/main.go` 的 worker 每秒先清扫再处理，日志分别说明“已重新入队”与“本批未完成、等待下一次清扫”，不再声称已完成实际跳过的重试。
|
||||
- 离页作废在途请求（R3）：`closeBook`/`closeChapter` 推进请求序号并清理 loading；`ImportView` 记录是否已卸载，卸载后的成功响应不再触发跳转。
|
||||
- 重试自愈（R4）：`retryChapter` 先把重试返回的章节状态应用到列表与阅读器并重新安排轮询，再做静默刷新。
|
||||
|
||||
验证：Go 全量用例 20 项通过（新增运行期恢复与尝试上限两项）；学习端单测 38 项通过，其中 7 项在整改前的代码上复现失败；真实联调确认追加路径可用、被中断的任务在运行中被自动恢复（约 0.5 秒，无需重启）、重试在首次刷新失败后仍自动显示最终结果。
|
||||
|
||||
|
||||
## 全 Go 正式架构决定（2026-09-11，#6）
|
||||
|
||||
用户已明确选择全 Go：正式英语分词、原文位置映射、本地词典解析和词形候选查询由 Go 后端完成，不运行 Python NLP 服务。此前“Python 建议/全 Go 未决”仅为历史决策记录，由本决定覆盖；spikes/english 保留历史验证，不接入产品。#6 按该方向实施，当前方案见工单最新启动评论；WordNet 3.0 仍为首个资源（英语释义），词形规则候选不等同于 spaCy 上下文消歧，原文及个人学习状态不按候选合并。
|
||||
|
||||
|
||||
## #6 全 Go 词典与阅读器（2026-09-11，已验收并合入 main）
|
||||
|
||||
`server/app/lexgo/wordnet.go` 负责固定 WordNet ZIP 校验/内存解析、Unicode 分词及词形候选；`dictionary.go` 负责资源与章节查词 API；`database.go` schema v4 新增单槽共享资源表 lexgo_dictionaries（元数据、enabled、SHA、ZIP LONGBLOB），已有学习数据不改写。每个 Router 按 SHA 缓存一个不可变词典，查询先读资源元数据，缓存未命中才读取 ZIP；进程重启从数据库恢复，不需要 Python NLP 或额外资源目录。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| GET /api/v1/dictionaries | 管理员；items + supported，状态 ready/disabled/unavailable，不返回 archive 或本机路径 |
|
||||
| POST /api/v1/dictionaries/import | 管理员；multipart name/language/version/source/format/file；返回 resource + duplicate；格式/来源/版本固定 |
|
||||
| PATCH /api/v1/dictionaries/:id | 管理员；{enabled:boolean}；返回 resource |
|
||||
| GET /api/v1/chapters/:id/tokens | 本人 ready 章节；{textSha256,tokens:[{text,start,end,startUtf16,endUtf16,kind}]} |
|
||||
| POST /api/v1/lookup | {chapterId,start,end}，cp半开范围；本人ready完整单词；返回 status/query/matchedForm/candidates/entries/resource |
|
||||
|
||||
管理端 `Dictionaries.vue` + `dictionaries.mjs` + session multipart 方法，复用 go-admin 导航/表单与身份失效保护。学习端 `useReaderLookup.ts` 校验原文片段/SHA/所有位置，`ReaderTokens.vue` 渲染可聚焦单词，`LookupPanel.vue` 展示释义及临时个人草稿。桌面侧栏，手机固定底部45dvh面板；关闭恢复焦点，仅无后续手动滚动时恢复自动调整前位置。旧响应在换词/换章/退出/离页后失效。
|
||||
|
||||
参考：[WordNet 数据格式](https://wordnet.princeton.edu/documentation/wndb5wn)、[词形规则](https://wordnet.princeton.edu/documentation/morphy7wn)。#3 仅历史实验，#6 不调用其实验服务。
|
||||
|
||||
## #7 个人词条与阅读器状态（2026-09-11，已验收并合入 main）
|
||||
|
||||
schema v5 新增 lexgo_terms：一个学习者对一个词形一条记录。身份键为 `(owner_id, language, term)`，`term` 是 Go 侧 `normalizeWord` 的结果（NFC、小写、弯撇号转直撇号），列使用 `utf8mb4_bin`，避免折叠 `resume`/`résumé`；`original_form` 保存最近一次保存的原词形供显示。`definition`/`examples` 是学习者自己的文本，例句按行存储；共享词典仍只在 `lexgo_dictionaries`，两者不混存。`status` 与 `level` 由数据库检查约束守住：只有 `learning` 允许 1～7，其他状态必须为 0。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| POST /api/v1/terms | 本人；`{chapterId,start,end,definition,examples[],status,level?}`；服务端按 #6 同一套 token 范围反推词形，`owner`/`language`/`term` 一律不接受客户端输入；唯一键 upsert，重复保存更新同一行；首次 201、更新 200，返回 `{term,created}` |
|
||||
| GET /api/v1/terms/:id | 本人；他人编号与不存在编号统一 404，不泄露存在性 |
|
||||
| GET /api/v1/chapters/:id/tokens | 在原响应上为 word 片段增加可选 `term:{id,status,level}`；服务端按本章词形分批（每批 500）查本人词条，其他章节保存的同形词同样命中 |
|
||||
|
||||
`server/app/lexgo/terms.go` 负责身份、状态/等级边界、文本上限、upsert 与章节点词状态；`database.go` 提供 v5；`dictionary.go` 的 tokens 与 lookup 共用 `wordAtRange`，保存与查询必须落在同一个完整单词范围上，所以客户端无法命名自己没有读到的词。个人词条不写审计日志。
|
||||
|
||||
学习端 `useReaderLookup.ts` 在原有查询状态上增加个人释义、例句、状态、已保存编号、预填与保存；打开已保存词先读 `GET /terms/:id`，读取失败时禁用保存，避免用空表单覆盖原内容。`LookupPanel.vue` 提供状态单选、释义与例句输入、保存与简短反馈；`ReaderTokens.vue` 按状态高亮 `is-new`/`is-learning`/`is-known`/`is-ignored`。换词、换章、离页、退出或切换账号都会清空表单、状态与高亮。管理端无改动。
|
||||
|
||||
## #8 复习调度与答题（2026-09-11，已验收并合入 main）
|
||||
|
||||
schema v6 新增 `lexgo_term_reviews`（每个个人词条一行排期：`due_at`、`review_count`、`correct_count`、`wrong_count`、`last_reviewed_at`）与 `lexgo_review_answers`（每次作答一条：`answer_key`、评分、状态/等级/间隔前后值、`result`、`requeued`、时间）。两条语句都是 `CREATE TABLE IF NOT EXISTS` 加 `INSERT IGNORE ... SELECT`，所以迁移是可重试的加法迁移；旧二进制回到 v5 仍可继续写 `lexgo_terms`，不需要改动个人词条表本身。既有已保存词汇在迁移中按 `due_at = created_at` 进入队列。
|
||||
|
||||
`server/app/lexgo/review.go` 负责间隔表、队列、评分转换、幂等与并发；`terms.go` 的保存路径通过 `syncTermReview` 维护排期行（`新词` 立即到期，显式 `学习中 level N` 排 `now + 间隔[N]`），计数在状态或等级变化时保留。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| GET /api/v1/reviews/queue | 本人＋当前语言；仅 `新词`/`学习中` 且 `due_at ≤ now`，按 `due_at, id` 排序、最多 50 条；返回 `{items, total}`，`total` 是全部到期数；不接受查询参数 |
|
||||
| POST /api/v1/reviews/:termId/answers | `{answerId, grade: correct\|wrong\|again, expectedDueAt}`；应用成功 201，重放或过期 200；返回首次结果 `result`(applied/stale) 与 `duplicate` 标记、前后状态/等级/到期时间、`requeued` 与词条新状态；加锁后再次读取答案键，所以并发的同键提交也返回记录而不是报错 |
|
||||
|
||||
学习端新增 `/review` 路由与书库、阅读器顶栏的「到期复习」入口；`stores/review.ts` 维护队列、本轮计数、评分与重学，`ReviewCard.vue`/`ReviewView.vue` 呈现正面（词＋挖空例句）、答案面（个人释义＋例句＋三个评分按钮）、完成页与空队列页。一次评分对应一个 `answerId`，失败重试复用同一个；换词后重新生成。切换账号或退出登录会清空队列、计数与当前卡片。
|
||||
|
||||
参考：[LinguaCafe Review.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Review/Review.vue)。上游的随机抽卡、阶段降级与快捷键不属于本单；#8 只实现本项目的固定间隔、固定顺序与幂等作答。
|
||||
|
||||
## #9 TXT 上传导入（2026-09-11，已验收并合入 main）
|
||||
|
||||
`server/app/lexgo/upload.go` 负责把上传的 TXT 解码后交给与粘贴相同的核心：解码、multipart 解析与两条路由，schema 无变化（沿用 #5 的 `lexgo_books`/`lexgo_chapters`/`lexgo_ingest_jobs`）。文件只在内存中存在，不写临时文件，客户端文件名不参与任何路径也不入库。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| POST /api/v1/books/upload | 本人；multipart：`requestId`、`title`、`language`（可省略，省略即英语）、`file`；新建书籍与首章 |
|
||||
| POST /api/v1/books/:id/chapters/upload | 本人且本人书籍；multipart：`requestId`、`title`、`file`；追加一章；不接受 `language` |
|
||||
|
||||
字段白名单之外的字段、重复字段、缺失 `file`、非 multipart 请求都返回 400；201 新建、200 重复、409 同编号换内容、404 他人书籍、401 未登录、429 已有文件正在上传（单槽并发门）。响应体与粘贴路径同为 `PasteResult`，所以学习端复用同一套跳转与轮询逻辑。
|
||||
|
||||
解码规则见业务规则页；实现上 `decodeTextUpload` 先按 UTF-16 BOM 识别并给出针对性提示，再剥离可选 UTF-8 BOM，然后用 `utf8.Valid` 整体校验，最后交给 `validatePaste`（非空、≤100000 码点）。因此上传与粘贴共享同一分章与任务规则：一次提交一章，`requestId` + 内容 SHA 幂等，worker 只发布已落库的原文。
|
||||
|
||||
学习端 `ImportView.vue` 增加「粘贴文本 / TXT 文件」来源切换（沿用已验收 v1 的切换与状态行），`stores/library.ts` 增加 `upload()` 与 `fileProblem`/`fileSizeLabel`，`session.request` 支持 `FormData`（multipart 请求不再被 JSON 化，边界由浏览器提供）。客户端预检只提前反馈，服务端结论为最终结论。
|
||||
|
||||
## #10 编辑与删除书籍章节（2026-09-11）
|
||||
|
||||
`server/app/lexgo/edit.go` 提供改名、编辑与删除；`ingest.go` 增加版本门控。**schema 无变化**：`chapters.content_sha256` 与 `jobs.content_sha256` 就是版本键，新增的 `superseded` 复用现有 `error_reason` 列。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| PATCH /api/v1/books/:id | 本人；`{title}`；返回 `{book}` |
|
||||
| GET /api/v1/chapters/:id/source | 本人任意状态；返回 `{source:{id,bookId,ordinal,title,text,status,contentSha256,charCount}}`，供编辑使用；不接受查询参数 |
|
||||
| PATCH /api/v1/chapters/:id | 本人；`{title?,text?}`；返回 `{chapter,job,versionChanged}`；正文变化才新建任务 |
|
||||
| DELETE /api/v1/books/:id | 本人；返回 `{deleted:{bookId,chapters}}`；FC 级联删除章节与任务 |
|
||||
| DELETE /api/v1/chapters/:id | 本人；返回 `{deleted:{chapterId,bookId,remaining}}`；删除后重排序号 |
|
||||
|
||||
**版本门控**（本单修掉的缺陷）：任务只在 `job.content_sha256 == chapter.content_sha256` 时才能影响章节。认领任务时用 JOIN 只取版本匹配的行，并先把过期版本任务一次性标为 `failed/superseded`；发布前再比对一次，不匹配就只把任务标为 `superseded` 并**完全不触碰章节**；恢复扫描同样先作废过期版本任务、只重排版本匹配的中断任务；重试接口拒绝版本不匹配的任务（409）。删除期间在途任务找不到章节时视为无事可做（级联已删除其任务行）。
|
||||
|
||||
`unprocessableReason` 保留一条内容一致性检查：存储的正文重新计算出的 SHA 必须等于该章节存储的 SHA，用于兜住绕过 API 的直接写入（`content_changed`），与版本门控互不重复。
|
||||
|
||||
学习端 `BookView.vue` 增加书名编辑对话框、章节编辑对话框（标题 + 正文，正文来自 source 接口）与两处确认弹窗（`ElMessageBox`），章节行增加「编辑」入口；`LibraryView.vue` 在书库被删后显示「书籍已删除 · 已保存的生词和短语仍保留在生词本。」；`stores/library.ts` 增加 `renameBook`、`updateChapter`、`loadChapterSource`、`deleteBook`、`deleteChapter`。正文编辑通过浏览器 textarea 输入，因此该章的行尾统一为 LF（粘贴与 TXT 导入仍保留原始 CRLF）。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Business-Rules-and-Glossary
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Business-Rules-and-Glossary.-
|
||||
wiki_revision: 227c6cb92d30490f0f1f4b1b9f29cdbed8559ee4
|
||||
synchronized_at: 2026-09-10T14:33:25Z
|
||||
wiki_revision: 96eec89f74db744d8807baa24b1d1d611ff6bf8f
|
||||
synchronized_at: 2026-09-13T15:20:16Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 业务规则与术语
|
||||
@@ -112,3 +112,117 @@ POST /lookup 接收 {surface,lemma?}。查词键单独 casefold/NFC/弯撇号转
|
||||
桌面鼠标/手机长按使用原生 Selection；正文中键盘 ←/→ 选相邻词，Shift+←/→ 从锚点扩缩连续范围，起点/终点按钮可用 Tab/Enter。Esc 或关闭清除选区及未保存编辑，保留滚动和键盘阅读位置；切换章节清除选区，当前页面内分别记住章节滚动位置。纯标点的新选区清除旧面板，防止操作上一个词。个人释义必须显式保存，状态变化也随保存提交；本小样仅写内存。
|
||||
|
||||
手机方案保留浏览器原生长按、选择手柄和滚动，面板最多占底部 42dvh，正文有底部阅读余量；键盘定位用滚动边距避开面板。未获得真实手机结果，不能判定手柄、系统菜单、虚拟键盘或触摸滚动冲突已经解决。
|
||||
|
||||
## #5 粘贴分章、任务与阅读规则 v1
|
||||
|
||||
用户于 2026-09-10 确认两项边界（工单 #5 评论 7644）：本单按 Go 处理、不接入 Python；一次粘贴等于一个章节。
|
||||
|
||||
- 固定分章规则：一次粘贴产生一个章节，不按空行或长度自动再分。新建书籍时书籍标题与首章标题同为提交的标题；追加时标题即新章节标题。分章规则变化属于需求变化，必须重新确认。
|
||||
- 长度与校验：标题去首尾空白后 1～120 个字符；正文必须含至少一个非空白字符；正文上限 100000 Unicode code point，超出返回 400；语言当前只接受 en。
|
||||
- 原文保真：正文按收到的字符串原样保存与返回，不做 NFC、大小写、换行或空白归一化；页面使用 white-space: pre-wrap 展示，制表符、连续空格与空行保持可见。处理完成前不返回原文。
|
||||
- 归属：书籍、章节与任务都记录认证账号的 owner_id；他人编号返回 404；管理员角色不解除学习数据的本人归属；后台任务只使用任务行的 owner，不信任客户端用户编号。
|
||||
- 任务状态：pending、processing、ready、failed，章节与任务共用同一词表。失败时返回固定原因码加可读中文提示，错误字段不保存正文。
|
||||
- 固定失败原因：unsupported_language、too_long、empty_text、content_changed。前三种只能由其他写入路径产生（例如语言调整或后续编辑功能）；content_changed 表示章节内容在处理前被改动，属于过期任务，必须重新提交，或恢复为提交时的内容后重试。
|
||||
- 幂等：客户端 requestId 与账号构成唯一键。同一 requestId 配同标题同正文的重复提交返回首次结果，不新建章节；同一 requestId 配不同标题或正文返回 409；并发重复提交同样只产生一个章节。重试复用原章节，只增加尝试次数。
|
||||
- 恢复：声明与完成分属两个事务。进程在声明后退出时，重启把 processing 的章节与任务放回 pending 并保留尝试次数，不产生重复章节。
|
||||
- 未决边界：Go＋Python NLP 与全 Go 路线尚未确认。本单只做 Go 校验、分章与发布，不产生 token、lemma 或词典索引；正式接入前必须由用户确认路线，再定义生产 token 契约。
|
||||
- 已知限制：本单未设置每账号书籍数量或总容量配额，只限制单次正文与请求体大小；删除书籍/章节属 #10，导入失败不会自动重试，只在启动时恢复被中断的 processing 任务。
|
||||
|
||||
## #5 审核整改（R1～R4，2026-09-11）
|
||||
|
||||
工单 #5 的整改记录见该工单评论；本节只记录长期有效的契约变化。
|
||||
|
||||
- 追加章节沿用所属书籍的语言：`POST /api/v1/books/:id/chapters` 的请求体只有 requestId、title、text，不接受 language；服务端仍拒绝未知字段，客户端发 language 会得到 400。新建书籍的 `POST /api/v1/books` 才带 language。学习端已按此拆分请求体，避免两个契约共用同一结构。
|
||||
- 导入任务的自动重试有上限：同一任务被 worker 领取的次数达到 5 次后，任务与章节转为 failed，原因码 `attempts_exhausted`，提示“处理多次失败，请重试或重新提交”。人工重试（POST /jobs/:id/retry）会重置尝试次数，因此人工操作不受该上限阻塞。
|
||||
- 运行期恢复不依赖重启：除了启动时的恢复，运行中的服务每次轮询都会把停留在 processing 且超过 15 秒的任务放回 pending，因此“领取已提交、完成事务失败”不会让章节永久卡在处理中。该阈值必须长于正常的领取到完成窗口；重复处理同一任务不会产生第二个章节，因为任务从不创建章节。
|
||||
- 学习端离开页面时作废在途请求：目录页与阅读页在关闭时推进各自的请求序号，晚到的响应不会写回状态或重启轮询；导入页在提交过程中离开后，晚到的成功响应不会把用户导航回书籍页。
|
||||
- 重试结果立即生效：重试被接受后先把返回的 pending 状态写入界面并继续轮询，因此紧随其后的一次刷新失败不会让页面停在处理失败。
|
||||
|
||||
|
||||
## 全 Go 正式架构决定（2026-09-11，#6）
|
||||
|
||||
用户已明确选择全 Go：正式英语分词、原文位置映射、本地词典解析和词形候选查询由 Go 后端完成，不运行 Python NLP 服务。此前“Python 建议/全 Go 未决”仅为历史决策记录，由本决定覆盖；spikes/english 保留历史验证，不接入产品。#6 按该方向实施，当前方案见工单最新启动评论；WordNet 3.0 仍为首个资源（英语释义），词形规则候选不等同于 spaCy 上下文消歧，原文及个人学习状态不按候选合并。
|
||||
|
||||
|
||||
## #6 正式词典规则（2026-09-11）
|
||||
|
||||
首个资源固定 Princeton WordNet 3.0，英语释义；ZIP 来源及 SHA 以 server/wordnet-resource.json 为准，LICENSE 原样保留于 server/WORDNET-LICENSE.txt。管理员上传指定包并配置显示名称，语言/来源/版本/格式固定；32MiB压缩、128MiB解压上限、成员与结构校验。资源为所有账号共享、仅管理员写；失败保持旧资源，重复上传复用id并启用，停用后查询返回 resource_missing。ZIP 随数据库备份，无运行时网络下载。
|
||||
|
||||
正式分词由 Go 完成：字母开始词，组合标记延续词，内部直/弯撇号连接字母；空白逐字保留，数字和符号为不可查询的 punctuation。start/end 是 Unicode code point 半开范围，另有UTF-16偏移；原文、CRLF、组合字符与emoji不归一化。只对查询键小写/NFC/撇号归一化。每次查询必须是本人ready章节内的完整单词，最大128码点，跨账号404，非法范围400。
|
||||
|
||||
exact优先；未命中再按WordNet异常表/词尾规则查候选，词性顺序n/v/a/r、最多12条释义，保留原数据s词性。返回lemma表示规则候选，不承诺上下文消歧；歧义不会合并个人学习状态。not_found与resource_missing区分，网络错误可重试，不阻断阅读。#3使用spaCy上下文lemma的实验路径由全Go规则候选替代。
|
||||
|
||||
个人释义目前仅当前选择的临时草稿，界面标记未保存；换词、关闭、换章、离页和身份变化清理。持久化与个人词汇状态由#7实现，不把临时输入宣传为保存成功。
|
||||
|
||||
## #7 个人词条规则（2026-09-11）
|
||||
|
||||
个人词条身份是「学习者＋语言＋规范化词形」：`normalizeWord` 做 NFC、小写与弯撇号转换，而原文、章节文本与显示用的原词形都不归一化。同一词形不同大小写是同一条记录；不同词形（`dog` 与 `dogs`）是不同记录，不按 WordNet 候选或 lemma 合并，与 #6「不按候选合并个人学习状态」一致。语言取自已登录学习者的英语空间，不从请求读取。
|
||||
|
||||
| 业务状态 | level | 含义 | 进入到期复习 | 原版 stage |
|
||||
|---|---|---|---|---|
|
||||
| new 新词 | 0 | 已保存、尚未开始学习；首次保存默认 | 由 #8 决定 | 2 |
|
||||
| learning 学习中 | 1～7 | 正在复习，越接近 7 越熟 | 是 | -1～-7 |
|
||||
| known 已知 | 0 | 已掌握，不再进入到期复习 | 否 | 0 |
|
||||
| ignored 忽略 | 0 | 明确忽略，不计入已知 | 否 | 1 |
|
||||
|
||||
上表是原版合并编码（状态与等级在同一字段）的显式替代，供 CSV 导出与旧数据迁移映射；#7 只保存与返回等级，#8 负责复习推进与到期时间。非 `learning` 状态携带非 0 等级、`learning` 等级超出 1～7、以及未知状态一律拒绝，`learning` 缺省等级为 1。
|
||||
|
||||
个人释义可为空（允许只记录状态），最长 2000 字符，可含换行与制表符；例句最多 5 条、每条最长 500 字符，不能为空行或含换行。保存失败保留学习者已输入的内容，成功后显示「已保存 · 状态」并立即更新正文高亮。
|
||||
|
||||
保存幂等由唯一键承担：重复提交同一词形只更新同一行，不产生第二条冲突记录；同一账号多端同时编辑为最后写入生效，本版不引入版本冲突拒绝。所有读写都属于会话本人：章节必须本人且已就绪，篡改 id、owner、language、term 或携带未知字段返回 400，他人编号与本人不可见编号统一 404。跨账号不共享任何数据与前端缓存，退出或切换账号后不保留前一账号的词条与高亮。
|
||||
|
||||
例句只保存学习者手输内容，不自动关联原文句子：当前分词只有词/空白/标点边界，没有句子切分规则。原型 v1 中「原文例句已关联到词条」是演示文案，不作为契约。等级选择器属于 #8/#12 的编辑界面，阅读器面板只提供四个状态。
|
||||
|
||||
## #8 复习规则（2026-09-11）
|
||||
|
||||
**间隔表**：答对后按新等级排期，等级上限 7。这是固定表，不是 FSRS，也不是上游「按等级选复习量少的日期」的算法。
|
||||
|
||||
| 等级 | 1 | 2 | 3 | 4 | 5 | 6 | 7（上限） |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 下次复习 | 1 天 | 2 天 | 4 天 | 7 天 | 15 天 | 30 天 | 60 天 |
|
||||
|
||||
**入队范围**：只有状态 `新词` 或 `学习中` 且 `due_at ≤ now` 的词条进入队列；`已知` 与 `忽略` 不入队。新保存的词立即到期（`due_at` = 保存时刻），保存后就能复习；手动把词条改为「学习中 level N」时下次复习为 `now + 间隔[N]`，避免刚标等级就被当成到期。只有新建词条或状态/等级实际变化才移动复习时间：只修改释义、例句或原词形时保留原有排期，否则编辑文本会把逾期词条挤出当天队列。保存未提及等级时保留已获得的等级，只有进入 `学习中` 才从 1 开始，因此阅读器面板的保存不会把等级重置为 1。已在队列中的词条被改成 `已知`/`忽略` 后再次作答会被拒绝（409），排队信息由服务端裁决而不是前端过滤。
|
||||
|
||||
**三个评分动作**：`correct` 认识/答对 → 等级 +1（封顶 7）、按新等级排期、离开本轮；`wrong` 不认识/答错 → `学习中` 降一级（最低 1）、`新词` 保持 `新词`、`due_at = now` 立即回到本轮；`again` 再学一次 → 等级与状态不变、`due_at = now` 立即回到本轮。计数上 `correct_count` 只统计 `correct`，`wrong_count` 统计 `wrong` 与 `again`，`review_count` 统计全部已应用作答。词条的原文、个人释义与例句不因复习改变。
|
||||
|
||||
**时区与到期边界**：`due_at` 以 UTC 绝对时刻存储，到期判定是 `due_at ≤ now`，不引入本地日边界。理由：MVP 没有用户时区设置（属 X 系列边界），绝对时刻在多账号自托管下语义一致、没有夏令时陷阱；代价是复习时刻会随首次作答时间漂移，例如 22:00 答对的 1 天间隔词条要到次日 22:00 才到期。原型界面的「到期复习 N / M」指当前到期队列的位置与本轮卡片数，不是自然日统计。
|
||||
|
||||
**幂等与并发**：客户端每次作答生成一个 `answerId`，服务端以 `UNIQUE(owner_id, answer_id)` 去重。同一个 `answerId` 再次提交返回首次结果并把 `duplicate` 标记为 true（`result` 仍是首次的 `applied` 或 `stale`），不改变等级、间隔和次数，因此客户端重试可以按首次结果计数；同一词条在别处已经被推进（提交回传的 `expectedDueAt` 与服务端当前 `due_at` 不一致）时记为 `result=stale`，同样不改变任何状态。因此网络重发、双击、双标签页作答都只记账一次。每次尝试都会落一条 `lexgo_review_answers`（含 `result`），这是「只记账一次」的证据，也供 #13 统计使用。
|
||||
|
||||
**归属与错误**：词条归属由服务端按会话裁决，`owner` 不接受客户端输入；他人词条与不存在的词条统一 404，未登录 401，未知评分/缺少 `expectedDueAt`/未知字段 400，词条已变成 `已知`/`忽略` 409。复习接口不写审计日志：答题属于私人学习内容。
|
||||
|
||||
**范围边界**：本单只做单词复习。短语复习归 #11，到期范围筛选与策略配置界面（X11）不做，练习模式（X08）不做，进度与「已知词/待复习」计数归 #13。#8 不改变 #7 定下的身份口径：复习状态按规范化词形归属，仍不按 WordNet lemma 候选合并。
|
||||
|
||||
## #9 TXT 上传规则（2026-09-11）
|
||||
|
||||
**支持的编码**：只接受 UTF-8，允许带可选 UTF-8 BOM。BOM 在解码时剥离，不进入原文；其余字节必须整体合法，任何非法序列**直接拒绝**，绝不使用替换字符，因此章节里不会出现学习者没有写过的乱码。UTF-16（含记事本「另存为 Unicode」产生的大小端 BOM）单独识别并提示「请另存为 UTF-8 后重试」；GB18030、Latin-1 等其他编码按非法 UTF-8 拒绝。UTF-16 支持不在本单范围。
|
||||
|
||||
**换行与空白**：不做任何归一化，CRLF、LF、制表符、行尾空格与空行按原字节保存，阅读器以 `pre-wrap` 原样呈现——与粘贴路径一致。
|
||||
|
||||
**大小上限**：文件字节上限 2 MiB；解码后再套用单章上限（非空、≤100000 码点）。超限返回 400 并明确提示，不截断、不部分导入。2 MiB 对 100000 码点的 UTF-8 文本有足够余量。
|
||||
|
||||
**文件生命周期与路径**：上传内容只存在于内存中，解码后直接进入导入事务；服务端**不创建临时文件**，所以没有需要清理或可能泄漏的文件；**客户端文件名不参与任何文件系统路径、也不写入数据库**，它只在选择文件时用于显示（并可预填标题）。因此文件名即使写成 `..\..\windows\system32\evil.txt` 也不会影响任何存储位置。上传并发按单槽限制，忙时返回 429。
|
||||
|
||||
**导入与幂等**：上传与粘贴共用同一套规则——一次提交一章，`requestId` + 内容 SHA 保证重复上传同一文件只产生一章（返回第一次的章节并标记 `duplicate`），同一 `requestId` 换成其他内容返回 409。任务状态、失败重试与崩溃恢复沿用 #5 的任务机制，不新增状态。标题规则与粘贴完全相同（去空白后非空、≤120 字符）；省略 `language` 时默认英语，与粘贴一致；追加章节不接受 `language`。
|
||||
|
||||
**范围边界**：不包含 EPUB、PDF、字幕与其他文件格式；不做按空行自动分章；不做 UTF-16/GB18030 转码；不做断点续传；不把来源文件名持久化（若将来需要「导入来源」溯源，另立范围）。
|
||||
|
||||
## #10 编辑、版本与删除规则（2026-09-11）
|
||||
|
||||
**可编辑内容**：书名、章节标题、章节正文。只有正文变化才重新处理；只改标题不改变处理状态，也不新建任务。
|
||||
|
||||
**版本规则**：`chapters.content_sha256` 是章节的版本键，`jobs.content_sha256` 是任务被创建时对应的版本。任务只能在版本匹配时影响章节：
|
||||
- 过期版本的任务被标为 `error_reason=superseded`（「章节内容已更新为新版本，本次处理已作废」），**不会**把章节标成失败、也不会发布旧文本；界面上失败章节的重试按钮只出现于当前版本的任务。
|
||||
- 任务重试要求版本匹配，否则 409，避免把当前章节拉回旧版本再失败一次。
|
||||
- 重复保存同一正文不是新版本：不新建任务、不改变状态。改回原内容（内容相同）同样不触发处理。
|
||||
- 编辑章节正文会保留章节编号与阅读入口（URL 不变），章节状态回到待处理，处理完成后原文即新版本。
|
||||
|
||||
**存储文本与版本一致性**：章节存储的正文重新计算出的 SHA 必须等于存储的 SHA；出现不一致（绕过 API 的直接写入）时按 `content_changed` 失败，不发布不确定内容。
|
||||
|
||||
**删除规则**：删除在事务内**硬删除**，并沿用现有外键级联清理：书籍 → 章节 → 任务。删除章节后剩余章节序号重排为连续（原型显示「剩余 N 章」，导航按序号取值）；并发删除同一章由书籍行锁序列化，结果是一个成功、一个 404。重复删除返回 404，不把「已经不存在」当成成功。恢复路径是数据库备份与完整恢复（#15 演练范围），产品不提供回收站或撤销。
|
||||
|
||||
**个人学习记录保留**：删除书籍或章节**不删除**个人词条、复习排期与作答记录，因为它们按学习者归属、不引用章节；界面也没有「来自某章节」的引用（例句是学习者输入的副本）。所以删除只影响书籍、章节与处理任务，这一点在原型确认文案中就写明：「本章正文将被删除，已保存词条保留」「删除这本书及其章节？已保存的生词和短语将保留」。
|
||||
|
||||
**归属与边界**：改名、编辑、删除、读取编辑用原文都严格按会话归属；他人资源与不存在资源统一 404，未登录 401，空标题/空正文/超长文本/未知字段 400。学习端正文编辑框的行尾会统一为 LF（浏览器 textarea 行为），粘贴与 TXT 导入路径仍然保留原始 CRLF 与空白。
|
||||
|
||||
**范围边界**：不做封面与音频附件（#21）、不做回收站/撤销、不做批量操作、不做章节跨书移动、不做语言变更。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Local-Development-and-Verification
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Local-Development-and-Verification.-
|
||||
wiki_revision: 694facbb7a861182122c9f6c69d28c58b005cb1a
|
||||
synchronized_at: 2026-09-10T14:52:22Z
|
||||
wiki_revision: 251fb5de71b8ba75da3cba6eee41454d5bbd22df
|
||||
synchronized_at: 2026-09-13T15:20:16Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 本地开发与验证
|
||||
@@ -246,3 +246,173 @@ node --test spikes/english/view.test.mjs
|
||||
|
||||
|
||||
#4 用户验收记录：2026-09-10T22:51:52+08:00 用户确认“#4通过验收”（评论 7636）。未补充手机型号/浏览器/操作记录，未重跑测试，未自动合并 PR。后续正式移动端集成应补真机回归。
|
||||
|
||||
## #5 粘贴导入与章节阅读（schema v3）
|
||||
|
||||
升级步骤（本机，仓库根执行）：停止 lexgo-api → `python scripts/server.py build` → `python scripts/server.py migrate` → 启动 lexgo-api。lexgo_dev 已从 v2 升到 v3，新增 lexgo_books、lexgo_chapters、lexgo_ingest_jobs；迁移前后 sys_user 4、lexgo_spaces 4、lexgo_sessions 3、lexgo_login_logs 23、lexgo_operation_logs 1 完全一致。托管实例重启后 /healthz 返回 200，两端首页仍为 200，lexgo-admin 与 lexgo-learner 的 PID 未变化。
|
||||
|
||||
回退：停止 API，把 lexgo_schema 中 id=1 的版本从 3 改回 2，并恢复上一二进制；三张新表保留不删除，旧程序不读写它们。重新升级时显式 migrate 重新执行 IF NOT EXISTS 语句即可；集成测试覆盖 v2→v3 的既有数据保留与 v2 标记下的重复迁移。未在开发库演练回退。
|
||||
|
||||
学习端入口：http://127.0.0.1:5173 → 登录 → 我的书库 → 粘贴文本导入 → 章节就绪后进入阅读。
|
||||
|
||||
测试命令与结果（仓库根执行；本单使用专用库 lexgo_test_issue5，不借用其他测试库）：
|
||||
|
||||
| 命令 | 本次结果 |
|
||||
|---|---|
|
||||
| `python scripts/server.py test-integration`（LEXGO_TEST_DB_NAME=lexgo_test_issue5） | 全部通过：18 个顶层用例，其中 #5 新增 8 个（7 个书库/章节/任务/阅读 + 1 个 v2→v3 数据保留），另含 12 个子用例；既有 10 个用例保持通过 |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner test:unit --run` | 3 个文件 31 项通过（session 9、library 16、reading 6） |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner build` | vue-tsc 类型检查与 vite 构建通过，退出码 0 |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner test:e2e` | 3 项通过（既有 auth 2 项 + 新增 reading 1 项，均为虚构 API 响应） |
|
||||
|
||||
### 真实 API + MySQL 实测（2026-09-10，lexgo_dev）
|
||||
|
||||
使用本单新建的虚构账号 issue5_a、issue5_b，口令只保存在忽略的 .local/issue5-accounts.json；未改动 admin、dev、learner_a、learner_b。脚本 .local/verify-issue5-api.ps1 只在本机运行，不输出口令。
|
||||
|
||||
- 粘贴：HTTP 201，章节与任务均为 pending，charCount 99。
|
||||
- 处理：实测状态序列 pending → ready，约 1132 ms（后台任务每秒轮询）；job attempts=1。
|
||||
- 阅读：originalText 与提交正文逐字符相等，CRLF、制表符、弯引号、破折号、省略号、é 加组合重音、emoji、行尾空格与空行全部保留；sha256 前缀 ce7357ea22a3。
|
||||
- 幂等：同一 requestId 重复提交 HTTP 200、duplicate=true、章节与任务编号不变；同一 requestId 换正文 HTTP 409；书库仍为 1 本。
|
||||
- 隔离：issue5_b 读取 issue5_a 的书籍、章节、任务以及追加、重试全部 404；请求体带 ownerId 与查询参数 ownerId 均 400；issue5_b 书库为空。
|
||||
- 追加与阅读导航：新章节 ordinal=2，处理后就绪，前后章节编号互相指向。
|
||||
- 校验：空标题、纯空白正文、非 en 语言、缺少 requestId、超过 100000 code point 分别返回 400 与可读中文提示。
|
||||
- 遗留 fixture：lexgo_dev 中 issue5_a 名下 1 本虚构书、2 个就绪章节（bookId=1，章节 1、2）。
|
||||
|
||||
浏览器实测：真实学习端 + 真实 API + 真实 MySQL 联测（临时 Playwright 用例，运行后删除）：issue5_a 登录 → 书库显示既有虚构书与“导入内容”入口 → 导入页粘贴含空行、制表符、行尾空格与 emoji 的正文 → 书库页由“处理中”变为“已就绪” → 阅读页 article.reader-text 的 textContent 与粘贴正文逐字符相等、computed white-space 为 pre-wrap → “下一章”切换到第二章且正文精确相等 → 390×844 视口下横向溢出 0 px。同一轮还运行了 3 项虚构 API 的既有 e2e，共 4 项通过。
|
||||
|
||||
截图保存在本机 .local/evidence/（library.png、book-after-paste.png、reader-desktop.png、reader-mobile-390.png）并作为工单附件上传，便于人工目视复核；本次会话模型不能读取图片，截图未经 Agent 目视检查，功能断言来自上面的程序化检查。
|
||||
|
||||
未验证：处理失败到重试的用户界面路径只由集成测试覆盖（无法通过 API 主动制造处理失败）；真实手机浏览器长按、手柄、滚动与虚拟键盘仍属 #4 缺口，本次只用桌面浏览器窄屏检查，不能当作真机结果；Python NLP 未接入，token、lemma 与词典仍为 #3 小样范围；生产并发、容量、备份恢复与部署不在本单范围。
|
||||
|
||||
## #5 审核整改验证（R1～R4，2026-09-11）
|
||||
|
||||
整改提交与完整证据见工单 #5 的整改评论。本次复核命令与结果（仓库根执行，专用测试库 lexgo_test_issue5）：
|
||||
|
||||
| 命令 | 本次结果 |
|
||||
|---|---|
|
||||
| `python scripts/server.py test-integration` | 20 个顶层用例全部通过（新增 `TestMySQLIngestRecoveryWithoutRestart`、`TestMySQLIngestAttemptsAreBoundedAndManualRetryRestarts`） |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner test:unit --run` | 38 项通过（library 21、reading 8、session 9） |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner build` | 通过（vue-tsc + vite） |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner test:e2e` | 3 项通过（虚构 API 响应） |
|
||||
|
||||
回归测试的有效性：新增的前端 7 项用例先在整改前的 `library.ts`/`ImportView.vue` 上运行并实际失败（追加发送 language、离页后响应写回、重试后停在失败），改回修复版本后全部通过。
|
||||
|
||||
真实联调（lexgo_dev，虚构账号 issue5_a）：
|
||||
|
||||
- R1 追加：真实学习端从书籍页进入“追加章节”，提交后回到书籍页，新章节就绪后可阅读；抓取到的请求体只有 requestId、title、text，无 language；正文逐字符相等。
|
||||
- R2 恢复：新建章节后用 SQL 把任务与章节置为 processing 且 updated_at 早于阈值（UTC 时间），**不重启服务**，运行期清扫在 544 ms 内把任务重新入队并发布为就绪，章节与任务编号不变，正文逐字符相等。
|
||||
- R4 重试：SQL 制造真实失败任务（content_changed）后，在浏览器点击“重试”并中断其后的第一次刷新请求，页面立即由“处理失败”变为“处理中”，随后自行变为“已就绪”，无需手工刷新。
|
||||
|
||||
注意：MySQL 会话时区为 SYSTEM（本机为 UTC+8），而服务按 UTC 存储 DATETIME；核对任务时间时使用 UTC_TIMESTAMP 而不是 NOW()，否则会出现 8 小时的假偏差。
|
||||
|
||||
未在本轮验证：R3 的真实浏览器时序（离页与响应同时发生）只由单测覆盖；真机手机证据仍属 #4 缺口。
|
||||
|
||||
|
||||
## #6 部署与验证（2026-09-11）
|
||||
|
||||
正式后端 schema v4，只增加 lexgo_dictionaries。显式 migrate 后启动纯Go lexgo.exe；Python scripts/server.py 仍是开发命令封装，产品运行不依赖Python NLP。
|
||||
|
||||
本机已从v3升级v4并重启lexgo-api，升级前后sys_user/lexgo_spaces/lexgo_sessions/lexgo_books/lexgo_chapters/lexgo_ingest_jobs计数一致。旧二进制保存在忽略的 .local/lexgo-pre-issue6.exe。回退：停止API，恢复旧二进制，将已确认v4的schema标记恢复3，保留新增资源表及全部学习数据，再启动旧API；不要删除数据或重新bootstrap。
|
||||
|
||||
管理端 http://127.0.0.1:5174 的“英语词典”页可下载指定包并导入/启停；本机已导入固定WordNet3.0，155287个词形/词性索引项。学习端 http://127.0.0.1:5173 打开本人ready章节，点词或Enter/空格查询，Escape关闭。went/mice应出现go/mouse候选。源包、本机凭据与测试证据仅存在忽略的.local，不进入Git。
|
||||
|
||||
验证：Go全包MySQL集成（专用lexgo_test_issue5）与go vet通过；学习端53单测、类型检查/构建、默认5173 Playwright3项通过；管理端31单测与lint通过，构建含已有Sass弃用与bundle体积提示；治理56测试与strict通过。实际ZIP解析、schema3→4原文/任务保留、权限、失败保留、重复启用、冷缓存路由重建都有覆盖；冷缓存测试不是完整备份恢复演练。
|
||||
|
||||
主审真实API联调使用真实管理端session模块经5174代理上传；经5173代理两测试账号分别创建虚构章节并验证精确/不规则词形、Unicode原文片段、越权404/普通用户管理403、停用/重复导入启用、错误ZIP保留资源。新增测试书籍id3/4、章节id8/9归issue5_a/issue5_b，没有修改其他账号的书籍。
|
||||
|
||||
常驻5173一度返回空白页：Vue模块转换500、代理缺失；同代码隔离服务正常，只重启lexgo-learner加载配置后恢复，默认E2E通过，未改启动配置。桌面交互浏览器工具因旧会话失效未完成手工真实UI联调；已有项目Playwright使用模拟API，真实API验证另列。手机仅窄屏自动测试，真机证据仍未补齐。
|
||||
|
||||
## #7 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue7，不借用其他测试库。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue7 python scripts/server.py test-integration` | 34 个顶层用例全部通过、0 跳过；含 #7 新增 3 个 MySQL 用例、4 个单元用例和 1 个 v4→v5 迁移用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 58 项单测、类型检查、构建、3 项默认 E2E 全部通过 |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：非法状态与等级边界、文本上限、词形身份与大小写合并、词形不按 lemma 合并、重复与并发保存只留一条记录、跨章节同形词状态一致、两账号互不影响、篡改 owner/language/term 被拒绝、非本人章节 404、未登录 401、未就绪章节 409、v4→v5 迁移保留既有数据与检查约束。
|
||||
|
||||
本机开发库 lexgo_dev 已显式从 v4 升级到 v5：升级前后 sys_user 6、lexgo_spaces 6、lexgo_sessions 8、lexgo_books 4、lexgo_chapters 9、lexgo_ingest_jobs 9、lexgo_dictionaries 1 全部不变，新增空的 lexgo_terms。旧二进制备份在忽略的 `.local/lexgo-pre-issue7.exe`。回退：停止 lexgo-api，恢复旧二进制，把 `lexgo_schema` 标记改回 4 后启动；保留 lexgo_terms 与全部既有数据，不删除数据、不重新 bootstrap。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 30 项检查通过（凭据只从本机安全配置读入进程），覆盖两个虚构测试账号 issue5_a/issue5_b 的登录、保存、幂等、状态边界、跨章节一致、跨账号隔离与越权拒绝；随后用临时 Playwright 用例在真实学习端＋真实 API 上以两个账号复核保存、重新加载后的高亮与预填，以及 390×844 窄屏底部面板。截图保存在本机 `.local/evidence/`（issue7-reader-desktop.png、issue7-reader-mobile-390.png、issue7-account-b.png），临时用例运行后删除。
|
||||
|
||||
未验证：真实手机触屏详细证据与完整备份恢复演练仍属既有缺口（#14/#15）；本单只用桌面浏览器窄屏检查，不当作真机结果。
|
||||
|
||||
## #8 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue8，不借用其他测试库。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue8 python scripts/server.py test-integration` | 41 个顶层用例全部通过、0 跳过；含 #8 新增 6 个复习用例与 1 个 v5→v6 迁移用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 71 项单测、类型检查、构建、5 项 E2E 全部通过 |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：间隔表与每个状态转换（含等级上下限）、到期边界（`due_at = now` 到期、早 1 毫秒不到期）、入队范围（已知/忽略不入队）、两账号队列互不可见、重复提交只记一次、同一词条双标签页 second 写入为 stale、作答归属 404、已知词条 409、未知评分/缺少到期时间/未知字段 400、显式等级的排期规则、幂等键唯一约束、v5→v6 保留既有词条与计数。
|
||||
|
||||
本机开发库 lexgo_dev 已显式从 v5 升级到 v6：升级前后 sys_user 6、lexgo_spaces 6、lexgo_sessions 5、lexgo_books 5、lexgo_chapters 10、lexgo_ingest_jobs 10、lexgo_dictionaries 1、lexgo_terms 4 全部不变；新增 `lexgo_term_reviews` 4 行（全部 `due_at = created_at`）与空的 `lexgo_review_answers`。旧二进制备份在忽略的 `.local/lexgo-pre-issue8.exe`。回退：停止 lexgo-api，恢复旧二进制，把 `lexgo_schema` 标记改回 5 后启动；新表不影响旧二进制写入个人词条，保留新表与全部既有数据，不删除数据、不重新 bootstrap。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 38 项检查通过（凭据只从本机安全配置读入进程，脚本可重复运行），覆盖到期即时入队、答对升级与 1 天排期、答错立即回队、重放与 stale 不重复推进、越权与非法输入拒绝、两账号互不影响、已知词条拒绝作答；随后用临时 Playwright 用例在真实学习端＋真实 API 上完成「登录→粘贴导入→点词查义→保存个人释义→到期复习作答→回到阅读器确认状态」的完整闭环，并复核 390×844 窄屏无横向溢出。截图保存在本机 `.local/evidence/`（issue8-reader-saved.png、issue8-review-revealed.png、issue8-review-summary.png、issue8-reader-reviewed.png、issue8-review-mobile-390.png），临时用例运行后删除。
|
||||
|
||||
运维记录：本次只有 lexgo-api（新二进制）与 lexgo-learner 被重启；学习端常驻 Vite 在长时间运行后一度对 `/src/style.css` 返回空样式表，导致 E2E 观察到 `white-space: normal`，重启后恢复，未改动代码或配置。其余实例保持 Running。
|
||||
|
||||
未验证：真实手机触屏详细证据与完整备份恢复演练仍属既有缺口（#14/#15）；本单只用桌面浏览器窄屏检查，不当作真机结果。并发只覆盖到「双标签页同一词条」这一层，没有做多用户压测。
|
||||
|
||||
## #8 审核整改（R1～R3，2026-09-11）
|
||||
|
||||
独立审核（Claude Code）只读审阅提交 `0328505`／`ec5ec2d`，指出三处影响验收标准第 2、3 条的问题。三点均先在 `lexgo_test_issue8` 写复现用例观察到失败，再修复并转绿。
|
||||
|
||||
| 问题 | 现象与根因 | 修复 | 回归用例 |
|
||||
|---|---|---|---|
|
||||
| R2 并发同键返回 500 | 两个同时到达、带同一 `answerId` 的请求都在加锁前查不到记录；后者进入 stale 分支插入相同 `answer_key`，触发唯一键冲突返回 500 | 取得词条行锁后再用加锁读复查一次答案键，命中直接返回首次结果；stale 插入遇到 1062 也转为返回记录 | `TestMySQLReviewConcurrentReplayOfOneAnswer`（两个 goroutine 同键提交，两个都 2xx、`review_count = 1`、只有一条答案记录） |
|
||||
| R3 编辑文本会重排复习 | `saveTerm` 无条件调用 `syncTermReview`，编辑释义/例句也会把 `due_at` 重算，逾期词条被挤出当天队列 | 保存前加锁读取旧行，只有新建或状态/等级实际变化才移动 `due_at`；缺行时补建排期行 | `TestMySQLReviewEditKeepsSchedule`（逾期 3 级词只改释义：`due_at` 与队列不变；改等级则重排） |
|
||||
| R3 附带发现：面板保存把等级重置为 1 | 阅读器面板只提交状态不提交等级，`termLevel` 对缺省等级一律返回 1，于是 4 级词改一个错字会掉到 1 级 | 保存未提及等级时保留已获得的等级；只有进入 `学习中` 才从 1 开始 | `TestMySQLReviewPanelSaveKeepsLevel`（4 级词面板式保存后仍为 4 级，且排期不变；退出再进入学习中则从 1 开始） |
|
||||
|
||||
**答案契约随之明确**：作答响应 `result` 只取 `applied`/`stale`，另加 `duplicate` 布尔标记。重放返回首次结果并把 `duplicate` 置真，客户端因此可以按首次结果计数：网络把响应丢掉后点「重试提交」拿到 `duplicate=true` 的 `applied`，本轮计数正常增加，完成页不会退化成「今天没有到期词条」。已应用的作答返回 201，重放与 stale 返回 200（R1）。
|
||||
|
||||
**提示与注释（R4、R5）**：卡片因 `stale` 离开时页面显示 `role="status"` 提示「该词已在其他页面复习，本次未计分。」，重放且首次为 stale 时显示「该词已按上一次的评分记录，未重复计分。」；本轮只解决卡片而没有新计分时，完成页显示「本轮没有新的计分：N 个词条已在其他页面复习。」。`answerId` 的作用域注释改为「每张卡片一个，失败重试复用」，与 `answerIdFor` 的实现一致。
|
||||
|
||||
**流程记录（R6）**：评论 7769 的方案写的是在 `lexgo_terms` 上增加列，实际实现改为独立表 `lexgo_term_reviews`（加法迁移可重试、不对既有表做 ALTER）。该变更在实施评论 7776 与 Wiki 中说明了原因，但没有按「数据结构变化先更新工单」的要求在实施前追加变更评论；本页与上文契约按实际实现记录，方案评论中的「新增列均有默认值」以独立表为准。
|
||||
|
||||
整改后重跑：Go 单元与集成测试（专用库 `lexgo_test_issue8`）44 个顶层用例全部通过、0 跳过；学习端 73 项单测、类型检查、构建与 5 项 E2E 通过；管理端 31 项与 lint 通过；治理 56 项与严格检查通过；真实 API＋MySQL 42 项检查通过（新增 4 项针对 R3 与重放契约）；真实浏览器复核面板保存与复习闭环通过。截图 `.local/evidence/issue8-fixed-summary.png`。
|
||||
|
||||
## #9 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue9，不借用其他测试库。本单**不改动数据库结构**，所以没有迁移步骤，回退只需换回旧二进制。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue9 python scripts/server.py test-integration` | 50 个顶层用例全部通过、0 跳过；含 #9 新增 6 个上传用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 80 项单测、类型检查、构建、6 项 E2E 全部通过 |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：有效 UTF-8（含 CRLF、制表符、弯引号、em dash、省略号、emoji、组合字符）字节级往返、UTF-8 BOM 剥离且不进原文、只有 BOM、非法 UTF-8、Latin-1、UTF-16 大小端、NUL 字节、空文件、只有空白、超限与恰好边界（2 MiB、100000 码点）、缺 `file`、缺标题、缺或错误 `language`、缺或过短 `requestId`、未知字段、追加路径携带 `language`、非 multipart 请求、未登录、恶意文件名不影响存储、重复上传只产生一章、同编号换内容 409、追加他人书籍 404、两账号隔离、上传与粘贴共用同一任务管线。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 26 项检查通过（凭据只从本机安全配置读入进程），覆盖有效文件与阅读器原文逐字节一致、BOM 不进入原文、五类无效文件、越权与未登录拒绝、两账号隔离、恶意文件名不泄漏；随后用临时 Playwright 用例在真实学习端完成「登录→切换 TXT→选择真实 UTF-8 文件→上传→处理中就绪→阅读器原文逐字符一致」的闭环，并复核 UTF-16 文件在浏览器预检阶段被拒。截图保存在本机 `.local/evidence/`（issue9-invalid-encoding.png、issue9-upload-processing.png、issue9-reader.png），临时用例运行后删除。
|
||||
|
||||
未验证：真实手机触屏详细证据与完整备份恢复演练仍属既有缺口（#14/#15）；本单只用桌面浏览器检查。大文件并发上传只按单槽并发门设计，没有做多用户压力测试。UTF-16/GB18030 转码与按空行自动分章不在本单。
|
||||
|
||||
## #10 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue9（沿用上一单的库），不借用其他测试库。本单**不改动数据库结构**，没有迁移步骤，回退只需换回旧二进制。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue9 python scripts/server.py test-integration` | 59 个顶层用例全部通过、0 跳过；含 #10 新增 9 个用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 94 项单测、类型检查、构建、7 项 E2E 全部通过 |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：改名不改状态也不新建任务、标题/正文校验、正文变化产生新版本且保留章节编号、**处理中改正文后旧任务只标 `superseded` 且不触碰新版本**、旧版本任务重试 409、重复保存与改回原内容不新建任务、内容与版本不一致时按 `content_changed` 失败、恢复扫描作废过期版本且不重排新版本、编辑用原文对任意状态可读、删除章节后序号连续且导航正确、删除书籍级联清理章节与任务、删除期间在途任务不复活内容、并发删除同一章一个成功一个 404、重复删除 404、跨用户改名/编辑/删除/读原文一律 404、未登录 401、个人词条与复习排期在删除后完全保留。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 36 项检查通过（凭据只从本机安全配置读入进程，脚本可重复运行并自行清理 fixture），覆盖改名、编辑产生新版本并重新处理、重复保存幂等、删除章节重排序号、删除书籍级联、个人词条与复习队列在删除后保留、越权与非法输入拒绝；随后用临时 Playwright 用例在真实学习端完成「导入→改名→编辑正文→新版就绪→删除章节→删除书籍→书库提示」的闭环，并核对阅读器原文等于新版本。截图保存在本机 `.local/evidence/`（issue10-book-after-edit.png、issue10-chapter-deleted.png、issue10-book-deleted.png），临时用例运行后删除。
|
||||
|
||||
未验证：真实手机触屏详细证据与完整备份恢复演练仍属既有缺口（#14/#15）；本单只用桌面浏览器检查。并发只覆盖「同一章节并发删除」与「处理中编辑」两类，没有做多用户压力测试。浏览器 textarea 会把该章的 CRLF 归一为 LF，属已知边界，已记入业务规则页。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Product-Requirements-Overview
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Product-Requirements-Overview.-
|
||||
wiki_revision: db82eeb1d509b8d3c4a2204143cfb0e829b6b083
|
||||
synchronized_at: 2026-09-10T12:43:32Z
|
||||
wiki_revision: 82d714b949d38f20586fbe42797b2dec6b0f19f1
|
||||
synchronized_at: 2026-09-13T15:20:17Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 产品需求总览
|
||||
@@ -223,3 +223,43 @@ Anki（U20）、YouTube/Jellyfin 远程字幕（U07/A08）、全量旧实例迁
|
||||
## 试用前管理能力增补
|
||||
|
||||
用户于 2026-09-10 验收 #2，并批准新增 #18 登录日志与操作审计：两个管理员列表、查询筛选分页、必要字段记录、权限及 90 天保留清理。沿用现有 go-admin 管理布局，无需独立高保真原型。已通过用户验收，纳入 #16“邀请用户试用前完成”；不替代 #3 英语分词和 #4 阅读选择验证，也不引入其他 go-admin 默认模块。
|
||||
|
||||
## 实施进度增补（#5）
|
||||
|
||||
2026-09-10：#5“粘贴英语文本，处理后进入本人章节阅读”已实现并待用户验收，覆盖 F01 基础（书库与章节）、F02（粘贴导入）、F04 基础（处理状态与失败重试）与 F05 原文（可读原文与章节切换）。本节取代此前“阅读、导入尚未实现”的表述：粘贴导入与原文阅读已实现；点词查词、词典、个人词语状态、复习与统计仍未实现（#6～#15）。schema 升级为 v3，新增 lexgo_books、lexgo_chapters、lexgo_ingest_jobs。
|
||||
|
||||
范围边界不变：本单按 Go 处理，不接入 Python NLP，因此不产生 token、lemma 或词典索引；Go＋Python NLP 与全 Go 路线仍未确认，正式接入前必须由用户确认。真机手机详细证据仍缺失（#4 缺口保持）。
|
||||
|
||||
|
||||
## 全 Go 正式架构决定（2026-09-11，#6）
|
||||
|
||||
用户已明确选择全 Go：正式英语分词、原文位置映射、本地词典解析和词形候选查询由 Go 后端完成，不运行 Python NLP 服务。此前“Python 建议/全 Go 未决”仅为历史决策记录，由本决定覆盖；spikes/english 保留历史验证，不接入产品。#6 按该方向实施，当前方案见工单最新启动评论；WordNet 3.0 仍为首个资源（英语释义），词形规则候选不等同于 spaCy 上下文消歧，原文及个人学习状态不按候选合并。
|
||||
|
||||
|
||||
## #6 交付范围更新（2026-09-11）
|
||||
|
||||
用户确认全Go后，英语词典配置与阅读器点词查义已实现待验收：共享WordNet3.0英语释义、管理员导入/启停、本人章节点击/键盘查词、加载/无结果/资源不可用/网络失败/关闭状态。词形结果是规则候选，不提供上下文词性消歧。个人释义输入是未保存临时草稿，#7才持久化。沿用已验收v1；手机底部45dvh面板自动测试通过，真机缺口保留。#5已关闭并合入main，#6尚不关闭。
|
||||
|
||||
## #7 交付范围更新（2026-09-11）
|
||||
|
||||
F07 的个人词语记录已于 2026-09-11 通过用户验收：阅读器可以保存与修改个人释义、例句和状态（新词/学习中/已知/忽略），同一词形在本人其他章节显示相同状态与高亮，两个账号的数据互不影响。个人释义与共享 WordNet 词典分开存储，个人释义不进入审计日志。schema 升级为 v5，新增 lexgo_terms。
|
||||
|
||||
仍未实现并留给后续工单：词汇库搜索与编辑（#12）、短语（#11）、到期复习与等级推进（#8）、阅读完成与进度（#13）、TXT 导入（#9）、书籍章节编辑删除（#10）。词形候选仍不提供上下文消歧，个人学习状态不按候选合并；例句不自动关联原文句子。
|
||||
|
||||
## #8 交付范围更新（2026-09-11）
|
||||
|
||||
F10 的单词到期复习已于 2026-09-11 通过用户验收（包含独立审核整改 R1～R3）：按固定间隔表取本人当前语言的到期词条，正面显示词与挖空例句，显示答案后按「认识/答对」「不认识/答错」「再学一次」评分；答对升级并排下次复习，答错降级并立即回到本轮，再学一次不改等级并回到本轮。重复提交、网络重发与双标签页都不会重复更新次数和间隔；已知与忽略的词条不入队。管理端无改动。
|
||||
|
||||
仍未实现并留给后续工单：短语复习（#11）、词汇库搜索与编辑（#12）、阅读完成与进度统计（#13）、TXT 导入（#9）、书籍章节编辑删除（#10）。复习范围筛选与策略配置（X11）、练习模式（X08）、FSRS 仍在范围外。
|
||||
|
||||
## #9 交付范围更新（2026-09-11）
|
||||
|
||||
F03 的 TXT 文件导入已于 2026-09-13 通过用户验收：学习端导入页新增「粘贴文本 / TXT 文件」来源切换，选择 UTF-8 的 .txt 文件后经大小、空文件与编码校验进入与粘贴相同的处理与阅读流程，失败可重试。只支持 UTF-8（允许可选 BOM）且不替换损坏字符；UTF-16 与其他编码会被明确拒绝；文件只在内存中解码、不写临时文件，客户端文件名不参与任何路径也不入库；重复上传同一文件只产生一章。schema 无变化。
|
||||
|
||||
仍未实现并留给后续工单：书籍与章节的编辑删除（#10）、短语选择与保存（#11）、词汇库搜索与编辑（#12）、阅读完成与进度（#13）、桌面与手机体验补齐（#14）、自托管试用交付与完整恢复（#15）。EPUB/PDF/字幕、UTF-16 转码、按空行自动分章与断点续传不在本单范围。
|
||||
|
||||
## #10 交付范围更新（2026-09-11）
|
||||
|
||||
F01 的编辑与删除已实现，待用户验收：学习端可改书名、改章节标题、编辑章节正文并按新版本重新处理，可用确认弹窗删除章节或整本书。编辑正文产生明确版本，旧处理结果被标为 `superseded` 而不覆盖新版本；删除在事务内完成并重排剩余章节序号，个人词条、复习排期与作答记录一律保留。本次没有数据库结构变化。
|
||||
|
||||
仍未实现并留给后续工单：短语选择与保存（#11）、词汇库搜索与编辑（#12）、阅读完成与进度（#13）、桌面与手机体验补齐（#14）、自托管试用交付与完整恢复（#15）。封面与音频附件（#21）、回收站/撤销、批量操作、章节跨书移动与语言变更不在本单范围。
|
||||
|
||||
+17
-2
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Home
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Home
|
||||
wiki_revision: a645b99632c70734b493ed030ad6d4d2acfd13be
|
||||
synchronized_at: 2026-09-10T14:52:14Z
|
||||
wiki_revision: ea264da6747bd141ec2d67011830320452563415
|
||||
synchronized_at: 2026-09-13T15:20:16Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# LexGo 文档入口
|
||||
@@ -63,3 +63,18 @@ Quant-UX 原型 v1 已通过用户验收。[桌面预览](https://qux.ilapage.cn
|
||||
|
||||
|
||||
#4 阅读选择小样已实现桌面鼠标/键盘、范围调整与原文位置验证，入口 http://127.0.0.1:5184/；固定版本 LinguaCafe 四项源码对照已记录。用户已验收并关闭 #4；真实手机详细测试证据仍缺失，详见本地验证页。#21 仍待实施。
|
||||
|
||||
#5 粘贴导入与章节阅读已实现，待用户验收：schema v3 新增 books/chapters/ingest_jobs，学习端具备粘贴导入、书库、处理状态与原文阅读；查词、词典与复习仍未实现。本单按 Go 处理，Go＋Python NLP 与全 Go 路线仍未决，正式接入前须用户确认。
|
||||
|
||||
|
||||
## 当前进度（2026-09-11）
|
||||
|
||||
#5已验收，#17/#19/#20/#22/#23按依赖顺序合入main。#6已按用户确认的全Go方向实现英语词典与阅读点词，2026-09-11通过用户验收，PR #25已合入main；原文/账户隔离保留。正式NLP不使用Python服务。管理端“英语词典”导入指定WordNet3.0，学习端打开本人章节即可查词。个人释义与学习状态已实现持久化（#7，待用户验收）。详见#6工单、架构和本地开发页面。
|
||||
|
||||
#7 个人词条已实现并于 2026-09-11 通过用户验收（schema v5 新增 lexgo_terms）：阅读器可保存释义、例句与状态，同一词形在本人其他章节显示一致高亮，两个账号数据独立；保存幂等，跨账号与篡改身份均被拒绝；PR #26 已 fast-forward-only 合入 main。
|
||||
|
||||
#8 单词到期复习已于 2026-09-11 通过用户验收（schema v6 新增 lexgo_term_reviews 与 lexgo_review_answers）：固定间隔表（1/2/4/7/15/30/60 天）、答对升级、答错或再学立即回队、已知与忽略不入队；重复提交、网络重发与双标签页都只记账一次，到期判定用 UTC 绝对时刻而不引入本地日边界。学习端新增「到期复习」入口，卡片正面显示词与挖空例句、答案面显示个人释义，并有完成页与空队列页。独立审核指出的并发同键 500、编辑文本重排复习与面板保存重置等级三项已整改并复测；PR #27 已 fast-forward-only 合入 main。第 3 阶段「首条学习闭环」#5～#8 全部验收。
|
||||
|
||||
#9 TXT 文件导入已于 2026-09-13 通过用户验收：导入页新增「粘贴文本 / TXT 文件」来源切换，只接受 UTF-8（允许可选 BOM）且不替换损坏字符，UTF-16 与其他编码会被明确拒绝；文件只在内存中解码、不写临时文件，客户端文件名不参与任何路径也不入库；上传与粘贴共用同一分章、任务与幂等规则，重复上传同一文件只产生一章。本次没有数据库结构变化；PR #28 已 fast-forward-only 合入 main。
|
||||
|
||||
#10 编辑与删除书籍章节已实现，待用户验收：可改书名、改章节标题、编辑章节正文并按新版本重新处理，也可用确认弹窗删除章节或整本书。编辑正文产生明确版本，旧处理结果会被标为 superseded 而不覆盖新版本；删除在事务内完成并重排剩余章节序号，已保存的个人词条、复习排期与作答记录一律保留。本次没有数据库结构变化。
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
import { expect, test, type Page } from '@playwright/test'
|
||||
|
||||
// Renaming, editing and deleting through the real dialogs of the accepted prototype, against
|
||||
// a mocked API.
|
||||
test('rename the book, edit a chapter into a new version and delete both', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-editor', role: 'learner' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const first = { id: 9, bookId: 1, ordinal: 1, title: 'First chapter', status: 'ready', charCount: 24, errorReason: '', errorMessage: '', jobId: 5, ...timestamps }
|
||||
const second = { id: 10, bookId: 1, ordinal: 2, title: 'Second chapter', status: 'ready', charCount: 12, errorReason: '', errorMessage: '', jobId: 6, ...timestamps }
|
||||
let bookTitle = 'A small step'
|
||||
let chapters = [first, second]
|
||||
let firstText = 'Mira opened the workshop.\n'
|
||||
let processing = false
|
||||
const requests: string[] = []
|
||||
|
||||
const listBook = () => ({
|
||||
book: { id: 1, title: bookTitle, language: 'en' },
|
||||
chapters: chapters.map(item => (item.id === 9 && processing ? { ...item, status: 'processing' } : item)),
|
||||
})
|
||||
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
requests.push(`${method} ${path}`)
|
||||
let data: unknown = null
|
||||
let status = 200
|
||||
if (path === '/api/v1/login') data = { token: 'fictional-session', user }
|
||||
else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books' && method === 'GET') {
|
||||
data = { items: bookTitle === '' ? [] : [{ id: 1, title: bookTitle, language: 'en', chapterCount: chapters.length, pendingCount: 0, processingCount: 0, readyCount: chapters.length, failedCount: 0, ...timestamps }] }
|
||||
} else if (path === '/api/v1/books/1' && method === 'PATCH') {
|
||||
bookTitle = (route.request().postDataJSON() as { title: string }).title
|
||||
data = { book: { id: 1, title: bookTitle, language: 'en' } }
|
||||
} else if (path === '/api/v1/books/1' && method === 'DELETE') {
|
||||
chapters = []
|
||||
bookTitle = ''
|
||||
data = { deleted: { bookId: 1, chapters: 2, remaining: 0 } }
|
||||
} else if (path === '/api/v1/books/1') data = listBook()
|
||||
else if (path === '/api/v1/chapters/9/source') data = { source: { id: 9, bookId: 1, ordinal: 1, title: first.title, text: firstText, status: 'ready', contentSha256: 'sha-a', charCount: [...firstText].length } }
|
||||
else if (path === '/api/v1/chapters/9' && method === 'PATCH') {
|
||||
const body = route.request().postDataJSON() as { title?: string; text?: string }
|
||||
const changed = body.text !== undefined && body.text !== firstText
|
||||
if (body.title !== undefined) first.title = body.title
|
||||
if (changed) {
|
||||
firstText = body.text as string
|
||||
processing = true
|
||||
first.status = 'pending'
|
||||
setTimeout(() => { processing = false; first.status = 'ready' }, 400)
|
||||
}
|
||||
status = 200
|
||||
data = { chapter: first, job: changed ? { id: 7, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps } : null, versionChanged: changed }
|
||||
} else if (path === '/api/v1/chapters/9' && method === 'DELETE') {
|
||||
chapters = chapters.filter(item => item.id !== 9).map((item, index) => ({ ...item, ordinal: index + 1 }))
|
||||
data = { deleted: { chapterId: 9, bookId: 1, remaining: chapters.length } }
|
||||
}
|
||||
await route.fulfill({ status, json: { code: 200, data } })
|
||||
})
|
||||
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
await page.getByRole('link', { name: 'A small step' }).click()
|
||||
await expect(page.getByRole('heading', { name: 'A small step' })).toBeVisible()
|
||||
await expect(page.getByText('封面使用系统默认样式')).toBeVisible()
|
||||
|
||||
// Rename the book through the dialog.
|
||||
await page.getByTestId('edit-book').click()
|
||||
await expect(page.getByLabel('书名', { exact: true })).toHaveValue('A small step')
|
||||
await page.getByLabel('书名', { exact: true }).fill('A long step')
|
||||
await page.getByTestId('save-book').click()
|
||||
await expect(page.getByTestId('book-notice')).toContainText('书名已更新')
|
||||
await expect(page.getByRole('heading', { name: 'A long step' })).toBeVisible()
|
||||
|
||||
// Edit the chapter text: the new version re-processes and later becomes ready again.
|
||||
await page.getByTestId('edit-chapter-9').click()
|
||||
await expect(page.getByLabel('章节标题', { exact: true })).toHaveValue('First chapter')
|
||||
await expect(page.getByLabel('正文', { exact: true })).toHaveValue(firstText)
|
||||
await page.getByLabel('正文', { exact: true }).fill('A replacement body.\n')
|
||||
await page.getByTestId('save-chapter').click()
|
||||
await expect(page.getByTestId('book-notice')).toContainText('已保存为新版本，正在重新处理')
|
||||
// The saved version is queued first and becomes readable again when the worker finishes.
|
||||
await expect(page.locator('.chapter-row').first()).toContainText('待处理')
|
||||
await expect(page.locator('.chapter-row').first()).toContainText('已就绪', { timeout: 10000 })
|
||||
|
||||
// Deleting a chapter asks first, then reports the remaining count.
|
||||
await page.getByTestId('edit-chapter-9').click()
|
||||
await page.getByTestId('delete-chapter').click()
|
||||
await expect(page.locator('.el-message-box__message').last()).toContainText('本章正文将被删除，已保存词条保留')
|
||||
await page.locator('.el-message-box').last().getByRole('button', { name: '取消' }).click()
|
||||
expect(requests.filter(entry => entry === 'DELETE /api/v1/chapters/9')).toHaveLength(0)
|
||||
await page.getByTestId('delete-chapter').click()
|
||||
await page.locator('.el-message-box').last().getByRole('button', { name: '确认删除章节' }).click()
|
||||
await expect(page.getByTestId('book-notice')).toContainText('章节已删除 · 剩余 1 章')
|
||||
await expect(page.locator('.chapter-row')).toHaveCount(1)
|
||||
|
||||
// Deleting the book asks first and returns to the library without it.
|
||||
await page.getByTestId('delete-book').click()
|
||||
await expect(page.locator('.el-message-box__message').last()).toContainText('已保存的生词和短语将保留')
|
||||
await page.locator('.el-message-box').last().getByRole('button', { name: '确认删除' }).click()
|
||||
await expect(page).toHaveURL(/\?deleted=\d+$/)
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
await expect(page.getByTestId('library-notice')).toContainText('书籍已删除 · 已保存的生词和短语仍保留在生词本')
|
||||
await expect(page.getByRole('link', { name: 'A long step' })).toHaveCount(0)
|
||||
})
|
||||
@@ -0,0 +1,157 @@
|
||||
import { expect, test } from '@playwright/test'
|
||||
|
||||
test('paste English text, watch a chapter finish processing, then read it verbatim', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-reader', role: 'learner' }
|
||||
const book = { id: 1, title: '虚构样例书', language: 'en' }
|
||||
const chapterTitle = '虚构样例第一章'
|
||||
// Line breaks, a tab and repeated spaces must survive the whole round trip.
|
||||
const pasted = 'First line of the chapter.\n\tIndented line.\nTwo spaces kept.\n\nLast line.\n'
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
// The worker reports the fresh chapter as processing until the worker settles it.
|
||||
let status: 'processing' | 'ready' = 'processing'
|
||||
// The personal record the learner saves during this run, served back on reload.
|
||||
let savedTerm: { id: number; term: string; originalForm: string; definition: string; examples: string[]; status: string; level: number } | null = null
|
||||
const chapterPayload = () => ({
|
||||
id: 55,
|
||||
bookId: book.id,
|
||||
ordinal: 1,
|
||||
title: chapterTitle,
|
||||
status,
|
||||
charCount: [...pasted].length,
|
||||
errorReason: '',
|
||||
errorMessage: '',
|
||||
jobId: 7,
|
||||
...timestamps,
|
||||
})
|
||||
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
let data: unknown = null
|
||||
let statusCode = 200
|
||||
if (path === '/api/v1/login') {
|
||||
expect(route.request().postDataJSON()).toEqual({ username: user.username, password: 'fictional-password' })
|
||||
data = { token: 'fictional-session', user }
|
||||
} else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books' && method === 'GET') {
|
||||
data = { items: [{ ...book, chapterCount: 0, pendingCount: 0, processingCount: 0, readyCount: 0, failedCount: 0, ...timestamps }] }
|
||||
} else if (path === '/api/v1/books' && method === 'POST') {
|
||||
const body = route.request().postDataJSON() as { requestId: string; title: string; text: string; language: string }
|
||||
expect(body.requestId).toMatch(/^[0-9a-f-]{36}$/)
|
||||
expect(body).toMatchObject({ title: chapterTitle, text: pasted, language: 'en' })
|
||||
statusCode = 201
|
||||
data = {
|
||||
book,
|
||||
chapter: chapterPayload(),
|
||||
job: { id: 7, bookId: book.id, chapterId: 55, status, attempts: 0, errorReason: '', errorMessage: '', ...timestamps },
|
||||
duplicate: false,
|
||||
}
|
||||
} else if (path === '/api/v1/books/1') data = { book, chapters: [chapterPayload()] }
|
||||
else if (path === '/api/v1/chapters/55') {
|
||||
data = {
|
||||
book,
|
||||
chapter: { ...chapterPayload(), contentSha256: 'fictional-sha256', ...(status === 'ready' ? { originalText: pasted } : {}) },
|
||||
navigation: { previousChapterId: null, nextChapterId: null },
|
||||
}
|
||||
} else if (path === '/api/v1/chapters/55/tokens') {
|
||||
let offset = 0
|
||||
const tokens = (pasted.match(/[A-Za-z]+|\s+|[^A-Za-z\s]+/g) ?? []).map(text => {
|
||||
const start = offset
|
||||
offset += text.length
|
||||
return { text, start, end: offset, startUtf16: start, endUtf16: offset, kind: /^[A-Za-z]+$/.test(text) ? 'word' : /^\s+$/.test(text) ? 'space' : 'punctuation' }
|
||||
})
|
||||
data = {
|
||||
textSha256: 'fictional-sha256',
|
||||
tokens: tokens.map(token => savedTerm && token.text === 'First'
|
||||
? { ...token, term: { id: savedTerm.id, status: savedTerm.status, level: savedTerm.level } }
|
||||
: token),
|
||||
}
|
||||
} else if (path === '/api/v1/terms' && method === 'POST') {
|
||||
const body = route.request().postDataJSON() as { chapterId: number; start: number; end: number; definition: string; examples: string[]; status: string }
|
||||
expect(body).toEqual({ chapterId: 55, start: 0, end: 5, definition: '虚构的个人释义', examples: ['A fictional example.'], status: 'new' })
|
||||
savedTerm = { id: 9, term: 'first', originalForm: 'First', definition: body.definition, examples: body.examples, status: body.status, level: 0 }
|
||||
statusCode = 201
|
||||
data = { term: savedTerm, created: true }
|
||||
} else if (path === '/api/v1/terms/9') data = { term: savedTerm }
|
||||
else if (path === '/api/v1/lookup') {
|
||||
expect(route.request().postDataJSON()).toEqual({ chapterId: 55, start: 0, end: 5 })
|
||||
data = { status: 'exact', query: 'first', matchedForm: 'first', candidates: [], entries: [{ lemma: 'first', pos: 'adjective', definition: 'Coming before all others.', examples: ['The first fictional chapter.'] }] }
|
||||
}
|
||||
await route.fulfill({ status: statusCode, json: { code: 200, data } })
|
||||
})
|
||||
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
|
||||
// The library lists the caller's book.
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
await expect(page.getByRole('link', { name: book.title })).toBeVisible()
|
||||
|
||||
// Paste text through the import form.
|
||||
await page.getByRole('button', { name: '导入内容' }).click()
|
||||
await expect(page.getByRole('heading', { name: '导入英文内容' })).toBeVisible()
|
||||
await page.getByLabel('标题').fill(chapterTitle)
|
||||
await page.getByLabel('正文').fill(pasted)
|
||||
await page.getByRole('button', { name: '开始处理' }).click()
|
||||
|
||||
// The new book opens with the chapter still processing…
|
||||
await expect(page).toHaveURL(/\/books\/1$/)
|
||||
await expect(page.getByText('处理中')).toBeVisible()
|
||||
|
||||
// …and the browser poll turns it ready without a page reload.
|
||||
status = 'ready'
|
||||
await expect(page.getByText('已就绪')).toBeVisible({ timeout: 15000 })
|
||||
|
||||
// Open the chapter and check the pasted text survived verbatim.
|
||||
await page.getByRole('link', { name: chapterTitle }).click()
|
||||
await expect(page).toHaveURL(/\/chapters\/55$/)
|
||||
const readerText = page.locator('.reader-text')
|
||||
await expect(readerText).toBeVisible()
|
||||
expect(await readerText.evaluate(element => element.textContent)).toBe(pasted)
|
||||
expect(await readerText.evaluate(element => getComputedStyle(element).whiteSpace)).toBe('pre-wrap')
|
||||
await expect(page.getByRole('button', { name: '上一章' })).toBeDisabled()
|
||||
await expect(page.getByRole('button', { name: '下一章' })).toBeDisabled()
|
||||
const word = page.locator('.reader-word').first()
|
||||
await expect(word).toHaveAttribute('aria-label', '查询 First')
|
||||
await word.focus()
|
||||
await word.press('Enter')
|
||||
await expect(page.getByText('Coming before all others.')).toBeVisible()
|
||||
expect(await readerText.evaluate(element => element.textContent)).toBe(pasted)
|
||||
|
||||
// Browser narrow viewport check only; this is not real-device acceptance.
|
||||
await page.setViewportSize({ width: 390, height: 844 })
|
||||
await word.click()
|
||||
await expect(page.getByText('Coming before all others.')).toBeVisible()
|
||||
const panelBounds = await page.locator('.lookup-panel').boundingBox()
|
||||
expect(panelBounds!.y + panelBounds!.height).toBeLessThanOrEqual(845)
|
||||
expect(panelBounds!.height).toBeLessThanOrEqual(844 * 0.46)
|
||||
expect(await page.locator('.lookup-panel').evaluate(element => getComputedStyle(element).position)).toBe('fixed')
|
||||
const wordBounds = await word.boundingBox()
|
||||
expect(wordBounds!.y + wordBounds!.height).toBeLessThanOrEqual(panelBounds!.y)
|
||||
expect(await page.getByLabel('我的释义 新词条').inputValue()).toBe('')
|
||||
|
||||
// Save a personal record: definition, example and the default status.
|
||||
await page.setViewportSize({ width: 1280, height: 900 })
|
||||
await page.getByLabel('我的释义 新词条').fill('虚构的个人释义')
|
||||
await page.getByLabel('例句 每行一条，最多 5 条').fill('A fictional example.')
|
||||
await page.getByRole('button', { name: '保存到生词本' }).click()
|
||||
await expect(page.getByText('已保存 · 新词')).toBeVisible()
|
||||
await expect(word).toHaveClass(/is-new/)
|
||||
expect(await readerText.evaluate(element => element.textContent)).toBe(pasted)
|
||||
await page.getByRole('button', { name: '关闭释义' }).press('Escape')
|
||||
await expect(page.locator('.lookup-panel')).toHaveCount(0)
|
||||
await expect(word).toBeFocused()
|
||||
|
||||
// Returning to the chapter shows the same state and the stored text.
|
||||
await page.reload()
|
||||
const reloadedWord = page.locator('.reader-word').first()
|
||||
await expect(reloadedWord).toHaveAttribute('aria-label', '查询 First，已保存')
|
||||
await expect(reloadedWord).toHaveClass(/is-new/)
|
||||
await reloadedWord.click()
|
||||
await expect(page.getByLabel('我的释义 已保存')).toHaveValue('虚构的个人释义')
|
||||
await expect(page.getByLabel('例句 每行一条，最多 5 条')).toHaveValue('A fictional example.')
|
||||
await expect(page.getByRole('radio', { name: '新词' })).toBeChecked()
|
||||
})
|
||||
@@ -0,0 +1,120 @@
|
||||
import { expect, test } from '@playwright/test'
|
||||
|
||||
// The review round against a mocked API: queue, reveal, grades, relearn and the summary.
|
||||
test('review the due words, requeue a missed one and finish the round', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-reviewer', role: 'learner' }
|
||||
const first = {
|
||||
id: 7, term: 'curiosity', originalForm: 'curiosity', definition: '好奇心；求知欲',
|
||||
examples: ['Learning begins with curiosity.'], status: 'new', level: 0,
|
||||
dueAt: '2026-09-11T10:00:00Z', reviewCount: 0,
|
||||
}
|
||||
const second = {
|
||||
id: 8, term: 'step', originalForm: 'step', definition: '一步',
|
||||
examples: ['Take a small step, every day.'], status: 'learning', level: 2,
|
||||
dueAt: '2026-09-11T10:00:00Z', reviewCount: 4,
|
||||
}
|
||||
// The wrong answer puts its word back in the same round, exactly as the server does.
|
||||
const submitted: { grade: string; answerId: string }[] = []
|
||||
let requeued = false
|
||||
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
let data: unknown = null
|
||||
let status = 200
|
||||
if (path === '/api/v1/login') data = { token: 'fictional-session', user }
|
||||
else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books') data = { items: [] }
|
||||
else if (path === '/api/v1/reviews/queue') {
|
||||
data = { items: requeued ? [second] : [first, second], total: requeued ? 1 : 2 }
|
||||
} else if (path.endsWith('/answers') && method === 'POST') {
|
||||
const body = route.request().postDataJSON() as { answerId: string; grade: string; expectedDueAt: string }
|
||||
submitted.push({ grade: body.grade, answerId: body.answerId })
|
||||
expect(body.answerId).toMatch(/^[0-9a-f-]{36}$/)
|
||||
const term = path.includes('8') ? second : first
|
||||
// The client answers the card it was shown, so it echoes that card's due time.
|
||||
expect(body.expectedDueAt).toBe(term.dueAt)
|
||||
const wrong = body.grade === 'wrong'
|
||||
requeued = wrong
|
||||
const dueAtAfter = wrong ? '2026-09-11T10:05:00Z' : '2026-09-12T10:00:00Z'
|
||||
term.dueAt = dueAtAfter
|
||||
status = 201
|
||||
data = {
|
||||
result: 'applied', grade: body.grade, requeued: wrong,
|
||||
statusBefore: term.status, statusAfter: wrong ? term.status : 'learning',
|
||||
levelBefore: term.level, levelAfter: wrong ? 1 : term.level + 1,
|
||||
dueAtBefore: body.expectedDueAt, dueAtAfter,
|
||||
item: { ...term, dueAt: dueAtAfter, reviewCount: term.reviewCount + 1 },
|
||||
}
|
||||
}
|
||||
await route.fulfill({ status, json: { code: 200, data } })
|
||||
})
|
||||
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
|
||||
// The library links into the due queue.
|
||||
await page.getByRole('link', { name: '到期复习' }).click()
|
||||
await expect(page).toHaveURL(/\/review$/)
|
||||
await expect(page.getByTestId('review-position')).toHaveText('到期复习 · 1 / 2')
|
||||
await expect(page.getByRole('heading', { name: 'curiosity' })).toBeVisible()
|
||||
// The answer stays hidden, and the example shows a blank instead of the word.
|
||||
await expect(page.getByText('Learning begins with _____.')).toBeVisible()
|
||||
await expect(page.getByTestId('review-definition')).toHaveCount(0)
|
||||
|
||||
await page.getByTestId('review-reveal').click()
|
||||
await expect(page.getByTestId('review-definition')).toHaveText('好奇心；求知欲')
|
||||
await expect(page.getByTestId('review-correct')).toBeFocused()
|
||||
await page.getByTestId('review-correct').click()
|
||||
|
||||
// The second word is learning level 2 and can be sent back into the round.
|
||||
await expect(page.getByTestId('review-position')).toHaveText('到期复习 · 2 / 2')
|
||||
await expect(page.getByText('学习中 · 等级 2')).toBeVisible()
|
||||
await page.getByTestId('review-reveal').click()
|
||||
await expect(page.getByTestId('review-definition')).toHaveText('一步')
|
||||
await page.getByTestId('review-wrong').click()
|
||||
|
||||
// Requeued: the same word asks again and the round grows instead of pretending it ended.
|
||||
await expect(page.getByTestId('review-position')).toHaveText('到期复习 · 3 / 3')
|
||||
await expect(page.getByTestId('review-definition')).toHaveCount(0)
|
||||
await page.getByTestId('review-reveal').click()
|
||||
await page.getByTestId('review-correct').click()
|
||||
|
||||
await expect(page.getByTestId('review-summary')).toContainText('复习了 2 个词条 · 共 3 次作答')
|
||||
await expect(page.getByTestId('review-summary')).toContainText('答对 2 · 答错或再学 1')
|
||||
expect(submitted.map(entry => entry.grade)).toEqual(['correct', 'wrong', 'correct'])
|
||||
expect(new Set(submitted.map(entry => entry.answerId)).size).toBe(3)
|
||||
|
||||
await page.getByTestId('review-finish').click()
|
||||
await expect(page).toHaveURL(/\/$/)
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
})
|
||||
|
||||
test('an empty due queue says so instead of showing a card', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-reviewer', role: 'learner' }
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
let data: unknown = null
|
||||
if (path === '/api/v1/login') data = { token: 'fictional-session', user }
|
||||
else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books') data = { items: [] }
|
||||
else if (path === '/api/v1/reviews/queue') data = { items: [], total: 0 }
|
||||
await route.fulfill({ json: { code: 200, data } })
|
||||
})
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
await page.goto('/review')
|
||||
await expect(page.getByTestId('review-empty')).toContainText('今天没有到期词条')
|
||||
await expect(page.getByTestId('review-card')).toHaveCount(0)
|
||||
// The narrow layout keeps the grading controls reachable without horizontal overflow.
|
||||
await page.setViewportSize({ width: 390, height: 844 })
|
||||
const overflow = await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth)
|
||||
expect(overflow).toBeLessThanOrEqual(0)
|
||||
})
|
||||
@@ -0,0 +1,76 @@
|
||||
import { expect, test } from '@playwright/test'
|
||||
|
||||
// The TXT upload path against a mocked API: pre-checks, multipart body and the hand-off to
|
||||
// the same processing screen the paste path uses.
|
||||
test('upload a UTF-8 TXT file and open the created book', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-uploader', role: 'learner' }
|
||||
const book = { id: 1, title: 'Studio Notes', language: 'en' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const pasted = 'Mira opened the workshop.\r\n\r\n\tThe sign read “A small step…”\n'
|
||||
let uploaded = false
|
||||
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
let data: unknown = null
|
||||
let status = 200
|
||||
if (path === '/api/v1/login') data = { token: 'fictional-session', user }
|
||||
else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books' && method === 'GET') data = { items: uploaded ? [{ ...book, chapterCount: 1, pendingCount: 0, processingCount: 0, readyCount: 1, failedCount: 0, ...timestamps }] : [] }
|
||||
else if (path === '/api/v1/books/upload') {
|
||||
// The upload must arrive as multipart: the fields and the file are inspected directly.
|
||||
expect(route.request().headers()['content-type']).toContain('multipart/form-data')
|
||||
const raw = route.request().postData() ?? ''
|
||||
expect(raw).toContain('name="requestId"')
|
||||
expect(raw).toContain('name="language"')
|
||||
expect(raw).toContain('name="title"')
|
||||
expect(raw).toContain('Studio Notes')
|
||||
expect(raw).toContain('filename="notes.txt"')
|
||||
expect(raw).toContain('Mira opened the workshop.')
|
||||
uploaded = true
|
||||
status = 201
|
||||
data = {
|
||||
book,
|
||||
chapter: { id: 9, bookId: 1, ordinal: 1, title: 'Studio Notes', status: 'pending', charCount: [...pasted].length, errorReason: '', errorMessage: '', jobId: 5, ...timestamps },
|
||||
job: { id: 5, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps },
|
||||
duplicate: false,
|
||||
}
|
||||
} else if (path === '/api/v1/books/1') data = { book, chapters: [{ id: 9, bookId: 1, ordinal: 1, title: 'Studio Notes', status: 'ready', charCount: [...pasted].length, errorReason: '', errorMessage: '', jobId: 5, ...timestamps }] }
|
||||
else if (path === '/api/v1/chapters/9') data = { book, chapter: { id: 9, bookId: 1, ordinal: 1, title: 'Studio Notes', status: 'ready', charCount: [...pasted].length, errorReason: '', errorMessage: '', jobId: 5, contentSha256: 'fictional-sha', originalText: pasted, ...timestamps }, navigation: { previousChapterId: null, nextChapterId: null } }
|
||||
await route.fulfill({ status, json: { code: 200, data } })
|
||||
})
|
||||
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
|
||||
await page.getByRole('button', { name: '导入内容' }).click()
|
||||
// Element Plus hides the native radio behind a styled span, so the label is what a person
|
||||
// clicks; the input still carries the checked state.
|
||||
await page.locator('label.el-radio', { hasText: 'TXT 文件' }).click()
|
||||
await expect(page.getByRole('radio', { name: 'TXT 文件' })).toBeChecked()
|
||||
await expect(page.locator('textarea#text')).toHaveCount(0)
|
||||
await expect(page.getByText('仅支持 UTF-8')).toBeVisible()
|
||||
|
||||
// An unusable file is refused in the browser, before any request is made.
|
||||
await page.setInputFiles('[data-testid="file-input"]', { name: 'notes.md', mimeType: 'text/markdown', buffer: Buffer.from('# heading\n') })
|
||||
await expect(page.getByText('请选择 .txt 文件。')).toBeVisible()
|
||||
|
||||
// A UTF-8 file is accepted and its metadata is shown; the title comes from the file name.
|
||||
await page.setInputFiles('[data-testid="file-input"]', { name: 'notes.txt', mimeType: 'text/plain', buffer: Buffer.from(pasted, 'utf8') })
|
||||
await expect(page.getByTestId('file-info')).toContainText('notes.txt · UTF-8')
|
||||
await expect(page.getByLabel('标题')).toHaveValue('notes')
|
||||
await page.getByLabel('标题').fill('Studio Notes')
|
||||
|
||||
await page.getByRole('button', { name: '上传并处理' }).click()
|
||||
await expect(page).toHaveURL(/\/books\/1$/)
|
||||
await expect(page.getByText('已就绪')).toBeVisible()
|
||||
await page.getByRole('link', { name: 'Studio Notes' }).click()
|
||||
const readerText = page.locator('.reader-text')
|
||||
await expect(readerText).toBeVisible()
|
||||
// The uploaded bytes reached the reader unchanged.
|
||||
expect(await readerText.evaluate(element => element.textContent)).toBe(pasted)
|
||||
})
|
||||
@@ -0,0 +1,237 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createRouter, createMemoryHistory, type Router } from 'vue-router'
|
||||
import { ElMessageBox } from 'element-plus'
|
||||
import BookView from '../views/BookView.vue'
|
||||
import { useLibraryStore, type ChapterSource, type ChapterSummary } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const user = { id: 42, username: 'fictional-editor', role: 'learner' as const }
|
||||
const book = { id: 1, title: 'A small step', language: 'en' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const chapter = (overrides: Partial<ChapterSummary> = {}): ChapterSummary => ({
|
||||
id: 9, bookId: 1, ordinal: 1, title: 'First chapter', status: 'ready', charCount: 12,
|
||||
errorReason: '', errorMessage: '', jobId: 5, ...timestamps, ...overrides,
|
||||
})
|
||||
const source: ChapterSource = { id: 9, bookId: 1, ordinal: 1, title: 'First chapter', text: 'Mira opened the workshop.\n', status: 'ready', contentSha256: 'sha-a', charCount: 24 }
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
const fail = (msg: string, status = 400) => new Response(JSON.stringify({ code: status, msg }), { status })
|
||||
let wrapper: VueWrapper | undefined
|
||||
|
||||
function stub(name: string) {
|
||||
return { template: `<div>${name}</div>` }
|
||||
}
|
||||
|
||||
async function viewAt(path: string): Promise<Router> {
|
||||
const router = createRouter({
|
||||
history: createMemoryHistory(),
|
||||
routes: [
|
||||
{ path: '/', component: stub('LibraryStub') },
|
||||
{ path: '/books/:id', component: stub('BookStub') },
|
||||
{ path: '/import', component: stub('ImportStub') },
|
||||
],
|
||||
})
|
||||
await router.push(path)
|
||||
await router.isReady()
|
||||
return router
|
||||
}
|
||||
|
||||
/** Routes the book page's own calls; a test can override any of them. */
|
||||
function mockApi(overrides: Record<string, (init?: RequestInit) => Response | Promise<Response>> = {}): MockInstance {
|
||||
return vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
const url = String(input)
|
||||
const method = String((init as RequestInit | undefined)?.method ?? 'GET')
|
||||
for (const [key, handler] of Object.entries(overrides)) {
|
||||
if (url.includes(key)) return handler(init as RequestInit)
|
||||
}
|
||||
if (url.endsWith('/books/1') && method === 'GET') return ok({ book, chapters: [chapter()] })
|
||||
if (url.includes('/chapters/9/source')) return ok({ source })
|
||||
// A rename answers with the stored book, and a delete reports what it removed.
|
||||
if (url.endsWith('/books/1') && method === 'PATCH') {
|
||||
const body = JSON.parse(String((init as RequestInit | undefined)?.body ?? '{}')) as { title?: string }
|
||||
return ok({ book: { ...book, title: body.title ?? book.title } })
|
||||
}
|
||||
if (url.endsWith('/books/1') && method === 'DELETE') return ok({ deleted: { bookId: 1, chapters: 3 } })
|
||||
if (url.endsWith('/books/1')) return ok({ book })
|
||||
if (url.includes('/chapters/9')) return ok({ chapter: chapter(), job: null, versionChanged: false })
|
||||
return ok({})
|
||||
})
|
||||
}
|
||||
|
||||
async function openBook() {
|
||||
useSessionStore().user = { ...user }
|
||||
const router = await viewAt('/books/1')
|
||||
wrapper = mount(BookView, { attachTo: document.body, global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
return { view: wrapper, router }
|
||||
}
|
||||
|
||||
describe('book editing store', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { vi.restoreAllMocks() })
|
||||
|
||||
it('renames a book and keeps the list entry in step', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const library = useLibraryStore()
|
||||
library.books = [{ ...book, chapterCount: 1, pendingCount: 0, processingCount: 0, readyCount: 1, failedCount: 0, ...timestamps }]
|
||||
library.book = { ...book }
|
||||
const renamed = await library.renameBook(1, ' New name ')
|
||||
expect(renamed.title).toBe('New name')
|
||||
expect(library.book?.title).toBe('New name')
|
||||
expect(library.books[0]?.title).toBe('New name')
|
||||
const call = fetchMock.mock.calls.find(([, init]) => (init as RequestInit | undefined)?.method === 'PATCH')!
|
||||
expect(String(call[0])).toBe('/api/v1/books/1')
|
||||
expect(JSON.parse(String((call[1] as RequestInit).body))).toEqual({ title: 'New name' })
|
||||
})
|
||||
|
||||
it('refuses an invalid title before calling the API', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const library = useLibraryStore()
|
||||
await expect(library.renameBook(1, ' ')).rejects.toThrow('请填写标题')
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('applies a new chapter version and drops the stale reader text', async () => {
|
||||
mockApi({ '/chapters/9': () => ok({ chapter: chapter({ status: 'pending', title: 'Edited' }), job: { id: 6, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps }, versionChanged: true }) })
|
||||
const library = useLibraryStore()
|
||||
library.chapters = [chapter()]
|
||||
library.chapter = { ...chapter(), contentSha256: 'sha-a', originalText: 'old text' }
|
||||
const result = await library.updateChapter(9, { title: 'Edited', text: 'new text' })
|
||||
expect(result.versionChanged).toBe(true)
|
||||
expect(library.chapters[0]).toMatchObject({ status: 'pending', title: 'Edited', jobId: 6 })
|
||||
// The reader must not keep showing text that is no longer the stored version.
|
||||
expect(library.chapter?.originalText).toBeUndefined()
|
||||
})
|
||||
|
||||
it('keeps the reader text when only the title changed', async () => {
|
||||
mockApi({ '/chapters/9': () => ok({ chapter: chapter({ title: 'Renamed' }), job: null, versionChanged: false }) })
|
||||
const library = useLibraryStore()
|
||||
library.chapters = [chapter()]
|
||||
library.chapter = { ...chapter(), contentSha256: 'sha-a', originalText: 'kept text' }
|
||||
await library.updateChapter(9, { title: 'Renamed' })
|
||||
expect(library.chapter?.originalText).toBe('kept text')
|
||||
expect(library.chapter?.title).toBe('Renamed')
|
||||
})
|
||||
|
||||
it('deletes a chapter, reloads the book and removes a deleted book from the list', async () => {
|
||||
mockApi({
|
||||
'/chapters/9': () => ok({ deleted: { chapterId: 9, bookId: 1, remaining: 2 } }),
|
||||
'/books/1': (init) => (init?.method === 'DELETE'
|
||||
? ok({ deleted: { bookId: 1, chapters: 3 } })
|
||||
: ok({ book, chapters: [] })),
|
||||
})
|
||||
const library = useLibraryStore()
|
||||
library.books = [{ ...book, chapterCount: 3, pendingCount: 0, processingCount: 0, readyCount: 3, failedCount: 0, ...timestamps }]
|
||||
library.book = { ...book }
|
||||
library.chapters = [chapter(), chapter({ id: 10, ordinal: 2 }), chapter({ id: 11, ordinal: 3 })]
|
||||
const deleted = await library.deleteChapter(9)
|
||||
expect(deleted.remaining).toBe(2)
|
||||
expect(library.chapters.map(item => item.id)).toEqual([])
|
||||
expect(library.book?.id).toBe(1)
|
||||
const removed = await library.deleteBook(1)
|
||||
expect(removed.chapters).toBe(3)
|
||||
expect(library.books).toHaveLength(0)
|
||||
expect(library.book).toBeNull()
|
||||
})
|
||||
|
||||
it('reads the editable source of any chapter state', async () => {
|
||||
mockApi({ '/chapters/9/source': () => ok({ source: { ...source, status: 'failed' } }) })
|
||||
const library = useLibraryStore()
|
||||
const loaded = await library.loadChapterSource(9)
|
||||
expect(loaded.text).toBe(source.text)
|
||||
expect(loaded.status).toBe('failed')
|
||||
})
|
||||
})
|
||||
|
||||
describe('book editing view', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear(); vi.restoreAllMocks() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; useLibraryStore().stopPolling() })
|
||||
|
||||
it('renames the book through the dialog and reports it', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-book"]').trigger('click'); await flushPromises()
|
||||
const input = view.get('#book-title')
|
||||
expect((input.element as HTMLInputElement).value).toBe('A small step')
|
||||
await input.setValue(' Edited name ')
|
||||
await view.get('[data-testid="save-book"]').trigger('click'); await flushPromises()
|
||||
const patch = fetchMock.mock.calls.find(([, init]) => (init as RequestInit | undefined)?.method === 'PATCH')!
|
||||
expect(JSON.parse(String((patch[1] as RequestInit).body))).toEqual({ title: 'Edited name' })
|
||||
expect(view.get('[data-testid="book-notice"]').text()).toContain('书名已更新')
|
||||
})
|
||||
|
||||
it('keeps the dialog open with the server message when renaming fails', async () => {
|
||||
mockApi({ '/books/1': (init) => (init?.method === 'PATCH' ? fail('书名已存在') : ok({ book, chapters: [chapter()] })) })
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-book"]').trigger('click'); await flushPromises()
|
||||
await view.get('#book-title').setValue('Rejected')
|
||||
await view.get('[data-testid="save-book"]').trigger('click'); await flushPromises()
|
||||
expect(view.text()).toContain('书名已存在')
|
||||
expect(view.get('#book-title')).toBeTruthy()
|
||||
})
|
||||
|
||||
it('asks before deleting the book and does nothing when the learner cancels', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const confirm = vi.spyOn(ElMessageBox, 'confirm').mockRejectedValue('cancel')
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="delete-book"]').trigger('click'); await flushPromises()
|
||||
expect(confirm).toHaveBeenCalledWith(expect.stringContaining('已保存的生词和短语将保留'), '删除书籍', expect.anything())
|
||||
expect(fetchMock.mock.calls.some(([, init]) => (init as RequestInit | undefined)?.method === 'DELETE')).toBe(false)
|
||||
})
|
||||
|
||||
it('deletes the book after confirmation and returns to the library', async () => {
|
||||
const fetchMock = mockApi()
|
||||
vi.spyOn(ElMessageBox, 'confirm').mockResolvedValue('confirm' as never)
|
||||
const { view, router } = await openBook()
|
||||
await view.get('[data-testid="delete-book"]').trigger('click'); await flushPromises()
|
||||
expect(fetchMock.mock.calls.some(([url, init]) => String(url).endsWith('/books/1') && (init as RequestInit | undefined)?.method === 'DELETE')).toBe(true)
|
||||
expect(router.currentRoute.value.path).toBe('/')
|
||||
expect(router.currentRoute.value.query.deleted).toBe('3')
|
||||
})
|
||||
|
||||
it('edits a chapter: loads the source, saves title and text and reports the new version', async () => {
|
||||
const fetchMock = mockApi({ '/chapters/9': (init) => (init?.method === 'PATCH'
|
||||
? ok({ chapter: chapter({ status: 'pending', title: 'Edited chapter' }), job: { id: 6, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps }, versionChanged: true })
|
||||
: ok({ source })) })
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-chapter-9"]').trigger('click'); await flushPromises()
|
||||
expect((view.get('#chapter-title').element as HTMLInputElement).value).toBe('First chapter')
|
||||
expect((view.get('#chapter-text').element as HTMLTextAreaElement).value).toBe(source.text)
|
||||
await view.get('#chapter-title').setValue('Edited chapter')
|
||||
await view.get('#chapter-text').setValue('A new body.\n')
|
||||
await view.get('[data-testid="save-chapter"]').trigger('click'); await flushPromises()
|
||||
const patch = fetchMock.mock.calls.find(([url, init]) => String(url).endsWith('/chapters/9') && (init as RequestInit | undefined)?.method === 'PATCH')!
|
||||
expect(JSON.parse(String((patch[1] as RequestInit).body))).toEqual({ title: 'Edited chapter', text: 'A new body.\n' })
|
||||
expect(view.get('[data-testid="book-notice"]').text()).toContain('已保存为新版本，正在重新处理')
|
||||
})
|
||||
|
||||
it('refuses an empty chapter body locally and keeps the dialog', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-chapter-9"]').trigger('click'); await flushPromises()
|
||||
await view.get('#chapter-text').setValue(' \n\t ')
|
||||
await view.get('[data-testid="save-chapter"]').trigger('click'); await flushPromises()
|
||||
expect(view.text()).toContain('请粘贴要导入的英文正文。')
|
||||
expect(fetchMock.mock.calls.some(([, init]) => (init as RequestInit | undefined)?.method === 'PATCH')).toBe(false)
|
||||
})
|
||||
|
||||
it('deletes a chapter after confirmation and reports the remaining count', async () => {
|
||||
const fetchMock = mockApi({ '/chapters/9': (init) => (init?.method === 'DELETE' ? ok({ deleted: { chapterId: 9, bookId: 1, remaining: 2 } }) : ok({ source })) })
|
||||
vi.spyOn(ElMessageBox, 'confirm').mockResolvedValue('confirm' as never)
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-chapter-9"]').trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="delete-chapter"]').trigger('click'); await flushPromises()
|
||||
expect(fetchMock.mock.calls.some(([url, init]) => String(url).endsWith('/chapters/9') && (init as RequestInit | undefined)?.method === 'DELETE')).toBe(true)
|
||||
expect(view.get('[data-testid="book-notice"]').text()).toContain('章节已删除 · 剩余 2 章')
|
||||
// Element Plus keeps a closed dialog in the DOM, so the closed state is what matters.
|
||||
expect(view.get('[data-testid="chapter-dialog"]').isVisible()).toBe(false)
|
||||
})
|
||||
|
||||
it('shows an empty chapter list with the import hint', async () => {
|
||||
mockApi({ '/books/1': () => ok({ book, chapters: [] }) })
|
||||
const { view } = await openBook()
|
||||
expect(view.get('[data-testid="empty-chapters"]').text()).toContain('这一本书还没有章节')
|
||||
expect(view.get('[data-testid="delete-book"]')).toBeTruthy()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,445 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import {
|
||||
NOT_FOUND_MESSAGE,
|
||||
POLL_INTERVAL_MS,
|
||||
TEXT_MAX_CODE_POINTS,
|
||||
canRetry,
|
||||
statusSummary,
|
||||
useLibraryStore,
|
||||
type ChapterDetail,
|
||||
} from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
// All accounts, books and texts in these tests are deliberately fictitious.
|
||||
const user = { id: 7, username: 'fictional-reader', role: 'learner' as const }
|
||||
const book = { id: 1, title: '虚构样例书', language: 'en' }
|
||||
const navigation = { previousChapterId: null, nextChapterId: null }
|
||||
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }), { status: 200 })
|
||||
const created = (data: unknown) => new Response(JSON.stringify({ code: 200, data }), { status: 201 })
|
||||
const httpError = (status: number, msg: string) => new Response(JSON.stringify({ code: status, msg }), { status })
|
||||
|
||||
const chapter = (overrides: Partial<ChapterDetail> = {}): ChapterDetail => ({
|
||||
id: 55,
|
||||
bookId: 1,
|
||||
ordinal: 1,
|
||||
title: '第一篇',
|
||||
status: 'pending',
|
||||
charCount: 120,
|
||||
errorReason: '',
|
||||
errorMessage: '',
|
||||
contentSha256: 'fictional-sha256',
|
||||
jobId: 7,
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
updatedAt: '2026-01-01T00:00:00Z',
|
||||
...overrides,
|
||||
})
|
||||
|
||||
const summary = (overrides: Partial<Record<string, number | string>> = {}) => ({
|
||||
...book,
|
||||
chapterCount: 0,
|
||||
pendingCount: 0,
|
||||
processingCount: 0,
|
||||
readyCount: 0,
|
||||
failedCount: 0,
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
updatedAt: '2026-01-01T00:00:00Z',
|
||||
...overrides,
|
||||
})
|
||||
|
||||
const job = { id: 7, bookId: 1, chapterId: 55, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
|
||||
const fetchMock = () => vi.mocked(globalThis.fetch)
|
||||
const paths = () => fetchMock().mock.calls.map(([input]) => String(input))
|
||||
const bodyOf = (index: number): Record<string, unknown> => JSON.parse(String(fetchMock().mock.calls[index]?.[1]?.body)) as Record<string, unknown>
|
||||
|
||||
async function signIn() {
|
||||
fetchMock().mockResolvedValueOnce(ok({ token: 'fictional-token', expiresAt: '2030-01-01', user }))
|
||||
const session = useSessionStore()
|
||||
await session.login(user.username, 'fictional-password')
|
||||
return session
|
||||
}
|
||||
|
||||
describe('learner library store', () => {
|
||||
beforeEach(() => {
|
||||
sessionStorage.clear()
|
||||
setActivePinia(createPinia())
|
||||
vi.restoreAllMocks()
|
||||
// Any request a test did not expect fails loudly instead of hanging.
|
||||
vi.spyOn(globalThis, 'fetch').mockImplementation(input => {
|
||||
throw new Error(`unexpected request: ${String(input)}`)
|
||||
})
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
// Never let a polling timer outlive its test.
|
||||
useLibraryStore().stopPolling()
|
||||
vi.useRealTimers()
|
||||
})
|
||||
|
||||
it('loads the library and summarises the count fields the API reports', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ items: [summary({ chapterCount: 5, readyCount: 2, processingCount: 1, pendingCount: 1, failedCount: 1 })] }))
|
||||
|
||||
await library.loadBooks()
|
||||
|
||||
expect(library.books).toHaveLength(1)
|
||||
expect(library.booksLoading).toBe(false)
|
||||
expect(library.booksError).toBe('')
|
||||
expect(statusSummary(library.books[0]!)).toBe('已就绪 2 · 处理中 1 · 待处理 1 · 失败 1')
|
||||
expect(paths()).toEqual(['/api/v1/login', '/api/v1/books'])
|
||||
expect(fetchMock().mock.calls[1]?.[1]?.headers).toMatchObject({ Authorization: 'Bearer fictional-token' })
|
||||
})
|
||||
|
||||
it('shows pending and processing separately instead of deriving one from a total', async () => {
|
||||
// processingCount is strictly "processing" now, so pendingCount must be read as given.
|
||||
const queued = summary({ chapterCount: 3, readyCount: 1, processingCount: 0, pendingCount: 2, failedCount: 0 })
|
||||
expect(statusSummary(queued)).toBe('已就绪 1 · 待处理 2')
|
||||
expect(statusSummary(queued)).not.toContain('处理中')
|
||||
const done = summary({ chapterCount: 1, readyCount: 1 })
|
||||
expect(statusSummary(done)).toBe('已就绪 1')
|
||||
expect(statusSummary(summary({ chapterCount: 0 }))).toBe('')
|
||||
})
|
||||
|
||||
it('loads a book detail whose chapters carry the job id used for retry', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapters: [chapter({ status: 'failed', jobId: 7 }), chapter({ id: 56, status: 'failed', jobId: null })] }))
|
||||
|
||||
await library.loadBook(1)
|
||||
|
||||
expect(library.book?.title).toBe('虚构样例书')
|
||||
expect(library.chapters).toHaveLength(2)
|
||||
expect(paths()[1]).toBe('/api/v1/books/1')
|
||||
// The chapter itself carries the job id, even for a freshly loaded book.
|
||||
expect(library.chapters[0]?.jobId).toBe(7)
|
||||
expect(canRetry(library.chapters[0]!)).toBe(true)
|
||||
// A null job id means the chapter has nothing to retry yet.
|
||||
expect(canRetry(library.chapters[1]!)).toBe(false)
|
||||
})
|
||||
|
||||
it('retries a failed chapter loaded fresh from the book detail, without any submit in this session', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
// No submit() call: this is a plain reload, the old workaround would hide retry here.
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapters: [chapter({ status: 'failed', errorMessage: '无法解析正文。', jobId: 7 })] }))
|
||||
await library.loadBook(1)
|
||||
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(ok({ job: { ...job, status: 'pending', attempts: 1 }, chapter: { id: 55, bookId: 1, jobId: 7 } }))
|
||||
.mockResolvedValueOnce(ok({ book, chapters: [chapter({ jobId: 7 })] }))
|
||||
await library.retryChapter(55)
|
||||
|
||||
expect(paths()).toContain('/api/v1/jobs/7/retry')
|
||||
expect(library.retryingChapterId).toBeNull()
|
||||
expect(library.chapters[0]?.status).toBe('pending')
|
||||
})
|
||||
|
||||
it('pastes a new book and reads the job id from the created chapter', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(created({ book, chapter: chapter(), job, duplicate: false }))
|
||||
|
||||
const bookId = await library.submit({ title: ' 第一篇 ', text: 'Hello world.\nSecond line.', target: { mode: 'new' } })
|
||||
|
||||
expect(bookId).toBe(1)
|
||||
expect(library.submitting).toBe(false)
|
||||
expect(library.submitError).toBe('')
|
||||
expect(paths()[1]).toBe('/api/v1/books')
|
||||
expect(bodyOf(1)).toEqual({ requestId: expect.any(String), title: '第一篇', text: 'Hello world.\nSecond line.', language: 'en' })
|
||||
})
|
||||
|
||||
it('appends to an existing book through the chapter endpoint', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(created({ chapter: chapter(), job, duplicate: false }))
|
||||
|
||||
const bookId = await library.submit({ title: '第二篇', text: 'Another text.', target: { mode: 'append', bookId: 1 } })
|
||||
|
||||
expect(bookId).toBe(1)
|
||||
expect(paths()[1]).toBe('/api/v1/books/1/chapters')
|
||||
})
|
||||
|
||||
it('reuses one requestId while the same unsent content keeps failing', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
const input = { title: '第一篇', text: 'Hello world.', target: { mode: 'new' as const } }
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(httpError(500, '服务器开小差了'))
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter(), job, duplicate: false }))
|
||||
|
||||
await expect(library.submit(input)).rejects.toThrow('服务器开小差了')
|
||||
expect(library.submitError).toBe('服务器开小差了')
|
||||
await library.submit(input)
|
||||
|
||||
// One chapter, not two: the retry of unchanged content reuses the requestId.
|
||||
expect(bodyOf(2).requestId).toBe(bodyOf(1).requestId)
|
||||
})
|
||||
|
||||
it('uses a fresh requestId after a successful submit and after the content changes', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter(), job, duplicate: false }))
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter({ id: 56 }), job, duplicate: false }))
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter({ id: 57 }), job, duplicate: false }))
|
||||
|
||||
await library.submit({ title: '第一篇', text: 'Hello world.', target: { mode: 'new' } })
|
||||
await library.submit({ title: '第一篇', text: 'Hello world.', target: { mode: 'new' } })
|
||||
await library.submit({ title: '第一篇', text: 'Hello world changed.', target: { mode: 'new' } })
|
||||
|
||||
expect(bodyOf(2).requestId).not.toBe(bodyOf(1).requestId)
|
||||
expect(bodyOf(3).requestId).not.toBe(bodyOf(2).requestId)
|
||||
})
|
||||
|
||||
it('rejects invalid input before sending anything', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
const sent = paths().length
|
||||
|
||||
await expect(library.submit({ title: ' ', text: 'Hello.', target: { mode: 'new' } })).rejects.toThrow('请填写标题。')
|
||||
await expect(library.submit({ title: '标题', text: ' \n\t ', target: { mode: 'new' } })).rejects.toThrow('请粘贴要导入的英文正文。')
|
||||
await expect(library.submit({ title: 'x'.repeat(121), text: 'Hello.', target: { mode: 'new' } })).rejects.toThrow('标题不能超过 120 个字符。')
|
||||
await expect(library.submit({ title: '标题', text: 'a'.repeat(TEXT_MAX_CODE_POINTS + 1), target: { mode: 'new' } })).rejects.toThrow(`正文不能超过 ${TEXT_MAX_CODE_POINTS} 个字符。`)
|
||||
|
||||
expect(paths().length).toBe(sent)
|
||||
expect(library.submitError).toBe(`正文不能超过 ${TEXT_MAX_CODE_POINTS} 个字符。`)
|
||||
})
|
||||
|
||||
it('measures the text limit in Unicode code points', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
// 100000 astral characters are 200000 UTF-16 units but still within the limit.
|
||||
fetchMock().mockResolvedValueOnce(created({ book, chapter: chapter(), job, duplicate: false }))
|
||||
|
||||
await expect(library.submit({ title: '标题', text: '😀'.repeat(TEXT_MAX_CODE_POINTS), target: { mode: 'new' } })).resolves.toBe(1)
|
||||
expect([...('😀'.repeat(TEXT_MAX_CODE_POINTS))].length).toBe(TEXT_MAX_CODE_POINTS)
|
||||
})
|
||||
|
||||
it('polls a pending chapter until it is ready and then stops', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter(), navigation }))
|
||||
|
||||
await library.loadChapter(55)
|
||||
expect(library.chapter?.status).toBe('pending')
|
||||
expect(library.readerText).toBe('')
|
||||
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Hello\nworld.' }), navigation }))
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS)
|
||||
|
||||
expect(library.chapter?.status).toBe('ready')
|
||||
expect(library.readerText).toBe('Hello\nworld.')
|
||||
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 3)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
it('retries a failed chapter through the chapter job id and resumes polling', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'failed', errorReason: 'decode_error', errorMessage: '无法解析正文。', jobId: 7 }), navigation }))
|
||||
|
||||
await library.loadChapter(55)
|
||||
expect(library.chapter?.status).toBe('failed')
|
||||
expect(canRetry(library.chapter!)).toBe(true)
|
||||
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(ok({ job: { ...job, status: 'pending', attempts: 1 }, chapter: { id: 55, bookId: 1, jobId: 7 } }))
|
||||
.mockResolvedValueOnce(ok({ book, chapter: chapter(), navigation }))
|
||||
await library.retryChapter(55)
|
||||
|
||||
expect(paths()).toContain('/api/v1/jobs/7/retry')
|
||||
expect(library.retryingChapterId).toBeNull()
|
||||
expect(library.chapter?.status).toBe('pending')
|
||||
|
||||
// The retry restarts polling for the chapter it re-queued.
|
||||
const before = paths().length
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Hello world.' }), navigation }))
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS)
|
||||
expect(paths().length).toBe(before + 1)
|
||||
expect(library.readerText).toBe('Hello world.')
|
||||
})
|
||||
|
||||
it('refuses to retry a chapter whose job id is null', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'failed', jobId: null }), navigation }))
|
||||
await library.loadChapter(55)
|
||||
|
||||
expect(canRetry(library.chapter!)).toBe(false)
|
||||
await expect(library.retryChapter(55)).rejects.toThrow('这一章暂时没有可重试的任务编号。')
|
||||
expect(paths()).toEqual(['/api/v1/login', '/api/v1/chapters/55'])
|
||||
})
|
||||
|
||||
it('reports another account id as 内容不存在 and stops polling for it', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(httpError(404, 'chapter not found'))
|
||||
|
||||
await library.loadChapter(99)
|
||||
expect(library.chapter).toBeNull()
|
||||
expect(library.chapterError).toBe(NOT_FOUND_MESSAGE)
|
||||
|
||||
fetchMock().mockResolvedValueOnce(httpError(404, 'book not found'))
|
||||
await library.loadBook(99)
|
||||
expect(library.book).toBeNull()
|
||||
expect(library.bookError).toBe(NOT_FOUND_MESSAGE)
|
||||
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 4)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
it('stops polling and ignores a late response once the session is cleared', async () => {
|
||||
vi.useFakeTimers()
|
||||
const session = await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter(), navigation }))
|
||||
await library.loadChapter(55)
|
||||
expect(library.chapter).not.toBeNull()
|
||||
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock().mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
fetchMock().mockResolvedValueOnce(ok(null))
|
||||
const late = library.loadChapter(55, { silent: true })
|
||||
const logout = session.logout()
|
||||
|
||||
finish(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Late text.' }), navigation }))
|
||||
await late
|
||||
await logout
|
||||
|
||||
expect(session.user).toBeNull()
|
||||
expect(library.chapter).toBeNull()
|
||||
expect(library.readerText).toBe('')
|
||||
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 4)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
it('keeps newer state when an older request answers later', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock().mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
fetchMock().mockResolvedValueOnce(ok({ items: [summary({ title: '较新的标题' })] }))
|
||||
|
||||
const stale = library.loadBooks()
|
||||
await library.loadBooks()
|
||||
finish(ok({ items: [summary({ title: '过期的标题' })] }))
|
||||
await stale
|
||||
|
||||
expect(library.books).toHaveLength(1)
|
||||
expect(library.books[0]?.title).toBe('较新的标题')
|
||||
})
|
||||
|
||||
// Regression R1: the append contract has no language field and the server rejects unknown
|
||||
// fields, so a client that sent one could never append.
|
||||
it('sends the language only when creating a book, never when appending', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(created({ book, chapter: chapter({ id: 54 }) }))
|
||||
await library.submit({ title: ' 新书 ', text: 'New book text.', target: { mode: 'new' } })
|
||||
expect(bodyOf(1)).toEqual({ requestId: expect.any(String), title: '新书', text: 'New book text.', language: 'en' })
|
||||
|
||||
fetchMock().mockResolvedValueOnce(created({ chapter: chapter({ id: 55, ordinal: 2 }) }))
|
||||
await library.submit({ title: '第二篇', text: 'Appended text.', target: { mode: 'append', bookId: 1 } })
|
||||
expect(String(fetchMock().mock.calls[2]?.[0])).toBe('/api/v1/books/1/chapters')
|
||||
expect(bodyOf(2)).toEqual({ requestId: expect.any(String), title: '第二篇', text: 'Appended text.' })
|
||||
expect(bodyOf(2)).not.toHaveProperty('language')
|
||||
})
|
||||
|
||||
// Regression R3: leaving a view must invalidate its in-flight request.
|
||||
it('ignores a book response that arrives after the book view was closed', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock().mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
|
||||
const pending = library.loadBook(1)
|
||||
library.closeBook()
|
||||
finish(ok({ book, chapters: [chapter()] }))
|
||||
await pending
|
||||
|
||||
expect(library.book).toBeNull()
|
||||
expect(library.chapters).toEqual([])
|
||||
expect(library.bookLoading).toBe(false)
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 3)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
it('ignores a chapter response that arrives after the reader was closed', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock().mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
|
||||
const pending = library.loadChapter(55)
|
||||
library.closeChapter()
|
||||
finish(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Late text.' }), navigation }))
|
||||
await pending
|
||||
|
||||
expect(library.chapter).toBeNull()
|
||||
expect(library.readerText).toBe('')
|
||||
expect(library.chapterLoading).toBe(false)
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 3)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
// Regression R4: an accepted retry must be visible and tracked even if the refresh fails.
|
||||
it('keeps tracking a retried chapter when the first refresh fails', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'failed', errorReason: 'content_changed', errorMessage: '内容在处理前发生变化。', jobId: 7 }), navigation }))
|
||||
await library.loadChapter(55)
|
||||
expect(library.chapter?.status).toBe('failed')
|
||||
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(ok({ job: { ...job, status: 'pending' }, chapter: chapter({ status: 'pending' }) }))
|
||||
.mockRejectedValueOnce(new Error('network down'))
|
||||
await library.retryChapter(55)
|
||||
|
||||
expect(paths()).toContain('/api/v1/jobs/7/retry')
|
||||
expect(library.chapter?.status).toBe('pending')
|
||||
expect(library.readerText).toBe('')
|
||||
|
||||
// The next poll still tracks the queued chapter and shows the final result.
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Recovered text.' }), navigation }))
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS)
|
||||
expect(library.chapter?.status).toBe('ready')
|
||||
expect(library.readerText).toBe('Recovered text.')
|
||||
})
|
||||
|
||||
it('keeps tracking a retried chapter from the book page when the first refresh fails', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapters: [chapter({ status: 'failed', errorReason: 'content_changed', errorMessage: '内容在处理前发生变化。' })] }))
|
||||
await library.loadBook(1)
|
||||
expect(library.chapters[0]?.status).toBe('failed')
|
||||
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(ok({ job: { ...job, status: 'pending' }, chapter: chapter({ status: 'pending' }) }))
|
||||
.mockRejectedValueOnce(new Error('network down'))
|
||||
await library.retryChapter(55)
|
||||
|
||||
expect(library.chapters[0]?.status).toBe('pending')
|
||||
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapters: [chapter({ status: 'ready' })] }))
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS)
|
||||
expect(library.chapters[0]?.status).toBe('ready')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,252 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createMemoryHistory, createRouter } from 'vue-router'
|
||||
import ReaderView from '../views/ReaderView.vue'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
import { useReaderLookup, type SavedTerm } from '../composables/useReaderLookup'
|
||||
import { effectScope, ref } from 'vue'
|
||||
import type { ChapterDetail } from '../stores/library'
|
||||
|
||||
// Fictitious text includes astral, combining, CRLF and repeated whitespace.
|
||||
const original = '😀 Cats\r\n café!'
|
||||
const fragments = [['😀', 'punctuation'], [' ', 'space'], ['Cats', 'word'], ['\r\n ', 'space'], ['café', 'word'], ['!', 'punctuation']]
|
||||
let cp = 0, utf16 = 0
|
||||
const tokens = fragments.map(([text = '', kind]) => {
|
||||
const token = { text, kind, start: cp, end: cp + [...text].length, startUtf16: utf16, endUtf16: utf16 + text.length }
|
||||
cp = token.end; utf16 = token.endUtf16
|
||||
return token
|
||||
})
|
||||
const chapter = { id: 55, bookId: 1, title: '虚构章节', status: 'ready', originalText: original, contentSha256: 'same-sha' }
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
const result = (query: string, status = 'exact') => ({ status, query, matchedForm: query, candidates: [], entries: status === 'exact' ? [{ lemma: query, pos: 'noun', definition: `Definition of ${query}`, examples: ['A fictional example.'] }] : [] })
|
||||
const savedTerm = (overrides: Partial<SavedTerm> = {}): SavedTerm => ({
|
||||
id: 7, term: 'cats', originalForm: 'Cats', definition: '猫', examples: ['A fictional example.'], status: 'learning', level: 2, ...overrides,
|
||||
})
|
||||
let wrapper: VueWrapper | undefined
|
||||
interface OpenOptions {
|
||||
lookup?: (body: Record<string, number>) => Promise<Response>
|
||||
tokenData?: unknown
|
||||
termRead?: () => Promise<Response>
|
||||
termWrite?: (body: Record<string, unknown>) => Promise<Response>
|
||||
}
|
||||
async function open(options: OpenOptions = {}) {
|
||||
const { lookup = async () => ok(result('cat')), tokenData = { textSha256: 'same-sha', tokens } } = options
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
const url = String(input)
|
||||
if (url.endsWith('/tokens')) return ok(tokenData)
|
||||
if (url.endsWith('/lookup')) return lookup(JSON.parse(String(init?.body)))
|
||||
if (url.includes('/terms/')) return options.termRead ? options.termRead() : ok({ term: savedTerm() })
|
||||
if (url.endsWith('/terms')) return options.termWrite ? options.termWrite(JSON.parse(String(init?.body))) : ok({ term: savedTerm(), created: true })
|
||||
return ok({ chapter, navigation: { previousChapterId: null, nextChapterId: 56 } })
|
||||
})
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
const router = createRouter({ history: createMemoryHistory(), routes: [{ path: '/chapters/:id', component: ReaderView }, { path: '/', component: { template: '<div />' } }] })
|
||||
await router.push('/chapters/55')
|
||||
wrapper = mount(ReaderView, { attachTo: document.body, global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
return { fetchMock, router, view: wrapper }
|
||||
}
|
||||
const callsTo = (fetchMock: MockInstance, suffix: string) => fetchMock.mock.calls.filter(call => String(call[0]).endsWith(suffix))
|
||||
// Personal state is attached to the word itself, never to a space or punctuation token.
|
||||
const withTerm = (text: string, term: { id: number; status: string; level: number }) => ({
|
||||
textSha256: 'same-sha',
|
||||
tokens: tokens.map(token => token.text === text ? { ...token, term } : token),
|
||||
})
|
||||
describe('reader word lookup', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; vi.restoreAllMocks() })
|
||||
it('keeps Unicode original text exact and sends only chapter and code-point offsets', async () => {
|
||||
const { view, fetchMock } = await open()
|
||||
expect(view.get('.reader-text').element.textContent).toBe(original)
|
||||
const words = view.findAll('.reader-word')
|
||||
expect(words).toHaveLength(2)
|
||||
await words[0]!.trigger('click'); await flushPromises()
|
||||
const call = callsTo(fetchMock, '/lookup')[0]!
|
||||
expect(call[1]?.method).toBe('POST')
|
||||
expect(JSON.parse(String(call[1]?.body))).toEqual({ chapterId: 55, start: 2, end: 6 })
|
||||
expect(view.text()).toContain('Definition of cat')
|
||||
expect(view.text()).toContain('A fictional example.')
|
||||
})
|
||||
it('discards older word responses and clears the selection form', async () => {
|
||||
let finish!: (response: Response) => void
|
||||
const { view } = await open({ lookup: body => body.start === 2 ? new Promise(resolve => { finish = resolve }) : Promise.resolve(ok(result('cafe', 'not_found'))) })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click')
|
||||
expect(view.text()).toContain('正在查询')
|
||||
await view.findAll('.reader-word')[1]!.trigger('click'); await flushPromises()
|
||||
await view.get('#term-definition').setValue('虚构私人草稿')
|
||||
finish(ok(result('OLD'))); await flushPromises()
|
||||
expect(view.text()).not.toContain('Definition of OLD')
|
||||
expect(view.text()).toContain('未找到释义')
|
||||
expect(view.text()).toContain('新词条')
|
||||
await view.findAll('.reader-word')[0]!.trigger('click')
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('')
|
||||
})
|
||||
it.each(['resource_missing', 'not_found', 'error'])('keeps reading and permits a saved word and retry for %s', async status => {
|
||||
const { view, fetchMock } = await open({ lookup: async () => { if (status === 'error') throw new Error('暂时无法查询'); return ok(result('cat', status)) } })
|
||||
await view.get('.reader-word').trigger('click'); await flushPromises()
|
||||
await view.get('#term-definition').setValue('仅当前词的虚构释义')
|
||||
expect(view.get('.reader-text').element.textContent).toBe(original)
|
||||
expect(view.get('[data-testid="term-save"]').attributes('disabled')).toBeUndefined()
|
||||
expect(callsTo(fetchMock, '/lookup')).toHaveLength(1)
|
||||
await view.get('[data-testid="lookup-retry"]').trigger('click'); await flushPromises()
|
||||
expect(callsTo(fetchMock, '/lookup')).toHaveLength(2)
|
||||
})
|
||||
it.each([{ textSha256: 'wrong', tokens }, { textSha256: 'same-sha', tokens: tokens.slice(1) }])('falls back to original when tokens do not match', async data => {
|
||||
const { view } = await open({ tokenData: data })
|
||||
expect(view.get('.reader-text').element.textContent).toBe(original)
|
||||
expect(view.find('.reader-word').exists()).toBe(false)
|
||||
expect(view.find('[data-testid="tokens-retry"]').exists()).toBe(true)
|
||||
})
|
||||
it('closes with Escape and restores the word focus without scrolling', async () => {
|
||||
const { view } = await open()
|
||||
const word = view.get('.reader-word').element as HTMLElement
|
||||
word.focus()
|
||||
await view.get('.reader-word').trigger('keydown', { key: 'Enter' }); await flushPromises()
|
||||
const focus = vi.spyOn(word, 'focus')
|
||||
await view.get('.lookup-panel').trigger('keydown', { key: 'Escape' })
|
||||
await flushPromises()
|
||||
expect(view.find('.lookup-panel').exists()).toBe(false)
|
||||
expect(document.activeElement).toBe(word)
|
||||
expect(focus).toHaveBeenCalledWith({ preventScroll: true })
|
||||
})
|
||||
it('retries a failed token fetch without losing original text', async () => {
|
||||
const { view, fetchMock } = await open()
|
||||
fetchMock.mockRejectedValueOnce(new Error('网络暂不可用'))
|
||||
// A new chapter load invalidates the old token rendering.
|
||||
const { useLibraryStore } = await import('../stores/library')
|
||||
useLibraryStore().chapter = { ...useLibraryStore().chapter!, contentSha256: 'retry-sha' }
|
||||
await flushPromises()
|
||||
expect(view.get('.reader-text').element.textContent).toBe(original)
|
||||
expect(view.find('.reader-word').exists()).toBe(false)
|
||||
fetchMock.mockResolvedValueOnce(ok({ textSha256: 'retry-sha', tokens }))
|
||||
await view.get('[data-testid="tokens-retry"]').trigger('click'); await flushPromises()
|
||||
expect(view.findAll('.reader-word')).toHaveLength(2)
|
||||
})
|
||||
it('does not accept a late token response after identity changes', async () => {
|
||||
const { view, fetchMock } = await open({ tokenData: { textSha256: 'wrong', tokens } })
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock.mockImplementationOnce(() => new Promise(resolve => { finish = resolve }))
|
||||
await view.get('[data-testid="tokens-retry"]').trigger('click')
|
||||
await useSessionStore().logout()
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
finish(ok({ textSha256: 'same-sha', tokens })); await flushPromises()
|
||||
expect(view.find('.reader-word').exists()).toBe(false)
|
||||
})
|
||||
it('moves a covered selected word above the sheet and restores the prior scroll on close', async () => {
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok(result('cat')))
|
||||
const scrollBy = vi.spyOn(window, 'scrollBy').mockImplementation(() => {})
|
||||
const scrollTo = vi.spyOn(window, 'scrollTo').mockImplementation(() => {})
|
||||
const scope = effectScope()
|
||||
const lookup = scope.run(() => useReaderLookup(ref(chapter as ChapterDetail)))!
|
||||
const element = document.createElement('span')
|
||||
vi.spyOn(element, 'getBoundingClientRect').mockReturnValue({ bottom: 700 } as DOMRect)
|
||||
lookup.select(tokens[2] as never, element)
|
||||
lookup.keepSelectionVisible(500)
|
||||
expect(scrollBy).toHaveBeenCalledWith({ top: 216, behavior: 'instant' })
|
||||
lookup.close()
|
||||
expect(scrollTo).toHaveBeenCalledWith({ top: 0, left: 0, behavior: 'instant' })
|
||||
scope.stop()
|
||||
})
|
||||
it('preserves a later manual reading scroll when closing the sheet', async () => {
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok(result('cat')))
|
||||
let scrollY = 100
|
||||
vi.spyOn(window, 'scrollY', 'get').mockImplementation(() => scrollY)
|
||||
vi.spyOn(window, 'scrollBy').mockImplementation((...args: unknown[]) => {
|
||||
const options = args[0] as ScrollToOptions
|
||||
scrollY += options.top ?? 0
|
||||
})
|
||||
const scrollTo = vi.spyOn(window, 'scrollTo').mockImplementation(() => {})
|
||||
const scope = effectScope()
|
||||
const lookup = scope.run(() => useReaderLookup(ref(chapter as ChapterDetail)))!
|
||||
const element = document.createElement('span')
|
||||
vi.spyOn(element, 'getBoundingClientRect').mockReturnValue({ bottom: 700 } as DOMRect)
|
||||
lookup.select(tokens[2] as never, element)
|
||||
lookup.keepSelectionVisible(500)
|
||||
scrollY += 300 // The reader continues down the page while the sheet is open.
|
||||
lookup.close()
|
||||
expect(scrollTo).not.toHaveBeenCalled()
|
||||
expect(scrollY).toBe(616)
|
||||
scope.stop()
|
||||
})
|
||||
it.each(['logout', 'chapter', 'unmount'])('invalidates pending lookup on %s', async action => {
|
||||
let finish!: (response: Response) => void
|
||||
const { view, router } = await open({ lookup: () => new Promise(resolve => { finish = resolve }) })
|
||||
await view.get('.reader-word').trigger('click')
|
||||
if (action === 'logout') await useSessionStore().logout()
|
||||
else if (action === 'chapter') await router.push('/chapters/56')
|
||||
else { view.unmount(); wrapper = undefined }
|
||||
finish(ok(result('OLD'))); await flushPromises()
|
||||
if (action === 'unmount') expect(document.querySelector('.lookup-panel')).toBeNull()
|
||||
else expect(view.find('.lookup-panel').exists()).toBe(false)
|
||||
expect(view.text()).not.toContain('Definition of OLD')
|
||||
})
|
||||
})
|
||||
describe('personal word records', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; vi.restoreAllMocks() })
|
||||
it('saves the definition, examples and status, then marks the word in the text', async () => {
|
||||
let body: Record<string, unknown> | undefined
|
||||
const { view, fetchMock } = await open({ termWrite: async value => { body = value; return ok({ term: savedTerm({ status: 'learning', level: 2 }), created: true }) } })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
await view.get('#term-definition').setValue(' 猫 ')
|
||||
await view.get('#term-examples').setValue('A fictional example.\n\n Second line ')
|
||||
await view.findAll('input[type="radio"]')[1]!.setValue()
|
||||
await view.get('[data-testid="term-save"]').trigger('click'); await flushPromises()
|
||||
expect(body).toEqual({ chapterId: 55, start: 2, end: 6, definition: ' 猫 ', examples: ['A fictional example.', 'Second line'], status: 'learning' })
|
||||
expect(view.text()).toContain('已保存 · 学习中')
|
||||
expect(view.findAll('.reader-word')[0]!.classes()).toContain('is-learning')
|
||||
expect(callsTo(fetchMock, '/terms')).toHaveLength(1)
|
||||
})
|
||||
it('keeps the typed input and reports the failure when saving fails', async () => {
|
||||
const { view } = await open({ termWrite: async () => { throw new Error('保存暂时不可用') } })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
await view.get('#term-definition').setValue('虚构释义')
|
||||
await view.get('[data-testid="term-save"]').trigger('click'); await flushPromises()
|
||||
expect(view.text()).toContain('保存暂时不可用')
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('虚构释义')
|
||||
expect(view.find('.lookup-panel').exists()).toBe(true)
|
||||
expect(view.findAll('.reader-word')[0]!.classes()).not.toContain('is-new')
|
||||
})
|
||||
it('loads the stored record when a saved word is opened and marks it in the text', async () => {
|
||||
const tokenData = withTerm('Cats', { id: 7, status: 'known', level: 0 })
|
||||
const { view, fetchMock } = await open({ tokenData, termRead: async () => ok({ term: savedTerm({ status: 'known', level: 0 }) }) })
|
||||
expect(view.findAll('.reader-word')[0]!.classes()).toContain('is-known')
|
||||
expect(view.findAll('.reader-word')[0]!.attributes('aria-label')).toContain('已保存')
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
expect(callsTo(fetchMock, '/terms/7')).toHaveLength(1)
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('猫')
|
||||
expect((view.get('#term-examples').element as HTMLTextAreaElement).value).toBe('A fictional example.')
|
||||
expect((view.findAll('input[type="radio"]')[2]!.element as HTMLInputElement).checked).toBe(true)
|
||||
expect(view.text()).toContain('已保存')
|
||||
})
|
||||
it('blocks saving until a failed read of the stored record is retried', async () => {
|
||||
const tokenData = withTerm('Cats', { id: 7, status: 'learning', level: 1 })
|
||||
let attempts = 0
|
||||
const { view, fetchMock } = await open({ tokenData, termRead: async () => { attempts++; if (attempts === 1) throw new Error('已保存的内容暂时无法读取'); return ok({ term: savedTerm() }) } })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
expect(view.text()).toContain('已保存的内容暂时无法读取')
|
||||
expect(view.get('[data-testid="term-save"]').attributes('disabled')).toBeDefined()
|
||||
await view.get('.reader-word').trigger('click'); await flushPromises()
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('猫')
|
||||
expect(view.get('[data-testid="term-save"]').attributes('disabled')).toBeUndefined()
|
||||
expect(callsTo(fetchMock, '/terms')).toHaveLength(0)
|
||||
})
|
||||
it('drops the previous account words and highlights after an identity change', async () => {
|
||||
const { view, fetchMock } = await open({ termWrite: async () => ok({ term: savedTerm({ status: 'new', level: 0 }), created: true }) })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="term-save"]').trigger('click'); await flushPromises()
|
||||
expect(view.findAll('.reader-word')[0]!.classes()).toContain('is-new')
|
||||
fetchMock.mockResolvedValueOnce(ok({ textSha256: 'same-sha', tokens }))
|
||||
useSessionStore().user = { id: 43, username: 'other-fictional', role: 'learner' }
|
||||
await flushPromises()
|
||||
expect(view.find('.lookup-panel').exists()).toBe(false)
|
||||
// The previous account's words and chapter are gone: the reader falls back to
|
||||
// nothing until the new identity loads its own chapter.
|
||||
expect(view.find('.reader-word').exists()).toBe(false)
|
||||
expect(view.find('.reader-text').exists()).toBe(false)
|
||||
expect(view.text()).not.toContain('已保存')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,265 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { flushPromises, mount } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createMemoryHistory, createRouter, type Router } from 'vue-router'
|
||||
import { defineComponent, h } from 'vue'
|
||||
import BookView from '../views/BookView.vue'
|
||||
import ImportView from '../views/ImportView.vue'
|
||||
import ReaderView from '../views/ReaderView.vue'
|
||||
import { useLibraryStore, type ChapterStatus } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
// All accounts, books and texts in these tests are deliberately fictitious.
|
||||
const user = { id: 42, username: 'fictional-reader', role: 'learner' as const }
|
||||
const book = { id: 1, title: '虚构样例书', language: 'en' }
|
||||
const navigation = { previousChapterId: null, nextChapterId: null }
|
||||
const pasted = 'First line.\n\tIndented line.\nTwo spaces kept.\n\nLast line.\n'
|
||||
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }), { status: 200 })
|
||||
const created = (data: unknown) => new Response(JSON.stringify({ code: 200, data }), { status: 201 })
|
||||
|
||||
function chapter(status: ChapterStatus, extra: Record<string, unknown> = {}) {
|
||||
return {
|
||||
id: 55,
|
||||
bookId: 1,
|
||||
ordinal: 1,
|
||||
title: '第一篇',
|
||||
status,
|
||||
charCount: 120,
|
||||
errorReason: '',
|
||||
errorMessage: '',
|
||||
jobId: 7,
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
updatedAt: '2026-01-01T00:00:00Z',
|
||||
...extra,
|
||||
}
|
||||
}
|
||||
|
||||
const stub = (name: string) => defineComponent({ name, render: () => h('div') })
|
||||
|
||||
async function viewAt(path: string): Promise<Router> {
|
||||
const router = createRouter({
|
||||
history: createMemoryHistory(),
|
||||
routes: [
|
||||
{ path: '/', component: stub('LibraryStub') },
|
||||
{ path: '/import', component: stub('ImportStub') },
|
||||
{ path: '/books/:id', component: stub('BookStub') },
|
||||
{ path: '/chapters/:id', component: stub('ChapterStub') },
|
||||
],
|
||||
})
|
||||
await router.push(path)
|
||||
await router.isReady()
|
||||
return router
|
||||
}
|
||||
|
||||
function signIn() {
|
||||
useSessionStore().user = { ...user }
|
||||
}
|
||||
|
||||
describe('learner reading views', () => {
|
||||
beforeEach(() => {
|
||||
sessionStorage.clear()
|
||||
setActivePinia(createPinia())
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
useLibraryStore().stopPolling()
|
||||
})
|
||||
|
||||
it('shows inline validation and sends nothing for an empty import form', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
signIn()
|
||||
const router = await viewAt('/import')
|
||||
const wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
await wrapper.find('form').trigger('submit')
|
||||
expect(wrapper.text()).toContain('请填写标题。')
|
||||
expect(wrapper.text()).toContain('请粘贴要导入的英文正文。')
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
|
||||
await wrapper.find('input#title').setValue(' ')
|
||||
await wrapper.find('textarea#text').setValue(' \n\t ')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
expect(wrapper.text()).toContain('请填写标题。')
|
||||
expect(wrapper.text()).toContain('请粘贴要导入的英文正文。')
|
||||
|
||||
await wrapper.find('input#title').setValue('虚构样例第一章')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
expect(wrapper.text()).not.toContain('请填写标题。')
|
||||
expect(wrapper.text()).toContain('请粘贴要导入的英文正文。')
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('submits valid pasted text and opens the created book', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter('pending'), job: { id: 7 }, duplicate: false }))
|
||||
signIn()
|
||||
const router = await viewAt('/import')
|
||||
const wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
// Element Plus assigns the input ids on mount, so wait for the first update.
|
||||
await flushPromises()
|
||||
|
||||
await wrapper.find('input#title').setValue('虚构样例第一章')
|
||||
await wrapper.find('textarea#text').setValue(pasted)
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
expect(JSON.parse(String(fetchMock.mock.calls[0]?.[1]?.body))).toMatchObject({ title: '虚构样例第一章', text: pasted, language: 'en' })
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('renders a ready chapter verbatim, keeping line breaks, tabs and repeated spaces', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ book, chapter: chapter('ready', { originalText: pasted, contentSha256: 'fictional' }), navigation }))
|
||||
signIn()
|
||||
const router = await viewAt('/chapters/55')
|
||||
const wrapper = mount(ReaderView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
const article = wrapper.find('.reader-text')
|
||||
expect(article.exists()).toBe(true)
|
||||
expect(article.element.textContent).toBe(pasted)
|
||||
expect(wrapper.text()).toContain('已就绪')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('shows the failure message of a failed chapter and no text', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ book, chapter: chapter('failed', { errorReason: 'decode_error', errorMessage: '无法解析正文，请检查编码。', jobId: null }), navigation }))
|
||||
signIn()
|
||||
const router = await viewAt('/chapters/55')
|
||||
const wrapper = mount(ReaderView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
expect(wrapper.text()).toContain('处理失败')
|
||||
expect(wrapper.text()).toContain('无法解析正文，请检查编码。')
|
||||
expect(wrapper.find('.reader-text').exists()).toBe(false)
|
||||
// A null job id means there is nothing to retry yet.
|
||||
expect(wrapper.text()).toContain('这一章暂时没有可重试的任务编号。')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('offers retry on a failed chapter through its own job id', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ book, chapter: chapter('failed', { errorMessage: '解析失败。', jobId: 7 }), navigation }))
|
||||
signIn()
|
||||
const router = await viewAt('/chapters/55')
|
||||
const wrapper = mount(ReaderView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
expect(wrapper.text()).toContain('解析失败。')
|
||||
const retryButton = wrapper.get('.notice .el-button')
|
||||
expect(retryButton.text()).toContain('重试处理')
|
||||
await retryButton.trigger('click')
|
||||
await flushPromises()
|
||||
|
||||
// No prior submit in this session: the job id comes from the chapter payload.
|
||||
const retryCall = fetchMock.mock.calls.find(([input]) => String(input).endsWith('/jobs/7/retry'))
|
||||
expect(retryCall?.[1]?.method).toBe('POST')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('lists chapters with their status labels and links only ready chapters', async () => {
|
||||
const chapters = [
|
||||
chapter('pending', { id: 55, ordinal: 1, title: '第一篇', jobId: 7 }),
|
||||
chapter('processing', { id: 56, ordinal: 2, title: '第二篇', jobId: 8 }),
|
||||
chapter('ready', { id: 57, ordinal: 3, title: '第三篇', jobId: 9 }),
|
||||
chapter('failed', { id: 58, ordinal: 4, title: '第四篇', errorMessage: '解析失败。', jobId: null }),
|
||||
chapter('failed', { id: 59, ordinal: 5, title: '第五篇', errorMessage: '编码错误。', jobId: 10 }),
|
||||
]
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(input => {
|
||||
if (String(input).endsWith('/jobs/10/retry')) {
|
||||
return Promise.resolve(ok({ job: { id: 10, status: 'pending', attempts: 1 }, chapter: { id: 59, bookId: 1, jobId: 10 } }))
|
||||
}
|
||||
return Promise.resolve(ok({ book, chapters }))
|
||||
})
|
||||
signIn()
|
||||
const router = await viewAt('/books/1')
|
||||
const wrapper = mount(BookView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
const text = wrapper.text()
|
||||
expect(text).toContain('待处理')
|
||||
expect(text).toContain('处理中')
|
||||
expect(text).toContain('已就绪')
|
||||
expect(text).toContain('处理失败')
|
||||
expect(text).toContain('解析失败。')
|
||||
expect(text).toContain('编码错误。')
|
||||
expect(wrapper.find('a[href="/chapters/57"]').exists()).toBe(true)
|
||||
expect(wrapper.find('a[href="/chapters/55"]').exists()).toBe(false)
|
||||
|
||||
// Retry is offered only for the failed chapter that carries a job id.
|
||||
const retryButtons = wrapper.findAll('.chapter-row .el-button').filter(button => button.text().includes('重试'))
|
||||
expect(retryButtons).toHaveLength(1)
|
||||
await retryButtons[0]!.trigger('click')
|
||||
await flushPromises()
|
||||
const retryCall = fetchMock.mock.calls.find(([input]) => String(input).endsWith('/jobs/10/retry'))
|
||||
expect(retryCall?.[1]?.method).toBe('POST')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
// Regression R1: appending must not send the language field that the append contract rejects.
|
||||
it('appends to an existing book through the chapter endpoint without a language field', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
.mockResolvedValueOnce(ok({
|
||||
items: [{
|
||||
...book,
|
||||
chapterCount: 1,
|
||||
pendingCount: 0,
|
||||
processingCount: 0,
|
||||
readyCount: 1,
|
||||
failedCount: 0,
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
updatedAt: '2026-01-01T00:00:00Z',
|
||||
}],
|
||||
}))
|
||||
.mockResolvedValueOnce(created({ chapter: chapter('pending', { id: 56, ordinal: 2 }), job: { id: 8 } }))
|
||||
signIn()
|
||||
const router = await viewAt('/import?book=1')
|
||||
const wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
await wrapper.find('input#title').setValue('第二篇')
|
||||
await wrapper.find('textarea#text').setValue(pasted)
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
|
||||
const appendCall = fetchMock.mock.calls.find(([input]) => String(input).endsWith('/books/1/chapters'))
|
||||
expect(appendCall?.[1]?.method).toBe('POST')
|
||||
const body = JSON.parse(String(appendCall?.[1]?.body)) as Record<string, unknown>
|
||||
expect(body).toEqual({ requestId: expect.any(String), title: '第二篇', text: pasted })
|
||||
expect(body).not.toHaveProperty('language')
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
// Regression R3: a submit that finishes after the user left the page must not navigate back.
|
||||
it('does not navigate after the user left the import page during a submit', async () => {
|
||||
let finish!: (response: Response) => void
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
.mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
signIn()
|
||||
const router = await viewAt('/import')
|
||||
const wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
await wrapper.find('input#title').setValue('虚构样例第一章')
|
||||
await wrapper.find('textarea#text').setValue(pasted)
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
|
||||
// The user leaves the page while the request is still open.
|
||||
wrapper.unmount()
|
||||
await router.push('/')
|
||||
await flushPromises()
|
||||
|
||||
finish(created({ book, chapter: chapter('pending'), job: { id: 7 } }))
|
||||
await flushPromises()
|
||||
|
||||
expect(router.currentRoute.value.path).toBe('/')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,246 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createMemoryHistory, createRouter } from 'vue-router'
|
||||
import ReviewView from '../views/ReviewView.vue'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
import { clozeSentence, useReviewStore, type ReviewItem } from '../stores/review'
|
||||
|
||||
const item = (overrides: Partial<ReviewItem> = {}): ReviewItem => ({
|
||||
id: 7, term: 'dogs', originalForm: 'Dogs', definition: '狗', examples: ['Dogs went home.'],
|
||||
status: 'new', level: 0, dueAt: '2026-09-11T10:00:00Z', reviewCount: 0, ...overrides,
|
||||
})
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
const answer = (overrides: Record<string, unknown> = {}) => ({
|
||||
result: 'applied', grade: 'correct', requeued: false, statusBefore: 'new', statusAfter: 'learning',
|
||||
levelBefore: 0, levelAfter: 1, dueAtBefore: '2026-09-11T10:00:00Z', dueAtAfter: '2026-09-12T10:00:00Z',
|
||||
item: item({ status: 'learning', level: 1, dueAt: '2026-09-12T10:00:00Z', reviewCount: 1 }), ...overrides,
|
||||
})
|
||||
const answerCalls = (fetchMock: MockInstance) => fetchMock.mock.calls.filter(call => String(call[0]).includes('/answers'))
|
||||
let wrapper: VueWrapper | undefined
|
||||
|
||||
describe('review prompt', () => {
|
||||
it('masks the word in the first example and keeps other sentences usable', () => {
|
||||
expect(clozeSentence(item())).toBe('_____ went home.')
|
||||
expect(clozeSentence(item({ term: "isn't", originalForm: "Isn't", examples: ["It isn’t over."] }))).toBe('It _____ over.')
|
||||
expect(clozeSentence(item({ examples: ['No matching word here.'] }))).toBe('No matching word here.')
|
||||
expect(clozeSentence(item({ examples: [] }))).toBeNull()
|
||||
expect(clozeSentence(item({ term: 'dog', originalForm: 'Dog', examples: ['Dogs are not the saved word.'] }))).toBe('Dogs are not the saved word.')
|
||||
})
|
||||
})
|
||||
|
||||
describe('review store', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear(); useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' } })
|
||||
afterEach(() => { vi.restoreAllMocks() })
|
||||
|
||||
it('loads the due queue and answers a word correctly', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
if (String(input).endsWith('/reviews/queue')) return ok({ items: [item()], total: 1 })
|
||||
return ok(answer())
|
||||
})
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
expect(review.queue).toHaveLength(1)
|
||||
expect(review.current?.id).toBe(7)
|
||||
expect(review.empty).toBe(false)
|
||||
review.reveal()
|
||||
expect(review.revealed).toBe(true)
|
||||
await review.answer('correct')
|
||||
const body = JSON.parse(String(answerCalls(fetchMock)[0]![1]?.body))
|
||||
expect(body).toEqual({ answerId: expect.stringMatching(/^[0-9a-f-]{36}$/), grade: 'correct', expectedDueAt: '2026-09-11T10:00:00Z' })
|
||||
expect(review.queue).toHaveLength(0)
|
||||
expect(review.answered).toBe(1)
|
||||
expect(review.correctCount).toBe(1)
|
||||
expect(review.finished).toBe(true)
|
||||
expect(review.revealed).toBe(false)
|
||||
})
|
||||
|
||||
it('requeues a word that is not recognised and counts it as wrong', async () => {
|
||||
const wrong = answer({ grade: 'wrong', requeued: true, levelAfter: 0, statusAfter: 'new', dueAtAfter: '2026-09-11T10:05:00Z', item: item({ dueAt: '2026-09-11T10:05:00Z' }) })
|
||||
vi.spyOn(globalThis, 'fetch').mockImplementation(async input => String(input).endsWith('/reviews/queue') ? ok({ items: [item()], total: 1 }) : ok(wrong))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
await review.answer('wrong')
|
||||
expect(review.queue).toHaveLength(1)
|
||||
expect(review.queue[0]!.dueAt).toBe('2026-09-11T10:05:00Z')
|
||||
expect(review.wrongCount).toBe(1)
|
||||
expect(review.correctCount).toBe(0)
|
||||
expect(review.wordsReviewed).toBe(1)
|
||||
expect(review.finished).toBe(false)
|
||||
})
|
||||
|
||||
it('treats a stale answer as no new score but finishes the round instead of reporting an empty queue', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockImplementation(async input => String(input).endsWith('/reviews/queue') ? ok({ items: [item()], total: 1 }) : ok(answer({ result: 'stale', requeued: false })))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
await review.answer('correct')
|
||||
expect(review.queue).toHaveLength(0)
|
||||
expect(review.answered).toBe(0)
|
||||
expect(review.correctCount).toBe(0)
|
||||
expect(review.resolved).toBe(1)
|
||||
expect(review.empty).toBe(false)
|
||||
expect(review.finished).toBe(true)
|
||||
expect(review.notice).toContain('已在其他页面复习')
|
||||
})
|
||||
|
||||
it('counts a replay of this client own answer after a lost response', async () => {
|
||||
let sent = 0
|
||||
vi.spyOn(globalThis, 'fetch').mockImplementation(async input => {
|
||||
if (String(input).endsWith('/reviews/queue')) return ok({ items: [item()], total: 1 })
|
||||
sent += 1
|
||||
// The first response never reaches the client, the retry reports the recorded answer.
|
||||
if (sent === 1) throw new Error('网络中断')
|
||||
return ok(answer({ result: 'applied', duplicate: true }))
|
||||
})
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
await review.answer('correct')
|
||||
expect(review.error).toContain('网络中断')
|
||||
expect(review.answered).toBe(0)
|
||||
await review.answer('correct')
|
||||
expect(review.answered).toBe(1)
|
||||
expect(review.correctCount).toBe(1)
|
||||
expect(review.wordsReviewed).toBe(1)
|
||||
expect(review.empty).toBe(false)
|
||||
expect(review.finished).toBe(true)
|
||||
expect(review.notice).toBe('')
|
||||
})
|
||||
|
||||
it('keeps the card and the same answer id when a submission fails, then retries once', async () => {
|
||||
let fail = true
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async input => {
|
||||
if (String(input).endsWith('/reviews/queue')) return ok({ items: [item()], total: 1 })
|
||||
if (fail) { fail = false; throw new Error('评分暂时无法提交') }
|
||||
return ok(answer())
|
||||
})
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
await review.answer('correct')
|
||||
expect(review.error).toContain('评分暂时无法提交')
|
||||
expect(review.queue).toHaveLength(1)
|
||||
expect(review.answered).toBe(0)
|
||||
await review.answer('correct')
|
||||
const calls = answerCalls(fetchMock)
|
||||
const ids = calls.map(call => JSON.parse(String(call[1]?.body)).answerId)
|
||||
expect(ids).toHaveLength(2)
|
||||
expect(ids[0]).toBe(ids[1])
|
||||
expect(review.queue).toHaveLength(0)
|
||||
expect(review.answered).toBe(1)
|
||||
})
|
||||
|
||||
it('reports a failed queue load without pretending the queue is empty', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockRejectedValue(new Error('网络暂不可用'))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
expect(review.error).toContain('网络暂不可用')
|
||||
expect(review.finished).toBe(false)
|
||||
expect(review.empty).toBe(false)
|
||||
})
|
||||
|
||||
it('drops the previous account queue and counters when the identity changes', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ items: [item()], total: 1 }))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
expect(review.queue).toHaveLength(1)
|
||||
useSessionStore().user = { id: 43, username: 'other-fictional', role: 'learner' }
|
||||
await flushPromises()
|
||||
expect(review.queue).toHaveLength(0)
|
||||
expect(review.current).toBeNull()
|
||||
expect(review.finished).toBe(false)
|
||||
expect(review.empty).toBe(false)
|
||||
})
|
||||
|
||||
it('reports the words still due beyond the fetched page', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ items: [item()], total: 3 }))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
expect(review.pending).toBe(2)
|
||||
})
|
||||
})
|
||||
|
||||
async function open(queueResult: unknown, onAnswer: (body: Record<string, unknown>) => Response | Promise<Response> = () => ok(answer())) {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
if (String(input).endsWith('/reviews/queue')) {
|
||||
if (queueResult instanceof Error) throw queueResult
|
||||
return ok(queueResult)
|
||||
}
|
||||
return onAnswer(JSON.parse(String(init?.body)))
|
||||
})
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
const router = createRouter({ history: createMemoryHistory(), routes: [{ path: '/', component: { template: '<div />' } }, { path: '/review', component: ReviewView }] })
|
||||
await router.push('/review')
|
||||
wrapper = mount(ReviewView, { attachTo: document.body, global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
return { view: wrapper, router, fetchMock }
|
||||
}
|
||||
|
||||
describe('review page', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; vi.restoreAllMocks() })
|
||||
|
||||
it('shows the masked example, hides the definition until the answer is revealed, then grades it', async () => {
|
||||
const { view, fetchMock } = await open({ items: [item()], total: 2 })
|
||||
expect(view.get('[data-testid="review-position"]').text()).toBe('到期复习 · 1 / 1')
|
||||
expect(view.text()).toContain('Dogs')
|
||||
expect(view.text()).toContain('_____ went home.')
|
||||
expect(view.find('[data-testid="review-definition"]').exists()).toBe(false)
|
||||
expect(document.activeElement).toBe(view.get('[data-testid="review-reveal"]').element)
|
||||
await view.get('[data-testid="review-reveal"]').trigger('click'); await flushPromises()
|
||||
expect(view.get('[data-testid="review-definition"]').text()).toContain('狗')
|
||||
expect(document.activeElement).toBe(view.get('[data-testid="review-correct"]').element)
|
||||
await view.get('[data-testid="review-correct"]').trigger('click'); await flushPromises()
|
||||
expect(answerCalls(fetchMock)).toHaveLength(1)
|
||||
expect(view.get('[data-testid="review-summary"]').text()).toContain('1 个词条')
|
||||
})
|
||||
|
||||
it('keeps the card and offers a retry when a grade fails', async () => {
|
||||
const { view } = await open({ items: [item()], total: 1 }, () => { throw new Error('评分暂时无法提交，请重试。') })
|
||||
await view.get('[data-testid="review-reveal"]').trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="review-wrong"]').trigger('click'); await flushPromises()
|
||||
expect(view.find('[data-testid="review-card"]').exists()).toBe(true)
|
||||
expect(view.text()).toContain('评分暂时无法提交')
|
||||
expect(view.find('[data-testid="review-retry"]').exists()).toBe(true)
|
||||
expect(view.find('[data-testid="review-summary"]').exists()).toBe(false)
|
||||
})
|
||||
|
||||
it('reports an empty queue and a load failure without pretending anything was reviewed', async () => {
|
||||
const emptyRun = await open({ items: [], total: 0 })
|
||||
expect(emptyRun.view.get('[data-testid="review-empty"]').text()).toContain('今天没有到期词条')
|
||||
await emptyRun.view.get('[data-testid="review-finish"]').trigger('click'); await flushPromises()
|
||||
expect(emptyRun.router.currentRoute.value.path).toBe('/')
|
||||
wrapper?.unmount(); wrapper = undefined
|
||||
|
||||
// A failed load is not an empty queue: it shows the error and a retry.
|
||||
const failing = await open(new Error('复习队列暂时无法加载'))
|
||||
expect(failing.view.get('[data-testid="review-reload"]').text()).toContain('重试')
|
||||
expect(failing.view.text()).toContain('复习队列暂时无法加载')
|
||||
expect(failing.view.find('[data-testid="review-empty"]').exists()).toBe(false)
|
||||
vi.restoreAllMocks()
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ items: [item()], total: 1 }))
|
||||
await failing.view.get('[data-testid="review-reload"]').trigger('click'); await flushPromises()
|
||||
expect(failing.view.get('[data-testid="review-position"]').text()).toBe('到期复习 · 1 / 1')
|
||||
})
|
||||
|
||||
it('offers the remaining due words after a round finishes', async () => {
|
||||
const { view } = await open({ items: [item()], total: 3 })
|
||||
await view.get('[data-testid="review-reveal"]').trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="review-correct"]').trigger('click'); await flushPromises()
|
||||
expect(view.get('[data-testid="review-more"]').text()).toContain('继续复习')
|
||||
expect(view.text()).toContain('还有 2 个词条到期')
|
||||
})
|
||||
|
||||
it('tells the learner when a card was already reviewed elsewhere', async () => {
|
||||
const { view } = await open({ items: [item()], total: 1 }, () => ok(answer({ result: 'stale', duplicate: false })))
|
||||
await view.get('[data-testid="review-reveal"]').trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="review-correct"]').trigger('click'); await flushPromises()
|
||||
expect(view.get('[data-testid="review-notice"]').text()).toContain('已在其他页面复习')
|
||||
expect(view.get('[data-testid="review-summary"]').text()).toContain('本轮没有新的计分')
|
||||
})
|
||||
|
||||
it('ends the round without submitting anything', async () => {
|
||||
const { view, router, fetchMock } = await open({ items: [item()], total: 1 })
|
||||
await view.get('[data-testid="review-end"]').trigger('click'); await flushPromises()
|
||||
expect(router.currentRoute.value.path).toBe('/')
|
||||
expect(answerCalls(fetchMock)).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,223 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createRouter, createMemoryHistory, type Router } from 'vue-router'
|
||||
import ImportView from '../views/ImportView.vue'
|
||||
import { fileProblem, fileSizeLabel, TXT_MAX_BYTES, useLibraryStore } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const user = { id: 42, username: 'fictional-uploader', role: 'learner' as const }
|
||||
const book = { id: 1, title: 'Uploaded Book', language: 'en' }
|
||||
const chapter = { id: 9, bookId: 1, ordinal: 1, title: 'Uploaded Book', status: 'pending', charCount: 12, errorReason: '', errorMessage: '', jobId: 5, createdAt: '', updatedAt: '' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
let wrapper: VueWrapper | undefined
|
||||
|
||||
function stub(name: string) {
|
||||
return { template: `<div>${name}</div>` }
|
||||
}
|
||||
|
||||
async function viewAt(path: string): Promise<Router> {
|
||||
const router = createRouter({
|
||||
history: createMemoryHistory(),
|
||||
routes: [
|
||||
{ path: '/', component: stub('LibraryStub') },
|
||||
{ path: '/import', component: stub('ImportStub') },
|
||||
{ path: '/books/:id', component: stub('BookStub') },
|
||||
],
|
||||
})
|
||||
await router.push(path)
|
||||
await router.isReady()
|
||||
return router
|
||||
}
|
||||
|
||||
/** jsdom has no file picker, so the input's files are set directly before dispatching change. */
|
||||
async function chooseFile(view: VueWrapper, file: File | null): Promise<void> {
|
||||
const input = view.get('[data-testid="file-input"]')
|
||||
Object.defineProperty(input.element, 'files', { value: file ? [file] : [], configurable: true })
|
||||
await input.trigger('change')
|
||||
await flushPromises()
|
||||
}
|
||||
|
||||
/** The TXT toggle is an Element Plus radio group; its hidden input carries the value. */
|
||||
async function useTxt(view: VueWrapper): Promise<void> {
|
||||
for (const input of view.findAll('input[type="radio"]')) {
|
||||
if ((input.element as HTMLInputElement).value === 'txt') {
|
||||
await input.setValue()
|
||||
await flushPromises()
|
||||
return
|
||||
}
|
||||
}
|
||||
throw new Error('TXT 文件 toggle not found')
|
||||
}
|
||||
|
||||
function uploadResponse() {
|
||||
return ok({ book, chapter, job: { id: 5, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps }, duplicate: false })
|
||||
}
|
||||
|
||||
describe('txt upload pre-checks', () => {
|
||||
it('mirrors the server limits for the file that was picked', () => {
|
||||
expect(fileProblem({ name: 'reading.txt', size: 2048 })).toBe('')
|
||||
expect(fileProblem({ name: 'READING.TXT', size: 1 })).toBe('')
|
||||
expect(fileProblem({ name: 'reading.md', size: 2048 })).toContain('.txt')
|
||||
expect(fileProblem({ name: 'empty.txt', size: 0 })).toContain('空的')
|
||||
expect(fileProblem({ name: 'big.txt', size: TXT_MAX_BYTES + 1 })).toContain('2 MiB')
|
||||
expect(fileProblem({ name: 'limit.txt', size: TXT_MAX_BYTES })).toBe('')
|
||||
expect(fileSizeLabel(512)).toBe('512 B')
|
||||
expect(fileSizeLabel(2048)).toBe('2 KB')
|
||||
expect(fileSizeLabel(1.5 * 1024 * 1024)).toBe('1.5 MB')
|
||||
})
|
||||
})
|
||||
|
||||
describe('txt upload view', () => {
|
||||
beforeEach(() => {
|
||||
setActivePinia(createPinia())
|
||||
sessionStorage.clear()
|
||||
vi.restoreAllMocks()
|
||||
useSessionStore().user = { ...user }
|
||||
})
|
||||
afterEach(() => {
|
||||
wrapper?.unmount()
|
||||
wrapper = undefined
|
||||
useLibraryStore().stopPolling()
|
||||
})
|
||||
|
||||
it('shows the file picker instead of the textarea and reports the picked file', async () => {
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
// Paste mode keeps the textarea; TXT mode replaces it with the picker.
|
||||
expect(wrapper.find('textarea#text').exists()).toBe(true)
|
||||
expect(wrapper.find('[data-testid="file-input"]').exists()).toBe(false)
|
||||
await useTxt(wrapper)
|
||||
expect(wrapper.find('textarea#text').exists()).toBe(false)
|
||||
expect(wrapper.find('[data-testid="file-input"]').exists()).toBe(true)
|
||||
expect(wrapper.text()).toContain('仅支持 UTF-8')
|
||||
|
||||
await chooseFile(wrapper, new File(['Mira opened the workshop.\n'], 'reading.txt', { type: 'text/plain' }))
|
||||
expect(wrapper.get('[data-testid="file-info"]').text()).toBe('reading.txt · UTF-8 · 26 B')
|
||||
// The title is prefilled from the file name and stays editable.
|
||||
expect((wrapper.get('input#title').element as HTMLInputElement).value).toBe('reading')
|
||||
})
|
||||
|
||||
it('refuses a file the server would refuse, before anything is sent', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
|
||||
await chooseFile(wrapper, new File(['# not a txt file\n'], 'notes.md'))
|
||||
expect(wrapper.text()).toContain('请选择 .txt 文件。')
|
||||
// A file that is not valid UTF-8 is rejected by the preview decode.
|
||||
await chooseFile(wrapper, new File([new Uint8Array([0x63, 0x61, 0x66, 0xe9, 0x0a])], 'latin1.txt'))
|
||||
expect(wrapper.text()).toContain('文件不是 UTF-8 编码')
|
||||
await wrapper.find('input#title').setValue('Latin One')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
expect(wrapper.text()).toContain('文件不是 UTF-8 编码')
|
||||
|
||||
// No file at all is also refused locally.
|
||||
await chooseFile(wrapper, null)
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
expect(wrapper.text()).toContain('请选择要导入的 TXT 文件。')
|
||||
})
|
||||
|
||||
it('uploads the file as multipart and opens the created book', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockResolvedValueOnce(uploadResponse())
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
const file = new File(['Mira opened the workshop.\n'], 'reading.txt', { type: 'text/plain' })
|
||||
await chooseFile(wrapper, file)
|
||||
await wrapper.find('input#title').setValue('上传的虚构章节')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
const [url, init] = fetchMock.mock.calls[0]!
|
||||
expect(String(url)).toBe('/api/v1/books/upload')
|
||||
expect(init?.method).toBe('POST')
|
||||
// A multipart body must not be replaced by JSON and must keep the browser's boundary.
|
||||
expect(init?.body).toBeInstanceOf(FormData)
|
||||
expect((init?.headers as Record<string, string>)['Content-Type']).toBeUndefined()
|
||||
const body = init?.body as FormData
|
||||
expect(body.get('requestId')).toMatch(/^[0-9a-f-]{36}$/)
|
||||
expect(body.get('title')).toBe('上传的虚构章节')
|
||||
expect(body.get('language')).toBe('en')
|
||||
expect((body.get('file') as File).name).toBe('reading.txt')
|
||||
expect(await (body.get('file') as File).text()).toBe('Mira opened the workshop.\n')
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
// The form starts clean, so the same file is not submitted twice by accident.
|
||||
expect((wrapper.get('[data-testid="file-input"]').element as HTMLInputElement).value).toBe('')
|
||||
})
|
||||
|
||||
it('reuses one request id when the same upload is retried after a failure', async () => {
|
||||
let attempt = 0
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async () => {
|
||||
attempt += 1
|
||||
if (attempt === 1) throw new Error('上传中断')
|
||||
return uploadResponse()
|
||||
})
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
await chooseFile(wrapper, new File(['Body.\n'], 'retry.txt'))
|
||||
await wrapper.find('input#title').setValue('Retry Upload')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(wrapper.text()).toContain('上传中断')
|
||||
// The picked file stays selected so the learner can retry without choosing it again.
|
||||
expect(wrapper.get('[data-testid="file-info"]').text()).toContain('retry.txt')
|
||||
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
const ids = fetchMock.mock.calls.map(call => (call[1]?.body as FormData).get('requestId'))
|
||||
expect(ids).toHaveLength(2)
|
||||
expect(ids[0]).toBe(ids[1])
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
})
|
||||
|
||||
it('appends an uploaded chapter to a chosen book without a language field', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
const url = String(input)
|
||||
if (url.endsWith('/books')) return ok({ items: [{ ...book, chapterCount: 1, pendingCount: 0, processingCount: 0, readyCount: 1, failedCount: 0, ...timestamps }] })
|
||||
expect(url).toBe('/api/v1/books/1/chapters/upload')
|
||||
const body = init?.body as FormData
|
||||
expect(body.get('language')).toBeNull()
|
||||
return ok({ chapter: { ...chapter, id: 10, ordinal: 2 }, job: { id: 6, bookId: 1, chapterId: 10, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps }, duplicate: false })
|
||||
})
|
||||
const router = await viewAt('/import?book=1')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
await chooseFile(wrapper, new File(['Second chapter.\n'], 'second.txt'))
|
||||
await wrapper.find('input#title').setValue('Appended Chapter')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
|
||||
const posts = fetchMock.mock.calls.filter(call => String(call[0]).includes('/upload'))
|
||||
expect(posts).toHaveLength(1)
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
})
|
||||
|
||||
it('keeps the server message and the form when the upload is rejected', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(new Response(JSON.stringify({ code: 400, msg: '文件不是 UTF-8 编码，请另存为 UTF-8 后重试' }), { status: 400 }))
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
await chooseFile(wrapper, new File(['Body.\n'], 'server-rejects.txt'))
|
||||
await wrapper.find('input#title').setValue('Server Rejects')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(wrapper.text()).toContain('文件不是 UTF-8 编码，请另存为 UTF-8 后重试')
|
||||
expect(wrapper.get('[data-testid="file-info"]').text()).toContain('server-rejects.txt')
|
||||
expect(router.currentRoute.value.path).toBe('/import')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,85 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted, onUnmounted, ref } from 'vue'
|
||||
import { ElButton, ElInput, ElOption, ElRadio, ElRadioGroup, ElSelect } from 'element-plus'
|
||||
import { TERM_STATUSES, type LookupResult, type TermStatus } from '../composables/useReaderLookup'
|
||||
|
||||
defineProps<{
|
||||
word: string
|
||||
result: LookupResult | null
|
||||
loading: boolean
|
||||
error: string
|
||||
saving: boolean
|
||||
saveError: string
|
||||
saved: string
|
||||
savedTermId: number | null
|
||||
prefilling: boolean
|
||||
prefillError: string
|
||||
canSave: boolean
|
||||
}>()
|
||||
const definition = defineModel<string>('definition', { required: true })
|
||||
const examples = defineModel<string>('examples', { required: true })
|
||||
const status = defineModel<TermStatus>('status', { required: true })
|
||||
const emit = defineEmits<{ close: []; retry: []; save: []; resize: [top: number] }>()
|
||||
const heading = ref<HTMLElement | null>(null)
|
||||
const panel = ref<HTMLElement | null>(null)
|
||||
let observer: ResizeObserver | undefined
|
||||
function resized() {
|
||||
if (window.innerWidth <= 760 && panel.value) emit('resize', panel.value.getBoundingClientRect().top)
|
||||
}
|
||||
onMounted(() => {
|
||||
heading.value?.focus({ preventScroll: true })
|
||||
resized()
|
||||
if (typeof ResizeObserver !== 'undefined' && panel.value) {
|
||||
observer = new ResizeObserver(resized)
|
||||
observer.observe(panel.value)
|
||||
}
|
||||
window.addEventListener('resize', resized)
|
||||
})
|
||||
onUnmounted(() => { observer?.disconnect(); window.removeEventListener('resize', resized) })
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<aside ref="panel" class="lookup-panel" aria-labelledby="lookup-heading" @keydown.esc.stop.prevent="$emit('close')">
|
||||
<header class="lookup-heading">
|
||||
<h2 id="lookup-heading" ref="heading" tabindex="-1">{{ word }}</h2>
|
||||
<ElButton text aria-label="关闭释义" @click="$emit('close')">关闭</ElButton>
|
||||
</header>
|
||||
<div class="lookup-content" aria-live="polite" :aria-busy="loading">
|
||||
<p v-if="loading" role="status" class="subtle">正在查询…</p>
|
||||
<p v-else-if="error" role="alert" class="lookup-message">{{ error }}</p>
|
||||
<template v-else-if="result">
|
||||
<p v-if="result.status === 'resource_missing'" class="lookup-message">词典资源暂不可用。</p>
|
||||
<p v-else-if="result.status === 'not_found'" class="lookup-message">未找到释义。</p>
|
||||
<p v-if="result.status === 'lemma'" class="subtle">词形匹配：{{ result.matchedForm }}（规则候选）</p>
|
||||
<ol v-if="result.entries.length" class="lookup-senses">
|
||||
<li v-for="(entry, index) in result.entries" :key="index">
|
||||
<p class="sense-heading"><strong>{{ entry.lemma }}</strong> <span>{{ entry.pos }}</span></p>
|
||||
<p lang="en">{{ entry.definition }}</p>
|
||||
<blockquote v-for="(example, exampleIndex) in entry.examples" :key="exampleIndex" lang="en">{{ example }}</blockquote>
|
||||
</li>
|
||||
</ol>
|
||||
<p v-if="result.resource" class="subtle">{{ result.resource.name }} · {{ result.resource.version }}</p>
|
||||
</template>
|
||||
<ElButton v-if="!loading && (error || result?.status === 'resource_missing' || result?.status === 'not_found')" data-testid="lookup-retry" @click="$emit('retry')">重试查询</ElButton>
|
||||
</div>
|
||||
<div class="lookup-term">
|
||||
<p class="lookup-term-title">学习状态</p>
|
||||
<ElRadioGroup v-model="status" :disabled="saving || prefilling || !!prefillError" aria-label="学习状态">
|
||||
<ElRadio v-for="item in TERM_STATUSES" :key="item.value" :value="item.value">{{ item.label }}</ElRadio>
|
||||
</ElRadioGroup>
|
||||
<p v-if="status === 'learning'" class="subtle">学习中按 1～7 级记录，等级由复习推进。</p>
|
||||
<label for="term-definition">我的释义 <span>{{ savedTermId ? '已保存' : '新词条' }}</span></label>
|
||||
<ElInput id="term-definition" v-model="definition" type="textarea" :rows="3" :disabled="saving || prefilling || !!prefillError" placeholder="记下当前词的释义" />
|
||||
<label for="term-examples">例句 <span>每行一条，最多 5 条</span></label>
|
||||
<ElInput id="term-examples" v-model="examples" type="textarea" :rows="2" :disabled="saving || prefilling || !!prefillError" placeholder="可选，每行一条" />
|
||||
<p v-if="prefilling" role="status" class="subtle">正在读取已保存的内容…</p>
|
||||
<p v-else-if="prefillError" role="alert" class="lookup-message">{{ prefillError }}</p>
|
||||
<p v-if="saved" role="status" class="lookup-saved">✓ {{ saved }}</p>
|
||||
<p v-if="saveError" role="alert" class="lookup-message">{{ saveError }}</p>
|
||||
<div class="lookup-actions">
|
||||
<ElButton type="primary" data-testid="term-save" :loading="saving" :disabled="!canSave" @click="$emit('save')">保存到生词本</ElButton>
|
||||
<ElButton text @click="$emit('close')">关闭，继续阅读</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
</aside>
|
||||
</template>
|
||||
@@ -0,0 +1,10 @@
|
||||
<script setup lang="ts">
|
||||
import { termStatusOf, type ReaderToken } from '../composables/useReaderLookup'
|
||||
defineProps<{ tokens: ReaderToken[]; original: string; selectedStart?: number }>()
|
||||
const emit = defineEmits<{ select: [token: ReaderToken, element: HTMLElement] }>()
|
||||
function select(token: ReaderToken, event: Event) { emit('select', token, event.currentTarget as HTMLElement) }
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<article class="reader-text"><template v-if="tokens.length"><template v-for="token in tokens" :key="token.start"><span v-if="token.kind === 'word'" role="button" tabindex="0" class="reader-word" :class="[termStatusOf(token) ? `is-${termStatusOf(token)}` : '', { 'is-selected': selectedStart === token.start }]" :aria-label="termStatusOf(token) ? `查询 ${token.text}，已保存` : `查询 ${token.text}`" :aria-pressed="selectedStart === token.start" @click="select(token, $event)" @keydown.enter.prevent="select(token, $event)" @keydown.space.prevent="select(token, $event)">{{ token.text }}</span><template v-else>{{ token.text }}</template></template></template><template v-else>{{ original }}</template></article>
|
||||
</template>
|
||||
@@ -0,0 +1,49 @@
|
||||
<script setup lang="ts">
|
||||
import { nextTick, ref, watch } from 'vue'
|
||||
import { ElButton } from 'element-plus'
|
||||
import { clozeSentence, type ReviewItem } from '../stores/review'
|
||||
|
||||
const props = defineProps<{ item: ReviewItem; position: number; total: number; revealed: boolean; busy: boolean; error: string }>()
|
||||
const emit = defineEmits<{ reveal: []; grade: [grade: 'correct' | 'wrong' | 'again']; end: []; retry: [] }>()
|
||||
// The word and its masked example are enough to answer; the definition stays hidden.
|
||||
const prompt = () => clozeSentence(props.item)
|
||||
const revealButton = ref<{ $el?: HTMLElement } | null>(null)
|
||||
const correctButton = ref<{ $el?: HTMLElement } | null>(null)
|
||||
// Keyboard users should reach the next action without tabbing through the page again.
|
||||
watch(() => props.item.id, async () => {
|
||||
await nextTick()
|
||||
if (!props.revealed) revealButton.value?.$el?.focus()
|
||||
}, { immediate: true })
|
||||
watch(() => props.revealed, async value => {
|
||||
if (!value) return
|
||||
await nextTick()
|
||||
correctButton.value?.$el?.focus()
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section class="review-card" aria-labelledby="review-word" data-testid="review-card">
|
||||
<p class="subtle" data-testid="review-position">到期复习 · {{ position }} / {{ total }}</p>
|
||||
<h2 id="review-word" tabindex="-1" class="review-word">{{ item.originalForm }}</h2>
|
||||
<p v-if="prompt()" class="review-example" lang="en">{{ prompt() }}</p>
|
||||
<p class="subtle">{{ item.status === 'learning' ? `学习中 · 等级 ${item.level}` : '新词 · 尚未复习' }}</p>
|
||||
<p v-if="error" role="alert" class="notice">{{ error }}</p>
|
||||
|
||||
<template v-if="!revealed">
|
||||
<ElButton ref="revealButton" type="primary" data-testid="review-reveal" :disabled="busy || !!error" @click="emit('reveal')">显示答案</ElButton>
|
||||
</template>
|
||||
<template v-else>
|
||||
<div class="review-answer">
|
||||
<p class="review-definition" data-testid="review-definition">{{ item.definition || '（未填写个人释义）' }}</p>
|
||||
<blockquote v-for="(example, index) in item.examples" :key="index" lang="en">{{ example }}</blockquote>
|
||||
</div>
|
||||
<div class="review-grades">
|
||||
<ElButton ref="correctButton" type="primary" data-testid="review-correct" :loading="busy" @click="emit('grade', 'correct')">认识 / 答对</ElButton>
|
||||
<ElButton data-testid="review-wrong" :disabled="busy" @click="emit('grade', 'wrong')">不认识 / 答错</ElButton>
|
||||
<ElButton data-testid="review-again" :disabled="busy" @click="emit('grade', 'again')">再学一次</ElButton>
|
||||
</div>
|
||||
</template>
|
||||
<p v-if="error" class="review-actions"><ElButton data-testid="review-retry" @click="emit('retry')">重试提交</ElButton></p>
|
||||
<p class="review-actions"><ElButton text data-testid="review-end" @click="emit('end')">结束本次复习</ElButton></p>
|
||||
</section>
|
||||
</template>
|
||||
@@ -0,0 +1,257 @@
|
||||
import { computed, onScopeDispose, ref, watch, type Ref } from 'vue'
|
||||
import type { ChapterDetail } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
export type TermStatus = 'new' | 'learning' | 'known' | 'ignored'
|
||||
|
||||
export interface TokenTerm { id: number; status: TermStatus; level: number }
|
||||
export interface ReaderToken {
|
||||
text: string
|
||||
start: number
|
||||
end: number
|
||||
startUtf16: number
|
||||
endUtf16: number
|
||||
kind: 'word' | 'space' | 'punctuation'
|
||||
term?: TokenTerm | null
|
||||
}
|
||||
export interface LookupResult {
|
||||
status: 'exact' | 'lemma' | 'not_found' | 'resource_missing'
|
||||
query: string
|
||||
matchedForm: string | null
|
||||
candidates: string[]
|
||||
entries: { lemma: string; pos: string; definition: string; examples: string[] }[]
|
||||
resource?: { name: string; version: string }
|
||||
}
|
||||
export interface SavedTerm {
|
||||
id: number
|
||||
term: string
|
||||
originalForm: string
|
||||
definition: string
|
||||
examples: string[]
|
||||
status: TermStatus
|
||||
level: number
|
||||
}
|
||||
interface TermResponse { term: SavedTerm }
|
||||
interface TokenResponse { textSha256: string; tokens: ReaderToken[] }
|
||||
|
||||
// The four learner-visible states and the level rule behind them. Only a learning
|
||||
// entry carries a level, so every other status reports level 0.
|
||||
export const TERM_STATUSES: { value: TermStatus; label: string }[] = [
|
||||
{ value: 'new', label: '新词' },
|
||||
{ value: 'learning', label: '学习中' },
|
||||
{ value: 'known', label: '已知' },
|
||||
{ value: 'ignored', label: '忽略' },
|
||||
]
|
||||
|
||||
export function termStatusLabel(status: TermStatus): string {
|
||||
return TERM_STATUSES.find(item => item.value === status)?.label ?? status
|
||||
}
|
||||
|
||||
/** Unknown or malformed personal state never becomes a highlight class. */
|
||||
export function termStatusOf(token: ReaderToken): TermStatus | null {
|
||||
const term = token.term
|
||||
if (!term || typeof term.id !== 'number' || term.id <= 0) return null
|
||||
return TERM_STATUSES.some(item => item.value === term.status) ? term.status : null
|
||||
}
|
||||
|
||||
// Validate every coordinate before enabling selection; original text is always the fallback.
|
||||
function matchesChapter(data: TokenResponse, chapter: ChapterDetail): boolean {
|
||||
if (!data || data.textSha256 !== chapter.contentSha256 || !Array.isArray(data.tokens)) return false
|
||||
const text = chapter.originalText ?? ''
|
||||
let cp = 0, utf16 = 0
|
||||
for (const token of data.tokens) {
|
||||
if (typeof token.text !== 'string' || !token.text || !['word', 'space', 'punctuation'].includes(token.kind)) return false
|
||||
if (token.start !== cp || token.startUtf16 !== utf16) return false
|
||||
cp += [...token.text].length
|
||||
utf16 += token.text.length
|
||||
if (token.end !== cp || token.endUtf16 !== utf16 || text.slice(token.startUtf16, utf16) !== token.text) return false
|
||||
}
|
||||
return utf16 === text.length && data.tokens.map(token => token.text).join('') === text
|
||||
}
|
||||
|
||||
export function useReaderLookup(chapter: Ref<ChapterDetail | null>) {
|
||||
const session = useSessionStore()
|
||||
const tokens = ref<ReaderToken[]>([])
|
||||
const tokensError = ref('')
|
||||
const tokensLoading = ref(false)
|
||||
const selected = ref<ReaderToken | null>(null)
|
||||
const result = ref<LookupResult | null>(null)
|
||||
const loading = ref(false)
|
||||
const error = ref('')
|
||||
|
||||
// The learner's own record for the selected word.
|
||||
const definition = ref('')
|
||||
const examples = ref('')
|
||||
const status = ref<TermStatus>('new')
|
||||
const savedTermId = ref<number | null>(null)
|
||||
const prefilling = ref(false)
|
||||
const prefillError = ref('')
|
||||
const saving = ref(false)
|
||||
const saveError = ref('')
|
||||
const saved = ref('')
|
||||
|
||||
let tokenSequence = 0, lookupSequence = 0, termSequence = 0, saveSequence = 0
|
||||
let origin: HTMLElement | null = null
|
||||
let scrollBeforeAdjustment: { top: number; left: number } | null = null
|
||||
let lastAdjustment: { top: number; left: number } | null = null
|
||||
|
||||
// Sending is blocked until the stored text is on screen, so a slow or failed read
|
||||
// can never let an empty form overwrite what the learner already wrote.
|
||||
const canSave = computed(() => selected.value !== null && !saving.value && !prefilling.value && !prefillError.value)
|
||||
|
||||
function exampleLines(): string[] {
|
||||
return examples.value.split('\n').map(line => line.trim()).filter(Boolean)
|
||||
}
|
||||
|
||||
function applyTerm(next: TokenTerm) {
|
||||
savedTermId.value = next.id
|
||||
const index = tokens.value.findIndex(token => token.start === selected.value?.start)
|
||||
const target = index >= 0 ? tokens.value[index] : undefined
|
||||
if (target) tokens.value[index] = { ...target, term: next }
|
||||
}
|
||||
|
||||
function stillAtAdjustment() {
|
||||
return lastAdjustment !== null && Math.abs(window.scrollY - lastAdjustment.top) <= 2
|
||||
&& Math.abs(window.scrollX - lastAdjustment.left) <= 2
|
||||
}
|
||||
|
||||
function keepSelectionVisible(panelTop: number) {
|
||||
if (!origin) return
|
||||
const coveredBy = origin.getBoundingClientRect().bottom - panelTop + 16
|
||||
if (coveredBy <= 0) return
|
||||
// A later manual scroll becomes the new reading position, including when
|
||||
// another resize subsequently needs to reveal the selected word again.
|
||||
if (!scrollBeforeAdjustment || !stillAtAdjustment()) scrollBeforeAdjustment = { top: window.scrollY, left: window.scrollX }
|
||||
window.scrollBy({ top: coveredBy, behavior: 'instant' })
|
||||
lastAdjustment = { top: window.scrollY, left: window.scrollX }
|
||||
}
|
||||
|
||||
function close(restoreFocus = true) {
|
||||
lookupSequence++
|
||||
termSequence++
|
||||
saveSequence++
|
||||
selected.value = null
|
||||
result.value = null
|
||||
loading.value = false
|
||||
error.value = ''
|
||||
definition.value = ''
|
||||
examples.value = ''
|
||||
status.value = 'new'
|
||||
savedTermId.value = null
|
||||
prefilling.value = false
|
||||
prefillError.value = ''
|
||||
saving.value = false
|
||||
saveError.value = ''
|
||||
saved.value = ''
|
||||
if (restoreFocus && origin?.isConnected) origin.focus({ preventScroll: true })
|
||||
if (restoreFocus && scrollBeforeAdjustment && stillAtAdjustment()) window.scrollTo({ ...scrollBeforeAdjustment, behavior: 'instant' })
|
||||
scrollBeforeAdjustment = null
|
||||
lastAdjustment = null
|
||||
origin = null
|
||||
}
|
||||
function reset() {
|
||||
tokenSequence++
|
||||
tokens.value = []
|
||||
tokensError.value = ''
|
||||
tokensLoading.value = false
|
||||
close(false)
|
||||
}
|
||||
async function loadTokens() {
|
||||
const current = chapter.value
|
||||
if (!session.user || current?.status !== 'ready') return
|
||||
const seq = ++tokenSequence
|
||||
tokensLoading.value = true
|
||||
tokensError.value = ''
|
||||
try {
|
||||
const data = await session.request<TokenResponse>(`chapters/${current.id}/tokens`)
|
||||
if (seq !== tokenSequence) return
|
||||
if (!matchesChapter(data, current)) throw new Error('分词与正文不一致，请重试。')
|
||||
tokens.value = data.tokens
|
||||
} catch (reason) {
|
||||
if (seq !== tokenSequence) return
|
||||
tokens.value = []
|
||||
tokensError.value = reason instanceof Error ? reason.message : '单词暂时无法加载。'
|
||||
} finally { if (seq === tokenSequence) tokensLoading.value = false }
|
||||
}
|
||||
async function lookup() {
|
||||
const current = chapter.value
|
||||
const token = selected.value
|
||||
if (!session.user || !current || !token) return
|
||||
const seq = ++lookupSequence
|
||||
result.value = null
|
||||
error.value = ''
|
||||
loading.value = true
|
||||
try {
|
||||
const data = await session.request<LookupResult>('lookup', 'POST', { chapterId: current.id, start: token.start, end: token.end })
|
||||
if (seq === lookupSequence) result.value = data
|
||||
} catch (reason) {
|
||||
if (seq === lookupSequence) error.value = reason instanceof Error ? reason.message : '暂时无法查询，请重试。'
|
||||
} finally { if (seq === lookupSequence) loading.value = false }
|
||||
}
|
||||
// Opening a word the learner already saved loads that record, so the panel shows
|
||||
// the stored text and the same id other chapters show.
|
||||
async function loadTerm(id: number) {
|
||||
const seq = ++termSequence
|
||||
prefilling.value = true
|
||||
prefillError.value = ''
|
||||
try {
|
||||
const data = await session.request<TermResponse>(`terms/${id}`)
|
||||
if (seq !== termSequence) return
|
||||
definition.value = data.term.definition
|
||||
examples.value = data.term.examples.join('\n')
|
||||
status.value = data.term.status
|
||||
applyTerm({ id: data.term.id, status: data.term.status, level: data.term.level })
|
||||
} catch (reason) {
|
||||
if (seq !== termSequence) return
|
||||
prefillError.value = reason instanceof Error ? reason.message : '已保存的内容暂时无法读取。'
|
||||
} finally { if (seq === termSequence) prefilling.value = false }
|
||||
}
|
||||
async function save() {
|
||||
const current = chapter.value
|
||||
const token = selected.value
|
||||
if (!session.user || !current || !token || !canSave.value) return
|
||||
const seq = ++saveSequence
|
||||
saving.value = true
|
||||
saveError.value = ''
|
||||
saved.value = ''
|
||||
try {
|
||||
const data = await session.request<TermResponse & { created: boolean }>('terms', 'POST', {
|
||||
chapterId: current.id, start: token.start, end: token.end,
|
||||
definition: definition.value, examples: exampleLines(), status: status.value,
|
||||
})
|
||||
if (seq !== saveSequence) return
|
||||
definition.value = data.term.definition
|
||||
examples.value = data.term.examples.join('\n')
|
||||
status.value = data.term.status
|
||||
applyTerm({ id: data.term.id, status: data.term.status, level: data.term.level })
|
||||
saved.value = `已保存 · ${termStatusLabel(data.term.status)}`
|
||||
} catch (reason) {
|
||||
if (seq !== saveSequence) return
|
||||
saveError.value = reason instanceof Error ? reason.message : '保存失败，请稍后重试。'
|
||||
} finally { if (seq === saveSequence) saving.value = false }
|
||||
}
|
||||
function select(token: ReaderToken, element: HTMLElement) {
|
||||
close(false)
|
||||
origin = element
|
||||
selected.value = token
|
||||
const existing = termStatusOf(token)
|
||||
if (existing !== null && token.term) {
|
||||
status.value = existing
|
||||
void loadTerm(token.term.id)
|
||||
}
|
||||
void lookup()
|
||||
}
|
||||
watch(() => [chapter.value?.id, chapter.value?.status, chapter.value?.contentSha256, chapter.value?.originalText], () => {
|
||||
reset()
|
||||
void loadTokens()
|
||||
}, { immediate: true, flush: 'sync' })
|
||||
// Watch the identity object, including clear → login for the same account: leaving
|
||||
// or switching an account drops every word saved by the previous one.
|
||||
watch(() => session.user, reset, { flush: 'sync' })
|
||||
onScopeDispose(reset)
|
||||
return {
|
||||
tokens, tokensError, tokensLoading, selected, result, loading, error,
|
||||
definition, examples, status, savedTermId, prefilling, prefillError, saving, saveError, saved, canSave,
|
||||
loadTokens, lookup, loadTerm, save, select, close, reset, keepSelectionVisible,
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,10 @@ import App from './App.vue'
|
||||
import router from './router'
|
||||
import 'element-plus/es/components/button/style/css'
|
||||
import 'element-plus/es/components/input/style/css'
|
||||
import 'element-plus/es/components/radio/style/css'
|
||||
import 'element-plus/es/components/radio-group/style/css'
|
||||
import 'element-plus/es/components/select/style/css'
|
||||
import 'element-plus/es/components/option/style/css'
|
||||
import './style.css'
|
||||
|
||||
createApp(App).use(createPinia()).use(router).mount('#app')
|
||||
|
||||
@@ -6,6 +6,10 @@ const router = createRouter({
|
||||
routes: [
|
||||
{ path: '/login', name: 'login', component: () => import('../views/LoginView.vue') },
|
||||
{ path: '/', name: 'library', meta: { private: true }, component: () => import('../views/LibraryView.vue') },
|
||||
{ path: '/review', name: 'review', meta: { private: true }, component: () => import('../views/ReviewView.vue') },
|
||||
{ path: '/import', name: 'import', meta: { private: true }, component: () => import('../views/ImportView.vue') },
|
||||
{ path: '/books/:id', name: 'book', meta: { private: true }, component: () => import('../views/BookView.vue') },
|
||||
{ path: '/chapters/:id', name: 'chapter', meta: { private: true }, component: () => import('../views/ReaderView.vue') },
|
||||
{ path: '/:pathMatch(.*)*', redirect: '/' },
|
||||
],
|
||||
})
|
||||
|
||||
@@ -0,0 +1,621 @@
|
||||
import { defineStore } from 'pinia'
|
||||
import { computed, ref, watch } from 'vue'
|
||||
import { ApiError, useSessionStore } from './session'
|
||||
|
||||
export type ChapterStatus = 'pending' | 'processing' | 'ready' | 'failed'
|
||||
|
||||
export interface BookRef { id: number; title: string; language: string }
|
||||
|
||||
export interface BookSummary extends BookRef {
|
||||
chapterCount: number
|
||||
pendingCount: number
|
||||
processingCount: number
|
||||
readyCount: number
|
||||
failedCount: number
|
||||
createdAt: string
|
||||
updatedAt: string
|
||||
}
|
||||
|
||||
export interface ChapterSummary {
|
||||
id: number
|
||||
bookId: number
|
||||
ordinal: number
|
||||
title: string
|
||||
status: ChapterStatus
|
||||
charCount: number
|
||||
errorReason: string
|
||||
errorMessage: string
|
||||
// Present wherever a chapter appears; null while the job id is unknown.
|
||||
jobId: number | null
|
||||
createdAt: string
|
||||
updatedAt: string
|
||||
}
|
||||
|
||||
export interface ChapterDetail extends ChapterSummary {
|
||||
contentSha256: string
|
||||
// Present only for ready chapters; never cached or faked for other statuses.
|
||||
originalText?: string
|
||||
}
|
||||
|
||||
export interface Job {
|
||||
id: number
|
||||
bookId: number
|
||||
chapterId: number
|
||||
status: ChapterStatus
|
||||
attempts: number
|
||||
errorReason: string
|
||||
errorMessage: string
|
||||
createdAt: string
|
||||
updatedAt: string
|
||||
}
|
||||
|
||||
export interface ChapterNavigation { previousChapterId: number | null; nextChapterId: number | null }
|
||||
|
||||
/** The editable text of one owned chapter; separate from the ready-only reader payload. */
|
||||
export interface ChapterSource {
|
||||
id: number
|
||||
bookId: number
|
||||
ordinal: number
|
||||
title: string
|
||||
text: string
|
||||
status: ChapterStatus
|
||||
contentSha256: string
|
||||
charCount: number
|
||||
}
|
||||
|
||||
export interface ChapterEdit { chapter: ChapterSummary; job: Job | null; versionChanged: boolean }
|
||||
export interface DeletionResult { bookId?: number; chapterId?: number; chapters?: number; remaining: number }
|
||||
|
||||
export type SubmitTarget = { mode: 'new' } | { mode: 'append'; bookId: number }
|
||||
|
||||
export interface SubmitInput { title: string; text: string; target: SubmitTarget }
|
||||
|
||||
export const POLL_INTERVAL_MS = 1500
|
||||
export const TITLE_MAX_LENGTH = 120
|
||||
export const TEXT_MAX_CODE_POINTS = 100000
|
||||
// Mirrors the server limit for one TXT upload.
|
||||
export const TXT_MAX_BYTES = 2 * 1024 * 1024
|
||||
export const NOT_FOUND_MESSAGE = '内容不存在。'
|
||||
export const LANGUAGE_LABEL = '英语'
|
||||
export const LANGUAGE_CODE = 'en'
|
||||
|
||||
const CHAPTER_STATUS_LABELS: Record<ChapterStatus, string> = {
|
||||
pending: '待处理',
|
||||
processing: '处理中',
|
||||
ready: '已就绪',
|
||||
failed: '处理失败',
|
||||
}
|
||||
|
||||
export function statusLabel(status: ChapterStatus): string {
|
||||
return CHAPTER_STATUS_LABELS[status]
|
||||
}
|
||||
|
||||
function isUnsettled(status: ChapterStatus): boolean {
|
||||
return status === 'pending' || status === 'processing'
|
||||
}
|
||||
|
||||
/** Mirrors the server rule: non-empty after trim and at most 120 characters. */
|
||||
export function titleProblem(title: string): string {
|
||||
const trimmed = title.trim()
|
||||
if (!trimmed) return '请填写标题。'
|
||||
if ([...trimmed].length > TITLE_MAX_LENGTH) return `标题不能超过 ${TITLE_MAX_LENGTH} 个字符。`
|
||||
return ''
|
||||
}
|
||||
|
||||
/** Mirrors the server rule: at least one non-whitespace character, at most 100000 code points. */
|
||||
export function textProblem(text: string): string {
|
||||
if (!text.trim()) return '请粘贴要导入的英文正文。'
|
||||
if ([...text].length > TEXT_MAX_CODE_POINTS) return `正文不能超过 ${TEXT_MAX_CODE_POINTS} 个字符。`
|
||||
return ''
|
||||
}
|
||||
|
||||
/**
|
||||
* Client-side pre-check for a TXT upload. The server validates the file again and stays the
|
||||
* only authority; this only tells the learner about an obviously unusable choice earlier.
|
||||
*/
|
||||
export function fileProblem(file: { name: string; size: number }): string {
|
||||
if (!/\.txt$/i.test(file.name)) return '请选择 .txt 文件。'
|
||||
if (file.size === 0) return '文件是空的，请选择包含英文正文的 UTF-8 TXT。'
|
||||
if (file.size > TXT_MAX_BYTES) return 'TXT 文件不能超过 2 MiB。'
|
||||
return ''
|
||||
}
|
||||
|
||||
/** A readable size for the selected file, e.g. `2 KB` or `1.5 MB`. */
|
||||
export function fileSizeLabel(bytes: number): string {
|
||||
if (bytes < 1024) return `${bytes} B`
|
||||
if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)} KB`
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
|
||||
}
|
||||
|
||||
/** A failed chapter can be retried as soon as the API told us its job id. */
|
||||
export function canRetry(chapter: Pick<ChapterSummary, 'status' | 'jobId'>): boolean {
|
||||
return chapter.status === 'failed' && chapter.jobId !== null
|
||||
}
|
||||
|
||||
/** Compact one-line status summary for a book card, e.g. `已就绪 2 · 处理中 1 · 待处理 1 · 失败 1`. */
|
||||
export function statusSummary(book: BookSummary): string {
|
||||
const parts: string[] = []
|
||||
if (book.readyCount > 0) parts.push(`已就绪 ${book.readyCount}`)
|
||||
if (book.processingCount > 0) parts.push(`处理中 ${book.processingCount}`)
|
||||
if (book.pendingCount > 0) parts.push(`待处理 ${book.pendingCount}`)
|
||||
if (book.failedCount > 0) parts.push(`失败 ${book.failedCount}`)
|
||||
return parts.join(' · ')
|
||||
}
|
||||
|
||||
interface LoadOptions { silent?: boolean }
|
||||
interface SubmitBookBody { requestId: string; title: string; text: string; language: 'en' }
|
||||
// An appended chapter owns the language of its book, so the append contract has no language
|
||||
// field; the server rejects unknown fields, and a client that sends one gets HTTP 400.
|
||||
interface SubmitChapterBody { requestId: string; title: string; text: string }
|
||||
export interface UploadInput { title: string; target: SubmitTarget; file: File }
|
||||
interface Created { bookId: number; chapter: ChapterSummary }
|
||||
|
||||
function emptyNavigation(): ChapterNavigation {
|
||||
return { previousChapterId: null, nextChapterId: null }
|
||||
}
|
||||
|
||||
export const useLibraryStore = defineStore('library', () => {
|
||||
const session = useSessionStore()
|
||||
|
||||
const books = ref<BookSummary[]>([])
|
||||
const booksLoading = ref(false)
|
||||
const booksError = ref('')
|
||||
|
||||
const book = ref<BookRef | null>(null)
|
||||
const chapters = ref<ChapterSummary[]>([])
|
||||
const bookLoading = ref(false)
|
||||
const bookError = ref('')
|
||||
|
||||
const chapter = ref<ChapterDetail | null>(null)
|
||||
const chapterBook = ref<BookRef | null>(null)
|
||||
const navigation = ref<ChapterNavigation>(emptyNavigation())
|
||||
const chapterLoading = ref(false)
|
||||
const chapterError = ref('')
|
||||
|
||||
const submitting = ref(false)
|
||||
const submitError = ref('')
|
||||
const retryingChapterId = ref<number | null>(null)
|
||||
|
||||
/** Reader text exists only for ready chapters and is never taken from a cache. */
|
||||
const readerText = computed(() => (chapter.value?.status === 'ready' ? chapter.value.originalText ?? '' : ''))
|
||||
|
||||
// Every request is tagged with a generation and the owning account so that a
|
||||
// late response can never repopulate the view after logout or an account switch.
|
||||
let generation = 0
|
||||
let booksSeq = 0
|
||||
let bookSeq = 0
|
||||
let chapterSeq = 0
|
||||
let pollTimer: number | undefined
|
||||
// One requestId per unsent form content: a double click or a repeat submit of
|
||||
// unchanged content must create one chapter, not two.
|
||||
let submissionKey = ''
|
||||
let submissionRequestId = ''
|
||||
|
||||
watch(() => session.user?.id ?? null, (next, previous) => {
|
||||
if (next !== previous) reset()
|
||||
}, { flush: 'sync' })
|
||||
|
||||
function ownerId(): number | null {
|
||||
return session.user?.id ?? null
|
||||
}
|
||||
|
||||
function isStale(version: number, owner: number | null): boolean {
|
||||
return version !== generation || ownerId() !== owner
|
||||
}
|
||||
|
||||
function failureMessage(reason: unknown, fallback: string): string {
|
||||
return reason instanceof Error && reason.message ? reason.message : fallback
|
||||
}
|
||||
|
||||
function isNotFound(reason: unknown): boolean {
|
||||
return reason instanceof ApiError && reason.status === 404
|
||||
}
|
||||
|
||||
function stopPolling(): void {
|
||||
if (pollTimer !== undefined) {
|
||||
window.clearTimeout(pollTimer)
|
||||
pollTimer = undefined
|
||||
}
|
||||
}
|
||||
|
||||
function needsPolling(): boolean {
|
||||
return (book.value !== null && chapters.value.some(item => isUnsettled(item.status)))
|
||||
|| (chapter.value !== null && isUnsettled(chapter.value.status))
|
||||
}
|
||||
|
||||
/** Schedules the next refresh, or stops polling when nothing is pending anymore. */
|
||||
function schedulePolling(): void {
|
||||
if (!needsPolling()) {
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
if (pollTimer !== undefined || !session.user) return
|
||||
pollTimer = window.setTimeout(() => {
|
||||
pollTimer = undefined
|
||||
void poll()
|
||||
}, POLL_INTERVAL_MS)
|
||||
}
|
||||
|
||||
async function poll(): Promise<void> {
|
||||
if (!session.user) {
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const bookId = book.value !== null && chapters.value.some(item => isUnsettled(item.status)) ? book.value.id : null
|
||||
const chapterId = chapter.value !== null && isUnsettled(chapter.value.status) ? chapter.value.id : null
|
||||
if (bookId !== null) await loadBook(bookId, { silent: true })
|
||||
if (chapterId !== null) await loadChapter(chapterId, { silent: true })
|
||||
if (isStale(version, owner)) {
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
schedulePolling()
|
||||
}
|
||||
|
||||
async function loadBooks(): Promise<void> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = ++booksSeq
|
||||
booksLoading.value = true
|
||||
booksError.value = ''
|
||||
try {
|
||||
const result = await session.request<{ items?: BookSummary[] } | null>('books')
|
||||
if (seq !== booksSeq || isStale(version, owner)) return
|
||||
const items = result?.items
|
||||
books.value = Array.isArray(items) ? items : []
|
||||
} catch (reason) {
|
||||
if (seq !== booksSeq || isStale(version, owner)) return
|
||||
booksError.value = failureMessage(reason, '书库暂时无法加载，请稍后重试。')
|
||||
} finally {
|
||||
if (seq === booksSeq && !isStale(version, owner)) booksLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function loadBook(id: number, options: LoadOptions = {}): Promise<void> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = ++bookSeq
|
||||
if (!options.silent) {
|
||||
bookLoading.value = true
|
||||
bookError.value = ''
|
||||
}
|
||||
try {
|
||||
const result = await session.request<{ book: BookRef; chapters?: ChapterSummary[] }>(`books/${id}`)
|
||||
if (seq !== bookSeq || isStale(version, owner)) return
|
||||
book.value = result.book
|
||||
chapters.value = Array.isArray(result.chapters) ? result.chapters : []
|
||||
bookError.value = ''
|
||||
schedulePolling()
|
||||
} catch (reason) {
|
||||
if (seq !== bookSeq || isStale(version, owner)) return
|
||||
if (isNotFound(reason)) {
|
||||
// Another account's id never resolves for this caller: report it and stop
|
||||
// instead of polling a resource that will not appear.
|
||||
book.value = null
|
||||
chapters.value = []
|
||||
bookError.value = NOT_FOUND_MESSAGE
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
// A failed background refresh keeps the data already on screen; the next
|
||||
// tick tries again and the user still sees the last known state.
|
||||
if (!options.silent) bookError.value = failureMessage(reason, '书籍暂时无法加载，请稍后重试。')
|
||||
} finally {
|
||||
if (seq === bookSeq && !isStale(version, owner)) bookLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function loadChapter(id: number, options: LoadOptions = {}): Promise<void> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = ++chapterSeq
|
||||
if (!options.silent) {
|
||||
chapterLoading.value = true
|
||||
chapterError.value = ''
|
||||
// Never keep the previous chapter's text under a new chapter id.
|
||||
if (chapter.value !== null && chapter.value.id !== id) {
|
||||
chapter.value = null
|
||||
chapterBook.value = null
|
||||
navigation.value = emptyNavigation()
|
||||
}
|
||||
}
|
||||
try {
|
||||
const result = await session.request<{ book: BookRef; chapter: ChapterDetail; navigation?: ChapterNavigation }>(`chapters/${id}`)
|
||||
if (seq !== chapterSeq || isStale(version, owner)) return
|
||||
chapterBook.value = result.book
|
||||
chapter.value = result.chapter
|
||||
navigation.value = result.navigation ?? emptyNavigation()
|
||||
chapterError.value = ''
|
||||
schedulePolling()
|
||||
} catch (reason) {
|
||||
if (seq !== chapterSeq || isStale(version, owner)) return
|
||||
if (isNotFound(reason)) {
|
||||
chapter.value = null
|
||||
chapterBook.value = null
|
||||
navigation.value = emptyNavigation()
|
||||
chapterError.value = NOT_FOUND_MESSAGE
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
if (!options.silent) chapterError.value = failureMessage(reason, '章节暂时无法加载，请稍后重试。')
|
||||
} finally {
|
||||
if (seq === chapterSeq && !isStale(version, owner)) chapterLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function submissionKeyOf(target: SubmitTarget, title: string, text: string): string {
|
||||
return target.mode === 'new' ? `new\n${title}\n${text}` : `append:${target.bookId}\n${title}\n${text}`
|
||||
}
|
||||
|
||||
async function createBook(body: SubmitBookBody): Promise<Created> {
|
||||
const result = await session.request<{ book: BookRef; chapter: ChapterSummary }>('books', 'POST', body)
|
||||
return { bookId: result.book.id, chapter: result.chapter }
|
||||
}
|
||||
|
||||
async function appendChapter(bookId: number, body: SubmitChapterBody): Promise<Created> {
|
||||
const result = await session.request<{ chapter: ChapterSummary }>(`books/${bookId}/chapters`, 'POST', body)
|
||||
return { bookId: result.chapter.bookId, chapter: result.chapter }
|
||||
}
|
||||
|
||||
/**
|
||||
* Submits pasted text. Returns the book id to open on success and throws on
|
||||
* failure; `submitError` always carries the message shown to the user.
|
||||
*/
|
||||
async function submit(input: SubmitInput): Promise<number> {
|
||||
const title = input.title.trim()
|
||||
const text = input.text
|
||||
const problem = titleProblem(title) || textProblem(text)
|
||||
if (problem) {
|
||||
submitError.value = problem
|
||||
throw new Error(problem)
|
||||
}
|
||||
|
||||
const key = submissionKeyOf(input.target, title, text)
|
||||
if (key !== submissionKey || submissionRequestId === '') {
|
||||
submissionKey = key
|
||||
submissionRequestId = crypto.randomUUID()
|
||||
}
|
||||
const requestId = submissionRequestId
|
||||
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
submitting.value = true
|
||||
submitError.value = ''
|
||||
try {
|
||||
// Only the new-book contract carries a language; appending inherits the book's language.
|
||||
const created = input.target.mode === 'new'
|
||||
? await createBook({ requestId, title, text, language: LANGUAGE_CODE })
|
||||
: await appendChapter(input.target.bookId, { requestId, title, text })
|
||||
if (isStale(version, owner)) throw new Error('登录状态已变化，请重新提交。')
|
||||
// The content was accepted; a later submit must use a fresh requestId.
|
||||
submissionKey = ''
|
||||
submissionRequestId = ''
|
||||
return created.bookId
|
||||
} catch (reason) {
|
||||
if (!isStale(version, owner)) submitError.value = failureMessage(reason, '提交失败，请稍后重试。')
|
||||
throw reason instanceof Error ? reason : new Error('提交失败，请稍后重试。')
|
||||
} finally {
|
||||
if (!isStale(version, owner)) submitting.value = false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Uploads one TXT file. The file and the title share the request id discipline of a paste,
|
||||
* so a repeated upload of the same file answers with the chapter it already created.
|
||||
*/
|
||||
async function upload(input: UploadInput): Promise<number> {
|
||||
const title = input.title.trim()
|
||||
const problem = titleProblem(title) || fileProblem(input.file)
|
||||
if (problem) {
|
||||
submitError.value = problem
|
||||
throw new Error(problem)
|
||||
}
|
||||
const key = `upload\n${title}\n${input.file.name}\n${input.file.size}\n${input.file.lastModified}`
|
||||
if (key !== submissionKey || submissionRequestId === '') {
|
||||
submissionKey = key
|
||||
submissionRequestId = crypto.randomUUID()
|
||||
}
|
||||
const requestId = submissionRequestId
|
||||
const form = new FormData()
|
||||
form.append('requestId', requestId)
|
||||
form.append('title', title)
|
||||
// Only the new-book contract carries a language; appending inherits the book's language.
|
||||
const path = input.target.mode === 'new' ? 'books/upload' : `books/${input.target.bookId}/chapters/upload`
|
||||
if (input.target.mode === 'new') form.append('language', LANGUAGE_CODE)
|
||||
form.append('file', input.file, input.file.name)
|
||||
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
submitting.value = true
|
||||
submitError.value = ''
|
||||
try {
|
||||
const result = await session.request<{ book?: BookRef; chapter: ChapterSummary }>(path, 'POST', form)
|
||||
if (isStale(version, owner)) throw new Error('登录状态已变化，请重新提交。')
|
||||
submissionKey = ''
|
||||
submissionRequestId = ''
|
||||
return result.book?.id ?? result.chapter.bookId
|
||||
} catch (reason) {
|
||||
if (!isStale(version, owner)) submitError.value = failureMessage(reason, '上传失败，请稍后重试。')
|
||||
throw reason instanceof Error ? reason : new Error('上传失败，请稍后重试。')
|
||||
} finally {
|
||||
if (!isStale(version, owner)) submitting.value = false
|
||||
}
|
||||
}
|
||||
|
||||
/** Renames one owned book; the reply is the stored book. */
|
||||
async function renameBook(id: number, title: string): Promise<BookRef> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const problem = titleProblem(title)
|
||||
if (problem) throw new Error(problem)
|
||||
const result = await session.request<{ book: BookRef }>(`books/${id}`, 'PATCH', { title: title.trim() })
|
||||
if (!isStale(version, owner)) {
|
||||
book.value = result.book
|
||||
const listed = books.value.find(item => item.id === id)
|
||||
if (listed) listed.title = result.book.title
|
||||
}
|
||||
return result.book
|
||||
}
|
||||
|
||||
/**
|
||||
* Saves a chapter title and/or text. A changed text becomes a new version and re-processes;
|
||||
* the same text answered again changes nothing.
|
||||
*/
|
||||
async function updateChapter(id: number, input: { title?: string; text?: string }): Promise<ChapterEdit> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
if (input.title !== undefined) {
|
||||
const problem = titleProblem(input.title)
|
||||
if (problem) throw new Error(problem)
|
||||
}
|
||||
if (input.text !== undefined) {
|
||||
const problem = textProblem(input.text)
|
||||
if (problem) throw new Error(problem)
|
||||
}
|
||||
const body: { title?: string; text?: string } = {}
|
||||
if (input.title !== undefined) body.title = input.title.trim()
|
||||
if (input.text !== undefined) body.text = input.text
|
||||
const result = await session.request<ChapterEdit>(`chapters/${id}`, 'PATCH', body)
|
||||
if (isStale(version, owner)) return result
|
||||
const merged = { ...result.chapter, jobId: result.job?.id ?? result.chapter.jobId }
|
||||
applyChapterSummary(merged)
|
||||
if (chapter.value !== null && chapter.value.id === id && result.versionChanged) {
|
||||
// The new version is not readable yet, so the reader must drop the previous text.
|
||||
chapter.value = { ...chapter.value, ...merged, originalText: undefined }
|
||||
}
|
||||
schedulePolling()
|
||||
return result
|
||||
}
|
||||
|
||||
/** Reads the editable text of one owned chapter, in any processing state. */
|
||||
async function loadChapterSource(id: number): Promise<ChapterSource> {
|
||||
const result = await session.request<{ source: ChapterSource }>(`chapters/${id}/source`)
|
||||
return result.source
|
||||
}
|
||||
|
||||
/** Deletes one owned book with its chapters and jobs; personal records stay. */
|
||||
async function deleteBook(id: number): Promise<DeletionResult> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const result = await session.request<{ deleted: DeletionResult }>(`books/${id}`, 'DELETE')
|
||||
if (!isStale(version, owner)) {
|
||||
books.value = books.value.filter(item => item.id !== id)
|
||||
if (book.value?.id === id) closeBook()
|
||||
}
|
||||
return result.deleted
|
||||
}
|
||||
|
||||
/** Deletes one owned chapter and closes the gap in the chapter order. */
|
||||
async function deleteChapter(id: number): Promise<DeletionResult> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const result = await session.request<{ deleted: DeletionResult }>(`chapters/${id}`, 'DELETE')
|
||||
if (isStale(version, owner)) return result.deleted
|
||||
chapters.value = chapters.value.filter(item => item.id !== id)
|
||||
if (book.value !== null) await loadBook(book.value.id, { silent: true })
|
||||
return result.deleted
|
||||
}
|
||||
|
||||
/** The job id comes from the chapter itself, wherever that chapter was loaded from. */
|
||||
function jobIdOf(chapterId: number): number | null {
|
||||
const target = chapters.value.find(item => item.id === chapterId)
|
||||
?? (chapter.value?.id === chapterId ? chapter.value : null)
|
||||
return target?.jobId ?? null
|
||||
}
|
||||
|
||||
/**
|
||||
* Applies a chapter summary coming from any response to the chapter list entry and to the
|
||||
* open reader, so a queued chapter is never displayed with the state or text it had before.
|
||||
*/
|
||||
function applyChapterSummary(summary: ChapterSummary): void {
|
||||
const index = chapters.value.findIndex(item => item.id === summary.id)
|
||||
if (index >= 0) chapters.value[index] = { ...chapters.value[index], ...summary }
|
||||
if (chapter.value !== null && chapter.value.id === summary.id) {
|
||||
const originalText = summary.status === 'ready' ? chapter.value.originalText : undefined
|
||||
chapter.value = { ...chapter.value, ...summary, originalText }
|
||||
}
|
||||
}
|
||||
|
||||
async function retryChapter(chapterId: number): Promise<void> {
|
||||
const jobId = jobIdOf(chapterId)
|
||||
if (jobId === null) throw new Error('这一章暂时没有可重试的任务编号。')
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
retryingChapterId.value = chapterId
|
||||
try {
|
||||
const result = await session.request<{ job: Job; chapter: ChapterSummary }>(`jobs/${jobId}/retry`, 'POST')
|
||||
if (isStale(version, owner)) return
|
||||
// The retry is accepted, so show the queued chapter and keep tracking it even if the
|
||||
// refresh below fails: a failed silent refresh must not freeze the view on the old error.
|
||||
applyChapterSummary(result.chapter)
|
||||
schedulePolling()
|
||||
// Refresh whatever is on screen to pick up the newest job state.
|
||||
if (book.value !== null && book.value.id === result.chapter.bookId) await loadBook(result.chapter.bookId, { silent: true })
|
||||
if (chapter.value !== null && chapter.value.id === result.chapter.id) await loadChapter(result.chapter.id, { silent: true })
|
||||
} catch (reason) {
|
||||
if (isStale(version, owner)) return
|
||||
throw reason instanceof Error ? reason : new Error('重试失败，请稍后重试。')
|
||||
} finally {
|
||||
if (!isStale(version, owner)) retryingChapterId.value = null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Releases the book view so polling stops when the page is left. The request sequence is
|
||||
* advanced first, so a response that arrives after this call cannot repopulate the view or
|
||||
* restart polling for a page the user already left.
|
||||
*/
|
||||
function closeBook(): void {
|
||||
bookSeq++
|
||||
book.value = null
|
||||
chapters.value = []
|
||||
bookLoading.value = false
|
||||
bookError.value = ''
|
||||
schedulePolling()
|
||||
}
|
||||
|
||||
/** Releases the reader view, invalidating in-flight loads the same way. */
|
||||
function closeChapter(): void {
|
||||
chapterSeq++
|
||||
chapter.value = null
|
||||
chapterBook.value = null
|
||||
navigation.value = emptyNavigation()
|
||||
chapterLoading.value = false
|
||||
chapterError.value = ''
|
||||
schedulePolling()
|
||||
}
|
||||
|
||||
function reset(): void {
|
||||
generation++
|
||||
stopPolling()
|
||||
books.value = []
|
||||
booksLoading.value = false
|
||||
booksError.value = ''
|
||||
book.value = null
|
||||
chapters.value = []
|
||||
bookLoading.value = false
|
||||
bookError.value = ''
|
||||
chapter.value = null
|
||||
chapterBook.value = null
|
||||
navigation.value = emptyNavigation()
|
||||
chapterLoading.value = false
|
||||
chapterError.value = ''
|
||||
submitting.value = false
|
||||
submitError.value = ''
|
||||
retryingChapterId.value = null
|
||||
submissionKey = ''
|
||||
submissionRequestId = ''
|
||||
}
|
||||
|
||||
return {
|
||||
books, booksLoading, booksError,
|
||||
book, chapters, bookLoading, bookError,
|
||||
chapter, chapterBook, navigation, chapterLoading, chapterError,
|
||||
submitting, submitError, retryingChapterId, readerText,
|
||||
loadBooks, loadBook, loadChapter, submit, upload, retryChapter,
|
||||
renameBook, updateChapter, loadChapterSource, deleteBook, deleteChapter,
|
||||
stopPolling, closeBook, closeChapter, reset,
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,233 @@
|
||||
import { computed, ref, watch } from 'vue'
|
||||
import { defineStore } from 'pinia'
|
||||
import { useSessionStore } from './session'
|
||||
|
||||
export type ReviewGrade = 'correct' | 'wrong' | 'again'
|
||||
// The outcome of an answer: applied when the word moved, stale when another screen had
|
||||
// already reviewed it. A replayed answer repeats the outcome it was given first.
|
||||
export type ReviewResult = 'applied' | 'stale'
|
||||
|
||||
export interface ReviewItem {
|
||||
id: number
|
||||
term: string
|
||||
originalForm: string
|
||||
definition: string
|
||||
examples: string[]
|
||||
status: 'new' | 'learning'
|
||||
level: number
|
||||
dueAt: string
|
||||
reviewCount: number
|
||||
}
|
||||
|
||||
export interface ReviewAnswerResult {
|
||||
result: ReviewResult
|
||||
duplicate: boolean
|
||||
grade: ReviewGrade
|
||||
requeued: boolean
|
||||
statusBefore: string
|
||||
statusAfter: string
|
||||
levelBefore: number
|
||||
levelAfter: number
|
||||
dueAtBefore: string
|
||||
dueAtAfter: string
|
||||
item: ReviewItem
|
||||
}
|
||||
|
||||
interface QueueResponse { items: ReviewItem[]; total: number }
|
||||
|
||||
function escapeRegExp(text: string): string {
|
||||
return text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
|
||||
}
|
||||
|
||||
/**
|
||||
* The prompt shown before the answer: the first personal example with the word masked.
|
||||
* A word without examples is shown on its own; the dictionary is never consulted here.
|
||||
*/
|
||||
export function clozeSentence(item: ReviewItem): string | null {
|
||||
const line = item.examples[0]
|
||||
if (!line) return null
|
||||
const forms = [...new Set([item.term, item.originalForm].filter(Boolean))]
|
||||
let masked = line
|
||||
for (const form of forms) {
|
||||
// Whole word, any case, and either apostrophe form, so the mask matches how the
|
||||
// learner typed the sentence.
|
||||
const pattern = new RegExp(`(?<![\\p{L}\\p{M}])${escapeRegExp(form).replace(/'/g, "['’]")}(?![\\p{L}\\p{M}])`, 'giu')
|
||||
masked = masked.replace(pattern, '_____')
|
||||
}
|
||||
return masked
|
||||
}
|
||||
|
||||
export const useReviewStore = defineStore('review', () => {
|
||||
const session = useSessionStore()
|
||||
|
||||
const queue = ref<ReviewItem[]>([])
|
||||
const loading = ref(false)
|
||||
const error = ref('')
|
||||
const busy = ref(false)
|
||||
const revealed = ref(false)
|
||||
const answered = ref(0)
|
||||
const correctCount = ref(0)
|
||||
const wrongCount = ref(0)
|
||||
// Distinct words in this round: a requeued word is answered again but is one word.
|
||||
const wordsReviewed = ref(0)
|
||||
// Cards this round took off the queue, counted for every outcome, and the reason a card
|
||||
// left without a new score.
|
||||
const resolved = ref(0)
|
||||
const notice = ref('')
|
||||
const seen = new Set<number>()
|
||||
// Words due beyond the fetched page, reported by the server for this round.
|
||||
const pending = ref(0)
|
||||
let started = ref(false)
|
||||
// One answer id per card: a retry of a failed submission reuses its key, so the server
|
||||
// answers the retry from the first outcome instead of counting the same action twice.
|
||||
let attempt: { itemId: number; answerId: string } | null = null
|
||||
let sequence = 0
|
||||
let generation = 0
|
||||
|
||||
const current = computed<ReviewItem | null>(() => queue.value[0] ?? null)
|
||||
// A round that resolved cards is finished even when every answer turned out to be a
|
||||
// replay or a stale submission; only a round that never had a card is empty.
|
||||
const finished = computed(() => started.value && !loading.value && queue.value.length === 0 && resolved.value > 0)
|
||||
const empty = computed(() => started.value && !loading.value && queue.value.length === 0 && resolved.value === 0)
|
||||
|
||||
watch(() => session.user?.id ?? null, (next, previous) => {
|
||||
if (next !== previous) reset()
|
||||
}, { flush: 'sync' })
|
||||
|
||||
function ownerId(): number | null {
|
||||
return session.user?.id ?? null
|
||||
}
|
||||
|
||||
function isStale(version: number, owner: number | null): boolean {
|
||||
return version !== generation || ownerId() !== owner
|
||||
}
|
||||
|
||||
function reset(): void {
|
||||
generation++
|
||||
sequence++
|
||||
queue.value = []
|
||||
loading.value = false
|
||||
error.value = ''
|
||||
busy.value = false
|
||||
revealed.value = false
|
||||
answered.value = 0
|
||||
correctCount.value = 0
|
||||
wrongCount.value = 0
|
||||
wordsReviewed.value = 0
|
||||
resolved.value = 0
|
||||
notice.value = ''
|
||||
pending.value = 0
|
||||
started.value = false
|
||||
attempt = null
|
||||
seen.clear()
|
||||
}
|
||||
|
||||
async function load(newRound = true): Promise<void> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = ++sequence
|
||||
loading.value = true
|
||||
error.value = ''
|
||||
revealed.value = false
|
||||
notice.value = ''
|
||||
attempt = null
|
||||
try {
|
||||
const result = await session.request<QueueResponse>('reviews/queue')
|
||||
if (seq !== sequence || isStale(version, owner)) return
|
||||
const items = Array.isArray(result?.items) ? result.items : []
|
||||
queue.value = items
|
||||
const total = typeof result?.total === 'number' ? result.total : items.length
|
||||
pending.value = Math.max(0, total - items.length)
|
||||
// A fresh round starts at zero; continuing with the next page of due words keeps
|
||||
// the counters of the round the learner is already in.
|
||||
if (newRound) {
|
||||
answered.value = 0
|
||||
correctCount.value = 0
|
||||
wrongCount.value = 0
|
||||
wordsReviewed.value = 0
|
||||
resolved.value = 0
|
||||
seen.clear()
|
||||
}
|
||||
started.value = true
|
||||
} catch (reason) {
|
||||
if (seq !== sequence || isStale(version, owner)) return
|
||||
error.value = reason instanceof Error ? reason.message : '复习队列暂时无法加载，请稍后重试。'
|
||||
} finally {
|
||||
if (seq === sequence && !isStale(version, owner)) loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function answerIdFor(item: ReviewItem): string {
|
||||
if (!attempt || attempt.itemId !== item.id) attempt = { itemId: item.id, answerId: crypto.randomUUID() }
|
||||
return attempt.answerId
|
||||
}
|
||||
|
||||
function reveal(): void {
|
||||
revealed.value = true
|
||||
}
|
||||
|
||||
/**
|
||||
* Sends one grade for the current card. A failed request keeps the card and its input
|
||||
* so the learner can retry; the same answer id is reused on that retry.
|
||||
*/
|
||||
async function answer(grade: ReviewGrade): Promise<void> {
|
||||
const item = current.value
|
||||
if (!item || busy.value || !session.user) return
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = sequence
|
||||
busy.value = true
|
||||
error.value = ''
|
||||
try {
|
||||
const result = await session.request<ReviewAnswerResult>(`reviews/${item.id}/answers`, 'POST', {
|
||||
answerId: answerIdFor(item), grade, expectedDueAt: item.dueAt,
|
||||
})
|
||||
if (seq !== sequence || isStale(version, owner)) return
|
||||
applyResult(item, result)
|
||||
} catch (reason) {
|
||||
if (seq !== sequence || isStale(version, owner)) return
|
||||
error.value = reason instanceof Error ? reason.message : '评分暂时无法提交，请重试。'
|
||||
} finally {
|
||||
if (seq === sequence && !isStale(version, owner)) busy.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function applyResult(item: ReviewItem, result: ReviewAnswerResult): void {
|
||||
// The card leaves the round in every outcome; a stale or replayed answer is the same
|
||||
// user action seen twice, not a second review.
|
||||
queue.value = queue.value.filter(entry => entry.id !== item.id)
|
||||
attempt = null
|
||||
revealed.value = false
|
||||
resolved.value += 1
|
||||
if (result.result === 'applied') {
|
||||
// A replay repeats the first outcome, and the first attempt may be this client's own
|
||||
// submission whose response was lost, so it counts exactly like that attempt.
|
||||
notice.value = ''
|
||||
answered.value += 1
|
||||
if (!seen.has(item.id)) {
|
||||
seen.add(item.id)
|
||||
wordsReviewed.value += 1
|
||||
}
|
||||
if (result.grade === 'correct') correctCount.value += 1
|
||||
else wrongCount.value += 1
|
||||
} else {
|
||||
notice.value = result.duplicate
|
||||
? '该词已按上一次的评分记录，未重复计分。'
|
||||
: '该词已在其他页面复习，本次未计分。'
|
||||
}
|
||||
if (result.requeued) {
|
||||
const next = result.item ?? { ...item, level: result.levelAfter, status: result.statusAfter as ReviewItem['status'], dueAt: result.dueAtAfter }
|
||||
queue.value = [...queue.value, { ...next, dueAt: result.dueAtAfter, level: result.levelAfter, status: result.statusAfter as ReviewItem['status'] }]
|
||||
}
|
||||
}
|
||||
|
||||
// Load the next page of due words, keeping the counters of the round in progress.
|
||||
async function continueRound(): Promise<void> {
|
||||
await load(false)
|
||||
}
|
||||
|
||||
return {
|
||||
queue, current, loading, error, busy, revealed, answered, correctCount, wrongCount, wordsReviewed, resolved,
|
||||
notice, pending, finished, empty, load, reveal, answer, continueRound, reset,
|
||||
}
|
||||
})
|
||||
@@ -2,6 +2,17 @@ import { defineStore } from 'pinia'
|
||||
import { ref } from 'vue'
|
||||
|
||||
export const TOKEN_KEY = 'lexgo-learner-token'
|
||||
// Carries the real HTTP status alongside the server message so callers can tell
|
||||
// "this id does not exist for me" (404) from a transient failure without
|
||||
// re-parsing the envelope. The message itself is unchanged.
|
||||
export class ApiError extends Error {
|
||||
readonly status: number
|
||||
constructor(message: string, status: number) {
|
||||
super(message)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
}
|
||||
}
|
||||
interface User { id: number; username: string; role: 'admin' | 'learner' }
|
||||
interface Space { ownerId: number; language: 'en' }
|
||||
interface Login { token: string; expiresAt: string; user: User }
|
||||
@@ -25,10 +36,12 @@ export const useSessionStore = defineStore('session', () => {
|
||||
}
|
||||
|
||||
async function request<T>(path: string, method = 'GET', body?: unknown, auth = token, version = generation): Promise<T> {
|
||||
// A multipart body carries its own content type with the boundary, so it is sent as is.
|
||||
const multipart = body instanceof FormData
|
||||
const response = await fetch(`/api/v1/${path}`, {
|
||||
method,
|
||||
headers: { ...(auth ? { Authorization: `Bearer ${auth}` } : {}), ...(body ? { 'Content-Type': 'application/json' } : {}) },
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
headers: { ...(auth ? { Authorization: `Bearer ${auth}` } : {}), ...(body && !multipart ? { 'Content-Type': 'application/json' } : {}) },
|
||||
body: body ? (multipart ? body : JSON.stringify(body)) : undefined,
|
||||
cache: 'no-store',
|
||||
})
|
||||
if (response.status === 401 && version === generation) {
|
||||
@@ -36,7 +49,7 @@ export const useSessionStore = defineStore('session', () => {
|
||||
notice.value = '登录已失效，请重新登录。'
|
||||
}
|
||||
const result = await response.json()
|
||||
if (!response.ok || result.code !== 200) throw new Error(result.msg || '请求失败，请稍后重试。')
|
||||
if (!response.ok || result.code !== 200) throw new ApiError(result.msg || '请求失败，请稍后重试。', response.status)
|
||||
return result.data as T
|
||||
}
|
||||
|
||||
@@ -91,5 +104,5 @@ export const useSessionStore = defineStore('session', () => {
|
||||
if (previousToken) await request<null>('logout', 'POST', undefined, previousToken, -1)
|
||||
}
|
||||
|
||||
return { user, space, notice, login, restore, logout, loadSpace }
|
||||
return { user, space, notice, login, restore, logout, loadSpace, request }
|
||||
})
|
||||
|
||||
@@ -51,6 +51,95 @@ h1 { font-size: 30px; font-weight: 600; margin: 14px 0; letter-spacing: 1px; }
|
||||
.empty-library .book-mark { color: #6e8967; width: 56px; height: 56px; }
|
||||
.empty-library h2 { font-weight: 500; font-size: 21px; margin: 26px 0 4px; }
|
||||
.loading { padding: 80px 24px; text-align: center; color: #748073; }
|
||||
/* Import / book / reader pages share one shell. */
|
||||
.page { max-width: 1120px; margin: 60px auto; padding: 0 28px; }
|
||||
.page-title { display: flex; align-items: center; justify-content: space-between; gap: 18px; flex-wrap: wrap; margin-bottom: 28px; }
|
||||
.page-title h1 { margin: 14px 0 6px; overflow-wrap: anywhere; }
|
||||
.page-actions { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; }
|
||||
.library-actions { display: flex; align-items: center; gap: 14px; flex-wrap: wrap; }
|
||||
.link-button { display: inline-flex; align-items: center; min-height: 42px; padding: 0 8px; color: #315c43; }
|
||||
.breadcrumb { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; margin: 0 0 4px; font-size: 14px; color: #748073; }
|
||||
.breadcrumb a { color: #315c43; }
|
||||
/* Library book cards. */
|
||||
.book-grid { list-style: none; margin: 0; padding: 0; display: grid; gap: 18px; grid-template-columns: repeat(auto-fill, minmax(290px, 1fr)); }
|
||||
.book-card { display: flex; flex-direction: column; gap: 10px; background: #fffdf8; border: 1px solid #e0e3d8; border-radius: 12px; padding: 22px; }
|
||||
.book-card .subtle { margin: 0; }
|
||||
.book-title { font-family: Georgia, 'Microsoft YaHei', serif; font-size: 21px; font-weight: 600; color: #233d31; text-decoration: none; overflow-wrap: anywhere; }
|
||||
.book-title:hover { color: #315c43; text-decoration: underline; }
|
||||
.status-summary { align-self: flex-start; margin: 0; padding: 6px 13px; border-radius: 20px; background: #eef2eb; color: #3d5b48; font-size: 13px; }
|
||||
/* Chapter and job status, identical vocabulary for both. */
|
||||
.status-chip { display: inline-flex; align-items: center; white-space: nowrap; padding: 5px 13px; border: 1px solid transparent; border-radius: 20px; font-size: 13px; }
|
||||
.status-pending { background: #f5f2e4; border-color: #e2dcc2; color: #7a6a35; }
|
||||
.status-processing { background: #eaf1f7; border-color: #c9dcea; color: #35566e; }
|
||||
.status-ready { background: #eef2eb; border-color: #cbd9c9; color: #315c43; }
|
||||
.status-failed { background: #fff0e7; border-color: #ebc3a8; color: #8b4324; }
|
||||
/* Import form. */
|
||||
.import-form { max-width: 720px; background: #fffdf8; border: 1px solid #e0e3d8; border-radius: 12px; padding: 28px; }
|
||||
.field-label { display: block; font-size: 14px; margin-bottom: 10px; }
|
||||
.fixed-value { margin: 0; padding: 12px 15px; border: 1px solid #d6dccf; border-radius: 8px; background: #fffefa; font-size: 15px; }
|
||||
.field-error { margin: 8px 0 0; color: #8b4324; font-size: 13px; line-height: 1.6; }
|
||||
.counter { margin: 8px 0 0; color: #748073; font-size: 13px; }
|
||||
.form-actions { display: flex; align-items: center; gap: 16px; flex-wrap: wrap; margin-top: 14px; }
|
||||
.import-form .el-textarea__inner { min-height: 220px; line-height: 1.9; }
|
||||
.book-select { width: 100%; }
|
||||
/* Book chapters. */
|
||||
.chapter-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 12px; }
|
||||
.chapter-row { display: grid; grid-template-columns: 40px minmax(0, 1fr) auto auto; align-items: center; gap: 14px; background: #fffdf8; border: 1px solid #e0e3d8; border-radius: 12px; padding: 16px 18px; }
|
||||
.chapter-ordinal { font-family: Georgia, serif; font-size: 17px; color: #8b9a8b; text-align: center; }
|
||||
.chapter-info { min-width: 0; }
|
||||
.chapter-name { display: inline-block; font-size: 16px; font-weight: 500; color: #233d31; text-decoration: none; overflow-wrap: anywhere; }
|
||||
a.chapter-name:hover { color: #315c43; text-decoration: underline; }
|
||||
.chapter-meta { margin: 4px 0 0; color: #748073; font-size: 13px; overflow-wrap: anywhere; }
|
||||
/* Reader keeps the pasted text exactly as it was, including line breaks and tabs. */
|
||||
.reader-page { max-width: 820px; }
|
||||
.reader-text { white-space: pre-wrap; overflow-wrap: break-word; margin: 26px 0 0; font-family: Georgia, 'Times New Roman', 'Microsoft YaHei', serif; font-size: 17px; line-height: 2; }
|
||||
.processing-hint { padding: 36px 0; color: #748073; }
|
||||
.reader-nav { display: flex; align-items: center; justify-content: space-between; gap: 14px; flex-wrap: wrap; margin-top: 36px; padding-top: 22px; border-top: 1px solid #e0e3d8; }
|
||||
.reader-page.has-lookup { max-width: 1120px; }
|
||||
.reader-workspace { display: grid; grid-template-columns: minmax(0, 1fr); gap: 32px; align-items: start; }
|
||||
.has-lookup .reader-workspace { grid-template-columns: minmax(0, 1fr) 320px; }
|
||||
.reader-body { min-width: 0; }
|
||||
.reader-word { cursor: pointer; border-radius: 3px; }
|
||||
/* Saved words keep one style per status everywhere they appear. */
|
||||
.reader-word.is-new { border-bottom: 2px dotted #bc803d; }
|
||||
.reader-word.is-learning { background: #f6e7c9; }
|
||||
.reader-word.is-known { background: #dee9d6; }
|
||||
.reader-word.is-ignored { color: #949c94; }
|
||||
.reader-word:hover, .reader-word.is-selected { background: #e3e9d9; color: #264a35; }
|
||||
.reader-word:focus-visible { outline: 2px solid #bc803d; outline-offset: 2px; background: #eef2eb; }
|
||||
.tokens-notice { color: #748073; font-size: 14px; margin-top: 24px; }
|
||||
.lookup-panel { position: sticky; top: 24px; margin-top: 26px; padding: 22px; border: 1px solid #d9decf; border-radius: 12px; background: #fffdf8; max-height: calc(100dvh - 48px); overflow-y: auto; overflow-wrap: anywhere; }
|
||||
.lookup-heading { display: flex; align-items: start; justify-content: space-between; gap: 10px; border-bottom: 1px solid #e0e3d8; padding-bottom: 12px; }
|
||||
.lookup-heading h2 { margin: 6px 0; font-family: Georgia, serif; font-size: 25px; }
|
||||
.lookup-content { font-size: 15px; line-height: 1.7; }
|
||||
.lookup-message { color: #8b4324; }
|
||||
.lookup-senses { padding-left: 22px; }
|
||||
.lookup-senses li { padding-left: 3px; margin-bottom: 20px; }
|
||||
.lookup-senses p { margin: 8px 0; }
|
||||
.sense-heading span { color: #748073; font-size: 13px; }
|
||||
.lookup-senses blockquote { border-left: 2px solid #cbd9c9; margin: 10px 0; padding-left: 12px; color: #687568; font-style: italic; }
|
||||
.lookup-term { border-top: 1px solid #e0e3d8; padding-top: 18px; margin-top: 20px; }
|
||||
.lookup-term label { display: flex; justify-content: space-between; gap: 10px; font-size: 14px; margin-top: 14px; }
|
||||
.lookup-term-title { display: flex; font-size: 14px; margin: 0; }
|
||||
.lookup-term label span { color: #8b794e; font-size: 12px; }
|
||||
.lookup-term .el-radio-group { margin-top: 8px; flex-wrap: wrap; gap: 4px 12px; }
|
||||
.lookup-term .el-textarea { margin-top: 8px; }
|
||||
.lookup-term .el-textarea textarea { font: inherit; line-height: 1.6; }
|
||||
.lookup-saved { color: #2f6b45; font-size: 14px; margin: 12px 0 0; }
|
||||
.lookup-actions { display: flex; gap: 8px; margin-top: 14px; flex-wrap: wrap; }
|
||||
.review-page { max-width: 680px; }
|
||||
.chapter-notice { margin: 12px 0 0; padding: 10px 14px; border: 1px solid #d9decf; border-radius: 8px; background: #fbf7ee; color: #6b5b3e; }
|
||||
.chapter-list .chapter-row { flex-wrap: wrap; }
|
||||
.review-notice { margin: 10px 0 0; padding: 10px 14px; border: 1px solid #d9decf; border-radius: 8px; background: #fbf7ee; color: #6b5b3e; }
|
||||
.review-card, .review-summary { margin-top: 26px; padding: 28px; border: 1px solid #d9decf; border-radius: 14px; background: #fffdf8; }
|
||||
.review-summary h2 { margin-top: 0; font-family: Georgia, serif; font-size: 24px; }
|
||||
.review-word { margin: 12px 0; font-family: Georgia, serif; font-size: 34px; }
|
||||
.review-example { margin: 12px 0; font-family: Georgia, serif; font-size: 19px; line-height: 1.8; }
|
||||
.review-answer { margin: 18px 0; padding-top: 18px; border-top: 1px solid #e0e3d8; }
|
||||
.review-definition { margin: 0 0 12px; font-size: 18px; }
|
||||
.review-answer blockquote { border-left: 2px solid #cbd9c9; margin: 10px 0; padding-left: 12px; color: #687568; font-style: italic; }
|
||||
.review-grades { display: flex; gap: 10px; flex-wrap: wrap; margin-top: 8px; }
|
||||
.review-actions { margin: 16px 0 0; }
|
||||
@media (max-width: 760px) {
|
||||
.login-page { grid-template-columns: 1fr; }
|
||||
.welcome { padding: 28px; }
|
||||
@@ -62,4 +151,19 @@ h1 { font-size: 30px; font-weight: 600; margin: 14px 0; letter-spacing: 1px; }
|
||||
.site-header nav { order: 3; flex-basis: 100%; padding-top: 8px; }
|
||||
.library { margin-top: 32px; padding: 0 20px; }
|
||||
h1 { font-size: 26px; }
|
||||
/* Single column, tap-friendly controls and no horizontal overflow. */
|
||||
.page { margin-top: 32px; padding: 0 20px; }
|
||||
.page-title { align-items: flex-start; }
|
||||
.library-title { align-items: flex-start; }
|
||||
.library-actions { width: 100%; justify-content: space-between; }
|
||||
.book-grid { grid-template-columns: 1fr; }
|
||||
.import-form { padding: 20px; }
|
||||
.chapter-row { grid-template-columns: 30px minmax(0, 1fr); align-items: start; row-gap: 10px; padding: 15px 16px; }
|
||||
.chapter-row .status-chip, .chapter-row .el-button { grid-column: 2; justify-self: start; }
|
||||
.reader-text { font-size: 16px; line-height: 1.95; }
|
||||
.has-lookup .reader-workspace { grid-template-columns: minmax(0, 1fr); gap: 20px; }
|
||||
.reader-page.has-lookup { padding-bottom: calc(45dvh + 28px); }
|
||||
.lookup-panel { position: fixed; inset: auto 0 0; z-index: 20; max-height: 45dvh; margin-top: 0; padding: 16px 20px max(20px, env(safe-area-inset-bottom)); border-radius: 16px 16px 0 0; box-shadow: 0 -5px 24px #233d3114; }
|
||||
.lookup-heading { position: sticky; top: -16px; z-index: 1; background: #fffdf8; }
|
||||
.reader-nav .el-button { flex: 1; }
|
||||
}
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
<script setup lang="ts">
|
||||
import { computed, onMounted, onUnmounted, ref, watch } from 'vue'
|
||||
import { RouterLink, useRoute, useRouter } from 'vue-router'
|
||||
import { ElButton, ElDialog, ElInput, ElMessageBox } from 'element-plus'
|
||||
import { canRetry, statusLabel, TEXT_MAX_CODE_POINTS, textProblem, titleProblem, useLibraryStore, type ChapterSource } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const session = useSessionStore()
|
||||
const library = useLibraryStore()
|
||||
const route = useRoute()
|
||||
const router = useRouter()
|
||||
const retryError = ref('')
|
||||
const notice = ref('')
|
||||
|
||||
// Renaming the book.
|
||||
const bookDialog = ref(false)
|
||||
const bookTitle = ref('')
|
||||
const bookError = ref('')
|
||||
// Editing one chapter: title and text, loaded from the source endpoint.
|
||||
const chapterDialog = ref(false)
|
||||
const chapterSource = ref<ChapterSource | null>(null)
|
||||
const chapterTitle = ref('')
|
||||
const chapterText = ref('')
|
||||
const chapterError = ref('')
|
||||
const chapterLoading = ref(false)
|
||||
const saving = ref(false)
|
||||
|
||||
const bookId = computed(() => Number(route.params.id))
|
||||
const chapterLength = computed(() => [...chapterText.value].length)
|
||||
|
||||
async function load() {
|
||||
retryError.value = ''
|
||||
await library.loadBook(bookId.value)
|
||||
}
|
||||
|
||||
async function retry(chapterId: number) {
|
||||
retryError.value = ''
|
||||
try { await library.retryChapter(chapterId) }
|
||||
catch (reason) { retryError.value = reason instanceof Error ? reason.message : '重试失败，请稍后重试。' }
|
||||
}
|
||||
|
||||
function openBookDialog() {
|
||||
bookTitle.value = library.book?.title ?? ''
|
||||
bookError.value = ''
|
||||
bookDialog.value = true
|
||||
}
|
||||
|
||||
async function saveBookTitle() {
|
||||
if (saving.value || library.book === null) return
|
||||
bookError.value = titleProblem(bookTitle.value)
|
||||
if (bookError.value) return
|
||||
saving.value = true
|
||||
try {
|
||||
await library.renameBook(library.book.id, bookTitle.value)
|
||||
bookDialog.value = false
|
||||
notice.value = '书名已更新。'
|
||||
} catch (reason) {
|
||||
bookError.value = reason instanceof Error ? reason.message : '保存失败，请稍后重试。'
|
||||
} finally {
|
||||
saving.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function openChapterDialog(chapterId: number) {
|
||||
chapterDialog.value = true
|
||||
chapterLoading.value = true
|
||||
chapterError.value = ''
|
||||
chapterSource.value = null
|
||||
try {
|
||||
const source = await library.loadChapterSource(chapterId)
|
||||
chapterSource.value = source
|
||||
chapterTitle.value = source.title
|
||||
chapterText.value = source.text
|
||||
} catch (reason) {
|
||||
chapterError.value = reason instanceof Error ? reason.message : '章节内容暂时无法加载，请稍后重试。'
|
||||
} finally {
|
||||
chapterLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function saveChapter() {
|
||||
const source = chapterSource.value
|
||||
if (saving.value || source === null) return
|
||||
chapterError.value = titleProblem(chapterTitle.value) || textProblem(chapterText.value)
|
||||
if (chapterError.value) return
|
||||
saving.value = true
|
||||
try {
|
||||
const edited = await library.updateChapter(source.id, { title: chapterTitle.value, text: chapterText.value })
|
||||
chapterDialog.value = false
|
||||
notice.value = edited.versionChanged ? '章节已保存为新版本，正在重新处理。' : '章节已保存。'
|
||||
} catch (reason) {
|
||||
chapterError.value = reason instanceof Error ? reason.message : '保存失败，请稍后重试。'
|
||||
} finally {
|
||||
saving.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function confirmDeleteBook() {
|
||||
if (library.book === null) return
|
||||
const id = library.book.id
|
||||
try {
|
||||
await ElMessageBox.confirm('删除这本书及其章节？已保存的生词和短语将保留。', '删除书籍', {
|
||||
confirmButtonText: '确认删除', cancelButtonText: '取消，保留书籍', type: 'warning',
|
||||
})
|
||||
} catch { return }
|
||||
try {
|
||||
const deleted = await library.deleteBook(id)
|
||||
await router.replace(`/?deleted=${deleted.chapters ?? 0}`)
|
||||
} catch (reason) {
|
||||
retryError.value = reason instanceof Error ? reason.message : '删除失败，请稍后重试。'
|
||||
}
|
||||
}
|
||||
|
||||
async function confirmDeleteChapter(chapterId: number, ordinal: number) {
|
||||
try {
|
||||
await ElMessageBox.confirm(`删除第 ${ordinal} 章？本章正文将被删除，已保存词条保留。`, '删除章节', {
|
||||
confirmButtonText: '确认删除章节', cancelButtonText: '取消', type: 'warning',
|
||||
})
|
||||
} catch { return }
|
||||
try {
|
||||
const deleted = await library.deleteChapter(chapterId)
|
||||
// The chapter no longer exists, so the dialog closes and the list reports the remainder.
|
||||
chapterDialog.value = false
|
||||
chapterSource.value = null
|
||||
notice.value = `章节已删除 · 剩余 ${deleted.remaining} 章`
|
||||
} catch (reason) {
|
||||
retryError.value = reason instanceof Error ? reason.message : '删除失败，请稍后重试。'
|
||||
}
|
||||
}
|
||||
|
||||
async function logout() {
|
||||
try { await session.logout() }
|
||||
catch { session.notice = '已退出此设备。服务器暂时无法连接，请稍后重试。' }
|
||||
finally { await router.replace('/login') }
|
||||
}
|
||||
|
||||
onMounted(load)
|
||||
watch(bookId, () => { void load() })
|
||||
// Leaving the page releases the book so polling stops.
|
||||
onUnmounted(() => library.closeBook())
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/">我的书库</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
<ElButton text @click="logout">退出登录</ElButton>
|
||||
</div>
|
||||
</header>
|
||||
<main class="page">
|
||||
<p class="breadcrumb"><RouterLink to="/">我的书库</RouterLink><span aria-hidden="true">/</span><span>{{ library.book?.title ?? '书籍' }}</span></p>
|
||||
<p v-if="library.bookLoading && !library.book" role="status" class="loading">正在加载…</p>
|
||||
<div v-else-if="library.bookError" class="notice">
|
||||
<p role="alert">{{ library.bookError }}</p>
|
||||
<ElButton @click="load">重试</ElButton>
|
||||
</div>
|
||||
<template v-else-if="library.book">
|
||||
<div class="page-title">
|
||||
<div>
|
||||
<h1>{{ library.book.title }}</h1>
|
||||
<p class="subtle">{{ library.chapters.length }} 个章节 · 语言 英语 · 封面使用系统默认样式</p>
|
||||
</div>
|
||||
<div class="page-actions">
|
||||
<ElButton data-testid="edit-book" @click="openBookDialog">编辑书名</ElButton>
|
||||
<RouterLink :to="`/import?book=${library.book.id}`" class="link-button">追加章节</RouterLink>
|
||||
<ElButton type="danger" plain data-testid="delete-book" @click="confirmDeleteBook">删除书籍</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
<p v-if="notice" role="status" class="chapter-notice" data-testid="book-notice">{{ notice }}</p>
|
||||
<p v-if="retryError" role="alert" class="notice">{{ retryError }}</p>
|
||||
<ul v-if="library.chapters.length" class="chapter-list" aria-label="章节列表">
|
||||
<li v-for="item in library.chapters" :key="item.id" class="chapter-row">
|
||||
<span class="chapter-ordinal">{{ item.ordinal }}</span>
|
||||
<div class="chapter-info">
|
||||
<RouterLink v-if="item.status === 'ready'" :to="`/chapters/${item.id}`" class="chapter-name">{{ item.title }}</RouterLink>
|
||||
<span v-else class="chapter-name">{{ item.title }}</span>
|
||||
<p class="chapter-meta">
|
||||
{{ item.charCount }} 字符
|
||||
<template v-if="item.status === 'failed' && item.errorMessage"> · {{ item.errorMessage }}</template>
|
||||
</p>
|
||||
</div>
|
||||
<span class="status-chip" :class="`status-${item.status}`">{{ statusLabel(item.status) }}</span>
|
||||
<ElButton
|
||||
v-if="canRetry(item)"
|
||||
size="small"
|
||||
:loading="library.retryingChapterId === item.id"
|
||||
@click="retry(item.id)"
|
||||
>重试</ElButton>
|
||||
<ElButton size="small" :data-testid="`edit-chapter-${item.id}`" @click="openChapterDialog(item.id)">编辑</ElButton>
|
||||
</li>
|
||||
</ul>
|
||||
<section v-else class="empty-library" aria-label="章节列表" data-testid="empty-chapters">
|
||||
<h2>这一本书还没有章节</h2>
|
||||
<p class="subtle">粘贴一段英文即可生成第一章。</p>
|
||||
</section>
|
||||
</template>
|
||||
|
||||
<ElDialog v-model="bookDialog" title="编辑书名" width="420" data-testid="book-dialog">
|
||||
<label for="book-title">书名</label>
|
||||
<ElInput id="book-title" v-model="bookTitle" type="text" maxlength="200" :disabled="saving" />
|
||||
<p class="subtle">封面使用系统默认样式。</p>
|
||||
<p v-if="bookError" role="alert" class="field-error">{{ bookError }}</p>
|
||||
<template #footer>
|
||||
<ElButton @click="bookDialog = false">取消</ElButton>
|
||||
<ElButton type="primary" :loading="saving" data-testid="save-book" @click="saveBookTitle">保存修改</ElButton>
|
||||
</template>
|
||||
</ElDialog>
|
||||
|
||||
<ElDialog v-model="chapterDialog" title="编辑章节" width="620" data-testid="chapter-dialog">
|
||||
<p v-if="chapterLoading" role="status" class="loading">正在加载…</p>
|
||||
<template v-else-if="chapterSource">
|
||||
<label for="chapter-title">章节标题</label>
|
||||
<ElInput id="chapter-title" v-model="chapterTitle" type="text" maxlength="200" :disabled="saving" />
|
||||
<label for="chapter-text">正文</label>
|
||||
<ElInput id="chapter-text" v-model="chapterText" type="textarea" :rows="12" :disabled="saving" />
|
||||
<p class="counter">{{ chapterLength }} / {{ TEXT_MAX_CODE_POINTS }} 字符</p>
|
||||
<p class="subtle">保存后会重新处理这一章；只改标题不会重新处理。</p>
|
||||
</template>
|
||||
<p v-if="chapterError" role="alert" class="field-error">{{ chapterError }}</p>
|
||||
<template #footer>
|
||||
<ElButton data-testid="cancel-chapter" @click="chapterDialog = false">取消</ElButton>
|
||||
<ElButton
|
||||
v-if="chapterSource"
|
||||
type="danger"
|
||||
plain
|
||||
:disabled="saving"
|
||||
data-testid="delete-chapter"
|
||||
@click="confirmDeleteChapter(chapterSource.id, chapterSource.ordinal)"
|
||||
>删除章节</ElButton>
|
||||
<ElButton type="primary" :loading="saving" :disabled="chapterLoading || !chapterSource" data-testid="save-chapter" @click="saveChapter">保存章节</ElButton>
|
||||
</template>
|
||||
</ElDialog>
|
||||
</main>
|
||||
</div>
|
||||
</template>
|
||||
@@ -0,0 +1,196 @@
|
||||
<script setup lang="ts">
|
||||
import { computed, onMounted, onUnmounted, ref, watch } from 'vue'
|
||||
import { RouterLink, useRoute, useRouter } from 'vue-router'
|
||||
import { ElButton, ElInput, ElOption, ElRadio, ElRadioGroup, ElSelect } from 'element-plus'
|
||||
import { LANGUAGE_LABEL, TEXT_MAX_CODE_POINTS, fileProblem, fileSizeLabel, textProblem, titleProblem, useLibraryStore, type SubmitTarget } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const session = useSessionStore()
|
||||
const library = useLibraryStore()
|
||||
const route = useRoute()
|
||||
const router = useRouter()
|
||||
|
||||
const title = ref('')
|
||||
const text = ref('')
|
||||
const mode = ref<'new' | 'append'>('new')
|
||||
const bookId = ref<number | undefined>(undefined)
|
||||
// The accepted prototype offers both sources on one screen; the server treats them the same.
|
||||
const source = ref<'paste' | 'txt'>('paste')
|
||||
const file = ref<File | null>(null)
|
||||
const fileInfo = ref<{ name: string; encoding: string; size: string } | null>(null)
|
||||
const titleError = ref('')
|
||||
const textError = ref('')
|
||||
const fileError = ref('')
|
||||
const bookError = ref('')
|
||||
|
||||
const length = computed(() => [...text.value].length)
|
||||
const busy = computed(() => library.submitting)
|
||||
|
||||
/** The browser pre-check only replaces the server, it never replaces its verdict. */
|
||||
async function readFile(selected: File): Promise<void> {
|
||||
fileError.value = ''
|
||||
fileInfo.value = null
|
||||
const problem = fileProblem(selected)
|
||||
if (problem) {
|
||||
file.value = null
|
||||
fileError.value = problem
|
||||
return
|
||||
}
|
||||
try {
|
||||
const decoder = new TextDecoder('utf-8', { fatal: true })
|
||||
decoder.decode(await selected.arrayBuffer())
|
||||
} catch {
|
||||
file.value = null
|
||||
fileError.value = '文件不是 UTF-8 编码，请另存为 UTF-8 后重试。'
|
||||
return
|
||||
}
|
||||
file.value = selected
|
||||
fileInfo.value = { name: selected.name, encoding: 'UTF-8', size: fileSizeLabel(selected.size) }
|
||||
// A file usually defines the title; the learner can still change it before submitting.
|
||||
if (!title.value.trim()) title.value = selected.name.replace(/\.txt$/i, '').slice(0, 120)
|
||||
}
|
||||
|
||||
function onFile(event: Event): void {
|
||||
const selected = (event.target as HTMLInputElement).files?.[0]
|
||||
file.value = null
|
||||
fileInfo.value = null
|
||||
fileError.value = ''
|
||||
if (selected) void readFile(selected)
|
||||
}
|
||||
|
||||
function requestedBookId(): number | undefined {
|
||||
const raw = Array.isArray(route.query.book) ? route.query.book[0] : route.query.book
|
||||
if (typeof raw !== 'string' || !/^\d+$/.test(raw)) return undefined
|
||||
const value = Number(raw)
|
||||
return Number.isSafeInteger(value) && value > 0 ? value : undefined
|
||||
}
|
||||
|
||||
onMounted(() => {
|
||||
// `?book=<id>` preselects "append to an existing book".
|
||||
const preselect = requestedBookId()
|
||||
if (preselect === undefined) return
|
||||
mode.value = 'append'
|
||||
bookId.value = preselect
|
||||
})
|
||||
|
||||
watch(mode, value => {
|
||||
// The select can only list the caller's own books.
|
||||
if (value === 'append') void library.loadBooks()
|
||||
})
|
||||
|
||||
watch(title, () => { titleError.value = '' })
|
||||
watch(text, () => { textError.value = '' })
|
||||
// Switching source clears the other source's complaints and its result.
|
||||
watch(source, () => {
|
||||
titleError.value = ''
|
||||
textError.value = ''
|
||||
fileError.value = ''
|
||||
library.submitError = ''
|
||||
})
|
||||
|
||||
async function submit() {
|
||||
if (busy.value) return
|
||||
titleError.value = titleProblem(title.value)
|
||||
textError.value = source.value === 'paste' ? textProblem(text.value) : ''
|
||||
fileError.value = source.value === 'txt' && !file.value ? (fileError.value || '请选择要导入的 TXT 文件。') : fileError.value
|
||||
bookError.value = mode.value === 'append' && bookId.value === undefined ? '请选择要追加的书籍。' : ''
|
||||
if (titleError.value || textError.value || fileError.value || bookError.value) return
|
||||
const target: SubmitTarget = mode.value === 'append' && bookId.value !== undefined
|
||||
? { mode: 'append', bookId: bookId.value }
|
||||
: { mode: 'new' }
|
||||
try {
|
||||
const createdBookId = source.value === 'txt' && file.value
|
||||
? await library.upload({ title: title.value, target, file: file.value })
|
||||
: await library.submit({ title: title.value, text: text.value, target })
|
||||
// A response that arrives after the user left this page must not navigate them back.
|
||||
if (disposed) return
|
||||
// The requestId was consumed by this submission, so the form starts clean.
|
||||
title.value = ''
|
||||
text.value = ''
|
||||
file.value = null
|
||||
fileInfo.value = null
|
||||
await router.replace(`/books/${createdBookId}`)
|
||||
} catch {
|
||||
// library.submitError already carries the server message for the template.
|
||||
}
|
||||
}
|
||||
|
||||
// The form may still be awaiting its submit when the user navigates away.
|
||||
let disposed = false
|
||||
onUnmounted(() => {
|
||||
disposed = true
|
||||
library.submitError = ''
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/">我的书库</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
</div>
|
||||
</header>
|
||||
<main class="page">
|
||||
<p class="breadcrumb"><RouterLink to="/">我的书库</RouterLink><span aria-hidden="true">/</span><span>导入内容</span></p>
|
||||
<div class="page-title">
|
||||
<div>
|
||||
<h1>导入英文内容</h1>
|
||||
<p class="subtle">粘贴英文正文，提交后系统会自动切分并处理章节。</p>
|
||||
</div>
|
||||
</div>
|
||||
<form class="import-form" novalidate @submit.prevent="submit">
|
||||
<div class="field">
|
||||
<span class="field-label">导入方式</span>
|
||||
<ElRadioGroup v-model="source" :disabled="busy" aria-label="导入方式">
|
||||
<ElRadio value="paste">粘贴文本</ElRadio>
|
||||
<ElRadio value="txt">TXT 文件</ElRadio>
|
||||
</ElRadioGroup>
|
||||
</div>
|
||||
<div class="field">
|
||||
<span class="field-label">语言</span>
|
||||
<p class="fixed-value">{{ LANGUAGE_LABEL }}</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="title">标题</label>
|
||||
<ElInput id="title" v-model="title" type="text" maxlength="200" placeholder="例如：虚构样例第一章" :disabled="busy" />
|
||||
<p v-if="titleError" role="alert" class="field-error">{{ titleError }}</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<span class="field-label">导入到</span>
|
||||
<ElRadioGroup v-model="mode" :disabled="busy" aria-label="导入目标">
|
||||
<ElRadio value="new">新建书籍</ElRadio>
|
||||
<ElRadio value="append">追加到已有书籍</ElRadio>
|
||||
</ElRadioGroup>
|
||||
</div>
|
||||
<div v-if="mode === 'append'" class="field">
|
||||
<label for="book">选择书籍</label>
|
||||
<ElSelect id="book" v-model="bookId" placeholder="请选择要追加的书籍" :disabled="busy" class="book-select">
|
||||
<ElOption v-for="item in library.books" :key="item.id" :label="item.title" :value="item.id" />
|
||||
</ElSelect>
|
||||
<p v-if="library.booksError" role="alert" class="field-error">{{ library.booksError }}</p>
|
||||
<p v-if="bookError" role="alert" class="field-error">{{ bookError }}</p>
|
||||
</div>
|
||||
<div v-if="source === 'paste'" class="field">
|
||||
<label for="text">正文</label>
|
||||
<ElInput id="text" v-model="text" type="textarea" :rows="12" placeholder="在此粘贴英文正文…" :disabled="busy" />
|
||||
<p class="counter">{{ length }} / {{ TEXT_MAX_CODE_POINTS }} 字符</p>
|
||||
<p v-if="textError" role="alert" class="field-error">{{ textError }}</p>
|
||||
</div>
|
||||
<div v-else class="field">
|
||||
<label for="file">TXT 文件</label>
|
||||
<input id="file" data-testid="file-input" type="file" accept=".txt,text/plain" :disabled="busy" @change="onFile" />
|
||||
<p v-if="fileInfo" class="counter" data-testid="file-info">{{ fileInfo.name }} · {{ fileInfo.encoding }} · {{ fileInfo.size }}</p>
|
||||
<p v-else class="subtle">仅支持 UTF-8 的 .txt 文件，最多 2 MiB；文件不会保存在服务器上。</p>
|
||||
<p v-if="fileError" role="alert" class="field-error">{{ fileError }}</p>
|
||||
</div>
|
||||
<p v-if="library.submitError" role="alert" class="notice">{{ library.submitError }}</p>
|
||||
<div class="form-actions">
|
||||
<ElButton type="primary" native-type="submit" :loading="busy" :disabled="busy">{{ source === 'txt' ? '上传并处理' : '开始处理' }}</ElButton>
|
||||
<RouterLink to="/" class="subtle">返回书库</RouterLink>
|
||||
</div>
|
||||
</form>
|
||||
</main>
|
||||
</div>
|
||||
</template>
|
||||
@@ -1,19 +1,30 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted, ref } from 'vue'
|
||||
import { RouterLink, useRouter } from 'vue-router'
|
||||
import { computed, onMounted, ref } from 'vue'
|
||||
import { RouterLink, useRoute, useRouter } from 'vue-router'
|
||||
import { ElButton } from 'element-plus'
|
||||
import BookMark from '../components/BookMark.vue'
|
||||
import { statusSummary, useLibraryStore } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
const session = useSessionStore()
|
||||
const library = useLibraryStore()
|
||||
const router = useRouter()
|
||||
const route = useRoute()
|
||||
const loading = ref(true)
|
||||
const error = ref('')
|
||||
// A deletion returns here with the count it removed, so the learner sees what happened.
|
||||
const deletedNotice = computed(() => {
|
||||
const raw = Array.isArray(route.query.deleted) ? route.query.deleted[0] : route.query.deleted
|
||||
if (typeof raw !== 'string' || !/^\d+$/.test(raw)) return ''
|
||||
return '书籍已删除 · 已保存的生词和短语仍保留在生词本。'
|
||||
})
|
||||
async function load() {
|
||||
loading.value = true
|
||||
error.value = ''
|
||||
try { await session.loadSpace() }
|
||||
catch (reason) { error.value = reason instanceof Error ? reason.message : '暂时无法加载，请重试。' }
|
||||
finally { loading.value = false }
|
||||
// The book list keeps its own error so one failing call still shows a retry.
|
||||
if (!error.value) await library.loadBooks()
|
||||
loading.value = false
|
||||
}
|
||||
async function logout() {
|
||||
try { await session.logout() }
|
||||
@@ -27,18 +38,36 @@ onMounted(load)
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/" class="active-nav">我的书库</RouterLink></nav>
|
||||
<nav aria-label="学习导航"><RouterLink to="/" class="active-nav">我的书库</RouterLink> · <RouterLink to="/review">到期复习</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
<ElButton text @click="logout">退出登录</ElButton>
|
||||
</div>
|
||||
</header>
|
||||
<main class="library">
|
||||
<div class="library-title"><div><h1>我的书库</h1><p class="subtle">你的阅读与学习，从这里开始。</p></div><span class="language">英语</span></div>
|
||||
<div class="library-title">
|
||||
<div><h1>我的书库</h1><p class="subtle">你的阅读与学习，从这里开始。</p></div>
|
||||
<div class="library-actions">
|
||||
<span class="language">英语</span>
|
||||
<ElButton type="primary" @click="router.push('/import')">导入内容</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
<p v-if="deletedNotice" role="status" class="chapter-notice" data-testid="library-notice">{{ deletedNotice }}</p>
|
||||
<p v-if="loading" role="status" class="loading">正在加载…</p>
|
||||
<div v-else-if="error" class="notice"><p role="alert">{{ error }}</p><ElButton @click="load">重试</ElButton></div>
|
||||
<section v-else-if="session.space" class="empty-library" aria-label="书库内容">
|
||||
<BookMark /><h2>书库还是空的</h2><p class="subtle">这里将收纳你的阅读内容。</p>
|
||||
<div v-else-if="error || library.booksError" class="notice">
|
||||
<p role="alert">{{ error || library.booksError }}</p>
|
||||
<ElButton @click="load">重试</ElButton>
|
||||
</div>
|
||||
<ul v-else-if="library.books.length" class="book-grid" aria-label="书籍列表">
|
||||
<li v-for="item in library.books" :key="item.id" class="book-card">
|
||||
<RouterLink :to="`/books/${item.id}`" class="book-title">{{ item.title }}</RouterLink>
|
||||
<p class="subtle">{{ item.chapterCount }} 个章节</p>
|
||||
<p v-if="statusSummary(item)" class="status-summary">{{ statusSummary(item) }}</p>
|
||||
<p v-else class="subtle">尚未导入章节</p>
|
||||
</li>
|
||||
</ul>
|
||||
<section v-else class="empty-library" aria-label="书库内容">
|
||||
<BookMark /><h2>书库还是空的</h2><p class="subtle">粘贴一段英文，开始你的第一篇阅读。</p>
|
||||
</section>
|
||||
</main>
|
||||
</div>
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
<script setup lang="ts">
|
||||
import { computed, onMounted, onUnmounted, ref, watch } from 'vue'
|
||||
import { RouterLink, useRoute, useRouter } from 'vue-router'
|
||||
import { ElButton } from 'element-plus'
|
||||
import { canRetry, statusLabel, useLibraryStore } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
import { useReaderLookup } from '../composables/useReaderLookup'
|
||||
import ReaderTokens from '../components/ReaderTokens.vue'
|
||||
import LookupPanel from '../components/LookupPanel.vue'
|
||||
|
||||
const session = useSessionStore()
|
||||
const library = useLibraryStore()
|
||||
const route = useRoute()
|
||||
const router = useRouter()
|
||||
const retryError = ref('')
|
||||
|
||||
const chapterId = computed(() => Number(route.params.id))
|
||||
const chapter = computed(() => library.chapter)
|
||||
const lookup = useReaderLookup(chapter)
|
||||
// Retry uses the job id the chapter carries, no matter where it was loaded from.
|
||||
const retryable = computed(() => chapter.value !== null && canRetry(chapter.value))
|
||||
|
||||
async function load() {
|
||||
lookup.reset()
|
||||
retryError.value = ''
|
||||
await library.loadChapter(chapterId.value)
|
||||
}
|
||||
|
||||
async function retry() {
|
||||
if (chapter.value === null) return
|
||||
retryError.value = ''
|
||||
try { await library.retryChapter(chapter.value.id) }
|
||||
catch (reason) { retryError.value = reason instanceof Error ? reason.message : '重试失败，请稍后重试。' }
|
||||
}
|
||||
|
||||
// Switching chapters reuses this component; only the route param changes.
|
||||
function goTo(id: number | null) {
|
||||
if (id === null) return
|
||||
void router.push(`/chapters/${id}`)
|
||||
}
|
||||
|
||||
async function logout() {
|
||||
try { await session.logout() }
|
||||
catch { session.notice = '已退出此设备。服务器暂时无法连接，请稍后重试。' }
|
||||
finally { await router.replace('/login') }
|
||||
}
|
||||
|
||||
onMounted(load)
|
||||
watch(chapterId, () => { void load() })
|
||||
// Leaving the page releases the chapter so polling stops.
|
||||
onUnmounted(() => library.closeChapter())
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/">我的书库</RouterLink> · <RouterLink to="/review">到期复习</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
<ElButton text @click="logout">退出登录</ElButton>
|
||||
</div>
|
||||
</header>
|
||||
<main class="page reader-page" :class="{ 'has-lookup': lookup.selected.value }" @keydown.esc="lookup.close()">
|
||||
<p v-if="library.chapterLoading && !chapter" role="status" class="loading">正在加载…</p>
|
||||
<div v-else-if="library.chapterError" class="notice">
|
||||
<p role="alert">{{ library.chapterError }}</p>
|
||||
<ElButton @click="load">重试</ElButton>
|
||||
</div>
|
||||
<template v-else-if="chapter">
|
||||
<p class="breadcrumb">
|
||||
<RouterLink to="/">我的书库</RouterLink>
|
||||
<span aria-hidden="true">/</span>
|
||||
<RouterLink v-if="library.chapterBook" :to="`/books/${library.chapterBook.id}`">{{ library.chapterBook.title }}</RouterLink>
|
||||
<span v-else>章节</span>
|
||||
</p>
|
||||
<div class="page-title">
|
||||
<h1>{{ chapter.title }}</h1>
|
||||
<span class="status-chip" :class="`status-${chapter.status}`">{{ statusLabel(chapter.status) }}</span>
|
||||
</div>
|
||||
<div v-if="chapter.status === 'failed'" class="notice">
|
||||
<p role="alert">{{ chapter.errorMessage || '这一章处理失败。' }}</p>
|
||||
<ElButton v-if="retryable" :loading="library.retryingChapterId === chapter.id" @click="retry">重试处理</ElButton>
|
||||
<p v-else class="subtle">这一章暂时没有可重试的任务编号。</p>
|
||||
</div>
|
||||
<p v-else-if="chapter.status !== 'ready'" role="status" class="processing-hint">这一章还在{{ statusLabel(chapter.status) }}，页面会自动刷新。</p>
|
||||
<p v-if="retryError" role="alert" class="notice">{{ retryError }}</p>
|
||||
<div v-if="chapter.status === 'ready'" class="reader-workspace">
|
||||
<div class="reader-body">
|
||||
<ReaderTokens :tokens="lookup.tokens.value" :original="library.readerText" :selected-start="lookup.selected.value?.start" @select="lookup.select" />
|
||||
<div v-if="lookup.tokensError.value" class="tokens-notice">
|
||||
<p role="status">{{ lookup.tokensError.value }}</p>
|
||||
<ElButton data-testid="tokens-retry" :loading="lookup.tokensLoading.value" @click="lookup.loadTokens">重试加载单词</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
<LookupPanel v-if="lookup.selected.value" :word="lookup.selected.value.text" :result="lookup.result.value" :loading="lookup.loading.value" :error="lookup.error.value" :saving="lookup.saving.value" :save-error="lookup.saveError.value" :saved="lookup.saved.value" :saved-term-id="lookup.savedTermId.value" :prefilling="lookup.prefilling.value" :prefill-error="lookup.prefillError.value" :can-save="lookup.canSave.value" v-model:definition="lookup.definition.value" v-model:examples="lookup.examples.value" v-model:status="lookup.status.value" @close="lookup.close()" @retry="lookup.lookup" @save="lookup.save" @resize="lookup.keepSelectionVisible" />
|
||||
</div>
|
||||
<nav class="reader-nav" aria-label="章节切换">
|
||||
<ElButton :disabled="library.navigation.previousChapterId === null" @click="goTo(library.navigation.previousChapterId)">上一章</ElButton>
|
||||
<ElButton :disabled="library.navigation.nextChapterId === null" @click="goTo(library.navigation.nextChapterId)">下一章</ElButton>
|
||||
</nav>
|
||||
</template>
|
||||
</main>
|
||||
</div>
|
||||
</template>
|
||||
@@ -0,0 +1,78 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted } from 'vue'
|
||||
import { RouterLink, useRouter } from 'vue-router'
|
||||
import { ElButton } from 'element-plus'
|
||||
import ReviewCard from '../components/ReviewCard.vue'
|
||||
import { useReviewStore, type ReviewItem } from '../stores/review'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const session = useSessionStore()
|
||||
const review = useReviewStore()
|
||||
const router = useRouter()
|
||||
|
||||
function endReview() { void router.push('/') }
|
||||
// Retrying reuses the grade of the failed attempt; the store keeps the answer id.
|
||||
let lastGrade: 'correct' | 'wrong' | 'again' = 'correct'
|
||||
function grade(value: 'correct' | 'wrong' | 'again') { lastGrade = value; void review.answer(value) }
|
||||
function retry() { void review.answer(lastGrade) }
|
||||
|
||||
onMounted(() => { void review.load() })
|
||||
async function logout() {
|
||||
try { await session.logout() }
|
||||
catch { session.notice = '已退出此设备。服务器暂时无法连接，请稍后重试。' }
|
||||
finally { await router.replace('/login') }
|
||||
}
|
||||
// The template guards for a card before rendering it; this keeps the prop type strict.
|
||||
const requireItem = (value: ReviewItem | null): ReviewItem => value as ReviewItem
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/">我的书库</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
<ElButton text @click="logout">退出登录</ElButton>
|
||||
</div>
|
||||
</header>
|
||||
<main class="page review-page">
|
||||
<h1>到期复习</h1>
|
||||
<p v-if="review.notice" role="status" class="review-notice" data-testid="review-notice">{{ review.notice }}</p>
|
||||
<p v-if="review.loading" role="status" class="loading">正在加载…</p>
|
||||
<div v-else-if="review.error && !review.current" class="notice">
|
||||
<p role="alert">{{ review.error }}</p>
|
||||
<ElButton data-testid="review-reload" @click="review.load()">重试</ElButton>
|
||||
</div>
|
||||
<section v-else-if="review.finished" class="review-summary" data-testid="review-summary">
|
||||
<h2>本次复习完成</h2>
|
||||
<p v-if="review.answered">复习了 {{ review.wordsReviewed }} 个词条 · 共 {{ review.answered }} 次作答</p>
|
||||
<p v-else class="subtle">本轮没有新的计分：{{ review.resolved }} 个词条已在其他页面复习。</p>
|
||||
<p class="subtle">答对 {{ review.correctCount }} · 答错或再学 {{ review.wrongCount }} · 已更新复习计划</p>
|
||||
<p v-if="review.pending" role="status" class="notice">还有 {{ review.pending }} 个词条到期。</p>
|
||||
<div class="review-grades">
|
||||
<ElButton v-if="review.pending" type="primary" data-testid="review-more" @click="review.continueRound()">继续复习</ElButton>
|
||||
<ElButton data-testid="review-finish" @click="endReview">继续阅读</ElButton>
|
||||
</div>
|
||||
</section>
|
||||
<section v-else-if="review.empty" class="review-summary" data-testid="review-empty">
|
||||
<h2>今天没有到期词条</h2>
|
||||
<p class="subtle">读一篇文章，积累下一次的词汇。</p>
|
||||
<ElButton data-testid="review-finish" @click="endReview">返回书库</ElButton>
|
||||
</section>
|
||||
<ReviewCard
|
||||
v-else-if="review.current"
|
||||
:item="requireItem(review.current)"
|
||||
:position="review.answered + 1"
|
||||
:total="review.answered + review.queue.length"
|
||||
:revealed="review.revealed"
|
||||
:busy="review.busy"
|
||||
:error="review.error"
|
||||
@reveal="review.reveal()"
|
||||
@grade="grade"
|
||||
@end="endReview"
|
||||
@retry="retry"
|
||||
/>
|
||||
</main>
|
||||
</div>
|
||||
</template>
|
||||
@@ -0,0 +1,31 @@
|
||||
WordNet Release 3.0
|
||||
|
||||
This software and database is being provided to you, the LICENSEE, by
|
||||
Princeton University under the following license. By obtaining, using
|
||||
and/or copying this software and database, you agree that you have
|
||||
read, understood, and will comply with these terms and conditions.:
|
||||
|
||||
Permission to use, copy, modify and distribute this software and
|
||||
database and its documentation for any purpose and without fee or
|
||||
royalty is hereby granted, provided that you agree to comply with
|
||||
the following copyright notice and statements, including the disclaimer,
|
||||
and that the same appear on ALL copies of the software, database and
|
||||
documentation, including modifications that you make for internal
|
||||
use or for distribution.
|
||||
|
||||
WordNet 3.0 Copyright 2006 by Princeton University. All rights reserved.
|
||||
|
||||
THIS SOFTWARE AND DATABASE IS PROVIDED "AS IS" AND PRINCETON
|
||||
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
|
||||
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PRINCETON
|
||||
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES OF MERCHANT-
|
||||
ABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE
|
||||
OF THE LICENSED SOFTWARE, DATABASE OR DOCUMENTATION WILL NOT
|
||||
INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR
|
||||
OTHER RIGHTS.
|
||||
|
||||
The name of Princeton University or Princeton may not be used in
|
||||
advertising or publicity pertaining to distribution of the software
|
||||
and/or database. Title to copyright in this software, database and
|
||||
any associated documentation shall at all times remain with
|
||||
Princeton University and LICENSEE agrees to preserve same.
|
||||
+163
-39
@@ -4,10 +4,11 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Migrate takes a connection-scoped lock. Only an empty or LexGo-owned schema is accepted.
|
||||
@@ -59,44 +60,26 @@ func Migrate(db *gorm.DB) error {
|
||||
if err = conn.QueryRowContext(ctx, "SELECT version,product FROM lexgo_schema WHERE id=1").Scan(¤t, &product); err != nil {
|
||||
return err
|
||||
}
|
||||
if product != "lexgo" || current < 0 || current > 2 {
|
||||
if product != "lexgo" || current < 0 || current > SchemaVersion {
|
||||
return errors.New("unknown schema version")
|
||||
}
|
||||
if current == 2 {
|
||||
return nil
|
||||
// Each known version contributes its own statements; the version row advances only
|
||||
// after every statement succeeded, so a partially applied migration can be retried.
|
||||
statements := make([]string, 0, 16)
|
||||
if current < 2 {
|
||||
statements = append(statements, schemaV2Statements...)
|
||||
}
|
||||
statements := []string{
|
||||
`CREATE TABLE IF NOT EXISTS sys_user (
|
||||
user_id BIGINT PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL UNIQUE,
|
||||
password VARCHAR(128) NOT NULL, nick_name VARCHAR(128) NOT NULL DEFAULT '', phone VARCHAR(11) NOT NULL DEFAULT '',
|
||||
role_id INT NOT NULL, salt VARCHAR(255) NOT NULL DEFAULT '', avatar VARCHAR(255) NOT NULL DEFAULT '',
|
||||
sex VARCHAR(255) NOT NULL DEFAULT '', email VARCHAR(128) NOT NULL DEFAULT '', dept_id BIGINT NOT NULL DEFAULT 0,
|
||||
post_id BIGINT NOT NULL DEFAULT 0, remark VARCHAR(255) NOT NULL DEFAULT '', status VARCHAR(4) NOT NULL DEFAULT '2',
|
||||
create_by BIGINT NOT NULL DEFAULT 0, update_by BIGINT NOT NULL DEFAULT 0,
|
||||
created_at DATETIME(3) NULL, updated_at DATETIME(3) NULL, deleted_at DATETIME(3) NULL,
|
||||
CHECK (role_id IN (1,2)), CHECK (status IN ('1','2'))
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_spaces (
|
||||
owner_id BIGINT PRIMARY KEY, language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_sessions (
|
||||
token_hash CHAR(64) CHARACTER SET ascii COLLATE ascii_bin PRIMARY KEY,
|
||||
owner_id BIGINT NOT NULL, expires_at DATETIME(3) NOT NULL,
|
||||
INDEX (owner_id), INDEX (expires_at),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_login_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL, ip VARCHAR(45) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_operation_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, actor_id BIGINT NOT NULL, actor_username VARCHAR(32) NOT NULL,
|
||||
target_id BIGINT NOT NULL DEFAULT 0, target_username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
action VARCHAR(32) NOT NULL, result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL,
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(actor_username,created_at), INDEX(target_username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
if current < 3 {
|
||||
statements = append(statements, schemaV3Statements...)
|
||||
}
|
||||
if current < 4 {
|
||||
statements = append(statements, schemaV4Statements...)
|
||||
}
|
||||
if current < 5 {
|
||||
statements = append(statements, schemaV5Statements...)
|
||||
}
|
||||
if current < 6 {
|
||||
statements = append(statements, schemaV6Statements...)
|
||||
}
|
||||
for i, s := range statements {
|
||||
if _, err = conn.ExecContext(ctx, s); err != nil {
|
||||
@@ -107,16 +90,157 @@ func Migrate(db *gorm.DB) error {
|
||||
return fmt.Errorf("migration statement %d failed", i+1)
|
||||
}
|
||||
}
|
||||
_, err = conn.ExecContext(ctx, "UPDATE lexgo_schema SET version=2 WHERE id=1")
|
||||
_, err = conn.ExecContext(ctx, fmt.Sprintf("UPDATE lexgo_schema SET version=%d WHERE id=1", SchemaVersion))
|
||||
return err
|
||||
}
|
||||
|
||||
// SchemaVersion is the version an explicit migration leaves behind, and the
|
||||
// version the server requires before it starts.
|
||||
const SchemaVersion = 6
|
||||
|
||||
// v6 adds review scheduling as its own table instead of altering lexgo_terms: every
|
||||
// statement stays additive and therefore retry-safe after a partial migration, and a
|
||||
// binary restored to v5 keeps writing personal terms unchanged. Existing saved words
|
||||
// enter the queue immediately, because a saved word has never been reviewed.
|
||||
var schemaV6Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_term_reviews (
|
||||
term_id BIGINT UNSIGNED PRIMARY KEY,
|
||||
owner_id BIGINT NOT NULL, language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
due_at DATETIME(3) NOT NULL, review_count INT NOT NULL DEFAULT 0,
|
||||
correct_count INT NOT NULL DEFAULT 0, wrong_count INT NOT NULL DEFAULT 0,
|
||||
last_reviewed_at DATETIME(3) NULL,
|
||||
INDEX idx_term_review_due (owner_id, language, due_at, term_id),
|
||||
FOREIGN KEY (term_id) REFERENCES lexgo_terms(id) ON DELETE CASCADE,
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`INSERT IGNORE INTO lexgo_term_reviews (term_id, owner_id, language, due_at)
|
||||
SELECT id, owner_id, language, created_at FROM lexgo_terms`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_review_answers (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT,
|
||||
owner_id BIGINT NOT NULL,
|
||||
answer_key CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
|
||||
term_id BIGINT UNSIGNED NOT NULL, grade VARCHAR(16) NOT NULL, result VARCHAR(16) NOT NULL,
|
||||
status_before VARCHAR(16) NOT NULL, status_after VARCHAR(16) NOT NULL,
|
||||
level_before TINYINT NOT NULL, level_after TINYINT NOT NULL,
|
||||
due_at_before DATETIME(3) NOT NULL, due_at_after DATETIME(3) NOT NULL,
|
||||
requeued BOOLEAN NOT NULL DEFAULT FALSE, created_at DATETIME(3) NOT NULL,
|
||||
UNIQUE KEY uq_review_answer (owner_id, answer_key),
|
||||
INDEX idx_review_answer_owner (owner_id, created_at, id), INDEX idx_review_answer_term (term_id, created_at),
|
||||
CHECK (grade IN ('correct','wrong','again')), CHECK (result IN ('applied','stale')),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY (term_id) REFERENCES lexgo_terms(id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
// v5 adds one learner's own records for a word form. Identity is the normalized
|
||||
// form under the owner's language, so re-saving the same word updates one row
|
||||
// instead of creating a second, conflicting record. utf8mb4_bin keeps the key
|
||||
// byte-exact: the Go side normalizes, and an accent-insensitive collation must
|
||||
// not fold "resume" and "résumé" into the same entry.
|
||||
var schemaV5Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_terms (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, owner_id BIGINT NOT NULL,
|
||||
language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
term VARCHAR(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
|
||||
original_form VARCHAR(191) NOT NULL,
|
||||
definition TEXT NOT NULL, examples TEXT NOT NULL,
|
||||
status VARCHAR(16) NOT NULL, level TINYINT NOT NULL DEFAULT 0,
|
||||
created_at DATETIME(3) NOT NULL, updated_at DATETIME(3) NOT NULL,
|
||||
UNIQUE KEY uq_term_identity (owner_id, language, term),
|
||||
CONSTRAINT ck_term_status CHECK (status IN ('new','learning','known','ignored')),
|
||||
CONSTRAINT ck_term_level CHECK ((status = 'learning' AND level BETWEEN 1 AND 7) OR (status <> 'learning' AND level = 0)),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
var schemaV4Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_dictionaries (
|
||||
id BIGINT PRIMARY KEY, name VARCHAR(120) NOT NULL, language VARCHAR(16) NOT NULL,
|
||||
version VARCHAR(32) NOT NULL, source VARCHAR(512) NOT NULL, format VARCHAR(32) NOT NULL,
|
||||
sha256 CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
|
||||
entry_count INT NOT NULL, enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
archive LONGBLOB NOT NULL, updated_at DATETIME(3) NOT NULL,
|
||||
CHECK (id = 1), CHECK (language = 'en')
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
var schemaV2Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS sys_user (
|
||||
user_id BIGINT PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL UNIQUE,
|
||||
password VARCHAR(128) NOT NULL, nick_name VARCHAR(128) NOT NULL DEFAULT '', phone VARCHAR(11) NOT NULL DEFAULT '',
|
||||
role_id INT NOT NULL, salt VARCHAR(255) NOT NULL DEFAULT '', avatar VARCHAR(255) NOT NULL DEFAULT '',
|
||||
sex VARCHAR(255) NOT NULL DEFAULT '', email VARCHAR(128) NOT NULL DEFAULT '', dept_id BIGINT NOT NULL DEFAULT 0,
|
||||
post_id BIGINT NOT NULL DEFAULT 0, remark VARCHAR(255) NOT NULL DEFAULT '', status VARCHAR(4) NOT NULL DEFAULT '2',
|
||||
create_by BIGINT NOT NULL DEFAULT 0, update_by BIGINT NOT NULL DEFAULT 0,
|
||||
created_at DATETIME(3) NULL, updated_at DATETIME(3) NULL, deleted_at DATETIME(3) NULL,
|
||||
CHECK (role_id IN (1,2)), CHECK (status IN ('1','2'))
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_spaces (
|
||||
owner_id BIGINT PRIMARY KEY, language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_sessions (
|
||||
token_hash CHAR(64) CHARACTER SET ascii COLLATE ascii_bin PRIMARY KEY,
|
||||
owner_id BIGINT NOT NULL, expires_at DATETIME(3) NOT NULL,
|
||||
INDEX (owner_id), INDEX (expires_at),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_login_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL, ip VARCHAR(45) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_operation_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, actor_id BIGINT NOT NULL, actor_username VARCHAR(32) NOT NULL,
|
||||
target_id BIGINT NOT NULL DEFAULT 0, target_username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
action VARCHAR(32) NOT NULL, result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL,
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(actor_username,created_at), INDEX(target_username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
// v3 adds the private library: books, chapters with byte-exact original text, and
|
||||
// persistent ingestion jobs. owner_id is denormalized onto chapters and jobs so every
|
||||
// query can filter by the authenticated identity without joining.
|
||||
var schemaV3Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_books (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, owner_id BIGINT NOT NULL,
|
||||
title VARCHAR(120) NOT NULL, language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
created_at DATETIME(3) NOT NULL, updated_at DATETIME(3) NOT NULL,
|
||||
INDEX (owner_id, updated_at, id),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_chapters (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, book_id BIGINT UNSIGNED NOT NULL, owner_id BIGINT NOT NULL,
|
||||
ordinal INT NOT NULL, title VARCHAR(120) NOT NULL,
|
||||
original_text MEDIUMTEXT NOT NULL, char_count INT NOT NULL DEFAULT 0,
|
||||
content_sha256 CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL DEFAULT '',
|
||||
status VARCHAR(16) NOT NULL DEFAULT 'pending', error_reason VARCHAR(32) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, updated_at DATETIME(3) NOT NULL,
|
||||
UNIQUE KEY uq_chapter_ordinal (book_id, ordinal), INDEX (owner_id, id),
|
||||
CHECK (status IN ('pending','processing','ready','failed')),
|
||||
FOREIGN KEY (book_id) REFERENCES lexgo_books(id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_ingest_jobs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, owner_id BIGINT NOT NULL,
|
||||
book_id BIGINT UNSIGNED NOT NULL, chapter_id BIGINT UNSIGNED NOT NULL,
|
||||
request_key CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
|
||||
content_sha256 CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
|
||||
status VARCHAR(16) NOT NULL DEFAULT 'pending', attempts INT NOT NULL DEFAULT 0,
|
||||
error_reason VARCHAR(32) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, updated_at DATETIME(3) NOT NULL, finished_at DATETIME(3) NULL,
|
||||
UNIQUE KEY uq_job_request (owner_id, request_key), INDEX (status, id),
|
||||
CHECK (status IN ('pending','processing','ready','failed')),
|
||||
FOREIGN KEY (book_id) REFERENCES lexgo_books(id) ON DELETE CASCADE,
|
||||
FOREIGN KEY (chapter_id) REFERENCES lexgo_chapters(id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
func CheckSchema(db *gorm.DB) error {
|
||||
var r struct {
|
||||
Version int
|
||||
Product string
|
||||
}
|
||||
if err := db.Table("lexgo_schema").Where("id=1").First(&r).Error; err != nil || r.Version != 2 || r.Product != "lexgo" {
|
||||
if err := db.Table("lexgo_schema").Where("id=1").First(&r).Error; err != nil || r.Version != SchemaVersion || r.Product != "lexgo" {
|
||||
return errors.New("run the explicit migration before starting")
|
||||
}
|
||||
return nil
|
||||
|
||||
@@ -0,0 +1,324 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// ID 1 is the sole English corpus slot. Keeping the validated ZIP in MySQL
|
||||
// makes ordinary database backups include the resource needed after a restart.
|
||||
type DictionaryResource struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement:false"`
|
||||
Name string
|
||||
Language string
|
||||
Version string
|
||||
Source string
|
||||
Format string
|
||||
SHA256 string `gorm:"column:sha256"`
|
||||
EntryCount int
|
||||
Enabled bool
|
||||
Archive []byte
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (DictionaryResource) TableName() string { return "lexgo_dictionaries" }
|
||||
|
||||
type DictionaryView struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Language string `json:"language"`
|
||||
Version string `json:"version"`
|
||||
Source string `json:"source"`
|
||||
Format string `json:"format"`
|
||||
Status string `json:"status"`
|
||||
Enabled bool `json:"enabled"`
|
||||
SHA256 string `json:"sha256"`
|
||||
EntryCount int `json:"entryCount"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func dictionaryView(r DictionaryResource, status string) DictionaryView {
|
||||
return DictionaryView{r.ID, r.Name, r.Language, r.Version, r.Source, r.Format, status, r.Enabled, r.SHA256, r.EntryCount, r.UpdatedAt}
|
||||
}
|
||||
|
||||
type DictionaryImportResult struct {
|
||||
Resource DictionaryView `json:"resource"`
|
||||
Duplicate bool `json:"duplicate"`
|
||||
}
|
||||
type ChapterTokens struct {
|
||||
TextSHA256 string `json:"textSha256"`
|
||||
Tokens []TextToken `json:"tokens"`
|
||||
}
|
||||
|
||||
// Each router keeps at most one immutable parsed corpus; no private chapter or
|
||||
// lookup data enters the cache. The mutex also coalesces simultaneous cold loads.
|
||||
type dictionaryCache struct {
|
||||
mu sync.Mutex
|
||||
sha string
|
||||
engine *WordNet
|
||||
}
|
||||
|
||||
func (cache *dictionaryCache) load(tx *gorm.DB, r DictionaryResource) (*WordNet, error) {
|
||||
cache.mu.Lock()
|
||||
defer cache.mu.Unlock()
|
||||
if r.SHA256 != WordNetSHA {
|
||||
return nil, errors.New("unsupported resource checksum")
|
||||
}
|
||||
if cache.sha == r.SHA256 && cache.engine != nil {
|
||||
return cache.engine, nil
|
||||
}
|
||||
var stored DictionaryResource
|
||||
if err := tx.Select("id", "archive").Where("id = ? AND sha256 = ?", r.ID, r.SHA256).First(&stored).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
engine, err := ParseWordNet(stored.Archive)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
cache.sha = r.SHA256
|
||||
cache.engine = engine
|
||||
return engine, nil
|
||||
}
|
||||
|
||||
func readyOwnedChapter(tx *gorm.DB, owner int, id int64) (Chapter, error) {
|
||||
var chapter Chapter
|
||||
err := tx.Where("id = ? AND owner_id = ?", id, owner).First(&chapter).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return chapter, failure(404, "章节不存在")
|
||||
}
|
||||
if err != nil {
|
||||
return chapter, err
|
||||
}
|
||||
if chapter.Status != statusReady {
|
||||
return chapter, failure(409, "章节尚未就绪，请稍后重试")
|
||||
}
|
||||
return chapter, nil
|
||||
}
|
||||
|
||||
func readDictionaryUpload(c *gin.Context) (DictionaryResource, error) {
|
||||
bad := failure(400, "词典上传无效，请使用指定的 WordNet 3.0 ZIP 和完整资源信息")
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, maxDictionaryZip+(64<<10))
|
||||
reader, err := c.Request.MultipartReader()
|
||||
if err != nil {
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
fields := map[string]string{}
|
||||
var archive []byte
|
||||
for {
|
||||
part, err := reader.NextPart()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
name := part.FormName()
|
||||
if name == "file" {
|
||||
if archive != nil || part.FileName() == "" {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
archive, err = io.ReadAll(io.LimitReader(part, maxDictionaryZip+1))
|
||||
if err != nil || len(archive) == 0 || len(archive) > maxDictionaryZip {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
} else {
|
||||
if part.FileName() != "" || (name != "name" && name != "language" && name != "version" && name != "source" && name != "format") {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
if _, exists := fields[name]; exists {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
value, e := io.ReadAll(io.LimitReader(part, 1025))
|
||||
if e != nil || len(value) > 1024 || !utf8.Valid(value) {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
fields[name] = string(value)
|
||||
}
|
||||
part.Close()
|
||||
}
|
||||
name := strings.TrimSpace(fields["name"])
|
||||
if name == "" || utf8.RuneCountInString(name) > 120 || strings.IndexFunc(name, unicode.IsControl) >= 0 || fields["language"] != "en" || fields["version"] != "3.0" || fields["format"] != "wordnet-3.0-zip" || fields["source"] != WordNetSource || len(archive) == 0 {
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
return DictionaryResource{ID: 1, Name: name, Language: "en", Version: "3.0", Source: WordNetSource, Format: "wordnet-3.0-zip", SHA256: WordNetSHA, Enabled: true, Archive: archive}, nil
|
||||
}
|
||||
|
||||
func registerDictionaryRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
cache := &dictionaryCache{}
|
||||
uploadGate := make(chan struct{}, 1)
|
||||
v.GET("/dictionaries", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "词典列表不接受查询参数")
|
||||
}
|
||||
var rows []DictionaryResource
|
||||
if err := tx.Omit("archive").Order("id").Find(&rows).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items := make([]DictionaryView, 0, len(rows))
|
||||
for _, row := range rows {
|
||||
status := "disabled"
|
||||
if row.Enabled {
|
||||
status = "ready"
|
||||
if _, err := cache.load(tx, row); err != nil {
|
||||
status = "unavailable"
|
||||
}
|
||||
}
|
||||
items = append(items, dictionaryView(row, status))
|
||||
}
|
||||
return gin.H{"items": items, "supported": gin.H{"name": "Princeton WordNet", "language": "en", "version": "3.0", "format": "wordnet-3.0-zip", "source": WordNetSource, "sha256": WordNetSHA}}, nil
|
||||
}))
|
||||
v.POST("/dictionaries/import", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
select {
|
||||
case uploadGate <- struct{}{}:
|
||||
defer func() { <-uploadGate }()
|
||||
default:
|
||||
return nil, failure(429, "已有词典正在导入，请稍后重试")
|
||||
}
|
||||
resource, err := readDictionaryUpload(c)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
engine, err := ParseWordNet(resource.Archive)
|
||||
if err != nil {
|
||||
return nil, failure(400, "词典文件校验失败，请选择指定的 WordNet 3.0 ZIP")
|
||||
}
|
||||
resource.EntryCount = engine.EntryCount
|
||||
resource.UpdatedAt = stamp(now())
|
||||
// INSERT ... ON CONFLICT followed by a locking read serializes even the first
|
||||
// concurrent import. A validated replacement and its metadata commit together.
|
||||
created := tx.Clauses(clause.OnConflict{DoNothing: true}).Create(&resource)
|
||||
if created.Error != nil {
|
||||
return nil, created.Error
|
||||
}
|
||||
var existing DictionaryResource
|
||||
if err = tx.Omit("archive").Clauses(clause.Locking{Strength: "UPDATE"}).Where("id = 1").First(&existing).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
duplicate := created.RowsAffected == 0 && existing.SHA256 == resource.SHA256
|
||||
if created.RowsAffected == 0 {
|
||||
if err = tx.Model(&DictionaryResource{}).Where("id = 1").Select("name", "language", "version", "source", "format", "sha256", "entry_count", "enabled", "archive", "updated_at").Updates(&resource).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
existing = resource
|
||||
}
|
||||
status := "ready"
|
||||
if !existing.Enabled {
|
||||
status = "disabled"
|
||||
}
|
||||
return DictionaryImportResult{dictionaryView(existing, status), duplicate}, nil
|
||||
}))
|
||||
v.PATCH("/dictionaries/:id", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
|
||||
if err != nil || id != 1 {
|
||||
return nil, failure(404, "词典不存在")
|
||||
}
|
||||
var input struct {
|
||||
Enabled *bool `json:"enabled"`
|
||||
}
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if input.Enabled == nil {
|
||||
return nil, failure(400, "请指定词典启用状态")
|
||||
}
|
||||
var resource DictionaryResource
|
||||
if err = tx.Omit("archive").Clauses(clause.Locking{Strength: "UPDATE"}).Where("id = ?", id).First(&resource).Error; errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil, failure(404, "词典不存在")
|
||||
} else if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if *input.Enabled {
|
||||
if _, err = cache.load(tx, resource); err != nil {
|
||||
return nil, failure(409, "词典资源不可用，请重新导入")
|
||||
}
|
||||
}
|
||||
resource.Enabled = *input.Enabled
|
||||
resource.UpdatedAt = stamp(now())
|
||||
if err = tx.Model(&resource).Updates(map[string]any{"enabled": resource.Enabled, "updated_at": resource.UpdatedAt}).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
status := "disabled"
|
||||
if resource.Enabled {
|
||||
status = "ready"
|
||||
}
|
||||
return gin.H{"resource": dictionaryView(resource, status)}, nil
|
||||
}))
|
||||
v.GET("/chapters/:id/tokens", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "分词接口不接受查询参数")
|
||||
}
|
||||
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
return nil, failure(404, "章节不存在")
|
||||
}
|
||||
chapter, err := readyOwnedChapter(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tokens := Tokenize(chapter.OriginalText)
|
||||
language, err := languageOf(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// The learner's own saved words are part of the chapter view, so the reader
|
||||
// shows the same status here as everywhere else.
|
||||
if err = attachTerms(tx, u.UserId, language, tokens); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ChapterTokens{chapter.ContentSHA256, tokens}, nil
|
||||
}))
|
||||
v.POST("/lookup", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
var input struct {
|
||||
ChapterID int64 `json:"chapterId"`
|
||||
Start *int `json:"start"`
|
||||
End *int `json:"end"`
|
||||
}
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
chapter, err := readyOwnedChapter(tx, u.UserId, input.ChapterID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if input.Start == nil || input.End == nil {
|
||||
return nil, failure(400, "请选择完整单词")
|
||||
}
|
||||
query, err := wordAtRange(chapter, *input.Start, *input.End)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
missing := LookupResult{Status: "resource_missing", Query: query, Candidates: []string{}, Entries: []DictionaryEntry{}}
|
||||
var resource DictionaryResource
|
||||
err = tx.Omit("archive").Where("id = 1 AND enabled = ?", true).First(&resource).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return missing, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
engine, err := cache.load(tx, resource)
|
||||
if err != nil {
|
||||
return missing, nil
|
||||
}
|
||||
result := engine.Lookup(query)
|
||||
result.Resource = &LookupResource{Name: resource.Name, Version: resource.Version}
|
||||
return result, nil
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"mime/multipart"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
)
|
||||
|
||||
func importDictionaryAPI(t *testing.T, r *gin.Engine, token string, raw []byte) (int, DictionaryImportResult) {
|
||||
t.Helper()
|
||||
var body bytes.Buffer
|
||||
w := multipart.NewWriter(&body)
|
||||
for k, v := range map[string]string{"name": "Princeton WordNet", "language": "en", "version": "3.0", "source": WordNetSource, "format": "wordnet-3.0-zip"} {
|
||||
if err := w.WriteField(k, v); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
p, err := w.CreateFormFile("file", "wordnet.zip")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
p.Write(raw)
|
||||
w.Close()
|
||||
req := httptest.NewRequest("POST", "/api/v1/dictionaries/import", &body)
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
response := httptest.NewRecorder()
|
||||
r.ServeHTTP(response, req)
|
||||
var e envelope
|
||||
json.Unmarshal(response.Body.Bytes(), &e)
|
||||
var result DictionaryImportResult
|
||||
json.Unmarshal(e.Data, &result)
|
||||
return response.Code, result
|
||||
}
|
||||
|
||||
func TestDictionaryAPIResourcesAndOwnership(t *testing.T) {
|
||||
db := testDB(t)
|
||||
raw, err := os.ReadFile("../../../.local/nlp-resources/wordnet.zip")
|
||||
if os.IsNotExist(err) {
|
||||
t.Skip("prepare WordNet fixture")
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Test-only dedicated database slot; no development/production resources touched.
|
||||
if err := db.Exec("DELETE FROM lexgo_dictionaries").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { db.Exec("DELETE FROM lexgo_dictionaries") })
|
||||
r := Router(db, time.Now)
|
||||
users := []admin.SysUser{}
|
||||
tokens := []string{}
|
||||
for i, role := range []int{1, 2, 2} {
|
||||
u := admin.SysUser{Username: randomName(fmt.Sprintf("dict%d", i)), Password: fixturePassword, RoleId: role, Status: "2"}
|
||||
if err := db.Create(&u).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
users = append(users, u)
|
||||
tokens = append(tokens, loginToken(t, r, u.Username, fixturePassword))
|
||||
}
|
||||
book := Book{OwnerID: users[1].UserId, Title: "Fictional", Language: "en", CreatedAt: stamp(time.Now()), UpdatedAt: stamp(time.Now())}
|
||||
if err := db.Create(&book).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
original := "😀 Dogs went. Cafe\u0301 123"
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: users[1].UserId, Ordinal: 1, Title: "Fictional", OriginalText: original, ContentSHA256: contentSHA(original), CharCount: len([]rune(original)), Status: statusReady, CreatedAt: stamp(time.Now()), UpdatedAt: stamp(time.Now())}
|
||||
if err := db.Create(&chapter).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lookup := func(token string, start, end int) (int, LookupResult) {
|
||||
code, data := callAPI(t, r, "POST", "/api/v1/lookup", token, map[string]any{"chapterId": chapter.ID, "start": start, "end": end})
|
||||
var got LookupResult
|
||||
json.Unmarshal(data, &got)
|
||||
return code, got
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", "/api/v1/dictionaries", tokens[1], nil); code != 403 {
|
||||
t.Fatal("learner resource list", code)
|
||||
}
|
||||
if code, _ := importDictionaryAPI(t, r, tokens[1], []byte("bad")); code != 403 {
|
||||
t.Fatal("learner import", code)
|
||||
}
|
||||
if code, _ := lookup(tokens[2], 2, 6); code != 404 {
|
||||
t.Fatal("foreign lookup", code)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", chapter.ID), tokens[2], nil); code != 404 {
|
||||
t.Fatal("foreign tokens", code)
|
||||
}
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "resource_missing" {
|
||||
t.Fatal("missing", code, got)
|
||||
}
|
||||
code, data := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", chapter.ID), tokens[1], nil)
|
||||
var analyzed ChapterTokens
|
||||
json.Unmarshal(data, &analyzed)
|
||||
if code != 200 || analyzed.TextSHA256 != contentSHA(original) || analyzed.Tokens[2].StartUtf16 != 3 {
|
||||
t.Fatal("tokens", code, string(data))
|
||||
}
|
||||
for _, span := range [][2]int{{-1, 3}, {2, 5}, {0, 1}, {2, 11}, {18, 21}, {1, 2}, {6, 2}} {
|
||||
if code, _ := lookup(tokens[1], span[0], span[1]); code != 400 {
|
||||
t.Fatal("invalid interval", span, code)
|
||||
}
|
||||
}
|
||||
code, imported := importDictionaryAPI(t, r, tokens[0], raw)
|
||||
if code != 200 || imported.Duplicate || imported.Resource.Status != "ready" {
|
||||
t.Fatal("import", code, imported)
|
||||
}
|
||||
if code, got := lookup(tokens[1], 7, 11); code != 200 || got.Status != "lemma" || *got.MatchedForm != "go" {
|
||||
t.Fatal("went", code, got)
|
||||
}
|
||||
code, duplicate := importDictionaryAPI(t, r, tokens[0], raw)
|
||||
if code != 200 || !duplicate.Duplicate || duplicate.Resource.ID != imported.Resource.ID {
|
||||
t.Fatal("duplicate", code, duplicate)
|
||||
}
|
||||
if code, _ := importDictionaryAPI(t, r, tokens[0], []byte("broken")); code != 400 {
|
||||
t.Fatal("bad import", code)
|
||||
}
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "lemma" {
|
||||
t.Fatal("failed import lost old resource", code, got)
|
||||
}
|
||||
endpoint := fmt.Sprintf("/api/v1/dictionaries/%d", imported.Resource.ID)
|
||||
if code, _ := callAPI(t, r, "PATCH", endpoint, tokens[1], map[string]bool{"enabled": false}); code != 403 {
|
||||
t.Fatal("learner toggle", code)
|
||||
}
|
||||
for _, enabled := range []bool{false, true} {
|
||||
if code, _ := callAPI(t, r, "PATCH", endpoint, tokens[0], map[string]bool{"enabled": enabled}); code != 200 {
|
||||
t.Fatal("toggle", code)
|
||||
}
|
||||
code, got := lookup(tokens[1], 2, 6)
|
||||
if code != 200 || (enabled && got.Status != "lemma") || (!enabled && got.Status != "resource_missing") {
|
||||
t.Fatal("enabled state", enabled, code, got)
|
||||
}
|
||||
}
|
||||
r = Router(db, time.Now) // A new router has an empty cache and reloads the persisted ZIP.
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "lemma" {
|
||||
t.Fatal("cold restart", code, got)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "PATCH", endpoint, tokens[0], map[string]bool{"enabled": false}); code != 200 {
|
||||
t.Fatal("disable before duplicate", code)
|
||||
}
|
||||
if code, result := importDictionaryAPI(t, r, tokens[0], raw); code != 200 || !result.Duplicate || !result.Resource.Enabled {
|
||||
t.Fatal("duplicate must re-enable", code, result)
|
||||
}
|
||||
var count int64
|
||||
if err := db.Model(&DictionaryResource{}).Count(&count).Error; err != nil || count != 1 {
|
||||
t.Fatal("duplicate created extra resource", count, err)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "PATCH", endpoint, tokens[0], map[string]any{"enabled": nil}); code != 400 {
|
||||
t.Fatal("null toggle", code)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "POST", "/api/v1/lookup", tokens[1], map[string]any{"chapterId": chapter.ID, "start": 2, "end": 6, "ownerId": users[2].UserId}); code != 400 {
|
||||
t.Fatal("unknown lookup input", code)
|
||||
}
|
||||
if err := db.Model(&DictionaryResource{}).Where("id=1").Update("archive", []byte("corrupt fixture")).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r = Router(db, time.Now)
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "resource_missing" {
|
||||
t.Fatal("corrupt cold resource", code, got)
|
||||
}
|
||||
if code, got := callAPI(t, r, "GET", "/api/v1/dictionaries", tokens[0], nil); code != 200 || !bytes.Contains(got, []byte(`"status":"unavailable"`)) {
|
||||
t.Fatal("corrupt resource state", code, string(got))
|
||||
}
|
||||
if code, result := importDictionaryAPI(t, r, tokens[0], raw); code != 200 || !result.Duplicate {
|
||||
t.Fatal("repair reimport", code, result)
|
||||
}
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "lemma" {
|
||||
t.Fatal("repair lookup", code, got)
|
||||
}
|
||||
if err := db.Model(&chapter).Update("status", statusPending).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code, _ := lookup(tokens[1], 2, 6); code != 409 {
|
||||
t.Fatal("pending lookup", code)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,286 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// Editing and deleting are strictly owner scoped. Nothing here touches personal terms,
|
||||
// review rows or answers: those belong to the learner, not to a book or a chapter.
|
||||
|
||||
type BookUpdateInput struct {
|
||||
Title string `json:"title"`
|
||||
}
|
||||
|
||||
type ChapterUpdateInput struct {
|
||||
Title *string `json:"title"`
|
||||
Text *string `json:"text"`
|
||||
}
|
||||
|
||||
// ChapterSource is the editable text of one owned chapter. It is separate from the reader
|
||||
// contract, which only exposes text once a chapter is ready, so a failed chapter can be
|
||||
// corrected and submitted again.
|
||||
type ChapterSource struct {
|
||||
ID int64 `json:"id"`
|
||||
BookID int64 `json:"bookId"`
|
||||
Ordinal int `json:"ordinal"`
|
||||
Title string `json:"title"`
|
||||
Text string `json:"text"`
|
||||
Status string `json:"status"`
|
||||
ContentSHA256 string `json:"contentSha256"`
|
||||
CharCount int `json:"charCount"`
|
||||
}
|
||||
|
||||
type ChapterEdit struct {
|
||||
Chapter ChapterSummary `json:"chapter"`
|
||||
Job *JobView `json:"job"`
|
||||
VersionChanged bool `json:"versionChanged"`
|
||||
}
|
||||
|
||||
type DeletionResult struct {
|
||||
BookID int64 `json:"bookId,omitempty"`
|
||||
ChapterID int64 `json:"chapterId,omitempty"`
|
||||
Chapters int `json:"chapters,omitempty"`
|
||||
Remaining int `json:"remaining"`
|
||||
}
|
||||
|
||||
// editTitle validates a title with the same rules a paste uses, so a renamed book or chapter
|
||||
// stays within the limits the list and reader already rely on.
|
||||
func editTitle(raw string) (string, error) {
|
||||
title := strings.TrimSpace(raw)
|
||||
if title == "" {
|
||||
return "", failure(400, "请填写标题")
|
||||
}
|
||||
if utf8.RuneCountInString(title) > maxTitleRunes {
|
||||
return "", failure(400, "标题最多 120 个字符")
|
||||
}
|
||||
return title, nil
|
||||
}
|
||||
|
||||
// lockOwnedChapter returns the caller's chapter or reports it as missing, so another
|
||||
// account's chapter id is never confirmed to exist.
|
||||
func lockOwnedChapter(tx *gorm.DB, owner int, chapterID int64, chapter *Chapter) error {
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", chapterID, owner).First(chapter).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return failure(404, "章节不存在")
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func RenameBook(tx *gorm.DB, owner int, bookID int64, input BookUpdateInput, now time.Time) (BookRef, error) {
|
||||
title, err := editTitle(input.Title)
|
||||
if err != nil {
|
||||
return BookRef{}, err
|
||||
}
|
||||
var book Book
|
||||
if err = lockOwnedBook(tx, owner, bookID, &book); err != nil {
|
||||
return BookRef{}, err
|
||||
}
|
||||
book.Title = title
|
||||
book.UpdatedAt = stamp(now)
|
||||
if err = tx.Model(&Book{}).Where("id = ? AND owner_id = ?", book.ID, owner).
|
||||
Updates(map[string]any{"title": title, "updated_at": book.UpdatedAt}).Error; err != nil {
|
||||
return BookRef{}, err
|
||||
}
|
||||
return bookRef(book), nil
|
||||
}
|
||||
|
||||
func ChapterEditSource(tx *gorm.DB, owner int, chapterID int64) (ChapterSource, error) {
|
||||
var chapter Chapter
|
||||
if err := tx.Where("id = ? AND owner_id = ?", chapterID, owner).First(&chapter).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ChapterSource{}, failure(404, "章节不存在")
|
||||
}
|
||||
return ChapterSource{}, err
|
||||
}
|
||||
return ChapterSource{
|
||||
ID: chapter.ID, BookID: chapter.BookID, Ordinal: chapter.Ordinal, Title: chapter.Title,
|
||||
Text: chapter.OriginalText, Status: chapter.Status, ContentSHA256: chapter.ContentSHA256,
|
||||
CharCount: chapter.CharCount,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// UpdateChapter renames a chapter and, when the text really changed, stores it as a new
|
||||
// version and queues a job for it. Only a changed version re-processes: a repeated save of
|
||||
// the same text is idempotent, and a title-only edit never touches the processing state.
|
||||
func UpdateChapter(tx *gorm.DB, owner int, chapterID int64, input ChapterUpdateInput, now time.Time) (ChapterEdit, error) {
|
||||
if input.Title == nil && input.Text == nil {
|
||||
return ChapterEdit{}, failure(400, "请选择要修改的内容")
|
||||
}
|
||||
var chapter Chapter
|
||||
if err := lockOwnedChapter(tx, owner, chapterID, &chapter); err != nil {
|
||||
return ChapterEdit{}, err
|
||||
}
|
||||
ts := stamp(now)
|
||||
updates := map[string]any{"updated_at": ts}
|
||||
if input.Title != nil {
|
||||
title, err := editTitle(*input.Title)
|
||||
if err != nil {
|
||||
return ChapterEdit{}, err
|
||||
}
|
||||
chapter.Title = title
|
||||
updates["title"] = title
|
||||
}
|
||||
var job *IngestJob
|
||||
if input.Text != nil {
|
||||
text := *input.Text
|
||||
if _, sha, count, err := validatePaste(chapter.Title, text); err != nil {
|
||||
return ChapterEdit{}, err
|
||||
} else if sha != chapter.ContentSHA256 {
|
||||
// A new version replaces the text and owns the chapter's state from here on.
|
||||
chapter.OriginalText, chapter.ContentSHA256, chapter.CharCount = text, sha, count
|
||||
chapter.Status, chapter.ErrorReason = statusPending, ""
|
||||
updates["original_text"] = text
|
||||
updates["content_sha256"] = sha
|
||||
updates["char_count"] = count
|
||||
updates["status"] = statusPending
|
||||
updates["error_reason"] = ""
|
||||
// The request key is derived from chapter and version, so one version has one job.
|
||||
key := contentSHA(fmt.Sprintf("edit:%d:%s", chapter.ID, sha))
|
||||
created := IngestJob{OwnerID: owner, BookID: chapter.BookID, ChapterID: chapter.ID,
|
||||
RequestKey: key, ContentSHA256: sha, Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err := tx.Create(&created).Error; err != nil {
|
||||
return ChapterEdit{}, err
|
||||
}
|
||||
job = &created
|
||||
}
|
||||
}
|
||||
if err := tx.Model(&Chapter{}).Where("id = ? AND owner_id = ?", chapter.ID, owner).Updates(updates).Error; err != nil {
|
||||
return ChapterEdit{}, err
|
||||
}
|
||||
result := ChapterEdit{VersionChanged: job != nil}
|
||||
if job != nil {
|
||||
view := jobView(*job)
|
||||
result.Job = &view
|
||||
}
|
||||
jobID := int64(0)
|
||||
if job != nil {
|
||||
jobID = job.ID
|
||||
}
|
||||
result.Chapter = chapterSummaryWithJob(chapter, &jobID)
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// DeleteBook removes the caller's book with its chapters and their jobs in one transaction.
|
||||
// Personal terms, review schedules and answers are not touched: they belong to the learner.
|
||||
func DeleteBook(tx *gorm.DB, owner int, bookID int64) (DeletionResult, error) {
|
||||
var book Book
|
||||
if err := lockOwnedBook(tx, owner, bookID, &book); err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
var chapters int64
|
||||
if err := tx.Model(&Chapter{}).Where("book_id = ? AND owner_id = ?", book.ID, owner).Count(&chapters).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
// Chapters and their jobs go with the book through the foreign keys.
|
||||
if err := tx.Where("id = ? AND owner_id = ?", book.ID, owner).Delete(&Book{}).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
return DeletionResult{BookID: book.ID, Chapters: int(chapters)}, nil
|
||||
}
|
||||
|
||||
// DeleteChapter removes one owned chapter with its jobs and closes the gap in the chapter
|
||||
// order, so "剩余 N 章" and the reader navigation stay contiguous.
|
||||
func DeleteChapter(tx *gorm.DB, owner int, chapterID int64) (DeletionResult, error) {
|
||||
var chapter Chapter
|
||||
if err := lockOwnedChapter(tx, owner, chapterID, &chapter); err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
// Lock the book too: two concurrent deletions in one book must not renumber each other.
|
||||
var book Book
|
||||
if err := lockOwnedBook(tx, owner, chapter.BookID, &book); err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
if err := tx.Where("id = ? AND owner_id = ?", chapter.ID, owner).Delete(&Chapter{}).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
// Ordering ascending decrements each row into a slot the previous row just freed, which
|
||||
// keeps the unique (book_id, ordinal) key satisfied throughout.
|
||||
if err := tx.Exec("UPDATE lexgo_chapters SET ordinal = ordinal - 1 WHERE book_id = ? AND owner_id = ? AND ordinal > ? ORDER BY ordinal ASC",
|
||||
chapter.BookID, owner, chapter.Ordinal).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
var remaining int64
|
||||
if err := tx.Model(&Chapter{}).Where("book_id = ? AND owner_id = ?", chapter.BookID, owner).Count(&remaining).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
return DeletionResult{ChapterID: chapter.ID, BookID: chapter.BookID, Remaining: int(remaining)}, nil
|
||||
}
|
||||
|
||||
func registerEditRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
v.PATCH("/books/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var input BookUpdateInput
|
||||
if err = decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
book, err := RenameBook(tx, u.UserId, id, input, now())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"book": book}, nil
|
||||
}))
|
||||
v.DELETE("/books/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result, err := DeleteBook(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"deleted": result}, nil
|
||||
}))
|
||||
v.GET("/chapters/:id/source", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "正文接口不接受查询参数")
|
||||
}
|
||||
id, err := pathID(c, "章节不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
source, err := ChapterEditSource(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"source": source}, nil
|
||||
}))
|
||||
v.PATCH("/chapters/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "章节不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var input ChapterUpdateInput
|
||||
if err = decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
edited, err := UpdateChapter(tx, u.UserId, id, input, now())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return edited, nil
|
||||
}))
|
||||
v.DELETE("/chapters/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "章节不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result, err := DeleteChapter(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"deleted": result}, nil
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,543 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
const editFixtureText = "Mira opened the workshop.\nThe sign read “A small step…”\n"
|
||||
|
||||
func TestEditTitleRules(t *testing.T) {
|
||||
if title, err := editTitle(" A small step "); err != nil || title != "A small step" {
|
||||
t.Fatalf("trimmed title: %q %v", title, err)
|
||||
}
|
||||
if _, err := editTitle(" "); err == nil {
|
||||
t.Fatal("an empty title must be rejected")
|
||||
}
|
||||
if _, err := editTitle(strings.Repeat("a", maxTitleRunes+1)); err == nil {
|
||||
t.Fatal("a title over the limit must be rejected")
|
||||
}
|
||||
if _, err := editTitle(strings.Repeat("a", maxTitleRunes)); err != nil {
|
||||
t.Fatalf("the exact title limit must be accepted: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func patchResource(t *testing.T, r *gin.Engine, token, path string, body any) (int, string, json.RawMessage) {
|
||||
t.Helper()
|
||||
return callRaw(t, r, "PATCH", path, token, body)
|
||||
}
|
||||
|
||||
func existingTitle(t *testing.T, r *gin.Engine, token string, bookID int64) string {
|
||||
t.Helper()
|
||||
code, detail := bookDetail(t, r, token, bookID)
|
||||
if code != 200 {
|
||||
t.Fatalf("book detail status %d", code)
|
||||
}
|
||||
return detail.Book.Title
|
||||
}
|
||||
|
||||
// TestMySQLRenameBookAndChapter covers renaming only: the text and the processing state stay
|
||||
// exactly as they were.
|
||||
func TestMySQLRenameBookAndChapter(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "rename-fixture-0001", "title": "Before Rename", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
before := chapterRow(t, db, pasted.Chapter.ID)
|
||||
|
||||
code, msg, data := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]string{"title": " After Rename "})
|
||||
if code != 200 {
|
||||
t.Fatalf("rename book status %d (%s)", code, msg)
|
||||
}
|
||||
var renamed struct {
|
||||
Book struct {
|
||||
ID int64
|
||||
Title string
|
||||
}
|
||||
}
|
||||
json.Unmarshal(data, &renamed)
|
||||
if renamed.Book.Title != "After Rename" || existingTitle(t, r, learner.Token, pasted.Book.ID) != "After Rename" {
|
||||
t.Fatalf("renamed book %+v", renamed.Book)
|
||||
}
|
||||
|
||||
code, msg, data = patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"title": "Chapter Two"})
|
||||
if code != 200 {
|
||||
t.Fatalf("rename chapter status %d (%s)", code, msg)
|
||||
}
|
||||
var edited ChapterEdit
|
||||
json.Unmarshal(data, &edited)
|
||||
if edited.Chapter.Title != "Chapter Two" || edited.VersionChanged || edited.Job != nil {
|
||||
t.Fatalf("rename must not re-process: %+v", edited)
|
||||
}
|
||||
after := chapterRow(t, db, pasted.Chapter.ID)
|
||||
if after.Title != "Chapter Two" || after.Status != statusReady || after.ContentSHA256 != before.ContentSHA256 || after.OriginalText != before.OriginalText {
|
||||
t.Fatalf("rename changed the content state: %+v", after)
|
||||
}
|
||||
var jobs int64
|
||||
db.Model(&IngestJob{}).Where("chapter_id = ?", pasted.Chapter.ID).Count(&jobs)
|
||||
if jobs != 1 {
|
||||
t.Fatalf("a rename created %d jobs", jobs)
|
||||
}
|
||||
|
||||
// Validation and ownership.
|
||||
for _, body := range []map[string]string{{"title": " "}, {"title": strings.Repeat("a", maxTitleRunes+1)}} {
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), body); code != 400 {
|
||||
t.Fatalf("invalid rename %v accepted", body)
|
||||
}
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]any{"title": "x", "ownerId": 9}); code != 400 {
|
||||
t.Fatal("an unknown field must be rejected")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]any{"name": "x"}); code != 400 {
|
||||
t.Fatal("a missing title must be rejected")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, other.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]string{"title": "Stolen"}); code != 404 {
|
||||
t.Fatal("another account must not rename this book")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, other.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"title": "Stolen"}); code != 404 {
|
||||
t.Fatal("another account must not rename this chapter")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, "", fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]string{"title": "Anonymous"}); code != 401 {
|
||||
t.Fatal("renaming requires a session")
|
||||
}
|
||||
if title := existingTitle(t, r, learner.Token, pasted.Book.ID); title != "After Rename" {
|
||||
t.Fatalf("a rejected rename changed the book: %q", title)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLChapterEditVersioning is the core of the ticket: an edit creates a new version, and
|
||||
// the older run must not fail or publish over it.
|
||||
func TestMySQLChapterEditVersioning(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "edit-fixture-0001", "title": "Editable", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// The first run is claimed but not finished, so the edit lands while it is in flight.
|
||||
clock := time.Now().UTC().Truncate(time.Millisecond)
|
||||
job, claimed, err := ClaimNextIngestJob(db, clock)
|
||||
if err != nil || !claimed || job.ChapterID != pasted.Chapter.ID {
|
||||
t.Fatalf("claim (claimed=%v): %v", claimed, err)
|
||||
}
|
||||
|
||||
newText := "Mira reopened the workshop.\r\n\r\nA newer version of the text.\n"
|
||||
code, msg, data := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"text": newText})
|
||||
if code != 200 {
|
||||
t.Fatalf("edit status %d (%s)", code, msg)
|
||||
}
|
||||
var edited ChapterEdit
|
||||
json.Unmarshal(data, &edited)
|
||||
if !edited.VersionChanged || edited.Job == nil || edited.Chapter.Status != statusPending {
|
||||
t.Fatalf("edit must queue a new version: %+v", edited)
|
||||
}
|
||||
row := chapterRow(t, db, pasted.Chapter.ID)
|
||||
if row.OriginalText != newText || row.ContentSHA256 != contentSHA(newText) || row.Status != statusPending || row.CharCount != len([]rune(newText)) {
|
||||
t.Fatalf("stored version: %+v", row)
|
||||
}
|
||||
if row.ID != pasted.Chapter.ID {
|
||||
t.Fatal("an edit must keep the chapter id, so the reading entry stays the same")
|
||||
}
|
||||
|
||||
// The in-flight run belongs to the previous version: it must not touch the chapter.
|
||||
if err := FinishIngestJob(t.Context(), db, job, clock.Add(time.Second)); err != nil {
|
||||
t.Fatalf("finishing the older run must not fail: %v", err)
|
||||
}
|
||||
var stale IngestJob
|
||||
if err := db.First(&stale, job.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if stale.Status != statusFailed || stale.ErrorReason != reasonSuperseded {
|
||||
t.Fatalf("the older run must be marked superseded: %+v", stale)
|
||||
}
|
||||
if row = chapterRow(t, db, pasted.Chapter.ID); row.Status != statusPending || row.ErrorReason != "" || row.OriginalText != newText {
|
||||
t.Fatalf("the older run changed the newer version: %+v", row)
|
||||
}
|
||||
// Retrying the superseded job is refused instead of reprocessing old text.
|
||||
code, _, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", stale.ID), learner.Token, nil)
|
||||
if code != 409 {
|
||||
t.Fatalf("retrying a superseded job: %d", code)
|
||||
}
|
||||
|
||||
// The new version publishes normally and the reader shows the new text.
|
||||
drainIngest(t, db)
|
||||
code, reader := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || reader.Chapter.Status != statusReady || reader.Chapter.OriginalText != newText {
|
||||
t.Fatalf("new version: status %d, %+v", code, reader.Chapter)
|
||||
}
|
||||
|
||||
// Saving the same text again is not a new version.
|
||||
var jobsBefore int64
|
||||
db.Model(&IngestJob{}).Where("chapter_id = ?", pasted.Chapter.ID).Count(&jobsBefore)
|
||||
code, _, data = patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"text": newText})
|
||||
json.Unmarshal(data, &edited)
|
||||
if code != 200 || edited.VersionChanged || edited.Job != nil {
|
||||
t.Fatalf("an unchanged text must not create a version: %+v", edited)
|
||||
}
|
||||
var jobsAfter int64
|
||||
db.Model(&IngestJob{}).Where("chapter_id = ?", pasted.Chapter.ID).Count(&jobsAfter)
|
||||
if jobsAfter != jobsBefore {
|
||||
t.Fatalf("an unchanged text created a job: %d -> %d", jobsBefore, jobsAfter)
|
||||
}
|
||||
if row = chapterRow(t, db, pasted.Chapter.ID); row.Status != statusReady {
|
||||
t.Fatalf("an unchanged save changed the status: %+v", row)
|
||||
}
|
||||
|
||||
// Text rules match a paste, and an empty body is refused.
|
||||
for _, body := range []map[string]string{{"text": " \n\t "}, {"text": strings.Repeat("a", maxChapterRunes+1)}} {
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), body); code != 400 {
|
||||
t.Fatalf("invalid text %v accepted", body)
|
||||
}
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{}); code != 400 {
|
||||
t.Fatal("an empty edit must be rejected")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]any{"text": "Body.\n", "ownerId": 9}); code != 400 {
|
||||
t.Fatal("an unknown field must be rejected")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLChapterSourceAnyStatus(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "source-fixture-0001", "title": "Source", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
readSource := func(token string, chapterID int64) (int, ChapterSource) {
|
||||
code, _, data := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/source", chapterID), token, nil)
|
||||
var payload struct {
|
||||
Source ChapterSource
|
||||
}
|
||||
if len(data) > 0 {
|
||||
json.Unmarshal(data, &payload)
|
||||
}
|
||||
return code, payload.Source
|
||||
}
|
||||
// A pending chapter has no reading text, but its edit source is available to its owner.
|
||||
if code, source := readSource(learner.Token, pasted.Chapter.ID); code != 200 || source.Text != editFixtureText || source.Status != statusPending || source.ContentSHA256 != contentSHA(editFixtureText) {
|
||||
t.Fatalf("pending source: %d %+v", code, source)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
if code, source := readSource(learner.Token, pasted.Chapter.ID); code != 200 || source.Status != statusReady || source.CharCount != len([]rune(editFixtureText)) {
|
||||
t.Fatalf("ready source: %d %+v", code, source)
|
||||
}
|
||||
if code, _ := readSource(other.Token, pasted.Chapter.ID); code != 404 {
|
||||
t.Fatal("another account must not read this source")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/source?id=1", pasted.Chapter.ID), learner.Token, nil); code != 400 {
|
||||
t.Fatal("the source endpoint must reject query parameters")
|
||||
}
|
||||
if code, _ := readSource(learner.Token, 999999); code != 404 {
|
||||
t.Fatal("an unknown chapter must be 404")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLDeleteChapterRenumbers closes the gap in the order and keeps personal records.
|
||||
func TestMySQLDeleteChapterRenumbers(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "delete-fixture-0001", "title": "Three Chapters", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("first paste %d", code)
|
||||
}
|
||||
second := pasteInto(t, r, learner.Token, first.Book.ID, "delete-fixture-0002", "Second", "Second chapter body.\n")
|
||||
third := pasteInto(t, r, learner.Token, first.Book.ID, "delete-fixture-0003", "Third", "Third chapter body.\n")
|
||||
drainIngest(t, db)
|
||||
// One personal word in the chapter that will be deleted, plus one in a surviving chapter.
|
||||
saved := saveWord(t, r, learner.Token, second.Chapter.ID, 0, 6, termStatusNew, nil)
|
||||
survivor := saveWord(t, r, learner.Token, first.Chapter.ID, 0, 4, termStatusNew, nil)
|
||||
before := termReviewRow(t, db, saved.Term.ID)
|
||||
|
||||
code, msg, data := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", second.Chapter.ID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("delete chapter status %d (%s)", code, msg)
|
||||
}
|
||||
var deleted struct {
|
||||
Deleted DeletionResult
|
||||
}
|
||||
json.Unmarshal(data, &deleted)
|
||||
if deleted.Deleted.ChapterID != second.Chapter.ID || deleted.Deleted.Remaining != 2 || deleted.Deleted.BookID != first.Book.ID {
|
||||
t.Fatalf("delete result: %+v", deleted.Deleted)
|
||||
}
|
||||
// The remaining chapters are contiguous and in the original order.
|
||||
code, detail := bookDetail(t, r, learner.Token, first.Book.ID)
|
||||
if code != 200 || len(detail.Chapters) != 2 {
|
||||
t.Fatalf("book detail after delete: %d %+v", code, detail.Chapters)
|
||||
}
|
||||
if detail.Chapters[0].ID != first.Chapter.ID || detail.Chapters[0].Ordinal != 1 || detail.Chapters[1].ID != third.Chapter.ID || detail.Chapters[1].Ordinal != 2 {
|
||||
t.Fatalf("renumbered chapters: %+v", detail.Chapters)
|
||||
}
|
||||
// Navigation follows the new order, and the deleted chapter is gone with its job.
|
||||
code, reader := readChapter(t, r, learner.Token, first.Chapter.ID)
|
||||
if code != 200 || reader.Navigation.NextChapterID == nil || *reader.Navigation.NextChapterID != third.Chapter.ID {
|
||||
t.Fatalf("navigation after delete: %d %+v", code, reader.Navigation)
|
||||
}
|
||||
if code, _ := readChapter(t, r, learner.Token, second.Chapter.ID); code != 404 {
|
||||
t.Fatal("a deleted chapter must be gone")
|
||||
}
|
||||
var jobs int64
|
||||
db.Model(&IngestJob{}).Where("chapter_id = ?", second.Chapter.ID).Count(&jobs)
|
||||
if jobs != 0 {
|
||||
t.Fatalf("the deleted chapter kept %d jobs", jobs)
|
||||
}
|
||||
// Personal records survive the deletion, including the schedule.
|
||||
var terms int64
|
||||
db.Model(&Term{}).Where("id IN ?", []int64{saved.Term.ID, survivor.Term.ID}).Count(&terms)
|
||||
var reviews int64
|
||||
db.Model(&TermReview{}).Where("term_id = ?", saved.Term.ID).Count(&reviews)
|
||||
if terms != 2 || reviews != 1 {
|
||||
t.Fatalf("deleting a chapter changed personal records: terms=%d reviews=%d", terms, reviews)
|
||||
}
|
||||
if after := termReviewRow(t, db, saved.Term.ID); !after.DueAt.Equal(before.DueAt) || after.ReviewCount != before.ReviewCount {
|
||||
t.Fatalf("the review schedule changed: %+v -> %+v", before, after)
|
||||
}
|
||||
// Repeated and foreign deletions.
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", second.Chapter.ID), learner.Token, nil); code != 404 {
|
||||
t.Fatal("a repeated delete must be 404")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", first.Chapter.ID), other.Token, nil); code != 404 {
|
||||
t.Fatal("another account must not delete this chapter")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", first.Chapter.ID), "", nil); code != 401 {
|
||||
t.Fatal("deleting requires a session")
|
||||
}
|
||||
if code, detail := bookDetail(t, r, learner.Token, first.Book.ID); code != 200 || len(detail.Chapters) != 2 {
|
||||
t.Fatalf("a rejected delete changed the book: %+v", detail.Chapters)
|
||||
}
|
||||
}
|
||||
|
||||
// pasteInto appends one chapter to an owned book.
|
||||
func pasteInto(t *testing.T, r *gin.Engine, token string, bookID int64, requestID, title, text string) pasteResponse {
|
||||
t.Helper()
|
||||
code, pasted := pasteChapter(t, r, token, bookID, map[string]string{"requestId": requestID, "title": title, "text": text})
|
||||
if code != 201 {
|
||||
t.Fatalf("append %s status %d", title, code)
|
||||
}
|
||||
return pasted
|
||||
}
|
||||
|
||||
func TestMySQLDeleteBookKeepsPersonalRecords(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "delete-book-0001", "title": "Doomed Book", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
pasteInto(t, r, learner.Token, first.Book.ID, "delete-book-0002", "Second", "Second chapter body.\n")
|
||||
drainIngest(t, db)
|
||||
saved := saveWord(t, r, learner.Token, first.Chapter.ID, 0, 4, termStatusNew, nil)
|
||||
survivorBook := pasteInto(t, r, learner.Token, first.Book.ID, "delete-book-0003", "Third", "Third body.\n")
|
||||
otherBook := func() int64 {
|
||||
code, kept := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "delete-book-keep", "title": "Kept Book", "text": "Kept body.\n", "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("kept book %d", code)
|
||||
}
|
||||
return kept.Book.ID
|
||||
}()
|
||||
drainIngest(t, db)
|
||||
|
||||
code, msg, data := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/books/%d", first.Book.ID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("delete book status %d (%s)", code, msg)
|
||||
}
|
||||
var deleted struct {
|
||||
Deleted DeletionResult
|
||||
}
|
||||
json.Unmarshal(data, &deleted)
|
||||
if deleted.Deleted.BookID != first.Book.ID || deleted.Deleted.Chapters != 3 {
|
||||
t.Fatalf("delete result: %+v", deleted.Deleted)
|
||||
}
|
||||
var books, chapters, jobs int64
|
||||
db.Model(&Book{}).Where("id = ?", first.Book.ID).Count(&books)
|
||||
db.Model(&Chapter{}).Where("book_id = ?", first.Book.ID).Count(&chapters)
|
||||
db.Model(&IngestJob{}).Where("book_id = ?", first.Book.ID).Count(&jobs)
|
||||
if books != 0 || chapters != 0 || jobs != 0 {
|
||||
t.Fatalf("the deleted book left rows: books=%d chapters=%d jobs=%d", books, chapters, jobs)
|
||||
}
|
||||
// The other book of the same account is untouched.
|
||||
if code, detail := bookDetail(t, r, learner.Token, otherBook); code != 200 || len(detail.Chapters) != 1 {
|
||||
t.Fatalf("the other book changed: %d %+v", code, detail.Chapters)
|
||||
}
|
||||
// Personal records and their schedule survive.
|
||||
var terms, reviews int64
|
||||
db.Model(&Term{}).Where("owner_id = ?", learner.ID).Count(&terms)
|
||||
db.Model(&TermReview{}).Where("term_id = ?", saved.Term.ID).Count(&reviews)
|
||||
if terms != 1 || reviews != 1 {
|
||||
t.Fatalf("deleting a book changed personal records: terms=%d reviews=%d", terms, reviews)
|
||||
}
|
||||
code, queue := reviewQueue(t, r, learner.Token)
|
||||
if code != 200 || len(queue.Items) != 1 || queue.Items[0].ID != saved.Term.ID {
|
||||
t.Fatalf("the saved word left the review queue: %d %+v", code, queue.Items)
|
||||
}
|
||||
// The deleted chapter and book are no longer readable, and repeats are 404.
|
||||
if code, _ := readChapter(t, r, learner.Token, survivorBook.Chapter.ID); code != 404 {
|
||||
t.Fatal("a chapter of the deleted book must be gone")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/books/%d", first.Book.ID), learner.Token, nil); code != 404 {
|
||||
t.Fatal("a repeated delete must be 404")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/books/%d", otherBook), other.Token, nil); code != 404 {
|
||||
t.Fatal("another account must not delete this book")
|
||||
}
|
||||
_, list := bookList(t, r, other.Token)
|
||||
if len(list.Items) != 0 {
|
||||
t.Fatalf("the other account must not see this library: %+v", list.Items)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLDeleteDuringProcessing proves a deleted chapter cannot come back through a run
|
||||
// that was already in flight.
|
||||
func TestMySQLDeleteDuringProcessing(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "delete-processing-01", "title": "In Flight", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
clock := time.Now().UTC().Truncate(time.Millisecond)
|
||||
job, claimed, err := ClaimNextIngestJob(db, clock)
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("claim: %v", err)
|
||||
}
|
||||
if code, msg, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), learner.Token, nil); code != 200 {
|
||||
t.Fatalf("deleting a processing chapter: %d (%s)", code, msg)
|
||||
}
|
||||
// The in-flight run finds nothing to publish and reports no error.
|
||||
if err := FinishIngestJob(t.Context(), db, job, clock.Add(time.Second)); err != nil {
|
||||
t.Fatalf("finishing a run for a deleted chapter: %v", err)
|
||||
}
|
||||
var chapters, jobs int64
|
||||
db.Model(&Chapter{}).Where("id = ?", pasted.Chapter.ID).Count(&chapters)
|
||||
db.Model(&IngestJob{}).Where("id = ?", job.ID).Count(&jobs)
|
||||
if chapters != 0 || jobs != 0 {
|
||||
t.Fatalf("a deleted chapter came back: chapters=%d jobs=%d", chapters, jobs)
|
||||
}
|
||||
// The book stays, with no chapters and a clear empty state for the client.
|
||||
code, detail := bookDetail(t, r, learner.Token, pasted.Book.ID)
|
||||
if code != 200 || len(detail.Chapters) != 0 {
|
||||
t.Fatalf("book after deleting its only chapter: %d %+v", code, detail.Chapters)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLConcurrentChapterDelete checks two deletions of one chapter in one book.
|
||||
func TestMySQLConcurrentChapterDelete(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "race-delete-0001", "title": "Race", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
pasteInto(t, r, learner.Token, first.Book.ID, "race-delete-0002", "Second", "Second body.\n")
|
||||
drainIngest(t, db)
|
||||
|
||||
var wg sync.WaitGroup
|
||||
codes := make(chan int, 2)
|
||||
for i := 0; i < 2; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
c, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", first.Chapter.ID), learner.Token, nil)
|
||||
codes <- c
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(codes)
|
||||
ok, missing := 0, 0
|
||||
for c := range codes {
|
||||
switch c {
|
||||
case 200:
|
||||
ok++
|
||||
case 404:
|
||||
missing++
|
||||
}
|
||||
}
|
||||
if ok != 1 || missing != 1 {
|
||||
t.Fatalf("concurrent deletes: ok=%d missing=%d", ok, missing)
|
||||
}
|
||||
// The single surviving chapter keeps ordinal 1.
|
||||
var remaining []Chapter
|
||||
db.Where("book_id = ?", first.Book.ID).Order("ordinal ASC").Find(&remaining)
|
||||
if len(remaining) != 1 || remaining[0].Ordinal != 1 {
|
||||
t.Fatalf("ordinals after concurrent delete: %+v", remaining)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLRecoverySkipsSupersededJobs proves the recovery sweep leaves a newer version alone.
|
||||
func TestMySQLRecoverySkipsSupersededJobs(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "recovery-fix-0001", "title": "Recovery", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
clock := time.Now().UTC().Truncate(time.Millisecond)
|
||||
job, claimed, err := ClaimNextIngestJob(db, clock)
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("claim: %v", err)
|
||||
}
|
||||
newText := "A replacement body.\n"
|
||||
if code, msg, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"text": newText}); code != 200 {
|
||||
t.Fatalf("edit status %d (%s)", code, msg)
|
||||
}
|
||||
// The old job looks stale to the sweep; it must be abandoned, not requeued.
|
||||
requeued, err := RequeueStaleIngestJobs(db, clock.Add(ingestStaleAfter+time.Second))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if requeued != 0 {
|
||||
t.Fatalf("the sweep requeued %d superseded job(s)", requeued)
|
||||
}
|
||||
var stale IngestJob
|
||||
if err := db.First(&stale, job.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if stale.Status != statusFailed || stale.ErrorReason != reasonSuperseded {
|
||||
t.Fatalf("the sweep left the old job %+v", stale)
|
||||
}
|
||||
if row := chapterRow(t, db, pasted.Chapter.ID); row.Status != statusPending || row.OriginalText != newText {
|
||||
t.Fatalf("the sweep changed the new version: %+v", row)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
if row := chapterRow(t, db, pasted.Chapter.ID); row.Status != statusReady || row.OriginalText != newText {
|
||||
t.Fatalf("the new version did not publish: %+v", row)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,263 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// The ingestion worker is deliberately small: the fixed paste rule stores one chapter per
|
||||
// submit, so "processing" only validates the persisted content and publishes the chapter.
|
||||
// Claiming and finishing are separate transactions on purpose. A durable claim means a
|
||||
// chapter observed as processing stays recoverable, whether the process stops or only the
|
||||
// finishing transaction fails.
|
||||
|
||||
// Recovery is one mechanism used from two places: the startup pass treats every processing row
|
||||
// as abandoned, while the running worker sweeps rows that have been processing longer than any
|
||||
// legitimate claim-to-finish window. A job that keeps failing ends in a readable failure
|
||||
// instead of looping forever, and a retried job never creates a second chapter.
|
||||
const (
|
||||
maxIngestAttempts = 5
|
||||
// ingestStaleAfter must stay longer than the longest legitimate claim-to-finish window,
|
||||
// otherwise a healthy job could be processed twice. Reprocessing is harmless for content
|
||||
// because a job never creates a chapter, only publishes the one it was created with.
|
||||
ingestStaleAfter = 15 * time.Second
|
||||
)
|
||||
|
||||
// RecoverIngestJobs requeues jobs and chapters left in processing by an unclean stop. It runs
|
||||
// once before the worker starts, for a single-instance deployment.
|
||||
func RecoverIngestJobs(db *gorm.DB, now time.Time) (int64, error) {
|
||||
return requeueStaleIngestJobs(db, now, 0)
|
||||
}
|
||||
|
||||
// RequeueStaleIngestJobs recovers jobs whose finishing transaction did not complete, so a
|
||||
// running service does not depend on a restart to make progress again.
|
||||
func RequeueStaleIngestJobs(db *gorm.DB, now time.Time) (int64, error) {
|
||||
return requeueStaleIngestJobs(db, now, ingestStaleAfter)
|
||||
}
|
||||
|
||||
func requeueStaleIngestJobs(db *gorm.DB, now time.Time, staleAfter time.Duration) (int64, error) {
|
||||
ts := stamp(now)
|
||||
cutoff := stamp(now.Add(-staleAfter))
|
||||
var requeued int64
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := abandonSupersededJobs(tx, ts); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := exhaustIngestJobs(tx, ts); err != nil {
|
||||
return err
|
||||
}
|
||||
stale := []int64{}
|
||||
// Only jobs that still describe the chapter's current version may be requeued: an
|
||||
// interrupted run of an older version must not pull the newer text back into processing.
|
||||
if err := tx.Table("lexgo_ingest_jobs AS j").
|
||||
Joins("JOIN lexgo_chapters AS c ON c.id = j.chapter_id AND c.content_sha256 = j.content_sha256").
|
||||
Where("j.status = ? AND j.attempts < ? AND j.updated_at <= ?", statusProcessing, maxIngestAttempts, cutoff).
|
||||
Pluck("j.id", &stale).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(stale) == 0 {
|
||||
return nil
|
||||
}
|
||||
if err := setIngestChapterStatus(tx, stale, statusPending, "", ts); err != nil {
|
||||
return err
|
||||
}
|
||||
result := tx.Model(&IngestJob{}).Where("id IN ?", stale).
|
||||
Updates(map[string]any{"status": statusPending, "updated_at": ts})
|
||||
if result.Error != nil {
|
||||
return result.Error
|
||||
}
|
||||
requeued = result.RowsAffected
|
||||
return nil
|
||||
})
|
||||
return requeued, err
|
||||
}
|
||||
|
||||
// exhaustIngestJobs fails jobs that used up the attempt budget, so nothing can stay queued or
|
||||
// claimed forever. The recorded reason is readable and a manual retry is still allowed.
|
||||
func exhaustIngestJobs(tx *gorm.DB, ts time.Time) error {
|
||||
var exhausted []int64
|
||||
if err := tx.Model(&IngestJob{}).
|
||||
Where("status IN ? AND attempts >= ?", []string{statusPending, statusProcessing}, maxIngestAttempts).
|
||||
Pluck("id", &exhausted).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(exhausted) == 0 {
|
||||
return nil
|
||||
}
|
||||
if err := setIngestChapterStatus(tx, exhausted, statusFailed, reasonAttemptsExhausted, ts); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Model(&IngestJob{}).Where("id IN ?", exhausted).Updates(map[string]any{
|
||||
"status": statusFailed, "error_reason": reasonAttemptsExhausted, "updated_at": ts, "finished_at": ts}).Error
|
||||
}
|
||||
|
||||
// setIngestChapterStatus mirrors a job outcome onto the chapters it owns.
|
||||
func setIngestChapterStatus(tx *gorm.DB, jobIDs []int64, status, reason string, ts time.Time) error {
|
||||
var chapterIDs []int64
|
||||
if err := tx.Model(&IngestJob{}).Where("id IN ?", jobIDs).Pluck("chapter_id", &chapterIDs).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(chapterIDs) == 0 {
|
||||
return nil
|
||||
}
|
||||
return tx.Model(&Chapter{}).Where("id IN ?", chapterIDs).
|
||||
Updates(map[string]any{"status": status, "error_reason": reason, "updated_at": ts}).Error
|
||||
}
|
||||
|
||||
// ClaimNextIngestJob takes the oldest pending job with attempts left and marks it processing in
|
||||
// its own transaction. The guarded update means only one worker can own a job.
|
||||
func ClaimNextIngestJob(db *gorm.DB, now time.Time) (IngestJob, bool, error) {
|
||||
ts := stamp(now)
|
||||
var job IngestJob
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := abandonSupersededJobs(tx, ts); err != nil {
|
||||
return err
|
||||
}
|
||||
// A job may only process the version it was created for, so the chapter join is part
|
||||
// of the claim and an edited chapter is never dragged back to processing.
|
||||
if err := tx.Table("lexgo_ingest_jobs AS j").
|
||||
Select("j.id, j.owner_id, j.book_id, j.chapter_id, j.request_key, j.content_sha256, j.status, j.attempts, j.error_reason, j.created_at, j.updated_at, j.finished_at").
|
||||
Joins("JOIN lexgo_chapters AS c ON c.id = j.chapter_id AND c.content_sha256 = j.content_sha256").
|
||||
Where("j.status = ? AND j.attempts < ?", statusPending, maxIngestAttempts).
|
||||
Order("j.id ASC").Limit(1).Find(&job).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if job.ID == 0 {
|
||||
return errNoIngestJob
|
||||
}
|
||||
claim := tx.Model(&IngestJob{}).Where("id = ? AND status = ?", job.ID, statusPending).
|
||||
Updates(map[string]any{"status": statusProcessing, "attempts": gorm.Expr("attempts + 1"), "updated_at": ts})
|
||||
if claim.Error != nil {
|
||||
return claim.Error
|
||||
}
|
||||
if claim.RowsAffected != 1 {
|
||||
return errJobTaken
|
||||
}
|
||||
if err := tx.Model(&Chapter{}).Where("id = ? AND owner_id = ? AND content_sha256 = ?", job.ChapterID, job.OwnerID, job.ContentSHA256).
|
||||
Updates(map[string]any{"status": statusProcessing, "updated_at": ts}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
job.Status = statusProcessing
|
||||
job.Attempts++
|
||||
job.UpdatedAt = ts
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) || errors.Is(err, errJobTaken) || errors.Is(err, errNoIngestJob) {
|
||||
return IngestJob{}, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return IngestJob{}, false, err
|
||||
}
|
||||
return job, true, nil
|
||||
}
|
||||
|
||||
var errJobTaken = errors.New("ingestion job already claimed")
|
||||
|
||||
// errNoIngestJob reports an empty queue, which is not a failure.
|
||||
var errNoIngestJob = errors.New("no ingestion job to claim")
|
||||
|
||||
// FinishIngestJob validates the persisted chapter and publishes it, or records a fixed
|
||||
// failure reason. The check runs again here because a worker must not trust that content
|
||||
// reached the table through the paste API.
|
||||
func FinishIngestJob(ctx context.Context, db *gorm.DB, job IngestJob, now time.Time) error {
|
||||
ts := stamp(now)
|
||||
return db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
var chapter Chapter
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", job.ChapterID, job.OwnerID).First(&chapter).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
// The chapter was deleted while this run was in flight; the cascade removed its
|
||||
// jobs too, so there is nothing left to publish.
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if chapter.ContentSHA256 != job.ContentSHA256 {
|
||||
// The chapter moved to a newer version: publish nothing and leave its state, which
|
||||
// belongs to the newer job, untouched.
|
||||
return tx.Model(&IngestJob{}).Where("id = ?", job.ID).Updates(map[string]any{
|
||||
"status": statusFailed, "error_reason": reasonSuperseded, "updated_at": ts, "finished_at": ts}).Error
|
||||
}
|
||||
var book Book
|
||||
if err := tx.Where("id = ? AND owner_id = ?", job.BookID, job.OwnerID).First(&book).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
if reason := unprocessableReason(book, chapter, job); reason != "" {
|
||||
if err := tx.Model(&IngestJob{}).Where("id = ?", job.ID).Updates(map[string]any{
|
||||
"status": statusFailed, "error_reason": reason, "updated_at": ts, "finished_at": ts}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Model(&Chapter{}).Where("id = ?", chapter.ID).Updates(map[string]any{
|
||||
"status": statusFailed, "error_reason": reason, "updated_at": ts}).Error
|
||||
}
|
||||
if err := tx.Model(&Chapter{}).Where("id = ?", chapter.ID).Updates(map[string]any{
|
||||
"status": statusReady, "char_count": utf8.RuneCountInString(chapter.OriginalText), "updated_at": ts}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Model(&IngestJob{}).Where("id = ?", job.ID).Updates(map[string]any{
|
||||
"status": statusReady, "error_reason": "", "updated_at": ts, "finished_at": ts}).Error
|
||||
})
|
||||
}
|
||||
|
||||
func unprocessableReason(book Book, chapter Chapter, job IngestJob) string {
|
||||
if book.Language != "en" {
|
||||
return reasonUnsupportedLanguage
|
||||
}
|
||||
if strings.TrimFunc(chapter.OriginalText, unicode.IsSpace) == "" {
|
||||
return reasonEmptyText
|
||||
}
|
||||
if utf8.RuneCountInString(chapter.OriginalText) > maxChapterRunes {
|
||||
return reasonTooLong
|
||||
}
|
||||
// The stored text and its stored version must agree. The job-versus-chapter version check
|
||||
// ran before this function, so a direct write that changed the text without updating its
|
||||
// version is the remaining case, and it is a different paste rather than this version.
|
||||
if contentSHA(chapter.OriginalText) != chapter.ContentSHA256 {
|
||||
return reasonContentChanged
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// abandonSupersededJobs fails jobs whose version is no longer the chapter's version. They
|
||||
// must never publish or fail the chapter, because another job owns its current state. It is
|
||||
// one multi-table statement: GORM's Updates does not carry a Joins clause into an UPDATE.
|
||||
func abandonSupersededJobs(tx *gorm.DB, ts time.Time) error {
|
||||
return tx.Exec(`UPDATE lexgo_ingest_jobs j JOIN lexgo_chapters c ON c.id = j.chapter_id
|
||||
SET j.status = ?, j.error_reason = ?, j.updated_at = ?, j.finished_at = ?
|
||||
WHERE j.status IN (?, ?) AND j.content_sha256 <> c.content_sha256`,
|
||||
statusFailed, reasonSuperseded, ts, ts, statusPending, statusProcessing).Error
|
||||
}
|
||||
|
||||
// ProcessIngestJobs drains up to limit pending jobs. Claiming and finishing each use their
|
||||
// own transaction, so an interrupted run simply leaves a job for recovery.
|
||||
func ProcessIngestJobs(ctx context.Context, db *gorm.DB, now func() time.Time, limit int) (int, error) {
|
||||
processed := 0
|
||||
for i := 0; i < limit; i++ {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return processed, err
|
||||
}
|
||||
job, claimed, err := ClaimNextIngestJob(db.WithContext(ctx), now())
|
||||
if err != nil {
|
||||
return processed, err
|
||||
}
|
||||
if !claimed {
|
||||
return processed, nil
|
||||
}
|
||||
if err = FinishIngestJob(ctx, db, job, now()); err != nil {
|
||||
return processed, err
|
||||
}
|
||||
processed++
|
||||
}
|
||||
return processed, nil
|
||||
}
|
||||
@@ -0,0 +1,675 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// Chapter and job share one status vocabulary so the client renders either without mapping.
|
||||
const (
|
||||
statusPending = "pending"
|
||||
statusProcessing = "processing"
|
||||
statusReady = "ready"
|
||||
statusFailed = "failed"
|
||||
)
|
||||
|
||||
// Fixed worker-side failure reasons. Only these codes are stored; the readable text is
|
||||
// produced at the API boundary so no user content can leak into an error field.
|
||||
const (
|
||||
reasonUnsupportedLanguage = "unsupported_language"
|
||||
reasonTooLong = "too_long"
|
||||
reasonEmptyText = "empty_text"
|
||||
reasonContentChanged = "content_changed"
|
||||
// reasonSuperseded marks a job whose chapter already moved to a newer content version.
|
||||
reasonSuperseded = "superseded"
|
||||
reasonAttemptsExhausted = "attempts_exhausted"
|
||||
)
|
||||
|
||||
const (
|
||||
maxChapterRunes = 100000
|
||||
maxTitleRunes = 120
|
||||
maxBooksPerList = 200
|
||||
// A 100000 code point paste stays well inside this even with JSON escaping; the limit
|
||||
// exists only so an oversized body is rejected before it is decoded.
|
||||
maxPasteBodyBytes = 4 << 20
|
||||
maxJSONBodyBytes = 16 * 1024
|
||||
)
|
||||
|
||||
var requestIDPattern = regexp.MustCompile(`^[A-Za-z0-9_-]{8,64}$`)
|
||||
|
||||
func reasonMessage(reason string) string {
|
||||
switch reason {
|
||||
case reasonUnsupportedLanguage:
|
||||
return "当前版本只支持英语内容"
|
||||
case reasonTooLong:
|
||||
return "内容超过单章上限（100000 个字符）"
|
||||
case reasonEmptyText:
|
||||
return "章节内容为空"
|
||||
case reasonSuperseded:
|
||||
return "章节内容已更新为新版本，本次处理已作废"
|
||||
case reasonContentChanged:
|
||||
return "内容在处理前发生变化，请重新提交"
|
||||
case reasonAttemptsExhausted:
|
||||
return "处理多次失败，请重试或重新提交"
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
type Book struct {
|
||||
ID int64 `gorm:"primaryKey"`
|
||||
OwnerID int
|
||||
Title string
|
||||
Language string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (Book) TableName() string { return "lexgo_books" }
|
||||
|
||||
type Chapter struct {
|
||||
ID int64 `gorm:"primaryKey"`
|
||||
BookID int64
|
||||
OwnerID int
|
||||
Ordinal int
|
||||
Title string
|
||||
OriginalText string
|
||||
CharCount int
|
||||
ContentSHA256 string
|
||||
Status string
|
||||
ErrorReason string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (Chapter) TableName() string { return "lexgo_chapters" }
|
||||
|
||||
type IngestJob struct {
|
||||
ID int64 `gorm:"primaryKey"`
|
||||
OwnerID int
|
||||
BookID int64
|
||||
ChapterID int64
|
||||
RequestKey string
|
||||
ContentSHA256 string
|
||||
Status string
|
||||
Attempts int
|
||||
ErrorReason string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
FinishedAt *time.Time
|
||||
}
|
||||
|
||||
func (IngestJob) TableName() string { return "lexgo_ingest_jobs" }
|
||||
|
||||
type BookSummary struct {
|
||||
ID int64 `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Language string `json:"language"`
|
||||
ChapterCount int `json:"chapterCount"`
|
||||
PendingCount int `json:"pendingCount"`
|
||||
ProcessingCount int `json:"processingCount"`
|
||||
ReadyCount int `json:"readyCount"`
|
||||
FailedCount int `json:"failedCount"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
type BookRef struct {
|
||||
ID int64 `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Language string `json:"language"`
|
||||
}
|
||||
|
||||
func bookRef(book Book) BookRef { return BookRef{book.ID, book.Title, book.Language} }
|
||||
|
||||
type ChapterSummary struct {
|
||||
ID int64 `json:"id"`
|
||||
BookID int64 `json:"bookId"`
|
||||
Ordinal int `json:"ordinal"`
|
||||
Title string `json:"title"`
|
||||
Status string `json:"status"`
|
||||
CharCount int `json:"charCount"`
|
||||
// JobID lets a client retry a failed chapter without keeping the submit response.
|
||||
JobID *int64 `json:"jobId"`
|
||||
ErrorReason string `json:"errorReason"`
|
||||
ErrorMessage string `json:"errorMessage"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
type ChapterView struct {
|
||||
ChapterSummary
|
||||
ContentSHA256 string `json:"contentSha256"`
|
||||
// OriginalText is returned only for a ready chapter, so unprocessed content cannot be
|
||||
// rendered as readable text by the client.
|
||||
OriginalText string `json:"originalText,omitempty"`
|
||||
}
|
||||
|
||||
func chapterSummary(c Chapter) ChapterSummary { return chapterSummaryWithJob(c, nil) }
|
||||
|
||||
func chapterSummaryWithJob(c Chapter, jobID *int64) ChapterSummary {
|
||||
return ChapterSummary{ID: c.ID, BookID: c.BookID, Ordinal: c.Ordinal, Title: c.Title,
|
||||
Status: c.Status, CharCount: c.CharCount, JobID: jobID, ErrorReason: c.ErrorReason,
|
||||
ErrorMessage: reasonMessage(c.ErrorReason), CreatedAt: c.CreatedAt, UpdatedAt: c.UpdatedAt}
|
||||
}
|
||||
|
||||
func chapterView(c Chapter, jobID *int64) ChapterView {
|
||||
view := ChapterView{ChapterSummary: chapterSummaryWithJob(c, jobID), ContentSHA256: c.ContentSHA256}
|
||||
if c.Status == statusReady {
|
||||
view.OriginalText = c.OriginalText
|
||||
}
|
||||
return view
|
||||
}
|
||||
|
||||
type JobView struct {
|
||||
ID int64 `json:"id"`
|
||||
BookID int64 `json:"bookId"`
|
||||
ChapterID int64 `json:"chapterId"`
|
||||
Status string `json:"status"`
|
||||
Attempts int `json:"attempts"`
|
||||
ErrorReason string `json:"errorReason"`
|
||||
ErrorMessage string `json:"errorMessage"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func jobView(j IngestJob) JobView {
|
||||
return JobView{j.ID, j.BookID, j.ChapterID, j.Status, j.Attempts, j.ErrorReason, reasonMessage(j.ErrorReason), j.CreatedAt, j.UpdatedAt}
|
||||
}
|
||||
|
||||
type Navigation struct {
|
||||
PreviousChapterID *int64 `json:"previousChapterId"`
|
||||
NextChapterID *int64 `json:"nextChapterId"`
|
||||
}
|
||||
|
||||
// ReaderResponse carries everything the reader needs for one chapter of the caller's own book.
|
||||
type ReaderResponse struct {
|
||||
Book BookRef `json:"book"`
|
||||
Chapter ChapterView `json:"chapter"`
|
||||
Navigation Navigation `json:"navigation"`
|
||||
}
|
||||
|
||||
type PasteResult struct {
|
||||
Book *BookRef `json:"book,omitempty"`
|
||||
Chapter ChapterSummary `json:"chapter"`
|
||||
Job JobView `json:"job"`
|
||||
Duplicate bool `json:"duplicate"`
|
||||
}
|
||||
|
||||
type PasteBookInput struct {
|
||||
RequestID string `json:"requestId"`
|
||||
Title string `json:"title"`
|
||||
Text string `json:"text"`
|
||||
Language string `json:"language"`
|
||||
}
|
||||
|
||||
type PasteChapterInput struct {
|
||||
RequestID string `json:"requestId"`
|
||||
Title string `json:"title"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
func stamp(now time.Time) time.Time { return now.UTC().Truncate(time.Millisecond) }
|
||||
|
||||
func contentSHA(text string) string {
|
||||
v := sha256.Sum256([]byte(text))
|
||||
return hex.EncodeToString(v[:])
|
||||
}
|
||||
|
||||
func requestKey(requestID string) (string, error) {
|
||||
if !requestIDPattern.MatchString(requestID) {
|
||||
return "", failure(400, "请求编号须为 8～64 位字母、数字、下划线或连字符")
|
||||
}
|
||||
return contentSHA(requestID), nil
|
||||
}
|
||||
|
||||
// validatePaste applies the fixed paste rules: a title within the cap, at least one
|
||||
// non-space character, and at most maxChapterRunes code points. The text itself is stored
|
||||
// exactly as received, so whitespace, punctuation and line breaks survive unchanged.
|
||||
func validatePaste(title, text string) (string, string, int, error) {
|
||||
name := strings.TrimSpace(title)
|
||||
if name == "" {
|
||||
return "", "", 0, failure(400, "请填写标题")
|
||||
}
|
||||
if utf8.RuneCountInString(name) > maxTitleRunes {
|
||||
return "", "", 0, failure(400, "标题最多 120 个字符")
|
||||
}
|
||||
if strings.TrimFunc(text, unicode.IsSpace) == "" {
|
||||
return "", "", 0, failure(400, "请输入正文内容")
|
||||
}
|
||||
count := utf8.RuneCountInString(text)
|
||||
if count > maxChapterRunes {
|
||||
return "", "", 0, failure(400, "正文超过单章上限（100000 个字符）")
|
||||
}
|
||||
return name, contentSHA(text), count, nil
|
||||
}
|
||||
|
||||
// PasteBook creates one book with its first chapter and the ingestion job. The paste and the
|
||||
// job are written in one transaction, so a rejected submit leaves no book behind.
|
||||
func PasteBook(db *gorm.DB, owner int, now time.Time, input PasteBookInput) (PasteResult, error) {
|
||||
title, sha, count, err := validatePaste(input.Title, input.Text)
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
if input.Language != "" && input.Language != "en" {
|
||||
return PasteResult{}, failure(400, "当前版本只支持英语内容")
|
||||
}
|
||||
key, err := requestKey(input.RequestID)
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
ts := stamp(now)
|
||||
var result PasteResult
|
||||
err = db.Transaction(func(tx *gorm.DB) error {
|
||||
existing, found, err := jobByRequest(tx, owner, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if found {
|
||||
reused, err := reusePaste(tx, existing, title, sha)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result = reused
|
||||
return nil
|
||||
}
|
||||
book := Book{OwnerID: owner, Title: title, Language: "en", CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&book).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: owner, Ordinal: 1, Title: title,
|
||||
OriginalText: input.Text, CharCount: count, ContentSHA256: sha,
|
||||
Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&chapter).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
job := IngestJob{OwnerID: owner, BookID: book.ID, ChapterID: chapter.ID, RequestKey: key,
|
||||
ContentSHA256: sha, Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&job).Error; err != nil {
|
||||
return pasteInsertError(err)
|
||||
}
|
||||
ref := bookRef(book)
|
||||
result = PasteResult{Book: &ref, Chapter: chapterSummaryWithJob(chapter, &job.ID), Job: jobView(job)}
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, errRequestReuse) {
|
||||
return reusePasteByRequest(db, owner, key, title, sha)
|
||||
}
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// PasteChapter appends one chapter to a book the caller already owns. The book row is locked
|
||||
// so two appends cannot claim the same ordinal.
|
||||
func PasteChapter(db *gorm.DB, owner int, bookID int64, now time.Time, input PasteChapterInput) (PasteResult, error) {
|
||||
title, sha, count, err := validatePaste(input.Title, input.Text)
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
key, err := requestKey(input.RequestID)
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
ts := stamp(now)
|
||||
var result PasteResult
|
||||
err = db.Transaction(func(tx *gorm.DB) error {
|
||||
var book Book
|
||||
if err = lockOwnedBook(tx, owner, bookID, &book); err != nil {
|
||||
return err
|
||||
}
|
||||
existing, found, err := jobByRequest(tx, owner, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if found {
|
||||
if existing.BookID != book.ID {
|
||||
return failure(409, "该请求编号已用于其他内容")
|
||||
}
|
||||
reused, err := reusePaste(tx, existing, title, sha)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result = reused
|
||||
return nil
|
||||
}
|
||||
var last int
|
||||
row := tx.Model(&Chapter{}).Where("book_id = ?", book.ID).Select("COALESCE(MAX(ordinal),0)").Row()
|
||||
if err = row.Scan(&last); err != nil {
|
||||
return err
|
||||
}
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: owner, Ordinal: last + 1, Title: title,
|
||||
OriginalText: input.Text, CharCount: count, ContentSHA256: sha,
|
||||
Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&chapter).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
job := IngestJob{OwnerID: owner, BookID: book.ID, ChapterID: chapter.ID, RequestKey: key,
|
||||
ContentSHA256: sha, Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&job).Error; err != nil {
|
||||
return pasteInsertError(err)
|
||||
}
|
||||
if err = tx.Model(&Book{}).Where("id = ?", book.ID).Update("updated_at", ts).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
result = PasteResult{Chapter: chapterSummaryWithJob(chapter, &job.ID), Job: jobView(job)}
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, errRequestReuse) {
|
||||
return reusePasteByRequest(db, owner, key, title, sha)
|
||||
}
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
var errRequestReuse = errors.New("ingestion request already accepted")
|
||||
|
||||
// pasteInsertError turns a unique-key conflict on the job insert into a request reuse. The
|
||||
// transaction must be abandoned: a concurrent submit that already committed is invisible to
|
||||
// this transaction's snapshot, and its rows are re-read with locking reads below.
|
||||
func pasteInsertError(err error) error {
|
||||
var dup *driver.MySQLError
|
||||
if errors.As(err, &dup) && dup.Number == 1062 {
|
||||
return errRequestReuse
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func reusePasteByRequest(db *gorm.DB, owner int, key, title, sha string) (PasteResult, error) {
|
||||
var result PasteResult
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
job, found, err := jobByRequestLatest(tx, owner, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !found {
|
||||
// The conflict came from another unique key, not from a repeated request id.
|
||||
return failure(409, "提交冲突，请重试")
|
||||
}
|
||||
result, err = reusePaste(tx, job, title, sha)
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func jobByRequest(tx *gorm.DB, owner int, key string) (IngestJob, bool, error) {
|
||||
return jobQuery(tx, owner, key, false)
|
||||
}
|
||||
|
||||
// jobByRequestLatest uses a locking read, which sees the latest committed row instead of this
|
||||
// transaction's older snapshot. It is required after a duplicate-key conflict: only the
|
||||
// competing transaction's commit can cause that conflict, and its rows are newer than the
|
||||
// snapshot this request already took.
|
||||
func jobByRequestLatest(tx *gorm.DB, owner int, key string) (IngestJob, bool, error) {
|
||||
return jobQuery(tx, owner, key, true)
|
||||
}
|
||||
|
||||
func jobQuery(tx *gorm.DB, owner int, key string, latest bool) (IngestJob, bool, error) {
|
||||
var job IngestJob
|
||||
query := tx
|
||||
if latest {
|
||||
query = tx.Clauses(clause.Locking{Strength: "UPDATE"})
|
||||
}
|
||||
err := query.Where("owner_id = ? AND request_key = ?", owner, key).First(&job).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return job, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return job, false, err
|
||||
}
|
||||
return job, true, nil
|
||||
}
|
||||
|
||||
// reusePaste answers a repeated submit with the first result instead of creating a second
|
||||
// chapter. A reused request id with different content or title is a conflict, not a retry.
|
||||
// Its reads are locking reads so the same answer works right after a duplicate-key conflict.
|
||||
func reusePaste(tx *gorm.DB, job IngestJob, title, sha string) (PasteResult, error) {
|
||||
var chapter Chapter
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", job.ChapterID, job.OwnerID).First(&chapter).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return PasteResult{}, failure(409, "该请求已提交过，请刷新后查看结果")
|
||||
}
|
||||
return PasteResult{}, err
|
||||
}
|
||||
if chapter.ContentSHA256 != sha || chapter.Title != title {
|
||||
return PasteResult{}, failure(409, "该请求编号已用于其他内容")
|
||||
}
|
||||
var book Book
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", job.BookID, job.OwnerID).First(&book).Error; err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
ref := bookRef(book)
|
||||
result := PasteResult{Chapter: chapterSummaryWithJob(chapter, &job.ID), Job: jobView(job), Duplicate: true}
|
||||
if chapter.Ordinal == 1 {
|
||||
result.Book = &ref
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func lockOwnedBook(tx *gorm.DB, owner int, bookID int64, book *Book) error {
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", bookID, owner).First(book).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
// Another user's book and a missing book are indistinguishable to the caller.
|
||||
return failure(404, "书籍不存在")
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func ListBooks(db *gorm.DB, owner int) ([]BookSummary, error) {
|
||||
var books []Book
|
||||
if err := db.Where("owner_id = ?", owner).Order("updated_at DESC, id DESC").Limit(maxBooksPerList).Find(&books).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items := make([]BookSummary, 0, len(books))
|
||||
ids := make([]int64, 0, len(books))
|
||||
for _, b := range books {
|
||||
items = append(items, BookSummary{ID: b.ID, Title: b.Title, Language: b.Language, CreatedAt: b.CreatedAt, UpdatedAt: b.UpdatedAt})
|
||||
ids = append(ids, b.ID)
|
||||
}
|
||||
if len(ids) == 0 {
|
||||
return items, nil
|
||||
}
|
||||
type row struct {
|
||||
BookID int64
|
||||
Status string
|
||||
Total int
|
||||
}
|
||||
var rows []row
|
||||
if err := db.Model(&Chapter{}).Select("book_id, status, COUNT(*) AS total").
|
||||
Where("owner_id = ? AND book_id IN ?", owner, ids).Group("book_id, status").Scan(&rows).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
index := make(map[int64]int, len(items))
|
||||
for i, item := range items {
|
||||
index[item.ID] = i
|
||||
}
|
||||
for _, r := range rows {
|
||||
i, ok := index[r.BookID]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
items[i].ChapterCount += r.Total
|
||||
switch r.Status {
|
||||
case statusReady:
|
||||
items[i].ReadyCount += r.Total
|
||||
case statusProcessing:
|
||||
items[i].ProcessingCount += r.Total
|
||||
case statusFailed:
|
||||
items[i].FailedCount += r.Total
|
||||
default:
|
||||
items[i].PendingCount += r.Total
|
||||
}
|
||||
}
|
||||
return items, nil
|
||||
}
|
||||
|
||||
func BookDetail(db *gorm.DB, owner int, bookID int64) (BookRef, []ChapterSummary, error) {
|
||||
var book Book
|
||||
if err := db.Where("id = ? AND owner_id = ?", bookID, owner).First(&book).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return BookRef{}, nil, failure(404, "书籍不存在")
|
||||
}
|
||||
return BookRef{}, nil, err
|
||||
}
|
||||
var chapters []Chapter
|
||||
if err := db.Where("book_id = ? AND owner_id = ?", book.ID, owner).Order("ordinal ASC").Find(&chapters).Error; err != nil {
|
||||
return BookRef{}, nil, err
|
||||
}
|
||||
items := make([]ChapterSummary, 0, len(chapters))
|
||||
ids := make([]int64, 0, len(chapters))
|
||||
for _, c := range chapters {
|
||||
ids = append(ids, c.ID)
|
||||
}
|
||||
jobs, err := jobIDsByChapter(db, owner, ids)
|
||||
if err != nil {
|
||||
return BookRef{}, nil, err
|
||||
}
|
||||
for _, c := range chapters {
|
||||
var jobID *int64
|
||||
if id, ok := jobs[c.ID]; ok {
|
||||
jobID = &id
|
||||
}
|
||||
items = append(items, chapterSummaryWithJob(c, jobID))
|
||||
}
|
||||
return bookRef(book), items, nil
|
||||
}
|
||||
|
||||
// jobIDsByChapter maps chapters to their ingestion job so a client can retry a failed chapter
|
||||
// without having kept the original submit response.
|
||||
func jobIDsByChapter(db *gorm.DB, owner int, chapterIDs []int64) (map[int64]int64, error) {
|
||||
ids := make(map[int64]int64, len(chapterIDs))
|
||||
if len(chapterIDs) == 0 {
|
||||
return ids, nil
|
||||
}
|
||||
var jobs []IngestJob
|
||||
if err := db.Select("id", "chapter_id").Where("owner_id = ? AND chapter_id IN ?", owner, chapterIDs).
|
||||
Order("id ASC").Find(&jobs).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Ascending order keeps the newest id if a chapter somehow has more than one job.
|
||||
for _, j := range jobs {
|
||||
ids[j.ChapterID] = j.ID
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
// ChapterDetail resolves a chapter strictly inside the caller's own books and returns the
|
||||
// original text only once the chapter is ready.
|
||||
func ChapterDetail(db *gorm.DB, owner int, chapterID int64) (ReaderResponse, error) {
|
||||
var chapter Chapter
|
||||
if err := db.Where("id = ? AND owner_id = ?", chapterID, owner).First(&chapter).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReaderResponse{}, failure(404, "章节不存在")
|
||||
}
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
var book Book
|
||||
if err := db.Where("id = ? AND owner_id = ?", chapter.BookID, owner).First(&book).Error; err != nil {
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
navigation := Navigation{}
|
||||
var previous, next Chapter
|
||||
if err := db.Where("book_id = ? AND owner_id = ? AND ordinal < ?", book.ID, owner, chapter.Ordinal).
|
||||
Order("ordinal DESC").First(&previous).Error; err == nil {
|
||||
navigation.PreviousChapterID = &previous.ID
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
if err := db.Where("book_id = ? AND owner_id = ? AND ordinal > ?", book.ID, owner, chapter.Ordinal).
|
||||
Order("ordinal ASC").First(&next).Error; err == nil {
|
||||
navigation.NextChapterID = &next.ID
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
jobs, err := jobIDsByChapter(db, owner, []int64{chapter.ID})
|
||||
if err != nil {
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
var jobID *int64
|
||||
if id, ok := jobs[chapter.ID]; ok {
|
||||
jobID = &id
|
||||
}
|
||||
return ReaderResponse{Book: bookRef(book), Chapter: chapterView(chapter, jobID), Navigation: navigation}, nil
|
||||
}
|
||||
|
||||
func JobDetail(db *gorm.DB, owner int, jobID int64) (JobView, error) {
|
||||
var job IngestJob
|
||||
if err := db.Where("id = ? AND owner_id = ?", jobID, owner).First(&job).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return JobView{}, failure(404, "任务不存在")
|
||||
}
|
||||
return JobView{}, err
|
||||
}
|
||||
return jobView(job), nil
|
||||
}
|
||||
|
||||
// RetryIngestJob requeues a failed job on the same chapter, so a retry can never create a
|
||||
// second chapter for one paste. An explicit retry also restarts the attempt budget, because a
|
||||
// person asking again should not be blocked by the bound that stops automatic loops.
|
||||
func RetryIngestJob(db *gorm.DB, owner int, jobID int64, now time.Time) (JobView, ChapterSummary, error) {
|
||||
ts := stamp(now)
|
||||
var job IngestJob
|
||||
var chapter Chapter
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", jobID, owner).First(&job).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return failure(404, "任务不存在")
|
||||
}
|
||||
return err
|
||||
}
|
||||
if job.Status != statusFailed {
|
||||
return failure(409, "只有失败的任务可以重试")
|
||||
}
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", job.ChapterID, owner).First(&chapter).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
// A job of an older content version is gone for good: retrying it would process text
|
||||
// the chapter no longer holds, so the newer job owns the chapter instead.
|
||||
if chapter.ContentSHA256 != job.ContentSHA256 {
|
||||
return failure(409, "该任务对应的是旧版本，请刷新后重试当前版本")
|
||||
}
|
||||
if err := tx.Model(&IngestJob{}).Where("id = ?", job.ID).
|
||||
Updates(map[string]any{"status": statusPending, "error_reason": "", "attempts": 0,
|
||||
"updated_at": ts, "finished_at": nil}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if err := tx.Model(&Chapter{}).Where("id = ?", chapter.ID).
|
||||
Updates(map[string]any{"status": statusPending, "error_reason": "", "updated_at": ts}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
job.Status = statusPending
|
||||
job.ErrorReason = ""
|
||||
job.Attempts = 0
|
||||
job.FinishedAt = nil
|
||||
job.UpdatedAt = ts
|
||||
chapter.Status = statusPending
|
||||
chapter.ErrorReason = ""
|
||||
chapter.UpdatedAt = ts
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return JobView{}, ChapterSummary{}, err
|
||||
}
|
||||
return jobView(job), chapterSummaryWithJob(chapter, &job.ID), nil
|
||||
}
|
||||
@@ -0,0 +1,914 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Fictional English fixture with the characters the paste contract must preserve exactly:
|
||||
// CRLF and LF, a tab, curly quotes, an em dash, an ellipsis, an emoji, a combining acute
|
||||
// accent, a trailing space run and an empty final line.
|
||||
const fixturePastedText = "Mira opened the workshop.\r\n\r\n\tThe sign read “A small step…” — café e\u0301 🙂\r\nTrailing spaces here: \n\n"
|
||||
|
||||
type libraryAccount struct {
|
||||
ID int
|
||||
Username string
|
||||
Token string
|
||||
}
|
||||
|
||||
type pasteResponse struct {
|
||||
Book *struct {
|
||||
ID int64
|
||||
Title string
|
||||
Language string
|
||||
}
|
||||
Chapter struct {
|
||||
ID int64
|
||||
BookID int64
|
||||
Ordinal int
|
||||
Title string
|
||||
Status string
|
||||
CharCount int
|
||||
ErrorReason string
|
||||
ErrorMessage string
|
||||
}
|
||||
Job struct {
|
||||
ID int64
|
||||
BookID int64
|
||||
ChapterID int64
|
||||
Status string
|
||||
Attempts int
|
||||
ErrorReason string
|
||||
ErrorMessage string
|
||||
}
|
||||
Duplicate bool
|
||||
}
|
||||
|
||||
type readerResponse struct {
|
||||
Book struct {
|
||||
ID int64
|
||||
Title string
|
||||
Language string
|
||||
}
|
||||
Chapter struct {
|
||||
ID int64
|
||||
BookID int64
|
||||
Ordinal int
|
||||
Title string
|
||||
Status string
|
||||
CharCount int
|
||||
ErrorReason string
|
||||
ErrorMessage string
|
||||
ContentSHA256 string
|
||||
OriginalText string
|
||||
JobID *int64
|
||||
}
|
||||
Navigation struct {
|
||||
PreviousChapterID *int64
|
||||
NextChapterID *int64
|
||||
}
|
||||
}
|
||||
|
||||
type bookDetailResponse struct {
|
||||
Book struct {
|
||||
ID int64
|
||||
Title string
|
||||
Language string
|
||||
}
|
||||
Chapters []struct {
|
||||
ID int64
|
||||
Ordinal int
|
||||
Title string
|
||||
Status string
|
||||
CharCount int
|
||||
JobID *int64
|
||||
ErrorMessage string
|
||||
}
|
||||
}
|
||||
|
||||
type bookListResponse struct {
|
||||
Items []struct {
|
||||
ID int64
|
||||
Title string
|
||||
ChapterCount int
|
||||
PendingCount int
|
||||
ProcessingCount int
|
||||
ReadyCount int
|
||||
FailedCount int
|
||||
}
|
||||
}
|
||||
|
||||
// callRaw keeps the API message, which is how a readable failure reason is asserted.
|
||||
func callRaw(t *testing.T, r *gin.Engine, method, path, token string, body any) (int, string, json.RawMessage) {
|
||||
t.Helper()
|
||||
b, _ := json.Marshal(body)
|
||||
q := httptest.NewRequest(method, path, bytes.NewReader(b))
|
||||
q.Header.Set("Content-Type", "application/json")
|
||||
if token != "" {
|
||||
q.Header.Set("Authorization", "Bearer "+token)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, q)
|
||||
var e struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data json.RawMessage `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &e); err != nil {
|
||||
t.Fatalf("invalid JSON for %s %s (status %d)", method, path, w.Code)
|
||||
}
|
||||
return w.Code, e.Msg, e.Data
|
||||
}
|
||||
|
||||
func libraryFixture(t *testing.T) (*gorm.DB, *gin.Engine, libraryAccount) {
|
||||
t.Helper()
|
||||
db := testDB(t)
|
||||
owner := admin.SysUser{Username: randomName("admin"), Password: fixturePassword, RoleId: 1, Status: "2"}
|
||||
if err := db.Create(&owner).Error; err != nil {
|
||||
t.Fatal("fixture admin creation failed")
|
||||
}
|
||||
r := Router(db, time.Now)
|
||||
return db, r, libraryAccount{owner.UserId, owner.Username, loginToken(t, r, owner.Username, fixturePassword)}
|
||||
}
|
||||
|
||||
func newLearner(t *testing.T, r *gin.Engine, adminToken string) libraryAccount {
|
||||
t.Helper()
|
||||
name := randomName("lib")
|
||||
code, msg, data := callRaw(t, r, "POST", "/api/v1/accounts", adminToken, map[string]string{"username": name, "password": fixturePassword})
|
||||
if code != 201 {
|
||||
t.Fatalf("create learner status %d (%s)", code, msg)
|
||||
}
|
||||
var created struct {
|
||||
ID int
|
||||
Username string
|
||||
}
|
||||
json.Unmarshal(data, &created)
|
||||
return libraryAccount{created.ID, created.Username, loginToken(t, r, name, fixturePassword)}
|
||||
}
|
||||
|
||||
func pasteBook(t *testing.T, r *gin.Engine, token string, body any) (int, pasteResponse) {
|
||||
t.Helper()
|
||||
code, msg, data := callRaw(t, r, "POST", "/api/v1/books", token, body)
|
||||
var out pasteResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("paste response: %v", err)
|
||||
}
|
||||
}
|
||||
if code >= 400 && msg == "" {
|
||||
t.Fatalf("paste failed with status %d and no message", code)
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func pasteChapter(t *testing.T, r *gin.Engine, token string, bookID int64, body any) (int, pasteResponse) {
|
||||
t.Helper()
|
||||
code, msg, data := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/books/%d/chapters", bookID), token, body)
|
||||
var out pasteResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("paste chapter response: %v", err)
|
||||
}
|
||||
}
|
||||
if code >= 400 && msg == "" {
|
||||
t.Fatalf("paste chapter failed with status %d and no message", code)
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func readChapter(t *testing.T, r *gin.Engine, token string, chapterID int64) (int, readerResponse) {
|
||||
t.Helper()
|
||||
code, _, data := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d", chapterID), token, nil)
|
||||
var out readerResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("reader response: %v", err)
|
||||
}
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func bookDetail(t *testing.T, r *gin.Engine, token string, bookID int64) (int, bookDetailResponse) {
|
||||
t.Helper()
|
||||
code, _, data := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/books/%d", bookID), token, nil)
|
||||
var out bookDetailResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("book response: %v", err)
|
||||
}
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func bookList(t *testing.T, r *gin.Engine, token string) (int, bookListResponse) {
|
||||
t.Helper()
|
||||
code, _, data := callRaw(t, r, "GET", "/api/v1/books", token, nil)
|
||||
var out bookListResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("book list response: %v", err)
|
||||
}
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func drainIngest(t *testing.T, db *gorm.DB) {
|
||||
t.Helper()
|
||||
if _, err := ProcessIngestJobs(t.Context(), db, time.Now, 50); err != nil {
|
||||
t.Fatalf("ingestion failed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func chapterRow(t *testing.T, db *gorm.DB, id int64) Chapter {
|
||||
t.Helper()
|
||||
var c Chapter
|
||||
if err := db.Where("id = ?", id).First(&c).Error; err != nil {
|
||||
t.Fatalf("chapter row: %v", err)
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func jobRow(t *testing.T, db *gorm.DB, id int64) IngestJob {
|
||||
t.Helper()
|
||||
var j IngestJob
|
||||
if err := db.Where("id = ?", id).First(&j).Error; err != nil {
|
||||
t.Fatalf("job row: %v", err)
|
||||
}
|
||||
return j
|
||||
}
|
||||
|
||||
func TestMySQLPasteToReaderFullPath(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
// Empty the queue so the claim below takes this test's own job.
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-request-full-path", "title": "The Workshop", "text": fixturePastedText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d, want 201", code)
|
||||
}
|
||||
if pasted.Book == nil || pasted.Book.ID == 0 || pasted.Book.Title != "The Workshop" || pasted.Book.Language != "en" {
|
||||
t.Fatalf("unexpected book %+v", pasted.Book)
|
||||
}
|
||||
if pasted.Chapter.Ordinal != 1 || pasted.Chapter.BookID != pasted.Book.ID || pasted.Chapter.Title != "The Workshop" {
|
||||
t.Fatalf("unexpected chapter %+v", pasted.Chapter)
|
||||
}
|
||||
if pasted.Chapter.Status != statusPending || pasted.Job.Status != statusPending || pasted.Duplicate {
|
||||
t.Fatalf("paste must queue a pending job, got chapter %q job %q", pasted.Chapter.Status, pasted.Job.Status)
|
||||
}
|
||||
if want := utf8.RuneCountInString(fixturePastedText); pasted.Chapter.CharCount != want {
|
||||
t.Fatalf("charCount %d, want %d", pasted.Chapter.CharCount, want)
|
||||
}
|
||||
|
||||
// A queued chapter has no readable text yet.
|
||||
code, pending := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || pending.Chapter.Status != statusPending || pending.Chapter.OriginalText != "" {
|
||||
t.Fatalf("pending chapter must not expose text: status %d, %+v", code, pending.Chapter)
|
||||
}
|
||||
_, queued := bookList(t, r, learner.Token)
|
||||
if len(queued.Items) != 1 || queued.Items[0].PendingCount != 1 || queued.Items[0].ProcessingCount != 0 || queued.Items[0].ReadyCount != 0 {
|
||||
t.Fatalf("queued book counts %+v", queued.Items)
|
||||
}
|
||||
|
||||
// Processing is a durable state: a claim survives a crash and is observable in between.
|
||||
job, claimed, err := ClaimNextIngestJob(db, time.Now())
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("claim failed (claimed=%v): %v", claimed, err)
|
||||
}
|
||||
if job.ID != pasted.Job.ID || job.Attempts != 1 {
|
||||
t.Fatalf("claimed job %+v, want id %d with 1 attempt", job, pasted.Job.ID)
|
||||
}
|
||||
code, processing := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || processing.Chapter.Status != statusProcessing || processing.Chapter.OriginalText != "" {
|
||||
t.Fatalf("processing chapter must not expose text: status %d, %+v", code, processing.Chapter)
|
||||
}
|
||||
if err = FinishIngestJob(t.Context(), db, job, time.Now()); err != nil {
|
||||
t.Fatalf("finish failed: %v", err)
|
||||
}
|
||||
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady {
|
||||
t.Fatalf("ready chapter status %d %q", code, ready.Chapter.Status)
|
||||
}
|
||||
if ready.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("original text changed:\n got %q\nwant %q", ready.Chapter.OriginalText, fixturePastedText)
|
||||
}
|
||||
if ready.Chapter.ContentSHA256 != contentSHA(fixturePastedText) {
|
||||
t.Fatal("content hash mismatch")
|
||||
}
|
||||
if ready.Chapter.CharCount != utf8.RuneCountInString(fixturePastedText) {
|
||||
t.Fatalf("charCount %d after processing", ready.Chapter.CharCount)
|
||||
}
|
||||
if ready.Navigation.PreviousChapterID != nil || ready.Navigation.NextChapterID != nil {
|
||||
t.Fatal("single chapter must not navigate")
|
||||
}
|
||||
if ready.Book.ID != pasted.Book.ID || ready.Book.Title != "The Workshop" {
|
||||
t.Fatalf("unexpected reader book %+v", ready.Book)
|
||||
}
|
||||
|
||||
code, _, data := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/jobs/%d", pasted.Job.ID), learner.Token, nil)
|
||||
var jobView struct {
|
||||
Job struct {
|
||||
Status string
|
||||
Attempts int
|
||||
ChapterID int64
|
||||
}
|
||||
}
|
||||
json.Unmarshal(data, &jobView)
|
||||
if code != 200 || jobView.Job.Status != statusReady || jobView.Job.Attempts != 1 || jobView.Job.ChapterID != pasted.Chapter.ID {
|
||||
t.Fatalf("job view %+v (status %d)", jobView.Job, code)
|
||||
}
|
||||
|
||||
// Appending keeps the fixed rule: one paste, one more chapter.
|
||||
code, appended := pasteChapter(t, r, learner.Token, pasted.Book.ID, map[string]string{
|
||||
"requestId": "fixture-request-append", "title": "Second Chapter", "text": "A single plain paragraph.\n"})
|
||||
if code != 201 || appended.Chapter.Ordinal != 2 || appended.Book != nil {
|
||||
t.Fatalf("append status %d chapter %+v book %+v", code, appended.Chapter, appended.Book)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, second := readChapter(t, r, learner.Token, appended.Chapter.ID)
|
||||
if code != 200 || second.Chapter.Status != statusReady || second.Chapter.OriginalText != "A single plain paragraph.\n" {
|
||||
t.Fatalf("appended chapter %+v", second.Chapter)
|
||||
}
|
||||
if second.Navigation.PreviousChapterID == nil || *second.Navigation.PreviousChapterID != pasted.Chapter.ID || second.Navigation.NextChapterID != nil {
|
||||
t.Fatalf("appended navigation %+v", second.Navigation)
|
||||
}
|
||||
code, first := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || first.Navigation.NextChapterID == nil || *first.Navigation.NextChapterID != appended.Chapter.ID {
|
||||
t.Fatalf("first chapter navigation %+v", first.Navigation)
|
||||
}
|
||||
|
||||
code, detail := bookDetail(t, r, learner.Token, pasted.Book.ID)
|
||||
if code != 200 || len(detail.Chapters) != 2 || detail.Chapters[0].Status != statusReady || detail.Chapters[1].Ordinal != 2 {
|
||||
t.Fatalf("book detail %+v", detail)
|
||||
}
|
||||
// The chapter list carries the job id, which is what a client needs to retry a failure.
|
||||
if detail.Chapters[0].JobID == nil || *detail.Chapters[0].JobID != pasted.Job.ID {
|
||||
t.Fatalf("book detail chapter job id %+v, want %d", detail.Chapters[0].JobID, pasted.Job.ID)
|
||||
}
|
||||
if ready.Chapter.JobID == nil || *ready.Chapter.JobID != pasted.Job.ID {
|
||||
t.Fatalf("reader chapter job id %+v, want %d", ready.Chapter.JobID, pasted.Job.ID)
|
||||
}
|
||||
code, list := bookList(t, r, learner.Token)
|
||||
if code != 200 || len(list.Items) != 1 || list.Items[0].ChapterCount != 2 || list.Items[0].ReadyCount != 2 || list.Items[0].FailedCount != 0 {
|
||||
t.Fatalf("book list %+v", list)
|
||||
}
|
||||
if list.Items[0].PendingCount != 0 || list.Items[0].ProcessingCount != 0 {
|
||||
t.Fatalf("published book must have no queued chapter: %+v", list.Items[0])
|
||||
}
|
||||
if list.Items[0].ID != pasted.Book.ID {
|
||||
t.Fatal("book list must only contain the caller's own book")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLIngestFailureReasonsAndRetry(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
reason string
|
||||
mutate func(book *Book, chapter *Chapter, job *IngestJob)
|
||||
}{
|
||||
{"valid", "", func(*Book, *Chapter, *IngestJob) {}},
|
||||
{"unsupported_language", reasonUnsupportedLanguage, func(book *Book, _ *Chapter, _ *IngestJob) {
|
||||
book.Language = "de"
|
||||
}},
|
||||
{"empty_text", reasonEmptyText, func(_ *Book, chapter *Chapter, job *IngestJob) {
|
||||
chapter.OriginalText = " \r\n\t "
|
||||
chapter.ContentSHA256 = contentSHA(chapter.OriginalText)
|
||||
job.ContentSHA256 = chapter.ContentSHA256
|
||||
}},
|
||||
{"too_long", reasonTooLong, func(_ *Book, chapter *Chapter, job *IngestJob) {
|
||||
chapter.OriginalText = strings.Repeat("a", maxChapterRunes+1)
|
||||
chapter.ContentSHA256 = contentSHA(chapter.OriginalText)
|
||||
job.ContentSHA256 = chapter.ContentSHA256
|
||||
}},
|
||||
{"content_changed", reasonContentChanged, func(_ *Book, chapter *Chapter, _ *IngestJob) {
|
||||
chapter.OriginalText = "Mira opened"
|
||||
}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": randomName("case"), "title": "Case " + tc.name, "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// These are the states another writing path could leave behind; the worker must
|
||||
// re-validate persisted content instead of trusting the paste API.
|
||||
book := Book{ID: pasted.Book.ID, OwnerID: learner.ID, Language: "en"}
|
||||
chapter := chapterRow(t, db, pasted.Chapter.ID)
|
||||
job := jobRow(t, db, pasted.Job.ID)
|
||||
tc.mutate(&book, &chapter, &job)
|
||||
if err := db.Model(&Book{}).Where("id = ?", book.ID).Update("language", book.Language).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", chapter.ID).
|
||||
Updates(map[string]any{"original_text": chapter.OriginalText, "content_sha256": chapter.ContentSHA256}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Model(&IngestJob{}).Where("id = ?", job.ID).
|
||||
Update("content_sha256", job.ContentSHA256).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, read := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 {
|
||||
t.Fatalf("read status %d", code)
|
||||
}
|
||||
if tc.reason == "" {
|
||||
if read.Chapter.Status != statusReady || read.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("valid content must reach ready, got %+v", read.Chapter)
|
||||
}
|
||||
return
|
||||
}
|
||||
if read.Chapter.Status != statusFailed || read.Chapter.ErrorReason != tc.reason {
|
||||
t.Fatalf("chapter %q with reason %q, want failed/%s", read.Chapter.Status, read.Chapter.ErrorReason, tc.reason)
|
||||
}
|
||||
// The readable reason is produced by the API and never stores user content.
|
||||
if read.Chapter.ErrorMessage == "" || strings.ContainsAny(read.Chapter.ErrorMessage, "\r\n") {
|
||||
t.Fatalf("unreadable failure message %q", read.Chapter.ErrorMessage)
|
||||
}
|
||||
if read.Chapter.OriginalText != "" {
|
||||
t.Fatal("failed chapter must not expose text")
|
||||
}
|
||||
var detail bookDetailResponse
|
||||
_, detail = bookDetail(t, r, learner.Token, pasted.Book.ID)
|
||||
if detail.Chapters[0].Status != statusFailed || detail.Chapters[0].ErrorMessage == "" {
|
||||
t.Fatalf("book detail must show the failure: %+v", detail.Chapters[0])
|
||||
}
|
||||
// Retry through the job id the chapter list exposes, which is the client's only path.
|
||||
if detail.Chapters[0].JobID == nil || *detail.Chapters[0].JobID != pasted.Job.ID {
|
||||
t.Fatalf("failed chapter must expose its job id: %+v", detail.Chapters[0])
|
||||
}
|
||||
code, _, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", *detail.Chapters[0].JobID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("retry status %d, want 200", code)
|
||||
}
|
||||
// Retrying reuses the same chapter: no second chapter for one paste.
|
||||
var count int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatalf("retry created %d chapters, want 1", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLRetryAfterContentRestoredPublishesSameChapter(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-retry", "title": "Retry Book", "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// Simulate a chapter whose stored text was replaced before processing.
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", pasted.Chapter.ID).
|
||||
Update("original_text", "Mira opened").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, failed := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || failed.Chapter.Status != statusFailed || failed.Chapter.ErrorReason != reasonContentChanged {
|
||||
t.Fatalf("expected content_changed failure, got %+v", failed.Chapter)
|
||||
}
|
||||
code, msg, _ := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", pasted.Job.ID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("retry status %d (%s)", code, msg)
|
||||
}
|
||||
// A pending retry exposes no text and does not create a new chapter.
|
||||
code, pending := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || pending.Chapter.Status != statusPending || pending.Chapter.OriginalText != "" {
|
||||
t.Fatalf("retry must return the chapter to pending, got %+v", pending.Chapter)
|
||||
}
|
||||
var count int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatalf("retry created %d chapters, want 1", count)
|
||||
}
|
||||
// Simulate the content being restored to what was submitted, then retry to completion.
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", pasted.Chapter.ID).
|
||||
Update("original_text", fixturePastedText).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("retry did not publish the same chapter: %+v", ready.Chapter)
|
||||
}
|
||||
job := jobRow(t, db, pasted.Job.ID)
|
||||
// A manual retry restarts the attempt budget, so this processing is attempt 1 again.
|
||||
if job.Attempts != 1 {
|
||||
t.Fatalf("attempts %d after a manual retry, want 1", job.Attempts)
|
||||
}
|
||||
code, msg, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", pasted.Job.ID), learner.Token, nil)
|
||||
if code != 409 || msg == "" {
|
||||
t.Fatalf("retry of a ready job status %d (%s), want 409 with a message", code, msg)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLRepeatedPasteIsIdempotent(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
body := map[string]string{"requestId": "fixture-request-repeat", "title": "Repeat", "text": fixturePastedText}
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, body)
|
||||
if code != 201 || first.Duplicate {
|
||||
t.Fatalf("first paste status %d duplicate %v", code, first.Duplicate)
|
||||
}
|
||||
code, second := pasteBook(t, r, learner.Token, body)
|
||||
if code != 200 || !second.Duplicate {
|
||||
t.Fatalf("repeat paste status %d duplicate %v, want 200 with duplicate", code, second.Duplicate)
|
||||
}
|
||||
if second.Chapter.ID != first.Chapter.ID || second.Job.ID != first.Job.ID || second.Book.ID != first.Book.ID {
|
||||
t.Fatalf("repeat paste returned different rows: %+v vs %+v", second, first)
|
||||
}
|
||||
var books, chapters int64
|
||||
db.Model(&Book{}).Where("owner_id = ?", learner.ID).Count(&books)
|
||||
db.Model(&Chapter{}).Where("owner_id = ?", learner.ID).Count(&chapters)
|
||||
if books != 1 || chapters != 1 {
|
||||
t.Fatalf("repeat paste created books=%d chapters=%d, want 1/1", books, chapters)
|
||||
}
|
||||
|
||||
// The same request id with different content is a conflict, not a silent reuse.
|
||||
for name, changed := range map[string]map[string]string{
|
||||
"different text": {"requestId": "fixture-request-repeat", "title": "Repeat", "text": "Mira opened the workshop."},
|
||||
"different title": {"requestId": "fixture-request-repeat", "title": "Another", "text": fixturePastedText},
|
||||
} {
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", learner.Token, changed)
|
||||
if code != 409 || msg == "" {
|
||||
t.Fatalf("%s: status %d (%s), want 409", name, code, msg)
|
||||
}
|
||||
}
|
||||
|
||||
// Two concurrent submits of one request id create exactly one chapter.
|
||||
name := randomName("race")
|
||||
body = map[string]string{"requestId": name, "title": "Race", "text": "A small step; a small step"}
|
||||
var wg sync.WaitGroup
|
||||
type attempt struct {
|
||||
code int
|
||||
msg string
|
||||
}
|
||||
results := make(chan attempt, 2)
|
||||
for i := 0; i < 2; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", learner.Token, body)
|
||||
results <- attempt{code, msg}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(results)
|
||||
created, reused := 0, 0
|
||||
for result := range results {
|
||||
switch result.code {
|
||||
case 201:
|
||||
created++
|
||||
case 200:
|
||||
reused++
|
||||
default:
|
||||
t.Fatalf("concurrent paste status %d (%s)", result.code, result.msg)
|
||||
}
|
||||
}
|
||||
if created != 1 || reused != 1 {
|
||||
t.Fatalf("concurrent paste created=%d reused=%d", created, reused)
|
||||
}
|
||||
db.Model(&Chapter{}).Where("owner_id = ?", learner.ID).Count(&chapters)
|
||||
if chapters != 2 {
|
||||
t.Fatalf("concurrent paste left %d chapters, want 2", chapters)
|
||||
}
|
||||
|
||||
// A request id already used by another book cannot be replayed by appending, while a
|
||||
// fresh request id appends normally.
|
||||
var raceBook Book
|
||||
if err := db.Where("owner_id = ? AND title = ?", learner.ID, "Race").First(&raceBook).Error; err != nil {
|
||||
t.Fatalf("race book: %v", err)
|
||||
}
|
||||
code, msg, _ := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/books/%d/chapters", first.Book.ID), learner.Token,
|
||||
map[string]string{"requestId": name, "title": "Append", "text": "A second chapter."})
|
||||
if code != 409 || msg == "" {
|
||||
t.Fatalf("cross-book request id status %d (%s), want 409", code, msg)
|
||||
}
|
||||
code, appended := pasteChapter(t, r, learner.Token, first.Book.ID, map[string]string{
|
||||
"requestId": randomName("append"), "title": "Append", "text": "A second chapter."})
|
||||
if code != 201 || appended.Chapter.Ordinal != 2 {
|
||||
t.Fatalf("append status %d chapter %+v", code, appended.Chapter)
|
||||
}
|
||||
if raceBook.ID == first.Book.ID {
|
||||
t.Fatal("idempotency fixtures must use different books")
|
||||
}
|
||||
// Append owns the language of its book, so the field is not part of that contract and the
|
||||
// strict decoder rejects it. The learner client must therefore not send it (regression R1).
|
||||
code, msg, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/books/%d/chapters", first.Book.ID), learner.Token,
|
||||
map[string]string{"requestId": randomName("append"), "title": "Strict", "text": "Strict contract.", "language": "en"})
|
||||
if code != 400 || msg == "" {
|
||||
t.Fatalf("append with language status %d (%s), want 400", code, msg)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLLibraryIsolationAndOwnership(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
a := newLearner(t, r, owner.Token)
|
||||
b := newLearner(t, r, owner.Token)
|
||||
|
||||
code, pasted := pasteBook(t, r, a.Token, map[string]string{
|
||||
"requestId": "fixture-request-isolation", "title": "Private Book", "text": "Only A may read this."})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// Force a failed job so the retry path is checked for another account too.
|
||||
if err := db.Model(&IngestJob{}).Where("id = ?", pasted.Job.ID).Updates(map[string]any{"status": statusFailed, "error_reason": reasonContentChanged}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var book Book
|
||||
var chapter Chapter
|
||||
var job IngestJob
|
||||
db.Where("id = ?", pasted.Book.ID).First(&book)
|
||||
db.Where("id = ?", pasted.Chapter.ID).First(&chapter)
|
||||
db.Where("id = ?", pasted.Job.ID).First(&job)
|
||||
if book.OwnerID != a.ID || chapter.OwnerID != a.ID || job.OwnerID != a.ID {
|
||||
t.Fatal("stored rows must belong to the authenticated account")
|
||||
}
|
||||
if chapter.BookID != book.ID || job.ChapterID != chapter.ID {
|
||||
t.Fatal("job and chapter must stay linked to the book")
|
||||
}
|
||||
|
||||
for name, token := range map[string]string{"other learner": b.Token, "administrator": owner.Token} {
|
||||
for _, path := range []string{
|
||||
fmt.Sprintf("/api/v1/books/%d", book.ID),
|
||||
fmt.Sprintf("/api/v1/chapters/%d", chapter.ID),
|
||||
fmt.Sprintf("/api/v1/jobs/%d", job.ID),
|
||||
} {
|
||||
code, _, _ := callRaw(t, r, "GET", path, token, nil)
|
||||
if code != 404 {
|
||||
t.Fatalf("%s GET %s status %d, want 404", name, path, code)
|
||||
}
|
||||
}
|
||||
code, _, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/books/%d/chapters", book.ID), token,
|
||||
map[string]string{"requestId": randomName("intruder"), "title": "Intruder", "text": "Intruder text."})
|
||||
if code != 404 {
|
||||
t.Fatalf("%s append status %d, want 404", name, code)
|
||||
}
|
||||
code, _, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", job.ID), token, nil)
|
||||
if code != 404 {
|
||||
t.Fatalf("%s retry status %d, want 404", name, code)
|
||||
}
|
||||
_, list := bookList(t, r, token)
|
||||
if len(list.Items) != 0 {
|
||||
t.Fatalf("%s sees %d books", name, len(list.Items))
|
||||
}
|
||||
}
|
||||
|
||||
// The caller's identity comes from the session, never from the request body or query.
|
||||
for _, payload := range []map[string]any{
|
||||
{"requestId": randomName("owner"), "title": "Spoof", "text": "Spoofed owner.", "ownerId": b.ID},
|
||||
{"requestId": randomName("owner"), "title": "Spoof", "text": "Spoofed owner.", "userId": b.ID},
|
||||
} {
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", a.Token, payload)
|
||||
if code != 400 || msg == "" {
|
||||
t.Fatalf("client-supplied owner status %d (%s), want 400", code, msg)
|
||||
}
|
||||
}
|
||||
code, msg, _ := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/books?ownerId=%d", b.ID), a.Token, nil)
|
||||
if code != 400 || msg == "" {
|
||||
t.Fatalf("query owner override status %d (%s), want 400", code, msg)
|
||||
}
|
||||
var books int64
|
||||
db.Model(&Book{}).Where("owner_id = ?", a.ID).Count(&books)
|
||||
if books != 1 {
|
||||
t.Fatalf("rejected requests created %d books", books)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLIngestRecoveryWithoutRestart(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-in-service-recovery", "title": "In-service recovery", "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
job, claimed, err := ClaimNextIngestJob(db, time.Now())
|
||||
if err != nil || !claimed || job.ID != pasted.Job.ID {
|
||||
t.Fatalf("claim failed (claimed=%v, job=%d): %v", claimed, job.ID, err)
|
||||
}
|
||||
|
||||
// The finishing transaction fails (context cancellation stands in for a timeout or a
|
||||
// database error). The claim is already committed, so the job stays processing.
|
||||
canceled, cancel := context.WithCancel(t.Context())
|
||||
cancel()
|
||||
if err = FinishIngestJob(canceled, db, job, time.Now()); err == nil {
|
||||
t.Fatal("a canceled finishing transaction must report an error")
|
||||
}
|
||||
var stuck IngestJob
|
||||
if err = db.Where("id = ?", pasted.Job.ID).First(&stuck).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if stuck.Status != statusProcessing {
|
||||
t.Fatalf("job status %q after a failed finish, want processing", stuck.Status)
|
||||
}
|
||||
// A manual retry cannot rescue it: only failed jobs are accepted.
|
||||
code, msg, _ := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", pasted.Job.ID), learner.Token, nil)
|
||||
if code != 409 || msg == "" {
|
||||
t.Fatalf("retry of a processing job status %d (%s), want 409", code, msg)
|
||||
}
|
||||
|
||||
// A sweep that is too early must leave a healthy claim alone.
|
||||
claimedAt := stuck.UpdatedAt
|
||||
if _, err = RequeueStaleIngestJobs(db, claimedAt.Add(time.Second)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if jobRow(t, db, pasted.Job.ID).Status != statusProcessing {
|
||||
t.Fatal("a fresh claim must not be requeued")
|
||||
}
|
||||
// Once the claim is older than the stale window, the running service recovers it.
|
||||
if _, err = RequeueStaleIngestJobs(db, claimedAt.Add(ingestStaleAfter+time.Second)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
recovered := jobRow(t, db, pasted.Job.ID)
|
||||
if recovered.Status != statusPending || recovered.Attempts != 1 || recovered.FinishedAt != nil {
|
||||
t.Fatalf("recovered job %+v", recovered)
|
||||
}
|
||||
if chapterRow(t, db, pasted.Chapter.ID).Status != statusPending {
|
||||
t.Fatal("recovered chapter must be pending")
|
||||
}
|
||||
// Recovery reuses the same rows: no second chapter, same task id.
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("recovery left %d chapters, want 1", chapters)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("recovered chapter %+v", ready.Chapter)
|
||||
}
|
||||
if ready.Chapter.ID != pasted.Chapter.ID || ready.Chapter.JobID == nil || *ready.Chapter.JobID != pasted.Job.ID {
|
||||
t.Fatal("recovery must keep the original chapter and task ids")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLIngestAttemptsAreBoundedAndManualRetryRestarts(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-attempt-budget", "title": "Attempt budget", "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// Spend the whole budget without any worker running.
|
||||
if err := db.Model(&IngestJob{}).Where("id = ?", pasted.Job.ID).Update("attempts", maxIngestAttempts).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, claimed, err := ClaimNextIngestJob(db, time.Now()); err != nil || claimed {
|
||||
t.Fatalf("claim claimed=%v (%v), want no claim once the budget is used", claimed, err)
|
||||
}
|
||||
if _, err := RequeueStaleIngestJobs(db, time.Now()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
exhausted := jobRow(t, db, pasted.Job.ID)
|
||||
if exhausted.Status != statusFailed || exhausted.ErrorReason != reasonAttemptsExhausted {
|
||||
t.Fatalf("exhausted job %+v", exhausted)
|
||||
}
|
||||
code, read := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || read.Chapter.Status != statusFailed || read.Chapter.ErrorReason != reasonAttemptsExhausted {
|
||||
t.Fatalf("exhausted chapter %+v", read.Chapter)
|
||||
}
|
||||
if read.Chapter.ErrorMessage == "" || read.Chapter.OriginalText != "" {
|
||||
t.Fatalf("exhausted chapter must fail readably without text: %+v", read.Chapter)
|
||||
}
|
||||
// The manual retry is still available and restarts the attempt budget.
|
||||
code, msg, _ := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", pasted.Job.ID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("manual retry status %d (%s), want 200", code, msg)
|
||||
}
|
||||
retried := jobRow(t, db, pasted.Job.ID)
|
||||
if retried.Status != statusPending || retried.Attempts != 0 || retried.ErrorReason != "" {
|
||||
t.Fatalf("retried job %+v", retried)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.ErrorReason != "" {
|
||||
t.Fatalf("chapter after manual retry %+v", ready.Chapter)
|
||||
}
|
||||
if ready.Chapter.ID != pasted.Chapter.ID {
|
||||
t.Fatal("manual retry must reuse the same chapter")
|
||||
}
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("attempt recovery left %d chapters, want 1", chapters)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLIngestRecoveryAfterRestart(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
// The test database is shared with other cases, so empty the queue first: the claim below
|
||||
// must take this test's own job, not a leftover one.
|
||||
drainIngest(t, db)
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-recovery", "title": "Recovery", "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// A crash between claim and finish leaves rows in processing.
|
||||
job, claimed, err := ClaimNextIngestJob(db, time.Now())
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("claim failed (claimed=%v): %v", claimed, err)
|
||||
}
|
||||
if job.ID != pasted.Job.ID {
|
||||
t.Fatalf("claimed job %d, want this test's job %d", job.ID, pasted.Job.ID)
|
||||
}
|
||||
if chapterRow(t, db, pasted.Chapter.ID).Status != statusProcessing || jobRow(t, db, pasted.Job.ID).Status != statusProcessing {
|
||||
t.Fatal("claim must persist the processing state")
|
||||
}
|
||||
requeued, err := RecoverIngestJobs(db, time.Now())
|
||||
if err != nil || requeued < 1 {
|
||||
t.Fatalf("recovery requeued %d (%v), want at least this test's job", requeued, err)
|
||||
}
|
||||
pending := jobRow(t, db, pasted.Job.ID)
|
||||
if pending.Status != statusPending || pending.Attempts != 1 || pending.FinishedAt != nil {
|
||||
t.Fatalf("recovered job %+v", pending)
|
||||
}
|
||||
if chapterRow(t, db, pasted.Chapter.ID).Status != statusPending {
|
||||
t.Fatal("recovered chapter must be pending")
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("recovered chapter %+v", ready.Chapter)
|
||||
}
|
||||
if jobRow(t, db, pasted.Job.ID).Attempts != 2 {
|
||||
t.Fatal("the recovered job must be processed as a second attempt")
|
||||
}
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("recovery left %d chapters, want 1", chapters)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLPasteRejectsInvalidInputAndLimits(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
|
||||
longTitle := strings.Repeat("T", maxTitleRunes+1)
|
||||
overLimit := strings.Repeat("a", maxChapterRunes+1)
|
||||
cases := []struct {
|
||||
name string
|
||||
payload map[string]string
|
||||
}{
|
||||
{"missing_request_id", map[string]string{"title": "T", "text": "Text."}},
|
||||
{"short_request_id", map[string]string{"requestId": "short", "title": "T", "text": "Text."}},
|
||||
{"empty_title", map[string]string{"requestId": randomName("invalid"), "title": " ", "text": "Text."}},
|
||||
{"long_title", map[string]string{"requestId": randomName("invalid"), "title": longTitle, "text": "Text."}},
|
||||
{"empty_text", map[string]string{"requestId": randomName("invalid"), "title": "T", "text": " \r\n\t "}},
|
||||
{"unsupported_language", map[string]string{"requestId": randomName("invalid"), "title": "T", "text": "Text.", "language": "zh"}},
|
||||
{"over_limit", map[string]string{"requestId": randomName("invalid"), "title": "T", "text": overLimit}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", learner.Token, tc.payload)
|
||||
if code != 400 || msg == "" {
|
||||
t.Fatalf("status %d (%s), want 400 with a message", code, msg)
|
||||
}
|
||||
})
|
||||
}
|
||||
// The boundary itself is accepted, and an oversized body is refused before decoding.
|
||||
code, accepted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-boundary-limit", "title": "At the limit", "text": strings.Repeat("a", maxChapterRunes)})
|
||||
if code != 201 || accepted.Chapter.CharCount != maxChapterRunes {
|
||||
t.Fatalf("boundary paste status %d charCount %d", code, accepted.Chapter.CharCount)
|
||||
}
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", learner.Token, map[string]string{
|
||||
"requestId": "fixture-body-limit", "title": "Too large", "text": strings.Repeat("a", maxPasteBodyBytes)})
|
||||
if code != 400 || !strings.Contains(msg, "过大") {
|
||||
t.Fatalf("oversized body status %d (%s), want 400 with 过大", code, msg)
|
||||
}
|
||||
var books, chapters int64
|
||||
db.Model(&Book{}).Where("owner_id = ?", learner.ID).Count(&books)
|
||||
db.Model(&Chapter{}).Where("owner_id = ?", learner.ID).Count(&chapters)
|
||||
if books != 1 || chapters != 1 {
|
||||
t.Fatalf("rejected input created books=%d chapters=%d", books, chapters)
|
||||
}
|
||||
}
|
||||
@@ -88,11 +88,216 @@ func emptyMigrationDB(t *testing.T) *gorm.DB {
|
||||
return db
|
||||
}
|
||||
|
||||
func TestMigrationFromV2PreservesExistingData(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
// Build a v2 database by hand: this is the state a deployed instance is in before #5.
|
||||
if err := db.Exec("CREATE TABLE lexgo_schema (id INT PRIMARY KEY,version INT,product VARCHAR(32))").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Exec("INSERT INTO lexgo_schema VALUES (1,2,'lexgo')").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, statement := range schemaV2Statements {
|
||||
if err := db.Exec(statement).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.Exec("INSERT INTO sys_user (user_id,username,password,role_id) VALUES (7,'fixture_v2','x',2)").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Exec("INSERT INTO lexgo_spaces (owner_id,language) VALUES (7,'en')").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatalf("v2 to v3 migration failed: %v", err)
|
||||
}
|
||||
if err := CheckSchema(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, table := range []string{"lexgo_books", "lexgo_chapters", "lexgo_ingest_jobs"} {
|
||||
var count int64
|
||||
db.Raw("SELECT COUNT(*) FROM information_schema.tables WHERE table_schema=DATABASE() AND table_name=?", table).Scan(&count)
|
||||
if count != 1 {
|
||||
t.Fatalf("migration did not create %s", table)
|
||||
}
|
||||
}
|
||||
var users, spaces int64
|
||||
db.Table("sys_user").Where("user_id = 7").Count(&users)
|
||||
db.Table("lexgo_spaces").Where("owner_id = 7").Count(&spaces)
|
||||
if users != 1 || spaces != 1 {
|
||||
t.Fatalf("migration changed existing rows: users=%d spaces=%d", users, spaces)
|
||||
}
|
||||
// A rollback marker set back to 2 can be upgraded again without touching data.
|
||||
if err := db.Exec("UPDATE lexgo_schema SET version=2 WHERE id=1").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatalf("re-upgrade failed: %v", err)
|
||||
}
|
||||
var version int
|
||||
db.Raw("SELECT version FROM lexgo_schema WHERE id=1").Scan(&version)
|
||||
if version != SchemaVersion {
|
||||
t.Fatalf("schema version %d after re-upgrade, want %d", version, SchemaVersion)
|
||||
}
|
||||
db.Table("sys_user").Where("user_id = 7").Count(&users)
|
||||
if users != 1 {
|
||||
t.Fatal("re-upgrade changed existing rows")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationFromV4KeepsLibraryAndAddsPersonalTerms(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
statements := []string{"CREATE TABLE lexgo_schema (id INT PRIMARY KEY,version INT,product VARCHAR(32))", "INSERT INTO lexgo_schema VALUES (1,4,'lexgo')"}
|
||||
statements = append(statements, schemaV2Statements...)
|
||||
statements = append(statements, schemaV3Statements...)
|
||||
statements = append(statements, schemaV4Statements...)
|
||||
for _, statement := range statements {
|
||||
if err := db.Exec(statement).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.Exec("INSERT INTO sys_user (user_id,username,password,role_id) VALUES (9,'fixture_v4','fictional-not-a-real-hash',2)").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := stamp(time.Now())
|
||||
book := Book{OwnerID: 9, Title: "Fictional migration", Language: "en", CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&book).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
original := "😀 Original e\u0301\r\n"
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: 9, Ordinal: 1, Title: "Fixture", OriginalText: original, ContentSHA256: contentSHA(original), Status: statusReady, CharCount: len([]rune(original)), CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&chapter).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resource := DictionaryResource{ID: 1, Name: "Fixture WordNet", Language: "en", Version: "3.0", Source: WordNetSource, Format: "wordnet-3.0-zip", SHA256: strings.Repeat("a", 64), EntryCount: 5, Enabled: true, Archive: []byte("fixture archive"), UpdatedAt: now}
|
||||
if err := db.Create(&resource).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err == nil {
|
||||
t.Fatal("old schema accepted before explicit migration")
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var restored Chapter
|
||||
if err := db.First(&restored, chapter.ID).Error; err != nil || restored.OriginalText != original || restored.ContentSHA256 != chapter.ContentSHA256 {
|
||||
t.Fatal("migration changed original chapter", err)
|
||||
}
|
||||
var restoredResource DictionaryResource
|
||||
if err := db.Omit("archive").First(&restoredResource, 1).Error; err != nil || restoredResource.SHA256 != resource.SHA256 || restoredResource.EntryCount != 5 {
|
||||
t.Fatal("migration changed the shared dictionary resource", err)
|
||||
}
|
||||
var terms int64
|
||||
if err := db.Model(&Term{}).Count(&terms).Error; err != nil || terms != 0 {
|
||||
t.Fatal("migration must create an empty personal term table", terms, err)
|
||||
}
|
||||
// The new table is usable and enforces the documented status boundary.
|
||||
valid := Term{OwnerID: 9, Language: "en", Term: "curiosity", OriginalForm: "Curiosity", Status: termStatusNew, CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&valid).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Create(&Term{OwnerID: 9, Language: "en", Term: "forged", OriginalForm: "forged", Status: "deleted", CreatedAt: now, UpdatedAt: now}).Error; err == nil {
|
||||
t.Fatal("an unknown status must be rejected by the schema check")
|
||||
}
|
||||
if err := db.Create(&Term{OwnerID: 9, Language: "en", Term: "leveled", OriginalForm: "leveled", Status: termStatusKnown, Level: 5, CreatedAt: now, UpdatedAt: now}).Error; err == nil {
|
||||
t.Fatal("a level outside 0～7 must be rejected by the schema check")
|
||||
}
|
||||
// Rolling the marker back for a binary rollback and upgrading again keeps rows.
|
||||
if err := db.Exec("UPDATE lexgo_schema SET version=4 WHERE id=1").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Model(&Term{}).Count(&terms).Error; err != nil || terms != 1 {
|
||||
t.Fatal("re-upgrade changed existing personal terms", terms, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationFromV5AddsReviewScheduling(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
statements := []string{"CREATE TABLE lexgo_schema (id INT PRIMARY KEY,version INT,product VARCHAR(32))", "INSERT INTO lexgo_schema VALUES (1,5,'lexgo')"}
|
||||
statements = append(statements, schemaV2Statements...)
|
||||
statements = append(statements, schemaV3Statements...)
|
||||
statements = append(statements, schemaV4Statements...)
|
||||
statements = append(statements, schemaV5Statements...)
|
||||
for _, statement := range statements {
|
||||
if err := db.Exec(statement).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.Exec("INSERT INTO sys_user (user_id,username,password,role_id) VALUES (11,'fixture_v5','fictional-not-a-real-hash',2)").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
saved := stamp(time.Now())
|
||||
term := Term{OwnerID: 11, Language: "en", Term: "curiosity", OriginalForm: "Curiosity", Definition: "虚构释义", Examples: "Fictional example.", Status: termStatusLearning, Level: 2, CreatedAt: saved, UpdatedAt: saved}
|
||||
if err := db.Create(&term).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err == nil {
|
||||
t.Fatal("old schema accepted before explicit migration")
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// The learner's own term is untouched and has never been reviewed.
|
||||
var restored Term
|
||||
if err := db.First(&restored, term.ID).Error; err != nil || restored.Definition != term.Definition || restored.Level != 2 || restored.Status != termStatusLearning {
|
||||
t.Fatal("migration changed the personal term", err)
|
||||
}
|
||||
// An existing saved word enters the queue immediately: due at the moment it was saved.
|
||||
var review TermReview
|
||||
if err := db.Where("term_id = ?", term.ID).First(&review).Error; err != nil {
|
||||
t.Fatal("existing terms need a schedule row", err)
|
||||
}
|
||||
if !review.DueAt.Equal(saved) || review.ReviewCount != 0 || review.CorrectCount != 0 || review.WrongCount != 0 || review.Language != "en" {
|
||||
t.Fatalf("schedule row: %#v", review)
|
||||
}
|
||||
var answers int64
|
||||
if err := db.Model(&ReviewAnswer{}).Count(&answers).Error; err != nil || answers != 0 {
|
||||
t.Fatal("migration must create an empty attempt log", answers, err)
|
||||
}
|
||||
// A second identical answer row is rejected by the unique answer key.
|
||||
digest, err := requestKey("migration-fixture-answer")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
record := ReviewAnswer{OwnerID: 11, AnswerKey: digest, TermID: term.ID, Grade: reviewGradeCorrect, Result: "applied", StatusBefore: termStatusNew, StatusAfter: termStatusLearning, LevelBefore: 0, LevelAfter: 1, DueAtBefore: saved, DueAtAfter: saved, CreatedAt: saved}
|
||||
if err := db.Create(&record).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
duplicate := record
|
||||
duplicate.ID = 0
|
||||
if err := db.Create(&duplicate).Error; err == nil {
|
||||
t.Fatal("the answer key must be unique per account")
|
||||
}
|
||||
// Rolling the marker back for a binary rollback and upgrading again keeps the rows.
|
||||
if err := db.Exec("UPDATE lexgo_schema SET version=5 WHERE id=1").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var reviews, attempts int64
|
||||
if err := db.Model(&TermReview{}).Count(&reviews).Error; err != nil || reviews != 1 {
|
||||
t.Fatalf("re-upgrade must not duplicate schedule rows: %d %v", reviews, err)
|
||||
}
|
||||
if err := db.Model(&ReviewAnswer{}).Count(&attempts).Error; err != nil || attempts != 1 {
|
||||
t.Fatalf("re-upgrade must keep the attempt log: %d %v", attempts, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationRefusesUnownedOrUnsupportedSchema(t *testing.T) {
|
||||
for _, tc := range []struct{ name, marker string }{
|
||||
{"empty_marker", ""},
|
||||
{"negative_version", "INSERT INTO lexgo_schema VALUES (1,-1,'lexgo')"},
|
||||
{"future_version", "INSERT INTO lexgo_schema VALUES (1,3,'lexgo')"},
|
||||
{"future_version", "INSERT INTO lexgo_schema VALUES (1,99,'lexgo')"},
|
||||
{"wrong_product", "INSERT INTO lexgo_schema VALUES (1,0,'another-app')"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
@@ -116,3 +321,53 @@ func TestMigrationRefusesUnownedOrUnsupportedSchema(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationFromV3PreservesLibraryAndJobs(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
statements := []string{"CREATE TABLE lexgo_schema (id INT PRIMARY KEY,version INT,product VARCHAR(32))", "INSERT INTO lexgo_schema VALUES (1,3,'lexgo')"}
|
||||
statements = append(statements, schemaV2Statements...)
|
||||
statements = append(statements, schemaV3Statements...)
|
||||
for _, statement := range statements {
|
||||
if err := db.Exec(statement).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.Exec("INSERT INTO sys_user (user_id,username,password,role_id) VALUES (8,'fixture_v3','fictional-not-a-real-hash',2)").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := stamp(time.Now())
|
||||
book := Book{OwnerID: 8, Title: "Fictional migration", Language: "en", CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&book).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
original := "😀 Original e\u0301\r\n"
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: 8, Ordinal: 1, Title: "Fixture", OriginalText: original, ContentSHA256: contentSHA(original), Status: statusReady, CharCount: len([]rune(original)), CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&chapter).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
job := IngestJob{OwnerID: 8, BookID: book.ID, ChapterID: chapter.ID, RequestKey: contentSHA("fixture-v3"), ContentSHA256: chapter.ContentSHA256, Status: statusReady, CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&job).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err == nil {
|
||||
t.Fatal("old schema accepted before explicit migration")
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var restored Chapter
|
||||
if err := db.First(&restored, chapter.ID).Error; err != nil || restored.OriginalText != original || restored.ContentSHA256 != chapter.ContentSHA256 || restored.Status != statusReady {
|
||||
t.Fatal("migration changed original chapter", err)
|
||||
}
|
||||
var restoredJob IngestJob
|
||||
if err := db.First(&restoredJob, job.ID).Error; err != nil || restoredJob.RequestKey != job.RequestKey || restoredJob.Status != statusReady {
|
||||
t.Fatal("migration changed job", err)
|
||||
}
|
||||
var resources int64
|
||||
if err := db.Model(&DictionaryResource{}).Count(&resources).Error; err != nil || resources != 0 {
|
||||
t.Fatal("migration must create empty resource table", resources, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,415 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// The three answers the accepted review prototype offers.
|
||||
const (
|
||||
reviewGradeCorrect = "correct"
|
||||
reviewGradeWrong = "wrong"
|
||||
reviewGradeAgain = "again"
|
||||
)
|
||||
|
||||
const (
|
||||
maxReviewLevel = 7
|
||||
// One queue page is enough for a daily round; the rest is fetched after finishing it.
|
||||
reviewQueueLimit = 50
|
||||
)
|
||||
|
||||
// reviewIntervals is the fixed schedule applied after a correct answer, indexed by
|
||||
// level-1. This is a fixed table, not FSRS: the decision table and its samples live in
|
||||
// the business rules page.
|
||||
var reviewIntervals = [maxReviewLevel]int{1, 2, 4, 7, 15, 30, 60}
|
||||
|
||||
var reviewGrades = map[string]bool{
|
||||
reviewGradeCorrect: true,
|
||||
reviewGradeWrong: true,
|
||||
reviewGradeAgain: true,
|
||||
}
|
||||
|
||||
// TermReview is the scheduling state of one personal term, kept in its own table so a
|
||||
// migration never alters the table that holds the learner's own text.
|
||||
type TermReview struct {
|
||||
TermID int64 `gorm:"primaryKey"`
|
||||
OwnerID int
|
||||
Language string
|
||||
DueAt time.Time
|
||||
ReviewCount int
|
||||
CorrectCount int
|
||||
WrongCount int
|
||||
LastReviewedAt *time.Time
|
||||
}
|
||||
|
||||
func (TermReview) TableName() string { return "lexgo_term_reviews" }
|
||||
|
||||
// ReviewAnswer records one attempt. The unique answer key is what makes a repeated
|
||||
// request, a double click or a network resend update nothing a second time.
|
||||
type ReviewAnswer struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
OwnerID int
|
||||
AnswerKey string `gorm:"column:answer_key"`
|
||||
TermID int64
|
||||
Grade string
|
||||
Result string
|
||||
StatusBefore string `gorm:"column:status_before"`
|
||||
StatusAfter string `gorm:"column:status_after"`
|
||||
LevelBefore int `gorm:"column:level_before"`
|
||||
LevelAfter int `gorm:"column:level_after"`
|
||||
DueAtBefore time.Time `gorm:"column:due_at_before"`
|
||||
DueAtAfter time.Time `gorm:"column:due_at_after"`
|
||||
Requeued bool
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
func (ReviewAnswer) TableName() string { return "lexgo_review_answers" }
|
||||
|
||||
type ReviewItem struct {
|
||||
ID int64 `json:"id"`
|
||||
Term string `json:"term"`
|
||||
OriginalForm string `json:"originalForm"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
Status string `json:"status"`
|
||||
Level int `json:"level"`
|
||||
DueAt time.Time `json:"dueAt"`
|
||||
ReviewCount int `json:"reviewCount"`
|
||||
}
|
||||
|
||||
type ReviewQueue struct {
|
||||
Items []ReviewItem `json:"items"`
|
||||
Total int `json:"total"`
|
||||
}
|
||||
|
||||
type ReviewAnswerInput struct {
|
||||
AnswerID string `json:"answerId"`
|
||||
Grade string `json:"grade"`
|
||||
ExpectedDueAt time.Time `json:"expectedDueAt"`
|
||||
}
|
||||
|
||||
type ReviewAnswerResult struct {
|
||||
// Result is the outcome of this answer: applied when the term moved, stale when another
|
||||
// screen already advanced it. A replayed answer repeats the outcome it was given first.
|
||||
Result string `json:"result"`
|
||||
// Duplicate reports that this answer key was already recorded and nothing changed now.
|
||||
Duplicate bool `json:"duplicate"`
|
||||
Grade string `json:"grade"`
|
||||
Requeued bool `json:"requeued"`
|
||||
StatusBefore string `json:"statusBefore"`
|
||||
StatusAfter string `json:"statusAfter"`
|
||||
LevelBefore int `json:"levelBefore"`
|
||||
LevelAfter int `json:"levelAfter"`
|
||||
DueAtBefore time.Time `json:"dueAtBefore"`
|
||||
DueAtAfter time.Time `json:"dueAtAfter"`
|
||||
Item ReviewItem `json:"item"`
|
||||
}
|
||||
|
||||
// reviewState is the part of a term a grade moves.
|
||||
type reviewState struct {
|
||||
Status string
|
||||
Level int
|
||||
DueAt time.Time
|
||||
}
|
||||
|
||||
func nextDue(level int, now time.Time) time.Time {
|
||||
if level < 1 {
|
||||
level = 1
|
||||
}
|
||||
if level > maxReviewLevel {
|
||||
level = maxReviewLevel
|
||||
}
|
||||
return stamp(now.AddDate(0, 0, reviewIntervals[level-1]))
|
||||
}
|
||||
|
||||
// applyGrade advances one term by exactly one answer. It returns the next state and
|
||||
// whether the item belongs to the current round again.
|
||||
func applyGrade(before reviewState, grade string, now time.Time) (reviewState, bool, error) {
|
||||
switch grade {
|
||||
case reviewGradeCorrect:
|
||||
level := before.Level + 1
|
||||
if level > maxReviewLevel {
|
||||
level = maxReviewLevel
|
||||
}
|
||||
return reviewState{Status: termStatusLearning, Level: level, DueAt: nextDue(level, now)}, false, nil
|
||||
case reviewGradeWrong:
|
||||
next := reviewState{Status: before.Status, Level: before.Level}
|
||||
// Only a learning word loses a level; a new word stays new until it is answered
|
||||
// correctly for the first time.
|
||||
if before.Status == termStatusLearning {
|
||||
next.Level = before.Level - 1
|
||||
if next.Level < 1 {
|
||||
next.Level = 1
|
||||
}
|
||||
}
|
||||
next.DueAt = stamp(now)
|
||||
return next, true, nil
|
||||
case reviewGradeAgain:
|
||||
return reviewState{Status: before.Status, Level: before.Level, DueAt: stamp(now)}, true, nil
|
||||
default:
|
||||
return before, false, failure(400, "评分无效")
|
||||
}
|
||||
}
|
||||
|
||||
func reviewItem(term Term, review TermReview) ReviewItem {
|
||||
return ReviewItem{
|
||||
ID: term.ID, Term: term.Term, OriginalForm: term.OriginalForm, Definition: term.Definition,
|
||||
Examples: splitExamples(term.Examples), Status: term.Status, Level: term.Level,
|
||||
DueAt: review.DueAt, ReviewCount: review.ReviewCount,
|
||||
}
|
||||
}
|
||||
|
||||
// ReviewQueueFor returns the caller's due words for the current language. Known and
|
||||
// ignored words stay out of the queue, and the queue is an absolute-instant comparison:
|
||||
// an item is due as soon as due_at is not in the future.
|
||||
func ReviewQueueFor(tx *gorm.DB, owner int, language string, now time.Time) (ReviewQueue, error) {
|
||||
queue := ReviewQueue{Items: []ReviewItem{}}
|
||||
due := func() *gorm.DB {
|
||||
return tx.Table("lexgo_terms AS t").
|
||||
Joins("JOIN lexgo_term_reviews AS r ON r.term_id = t.id").
|
||||
Where("t.owner_id = ? AND r.owner_id = ? AND t.language = ? AND r.language = ?", owner, owner, language, language).
|
||||
Where("t.status IN ?", []string{termStatusNew, termStatusLearning}).
|
||||
Where("r.due_at <= ?", stamp(now))
|
||||
}
|
||||
var total int64
|
||||
if err := due().Count(&total).Error; err != nil {
|
||||
return queue, err
|
||||
}
|
||||
queue.Total = int(total)
|
||||
var rows []struct {
|
||||
ID int64
|
||||
Term string
|
||||
OriginalForm string
|
||||
Definition string
|
||||
Examples string
|
||||
Status string
|
||||
Level int
|
||||
DueAt time.Time
|
||||
ReviewCount int
|
||||
}
|
||||
if err := due().
|
||||
Select("t.id, t.term, t.original_form, t.definition, t.examples, t.status, t.level, r.due_at, r.review_count").
|
||||
Order("r.due_at, t.id").Limit(reviewQueueLimit).Scan(&rows).Error; err != nil {
|
||||
return queue, err
|
||||
}
|
||||
for _, row := range rows {
|
||||
queue.Items = append(queue.Items, ReviewItem{
|
||||
ID: row.ID, Term: row.Term, OriginalForm: row.OriginalForm, Definition: row.Definition,
|
||||
Examples: splitExamples(row.Examples), Status: row.Status, Level: row.Level,
|
||||
DueAt: row.DueAt, ReviewCount: row.ReviewCount,
|
||||
})
|
||||
}
|
||||
return queue, nil
|
||||
}
|
||||
|
||||
// syncTermReview keeps the scheduling row in step with a saved term. A new word is due
|
||||
// immediately so saving it starts the loop; a word saved straight into a level waits for
|
||||
// that level's interval, so a manual level is not silently re-queued today. When the caller
|
||||
// does not ask for a reschedule, only a missing row is created and the existing date and
|
||||
// counters stay untouched.
|
||||
func syncTermReview(tx *gorm.DB, term Term, reschedule bool, now time.Time) error {
|
||||
due := stamp(now)
|
||||
if term.Status == termStatusLearning && term.Level >= 1 {
|
||||
due = nextDue(term.Level, now)
|
||||
}
|
||||
var existing TermReview
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("term_id = ?", term.ID).First(&existing).Error
|
||||
switch {
|
||||
case errors.Is(err, gorm.ErrRecordNotFound):
|
||||
return tx.Create(&TermReview{TermID: term.ID, OwnerID: term.OwnerID, Language: term.Language, DueAt: due}).Error
|
||||
case err != nil:
|
||||
return err
|
||||
case reschedule:
|
||||
return tx.Model(&TermReview{}).Where("term_id = ?", term.ID).Update("due_at", due).Error
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func lockedReview(tx *gorm.DB, owner int, term Term) (TermReview, error) {
|
||||
var review TermReview
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("term_id = ? AND owner_id = ?", term.ID, owner).First(&review).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
// A term written before this table existed is due at the moment it was saved.
|
||||
review = TermReview{TermID: term.ID, OwnerID: owner, Language: term.Language, DueAt: stamp(term.CreatedAt)}
|
||||
if err = tx.Create(&review).Error; err != nil {
|
||||
return review, err
|
||||
}
|
||||
return review, nil
|
||||
}
|
||||
return review, err
|
||||
}
|
||||
|
||||
// lockedAnswerKey reads one recorded answer under a lock. It is called again after the term
|
||||
// row is locked, because a locking read sees the newest committed row while the plain read
|
||||
// before it may still see this transaction's snapshot.
|
||||
func lockedAnswerKey(tx *gorm.DB, owner int, key string) (ReviewAnswer, error) {
|
||||
var stored ReviewAnswer
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("owner_id = ? AND answer_key = ?", owner, key).First(&stored).Error
|
||||
return stored, err
|
||||
}
|
||||
|
||||
// AnswerReview applies one grade to one owned term exactly once. A repeated answer key
|
||||
// returns the first outcome, and an answer whose due time no longer matches (another tab
|
||||
// already advanced the term) is recorded as stale without moving anything.
|
||||
func AnswerReview(tx *gorm.DB, owner int, termID int64, input ReviewAnswerInput, now time.Time) (ReviewAnswerResult, error) {
|
||||
if !reviewGrades[input.Grade] {
|
||||
return ReviewAnswerResult{}, failure(400, "评分无效")
|
||||
}
|
||||
if input.ExpectedDueAt.IsZero() {
|
||||
return ReviewAnswerResult{}, failure(400, "请提交队列中的到期时间")
|
||||
}
|
||||
key, err := requestKey(input.AnswerID)
|
||||
if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
if stored, err := lockedAnswerKey(tx, owner, key); err == nil {
|
||||
return replayedAnswer(tx, stored, termID)
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
// Ownership first: another account's term and a missing term answer identically.
|
||||
var term Term
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("id = ? AND owner_id = ?", termID, owner).First(&term).Error; errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReviewAnswerResult{}, failure(404, "词条不存在")
|
||||
} else if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
// Simultaneous submissions of one answer id are serialized by the term lock above; the
|
||||
// second one now sees the recorded answer and reports it instead of counting again.
|
||||
if stored, err := lockedAnswerKey(tx, owner, key); err == nil {
|
||||
return replayedAnswer(tx, stored, termID)
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
review, err := lockedReview(tx, owner, term)
|
||||
if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
if term.Status != termStatusNew && term.Status != termStatusLearning {
|
||||
return ReviewAnswerResult{}, failure(409, "该词条已不在复习队列，请刷新队列")
|
||||
}
|
||||
before := reviewState{Status: term.Status, Level: term.Level, DueAt: review.DueAt}
|
||||
if !input.ExpectedDueAt.Equal(review.DueAt) {
|
||||
return staleAnswer(tx, owner, term, review, input, now)
|
||||
}
|
||||
next, requeued, err := applyGrade(before, input.Grade, now)
|
||||
if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
when := stamp(now)
|
||||
if err := tx.Model(&Term{}).Where("id = ? AND owner_id = ?", term.ID, owner).
|
||||
Updates(map[string]any{"status": next.Status, "level": next.Level, "updated_at": when}).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
counters := map[string]any{
|
||||
"due_at": next.DueAt, "review_count": gorm.Expr("review_count + 1"),
|
||||
"last_reviewed_at": when,
|
||||
}
|
||||
// "again" is not a correct answer either: the word was not recognised this time.
|
||||
if input.Grade == reviewGradeCorrect {
|
||||
counters["correct_count"] = gorm.Expr("correct_count + 1")
|
||||
} else {
|
||||
counters["wrong_count"] = gorm.Expr("wrong_count + 1")
|
||||
}
|
||||
if err := tx.Model(&TermReview{}).Where("term_id = ? AND owner_id = ?", term.ID, owner).Updates(counters).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
term.Status, term.Level, term.UpdatedAt = next.Status, next.Level, when
|
||||
review.DueAt, review.ReviewCount, review.LastReviewedAt = next.DueAt, review.ReviewCount+1, &when
|
||||
record := ReviewAnswer{
|
||||
OwnerID: owner, AnswerKey: key, TermID: term.ID, Grade: input.Grade, Result: "applied",
|
||||
StatusBefore: before.Status, StatusAfter: next.Status, LevelBefore: before.Level, LevelAfter: next.Level,
|
||||
DueAtBefore: before.DueAt, DueAtAfter: next.DueAt, Requeued: requeued, CreatedAt: when,
|
||||
}
|
||||
if err := tx.Create(&record).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
return ReviewAnswerResult{
|
||||
Result: "applied", Grade: input.Grade, Requeued: requeued,
|
||||
StatusBefore: before.Status, StatusAfter: next.Status, LevelBefore: before.Level, LevelAfter: next.Level,
|
||||
DueAtBefore: before.DueAt, DueAtAfter: next.DueAt, Item: reviewItem(term, review),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// staleAnswer records that this attempt changed nothing because the term had already
|
||||
// moved on. It is a normal outcome of two open tabs, not an error.
|
||||
func staleAnswer(tx *gorm.DB, owner int, term Term, review TermReview, input ReviewAnswerInput, now time.Time) (ReviewAnswerResult, error) {
|
||||
key, err := requestKey(input.AnswerID)
|
||||
if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
record := ReviewAnswer{
|
||||
OwnerID: owner, AnswerKey: key, TermID: term.ID, Grade: input.Grade, Result: "stale",
|
||||
StatusBefore: term.Status, StatusAfter: term.Status, LevelBefore: term.Level, LevelAfter: term.Level,
|
||||
DueAtBefore: review.DueAt, DueAtAfter: review.DueAt, CreatedAt: stamp(now),
|
||||
}
|
||||
if err := tx.Create(&record).Error; err != nil {
|
||||
// Losing a race to another transaction that recorded this very answer id is not an
|
||||
// error: report the outcome that was stored first.
|
||||
var duplicate *driver.MySQLError
|
||||
if errors.As(err, &duplicate) && duplicate.Number == 1062 {
|
||||
if stored, readErr := lockedAnswerKey(tx, owner, key); readErr == nil {
|
||||
return replayedAnswer(tx, stored, term.ID)
|
||||
}
|
||||
}
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
return ReviewAnswerResult{
|
||||
Result: "stale", Grade: input.Grade, Requeued: false,
|
||||
StatusBefore: term.Status, StatusAfter: term.Status, LevelBefore: term.Level, LevelAfter: term.Level,
|
||||
DueAtBefore: review.DueAt, DueAtAfter: review.DueAt, Item: reviewItem(term, review),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// replayedAnswer reports the outcome already recorded for this answer key. Nothing is
|
||||
// advanced a second time, so a resend cannot change counts or intervals.
|
||||
func replayedAnswer(tx *gorm.DB, stored ReviewAnswer, termID int64) (ReviewAnswerResult, error) {
|
||||
if stored.TermID != termID {
|
||||
return ReviewAnswerResult{}, failure(400, "请求编号已用于其他词条")
|
||||
}
|
||||
var term Term
|
||||
if err := tx.Where("id = ? AND owner_id = ?", stored.TermID, stored.OwnerID).First(&term).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
var review TermReview
|
||||
if err := tx.Where("term_id = ? AND owner_id = ?", stored.TermID, stored.OwnerID).First(&review).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
return ReviewAnswerResult{
|
||||
Result: stored.Result, Duplicate: true, Grade: stored.Grade, Requeued: stored.Requeued,
|
||||
StatusBefore: stored.StatusBefore, StatusAfter: stored.StatusAfter,
|
||||
LevelBefore: stored.LevelBefore, LevelAfter: stored.LevelAfter,
|
||||
DueAtBefore: stored.DueAtBefore, DueAtAfter: stored.DueAtAfter, Item: reviewItem(term, review),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func registerReviewRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
v.GET("/reviews/queue", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "复习队列不接受查询参数")
|
||||
}
|
||||
language, err := languageOf(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ReviewQueueFor(tx, u.UserId, language, now())
|
||||
}))
|
||||
v.POST("/reviews/:termId/answers", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
termID, err := strconv.ParseInt(c.Param("termId"), 10, 64)
|
||||
if err != nil || termID <= 0 {
|
||||
return nil, failure(404, "词条不存在")
|
||||
}
|
||||
var input ReviewAnswerInput
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return AnswerReview(tx, u.UserId, termID, input, now())
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,501 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// TestReviewDecisionTable pins the fixed schedule and every state transition. It is the
|
||||
// design evidence the ticket asks for, so a change to the table must fail here first.
|
||||
func TestReviewDecisionTable(t *testing.T) {
|
||||
levels := []int{1, 2, 3, 4, 5, 6, 7}
|
||||
days := []int{1, 2, 4, 7, 15, 30, 60}
|
||||
if len(reviewIntervals) != len(levels) {
|
||||
t.Fatalf("interval table has %d entries, want %d", len(reviewIntervals), len(levels))
|
||||
}
|
||||
now := time.Date(2026, 9, 11, 10, 30, 0, 0, time.UTC)
|
||||
for index, level := range levels {
|
||||
if reviewIntervals[level-1] != days[index] {
|
||||
t.Fatalf("level %d interval %d, want %d", level, reviewIntervals[level-1], days[index])
|
||||
}
|
||||
next, requeued, err := applyGrade(reviewState{Status: termStatusLearning, Level: level, DueAt: now}, reviewGradeCorrect, now)
|
||||
want := level + 1
|
||||
if want > maxReviewLevel {
|
||||
want = maxReviewLevel
|
||||
}
|
||||
if err != nil || requeued || next.Level != want || !next.DueAt.Equal(now.AddDate(0, 0, reviewIntervals[want-1])) || next.Status != termStatusLearning {
|
||||
t.Fatalf("level %d correct: %#v requeued=%v err=%v", level, next, requeued, err)
|
||||
}
|
||||
}
|
||||
cases := []struct {
|
||||
name string
|
||||
before reviewState
|
||||
grade string
|
||||
status string
|
||||
level int
|
||||
dueAt time.Time
|
||||
requeued bool
|
||||
}{
|
||||
{"new correct becomes learning level 1", reviewState{Status: termStatusNew}, reviewGradeCorrect, termStatusLearning, 1, now.AddDate(0, 0, 1), false},
|
||||
{"new wrong stays new and requeues", reviewState{Status: termStatusNew}, reviewGradeWrong, termStatusNew, 0, now, true},
|
||||
{"new again stays new and requeues", reviewState{Status: termStatusNew}, reviewGradeAgain, termStatusNew, 0, now, true},
|
||||
{"learning 4 wrong drops to 3", reviewState{Status: termStatusLearning, Level: 4}, reviewGradeWrong, termStatusLearning, 3, now, true},
|
||||
{"learning 1 wrong floors at 1", reviewState{Status: termStatusLearning, Level: 1}, reviewGradeWrong, termStatusLearning, 1, now, true},
|
||||
{"learning 7 wrong drops to 6", reviewState{Status: termStatusLearning, Level: 7}, reviewGradeWrong, termStatusLearning, 6, now, true},
|
||||
{"again keeps the level", reviewState{Status: termStatusLearning, Level: 3}, reviewGradeAgain, termStatusLearning, 3, now, true},
|
||||
{"level 7 correct stays at 7", reviewState{Status: termStatusLearning, Level: 7}, reviewGradeCorrect, termStatusLearning, 7, now.AddDate(0, 0, 60), false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
before := tc.before
|
||||
before.DueAt = now
|
||||
next, requeued, err := applyGrade(before, tc.grade, now)
|
||||
if err != nil || requeued != tc.requeued || next.Status != tc.status || next.Level != tc.level || !next.DueAt.Equal(tc.dueAt) {
|
||||
t.Fatalf("%s: %#v requeued=%v err=%v", tc.name, next, requeued, err)
|
||||
}
|
||||
}
|
||||
if _, _, err := applyGrade(reviewState{Status: termStatusNew, DueAt: now}, "maybe", now); err == nil {
|
||||
t.Fatal("an unknown grade must be rejected")
|
||||
}
|
||||
}
|
||||
|
||||
// reviewFixture builds one learner, one book and one ready chapter under a clock the
|
||||
// test can move, so due boundaries are exact.
|
||||
func reviewFixture(t *testing.T, db *gorm.DB, text string) (*gin.Engine, *time.Time, admin.SysUser, string, Chapter) {
|
||||
t.Helper()
|
||||
clock := time.Now().UTC().Truncate(time.Millisecond)
|
||||
r := Router(db, func() time.Time { return clock })
|
||||
user, token, chapters := termFixture(t, db, r, text)
|
||||
return r, &clock, user, token, chapters[0]
|
||||
}
|
||||
|
||||
func reviewQueue(t *testing.T, r *gin.Engine, token string) (int, ReviewQueue) {
|
||||
t.Helper()
|
||||
code, data := callAPI(t, r, "GET", "/api/v1/reviews/queue", token, nil)
|
||||
var queue ReviewQueue
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &queue)
|
||||
}
|
||||
return code, queue
|
||||
}
|
||||
|
||||
func answerReview(t *testing.T, r *gin.Engine, token string, termID int64, body map[string]any) (int, ReviewAnswerResult) {
|
||||
t.Helper()
|
||||
code, data := callAPI(t, r, "POST", fmt.Sprintf("/api/v1/reviews/%d/answers", termID), token, body)
|
||||
var result ReviewAnswerResult
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &result)
|
||||
}
|
||||
return code, result
|
||||
}
|
||||
|
||||
func answerBody(answerID, grade string, dueAt time.Time) map[string]any {
|
||||
return map[string]any{"answerId": answerID, "grade": grade, "expectedDueAt": dueAt.UTC().Format(time.RFC3339Nano)}
|
||||
}
|
||||
|
||||
// saveWord saves one word span of an owned chapter and returns the stored record.
|
||||
func saveWord(t *testing.T, r *gin.Engine, token string, chapterID int64, start, end int, status string, level *int) TermSave {
|
||||
t.Helper()
|
||||
body := map[string]any{"chapterId": chapterID, "start": start, "end": end, "definition": "虚构释义", "status": status}
|
||||
if level != nil {
|
||||
body["level"] = *level
|
||||
}
|
||||
code, saved := saveTermAPI(t, r, token, body)
|
||||
if code != 201 && code != 200 {
|
||||
t.Fatalf("save word %d-%d: %d %#v", start, end, code, saved)
|
||||
}
|
||||
return saved
|
||||
}
|
||||
|
||||
func termReviewRow(t *testing.T, db *gorm.DB, termID int64) TermReview {
|
||||
t.Helper()
|
||||
var review TermReview
|
||||
if err := db.Where("term_id = ?", termID).First(&review).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return review
|
||||
}
|
||||
|
||||
// TestMySQLReviewQueueDueBoundaryAndScope covers the due boundary, the queue scope and
|
||||
// per-account isolation.
|
||||
func TestMySQLReviewQueueDueBoundaryAndScope(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
_, otherToken, otherChapters := termFixture(t, db, r, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
|
||||
// A saved word is due immediately: its due time is exactly the save time.
|
||||
code, queue := reviewQueue(t, r, token)
|
||||
if code != 200 || queue.Total != 1 || len(queue.Items) != 1 || queue.Items[0].ID != dogs.Term.ID {
|
||||
t.Fatalf("a new word must be due at once: %d %#v", code, queue)
|
||||
}
|
||||
item := queue.Items[0]
|
||||
if !item.DueAt.Equal(*clock) || item.Status != termStatusNew || item.Level != 0 || item.ReviewCount != 0 || item.Examples == nil {
|
||||
t.Fatalf("queue item: %#v", item)
|
||||
}
|
||||
if !termReviewRow(t, db, dogs.Term.ID).DueAt.Equal(*clock) {
|
||||
t.Fatal("the schedule row must hold the save time")
|
||||
}
|
||||
|
||||
// due_at == now is due; one millisecond earlier is not.
|
||||
*clock = item.DueAt.Add(-time.Millisecond)
|
||||
if _, early := reviewQueue(t, r, token); early.Total != 0 {
|
||||
t.Fatalf("an item due later must stay out of the queue: %#v", early)
|
||||
}
|
||||
*clock = item.DueAt.Add(time.Millisecond)
|
||||
if _, late := reviewQueue(t, r, token); late.Total != 1 {
|
||||
t.Fatal("an overdue item must be due")
|
||||
}
|
||||
|
||||
// Another account saves the same word in its own chapter without touching this queue.
|
||||
otherDogs := saveWord(t, r, otherToken, otherChapters[0].ID, 0, 4, termStatusNew, nil)
|
||||
if otherDogs.Term.ID == dogs.Term.ID {
|
||||
t.Fatal("two accounts must own separate records for the same word")
|
||||
}
|
||||
code, mine := reviewQueue(t, r, token)
|
||||
if code != 200 || len(mine.Items) != 1 || mine.Items[0].ID != dogs.Term.ID {
|
||||
t.Fatalf("queues must not mix accounts: %d %#v", code, mine)
|
||||
}
|
||||
code, theirs := reviewQueue(t, r, otherToken)
|
||||
if code != 200 || len(theirs.Items) != 1 || theirs.Items[0].ID != otherDogs.Term.ID {
|
||||
t.Fatalf("the other account must see its own queue: %d %#v", code, theirs)
|
||||
}
|
||||
|
||||
// Known and ignored words never enter the queue; returning to new makes it due again.
|
||||
for _, status := range []string{termStatusKnown, termStatusIgnored} {
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": status}); code != 200 {
|
||||
t.Fatalf("marking %s failed", status)
|
||||
}
|
||||
if _, scoped := reviewQueue(t, r, token); scoped.Total != 0 {
|
||||
t.Fatalf("%s must stay out of the queue: %#v", status, scoped)
|
||||
}
|
||||
}
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 200 {
|
||||
t.Fatal("returning to a new word failed")
|
||||
}
|
||||
if _, again := reviewQueue(t, r, token); again.Total != 1 {
|
||||
t.Fatalf("a new word must be due again: %#v", again)
|
||||
}
|
||||
|
||||
if code, _ := callAPI(t, r, "GET", "/api/v1/reviews/queue?status=new", token, nil); code != 400 {
|
||||
t.Fatal("the queue must reject query parameters")
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", "/api/v1/reviews/queue", "", nil); code != 401 {
|
||||
t.Fatal("the queue requires a session")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewAnswerTransitions answers one word three times and checks the term, the
|
||||
// counters and the recorded attempt after each grade.
|
||||
func TestMySQLReviewAnswerTransitions(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, user, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
termID := dogs.Term.ID
|
||||
|
||||
// Correct: a new word becomes learning level 1 and leaves the round.
|
||||
_, queue := reviewQueue(t, r, token)
|
||||
code, first := answerReview(t, r, token, termID, answerBody("answer-correct-0001", reviewGradeCorrect, queue.Items[0].DueAt))
|
||||
if code != 201 || first.Result != "applied" || first.Requeued {
|
||||
t.Fatalf("correct answer: %d %#v", code, first)
|
||||
}
|
||||
if first.StatusAfter != termStatusLearning || first.LevelAfter != 1 || first.Item.Level != 1 || first.Item.ReviewCount != 1 {
|
||||
t.Fatalf("correct transition: %#v", first)
|
||||
}
|
||||
if !first.DueAtAfter.Equal(clock.AddDate(0, 0, 1)) || !first.DueAtBefore.Equal(*clock) {
|
||||
t.Fatalf("correct schedule: %#v", first)
|
||||
}
|
||||
if _, empty := reviewQueue(t, r, token); empty.Total != 0 {
|
||||
t.Fatal("a correctly answered word must leave the queue until it is due again")
|
||||
}
|
||||
// One interval later the word is due again; a wrong answer requeues it immediately.
|
||||
// Sessions last eight hours of the injected clock, so the jump forward needs a login.
|
||||
*clock = first.DueAtAfter
|
||||
token = loginToken(t, r, user.Username, fixturePassword)
|
||||
_, queue = reviewQueue(t, r, token)
|
||||
if queue.Total != 1 || queue.Items[0].Level != 1 || queue.Items[0].ReviewCount != 1 {
|
||||
t.Fatalf("second round queue: %#v", queue)
|
||||
}
|
||||
code, wrong := answerReview(t, r, token, termID, answerBody("answer-wrong-000002", reviewGradeWrong, queue.Items[0].DueAt))
|
||||
if code != 201 || !wrong.Requeued || wrong.LevelAfter != 1 || wrong.StatusAfter != termStatusLearning || !wrong.DueAtAfter.Equal(*clock) {
|
||||
t.Fatalf("wrong answer: %d %#v", code, wrong)
|
||||
}
|
||||
_, queue = reviewQueue(t, r, token)
|
||||
if queue.Total != 1 || queue.Items[0].Level != 1 {
|
||||
t.Fatalf("requeued item: %#v", queue)
|
||||
}
|
||||
code, again := answerReview(t, r, token, termID, answerBody("answer-again-000003", reviewGradeAgain, queue.Items[0].DueAt))
|
||||
if code != 201 || !again.Requeued || again.LevelBefore != 1 || again.LevelAfter != 1 {
|
||||
t.Fatalf("again answer: %d %#v", code, again)
|
||||
}
|
||||
|
||||
// Counters and the attempt log accumulated exactly three answers.
|
||||
review := termReviewRow(t, db, termID)
|
||||
if review.ReviewCount != 3 || review.CorrectCount != 1 || review.WrongCount != 2 || review.LastReviewedAt == nil {
|
||||
t.Fatalf("counters: %#v", review)
|
||||
}
|
||||
var term Term
|
||||
if err := db.First(&term, termID).Error; err != nil || term.Status != termStatusLearning || term.Level != 1 || term.Definition != "虚构释义" {
|
||||
t.Fatalf("the learner's own text must not change: %#v %v", term, err)
|
||||
}
|
||||
var answers int64
|
||||
if err := db.Model(&ReviewAnswer{}).Where("owner_id = ?", user.UserId).Count(&answers).Error; err != nil || answers != 3 {
|
||||
t.Fatalf("attempt log: %d %v", answers, err)
|
||||
}
|
||||
var logged ReviewAnswer
|
||||
if err := db.Where("term_id = ? AND grade = ?", termID, reviewGradeCorrect).First(&logged).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if logged.Result != "applied" || logged.StatusBefore != termStatusNew || logged.LevelAfter != 1 || logged.Requeued {
|
||||
t.Fatalf("logged attempt: %#v", logged)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewAnswerIdempotencyAndStaleTabs is the core of the acceptance: a resend
|
||||
// must not count twice, and a second tab must not advance a term that already moved.
|
||||
func TestMySQLReviewAnswerIdempotencyAndStaleTabs(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
termID := dogs.Term.ID
|
||||
_, queue := reviewQueue(t, r, token)
|
||||
seen := queue.Items[0].DueAt
|
||||
|
||||
// Two tabs answer the same word with different answer ids but the same seen due time.
|
||||
code, applied := answerReview(t, r, token, termID, answerBody("tab-one-answer-0001", reviewGradeCorrect, seen))
|
||||
if code != 201 || applied.Result != "applied" {
|
||||
t.Fatalf("first tab: %d %#v", code, applied)
|
||||
}
|
||||
code, stale := answerReview(t, r, token, termID, answerBody("tab-two-answer-0002", reviewGradeCorrect, seen))
|
||||
if code != 200 || stale.Result != "stale" || stale.Duplicate || stale.Requeued {
|
||||
t.Fatalf("second tab must not advance the term: %d %#v", code, stale)
|
||||
}
|
||||
if stale.LevelAfter != applied.LevelAfter || !stale.DueAtAfter.Equal(applied.DueAtAfter) {
|
||||
t.Fatalf("a stale answer changed state: %#v", stale)
|
||||
}
|
||||
|
||||
// A resend of the first answer returns the first outcome, flagged as a replay, without
|
||||
// another update. The client can therefore count it exactly like the original answer.
|
||||
code, duplicate := answerReview(t, r, token, termID, answerBody("tab-one-answer-0001", reviewGradeCorrect, seen))
|
||||
if code != 200 || duplicate.Result != "applied" || !duplicate.Duplicate {
|
||||
t.Fatalf("resend: %d %#v", code, duplicate)
|
||||
}
|
||||
if duplicate.LevelAfter != applied.LevelAfter || !duplicate.DueAtAfter.Equal(applied.DueAtAfter) || duplicate.Item.ReviewCount != 1 {
|
||||
t.Fatalf("a resend changed state: %#v", duplicate)
|
||||
}
|
||||
// Counters moved exactly once, and the replay is not a new attempt row: the applied
|
||||
// answer and the stale one are two rows, the duplicate returns the first row.
|
||||
review := termReviewRow(t, db, termID)
|
||||
if review.ReviewCount != 1 || review.CorrectCount != 1 || review.WrongCount != 0 {
|
||||
t.Fatalf("counters after three attempts: %#v", review)
|
||||
}
|
||||
var attempts int64
|
||||
if err := db.Model(&ReviewAnswer{}).Where("term_id = ?", termID).Count(&attempts).Error; err != nil || attempts != 2 {
|
||||
t.Fatalf("each distinct attempt must be recorded once: %d %v", attempts, err)
|
||||
}
|
||||
|
||||
// Reusing an answer id for another word is a client error, not a new answer.
|
||||
went := saveWord(t, r, token, chapter.ID, 5, 9, termStatusNew, nil)
|
||||
code, _ = answerReview(t, r, token, went.Term.ID, answerBody("tab-one-answer-0001", reviewGradeCorrect, *clock))
|
||||
if code != 400 {
|
||||
t.Fatalf("a reused answer id must be rejected: %d", code)
|
||||
}
|
||||
// A genuinely new answer for the other word still works.
|
||||
code, fresh := answerReview(t, r, token, went.Term.ID, answerBody("went-answer-000004", reviewGradeCorrect, *clock))
|
||||
if code != 201 || fresh.LevelAfter != 1 {
|
||||
t.Fatalf("a later answer must work: %d %#v", code, fresh)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewAnswerErrorsAndOwnership covers the rejected shapes of the answer
|
||||
// endpoint.
|
||||
func TestMySQLReviewAnswerErrorsAndOwnership(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
_, otherToken, _ := termFixture(t, db, r, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
termID := dogs.Term.ID
|
||||
seen := *clock
|
||||
|
||||
if code, _ := answerReview(t, r, otherToken, termID, answerBody("foreign-answer-0001", reviewGradeCorrect, seen)); code != 404 {
|
||||
t.Fatal("another account's term must be 404")
|
||||
}
|
||||
if code, _ := answerReview(t, r, token, 999999, answerBody("missing-answer-0002", reviewGradeCorrect, seen)); code != 404 {
|
||||
t.Fatal("an unknown term must be 404")
|
||||
}
|
||||
for _, id := range []string{"0", "-3", "abc"} {
|
||||
if code, _ := callAPI(t, r, "POST", "/api/v1/reviews/"+id+"/answers", token, answerBody("bad-id-000003", reviewGradeCorrect, seen)); code != 404 {
|
||||
t.Fatalf("invalid term id %s: %d", id, code)
|
||||
}
|
||||
}
|
||||
for _, body := range []map[string]any{
|
||||
{"answerId": "grade-missing-0001", "expectedDueAt": seen.Format(time.RFC3339Nano)},
|
||||
{"answerId": "grade-unknown-0002", "grade": "maybe", "expectedDueAt": seen.Format(time.RFC3339Nano)},
|
||||
answerBody("short", reviewGradeCorrect, seen),
|
||||
{"answerId": "due-missing-0003", "grade": reviewGradeCorrect},
|
||||
{"answerId": "extra-field-0004", "grade": reviewGradeCorrect, "expectedDueAt": seen.Format(time.RFC3339Nano), "level": 7},
|
||||
} {
|
||||
if code, _ := answerReview(t, r, token, termID, body); code != 400 {
|
||||
t.Fatalf("invalid body %v must be 400: %d", body, code)
|
||||
}
|
||||
}
|
||||
if code, _ := callAPI(t, r, "POST", fmt.Sprintf("/api/v1/reviews/%d/answers", termID), "", answerBody("anonymous-000001", reviewGradeCorrect, seen)); code != 401 {
|
||||
t.Fatal("answering requires a session")
|
||||
}
|
||||
// A word marked as known in another screen is no longer reviewable.
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusKnown}); code != 200 {
|
||||
t.Fatal("marking the word known failed")
|
||||
}
|
||||
code, result := answerReview(t, r, token, termID, answerBody("known-answer-0005", reviewGradeCorrect, seen))
|
||||
if code != 409 || result.Result != "" {
|
||||
t.Fatalf("a known word must refuse an answer: %d %#v", code, result)
|
||||
}
|
||||
if review := termReviewRow(t, db, termID); review.ReviewCount != 0 {
|
||||
t.Fatalf("a refused answer must not count: %#v", review)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewScheduleOnManualStatus checks the documented rule for words saved with
|
||||
// an explicit status: new is due at once, learning waits for its level's interval.
|
||||
func TestMySQLReviewScheduleOnManualStatus(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
level := 3
|
||||
learning := saveWord(t, r, token, chapter.ID, 0, 4, termStatusLearning, &level)
|
||||
if review := termReviewRow(t, db, learning.Term.ID); !review.DueAt.Equal(clock.AddDate(0, 0, 4)) {
|
||||
t.Fatalf("a manual level must wait for its interval: %v", review.DueAt)
|
||||
}
|
||||
if _, queue := reviewQueue(t, r, token); queue.Total != 0 {
|
||||
t.Fatalf("a manually leveled word is not due today: %#v", queue)
|
||||
}
|
||||
// Changing it back to a new word makes it due immediately again.
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 200 {
|
||||
t.Fatal("resetting the status failed")
|
||||
}
|
||||
if _, queue := reviewQueue(t, r, token); queue.Total != 1 || queue.Items[0].Level != 0 {
|
||||
t.Fatalf("a new word is due at once: %#v", queue)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewConcurrentReplayOfOneAnswer reproduces the review finding that two
|
||||
// simultaneous submissions of one answer id must both get the recorded outcome.
|
||||
func TestMySQLReviewConcurrentReplayOfOneAnswer(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
_, queue := reviewQueue(t, r, token)
|
||||
seen := queue.Items[0].DueAt
|
||||
body := answerBody("concurrent-answer-01", reviewGradeCorrect, seen)
|
||||
|
||||
var wg sync.WaitGroup
|
||||
codes := make(chan int, 2)
|
||||
results := make(chan string, 2)
|
||||
for i := 0; i < 2; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
code, data := callAPI(t, r, "POST", fmt.Sprintf("/api/v1/reviews/%d/answers", dogs.Term.ID), token, body)
|
||||
var result ReviewAnswerResult
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &result)
|
||||
}
|
||||
codes <- code
|
||||
results <- result.Result
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(codes)
|
||||
close(results)
|
||||
ok, bad := 0, 0
|
||||
for code := range codes {
|
||||
if code == 200 || code == 201 {
|
||||
ok++
|
||||
} else {
|
||||
bad++
|
||||
t.Logf("status %d", code)
|
||||
}
|
||||
}
|
||||
kinds := []string{}
|
||||
for kind := range results {
|
||||
kinds = append(kinds, kind)
|
||||
}
|
||||
if ok != 2 || bad != 0 {
|
||||
t.Fatalf("both submissions must succeed: ok=%d bad=%d kinds=%v clock=%v", ok, bad, kinds, *clock)
|
||||
}
|
||||
review := termReviewRow(t, db, dogs.Term.ID)
|
||||
if review.ReviewCount != 1 {
|
||||
t.Fatalf("concurrent replay counted %d times", review.ReviewCount)
|
||||
}
|
||||
var attempts int64
|
||||
if err := db.Model(&ReviewAnswer{}).Where("term_id = ?", dogs.Term.ID).Count(&attempts).Error; err != nil || attempts != 1 {
|
||||
t.Fatalf("concurrent replay logged %d attempts", attempts)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewEditKeepsSchedule reproduces the review finding that editing only the
|
||||
// learner's own text must not move a review date (decision D2).
|
||||
func TestMySQLReviewEditKeepsSchedule(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
level := 3
|
||||
learning := saveWord(t, r, token, chapter.ID, 0, 4, termStatusLearning, &level)
|
||||
// Make the word overdue, as it would be after a missed day.
|
||||
overdue := clock.AddDate(0, 0, -5)
|
||||
if err := db.Model(&TermReview{}).Where("term_id = ?", learning.Term.ID).Update("due_at", overdue).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Editing only the personal text must keep the schedule exactly as it was.
|
||||
code, saved := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "改过的释义", "status": termStatusLearning, "level": 3})
|
||||
if code != 200 || saved.Term.Level != 3 {
|
||||
t.Fatalf("text edit: %d %#v", code, saved)
|
||||
}
|
||||
if review := termReviewRow(t, db, learning.Term.ID); !review.DueAt.Equal(overdue) {
|
||||
t.Fatalf("editing the text moved the due time: %v want %v", review.DueAt, overdue)
|
||||
}
|
||||
if _, queue := reviewQueue(t, r, token); queue.Total != 1 {
|
||||
t.Fatalf("an overdue word must stay due after a text edit: %#v", queue)
|
||||
}
|
||||
// Changing the level is a scheduling action and does move the due time.
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusLearning, "level": 5}); code != 200 {
|
||||
t.Fatal("level change failed")
|
||||
}
|
||||
if review := termReviewRow(t, db, learning.Term.ID); !review.DueAt.Equal(clock.AddDate(0, 0, 15)) {
|
||||
t.Fatalf("a level change must reschedule: %v", review.DueAt)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewPanelSaveKeepsLevel reproduces the related defect found while checking the
|
||||
// review: the reader panel saves a status without a level, and that must not reset a level
|
||||
// the learner already earned.
|
||||
func TestMySQLReviewPanelSaveKeepsLevel(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
level := 4
|
||||
learning := saveWord(t, r, token, chapter.ID, 0, 4, termStatusLearning, &level)
|
||||
overdue := clock.AddDate(0, 0, -2)
|
||||
if err := db.Model(&TermReview{}).Where("term_id = ?", learning.Term.ID).Update("due_at", overdue).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// This is exactly what the reader panel sends: status only, no level.
|
||||
code, saved := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "改过的释义", "status": termStatusLearning})
|
||||
if code != 200 || saved.Term.Level != 4 {
|
||||
t.Fatalf("a panel save must keep the learned level: %d %#v", code, saved)
|
||||
}
|
||||
if review := termReviewRow(t, db, learning.Term.ID); !review.DueAt.Equal(overdue) {
|
||||
t.Fatalf("a panel save moved the due time: %v want %v", review.DueAt, overdue)
|
||||
}
|
||||
// Leaving learning and re-entering it is a real transition and starts at level 1.
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 200 {
|
||||
t.Fatal("reset to new failed")
|
||||
}
|
||||
code, again := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusLearning})
|
||||
if code != 200 || again.Term.Level != 1 {
|
||||
t.Fatalf("re-entering learning starts at 1: %d %#v", code, again)
|
||||
}
|
||||
}
|
||||
+112
-3
@@ -129,6 +129,26 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
if c.Request.Method == "POST" && c.FullPath() == "/api/v1/accounts" {
|
||||
status = 201
|
||||
}
|
||||
// A repeated paste is answered from the first result, so it is not a new resource.
|
||||
// A pasted or uploaded chapter is one resource; a repeated submit is not.
|
||||
if c.Request.Method == "POST" && (c.FullPath() == "/api/v1/books" || c.FullPath() == "/api/v1/books/:id/chapters" ||
|
||||
c.FullPath() == "/api/v1/books/upload" || c.FullPath() == "/api/v1/books/:id/chapters/upload") {
|
||||
if paste, ok := data.(PasteResult); ok && !paste.Duplicate {
|
||||
status = 201
|
||||
}
|
||||
}
|
||||
// Saving the same word again is an update of one record, not a new resource.
|
||||
if c.Request.Method == "POST" && c.FullPath() == "/api/v1/terms" {
|
||||
if saved, ok := data.(TermSave); ok && saved.Created {
|
||||
status = 201
|
||||
}
|
||||
}
|
||||
// A replayed or stale answer changed nothing, so it is not a new answer.
|
||||
if c.Request.Method == "POST" && c.FullPath() == "/api/v1/reviews/:termId/answers" {
|
||||
if answer, ok := data.(ReviewAnswerResult); ok && answer.Result == "applied" && !answer.Duplicate {
|
||||
status = 201
|
||||
}
|
||||
}
|
||||
respond(c, status, data, err)
|
||||
}
|
||||
}
|
||||
@@ -213,18 +233,107 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
}
|
||||
return updateAccount(tx, id, u.UserId, input)
|
||||
}))
|
||||
v.POST("/books", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
var input PasteBookInput
|
||||
if err := decodeLimit(c, &input, maxPasteBodyBytes); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return PasteBook(tx, u.UserId, now(), input)
|
||||
}))
|
||||
v.GET("/books", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "书库不接受查询参数")
|
||||
}
|
||||
items, err := ListBooks(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"items": items}, nil
|
||||
}))
|
||||
v.GET("/books/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
book, chapters, err := BookDetail(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"book": book, "chapters": chapters}, nil
|
||||
}))
|
||||
v.POST("/books/:id/chapters", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var input PasteChapterInput
|
||||
if err := decodeLimit(c, &input, maxPasteBodyBytes); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return PasteChapter(tx, u.UserId, id, now(), input)
|
||||
}))
|
||||
v.GET("/chapters/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "章节不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ChapterDetail(tx, u.UserId, id)
|
||||
}))
|
||||
v.GET("/jobs/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "任务不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
job, err := JobDetail(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"job": job}, nil
|
||||
}))
|
||||
v.POST("/jobs/:id/retry", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "任务不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
job, chapter, err := RetryIngestJob(tx, u.UserId, id, now())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"job": job, "chapter": chapter}, nil
|
||||
}))
|
||||
registerDictionaryRoutes(v, protect, now)
|
||||
registerTermRoutes(v, protect, now)
|
||||
registerReviewRoutes(v, protect, now)
|
||||
registerUploadRoutes(v, protect, now)
|
||||
registerEditRoutes(v, protect, now)
|
||||
r.NoRoute(func(c *gin.Context) { respond(c, 404, nil, failure(404, "页面或接口不存在")) })
|
||||
return r
|
||||
}
|
||||
|
||||
func decode(c *gin.Context, value any) error {
|
||||
// pathID reads a positive numeric path parameter; a malformed id is reported like a
|
||||
// missing resource so it cannot be used to probe for other accounts' rows.
|
||||
func pathID(c *gin.Context, message string) (int64, error) {
|
||||
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
return 0, failure(404, message)
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
func decode(c *gin.Context, value any) error { return decodeLimit(c, value, maxJSONBodyBytes) }
|
||||
|
||||
func decodeLimit(c *gin.Context, value any, limit int64) error {
|
||||
if !strings.HasPrefix(c.GetHeader("Content-Type"), "application/json") {
|
||||
return failure(400, "请使用 JSON 请求")
|
||||
}
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, 16*1024)
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, limit)
|
||||
d := json.NewDecoder(c.Request.Body)
|
||||
d.DisallowUnknownFields()
|
||||
if d.Decode(value) != nil {
|
||||
if err := d.Decode(value); err != nil {
|
||||
var tooLarge *http.MaxBytesError
|
||||
if errors.As(err, &tooLarge) {
|
||||
return failure(400, "内容过大，请减少后重试")
|
||||
}
|
||||
return failure(400, "请求内容无效")
|
||||
}
|
||||
if d.Decode(new(any)) != io.EOF {
|
||||
|
||||
@@ -0,0 +1,373 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// The four learner-visible statuses are stored explicitly instead of the upstream
|
||||
// merged status/level code. The mapping stays available for CSV export and for
|
||||
// migrating an existing LinguaCafe instance: new=2, ignored=1, known=0, and
|
||||
// learning level N=-N. A level is only meaningful while learning, so every other
|
||||
// status keeps level 0 and the database check constraints repeat that rule.
|
||||
const (
|
||||
termStatusNew = "new"
|
||||
termStatusLearning = "learning"
|
||||
termStatusKnown = "known"
|
||||
termStatusIgnored = "ignored"
|
||||
)
|
||||
|
||||
const (
|
||||
termFormLimit = 128
|
||||
termDefinitionLimit = 2000
|
||||
termExampleLimit = 500
|
||||
termExamplesLimit = 5
|
||||
termLevelMax = 7
|
||||
// One chapter may contain many distinct words, so identity lookups are batched
|
||||
// instead of sending an unbounded IN list.
|
||||
termLookupBatch = 500
|
||||
)
|
||||
|
||||
var termStatuses = map[string]bool{
|
||||
termStatusNew: true,
|
||||
termStatusLearning: true,
|
||||
termStatusKnown: true,
|
||||
termStatusIgnored: true,
|
||||
}
|
||||
|
||||
// Term is one learner's own record for a word form. Identity is the normalized
|
||||
// form: the original spelling is kept for display only.
|
||||
type Term struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
OwnerID int
|
||||
Language string
|
||||
Term string
|
||||
OriginalForm string `gorm:"column:original_form"`
|
||||
Definition string
|
||||
Examples string
|
||||
Status string
|
||||
Level int
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (Term) TableName() string { return "lexgo_terms" }
|
||||
|
||||
type TermView struct {
|
||||
ID int64 `json:"id"`
|
||||
Language string `json:"language"`
|
||||
Term string `json:"term"`
|
||||
OriginalForm string `json:"originalForm"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
Status string `json:"status"`
|
||||
Level int `json:"level"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
// TokenTerm is what a reader token needs to render its own highlight: the entry
|
||||
// id for the follow-up read, and enough state to style the word.
|
||||
type TokenTerm struct {
|
||||
ID int64 `json:"id"`
|
||||
Status string `json:"status"`
|
||||
Level int `json:"level"`
|
||||
}
|
||||
|
||||
type TermSave struct {
|
||||
Term TermView `json:"term"`
|
||||
Created bool `json:"created"`
|
||||
}
|
||||
|
||||
// splitExamples returns the stored examples as a list; the column holds one per line.
|
||||
func splitExamples(text string) []string {
|
||||
if text == "" {
|
||||
return []string{}
|
||||
}
|
||||
return strings.Split(text, "\n")
|
||||
}
|
||||
|
||||
func termView(t Term) TermView {
|
||||
return TermView{t.ID, t.Language, t.Term, t.OriginalForm, t.Definition, splitExamples(t.Examples), t.Status, t.Level, t.CreatedAt, t.UpdatedAt}
|
||||
}
|
||||
|
||||
// termLevel enforces the documented status/level boundary: only a learning entry carries a
|
||||
// level, every other status must leave the level at 0, and entering learning starts at 1.
|
||||
// A save that does not mention a level keeps the level the learner already earned, so
|
||||
// editing a definition can never roll a word back to level 1.
|
||||
func termLevel(status string, level *int, previous Term, exists bool) (int, error) {
|
||||
if !termStatuses[status] {
|
||||
return 0, failure(400, "词语状态无效")
|
||||
}
|
||||
if status == termStatusLearning {
|
||||
if level != nil && *level != 0 {
|
||||
if *level < 1 || *level > termLevelMax {
|
||||
return 0, failure(400, "学习等级须为 1～7")
|
||||
}
|
||||
return *level, nil
|
||||
}
|
||||
if exists && previous.Status == termStatusLearning && previous.Level >= 1 {
|
||||
return previous.Level, nil
|
||||
}
|
||||
return 1, nil
|
||||
}
|
||||
if level != nil && *level != 0 {
|
||||
return 0, failure(400, "只有学习中的词语可以设置等级")
|
||||
}
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
func hasControlRune(text string, allowNewline bool) bool {
|
||||
for _, r := range text {
|
||||
if allowNewline && (r == '\n' || r == '\t') {
|
||||
continue
|
||||
}
|
||||
if unicode.IsControl(r) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// termContent validates the learner's own text and returns it in storage form:
|
||||
// the definition as typed (trimmed) and examples joined by newline.
|
||||
func termContent(definition string, examples []string) (string, string, error) {
|
||||
definition = strings.TrimSpace(definition)
|
||||
if utf8.RuneCountInString(definition) > termDefinitionLimit {
|
||||
return "", "", failure(400, "个人释义不能超过 2000 个字符")
|
||||
}
|
||||
if hasControlRune(definition, true) {
|
||||
return "", "", failure(400, "个人释义包含不支持的字符")
|
||||
}
|
||||
if len(examples) > termExamplesLimit {
|
||||
return "", "", failure(400, "例句不能超过 5 条")
|
||||
}
|
||||
cleaned := make([]string, 0, len(examples))
|
||||
for _, example := range examples {
|
||||
example = strings.TrimSpace(example)
|
||||
if example == "" {
|
||||
return "", "", failure(400, "例句不能为空行")
|
||||
}
|
||||
if utf8.RuneCountInString(example) > termExampleLimit {
|
||||
return "", "", failure(400, "每条例句不能超过 500 个字符")
|
||||
}
|
||||
if hasControlRune(example, false) {
|
||||
return "", "", failure(400, "例句包含不支持的字符")
|
||||
}
|
||||
cleaned = append(cleaned, example)
|
||||
}
|
||||
return definition, strings.Join(cleaned, "\n"), nil
|
||||
}
|
||||
|
||||
// wordAtRange returns the word the learner actually selected. Identity is always
|
||||
// derived from the server's own tokens, so a client cannot name a word, user or
|
||||
// language that it did not read from this chapter.
|
||||
func wordAtRange(chapter Chapter, start, end int) (string, error) {
|
||||
for _, token := range Tokenize(chapter.OriginalText) {
|
||||
if token.Kind == "word" && token.Start == start && token.End == end {
|
||||
if utf8.RuneCountInString(token.Text) <= termFormLimit {
|
||||
return token.Text, nil
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
return "", failure(400, "请选择不超过 128 个字符的完整单词")
|
||||
}
|
||||
|
||||
// languageOf reads the owner's study language. A missing space row falls back to
|
||||
// the English default that account creation and login already establish.
|
||||
func languageOf(tx *gorm.DB, owner int) (string, error) {
|
||||
var space Space
|
||||
err := tx.Where("owner_id = ?", owner).First(&space).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return "en", nil
|
||||
}
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return space.Language, nil
|
||||
}
|
||||
|
||||
// previousTerm reads the row this save is about to change under a lock, so the level and
|
||||
// the review schedule can be compared with what the learner already had.
|
||||
func previousTerm(tx *gorm.DB, owner int, language, term string) (Term, bool, error) {
|
||||
var existing Term
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("owner_id = ? AND language = ? AND term = ?", owner, language, term).First(&existing).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return Term{}, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return Term{}, false, err
|
||||
}
|
||||
return existing, true, nil
|
||||
}
|
||||
|
||||
// saveTerm writes one identity with INSERT ... ON DUPLICATE KEY UPDATE: a repeated
|
||||
// save updates the same row instead of adding a second, conflicting record, and
|
||||
// two concurrent saves of the same word still leave exactly one.
|
||||
func saveTerm(tx *gorm.DB, owner int, language, word string, fields TermFields, now time.Time) (TermSave, error) {
|
||||
when := stamp(now)
|
||||
row := Term{
|
||||
OwnerID: owner, Language: language, Term: normalizeWord(word), OriginalForm: word,
|
||||
Definition: fields.Definition, Examples: fields.Examples, Status: fields.Status, Level: fields.Level,
|
||||
CreatedAt: when, UpdatedAt: when,
|
||||
}
|
||||
// MySQL reports affected rows 1 for an insert and 0 or 2 for an update, so the
|
||||
// counter distinguishes "created" from "saved again" without a second read.
|
||||
insert := tx.Clauses(clause.OnConflict{
|
||||
Columns: []clause.Column{{Name: "owner_id"}, {Name: "language"}, {Name: "term"}},
|
||||
DoUpdates: clause.Assignments(map[string]any{
|
||||
"original_form": row.OriginalForm, "definition": row.Definition, "examples": row.Examples,
|
||||
"status": row.Status, "level": row.Level, "updated_at": row.UpdatedAt,
|
||||
}),
|
||||
}).Create(&row)
|
||||
if insert.Error != nil {
|
||||
return TermSave{}, insert.Error
|
||||
}
|
||||
var stored Term
|
||||
if err := tx.Where("owner_id = ? AND language = ? AND term = ?", owner, language, row.Term).First(&stored).Error; err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
// A saved word always owns a schedule row, but the date only moves for a new word or a
|
||||
// real status/level change: editing a definition must not push a word out of today's
|
||||
// queue (decision D2).
|
||||
reschedule := !fields.Exists || fields.PreviousStatus != stored.Status || fields.PreviousLevel != stored.Level
|
||||
if err := syncTermReview(tx, stored, reschedule, now); err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
return TermSave{termView(stored), insert.RowsAffected == 1}, nil
|
||||
}
|
||||
|
||||
// attachTerms marks the tokens this learner already saved. Matching is by
|
||||
// normalized form across the whole vocabulary, so a word saved in another chapter
|
||||
// is highlighted here with the same status.
|
||||
func attachTerms(tx *gorm.DB, owner int, language string, tokens []TextToken) error {
|
||||
keys := make([]string, 0, 16)
|
||||
seen := map[string]bool{}
|
||||
for _, token := range tokens {
|
||||
if token.Kind != "word" || utf8.RuneCountInString(token.Text) > termFormLimit {
|
||||
continue
|
||||
}
|
||||
key := normalizeWord(token.Text)
|
||||
if !seen[key] {
|
||||
seen[key] = true
|
||||
keys = append(keys, key)
|
||||
}
|
||||
}
|
||||
byKey := map[string]TokenTerm{}
|
||||
for start := 0; start < len(keys); start += termLookupBatch {
|
||||
end := min(start+termLookupBatch, len(keys))
|
||||
var rows []Term
|
||||
if err := tx.Select("id", "term", "status", "level").
|
||||
Where("owner_id = ? AND language = ? AND term IN ?", owner, language, keys[start:end]).
|
||||
Find(&rows).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
for _, row := range rows {
|
||||
byKey[row.Term] = TokenTerm{ID: row.ID, Status: row.Status, Level: row.Level}
|
||||
}
|
||||
}
|
||||
for i := range tokens {
|
||||
if tokens[i].Kind != "word" {
|
||||
continue
|
||||
}
|
||||
if term, ok := byKey[normalizeWord(tokens[i].Text)]; ok {
|
||||
value := term
|
||||
tokens[i].Term = &value
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// TermFields carries the validated learner text and state into storage, together with the
|
||||
// status and level the row had before this save.
|
||||
type TermFields struct {
|
||||
Definition string
|
||||
Examples string
|
||||
Status string
|
||||
Level int
|
||||
PreviousStatus string
|
||||
PreviousLevel int
|
||||
Exists bool
|
||||
}
|
||||
|
||||
type TermInput struct {
|
||||
ChapterID int64 `json:"chapterId"`
|
||||
Start *int `json:"start"`
|
||||
End *int `json:"end"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
Status string `json:"status"`
|
||||
Level *int `json:"level"`
|
||||
}
|
||||
|
||||
// registerTermRoutes exposes the learner's own word records. Nothing here is
|
||||
// written to the audit log: personal learning content stays out of it, and the
|
||||
// account-level audit already covers administrative changes.
|
||||
func registerTermRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
v.POST("/terms", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
var input TermInput
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if input.Start == nil || input.End == nil {
|
||||
return nil, failure(400, "请选择完整单词")
|
||||
}
|
||||
chapter, err := readyOwnedChapter(tx, u.UserId, input.ChapterID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
word, err := wordAtRange(chapter, *input.Start, *input.End)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
language, err := languageOf(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// The previous row decides whether a missing level keeps the earned one and whether
|
||||
// the review date may move at all.
|
||||
previous, exists, err := previousTerm(tx, u.UserId, language, normalizeWord(word))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
level, err := termLevel(input.Status, input.Level, previous, exists)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
definition, examples, err := termContent(input.Definition, input.Examples)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
fields := TermFields{
|
||||
Definition: definition, Examples: examples, Status: input.Status, Level: level,
|
||||
PreviousStatus: previous.Status, PreviousLevel: previous.Level, Exists: exists,
|
||||
}
|
||||
return saveTerm(tx, u.UserId, language, word, fields, now())
|
||||
}))
|
||||
v.GET("/terms/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
return nil, failure(404, "词条不存在")
|
||||
}
|
||||
// Another account's id and a missing id answer identically, so the response
|
||||
// never confirms that someone else's entry exists.
|
||||
var term Term
|
||||
if err := tx.Where("id = ? AND owner_id = ?", id, u.UserId).First(&term).Error; errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil, failure(404, "词条不存在")
|
||||
} else if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"term": termView(term)}, nil
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,378 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func TestTermLevelBoundary(t *testing.T) {
|
||||
level := func(v int) *int { return &v }
|
||||
// The stored state the save is about to change: a level-4 learning word.
|
||||
stored := Term{Status: termStatusLearning, Level: 4}
|
||||
cases := []struct {
|
||||
status string
|
||||
level *int
|
||||
previous Term
|
||||
exists bool
|
||||
want int
|
||||
ok bool
|
||||
}{
|
||||
{termStatusNew, nil, Term{}, false, 0, true},
|
||||
{termStatusNew, level(0), Term{}, false, 0, true},
|
||||
{termStatusNew, level(1), Term{}, false, 0, false},
|
||||
{termStatusKnown, level(0), stored, true, 0, true},
|
||||
{termStatusKnown, level(3), stored, true, 0, false},
|
||||
{termStatusIgnored, level(0), stored, true, 0, true},
|
||||
{termStatusIgnored, level(-1), stored, true, 0, false},
|
||||
{termStatusLearning, nil, Term{}, false, 1, true},
|
||||
{termStatusLearning, level(0), Term{}, false, 1, true},
|
||||
{termStatusLearning, level(1), stored, true, 1, true},
|
||||
{termStatusLearning, level(7), stored, true, 7, true},
|
||||
{termStatusLearning, level(8), stored, true, 0, false},
|
||||
{termStatusLearning, level(-1), stored, true, 0, false},
|
||||
{"", nil, stored, true, 0, false},
|
||||
{"Learning", nil, stored, true, 0, false},
|
||||
{"deleted", nil, stored, true, 0, false},
|
||||
}
|
||||
// A save that does not mention a level keeps the earned one instead of resetting it.
|
||||
cases = append(cases, struct {
|
||||
status string
|
||||
level *int
|
||||
previous Term
|
||||
exists bool
|
||||
want int
|
||||
ok bool
|
||||
}{termStatusLearning, nil, stored, true, 4, true})
|
||||
for _, tc := range cases {
|
||||
got, err := termLevel(tc.status, tc.level, tc.previous, tc.exists)
|
||||
if tc.ok && (err != nil || got != tc.want) {
|
||||
t.Fatalf("%s/%v: got %d, %v; want %d", tc.status, tc.level, got, err, tc.want)
|
||||
}
|
||||
if !tc.ok && err == nil {
|
||||
t.Fatalf("%s/%v: invalid state accepted as %d", tc.status, tc.level, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTermContentRules(t *testing.T) {
|
||||
definition, examples, err := termContent(" 好奇\t心\n求知 ", []string{" A fictional line. ", "Second line"})
|
||||
if err != nil || definition != "好奇\t心\n求知" || examples != "A fictional line.\nSecond line" {
|
||||
t.Fatalf("trim and join: %q %q %v", definition, examples, err)
|
||||
}
|
||||
if definition, examples, err = termContent(" ", nil); err != nil || definition != "" || examples != "" {
|
||||
t.Fatalf("an empty definition is allowed: %q %q %v", definition, examples, err)
|
||||
}
|
||||
rejected := []struct {
|
||||
name string
|
||||
def string
|
||||
examples []string
|
||||
}{
|
||||
{"definition too long", strings.Repeat("字", termDefinitionLimit+1), nil},
|
||||
{"definition control character", "a\u0007b", nil},
|
||||
{"empty example", "ok", []string{"fine", " "}},
|
||||
{"example with newline", "ok", []string{"two\nlines"}},
|
||||
{"example too long", "ok", []string{strings.Repeat("a", termExampleLimit+1)}},
|
||||
{"too many examples", "ok", []string{"1", "2", "3", "4", "5", "6"}},
|
||||
}
|
||||
for _, tc := range rejected {
|
||||
if _, _, err := termContent(tc.def, tc.examples); err == nil {
|
||||
t.Fatalf("%s was accepted", tc.name)
|
||||
}
|
||||
}
|
||||
if _, _, err := termContent(strings.Repeat("字", termDefinitionLimit), []string{strings.Repeat("a", termExampleLimit)}); err != nil {
|
||||
t.Fatalf("boundary values must be accepted: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWordAtRangeUsesServerTokens(t *testing.T) {
|
||||
chapter := Chapter{OriginalText: "Cats went. Dogs"}
|
||||
for _, tc := range []struct {
|
||||
start, end int
|
||||
want string
|
||||
}{
|
||||
{0, 4, "Cats"},
|
||||
{5, 9, "went"},
|
||||
{11, 15, "Dogs"},
|
||||
} {
|
||||
got, err := wordAtRange(chapter, tc.start, tc.end)
|
||||
if err != nil || got != tc.want {
|
||||
t.Fatalf("%d-%d: got %q %v, want %q", tc.start, tc.end, got, err, tc.want)
|
||||
}
|
||||
}
|
||||
for _, span := range [][2]int{{0, 3}, {0, 5}, {4, 5}, {9, 10}, {10, 11}, {0, 15}, {-1, 3}, {12, 15}, {0, 0}} {
|
||||
if _, err := wordAtRange(chapter, span[0], span[1]); err == nil {
|
||||
t.Fatalf("range %v accepted as a word", span)
|
||||
}
|
||||
}
|
||||
long := "a" + strings.Repeat("b", termFormLimit)
|
||||
if _, err := wordAtRange(Chapter{OriginalText: long}, 0, len([]rune(long))); err == nil {
|
||||
t.Fatal("over-long word accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTermViewSplitsExamples(t *testing.T) {
|
||||
base := Term{ID: 3, Language: "en", Term: "curiosity", OriginalForm: "Curiosity", Status: termStatusNew}
|
||||
if got := termView(base); len(got.Examples) != 0 || got.Examples == nil {
|
||||
t.Fatalf("empty examples must serialize as []: %#v", got.Examples)
|
||||
}
|
||||
base.Examples = "One.\nTwo."
|
||||
if got := termView(base); len(got.Examples) != 2 || got.Examples[1] != "Two." {
|
||||
t.Fatalf("examples: %#v", got.Examples)
|
||||
}
|
||||
}
|
||||
|
||||
// termFixture creates one learner with a private book holding one ready chapter per
|
||||
// text. Fictional content only; nothing here touches another account's data.
|
||||
func termFixture(t *testing.T, db *gorm.DB, r *gin.Engine, texts ...string) (admin.SysUser, string, []Chapter) {
|
||||
t.Helper()
|
||||
u := admin.SysUser{Username: randomName("term"), Password: fixturePassword, RoleId: 2, Status: "2"}
|
||||
if err := db.Create(&u).Error; err != nil {
|
||||
t.Fatal("fixture learner creation failed")
|
||||
}
|
||||
now := stamp(time.Now())
|
||||
book := Book{OwnerID: u.UserId, Title: "Fictional terms", Language: "en", CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&book).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var chapters []Chapter
|
||||
for index, text := range texts {
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: u.UserId, Ordinal: index + 1, Title: fmt.Sprintf("Fictional %d", index+1),
|
||||
OriginalText: text, ContentSHA256: contentSHA(text), CharCount: len([]rune(text)), Status: statusReady, CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&chapter).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
chapters = append(chapters, chapter)
|
||||
}
|
||||
return u, loginToken(t, r, u.Username, fixturePassword), chapters
|
||||
}
|
||||
|
||||
func saveTermAPI(t *testing.T, r *gin.Engine, token string, body map[string]any) (int, TermSave) {
|
||||
t.Helper()
|
||||
code, data := callAPI(t, r, "POST", "/api/v1/terms", token, body)
|
||||
var saved TermSave
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &saved)
|
||||
}
|
||||
return code, saved
|
||||
}
|
||||
|
||||
func chapterTokens(t *testing.T, r *gin.Engine, token string, chapterID int64) (int, ChapterTokens) {
|
||||
t.Helper()
|
||||
code, data := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", chapterID), token, nil)
|
||||
var analyzed ChapterTokens
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &analyzed)
|
||||
}
|
||||
return code, analyzed
|
||||
}
|
||||
|
||||
// wordTokens returns only the selectable words, so a test can talk about the third
|
||||
// word instead of the sixth token.
|
||||
func wordTokens(tokens []TextToken) []TextToken {
|
||||
words := make([]TextToken, 0, len(tokens))
|
||||
for _, token := range tokens {
|
||||
if token.Kind == "word" {
|
||||
words = append(words, token)
|
||||
}
|
||||
}
|
||||
return words
|
||||
}
|
||||
|
||||
// TestMySQLTermIdentityAndCrossChapterConsistency covers the acceptance items that
|
||||
// one saved word stays one record, that case is not a second word, that another
|
||||
// inflection is its own entry, and that another chapter shows the same state.
|
||||
func TestMySQLTermIdentityAndCrossChapterConsistency(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r := Router(db, time.Now)
|
||||
learner, token, chapters := termFixture(t, db, r, "Dogs dogs dog.", "Dogs elsewhere")
|
||||
chapter, elsewhere := chapters[0], chapters[1]
|
||||
|
||||
code, saved := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "狗", "examples": []string{"Fictional dogs."}, "status": termStatusNew,
|
||||
})
|
||||
if code != 201 || !saved.Created || saved.Term.ID == 0 {
|
||||
t.Fatalf("first save: %d %#v", code, saved)
|
||||
}
|
||||
if saved.Term.Term != "dogs" || saved.Term.OriginalForm != "Dogs" || saved.Term.Language != "en" || saved.Term.Level != 0 {
|
||||
t.Fatalf("stored identity: %#v", saved.Term)
|
||||
}
|
||||
first := saved.Term.ID
|
||||
|
||||
// The exact same save is idempotent: same record, same status code, no duplicate.
|
||||
code, repeated := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "狗", "examples": []string{"Fictional dogs."}, "status": termStatusNew,
|
||||
})
|
||||
if code != 200 || repeated.Created || repeated.Term.ID != first {
|
||||
t.Fatalf("identical repeat must update one record: %d %#v", code, repeated)
|
||||
}
|
||||
|
||||
// Repeating the save updates the same record instead of adding a second one.
|
||||
code, again := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "狗；犬", "examples": []string{}, "status": termStatusLearning, "level": 3,
|
||||
})
|
||||
if code != 200 || again.Created || again.Term.ID != first || again.Term.Definition != "狗；犬" || again.Term.Status != termStatusLearning || again.Term.Level != 3 {
|
||||
t.Fatalf("repeated save: %d %#v", code, again)
|
||||
}
|
||||
// The same word in another case is the same identity.
|
||||
code, lower := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 5, "end": 9, "definition": "狗", "status": termStatusKnown,
|
||||
})
|
||||
if code != 200 || lower.Created || lower.Term.ID != first || lower.Term.OriginalForm != "dogs" || lower.Term.Level != 0 {
|
||||
t.Fatalf("case folded identity: %d %#v", code, lower)
|
||||
}
|
||||
// A different inflection stays its own entry instead of merging by lemma.
|
||||
code, inflection := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 10, "end": 13, "definition": "狗（单数）", "status": termStatusNew,
|
||||
})
|
||||
if code != 201 || inflection.Term.ID == first {
|
||||
t.Fatalf("inflection must be a separate entry: %d %#v", code, inflection)
|
||||
}
|
||||
var owned int64
|
||||
if err := db.Model(&Term{}).Where("owner_id = ?", learner.UserId).Count(&owned).Error; err != nil || owned != 2 {
|
||||
t.Fatalf("one word must stay one record: count=%d err=%v", owned, err)
|
||||
}
|
||||
|
||||
// The status saved here is the status the reader shows for this chapter.
|
||||
code, tokens := chapterTokens(t, r, token, chapter.ID)
|
||||
words := wordTokens(tokens.Tokens)
|
||||
if code != 200 || len(words) != 3 {
|
||||
t.Fatalf("tokens: %d %#v", code, tokens.Tokens)
|
||||
}
|
||||
if words[0].Term == nil || words[0].Term.ID != first || words[0].Term.Status != termStatusKnown {
|
||||
t.Fatalf("first token term: %#v", words[0])
|
||||
}
|
||||
if words[2].Term == nil || words[2].Term.ID != inflection.Term.ID {
|
||||
t.Fatalf("second entry token term: %#v", words[2])
|
||||
}
|
||||
for _, token := range tokens.Tokens {
|
||||
if token.Kind != "word" && token.Term != nil {
|
||||
t.Fatalf("spaces and punctuation must not carry a term: %#v", token)
|
||||
}
|
||||
}
|
||||
// Another chapter of the same learner shows the same entry without saving again.
|
||||
code, crossTokens := chapterTokens(t, r, token, elsewhere.ID)
|
||||
crossWords := wordTokens(crossTokens.Tokens)
|
||||
if code != 200 || len(crossWords) != 2 || crossWords[0].Term == nil || crossWords[0].Term.ID != first || crossWords[0].Term.Status != termStatusKnown {
|
||||
t.Fatalf("cross-chapter highlight: %d %#v", code, crossTokens.Tokens)
|
||||
}
|
||||
// Another account sees the same text with no personal state at all.
|
||||
_, otherToken, otherChapters := termFixture(t, db, r, "Dogs elsewhere")
|
||||
code, otherTokens := chapterTokens(t, r, otherToken, otherChapters[0].ID)
|
||||
otherWords := wordTokens(otherTokens.Tokens)
|
||||
if code != 200 || len(otherWords) != 2 || otherWords[0].Term != nil {
|
||||
t.Fatalf("another account must not see a term: %d %#v", code, otherTokens.Tokens)
|
||||
}
|
||||
if code, _ := chapterTokens(t, r, otherToken, chapter.ID); code != 404 {
|
||||
t.Fatalf("foreign chapter must be 404: %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTermIsolationAndInputRules covers ownership, tampered input and the
|
||||
// status/level boundary through the real HTTP surface.
|
||||
func TestMySQLTermIsolationAndInputRules(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r := Router(db, time.Now)
|
||||
_, ownerToken, chapters := termFixture(t, db, r, "Cats went.")
|
||||
chapter := chapters[0]
|
||||
otherUser, otherToken, _ := termFixture(t, db, r, "Cats went.")
|
||||
|
||||
code, saved := saveTermAPI(t, r, ownerToken, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "猫", "status": termStatusNew})
|
||||
if code != 201 {
|
||||
t.Fatalf("owner save: %d", code)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/terms/%d", saved.Term.ID), ownerToken, nil); code != 200 {
|
||||
t.Fatalf("owner read: %d", code)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/terms/%d", saved.Term.ID), otherToken, nil); code != 404 {
|
||||
t.Fatalf("foreign read must be 404: %d", code)
|
||||
}
|
||||
for _, id := range []string{"0", "-1", "abc", "99999999999999999999"} {
|
||||
if code, _ := callAPI(t, r, "GET", "/api/v1/terms/"+id, ownerToken, nil); code != 404 {
|
||||
t.Fatalf("invalid id %s: %d", id, code)
|
||||
}
|
||||
}
|
||||
if code, _ := callAPI(t, r, "POST", "/api/v1/terms", "", map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 401 {
|
||||
t.Fatal("anonymous save must be 401")
|
||||
}
|
||||
if code, _ := saveTermAPI(t, r, otherToken, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 404 {
|
||||
t.Fatal("saving into a foreign chapter must be 404")
|
||||
}
|
||||
// A client cannot name the owner, the language, the word or the record.
|
||||
for _, body := range []map[string]any{
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "ownerId": otherUser.UserId},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "language": "fr"},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "term": "forged"},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "id": 1},
|
||||
} {
|
||||
if code, _ := saveTermAPI(t, r, ownerToken, body); code != 400 {
|
||||
t.Fatalf("tampered input %v must be 400: %d", body, code)
|
||||
}
|
||||
}
|
||||
// Invalid or partial ranges, an unknown status and a forbidden level.
|
||||
for _, body := range []map[string]any{
|
||||
{"chapterId": chapter.ID, "start": 0, "status": termStatusNew},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 3, "status": termStatusNew},
|
||||
{"chapterId": chapter.ID, "start": 1, "end": 4, "status": termStatusNew},
|
||||
{"chapterId": chapter.ID, "start": 4, "end": 5, "status": termStatusNew},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": "deleted"},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusKnown, "level": 4},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusLearning, "level": 8},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "definition": strings.Repeat("a", termDefinitionLimit+1)},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "examples": []string{"1", "2", "3", "4", "5", "6"}},
|
||||
} {
|
||||
if code, _ := saveTermAPI(t, r, ownerToken, body); code != 400 {
|
||||
t.Fatalf("invalid input %v must be 400: %d", body, code)
|
||||
}
|
||||
}
|
||||
// Entering learning without a level starts at 1 and still updates the same entry.
|
||||
code, learning := saveTermAPI(t, r, ownerToken, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusLearning})
|
||||
if code != 200 || learning.Term.ID != saved.Term.ID || learning.Term.Level != 1 {
|
||||
t.Fatalf("learning default level: %d %#v", code, learning)
|
||||
}
|
||||
// A chapter that is not ready refuses the save instead of storing a word.
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", chapter.ID).Update("status", statusPending).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code, _ := saveTermAPI(t, r, ownerToken, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 409 {
|
||||
t.Fatal("pending chapter must be 409")
|
||||
}
|
||||
// An unknown chapter id never confirms whether it belongs to someone else.
|
||||
if code, _ := saveTermAPI(t, r, ownerToken, map[string]any{"chapterId": 999999, "start": 0, "end": 4, "status": termStatusNew}); code != 404 {
|
||||
t.Fatal("unknown chapter must be 404")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTermTextRoundTrip checks the learner's own text, including characters
|
||||
// that are never normalized away.
|
||||
func TestMySQLTermTextRoundTrip(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r := Router(db, time.Now)
|
||||
_, token, chapters := termFixture(t, db, r, "Café Dogs")
|
||||
code, saved := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapters[0].ID, "start": 0, "end": 4,
|
||||
"definition": "咖啡\n附带说明", "examples": []string{"A fictional example.", "😀 second"},
|
||||
"status": termStatusLearning, "level": 7,
|
||||
})
|
||||
if code != 201 {
|
||||
t.Fatalf("save: %d", code)
|
||||
}
|
||||
code, data := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/terms/%d", saved.Term.ID), token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("read: %d", code)
|
||||
}
|
||||
var response struct {
|
||||
Term TermView `json:"term"`
|
||||
}
|
||||
json.Unmarshal(data, &response)
|
||||
if response.Term.Definition != "咖啡\n附带说明" || len(response.Term.Examples) != 2 || response.Term.Examples[1] != "😀 second" || response.Term.Level != 7 {
|
||||
t.Fatalf("round trip: %#v", response.Term)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,183 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// A TXT upload is decoded in memory and handed to the same paste pipeline. Nothing is
|
||||
// written to disk: there is no temporary file to leak, and the client file name never
|
||||
// becomes a path, so it cannot reach outside the request.
|
||||
const maxTextUploadBytes = 2 << 20
|
||||
|
||||
var (
|
||||
utf8BOM = []byte{0xEF, 0xBB, 0xBF}
|
||||
utf16BOMBig = []byte{0xFE, 0xFF}
|
||||
utf16BOMSmall = []byte{0xFF, 0xFE}
|
||||
)
|
||||
|
||||
// decodeTextUpload turns an uploaded TXT into the exact text the reader will show. Only
|
||||
// UTF-8 is accepted: an optional BOM is removed before the text is validated, and any byte
|
||||
// that is not valid UTF-8 rejects the file instead of being replaced, so a chapter never
|
||||
// contains a substitute character the learner did not write.
|
||||
func decodeTextUpload(raw []byte) (string, error) {
|
||||
if len(raw) > maxTextUploadBytes {
|
||||
return "", failure(400, "TXT 文件不能超过 2 MiB")
|
||||
}
|
||||
if bytes.HasPrefix(raw, utf16BOMBig) || bytes.HasPrefix(raw, utf16BOMSmall) {
|
||||
return "", failure(400, "文件是 UTF-16 编码，请另存为 UTF-8 后重试")
|
||||
}
|
||||
raw = bytes.TrimPrefix(raw, utf8BOM)
|
||||
if !utf8.Valid(raw) {
|
||||
return "", failure(400, "文件不是 UTF-8 编码，请另存为 UTF-8 后重试")
|
||||
}
|
||||
if bytes.IndexByte(raw, 0) >= 0 {
|
||||
return "", failure(400, "文件包含无法处理的字符，请另存为纯文本后重试")
|
||||
}
|
||||
return string(raw), nil
|
||||
}
|
||||
|
||||
type textUpload struct {
|
||||
RequestID string
|
||||
Title string
|
||||
Language string
|
||||
Text string
|
||||
}
|
||||
|
||||
// readTextUpload parses the multipart submission: one file part plus a whitelist of text
|
||||
// fields. The uploaded name is never read, not even for validation, because it only serves
|
||||
// display in the browser.
|
||||
func readTextUpload(c *gin.Context, allowLanguage bool) (textUpload, error) {
|
||||
bad := failure(400, "TXT 上传无效，请选择 UTF-8 的 .txt 文件并填写标题")
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, maxTextUploadBytes+(64<<10))
|
||||
reader, err := c.Request.MultipartReader()
|
||||
if err != nil {
|
||||
return textUpload{}, bad
|
||||
}
|
||||
fields := map[string]string{}
|
||||
var content []byte
|
||||
for {
|
||||
part, err := reader.NextPart()
|
||||
if errors.Is(err, io.EOF) {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return textUpload{}, uploadBodyError(err, bad)
|
||||
}
|
||||
name := part.FormName()
|
||||
if name == "file" {
|
||||
if content != nil {
|
||||
part.Close()
|
||||
return textUpload{}, bad
|
||||
}
|
||||
content, err = io.ReadAll(io.LimitReader(part, maxTextUploadBytes+1))
|
||||
part.Close()
|
||||
if err != nil {
|
||||
return textUpload{}, uploadBodyError(err, bad)
|
||||
}
|
||||
if len(content) == 0 || len(content) > maxTextUploadBytes {
|
||||
return textUpload{}, failure(400, "TXT 文件不能为空且不能超过 2 MiB")
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !textUploadField(name, allowLanguage) {
|
||||
part.Close()
|
||||
return textUpload{}, bad
|
||||
}
|
||||
if _, exists := fields[name]; exists {
|
||||
part.Close()
|
||||
return textUpload{}, bad
|
||||
}
|
||||
value, readErr := io.ReadAll(io.LimitReader(part, 1025))
|
||||
part.Close()
|
||||
if readErr != nil {
|
||||
return textUpload{}, uploadBodyError(readErr, bad)
|
||||
}
|
||||
if len(value) > 1024 || !utf8.Valid(value) {
|
||||
return textUpload{}, bad
|
||||
}
|
||||
fields[name] = string(value)
|
||||
}
|
||||
if content == nil {
|
||||
return textUpload{}, bad
|
||||
}
|
||||
text, err := decodeTextUpload(content)
|
||||
if err != nil {
|
||||
return textUpload{}, err
|
||||
}
|
||||
return textUpload{RequestID: fields["requestId"], Title: fields["title"], Language: fields["language"], Text: text}, nil
|
||||
}
|
||||
|
||||
func textUploadField(name string, allowLanguage bool) bool {
|
||||
switch name {
|
||||
case "requestId", "title":
|
||||
return true
|
||||
case "language":
|
||||
return allowLanguage
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// An oversized body is reported as a size problem, not as an invalid upload.
|
||||
func uploadBodyError(err error, bad error) error {
|
||||
var tooLarge *http.MaxBytesError
|
||||
if errors.As(err, &tooLarge) {
|
||||
return failure(400, "TXT 文件不能超过 2 MiB")
|
||||
}
|
||||
return bad
|
||||
}
|
||||
|
||||
// registerUploadRoutes reuses the paste pipeline: the decoded file becomes the same
|
||||
// PasteBook/PasteChapter input, so idempotency, ownership and the ingest job behave exactly
|
||||
// as they do for pasted text. One upload at a time keeps concurrent large decodes bounded.
|
||||
func registerUploadRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
gate := make(chan struct{}, 1)
|
||||
enter := func() error {
|
||||
select {
|
||||
case gate <- struct{}{}:
|
||||
return nil
|
||||
default:
|
||||
return failure(429, "已有文件正在上传，请稍后重试")
|
||||
}
|
||||
}
|
||||
leave := func() { <-gate }
|
||||
v.POST("/books/upload", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if err := enter(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer leave()
|
||||
upload, err := readTextUpload(c, true)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return PasteBook(tx, u.UserId, now(), PasteBookInput{
|
||||
RequestID: upload.RequestID, Title: upload.Title, Text: upload.Text, Language: upload.Language,
|
||||
})
|
||||
}))
|
||||
v.POST("/books/:id/chapters/upload", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
bookID, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := enter(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer leave()
|
||||
upload, err := readTextUpload(c, false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return PasteChapter(tx, u.UserId, bookID, now(), PasteChapterInput{
|
||||
RequestID: upload.RequestID, Title: upload.Title, Text: upload.Text,
|
||||
})
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,376 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"mime/multipart"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
// uploadFixture is the same character set the paste contract preserves, written as the bytes
|
||||
// a file would hold: CRLF and LF, a tab, curly quotes, an em dash, an ellipsis, an emoji, a
|
||||
// combining acute accent, a trailing space run and an empty final line.
|
||||
const uploadFixture = "Mira opened the workshop.\r\n\r\n\tThe sign read “A small step…” — café e\u0301 🙂\r\nTrailing spaces here: \n\n"
|
||||
|
||||
// uploadFile posts one multipart TXT submission. A nil file means "no file part", and an
|
||||
// empty name means "no file name", so the rejection paths stay testable.
|
||||
func uploadFile(t *testing.T, r *gin.Engine, token, path, fileName string, content []byte, fields map[string]string) (int, string, pasteResponse) {
|
||||
t.Helper()
|
||||
var body bytes.Buffer
|
||||
writer := multipart.NewWriter(&body)
|
||||
names := make([]string, 0, len(fields))
|
||||
for name := range fields {
|
||||
names = append(names, name)
|
||||
}
|
||||
// Field order must be stable so a failure message is reproducible.
|
||||
for _, name := range []string{"requestId", "title", "language"} {
|
||||
if value, ok := fields[name]; ok {
|
||||
if err := writer.WriteField(name, value); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
names = removeString(names, name)
|
||||
}
|
||||
}
|
||||
for _, name := range names {
|
||||
if err := writer.WriteField(name, fields[name]); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if content != nil {
|
||||
part, err := writer.CreateFormFile("file", fileName)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = part.Write(content); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
writer.Close()
|
||||
request := httptest.NewRequest("POST", path, &body)
|
||||
request.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
if token != "" {
|
||||
request.Header.Set("Authorization", "Bearer "+token)
|
||||
}
|
||||
response := httptest.NewRecorder()
|
||||
r.ServeHTTP(response, request)
|
||||
var envelope struct {
|
||||
Msg string `json:"msg"`
|
||||
Data json.RawMessage `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(response.Body.Bytes(), &envelope); err != nil {
|
||||
t.Fatalf("upload response for %s: %v", path, err)
|
||||
}
|
||||
var out pasteResponse
|
||||
if len(envelope.Data) > 0 {
|
||||
json.Unmarshal(envelope.Data, &out)
|
||||
}
|
||||
return response.Code, envelope.Msg, out
|
||||
}
|
||||
|
||||
func removeString(values []string, target string) []string {
|
||||
result := values[:0]
|
||||
for _, value := range values {
|
||||
if value != target {
|
||||
result = append(result, value)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func TestDecodeTextUploadRules(t *testing.T) {
|
||||
// Valid UTF-8 is returned exactly as received, including unusual but legal characters.
|
||||
if text, err := decodeTextUpload([]byte(uploadFixture)); err != nil || text != uploadFixture {
|
||||
t.Fatalf("valid file: %q %v", text, err)
|
||||
}
|
||||
// A UTF-8 BOM is removed and never becomes part of the original text.
|
||||
withBOM := append(append([]byte{}, utf8BOM...), []byte("BOM before text\n")...)
|
||||
if text, err := decodeTextUpload(withBOM); err != nil || text != "BOM before text\n" {
|
||||
t.Fatalf("BOM file: %q %v", text, err)
|
||||
}
|
||||
// Only a BOM leaves an empty text, which the paste rules then reject.
|
||||
if text, err := decodeTextUpload(append([]byte{}, utf8BOM...)); err != nil || text != "" {
|
||||
t.Fatalf("BOM only: %q %v", text, err)
|
||||
}
|
||||
rejected := []struct {
|
||||
name string
|
||||
content []byte
|
||||
message string
|
||||
}{
|
||||
{"invalid UTF-8", []byte{0x41, 0x80, 0x42}, "UTF-8"},
|
||||
{"latin-1 text", []byte("caf\xe9 plain\n"), "UTF-8"},
|
||||
{"UTF-16 little endian", []byte{0xFF, 0xFE, 0x41, 0x00}, "UTF-16"},
|
||||
{"UTF-16 big endian", []byte{0xFE, 0xFF, 0x00, 0x41}, "UTF-16"},
|
||||
{"NUL byte", []byte("text\x00more"), "无法处理"},
|
||||
}
|
||||
for _, tc := range rejected {
|
||||
text, err := decodeTextUpload(tc.content)
|
||||
if err == nil || text != "" {
|
||||
t.Fatalf("%s was accepted as %q", tc.name, text)
|
||||
}
|
||||
api, ok := err.(*apiError)
|
||||
if !ok || api.status != 400 || !strings.Contains(api.message, tc.message) {
|
||||
t.Fatalf("%s message: %v", tc.name, err)
|
||||
}
|
||||
}
|
||||
// The size boundary is exact on both sides, and the byte limit is checked before decoding.
|
||||
if _, err := decodeTextUpload(bytes.Repeat([]byte("a"), maxTextUploadBytes)); err != nil {
|
||||
t.Fatalf("a file at the size limit must be accepted: %v", err)
|
||||
}
|
||||
if _, err := decodeTextUpload(bytes.Repeat([]byte("a"), maxTextUploadBytes+1)); err == nil {
|
||||
t.Fatal("a file over the size limit must be rejected")
|
||||
}
|
||||
// The paste rules still bound one chapter, so the byte limit cannot smuggle in more text.
|
||||
if _, _, _, err := validatePaste("title", strings.Repeat("a", maxChapterRunes)); err != nil {
|
||||
t.Fatalf("the exact chapter limit must be accepted: %v", err)
|
||||
}
|
||||
if _, _, _, err := validatePaste("title", strings.Repeat("a", maxChapterRunes+1)); err == nil {
|
||||
t.Fatal("the chapter code point limit must still apply to uploaded text")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadImportPath covers the accepted file: it creates the same book, chapter
|
||||
// and job as a paste, and the reader text equals the file byte for byte.
|
||||
func TestMySQLTextUploadImportPath(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
fields := map[string]string{"requestId": "upload-fixture-0001", "title": "The Workshop Upload", "language": "en"}
|
||||
code, msg, uploaded := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "workshop.txt", []byte(uploadFixture), fields)
|
||||
if code != 201 {
|
||||
t.Fatalf("upload status %d (%s)", code, msg)
|
||||
}
|
||||
if uploaded.Book == nil || uploaded.Book.Title != "The Workshop Upload" || uploaded.Book.Language != "en" {
|
||||
t.Fatalf("unexpected book %+v", uploaded.Book)
|
||||
}
|
||||
if uploaded.Chapter.Ordinal != 1 || uploaded.Chapter.Status != statusPending || uploaded.Duplicate {
|
||||
t.Fatalf("unexpected chapter %+v", uploaded.Chapter)
|
||||
}
|
||||
|
||||
// The worker publishes the chapter, and the reader shows exactly the file content.
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, uploaded.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.OriginalText != uploadFixture {
|
||||
t.Fatalf("reader text: status %d, %+v", code, ready.Chapter)
|
||||
}
|
||||
if want := len([]rune(uploadFixture)); ready.Chapter.CharCount != want {
|
||||
t.Fatalf("charCount %d, want %d", ready.Chapter.CharCount, want)
|
||||
}
|
||||
|
||||
// A UTF-8 BOM is stripped, so the reader never shows it.
|
||||
bomFields := map[string]string{"requestId": "upload-bom-000002", "title": "BOM Upload", "language": "en"}
|
||||
withBOM := append(append([]byte{}, utf8BOM...), []byte("Plain text with BOM.\n")...)
|
||||
code, msg, bom := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "bom.txt", withBOM, bomFields)
|
||||
if code != 201 {
|
||||
t.Fatalf("BOM upload status %d (%s)", code, msg)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
if code, read := readChapter(t, r, learner.Token, bom.Chapter.ID); code != 200 || read.Chapter.OriginalText != "Plain text with BOM.\n" {
|
||||
t.Fatalf("BOM text: status %d, %q", code, read.Chapter.OriginalText)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadIdempotencyAndAppend reuses the paste job rules: one file yields one
|
||||
// chapter, a repeated upload answers with that chapter, and the same request id with other
|
||||
// content is a conflict.
|
||||
func TestMySQLTextUploadIdempotencyAndAppend(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
fields := map[string]string{"requestId": "upload-repeat-0001", "title": "Repeat Upload", "language": "en"}
|
||||
code, msg, first := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "repeat.txt", []byte("First upload body.\n"), fields)
|
||||
if code != 201 {
|
||||
t.Fatalf("first upload %d (%s)", code, msg)
|
||||
}
|
||||
code, msg, again := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "repeat.txt", []byte("First upload body.\n"), fields)
|
||||
if code != 200 || !again.Duplicate || again.Chapter.ID != first.Chapter.ID {
|
||||
t.Fatalf("repeat upload %d (%s): %+v", code, msg, again)
|
||||
}
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", first.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("a repeated upload created %d chapters", chapters)
|
||||
}
|
||||
// The same request id with other content is a conflict, not a second chapter.
|
||||
code, msg, _ = uploadFile(t, r, learner.Token, "/api/v1/books/upload", "repeat.txt", []byte("Different body.\n"), fields)
|
||||
if code != 409 {
|
||||
t.Fatalf("changed content status %d (%s)", code, msg)
|
||||
}
|
||||
|
||||
// Appending uses the same rules and the same job pipeline.
|
||||
appendFields := map[string]string{"requestId": "upload-append-0002", "title": "Second Chapter"}
|
||||
path := fmt.Sprintf("/api/v1/books/%d/chapters/upload", first.Book.ID)
|
||||
code, msg, appended := uploadFile(t, r, learner.Token, path, "second.txt", []byte("A single plain paragraph.\n"), appendFields)
|
||||
if code != 201 {
|
||||
t.Fatalf("append upload %d (%s)", code, msg)
|
||||
}
|
||||
if appended.Chapter.Ordinal != 2 || appended.Chapter.BookID != first.Book.ID {
|
||||
t.Fatalf("unexpected appended chapter %+v", appended.Chapter)
|
||||
}
|
||||
code, msg, againAppend := uploadFile(t, r, learner.Token, path, "second.txt", []byte("A single plain paragraph.\n"), appendFields)
|
||||
if code != 200 || !againAppend.Duplicate || againAppend.Chapter.ID != appended.Chapter.ID {
|
||||
t.Fatalf("repeated append %d (%s): %+v", code, msg, againAppend)
|
||||
}
|
||||
// The append endpoint does not accept a language field: the book owns the language.
|
||||
code, msg, _ = uploadFile(t, r, learner.Token, path, "second.txt", []byte("Another body.\n"),
|
||||
map[string]string{"requestId": "upload-append-lang-0004", "title": "Second Chapter", "language": "en"})
|
||||
if code != 400 {
|
||||
t.Fatalf("append with a language field: status %d (%s)", code, msg)
|
||||
}
|
||||
|
||||
// Another account cannot append into this book, and never learns whether it exists.
|
||||
code, foreignMsg, _ := uploadFile(t, r, other.Token, path, "second.txt", []byte("Foreign body.\n"), map[string]string{"requestId": "upload-foreign-0003", "title": "Foreign"})
|
||||
if code != 404 {
|
||||
t.Fatalf("foreign append status %d (%s)", code, foreignMsg)
|
||||
}
|
||||
// The other account's own library stays empty.
|
||||
_, list := bookList(t, r, other.Token)
|
||||
if len(list.Items) != 0 {
|
||||
t.Fatalf("the other account must not see this book: %+v", list.Items)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadRejectsInvalidSubmissions covers the validation surface, including the
|
||||
// client file name, which is never used as a path.
|
||||
func TestMySQLTextUploadRejectsInvalidSubmissions(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
base := map[string]string{"requestId": "upload-invalid-0001", "title": "Invalid Upload", "language": "en"}
|
||||
cases := []struct {
|
||||
name string
|
||||
fileName string
|
||||
content []byte
|
||||
fields map[string]string
|
||||
status int
|
||||
}{
|
||||
{"no file part", "", nil, base, 400},
|
||||
{"empty file", "empty.txt", []byte{}, base, 400},
|
||||
{"empty file part", "empty.txt", []byte{}, base, 400},
|
||||
{"whitespace only", "blank.txt", []byte(" \n\t\n"), base, 400},
|
||||
{"BOM only", "bom.txt", append([]byte{}, utf8BOM...), base, 400},
|
||||
{"invalid UTF-8", "latin1.txt", []byte("caf\xe9\n"), base, 400},
|
||||
{"UTF-16 file", "unicode.txt", []byte{0xFF, 0xFE, 0x41, 0x00}, base, 400},
|
||||
{"oversized file", "big.txt", bytes.Repeat([]byte("a"), maxTextUploadBytes+1), base, 400},
|
||||
{"missing request id", "text.txt", []byte("Body.\n"), map[string]string{"title": "Invalid Upload", "language": "en"}, 400},
|
||||
{"missing title", "text.txt", []byte("Body.\n"), map[string]string{"requestId": "upload-invalid-0002", "language": "en"}, 400},
|
||||
{"unsupported language", "text.txt", []byte("Body.\n"), map[string]string{"requestId": "upload-invalid-0004", "title": "Invalid Upload", "language": "fr"}, 400},
|
||||
{"short request id", "text.txt", []byte("Body.\n"), map[string]string{"requestId": "short", "title": "Invalid Upload", "language": "en"}, 400},
|
||||
{"unknown field", "text.txt", []byte("Body.\n"), map[string]string{"requestId": "upload-invalid-0005", "title": "Invalid Upload", "language": "en", "ownerId": "9"}, 400},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
code, msg, _ := uploadFile(t, r, learner.Token, "/api/v1/books/upload", tc.fileName, tc.content, tc.fields)
|
||||
if code != tc.status {
|
||||
t.Fatalf("%s: status %d (%s), want %d", tc.name, code, msg, tc.status)
|
||||
}
|
||||
if msg == "" {
|
||||
t.Fatalf("%s: rejection without a readable message", tc.name)
|
||||
}
|
||||
}
|
||||
// No rejected submission left a book behind.
|
||||
_, list := bookList(t, r, learner.Token)
|
||||
if len(list.Items) != 0 {
|
||||
t.Fatalf("rejected uploads created books: %+v", list.Items)
|
||||
}
|
||||
|
||||
// A missing language field follows the paste rule and defaults to English.
|
||||
code, msg, defaulted := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "default.txt", []byte("Body without language.\n"),
|
||||
map[string]string{"requestId": "upload-default-lang-0007", "title": "Default Language"})
|
||||
if code != 201 || defaulted.Book == nil || defaulted.Book.Language != "en" {
|
||||
t.Fatalf("missing language must default to English: status %d (%s) book %+v", code, msg, defaulted.Book)
|
||||
}
|
||||
|
||||
// The uploaded name is only a display string: a traversal-shaped name changes nothing.
|
||||
hostile := map[string]string{"requestId": "upload-hostile-0006", "title": "Hostile Name", "language": "en"}
|
||||
code, msg, uploaded := uploadFile(t, r, learner.Token, "/api/v1/books/upload", `..\..\windows\system32\evil.txt`, []byte("Hostile but harmless.\n"), hostile)
|
||||
if code != 201 {
|
||||
t.Fatalf("hostile name status %d (%s)", code, msg)
|
||||
}
|
||||
var chapter Chapter
|
||||
if err := db.First(&chapter, uploaded.Chapter.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var book Book
|
||||
if err := db.First(&book, uploaded.Book.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, field := range []string{chapter.Title, chapter.OriginalText, book.Title} {
|
||||
if strings.Contains(field, "evil") || strings.Contains(field, "system32") || strings.Contains(field, `..`) {
|
||||
t.Fatalf("the uploaded name leaked into stored data: %q", field)
|
||||
}
|
||||
}
|
||||
|
||||
// An unrelated content type is not a multipart upload.
|
||||
request := httptest.NewRequest("POST", "/api/v1/books/upload", strings.NewReader(`{"requestId":"json-upload-0001"}`))
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
request.Header.Set("Authorization", "Bearer "+learner.Token)
|
||||
response := httptest.NewRecorder()
|
||||
r.ServeHTTP(response, request)
|
||||
if response.Code != 400 {
|
||||
t.Fatalf("JSON body accepted as an upload: %d", response.Code)
|
||||
}
|
||||
// Uploading without a session is rejected before any parsing.
|
||||
code, _, _ = uploadFile(t, r, "", "/api/v1/books/upload", "text.txt", []byte("Body.\n"), base)
|
||||
if code != 401 {
|
||||
t.Fatalf("anonymous upload status %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadKeepsChapterLimit proves the byte cap cannot bypass the one-chapter
|
||||
// code point rule, and that a large but legal file is stored completely.
|
||||
func TestMySQLTextUploadKeepsChapterLimit(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
// Exactly at the chapter limit: accepted, and the stored text keeps its full length.
|
||||
atLimit := strings.Repeat("a", maxChapterRunes-1) + "\n"
|
||||
fields := map[string]string{"requestId": "upload-limit-0001", "title": "At The Limit", "language": "en"}
|
||||
code, msg, uploaded := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "limit.txt", []byte(atLimit), fields)
|
||||
if code != 201 {
|
||||
t.Fatalf("upload at the chapter limit %d (%s)", code, msg)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
if code, read := readChapter(t, r, learner.Token, uploaded.Chapter.ID); code != 200 || read.Chapter.OriginalText != atLimit {
|
||||
t.Fatalf("chapter at the limit: status %d, length %d", code, len([]rune(read.Chapter.OriginalText)))
|
||||
}
|
||||
// One code point more is rejected by the same rule that already applies to a paste.
|
||||
overLimit := strings.Repeat("a", maxChapterRunes+1)
|
||||
code, msg, _ = uploadFile(t, r, learner.Token, "/api/v1/books/upload", "over.txt", []byte(overLimit), map[string]string{"requestId": "upload-limit-0002", "title": "Over The Limit", "language": "en"})
|
||||
if code != 400 || !strings.Contains(msg, "100000") {
|
||||
t.Fatalf("upload over the chapter limit: status %d (%s)", code, msg)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadAndPasteShareOnePipeline checks the two entry points cannot produce a
|
||||
// second chapter for the same submitted content when the client stays on one request id.
|
||||
func TestMySQLTextUploadAndPasteShareOnePipeline(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
fields := map[string]string{"requestId": "upload-shared-0001", "title": "Shared Pipeline", "language": "en"}
|
||||
code, msg, uploaded := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "shared.txt", []byte("Shared body.\n"), fields)
|
||||
if code != 201 {
|
||||
t.Fatalf("upload %d (%s)", code, msg)
|
||||
}
|
||||
// The same request id through the paste endpoint answers with the uploaded chapter.
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "upload-shared-0001", "title": "Shared Pipeline", "text": "Shared body.\n", "language": "en"})
|
||||
if code != 200 || !pasted.Duplicate || pasted.Chapter.ID != uploaded.Chapter.ID {
|
||||
t.Fatalf("paste after upload %d: %+v", code, pasted)
|
||||
}
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", uploaded.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("the two entry points created %d chapters", chapters)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,338 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bufio"
|
||||
"bytes"
|
||||
"errors"
|
||||
"io"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode"
|
||||
|
||||
"golang.org/x/text/unicode/norm"
|
||||
)
|
||||
|
||||
const WordNetSHA = "cbda5ea6eef7f36a97a43d4a75f85e07fccbb4f23657d27b4ccbc93e2646ab59"
|
||||
const WordNetSource = "https://raw.githubusercontent.com/nltk/nltk_data/96f9b3252457a2b97e52aec64c3dfceeb5c312d5/packages/corpora/wordnet.zip"
|
||||
const maxDictionaryZip = 32 << 20
|
||||
const maxDictionaryInflated = 128 << 20
|
||||
|
||||
type TextToken struct {
|
||||
Text string `json:"text"`
|
||||
Start int `json:"start"`
|
||||
End int `json:"end"`
|
||||
StartUtf16 int `json:"startUtf16"`
|
||||
EndUtf16 int `json:"endUtf16"`
|
||||
Kind string `json:"kind"`
|
||||
Term *TokenTerm `json:"term,omitempty"`
|
||||
}
|
||||
|
||||
// Tokenize never normalizes text. Letters begin words; marks continue them and
|
||||
// apostrophes join letters. Numbers and symbols are non-selectable punctuation.
|
||||
// All offsets are half-open, in code points and UTF-16 code units respectively.
|
||||
func Tokenize(text string) []TextToken {
|
||||
runes := []rune(text)
|
||||
result := make([]TextToken, 0)
|
||||
utf16Offset := 0
|
||||
for i := 0; i < len(runes); {
|
||||
start, start16 := i, utf16Offset
|
||||
kind := "punctuation"
|
||||
if unicode.IsLetter(runes[i]) {
|
||||
kind = "word"
|
||||
} else if unicode.IsSpace(runes[i]) {
|
||||
kind = "space"
|
||||
}
|
||||
i++
|
||||
for i < len(runes) {
|
||||
if kind == "word" && (unicode.IsLetter(runes[i]) || unicode.IsMark(runes[i]) || ((runes[i] == '\'' || runes[i] == '’') && i+1 < len(runes) && unicode.IsLetter(runes[i+1]))) {
|
||||
i++
|
||||
continue
|
||||
}
|
||||
if kind == "space" && unicode.IsSpace(runes[i]) {
|
||||
i++
|
||||
continue
|
||||
}
|
||||
break
|
||||
}
|
||||
for _, r := range runes[start:i] {
|
||||
utf16Offset++
|
||||
if r > 0xffff {
|
||||
utf16Offset++
|
||||
}
|
||||
}
|
||||
result = append(result, TextToken{Text: string(runes[start:i]), Start: start, End: i, StartUtf16: start16, EndUtf16: utf16Offset, Kind: kind})
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func normalizeWord(word string) string {
|
||||
return norm.NFC.String(strings.ToLower(strings.ReplaceAll(word, "’", "'")))
|
||||
}
|
||||
|
||||
type DictionaryEntry struct {
|
||||
Lemma string `json:"lemma"`
|
||||
POS string `json:"pos"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
}
|
||||
type LookupResult struct {
|
||||
Status string `json:"status"`
|
||||
Query string `json:"query"`
|
||||
MatchedForm *string `json:"matchedForm"`
|
||||
Candidates []string `json:"candidates"`
|
||||
Entries []DictionaryEntry `json:"entries"`
|
||||
Resource *LookupResource `json:"resource,omitempty"`
|
||||
}
|
||||
type LookupResource struct {
|
||||
Name string `json:"name"`
|
||||
Version string `json:"version"`
|
||||
}
|
||||
type wordNetPOS struct {
|
||||
pos string
|
||||
index map[string][]int
|
||||
data map[int]DictionaryEntry
|
||||
exceptions map[string][]string
|
||||
}
|
||||
type WordNet struct {
|
||||
parts []wordNetPOS
|
||||
EntryCount int
|
||||
}
|
||||
|
||||
// ParseWordNet accepts only the pinned official corpus. No archive member is ever
|
||||
// extracted to disk; both declared and actual inflated sizes are bounded.
|
||||
func ParseWordNet(raw []byte) (*WordNet, error) {
|
||||
if len(raw) > maxDictionaryZip || contentSHA(string(raw)) != WordNetSHA {
|
||||
return nil, errors.New("WordNet ZIP checksum mismatch")
|
||||
}
|
||||
return parseWordNetArchive(raw)
|
||||
}
|
||||
|
||||
func readWordNetArchive(raw []byte) (map[string][]byte, error) {
|
||||
if len(raw) > maxDictionaryZip {
|
||||
return nil, errors.New("archive too large")
|
||||
}
|
||||
reader, err := zip.NewReader(bytes.NewReader(raw), int64(len(raw)))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(reader.File) > 64 {
|
||||
return nil, errors.New("too many archive members")
|
||||
}
|
||||
files := map[string][]byte{}
|
||||
seen := map[string]bool{}
|
||||
total := int64(0)
|
||||
for _, f := range reader.File {
|
||||
name := f.Name
|
||||
if strings.Contains(name, "\\") || path.IsAbs(name) || strings.Contains(name, ":") || path.Clean(name) != strings.TrimSuffix(name, "/") || !strings.HasPrefix(name, "wordnet/") || seen[name] {
|
||||
return nil, errors.New("unsafe archive member")
|
||||
}
|
||||
seen[name] = true
|
||||
if !f.FileInfo().IsDir() && !f.Mode().IsRegular() {
|
||||
return nil, errors.New("unsupported archive member")
|
||||
}
|
||||
if f.FileInfo().IsDir() {
|
||||
continue
|
||||
}
|
||||
if f.UncompressedSize64 > maxDictionaryInflated || total+int64(f.UncompressedSize64) > maxDictionaryInflated {
|
||||
return nil, errors.New("inflated archive too large")
|
||||
}
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(rc, maxDictionaryInflated-total+1))
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
total += int64(len(data))
|
||||
if total > maxDictionaryInflated {
|
||||
return nil, errors.New("inflated archive too large")
|
||||
}
|
||||
files[strings.TrimPrefix(name, "wordnet/")] = data
|
||||
}
|
||||
for _, name := range []string{"LICENSE", "README", "lexnames", "index.noun", "index.verb", "index.adj", "index.adv", "data.noun", "data.verb", "data.adj", "data.adv", "noun.exc", "verb.exc", "adj.exc", "adv.exc"} {
|
||||
if len(files[name]) == 0 {
|
||||
return nil, errors.New("missing required WordNet member")
|
||||
}
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func parseWordNetArchive(raw []byte) (*WordNet, error) {
|
||||
files, err := readWordNetArchive(raw)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := &WordNet{}
|
||||
for i, suffix := range []string{"noun", "verb", "adj", "adv"} {
|
||||
part := wordNetPOS{pos: []string{"n", "v", "a", "r"}[i], index: map[string][]int{}, data: map[int]DictionaryEntry{}, exceptions: map[string][]string{}}
|
||||
scanner := bufio.NewScanner(bytes.NewReader(files["data."+suffix]))
|
||||
scanner.Buffer(make([]byte, 4096), 1<<20)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if line == "" || line[0] == ' ' {
|
||||
continue
|
||||
}
|
||||
header, gloss, ok := strings.Cut(line, "|")
|
||||
fields := strings.Fields(header)
|
||||
if !ok || len(fields) < 6 {
|
||||
return nil, errors.New("invalid WordNet data record")
|
||||
}
|
||||
offset, e := strconv.Atoi(fields[0])
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
if _, exists := part.data[offset]; exists {
|
||||
return nil, errors.New("duplicate synset offset")
|
||||
}
|
||||
definition, examples := wordNetGloss(gloss)
|
||||
part.data[offset] = DictionaryEntry{POS: fields[2], Definition: definition, Examples: examples}
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
scanner = bufio.NewScanner(bytes.NewReader(files["index."+suffix]))
|
||||
scanner.Buffer(make([]byte, 4096), 1<<20)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if line == "" || line[0] == ' ' {
|
||||
continue
|
||||
}
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) < 7 {
|
||||
return nil, errors.New("invalid WordNet index record")
|
||||
}
|
||||
count, e := strconv.Atoi(fields[2])
|
||||
if e != nil || count < 1 || count > 1000 {
|
||||
return nil, errors.New("invalid sense count")
|
||||
}
|
||||
pointers, e := strconv.Atoi(fields[3])
|
||||
if e != nil || pointers < 0 || len(fields) != 6+pointers+count {
|
||||
return nil, errors.New("invalid index offsets")
|
||||
}
|
||||
offsets := make([]int, 0, count)
|
||||
for _, rawOffset := range fields[6+pointers:] {
|
||||
offset, e := strconv.Atoi(rawOffset)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
if _, ok := part.data[offset]; !ok {
|
||||
return nil, errors.New("missing synset")
|
||||
}
|
||||
offsets = append(offsets, offset)
|
||||
}
|
||||
if _, exists := part.index[fields[0]]; exists {
|
||||
return nil, errors.New("duplicate index key")
|
||||
}
|
||||
part.index[fields[0]] = offsets
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
scanner = bufio.NewScanner(bytes.NewReader(files[suffix+".exc"]))
|
||||
for scanner.Scan() {
|
||||
fields := strings.Fields(scanner.Text())
|
||||
if len(fields) < 2 {
|
||||
return nil, errors.New("invalid exception record")
|
||||
}
|
||||
part.exceptions[fields[0]] = fields[1:]
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result.EntryCount += len(part.index)
|
||||
result.parts = append(result.parts, part)
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func wordNetGloss(gloss string) (string, []string) {
|
||||
examples := []string{}
|
||||
definition := strings.Builder{}
|
||||
for {
|
||||
before, after, ok := strings.Cut(gloss, "\"")
|
||||
definition.WriteString(before)
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
example, rest, closed := strings.Cut(after, "\"")
|
||||
if !closed {
|
||||
definition.WriteString(after)
|
||||
break
|
||||
}
|
||||
examples = append(examples, example)
|
||||
gloss = rest
|
||||
}
|
||||
return strings.Trim(strings.TrimSpace(definition.String()), "; "), examples
|
||||
}
|
||||
|
||||
var morphyRules = map[string][][2]string{
|
||||
"n": {{"s", ""}, {"ses", "s"}, {"ves", "f"}, {"xes", "x"}, {"zes", "z"}, {"ches", "ch"}, {"shes", "sh"}, {"men", "man"}, {"ies", "y"}},
|
||||
"v": {{"s", ""}, {"ies", "y"}, {"es", "e"}, {"es", ""}, {"ed", "e"}, {"ed", ""}, {"ing", "e"}, {"ing", ""}},
|
||||
"a": {{"er", ""}, {"est", ""}, {"er", "e"}, {"est", "e"}},
|
||||
}
|
||||
|
||||
// Morphy yields possible dictionary forms, never contextual POS or a merged
|
||||
// learning identity. Exact forms win before exceptions and suffix detachment.
|
||||
func (wn *WordNet) Lookup(query string) LookupResult {
|
||||
result := LookupResult{Status: "not_found", Query: query, Candidates: []string{}, Entries: []DictionaryEntry{}}
|
||||
key := normalizeWord(query)
|
||||
add := func(form string, part wordNetPOS) {
|
||||
if len(part.index[form]) == 0 {
|
||||
return
|
||||
}
|
||||
found := false
|
||||
for _, candidate := range result.Candidates {
|
||||
if candidate == form {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
result.Candidates = append(result.Candidates, form)
|
||||
}
|
||||
for _, offset := range part.index[form] {
|
||||
if len(result.Entries) >= 12 {
|
||||
break
|
||||
}
|
||||
entry := part.data[offset]
|
||||
entry.Lemma = strings.ReplaceAll(form, "_", " ")
|
||||
result.Entries = append(result.Entries, entry)
|
||||
}
|
||||
}
|
||||
for _, part := range wn.parts {
|
||||
add(key, part)
|
||||
}
|
||||
if len(result.Entries) > 0 {
|
||||
result.Status = "exact"
|
||||
} else {
|
||||
for _, part := range wn.parts {
|
||||
forms := part.exceptions[key]
|
||||
if len(forms) == 0 {
|
||||
for _, rule := range morphyRules[part.pos] {
|
||||
if strings.HasSuffix(key, rule[0]) && len(key) > len(rule[0]) {
|
||||
forms = append(forms, strings.TrimSuffix(key, rule[0])+rule[1])
|
||||
}
|
||||
}
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, form := range forms {
|
||||
if !seen[form] {
|
||||
add(form, part)
|
||||
seen[form] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(result.Entries) > 0 {
|
||||
result.Status = "lemma"
|
||||
}
|
||||
}
|
||||
if len(result.Candidates) > 0 {
|
||||
form := result.Candidates[0]
|
||||
result.MatchedForm = &form
|
||||
}
|
||||
return result
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode/utf16"
|
||||
)
|
||||
|
||||
func TestArchiveRejectsUnsafeMembersAndMissingLicense(t *testing.T) {
|
||||
for _, name := range []string{"../data.noun", "wordnet/../data.noun", "/wordnet/data.noun", "wordnet\\data.noun", "wordnet/C:data.noun", "wordnet/LICENSE"} {
|
||||
var raw bytes.Buffer
|
||||
w := zip.NewWriter(&raw)
|
||||
entry, err := w.Create(name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entry.Write([]byte("fictional fixture"))
|
||||
w.Close()
|
||||
if _, err := readWordNetArchive(raw.Bytes()); err == nil {
|
||||
t.Fatalf("accepted unsafe/incomplete archive %s", name)
|
||||
}
|
||||
}
|
||||
var raw bytes.Buffer
|
||||
w := zip.NewWriter(&raw)
|
||||
for i := 0; i < 2; i++ {
|
||||
entry, _ := w.Create("wordnet/LICENSE")
|
||||
entry.Write([]byte("fixture"))
|
||||
}
|
||||
w.Close()
|
||||
if _, err := readWordNetArchive(raw.Bytes()); err == nil {
|
||||
t.Fatal("accepted duplicate member")
|
||||
}
|
||||
if _, err := ParseWordNet(make([]byte, maxDictionaryZip+1)); err == nil {
|
||||
t.Fatal("accepted oversized archive")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMorphyCandidatesAndExactPrecedence(t *testing.T) {
|
||||
wn := &WordNet{parts: []wordNetPOS{{pos: "v", index: map[string][]int{"axes": {1}, "ax": {1}, "axe": {2}}, data: map[int]DictionaryEntry{1: {POS: "v", Definition: "fixture one", Examples: []string{}}, 2: {POS: "v", Definition: "fixture two", Examples: []string{}}}, exceptions: map[string][]string{}}}}
|
||||
if got := wn.Lookup("AXES"); got.Status != "exact" || len(got.Candidates) != 1 || *got.MatchedForm != "axes" {
|
||||
t.Fatal("exact must precede lemma", got)
|
||||
}
|
||||
delete(wn.parts[0].index, "axes")
|
||||
got := wn.Lookup("axes")
|
||||
if got.Status != "lemma" || strings.Join(got.Candidates, ",") != "axe,ax" {
|
||||
t.Fatal("deterministic candidate forms", got)
|
||||
}
|
||||
definition, examples := wordNetGloss(`a fictional gloss; "one example"; "second example"`)
|
||||
if definition != "a fictional gloss" || len(examples) != 2 {
|
||||
t.Fatal("gloss", definition, examples)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTokensPreserveUnicodeOriginal(t *testing.T) {
|
||||
text := "😀 Cafe\u0301 can’t\r\nDogs 123 中文!"
|
||||
tokens := Tokenize(text)
|
||||
runes := []rune(text)
|
||||
cursor, u16 := 0, 0
|
||||
words := []string{}
|
||||
for _, token := range tokens {
|
||||
if token.Start != cursor || token.StartUtf16 != u16 || string(runes[token.Start:token.End]) != token.Text {
|
||||
t.Fatalf("incorrect token %#v", token)
|
||||
}
|
||||
cursor = token.End
|
||||
u16 += len(utf16.Encode([]rune(token.Text)))
|
||||
if token.EndUtf16 != u16 {
|
||||
t.Fatal("incorrect UTF-16 end")
|
||||
}
|
||||
if token.Kind == "word" {
|
||||
words = append(words, token.Text)
|
||||
}
|
||||
}
|
||||
if cursor != len(runes) || strings.Join(words, "|") != "Cafe\u0301|can’t|Dogs|中文" {
|
||||
t.Fatalf("coverage/words %v", words)
|
||||
}
|
||||
if normalizeWord("CAFE\u0301") != "café" || normalizeWord("CAN’T") != "can't" {
|
||||
t.Fatal("normalization")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOfficialWordNet(t *testing.T) {
|
||||
raw, err := os.ReadFile("../../../.local/nlp-resources/wordnet.zip")
|
||||
if os.IsNotExist(err) {
|
||||
t.Skip("explicitly prepared official WordNet fixture unavailable")
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
engine, err := ParseWordNet(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if engine.EntryCount < 150000 {
|
||||
t.Fatalf("entry count %d", engine.EntryCount)
|
||||
}
|
||||
for _, tc := range []struct{ query, status, lemma string }{{"dog", "exact", "dog"}, {"went", "lemma", "go"}, {"mice", "lemma", "mouse"}, {"zzzznonword", "not_found", ""}} {
|
||||
got := engine.Lookup(tc.query)
|
||||
if got.Status != tc.status || (tc.lemma != "" && (got.MatchedForm == nil || *got.MatchedForm != tc.lemma)) {
|
||||
t.Fatalf("%s: %#v", tc.query, got)
|
||||
}
|
||||
if tc.lemma != "" && (len(got.Entries) == 0 || got.Entries[0].Definition == "") {
|
||||
t.Fatal("missing definition")
|
||||
}
|
||||
}
|
||||
raw[100] ^= 1
|
||||
if _, err := ParseWordNet(raw); err == nil {
|
||||
t.Fatal("accepted corrupt checksum")
|
||||
}
|
||||
}
|
||||
@@ -82,7 +82,7 @@ func run() error {
|
||||
if err = lexgo.Migrate(db); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println("LexGo schema version 2 ready")
|
||||
fmt.Println("LexGo schema version", lexgo.SchemaVersion, "ready")
|
||||
return nil
|
||||
}
|
||||
if err = lexgo.CheckSchema(db); err != nil {
|
||||
@@ -112,6 +112,36 @@ func run() error {
|
||||
srv := &http.Server{Addr: addr, Handler: lexgo.Router(db, time.Now), ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 15 * time.Second, WriteTimeout: 15 * time.Second, IdleTimeout: 60 * time.Second, MaxHeaderBytes: 1 << 20}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
// A single instance owns the worker. Startup recovery returns chapters left in processing
|
||||
// by an unclean stop, and the running loop sweeps jobs whose finishing transaction failed,
|
||||
// so a transient database problem does not need a restart.
|
||||
if _, err = lexgo.RecoverIngestJobs(db, time.Now()); err != nil {
|
||||
return errors.New("ingestion recovery failed")
|
||||
}
|
||||
go func() {
|
||||
ticker := time.NewTicker(time.Second)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
jobCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||
requeued, recoverErr := lexgo.RequeueStaleIngestJobs(db, time.Now())
|
||||
if recoverErr != nil {
|
||||
if ctx.Err() == nil {
|
||||
log.Print("ingestion recovery failed; the next sweep retries it")
|
||||
}
|
||||
} else if requeued > 0 && ctx.Err() == nil {
|
||||
log.Printf("requeued %d interrupted ingestion job(s)", requeued)
|
||||
}
|
||||
if _, err := lexgo.ProcessIngestJobs(jobCtx, db, time.Now, 20); err != nil && ctx.Err() == nil {
|
||||
log.Print("ingestion batch stopped before finishing; the claimed job stays processing until the next sweep requeues it")
|
||||
}
|
||||
cancel()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
}
|
||||
}()
|
||||
go func() {
|
||||
ticker := time.NewTicker(time.Hour)
|
||||
defer ticker.Stop()
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"name": "Princeton WordNet",
|
||||
"language": "en",
|
||||
"version": "3.0",
|
||||
"format": "wordnet-3.0-zip",
|
||||
"source": "https://raw.githubusercontent.com/nltk/nltk_data/96f9b3252457a2b97e52aec64c3dfceeb5c312d5/packages/corpora/wordnet.zip",
|
||||
"sha256": "cbda5ea6eef7f36a97a43d4a75f85e07fccbb4f23657d27b4ccbc93e2646ab59",
|
||||
"license": "WORDNET-LICENSE.txt",
|
||||
"licenseUrl": "https://wordnet.princeton.edu/license-and-commercial-use",
|
||||
"documentation": [
|
||||
"https://wordnet.princeton.edu/documentation/wndb5wn",
|
||||
"https://wordnet.princeton.edu/documentation/morphy7wn"
|
||||
],
|
||||
"runtime": "Go only; explicitly imported ZIP persisted in MySQL; no runtime download or Python process"
|
||||
}
|
||||
Reference in New Issue
Block a user