Compare commits
26
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 ila
|
a658fa6fd1 | ||
|
ila
|
b1b76cfb7c | ||
|
ila
|
bd77aa0a46 | ||
|
ila
|
1319c56cf8 | ||
|
ila
|
1d19b2b92f | ||
|
ila
|
35ed692c4b | ||
|
ila
|
65b50d5038 | ||
|
ila
|
fb256d4916 | ||
|
ila
|
61d3f63c69 | ||
|
ila
|
cd2b893bba | ||
|
ila
|
85b8e24429 | ||
|
ila
|
325849816e | ||
|
ila
|
ec5ec2db35 | ||
|
ila
|
0328505b77 | ||
|
ila
|
527d8af0c4 | ||
|
ila
|
a5a8c3c72d | ||
|
ila
|
8c0946af9f | ||
|
ila
|
33182e584b | ||
|
ila
|
ad31ea4225 | ||
|
ila
|
8310438bee | ||
|
ila
|
b18f9cc4a5 | ||
|
ila
|
a55708cd37 | ||
|
ila
|
ed5da31888 | ||
|
ila
|
8d4a095590 | ||
|
ila
|
2b6aa90813 | ||
|
ila
|
b7c976eb75 |
@@ -258,7 +258,7 @@ MVP 内所有单元任务通过后才能做 MVP 集成验收；MVP 通过后才
|
||||
|
||||
- 治理模式：轻量。数据库：MySQL 8（用户于 2026-09-10 确认）；具体小版本在工程验证后锁定。
|
||||
- 远端：https://git.ilapage.cn/OPC/lexgo.git；分支 main。不得把邻接 dev_harness 工作区当成本项目工作区。
|
||||
- 工程基础 #2 已通过用户验收：server 基于指定 go-admin 选用模型扩展账号/会话 API，admin 复用 go-admin-ui，learner 为独立 Vue 3 + TypeScript + Vite 工程。默认英语；阅读、导入、词典、复习及 Python NLP 尚未实现或验证。
|
||||
- 工程基础 #2 已通过用户验收：server 基于指定 go-admin 选用模型扩展账号/会话 API，admin 复用 go-admin-ui，learner 为独立 Vue 3 + TypeScript + Vite 工程。默认英语；阅读、导入、词典与复习尚未接入产品；#3 独立 Python NLP/词典验证小样已通过用户验收。
|
||||
- 原四份研究保留为历史参考；PostgreSQL 建议被 MySQL 8 决策覆盖，U/A/N 索引用于追踪而不是批准所有范围。
|
||||
- 用户/语言数据所有权、Unicode 原文位置、任务和复习幂等、完整备份恢复是后续方案的必要验收边界。
|
||||
- 当前 MCP 连接其他 Gitea 站点，需使用目标站点 API 时记录原因；凭据仅从安全配置进入进程。
|
||||
@@ -273,8 +273,16 @@ MVP 内所有单元任务通过后才能做 MVP 集成验收；MVP 通过后才
|
||||
|
||||
- F01～F12 已确认进入 MVP，X 系列本轮不纳入；用户指定 Quant-UX 原型，原型审核后才拆实施工单。默认学习语言已确认英语，原型语料为虚构样例；不导出本地 HTML。
|
||||
- 原型尽量减少说明文字，与目标页面一致：产品页只保留实际字段、操作与必要反馈；功能编号、模拟边界和审核说明放在独立导览或工单。
|
||||
- 当前 Quant-UX v1 已获用户验收（工单 #1 评论 7498），实施总览为 #16、单元工单为 #2～#15；LinguaCafe 对照与真实划词验证由 #4 承接，不能记作已完成。
|
||||
- 当前 Quant-UX v1 已获用户验收（工单 #1 评论 7498），实施总览为 #16、单元工单为 #2～#15；LinguaCafe 源码对照与桌面划词验证已由 #4 交付并获用户验收；真机详细测试证据缺口仍保留。
|
||||
- 学习端和管理端均使用账号（用户名）＋密码登录，账号不要求邮箱格式，邮箱不作为必填登录标识；后端独立校验管理权限与本人学习数据归属。
|
||||
- 已验证 MySQL 8.4.3，本机 127.0.0.1:3308；开发库 lexgo_dev、测试库 lexgo_test_issue2。密码只从环境或忽略的 .env.local 读取。迁移测试只能使用 lexgo_test_ 前缀专用库，不能借用其他数据库。
|
||||
- 后端命令使用 `python scripts/server.py migrate|bootstrap|serve|build|test|test-integration`；仅显式 migrate 修改表。bootstrap 只接受尚无账号的 LexGo 库，不覆盖已有管理员。Go 1.26.5、Node 22.22.1、pnpm 9.15.1；两端分别构建。
|
||||
- #18 登录日志与操作审计已通过用户验收：schema v2 显式迁移；日志只保存白名单字段，禁止保存凭据、请求/响应正文及私人学习内容。仅管理员查询，默认保留 90 天；启动/每小时及 `python scripts/server.py audit-cleanup` 仅清理两张审计表的过期记录。
|
||||
- #3 独立小样位于 `spikes/english/`，使用 `.local/nlp-venv/Scripts/python.exe`（3.12.12）运行；固定 spaCy 3.8.7、英语模型 3.8.0、NLTK 3.9.2、WordNet 3.0。资源仅显式准备时下载，摘要见 resources.json。不得把本机无账号的实验接口用于正式学习端；后续集成仍需 Go 授权、数据归属和任务设计。原文不归一化，位置区分 cp/UTF-8/UTF-16，lemma 不自动合并学习状态。
|
||||
- #4 独立阅读选择小样位于 `spikes/selection/`，`python spikes/selection/serve.py` 默认仅本机 5184。桌面鼠标/键盘与 11 项测试已验证，#4 已获用户验收并关闭；真实手机长按/手柄/滚动详细证据仍未提供；禁止把窄屏桌面当作真机验收。Intl.Segmenter 只用于 UI 范围验证，不替代 #3 NLP；释义保存只在内存。固定 LinguaCafe 源码对照和与 v1 的差异记录见架构 Wiki。
|
||||
- 2026-09-11 用户确认正式 NLP/词典采用全 Go。#5 已验收并合入 main；#6 使用 Go WordNet 解析和词形候选、Go Unicode 原文分片、schema v4 共享词典资源表，不调用 Python NLP。WordNet 3.0 ZIP 来源与摘要见 `server/wordnet-resource.json`，许可保留在 `server/WORDNET-LICENSE.txt`。词形候选不等于上下文消歧，不自动合并个人学习状态；#3 Python 小样只保留历史验证。当前词典仅英语释义，个人释义输入为临时草稿，持久化归 #7。
|
||||
- 2026-09-11 用户确认 #7 个人词条口径（三项由 Agent 定案）：身份为「学习者＋语言＋规范化词形」，大小写合并但**不按 lemma/候选合并**（`dog` 与 `dogs` 是两条记录）；首次保存默认「新词」；状态为 新词/学习中/已知/忽略，只有「学习中」带 1～7 级，对应原版 stage 2/1/0/-1～-7；例句只保存手输内容，不自动关联原文句子。schema v5 新增 `lexgo_terms`（唯一键加状态/等级检查约束），个人释义与共享词典分离且不进入审计日志；等级编辑 UI 归 #8/#12。
|
||||
- 2026-09-11 用户确认 #11 短语口径：短语与单词**共用 `lexgo_terms`**，身份键为按阅读顺序的规范化词形以单个空格连接（单词键不含空格），因此 `kind` 与词数由身份键派生，**不新增列、无迁移**；范围两端对齐整词、内部标点与换行保留但不参与身份比较，2～12 词、键 ≤128 字符、片段 ≤191 字符，切进单词的范围 400。跨章节匹配按连续词形比对，重叠取**最左最长**；短语高亮覆盖内部单词但**不修改单词数据**，点击已保存短语优先打开短语面板；失效引用回退＝高亮消失但词条与复习排期保留。短语进同一到期队列与同一套幂等作答，复习卡把整段短语挖成一个空；选择用原生拖选与手机系统手柄，不拦截 touchmove。同义形式合并、词性消歧、跨书移动、批量编辑（#12）与真机手感不在范围内。
|
||||
- 2026-09-11 用户确认 #10 编辑与删除口径：可改名、改章节标题、编辑章节正文；**只有正文变化才重新处理**，重复保存或改回原内容不新建任务，只改标题不改状态。版本键是 `content_sha256`：任务只在与章节版本一致时才能影响章节，过期版本任务标为 `error_reason=superseded` 且**完全不触碰章节**（认领、恢复扫描、重试都按版本裁决）；存储文本重算 SHA 与存储 SHA 不一致时按 `content_changed` 失败。删除为事务内硬删除 + 外键级联，删章后重排序号；个人词条、复习排期与作答记录**不随删除清理**。回收站/撤销、批量操作、章节跨书移动与语言变更不在范围内。编辑器行尾归一为 LF 是已知边界。
|
||||
- 2026-09-11 用户确认 #9 TXT 导入口径：只接受 UTF-8（允许可选 BOM，解码时剥离且不进入原文），非法字节整体拒绝、不使用替换字符；UTF-16 按 BOM 识别后明确拒绝，GB18030 等按非法 UTF-8 拒绝。文件字节上限 2 MiB，之后仍套用单章 100000 码点上限；换行与空白不归一化。文件只在内存中解码、不创建临时文件，客户端文件名不参与任何路径也不入库。解码后交给既有 `PasteBook`/`PasteChapter`，分章（一次提交一章）、`requestId` 幂等与任务恢复与粘贴一致；不改 schema。EPUB/PDF/字幕、UTF-16 转码、按空行自动分章与断点续传不在范围内。
|
||||
- 2026-09-11 用户确认 #8 到期单词复习决策表：固定间隔表 1/2/4/7/15/30/60 天，答对升级封顶 7、答错降级最低 1、再学一次不改等级，答错与再学立即回队；已知/忽略不入队，新保存的词立即到期，显式「学习中 level N」排 now+间隔[N]；只有新建或状态/等级实际变化才移动复习时间，编辑释义或例句保留原排期，保存未提及等级时保留已获得等级。到期判定用 UTC 绝对时刻（`due_at ≤ now`），不引入本地日边界。作答按 `answerId` 去重并以 `expectedDueAt` 判定过期标签页，重复提交、网络重发与双标签页都不得重复更新次数与间隔（作答响应 `result` 只取 applied/stale，重放另用 `duplicate` 标记并返回首次结果）；`correct_count` 只计答对，`wrong_count` 计答错与再学。短语复习归 #11，进度统计归 #13，不做策略配置 UI（X11）、练习模式（X08）与 FSRS。
|
||||
|
||||
@@ -2,11 +2,13 @@
|
||||
|
||||
面向自托管场景的阅读式语言学习项目，规划提供内容导入、阅读查词、词汇与短语、复习和实例管理。
|
||||
|
||||
已确认：**DevHarness 轻量模式、MySQL 8、go-admin 管理端**。工程基础 #2 已通过验收：两端用户名登录、学习账号管理、可撤销会话和本人英语空空间。管理端基于指定 go-admin/go-admin-ui 选用模块，学习端为独立 Vue 3 + TypeScript + Vite 工程，共用 Go 后端和 MySQL 8.4.3。#18 登录日志与操作审计已通过用户验收，支持管理员查询和 90 天保留清理。阅读、导入、词典与复习尚未实现。MVP 定位为“支持多账号、数据独立的自托管学习工具”，先邀请少量用户使用；F01～F12 已确认，X 系列后置。
|
||||
已确认：**DevHarness 轻量模式、MySQL 8、go-admin 管理端**。工程基础 #2 已通过验收：两端用户名登录、学习账号管理、可撤销会话和本人英语空空间。管理端基于指定 go-admin/go-admin-ui 选用模块，学习端为独立 Vue 3 + TypeScript + Vite 工程，共用 Go 后端和 MySQL 8.4.3。#18 登录日志与操作审计已通过用户验收，支持管理员查询和 90 天保留清理。#5 粘贴导入与章节原文阅读已验收并合入 main；#6 全 Go 英语词典与点词查义已通过用户验收，PR #25 已合入 main；#7 个人词条与学习状态已通过用户验收，PR #26 已合入 main；#8 到期单词复习已通过用户验收，PR #27 已合入 main；#9 上传 TXT、校验编码后导入本人书库已通过用户验收，PR #28 已合入 main；#10 编辑与删除本人书籍章节已通过用户验收，PR #29 已合入 main。短语、复习进度与词汇库仍未实现。MVP 定位为“支持多账号、数据独立的自托管学习工具”，先邀请少量用户使用；F01～F12 已确认，X 系列后置。
|
||||
|
||||
- [文档入口](docs/README.md) · [线上 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Home)
|
||||
- [英语分词与离线词典验证小样](spikes/english/README.md)（#3 已验收，独立本机入口）
|
||||
- [阅读选择验证小样](spikes/selection/README.md)（#4 已验收，真机详细测试证据缺口保留）
|
||||
- [项目档案](docs/00-project-profile.md) · [需求总览](docs/09-product-requirements-overview.md)
|
||||
- [工作量估算](docs/10-workload-estimate.md)：#2 验收后原范围剩余 46～75 人日，新增 #18 的 3～5 人日计划后为 49～80 人日；技术验证后重估，旧全量研究仅供参考。
|
||||
- [工作量估算](docs/10-workload-estimate.md)：#2、#3、#4、#5、#6、#7、#8、#9、#10、#18 已验收，原规划中的 #5 已完成；剩余 #11～#15 与新增 #21、#24 按工单复核；后续结合集成结果重估，旧全量研究仅供参考。
|
||||
- [四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16)：14 张单元工单，工程基础 → 技术验证 → 首条学习闭环 → 补齐 MVP；原型 v1 已获用户验收。两端使用账号（用户名）＋密码登录，不要求邮箱。
|
||||
- [原型工单 #1](https://git.ilapage.cn/OPC/lexgo/issues/1)：Quant-UX 桌面/手机原型 v1，预览入口与审核记录见工单及需求总览。
|
||||
- [工作流](docs/01-workflow.md) · [开发与验证](docs/04-local-development-and-verification.md)
|
||||
@@ -17,7 +19,15 @@
|
||||
|
||||
本地入口：学习端 http://127.0.0.1:5173，管理端 http://127.0.0.1:5174。完整安装与测试命令见[开发与验证](docs/04-local-development-and-verification.md)。账号使用用户名，无需邮箱；没有随代码交付的默认密码。
|
||||
|
||||
本机三个服务已由 `D:/supervisord/supervisord.conf` 中的 lexgo-learner、lexgo-admin、lexgo-api 托管，使用时不要重复手动启动同端口。当前 schema v2；从 #2 升级时停止 API，执行 build、migrate 后再启动。审计过期清理可执行 `python scripts/server.py audit-cleanup`，只影响超过 90 天的日志。
|
||||
本机三个服务已由 `D:/supervisord/supervisord.conf` 中的 lexgo-learner、lexgo-admin、lexgo-api 托管，使用时不要重复手动启动同端口。当前 schema v4；从 #2/#18 升级时停止 API，执行 build、migrate 后再启动。审计过期清理可执行 `python scripts/server.py audit-cleanup`，只影响超过 90 天的日志。
|
||||
|
||||
## 英语词典与点词查义（#6）
|
||||
|
||||
正式后端为纯 Go，`lexgo.exe serve` 不启动 Python NLP。开发辅助脚本与历史小样保留。
|
||||
|
||||
管理员在“英语词典”页下载并导入指定 WordNet 3.0 ZIP；来源、固定摘要及许可见 [资源清单](server/wordnet-resource.json)。词典提供英语释义，保存在 MySQL 中；导入失败保留当前资源，重复导入不新增资源并重新启用。也可在管理页停用。
|
||||
|
||||
学习者打开本人就绪章节，点击词语或聚焦后按 Enter/空格查询；Escape/关闭返回阅读。精确词形优先，未命中再查规则候选，例如 `went → go`。个人释义当前是未保存的临时草稿，保存功能在 #7 实现。真机测试证据尚未补齐。
|
||||
|
||||
## 文档与治理
|
||||
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
export const MAX_DICTIONARY_BYTES = 32 * 1024 * 1024
|
||||
|
||||
export function createDictionaryLoader(session, state) {
|
||||
let revision = 0
|
||||
function invalidate() {
|
||||
revision++
|
||||
Object.assign(state, { items: [], supported: null, loading: false, saving: false, error: '', notice: '' })
|
||||
}
|
||||
async function perform(operation, saving, apply) {
|
||||
const current = ++revision
|
||||
const generation = session.state.generation
|
||||
const stale = () => current !== revision || generation !== session.state.generation
|
||||
Object.assign(state, { loading: !saving, saving, error: '', notice: '' })
|
||||
try {
|
||||
const result = await operation()
|
||||
if (stale()) return false
|
||||
apply(result)
|
||||
return true
|
||||
} catch (error) {
|
||||
if (!stale()) state.error = error instanceof Error ? error.message : '词典操作失败，请重试'
|
||||
return false
|
||||
} finally {
|
||||
if (!stale()) { state.loading = false; state.saving = false }
|
||||
}
|
||||
}
|
||||
return {
|
||||
invalidate,
|
||||
load() {
|
||||
return perform(() => session.listDictionaries(), false, result => {
|
||||
state.items = result.items
|
||||
state.supported = result.supported
|
||||
})
|
||||
},
|
||||
import(form) {
|
||||
return perform(() => session.importDictionary(form), true, result => {
|
||||
state.items = [result.resource]
|
||||
state.notice = result.duplicate ? '该词典已存在，已启用，未重复导入。' : '词典导入成功。'
|
||||
})
|
||||
},
|
||||
toggle(id, enabled) {
|
||||
return perform(() => session.setDictionaryEnabled(id, enabled), true, result => {
|
||||
state.items = state.items.map(item => item.id === result.resource.id ? result.resource : item)
|
||||
state.notice = result.resource.enabled ? '词典已启用。' : '词典已停用。'
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
<div class="brand">{{ sidebar.opened ? 'LexGo 管理' : 'LG' }}</div>
|
||||
<el-menu :default-active="$route.path" :collapse="!sidebar.opened" router>
|
||||
<el-menu-item index="/accounts"><span>账号管理</span></el-menu-item>
|
||||
<el-menu-item index="/dictionaries"><span>英语词典</span></el-menu-item>
|
||||
<el-menu-item index="/login-logs"><span>登录日志</span></el-menu-item>
|
||||
<el-menu-item index="/operation-logs"><span>操作日志</span></el-menu-item>
|
||||
</el-menu>
|
||||
|
||||
@@ -4,11 +4,13 @@ import Layout from '../layout/index.vue'
|
||||
import Login from '../views/Login.vue'
|
||||
import Accounts from '../views/Accounts.vue'
|
||||
import AuditLogs from '../views/AuditLogs.vue'
|
||||
import Dictionaries from '../views/Dictionaries.vue'
|
||||
const router = createRouter({ history: createWebHashHistory(), routes: [
|
||||
{ path: '/login', component: Login },
|
||||
{ path: '/', component: Layout, children: [
|
||||
{ path: '', redirect: '/accounts' },
|
||||
{ path: 'accounts', component: Accounts, meta: { title: '账号管理' } },
|
||||
{ path: 'dictionaries', component: Dictionaries, meta: { title: '英语词典' } },
|
||||
{ path: 'login-logs', component: AuditLogs, props: { kind: 'login' }, meta: { title: '登录日志' } },
|
||||
{ path: 'operation-logs', component: AuditLogs, props: { kind: 'operation' }, meta: { title: '操作日志' } }
|
||||
] },
|
||||
|
||||
+17
-2
@@ -20,10 +20,11 @@ export function createSession({ fetch, storage, changed = () => {} }) {
|
||||
if (generation !== state.generation) throw new Error('会话已变化，请重新操作')
|
||||
}
|
||||
async function request(path, method = 'GET', body, token = state.token, generation = state.generation) {
|
||||
const multipart = typeof FormData !== 'undefined' && body instanceof FormData
|
||||
const result = await fetch('/api/v1' + path, {
|
||||
method,
|
||||
headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: 'Bearer ' + token } : {}) },
|
||||
...(body === undefined ? {} : { body: JSON.stringify(body) }),
|
||||
headers: { ...(!multipart ? { 'Content-Type': 'application/json' } : {}), ...(token ? { Authorization: 'Bearer ' + token } : {}) },
|
||||
...(body === undefined ? {} : { body: multipart ? body : JSON.stringify(body) }),
|
||||
cache: 'no-store'
|
||||
})
|
||||
const payload = await result.json()
|
||||
@@ -84,6 +85,20 @@ export function createSession({ fetch, storage, changed = () => {} }) {
|
||||
} catch (error) { if (generation === state.generation) clear(); throw error }
|
||||
},
|
||||
async logout() { const token = state.token; clear(); await revoke(token) },
|
||||
async listDictionaries() {
|
||||
authorized()
|
||||
return request('/dictionaries')
|
||||
},
|
||||
async importDictionary(form) {
|
||||
authorized()
|
||||
if (!(form instanceof FormData)) throw new Error('请选择词典文件')
|
||||
return request('/dictionaries/import', 'POST', form)
|
||||
},
|
||||
async setDictionaryEnabled(id, enabled) {
|
||||
authorized()
|
||||
if (!Number.isSafeInteger(id) || id <= 0 || typeof enabled !== 'boolean') throw new Error('词典参数无效')
|
||||
return request('/dictionaries/' + id, 'PATCH', { enabled })
|
||||
},
|
||||
async queryAuditLogs(kind, filters) {
|
||||
authorized()
|
||||
const generation = state.generation
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
<template>
|
||||
<basic-layout><template #wrapper>
|
||||
<el-card>
|
||||
<div class="toolbar"><div><h1>英语词典</h1><p class="subtle">供所有学习账号查询</p></div><el-button :disabled="busy" @click="reload">刷新</el-button></div>
|
||||
<el-alert v-if="resources.error" :title="resources.error" type="error" :closable="false" show-icon />
|
||||
<el-alert v-if="resources.notice" :title="resources.notice" type="success" :closable="false" show-icon />
|
||||
<el-table v-loading="resources.loading" :data="resources.items" border :empty-text="resources.error ? '加载失败，请重试' : '尚未导入词典'">
|
||||
<el-table-column label="名称" prop="name" min-width="175" />
|
||||
<el-table-column label="语言" width="80"><template #default>英语</template></el-table-column>
|
||||
<el-table-column label="版本" prop="version" width="80" />
|
||||
<el-table-column label="来源" prop="source" min-width="160" show-overflow-tooltip />
|
||||
<el-table-column label="格式" prop="format" min-width="160" />
|
||||
<el-table-column label="状态" width="95"><template #default="scope"><el-tag :type="scope.row.status === 'ready' ? 'success' : 'info'">{{ statusLabel(scope.row.status) }}</el-tag></template></el-table-column>
|
||||
<el-table-column label="词条数" prop="entryCount" min-width="95" />
|
||||
<el-table-column label="操作" width="100"><template #default="scope"><el-button :disabled="busy" link type="primary" @click="toggle(scope.row)">{{ scope.row.enabled ? '停用' : '启用' }}</el-button></template></el-table-column>
|
||||
</el-table>
|
||||
</el-card>
|
||||
<el-card class="import-card">
|
||||
<h2>导入词典</h2>
|
||||
<el-form class="dictionary-form" label-position="top" @submit.prevent="upload">
|
||||
<el-form-item label="名称"><el-input v-model="form.name" maxlength="120" :disabled="busy" /></el-form-item>
|
||||
<div class="form-row">
|
||||
<el-form-item label="语言"><el-input model-value="英语" disabled /></el-form-item>
|
||||
<el-form-item label="版本"><el-input v-model="form.version" readonly /></el-form-item>
|
||||
</div>
|
||||
<el-form-item label="来源"><el-input v-model="form.source" readonly /></el-form-item>
|
||||
<el-form-item label="格式"><el-input v-model="form.format" readonly /></el-form-item>
|
||||
<el-form-item label="词典文件">
|
||||
<input id="dictionary-file" ref="fileInput" type="file" accept=".zip,application/zip" aria-label="词典文件" :disabled="busy" @change="selectFile">
|
||||
</el-form-item>
|
||||
<p class="subtle">WordNet 3.0 ZIP · 上限 32 MiB · 英语释义</p>
|
||||
<p v-if="resources.supported?.source"><a :href="downloadURL" target="_blank" rel="noopener noreferrer">下载支持的词典文件</a></p>
|
||||
<p class="subtle">导入失败时保留当前词典。</p>
|
||||
<p v-if="fileError" role="alert" class="file-error">{{ fileError }}</p>
|
||||
<el-button type="primary" native-type="submit" :loading="resources.saving" :disabled="resources.loading || !file">导入并启用</el-button>
|
||||
</el-form>
|
||||
</el-card>
|
||||
</template></basic-layout>
|
||||
</template>
|
||||
<script>
|
||||
import BasicLayout from '../layout/BasicLayout.vue'
|
||||
import { session } from '../store'
|
||||
import { createDictionaryLoader, MAX_DICTIONARY_BYTES } from '../dictionaries.mjs'
|
||||
const downloadURL = 'https://raw.githubusercontent.com/nltk/nltk_data/96f9b3252457a2b97e52aec64c3dfceeb5c312d5/packages/corpora/wordnet.zip'
|
||||
export default {
|
||||
name: 'DictionariesView',
|
||||
components: { BasicLayout },
|
||||
data: () => ({
|
||||
resources: { items: [], supported: null, loading: false, saving: false, error: '', notice: '' },
|
||||
form: { name: 'Princeton WordNet', language: 'en', version: '3.0', source: downloadURL, format: 'wordnet-3.0-zip' },
|
||||
file: null, fileError: '', downloadURL
|
||||
}),
|
||||
computed: { busy() { return this.resources.loading || this.resources.saving } },
|
||||
watch: { '$store.state.generation': { flush: 'sync', handler() { this.loader.invalidate(); this.clearFile() } } },
|
||||
created() { this.loader = createDictionaryLoader(session, this.resources) },
|
||||
mounted() { this.reload() },
|
||||
beforeUnmount() { this.loader.invalidate(); this.file = null },
|
||||
methods: {
|
||||
reload() { return this.loader.load() },
|
||||
statusLabel(status) { return { ready: '可用', disabled: '已停用', unavailable: '不可用' }[status] || '不可用' },
|
||||
clearFile() { this.file = null; this.fileError = ''; if (this.$refs.fileInput) this.$refs.fileInput.value = '' },
|
||||
selectFile(event) {
|
||||
this.fileError = ''
|
||||
const selected = event.target.files?.[0]
|
||||
this.file = null
|
||||
if (!selected) return
|
||||
if (!selected.name.toLowerCase().endsWith('.zip') || selected.size === 0 || selected.size > MAX_DICTIONARY_BYTES) {
|
||||
this.fileError = '请选择不超过 32 MiB 的 ZIP 文件。'
|
||||
event.target.value = ''
|
||||
return
|
||||
}
|
||||
this.file = selected
|
||||
},
|
||||
async upload() {
|
||||
if (this.busy || !this.file) return
|
||||
this.fileError = ''
|
||||
if (!this.form.name.trim() || !this.form.source.trim()) { this.fileError = '请填写名称和来源。'; return }
|
||||
const data = new FormData()
|
||||
for (const [key, value] of Object.entries(this.form)) data.append(key, value.trim())
|
||||
data.append('file', this.file)
|
||||
if (await this.loader.import(data)) this.clearFile()
|
||||
},
|
||||
toggle(resource) { return this.loader.toggle(resource.id, !resource.enabled) }
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<style scoped>
|
||||
.toolbar { display:flex; align-items:center; justify-content:space-between; gap:16px; margin-bottom:20px; }
|
||||
h1 { font-size:20px; margin:0 0 8px; }
|
||||
h2 { font-size:18px; margin:0 0 24px; }
|
||||
.subtle { color:#606266; font-size:13px; line-height:1.6; }
|
||||
.toolbar p { margin:0; }
|
||||
.import-card { margin-top:20px; }
|
||||
.dictionary-form { max-width:600px; }
|
||||
.form-row { display:flex; gap:16px; }
|
||||
.form-row .el-form-item { flex:1; min-width:0; }
|
||||
.file-error { color:#b42318; }
|
||||
.el-alert { margin-bottom:16px; }
|
||||
input[type=file] { max-width:100%; }
|
||||
a { color:#176b63; }
|
||||
</style>
|
||||
@@ -0,0 +1,55 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import { createSession } from '../src/session.mjs'
|
||||
|
||||
const response = (data, status = 200) => ({ ok: status < 400, status, json: async () => ({ data, msg: '导入失败' }) })
|
||||
function setup() {
|
||||
const pending = []
|
||||
const session = createSession({ fetch: (url, options) => new Promise(resolve => pending.push({ url, options, resolve })), storage: { getItem() {}, setItem() {}, removeItem() {} } })
|
||||
session.state.user = { id: 1, username: 'fixture.admin', role: 'admin' }
|
||||
session.state.token = 'fictional-test-token'
|
||||
return { session, pending }
|
||||
}
|
||||
|
||||
test('dictionary upload sends multipart with authorization and lets browser set boundary', async () => {
|
||||
const { session, pending } = setup()
|
||||
const form = new FormData()
|
||||
form.append('name', 'Princeton WordNet')
|
||||
form.append('file', new Blob(['fictional-archive']), 'wordnet.zip')
|
||||
assert.equal(typeof session.importDictionary, 'function')
|
||||
const request = session.importDictionary(form)
|
||||
assert.equal(pending[0].url, '/api/v1/dictionaries/import')
|
||||
assert.equal(pending[0].options.method, 'POST')
|
||||
assert.equal(pending[0].options.body, form)
|
||||
assert.equal(pending[0].options.headers['Content-Type'], undefined)
|
||||
assert.equal(pending[0].options.headers.Authorization, 'Bearer fictional-test-token')
|
||||
pending[0].resolve(response({ resource: { id: 1 }, duplicate: false }))
|
||||
assert.deepEqual(await request, { resource: { id: 1 }, duplicate: false })
|
||||
})
|
||||
|
||||
test('dictionary read and toggle follow API contract, ordinary accounts cannot mutate', async () => {
|
||||
const { session, pending } = setup()
|
||||
assert.equal(typeof session.listDictionaries, 'function')
|
||||
const read = session.listDictionaries()
|
||||
assert.equal(pending[0].url, '/api/v1/dictionaries')
|
||||
pending[0].resolve(response({ items: [] }))
|
||||
await read
|
||||
const toggle = session.setDictionaryEnabled(1, false)
|
||||
assert.equal(pending[1].url, '/api/v1/dictionaries/1')
|
||||
assert.deepEqual(JSON.parse(pending[1].options.body), { enabled: false })
|
||||
pending[1].resolve(response({ resource: { id: 1, enabled: false } }))
|
||||
await toggle
|
||||
session.state.user.role = 'learner'
|
||||
await assert.rejects(session.importDictionary(new FormData()), /管理员/)
|
||||
await assert.rejects(session.setDictionaryEnabled(1, true), /管理员/)
|
||||
assert.equal(pending.length, 2)
|
||||
})
|
||||
|
||||
test('dictionary response cannot cross a logout or account switch', async () => {
|
||||
const { session, pending } = setup()
|
||||
assert.equal(typeof session.importDictionary, 'function')
|
||||
const request = session.importDictionary(new FormData())
|
||||
session.clear()
|
||||
pending[0].resolve(response({ resource: { id: 1 }, duplicate: false }))
|
||||
await assert.rejects(request, /会话已变化/)
|
||||
})
|
||||
@@ -0,0 +1,57 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
|
||||
async function fixture() {
|
||||
const { createDictionaryLoader } = await import('../src/dictionaries.mjs')
|
||||
const pending = []
|
||||
const later = () => new Promise((resolve, reject) => pending.push({ resolve, reject }))
|
||||
const session = { state: { generation: 1 }, listDictionaries: later, importDictionary: later, setDictionaryEnabled: later }
|
||||
const state = { items: [{ id: 1, name: 'Existing', enabled: true }], supported: null, loading: false, saving: false, error: '', notice: '' }
|
||||
return { loader: createDictionaryLoader(session, state), state, session, pending }
|
||||
}
|
||||
|
||||
test('failed upload preserves the existing resource and exposes the error', async () => {
|
||||
const { loader, state, pending } = await fixture()
|
||||
const action = loader.import(new FormData())
|
||||
assert.equal(state.saving, true)
|
||||
pending[0].reject(new Error('文件摘要不匹配'))
|
||||
assert.equal(await action, false)
|
||||
assert.equal(state.items[0].name, 'Existing')
|
||||
assert.equal(state.error, '文件摘要不匹配')
|
||||
assert.equal(state.saving, false)
|
||||
})
|
||||
|
||||
test('accepted mutation replaces old state directly, without a second refresh dependency', async () => {
|
||||
const { loader, state, pending } = await fixture()
|
||||
const action = loader.toggle(1, false)
|
||||
pending[0].resolve({ resource: { id: 1, name: 'Existing', enabled: false, status: 'disabled' } })
|
||||
assert.equal(await action, true)
|
||||
assert.equal(state.items[0].enabled, false)
|
||||
assert.equal(pending.length, 1)
|
||||
})
|
||||
|
||||
test('an old list cannot overwrite a newer import and duplicate import is explicit', async () => {
|
||||
const { loader, state, pending } = await fixture()
|
||||
const read = loader.load()
|
||||
const action = loader.import(new FormData())
|
||||
pending[1].resolve({ resource: { id: 1, name: 'Imported', enabled: true }, duplicate: true })
|
||||
await action
|
||||
pending[0].resolve({ items: [{ id: 1, name: 'Old' }] })
|
||||
await read
|
||||
assert.equal(state.items[0].name, 'Imported')
|
||||
assert.match(state.notice, /已存在/)
|
||||
})
|
||||
|
||||
test('page exit or session change discards late responses', async () => {
|
||||
const { loader, state, pending, session } = await fixture()
|
||||
const action = loader.import(new FormData())
|
||||
loader.invalidate()
|
||||
pending[0].resolve({ resource: { id: 1, name: 'Late' } })
|
||||
assert.equal(await action, false)
|
||||
assert.deepEqual(state.items, [])
|
||||
const read = loader.load()
|
||||
session.state.generation++
|
||||
pending[1].resolve({ items: [{ id: 1, name: 'Other session' }] })
|
||||
await read
|
||||
assert.deepEqual(state.items, [])
|
||||
})
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Project-Profile
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Project-Profile.-
|
||||
wiki_revision: 40ae5e3a0521195da4f1a9cd9f42beec15da20d4
|
||||
synchronized_at: 2026-09-10T12:43:19Z
|
||||
wiki_revision: 08d992aee15b7fd5cea0cdbb4eb0b90583068449
|
||||
synchronized_at: 2026-09-14T13:37:59Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# LexGo 项目档案
|
||||
@@ -37,7 +37,7 @@ LinguaCafe 参考基线由现有调研记录为 `c1ea298ce40c65b9dd33e9b26fd2e52
|
||||
|---|---|---|---|
|
||||
| D01 | 管理端底座 | **已确认使用 `D:/github_project/goadmin` 的 go-admin + go-admin-ui**，依据用户指令“管理端用D:\github_project\goadmin”；替代 gin-vue-admin 建议 | M0 验证指定本地版本配套、MySQL 8 及数据隔离，不再比较管理底座 |
|
||||
| D02 | 数据库 | **已确认 MySQL 8**；依据 2026-09-10 用户原话“使用mysql8”。覆盖两份分析的数据库分歧，具体小版本待基线验证锁定 | 按单一 MySQL 8 数据层估算，不做 PostgreSQL 双库兼容 |
|
||||
| D03 | NLP | 保留 Python 为建议，纯 Go 是否硬约束未确认 | 首发语言质量与开发量 |
|
||||
| D03 | NLP | **用户于2026-09-11确认正式产品全 Go**；Python小样仅历史验证 | 使用词典词形候选，不承诺上下文消歧 |
|
||||
| D04 | 默认学习语言 | 用户已确认英语；中文/日语后续需独立范围与验收 | 分词、读音和 UI |
|
||||
| D05 | 用户范围 | **已确认：支持多账号、数据独立的自托管学习工具，先邀请少量用户使用** | 首版按多用户归属和隔离设计；不自动加入公开注册、邀请链接或组织租户 |
|
||||
| D06 | 旧数据 | 全量迁移是否需要未确认，CSV 与完整迁移不同 | 迁移另估 |
|
||||
@@ -136,3 +136,29 @@ server 是 go-admin 的选用模块接入：原样保留 SysUser、SysDept、必
|
||||
## 日志审计基线（#18）
|
||||
|
||||
管理端新增两个审计列表，后端为 LexGo 自有日志模型与接口，参照 go-admin 模块布局但不复制其原始参数/响应持久化逻辑。schema v2 新增 lexgo_login_logs、lexgo_operation_logs；保留 90 天，启动和每小时分批清理，也可显式 audit-cleanup。新建/重置密码均为 6～72 UTF-8 字节；初始 bootstrap 保留 10 字节下限。#2 已验收，#18 已验收；默认模块中的其他候选未纳入。
|
||||
|
||||
|
||||
## 全 Go 正式架构决定（2026-09-11，#6）
|
||||
|
||||
用户已明确选择全 Go：正式英语分词、原文位置映射、本地词典解析和词形候选查询由 Go 后端完成，不运行 Python NLP 服务。此前“Python 建议/全 Go 未决”仅为历史决策记录，由本决定覆盖；spikes/english 保留历史验证，不接入产品。#6 按该方向实施，当前方案见工单最新启动评论；WordNet 3.0 仍为首个资源（英语释义），词形规则候选不等同于 spaCy 上下文消歧，原文及个人学习状态不按候选合并。
|
||||
|
||||
|
||||
## #6 当前工程状态（2026-09-11）
|
||||
|
||||
#5已验收且相关前置PR均已合入main。#6正式词典/点词查义已按全Go实现，已于2026-09-11通过用户验收，PR #25已合入main；server schema v4，管理端新增英语词典页，学习端加入Go分片及查词面板。正式Go进程不依赖Python NLP，WordNet包随数据库持久化。#3小样仍是历史验证；保存个人释义和状态归#7，音频封面#21、列表优化#24尚未实施。
|
||||
|
||||
## #7 当前工程状态（2026-09-11）
|
||||
|
||||
#6已通过用户验收并合入main。#7个人词条、学习状态与阅读器高亮已按用户确认口径实现，并于2026-09-11通过用户验收，PR #26已fast-forward-only合入main；server schema v5新增lexgo_terms，学习端面板可保存释义、例句与状态并按状态高亮，同一词形在本人其他章节显示一致。个人释义与共享词典分离，且不进入审计日志。词汇库#12、短语#11、到期复习#8、进度#13、音频封面#21与列表优化#24尚未实施。
|
||||
|
||||
## #8 当前工程状态（2026-09-11）
|
||||
|
||||
#7已通过用户验收并合入main。#8到期单词复习已按用户确认的决策表实现，并于2026-09-11通过用户验收，PR #27已fast-forward-only合入main；server schema v6新增lexgo_term_reviews（排期与计数）与lexgo_review_answers（作答记录），固定间隔1/2/4/7/15/30/60天，答对升级封顶7、答错降级最低1、再学一次不改等级，答错与再学立即回队，已知与忽略不入队。作答按answerId去重并以expectedDueAt判定过期标签页，重复提交、网络重发与双标签页都只记一次；到期判定用UTC绝对时刻，不引入本地日边界。独立审核（Claude Code）指出的并发同键500、编辑文本重排复习、面板保存重置等级三项已整改并复测。第3阶段「首条学习闭环」（#5～#8）至此全部验收；剩余#9～#15与#21、#24尚未实施。
|
||||
|
||||
## #9 当前工程状态（2026-09-13）
|
||||
|
||||
#8已通过用户验收并合入main。#9 TXT文件导入已按用户确认的契约实现，并于2026-09-13通过用户验收，PR #28已fast-forward-only合入main；本单没有数据库结构变化，schema仍为v6。解码只接受UTF-8（可选BOM剥离且不进原文），非法字节整体拒绝、不使用替换字符，UTF-16按BOM识别后明确拒绝；文件字节上限2MiB并仍受单章100000码点约束；换行与空白不归一化。文件只在内存中解码，不创建临时文件，客户端文件名不参与任何路径也不入库。解码后交给既有PasteBook/PasteChapter，分章、任务幂等与恢复与粘贴一致。第4阶段已完成#9；剩余#10～#15与#21、#24尚未实施。
|
||||
|
||||
## #10 当前工程状态（2026-09-14）
|
||||
|
||||
#9已通过用户验收并合入main。#10编辑与删除书籍章节已按用户确认的契约实现，并于2026-09-14通过用户验收，PR #29已fast-forward-only合入main；本单没有数据库结构变化，schema仍为v6。可改书名、章节标题与章节正文；只有正文变化才重新处理，重复保存或改回原内容不新建任务，只改标题不改变处理状态。版本门控按content_sha256执行：过期版本任务被标为superseded且完全不触碰章节，认领、恢复扫描与重试都按版本裁决，存储文本重算SHA与存储SHA不一致时按content_changed失败——这修掉了「处理中编辑导致新版本被标失败」的既有缺陷。删除为事务内硬删除加外键级联，删章后重排序号；个人词条、复习排期与作答记录不随删除清理。第4阶段已完成#9与#10；剩余#11～#15与#21、#24尚未实施。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Architecture-and-Code-Map
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Architecture-and-Code-Map.-
|
||||
wiki_revision: 3b6f6db7f0fe1019b5811f70af8d6174cec0b950
|
||||
synchronized_at: 2026-09-10T12:36:57Z
|
||||
wiki_revision: cc1522fceb8e420cee17f509101f61986b3d7ff9
|
||||
synchronized_at: 2026-09-14T14:21:32Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 架构与代码地图
|
||||
@@ -23,7 +23,7 @@ Go 承担业务与后台任务，浏览器提供阅读学习界面，NLP 保留
|
||||
| `dev_scripts/harness.py`、`dev_scripts/wiki_docs.py` | 原样复制的 DevHarness 工具 | 治理工具入口，无业务 API |
|
||||
| `tests/` | 上游治理工具与文档结构测试 | 不验证阅读、NLP 或 SRS |
|
||||
|
||||
不存在产品入口、数据库迁移或前端页面。拟定职责：identity（身份）、library（书库）、ingestion（导入）、lexicon（全局词典）、vocabulary（个人词语）、review（复习）、progress（统计）、administration（管理）。底座固定后再决定具体目录。
|
||||
工程入口与已实现模块见下方 #2/#18；学习领域拟定职责：identity（身份）、library（书库）、ingestion（导入）、lexicon（全局词典）、vocabulary（个人词语）、review（复习）、progress（统计）、administration（管理）。底座固定后再决定具体目录。
|
||||
|
||||
## 两条主要执行路径
|
||||
|
||||
@@ -47,7 +47,7 @@ Go 承担业务与后台任务，浏览器提供阅读学习界面，NLP 保留
|
||||
|
||||
## 学习端与管理端的目标架构
|
||||
|
||||
技术方向已纳入本轮方案，以下是目标结构，尚无对应产品目录或可运行应用。
|
||||
以下是目标结构；账号、管理端与学习空空间已由 #2 实现。英语 NLP 已完成 #3 独立验证，尚未接入业务 API。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
@@ -56,7 +56,7 @@ flowchart TD
|
||||
API --> B[独立学习业务模块]
|
||||
B --> DB[(MySQL 8)]
|
||||
B --> W[后台任务：具体选型待验证]
|
||||
W --> N[NLP 服务：Python 方案待确认]
|
||||
W --> N[NLP：Python 小样已验证，生产集成待实施]
|
||||
```
|
||||
|
||||
| 交付部分 | 建设方式 | 复用与自建边界 |
|
||||
@@ -131,3 +131,200 @@ server/app/lexgo/audit.go 定义两类白名单字段日志、筛选分页、失
|
||||
database.go 显式迁移至 v2，两张新增表均以 created_at/id 建立排序清理索引，账号字段建查询索引，无业务表级联删除。cmd/lexgo/main.go 的服务进程在启动和每小时执行审计清理，每次最多运行一分钟、每批删除 1000 条，仅影响过期审计记录。
|
||||
|
||||
admin/src/views/AuditLogs.vue 通过 kind 复用登录/操作列表；audit-logs.mjs 负责筛选编码和请求序号，session.mjs 继续进行管理员及会话 generation 校验。切换页面/账号清空日志，普通翻页保留总数，防止分页组件跳回第一页。菜单与标题按当前路由显示。
|
||||
|
||||
|
||||
## 英语分词与本地词典验证（#3，已验收）
|
||||
|
||||
`spikes/english/` 是独立可运行验证小样，不是学习端生产功能。推荐后续采用 Python 3.12.12、spaCy 3.8.7、en_core_web_sm 3.8.0（保留 tok2vec/tagger/attribute_ruler/lemmatizer，停用 parser/ner）和 NLTK 3.9.2 读取 WordNet 3.0。Go 继续管理用户、权限、任务和持久数据，后续通过显式契约调用 NLP；本单未新增 Go API、MySQL 表或常驻部署实例。
|
||||
|
||||
| 文件 | 作用 |
|
||||
|---|---|
|
||||
| engine.py | 原文分词、lemma、三个位置单位、直接/lemma 查词；只读本地资源 |
|
||||
| app.py、index.html、app.mjs、view.mjs、style.css | loopback 临时 HTTP 小样、输入/阅读/查词结果；单进程串行，输入不落盘 |
|
||||
| resources.json、setup_resources.py、requirements.lock | 固定版本、来源和 SHA256；显式联网准备，运行期无自动下载 |
|
||||
| test_engine.py、test_app.py、view.test.mjs | 真实模型离线验证、HTTP 边界与浏览器偏移/迟到响应测试 |
|
||||
| benchmark.py、benchmark-result.json | 虚构语料的候选对照、长文/查询实测及环境样本 |
|
||||
|
||||
WordNet 使用 ZIP 内原始 index/data/exception 文件，不使用 SysDict 或新增业务库。NLTK 默认 synsets 会隐式词形还原，本小样直接读取其固定版本索引以区分 exact 和显式 lemma；禁用依赖全局 corpus 的 OMW 跨版本映射，只接受 WordNet 3.0。升级 NLTK 或词典时必须重跑契约测试。
|
||||
|
||||
候选比较：正则分词＋WordNet 默认名词 morphology 依赖少、速度快，但不具备上下文判断，缩写和词性歧义处理弱；纯 Go 规则同样需要自行维护这些语言规则。本次 spaCy 在 12 个明确样例中答对 11 个，基线 6 个，因此推荐保留独立 Python NLP 边界。样例量不足以证明总体准确率；不宣称部署或正式阅读功能已完成。
|
||||
|
||||
|
||||
## #4 阅读选择小样与 LinguaCafe 对照
|
||||
|
||||
小样位于 spikes/selection：serve.py 只提供白名单静态文件，app.mjs 负责 DOM 原生选择/键盘/面板状态，range.mjs 负责原文范围和匹配，fixtures.mjs 提供两章虚构文本。保存仅在内存 Map，刷新清空，不接入账号、MySQL、正式词典、复习或 #21 附件。使用 Intl.Segmenter 的词与字形边界验证 UI，不能替代 #3 的 spaCy 结果；正式阅读器必须以章节原文、内容版本与 NLP tokens 为共同基准。
|
||||
|
||||
参考版本固定 LinguaCafe c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8。本次实际读取以下源码；未启动 LinguaCafe、未亲测其浏览器或手机行为。下表“参考行为”是源码证据；LexGo 方案是独立实现与取舍，不应标为上游已经验证的体验。
|
||||
|
||||
| 项目 | 参考行为与来源 | LexGo 采用及保留差异 |
|
||||
|---|---|---|
|
||||
| 阅读器 | TextBlockGroup.vue L56–100 用词/短语 stage、selected、hover 等状态；InteractiveTextStyling.scss L20–126 由主题决定颜色。TextReaderChapterList.vue L15–43 展示章节统计及已处理章节阅读入口；TextReader.vue L195–214 提供阅读完成后的书库/下一章 | 保留正文上下文、词语状态与选中区分、章节切换；小样以两章和学习中/已认识/忽略验证，章节统计及已读持久化仍属后续功能。侧栏/底部面板为 LexGo 布局，不宣称复制上游排版 |
|
||||
| 查词面板 | VocabularyBox.vue L124–169、374–404 区分 Translation 与字典搜索，新短语需 Save phrase；TextBlockGroup.vue L1351–1380 失选时自动保存单词/已有短语。VocabularySearchBox.vue L149–179 及保存 catch 未证明完整错误反馈 | 沿用 v1 的词典释义/我的释义、明确保存、关闭继续阅读。LexGo 关闭不自动保存；保存后可见反馈，无词典/无结果仍可手填。小样用固定虚构释义与故障状态，不把它当成真实词典/网络重试验证 |
|
||||
| 短语与键盘 | TextBlockGroup.vue L399–599 自定义鼠标范围；L374–457 为手机 500ms 长按及后续触摸移动，选区开始后阻止默认滚动；L1071–1202 的 Shift+方向键跳高亮词，而非扩展范围，Esc 失选 | LexGo 采用原生鼠标拖选、手机长按/系统手柄，不拦截 touchmove；起止端点按钮可调整。←/→ 相邻词，Shift+←/→ 扩缩范围，Esc 取消。原型的预设短语按钮被真实正文选区替代，属于 #4 明确要求的验证；手机手柄是否与底部面板冲突仍待真机 |
|
||||
| 复习 | Review.vue L270–345、559–692 为 Reveal→I was correct/Again，正确移除卡、Again 保留并随机抽剩余卡，最后一张正确完成；ReviewHotkeyInformationDialog.vue L14–24 提供快捷键。练习模式不写状态 | 已验收 v1 保留中文显答、答对/答错、再学与完成，单词/短语分别有状态。上游随机下一卡、阶段降级和快捷键不是本单已实现行为；#8 再明确调度、重学和幂等，#4 不新增复习引擎 |
|
||||
|
||||
源码链接：
|
||||
- [resources/js/components/Text/TextBlockGroup.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Text/TextBlockGroup.vue)
|
||||
- [resources/sass/Text/InteractiveTextStyling.scss](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/sass/Text/InteractiveTextStyling.scss)
|
||||
- [resources/js/components/TextReader/TextReaderChapterList.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/TextReader/TextReaderChapterList.vue)
|
||||
- [resources/js/components/TextReader/TextReader.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/TextReader/TextReader.vue)
|
||||
- [resources/js/components/Text/VocabularyBox.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Text/VocabularyBox.vue)
|
||||
- [resources/js/components/Text/VocabularySearchBox.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Text/VocabularySearchBox.vue)
|
||||
- [resources/js/components/TextReader/TextReaderHotkeyInformationDialog.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/TextReader/TextReaderHotkeyInformationDialog.vue)
|
||||
- [resources/js/components/Review/Review.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Review/Review.vue)
|
||||
- [resources/js/components/Review/ReviewHotkeyInformationDialog.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Review/ReviewHotkeyInformationDialog.vue)
|
||||
|
||||
上游 LICENSE 为 GPL v3，本单只核对并描述行为，没有移植源码。#1 评论 7497 的四项对照登记由本节补充；原型 v1 的保存/关闭主要流程保持，不更改其历史验收记录，不声称已经在 Quant-UX 新建修订版。范围调整与原生手柄作为可运行小样验证，若真机结果导致主要流程变化，应先更新关键原型状态并由用户确认。
|
||||
|
||||
## 粘贴导入、章节与阅读（#5，schema v3）
|
||||
|
||||
#5 实现了目标路径一的第一段可运行链路：粘贴英语文本 → 持久导入任务 → 固定分章 → 处理中/就绪 → 本人阅读原文。Go 单进程同时承担 API 与后台任务。本单不接入 Python NLP，token、lemma 与词典索引仍待 #6，路线未决边界见业务规则页。
|
||||
|
||||
| 路径 | 职责 |
|
||||
|---|---|
|
||||
| server/app/lexgo/database.go | schema v3 显式迁移：lexgo_books、lexgo_chapters、lexgo_ingest_jobs；按版本累加语句，版本行只在全部语句成功后推进 |
|
||||
| server/app/lexgo/library.go | 粘贴校验与固定分章、书籍/章节/任务写入、本人归属查询、重试与请求幂等 |
|
||||
| server/app/lexgo/ingest.go | 任务声明 claim、处理完成、固定失败原因、启动恢复 |
|
||||
| server/app/lexgo/router.go | 新增书籍/章节/任务路由；粘贴请求使用独立的 4 MiB 体积上限 |
|
||||
| server/cmd/lexgo/main.go | serve 启动时恢复遗留任务，并按秒轮询处理待处理任务 |
|
||||
| learner/src/stores/library.ts、views/ImportView.vue、BookView.vue、ReaderView.vue | 粘贴导入、书库与章节状态、失败重试、原文阅读 |
|
||||
|
||||
任务状态为 pending → processing → ready/failed，章节与任务共用同一套状态词。声明与完成分属两个事务：声明一经提交，即使进程随即退出，也只会留下可被启动恢复重新入队的 processing 记录。
|
||||
|
||||
### 粘贴导入 API v1（#5）
|
||||
|
||||
| 方法与路径 | 行为和权限 |
|
||||
|---|---|
|
||||
| POST /api/v1/books | {requestId,title,text,language?}；创建书籍＋首个章节＋导入任务；重复 requestId 返回首次结果（HTTP 200，duplicate=true） |
|
||||
| POST /api/v1/books/:id/chapters | 向本人书籍追加一个章节 |
|
||||
| GET /api/v1/books | 本人书库与章节状态计数；拒绝查询参数，避免用参数替换认证身份 |
|
||||
| GET /api/v1/books/:id | 本人书籍与章节列表，含 jobId 与可读失败原因 |
|
||||
| GET /api/v1/chapters/:id | 本人章节详情；仅 ready 时返回 originalText，并附带前后章节编号 |
|
||||
| GET /api/v1/jobs/:id | 本人任务状态、尝试次数与失败原因 |
|
||||
| POST /api/v1/jobs/:id/retry | 仅失败任务可重试；复用同一章节，不新建章节 |
|
||||
|
||||
所有接口按认证身份过滤 owner_id；他人书籍、章节或任务编号统一返回 404，管理员角色也不能解除学习数据的本人归属过滤。后台任务只使用任务行内的 owner_id，不接受客户端用户编号；请求体含未知字段（例如 ownerId）直接返回 400。
|
||||
|
||||
### schema v3
|
||||
|
||||
lexgo_books(owner_id, title, language)、lexgo_chapters(book_id, owner_id, ordinal, title, original_text MEDIUMTEXT, char_count, content_sha256, status, error_reason) 与 lexgo_ingest_jobs(owner_id, book_id, chapter_id, request_key, content_sha256, status, attempts, error_reason, finished_at)。owner_id 在章节与任务上冗余存放，使任何查询都能直接按认证身份过滤而不依赖连接；UNIQUE(book_id, ordinal) 与 UNIQUE(owner_id, request_key) 分别阻止重复章节与重复提交。启动检查要求版本 3，服务不自动迁移。
|
||||
|
||||
并发重复提交：请求命中 request_key 唯一键冲突后，用加锁读读取已提交结果，因为该请求事务的快照早于并发提交；因此两个并发相同提交只会产生一个章节，另一个得到 duplicate=true 的首次结果。
|
||||
|
||||
## #5 审核整改（R1～R4，2026-09-11）
|
||||
|
||||
提交见工单 #5 的整改评论；本条记录实现与验证方式。
|
||||
|
||||
- 追加契约（R1）：学习端把新建与追加拆成两个请求类型，追加不发送 language；后端保持严格解码，并新增回归测试断言“追加带 language 返回 400、不带则 201”，学习端单测断言追加请求体只有 requestId/title/text。
|
||||
- 运行期任务恢复（R2）：`server/app/lexgo/ingest.go` 的恢复逻辑合并为一处——启动恢复使用阈值 0，运行期每轮清扫使用 15 秒阈值并把超过 5 次尝试的任务置为 failed（原因码 attempts_exhausted）；`cmd/lexgo/main.go` 的 worker 每秒先清扫再处理，日志分别说明“已重新入队”与“本批未完成、等待下一次清扫”，不再声称已完成实际跳过的重试。
|
||||
- 离页作废在途请求（R3）：`closeBook`/`closeChapter` 推进请求序号并清理 loading；`ImportView` 记录是否已卸载，卸载后的成功响应不再触发跳转。
|
||||
- 重试自愈（R4）：`retryChapter` 先把重试返回的章节状态应用到列表与阅读器并重新安排轮询，再做静默刷新。
|
||||
|
||||
验证：Go 全量用例 20 项通过（新增运行期恢复与尝试上限两项）；学习端单测 38 项通过，其中 7 项在整改前的代码上复现失败；真实联调确认追加路径可用、被中断的任务在运行中被自动恢复（约 0.5 秒，无需重启）、重试在首次刷新失败后仍自动显示最终结果。
|
||||
|
||||
|
||||
## 全 Go 正式架构决定（2026-09-11，#6）
|
||||
|
||||
用户已明确选择全 Go：正式英语分词、原文位置映射、本地词典解析和词形候选查询由 Go 后端完成，不运行 Python NLP 服务。此前“Python 建议/全 Go 未决”仅为历史决策记录，由本决定覆盖；spikes/english 保留历史验证，不接入产品。#6 按该方向实施，当前方案见工单最新启动评论；WordNet 3.0 仍为首个资源（英语释义），词形规则候选不等同于 spaCy 上下文消歧，原文及个人学习状态不按候选合并。
|
||||
|
||||
|
||||
## #6 全 Go 词典与阅读器（2026-09-11，已验收并合入 main）
|
||||
|
||||
`server/app/lexgo/wordnet.go` 负责固定 WordNet ZIP 校验/内存解析、Unicode 分词及词形候选；`dictionary.go` 负责资源与章节查词 API；`database.go` schema v4 新增单槽共享资源表 lexgo_dictionaries（元数据、enabled、SHA、ZIP LONGBLOB），已有学习数据不改写。每个 Router 按 SHA 缓存一个不可变词典，查询先读资源元数据，缓存未命中才读取 ZIP；进程重启从数据库恢复，不需要 Python NLP 或额外资源目录。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| GET /api/v1/dictionaries | 管理员；items + supported，状态 ready/disabled/unavailable，不返回 archive 或本机路径 |
|
||||
| POST /api/v1/dictionaries/import | 管理员；multipart name/language/version/source/format/file；返回 resource + duplicate；格式/来源/版本固定 |
|
||||
| PATCH /api/v1/dictionaries/:id | 管理员；{enabled:boolean}；返回 resource |
|
||||
| GET /api/v1/chapters/:id/tokens | 本人 ready 章节；{textSha256,tokens:[{text,start,end,startUtf16,endUtf16,kind}]} |
|
||||
| POST /api/v1/lookup | {chapterId,start,end}，cp半开范围；本人ready完整单词；返回 status/query/matchedForm/candidates/entries/resource |
|
||||
|
||||
管理端 `Dictionaries.vue` + `dictionaries.mjs` + session multipart 方法，复用 go-admin 导航/表单与身份失效保护。学习端 `useReaderLookup.ts` 校验原文片段/SHA/所有位置，`ReaderTokens.vue` 渲染可聚焦单词，`LookupPanel.vue` 展示释义及临时个人草稿。桌面侧栏，手机固定底部45dvh面板；关闭恢复焦点，仅无后续手动滚动时恢复自动调整前位置。旧响应在换词/换章/退出/离页后失效。
|
||||
|
||||
参考：[WordNet 数据格式](https://wordnet.princeton.edu/documentation/wndb5wn)、[词形规则](https://wordnet.princeton.edu/documentation/morphy7wn)。#3 仅历史实验，#6 不调用其实验服务。
|
||||
|
||||
## #7 个人词条与阅读器状态（2026-09-11，已验收并合入 main）
|
||||
|
||||
schema v5 新增 lexgo_terms：一个学习者对一个词形一条记录。身份键为 `(owner_id, language, term)`，`term` 是 Go 侧 `normalizeWord` 的结果（NFC、小写、弯撇号转直撇号），列使用 `utf8mb4_bin`，避免折叠 `resume`/`résumé`；`original_form` 保存最近一次保存的原词形供显示。`definition`/`examples` 是学习者自己的文本，例句按行存储；共享词典仍只在 `lexgo_dictionaries`，两者不混存。`status` 与 `level` 由数据库检查约束守住：只有 `learning` 允许 1～7，其他状态必须为 0。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| POST /api/v1/terms | 本人；`{chapterId,start,end,definition,examples[],status,level?}`；服务端按 #6 同一套 token 范围反推词形，`owner`/`language`/`term` 一律不接受客户端输入；唯一键 upsert，重复保存更新同一行；首次 201、更新 200，返回 `{term,created}` |
|
||||
| GET /api/v1/terms/:id | 本人；他人编号与不存在编号统一 404，不泄露存在性 |
|
||||
| GET /api/v1/chapters/:id/tokens | 在原响应上为 word 片段增加可选 `term:{id,status,level}`；服务端按本章词形分批（每批 500）查本人词条，其他章节保存的同形词同样命中 |
|
||||
|
||||
`server/app/lexgo/terms.go` 负责身份、状态/等级边界、文本上限、upsert 与章节点词状态；`database.go` 提供 v5；`dictionary.go` 的 tokens 与 lookup 共用 `wordAtRange`，保存与查询必须落在同一个完整单词范围上，所以客户端无法命名自己没有读到的词。个人词条不写审计日志。
|
||||
|
||||
学习端 `useReaderLookup.ts` 在原有查询状态上增加个人释义、例句、状态、已保存编号、预填与保存；打开已保存词先读 `GET /terms/:id`，读取失败时禁用保存，避免用空表单覆盖原内容。`LookupPanel.vue` 提供状态单选、释义与例句输入、保存与简短反馈；`ReaderTokens.vue` 按状态高亮 `is-new`/`is-learning`/`is-known`/`is-ignored`。换词、换章、离页、退出或切换账号都会清空表单、状态与高亮。管理端无改动。
|
||||
|
||||
## #8 复习调度与答题（2026-09-11，已验收并合入 main）
|
||||
|
||||
schema v6 新增 `lexgo_term_reviews`（每个个人词条一行排期：`due_at`、`review_count`、`correct_count`、`wrong_count`、`last_reviewed_at`）与 `lexgo_review_answers`（每次作答一条：`answer_key`、评分、状态/等级/间隔前后值、`result`、`requeued`、时间）。两条语句都是 `CREATE TABLE IF NOT EXISTS` 加 `INSERT IGNORE ... SELECT`，所以迁移是可重试的加法迁移；旧二进制回到 v5 仍可继续写 `lexgo_terms`，不需要改动个人词条表本身。既有已保存词汇在迁移中按 `due_at = created_at` 进入队列。
|
||||
|
||||
`server/app/lexgo/review.go` 负责间隔表、队列、评分转换、幂等与并发；`terms.go` 的保存路径通过 `syncTermReview` 维护排期行（`新词` 立即到期，显式 `学习中 level N` 排 `now + 间隔[N]`），计数在状态或等级变化时保留。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| GET /api/v1/reviews/queue | 本人＋当前语言；仅 `新词`/`学习中` 且 `due_at ≤ now`，按 `due_at, id` 排序、最多 50 条；返回 `{items, total}`，`total` 是全部到期数；不接受查询参数 |
|
||||
| POST /api/v1/reviews/:termId/answers | `{answerId, grade: correct\|wrong\|again, expectedDueAt}`；应用成功 201，重放或过期 200；返回首次结果 `result`(applied/stale) 与 `duplicate` 标记、前后状态/等级/到期时间、`requeued` 与词条新状态；加锁后再次读取答案键，所以并发的同键提交也返回记录而不是报错 |
|
||||
|
||||
学习端新增 `/review` 路由与书库、阅读器顶栏的「到期复习」入口；`stores/review.ts` 维护队列、本轮计数、评分与重学，`ReviewCard.vue`/`ReviewView.vue` 呈现正面（词＋挖空例句）、答案面（个人释义＋例句＋三个评分按钮）、完成页与空队列页。一次评分对应一个 `answerId`，失败重试复用同一个；换词后重新生成。切换账号或退出登录会清空队列、计数与当前卡片。
|
||||
|
||||
参考：[LinguaCafe Review.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Review/Review.vue)。上游的随机抽卡、阶段降级与快捷键不属于本单；#8 只实现本项目的固定间隔、固定顺序与幂等作答。
|
||||
|
||||
## #9 TXT 上传导入（2026-09-11，已验收并合入 main）
|
||||
|
||||
`server/app/lexgo/upload.go` 负责把上传的 TXT 解码后交给与粘贴相同的核心：解码、multipart 解析与两条路由，schema 无变化（沿用 #5 的 `lexgo_books`/`lexgo_chapters`/`lexgo_ingest_jobs`）。文件只在内存中存在，不写临时文件，客户端文件名不参与任何路径也不入库。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| POST /api/v1/books/upload | 本人；multipart：`requestId`、`title`、`language`（可省略，省略即英语）、`file`；新建书籍与首章 |
|
||||
| POST /api/v1/books/:id/chapters/upload | 本人且本人书籍；multipart：`requestId`、`title`、`file`；追加一章；不接受 `language` |
|
||||
|
||||
字段白名单之外的字段、重复字段、缺失 `file`、非 multipart 请求都返回 400；201 新建、200 重复、409 同编号换内容、404 他人书籍、401 未登录、429 已有文件正在上传（单槽并发门）。响应体与粘贴路径同为 `PasteResult`，所以学习端复用同一套跳转与轮询逻辑。
|
||||
|
||||
解码规则见业务规则页；实现上 `decodeTextUpload` 先按 UTF-16 BOM 识别并给出针对性提示，再剥离可选 UTF-8 BOM，然后用 `utf8.Valid` 整体校验，最后交给 `validatePaste`（非空、≤100000 码点）。因此上传与粘贴共享同一分章与任务规则：一次提交一章，`requestId` + 内容 SHA 幂等，worker 只发布已落库的原文。
|
||||
|
||||
学习端 `ImportView.vue` 增加「粘贴文本 / TXT 文件」来源切换（沿用已验收 v1 的切换与状态行），`stores/library.ts` 增加 `upload()` 与 `fileProblem`/`fileSizeLabel`，`session.request` 支持 `FormData`（multipart 请求不再被 JSON 化，边界由浏览器提供）。客户端预检只提前反馈，服务端结论为最终结论。
|
||||
|
||||
## #10 编辑与删除书籍章节（2026-09-11，已验收并合入 main）
|
||||
|
||||
`server/app/lexgo/edit.go` 提供改名、编辑与删除；`ingest.go` 增加版本门控。**schema 无变化**：`chapters.content_sha256` 与 `jobs.content_sha256` 就是版本键，新增的 `superseded` 复用现有 `error_reason` 列。
|
||||
|
||||
| 接口 | 权限与输入/输出 |
|
||||
|---|---|
|
||||
| PATCH /api/v1/books/:id | 本人；`{title}`；返回 `{book}` |
|
||||
| GET /api/v1/chapters/:id/source | 本人任意状态；返回 `{source:{id,bookId,ordinal,title,text,status,contentSha256,charCount}}`，供编辑使用；不接受查询参数 |
|
||||
| PATCH /api/v1/chapters/:id | 本人；`{title?,text?}`；返回 `{chapter,job,versionChanged}`；正文变化才新建任务 |
|
||||
| DELETE /api/v1/books/:id | 本人；返回 `{deleted:{bookId,chapters}}`；FC 级联删除章节与任务 |
|
||||
| DELETE /api/v1/chapters/:id | 本人；返回 `{deleted:{chapterId,bookId,remaining}}`；删除后重排序号 |
|
||||
|
||||
**版本门控**（本单修掉的缺陷）：任务只在 `job.content_sha256 == chapter.content_sha256` 时才能影响章节。认领任务时用 JOIN 只取版本匹配的行，并先把过期版本任务一次性标为 `failed/superseded`；发布前再比对一次，不匹配就只把任务标为 `superseded` 并**完全不触碰章节**；恢复扫描同样先作废过期版本任务、只重排版本匹配的中断任务；重试接口拒绝版本不匹配的任务（409）。删除期间在途任务找不到章节时视为无事可做（级联已删除其任务行）。
|
||||
|
||||
`unprocessableReason` 保留一条内容一致性检查：存储的正文重新计算出的 SHA 必须等于该章节存储的 SHA，用于兜住绕过 API 的直接写入（`content_changed`），与版本门控互不重复。
|
||||
|
||||
学习端 `BookView.vue` 增加书名编辑对话框、章节编辑对话框（标题 + 正文，正文来自 source 接口）与两处确认弹窗（`ElMessageBox`），章节行增加「编辑」入口；`LibraryView.vue` 在书库被删后显示「书籍已删除 · 已保存的生词和短语仍保留在生词本。」；`stores/library.ts` 增加 `renameBook`、`updateChapter`、`loadChapterSource`、`deleteBook`、`deleteChapter`。正文编辑通过浏览器 textarea 输入，因此该章的行尾统一为 LF（粘贴与 TXT 导入仍保留原始 CRLF）。
|
||||
|
||||
## #11 短语选择、保存与复习（2026-09-11）
|
||||
|
||||
短语与单词共用一张表和一套复习机制：`lexgo_terms` 的 `term` 列存身份键，**单词键不含空格、短语键以空格分隔**，所以「词或短语」不需要额外列，也不需要第二套排期/队列/作答逻辑。`kind` 与词数由身份键在服务端派生（`termKind`/`termWordCount`），视图与队列项随响应返回。
|
||||
|
||||
`server/app/lexgo/phrase.go`：
|
||||
|
||||
| 部分 | 职责 |
|
||||
|---|---|
|
||||
| `phraseWords` | 从本人 ready 章节的 token 里取完全落在选区内的词；**切进单词的范围直接 400**，不静默丢弃；2～12 个词 |
|
||||
| `phraseKey` / `phraseSource` | 身份键＝按顺序的规范化词形以单个空格连接；显示片段＝选区原文（内部标点与换行保留） |
|
||||
| `phraseMatches` | 跨章节匹配：按首词分组后顺序比对词形，候选按 (起点, 长度降序, id) 排序并取**最左最长**的互不重叠集合 |
|
||||
| `phrasesForChapter` | 按 `term LIKE '% %'` 取本人短语并匹配，供 tokens 响应使用 |
|
||||
| `SavePhrase` | 由服务端推导身份后走与单词相同的 `saveTerm` upsert；`kind` 冲突返回 409 |
|
||||
|
||||
| 接口 | 说明 |
|
||||
|---|---|
|
||||
| POST /api/v1/phrases | `{chapterId,start,end,definition,examples[],status,level?}`；本人 ready 章节；服务端推导词序列与身份，不接受客户端身份；首次 201、重复 200 |
|
||||
| GET /api/v1/terms/:id | 复用；响应增加 `kind` 与 `wordCount` |
|
||||
| GET /api/v1/chapters/:id/tokens | 响应增加 `phrases:[{id,status,wordCount,startToken,endToken}]` |
|
||||
| GET /api/v1/reviews/queue | 队列项增加 `kind` 与 `wordCount`；短语与单词同一队列、同一作答接口 |
|
||||
|
||||
学习端：`composables/readerRange.ts` 是纯函数层（整词对齐、内部保留、端点按词调整、命中优先级、区间换算），`composables/useTextSelection.ts` 监听 `selectionchange`（100ms 去抖）与 document 的 `pointerup` 读取浏览器原生选区并映射为 token 索引，**不拦截 touchmove、不 preventDefault**；`ReaderTokens.vue` 为每个 token 输出 `data-token-index` 与短语区间样式；`ReaderView.vue` 负责把选区变成短语、shift 点击扩展、以及面板端点调整；`LookupPanel.vue` 增加短语标题与四个端点按钮；复习卡用 `maskedPrompt` 把整段短语挖成一个空。已保存短语点击优先打开短语面板，单词数据不受影响。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Business-Rules-and-Glossary
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Business-Rules-and-Glossary.-
|
||||
wiki_revision: 7061627fed9b5acd74fbd2369f27acadc0fd2e2a
|
||||
synchronized_at: 2026-09-10T12:36:59Z
|
||||
wiki_revision: fb3ce7506f1eb1940d7aa57226d22e4f8a270633
|
||||
synchronized_at: 2026-09-14T14:21:32Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 业务规则与术语
|
||||
@@ -80,3 +80,165 @@ M0 固定首发语言语料、词条身份规则、短语选择与重叠规则
|
||||
- 日志绝不保存密码、token、Cookie、请求/响应正文、错误堆栈或私人学习内容。合法账号、IP 属于本功能必要的审计数据，仅管理员可查询。
|
||||
- GET /api/v1/login-logs 与 /operation-logs：未登录 401，学习者 403；page 默认 1，limit 默认 20、最大 100。username 精确匹配；操作日志匹配操作人或目标账号。result 为 success/failure，action 限定枚举，from/to 为 RFC3339。返回 data.items/total/page/limit，按时间和编号倒序，时间按毫秒存储、浏览器按本地时区展示。
|
||||
- 固定保留最近 90 天，查询即排除过期记录；默认起止为保留边界和当前时间。清理只删除 created_at 严格早于边界的两表记录，不影响账号、空间、会话。无清空全部或导出按钮；本期不开放保留时长配置。
|
||||
|
||||
|
||||
## #3 英语位置与查询实验契约 v1
|
||||
|
||||
实验版本 `english-spike-v1`。POST /analyze 接收 {text}，返回 status、contract_version、original_text、text_sha256 和 tokens；仅为本机小样接口，不是生产 API。原文以收到的字符串为准，不先做 NFC、大小写、换行或空白归一化；SHA256 对原文 UTF-8 字节计算。tokens 连续覆盖全文，拼接 text 必须逐字符等于原文，空白也有独立区间。空串合法；最多 100000 Unicode code point，拒绝孤立代理项。kind 为 word/space/punctuation；word 是本小样的可点 token 类别，也可能包含数字或 emoji，不保证是自然语言词条。
|
||||
|
||||
每个 token 提供 text、lemma、kind 与半开区间 [start,end)：
|
||||
|
||||
| 字段后缀 | 单位与使用方 |
|
||||
|---|---|
|
||||
| cp | Unicode code point，Python 字符串索引；不是用户感知字形 |
|
||||
| utf8 | UTF-8 字节，可供 Go string 字节切片 |
|
||||
| utf16 | UTF-16 code unit，JavaScript String.slice / DOM 文本位置 |
|
||||
|
||||
例：原文 `A🙂é`，emoji 的 cp=[1,2)、utf8=[1,5)、utf16=[1,3)；后面的 e 加组合重音共两个 code point，cp=[2,4)、utf8=[5,8)、utf16=[3,5)。不能把这些单位混用，也不能把组合字符或 ZWJ 序列的 code point 数当作可见字符数。浏览器先逐 token 校验 UTF-16 切片并检查完整重建再展示；textarea 会按浏览器规范将换行转为 LF，所以 HTTP 契约保证收到的原文，不承诺还原剪贴板进入 textarea 前的 CRLF。服务端 CRLF 原文测试单独覆盖。
|
||||
|
||||
POST /lookup 接收 {surface,lemma?}。查词键单独 casefold/NFC/弯撇号转 ASCII，不改变原文位置；先精确查询 surface，再尝试调用方提供的 lemma。结果 status 为 exact、lemma、not_found 或 resource_missing，含 matched_form 和最多 12 条 entries（lemma/pos/definition/examples）。`dog` 直接命中；点击 `went` 可用上下文 lemma `go` 回退；手动只输入 `went` 不猜词性而返回未找到。词典缺失和模型缺失分别标识 wordnet/model，不能伪装成查无结果。
|
||||
|
||||
实验查询无用户状态、写入或缓存私人输入；重复查询确定性返回，原文哈希可检测文本版本变化，但尚未定义生产 token ID、任务幂等或个人词语合并规则。lemma 不等于学习状态身份，禁止自动合并原词/词元。WordNet 仅英英释义，按 n/v/a/r 与原生 sense 顺序截取，不做上下文义项排序、翻译或发音；`The leaves fell.` 的 leaves 实测被模型错误还原为 leave，此限制保留供后续用户选择/修正方案参考。
|
||||
|
||||
|
||||
## #4 阅读选择与重叠匹配建议
|
||||
|
||||
位置始终引用当前章节不变的原文，半开区间 [start,end)。浏览器 DOM Range 的 start/end 先按正文文本节点累计为 UTF-16，再扩展至触及的完整可选择词元；反向拖选得到同一规范区间。首尾空白/纯标点不独立成词，选区内部标点、空白和换行原样保留。emoji 字形可选，代理对、组合重音和 ZWJ 不拆开；cp/utf8/utf16 均与 #3 单位约定一致。
|
||||
|
||||
推荐以章节 ID、内容版本、原文区间标识一次出现，不使用短语文字定位所有实例。重复出现的 a small step 保持不同 offsets；查找采用区分大小写的精确原文匹配，并要求完整词元端点，不擅自折叠空白或 NFC。例：`a small step; a small step` 两次区间分别为 [0,12)、[14,26)。跨行 `small\nstep` 不与 `small step` 自动等同。
|
||||
|
||||
重叠匹配保留所有候选；只在显示层采用“起点靠前优先，同起点长范围优先，跳过已占用区间”，不合并或删除词条身份。例如 a small step 与 small step 都可保存，前者优先显示；当前用户选区高亮优先于已保存状态。#11 正式集成前再依据实际语言 tokens 验证候选索引与长度限制，本小样不继承未经实测的上游 14 词上限。
|
||||
|
||||
桌面鼠标/手机长按使用原生 Selection；正文中键盘 ←/→ 选相邻词，Shift+←/→ 从锚点扩缩连续范围，起点/终点按钮可用 Tab/Enter。Esc 或关闭清除选区及未保存编辑，保留滚动和键盘阅读位置；切换章节清除选区，当前页面内分别记住章节滚动位置。纯标点的新选区清除旧面板，防止操作上一个词。个人释义必须显式保存，状态变化也随保存提交；本小样仅写内存。
|
||||
|
||||
手机方案保留浏览器原生长按、选择手柄和滚动，面板最多占底部 42dvh，正文有底部阅读余量；键盘定位用滚动边距避开面板。未获得真实手机结果，不能判定手柄、系统菜单、虚拟键盘或触摸滚动冲突已经解决。
|
||||
|
||||
## #5 粘贴分章、任务与阅读规则 v1
|
||||
|
||||
用户于 2026-09-10 确认两项边界（工单 #5 评论 7644）：本单按 Go 处理、不接入 Python；一次粘贴等于一个章节。
|
||||
|
||||
- 固定分章规则：一次粘贴产生一个章节，不按空行或长度自动再分。新建书籍时书籍标题与首章标题同为提交的标题；追加时标题即新章节标题。分章规则变化属于需求变化，必须重新确认。
|
||||
- 长度与校验：标题去首尾空白后 1～120 个字符；正文必须含至少一个非空白字符；正文上限 100000 Unicode code point，超出返回 400；语言当前只接受 en。
|
||||
- 原文保真：正文按收到的字符串原样保存与返回，不做 NFC、大小写、换行或空白归一化；页面使用 white-space: pre-wrap 展示，制表符、连续空格与空行保持可见。处理完成前不返回原文。
|
||||
- 归属：书籍、章节与任务都记录认证账号的 owner_id；他人编号返回 404；管理员角色不解除学习数据的本人归属；后台任务只使用任务行的 owner，不信任客户端用户编号。
|
||||
- 任务状态：pending、processing、ready、failed，章节与任务共用同一词表。失败时返回固定原因码加可读中文提示，错误字段不保存正文。
|
||||
- 固定失败原因：unsupported_language、too_long、empty_text、content_changed。前三种只能由其他写入路径产生（例如语言调整或后续编辑功能）；content_changed 表示章节内容在处理前被改动，属于过期任务，必须重新提交，或恢复为提交时的内容后重试。
|
||||
- 幂等：客户端 requestId 与账号构成唯一键。同一 requestId 配同标题同正文的重复提交返回首次结果，不新建章节；同一 requestId 配不同标题或正文返回 409；并发重复提交同样只产生一个章节。重试复用原章节，只增加尝试次数。
|
||||
- 恢复：声明与完成分属两个事务。进程在声明后退出时，重启把 processing 的章节与任务放回 pending 并保留尝试次数，不产生重复章节。
|
||||
- 未决边界：Go＋Python NLP 与全 Go 路线尚未确认。本单只做 Go 校验、分章与发布，不产生 token、lemma 或词典索引；正式接入前必须由用户确认路线，再定义生产 token 契约。
|
||||
- 已知限制：本单未设置每账号书籍数量或总容量配额，只限制单次正文与请求体大小；删除书籍/章节属 #10，导入失败不会自动重试，只在启动时恢复被中断的 processing 任务。
|
||||
|
||||
## #5 审核整改（R1～R4，2026-09-11）
|
||||
|
||||
工单 #5 的整改记录见该工单评论；本节只记录长期有效的契约变化。
|
||||
|
||||
- 追加章节沿用所属书籍的语言：`POST /api/v1/books/:id/chapters` 的请求体只有 requestId、title、text，不接受 language；服务端仍拒绝未知字段，客户端发 language 会得到 400。新建书籍的 `POST /api/v1/books` 才带 language。学习端已按此拆分请求体，避免两个契约共用同一结构。
|
||||
- 导入任务的自动重试有上限：同一任务被 worker 领取的次数达到 5 次后，任务与章节转为 failed，原因码 `attempts_exhausted`，提示“处理多次失败，请重试或重新提交”。人工重试（POST /jobs/:id/retry）会重置尝试次数，因此人工操作不受该上限阻塞。
|
||||
- 运行期恢复不依赖重启：除了启动时的恢复，运行中的服务每次轮询都会把停留在 processing 且超过 15 秒的任务放回 pending，因此“领取已提交、完成事务失败”不会让章节永久卡在处理中。该阈值必须长于正常的领取到完成窗口；重复处理同一任务不会产生第二个章节，因为任务从不创建章节。
|
||||
- 学习端离开页面时作废在途请求：目录页与阅读页在关闭时推进各自的请求序号，晚到的响应不会写回状态或重启轮询；导入页在提交过程中离开后，晚到的成功响应不会把用户导航回书籍页。
|
||||
- 重试结果立即生效：重试被接受后先把返回的 pending 状态写入界面并继续轮询，因此紧随其后的一次刷新失败不会让页面停在处理失败。
|
||||
|
||||
|
||||
## 全 Go 正式架构决定（2026-09-11，#6）
|
||||
|
||||
用户已明确选择全 Go：正式英语分词、原文位置映射、本地词典解析和词形候选查询由 Go 后端完成，不运行 Python NLP 服务。此前“Python 建议/全 Go 未决”仅为历史决策记录，由本决定覆盖；spikes/english 保留历史验证，不接入产品。#6 按该方向实施，当前方案见工单最新启动评论；WordNet 3.0 仍为首个资源（英语释义），词形规则候选不等同于 spaCy 上下文消歧，原文及个人学习状态不按候选合并。
|
||||
|
||||
|
||||
## #6 正式词典规则（2026-09-11）
|
||||
|
||||
首个资源固定 Princeton WordNet 3.0，英语释义；ZIP 来源及 SHA 以 server/wordnet-resource.json 为准，LICENSE 原样保留于 server/WORDNET-LICENSE.txt。管理员上传指定包并配置显示名称，语言/来源/版本/格式固定；32MiB压缩、128MiB解压上限、成员与结构校验。资源为所有账号共享、仅管理员写；失败保持旧资源，重复上传复用id并启用，停用后查询返回 resource_missing。ZIP 随数据库备份，无运行时网络下载。
|
||||
|
||||
正式分词由 Go 完成：字母开始词，组合标记延续词，内部直/弯撇号连接字母；空白逐字保留，数字和符号为不可查询的 punctuation。start/end 是 Unicode code point 半开范围，另有UTF-16偏移；原文、CRLF、组合字符与emoji不归一化。只对查询键小写/NFC/撇号归一化。每次查询必须是本人ready章节内的完整单词，最大128码点，跨账号404，非法范围400。
|
||||
|
||||
exact优先；未命中再按WordNet异常表/词尾规则查候选，词性顺序n/v/a/r、最多12条释义，保留原数据s词性。返回lemma表示规则候选，不承诺上下文消歧；歧义不会合并个人学习状态。not_found与resource_missing区分，网络错误可重试，不阻断阅读。#3使用spaCy上下文lemma的实验路径由全Go规则候选替代。
|
||||
|
||||
个人释义目前仅当前选择的临时草稿，界面标记未保存；换词、关闭、换章、离页和身份变化清理。持久化与个人词汇状态由#7实现，不把临时输入宣传为保存成功。
|
||||
|
||||
## #7 个人词条规则（2026-09-11）
|
||||
|
||||
个人词条身份是「学习者＋语言＋规范化词形」：`normalizeWord` 做 NFC、小写与弯撇号转换，而原文、章节文本与显示用的原词形都不归一化。同一词形不同大小写是同一条记录；不同词形（`dog` 与 `dogs`）是不同记录，不按 WordNet 候选或 lemma 合并，与 #6「不按候选合并个人学习状态」一致。语言取自已登录学习者的英语空间，不从请求读取。
|
||||
|
||||
| 业务状态 | level | 含义 | 进入到期复习 | 原版 stage |
|
||||
|---|---|---|---|---|
|
||||
| new 新词 | 0 | 已保存、尚未开始学习；首次保存默认 | 由 #8 决定 | 2 |
|
||||
| learning 学习中 | 1～7 | 正在复习，越接近 7 越熟 | 是 | -1～-7 |
|
||||
| known 已知 | 0 | 已掌握，不再进入到期复习 | 否 | 0 |
|
||||
| ignored 忽略 | 0 | 明确忽略，不计入已知 | 否 | 1 |
|
||||
|
||||
上表是原版合并编码（状态与等级在同一字段）的显式替代，供 CSV 导出与旧数据迁移映射；#7 只保存与返回等级，#8 负责复习推进与到期时间。非 `learning` 状态携带非 0 等级、`learning` 等级超出 1～7、以及未知状态一律拒绝，`learning` 缺省等级为 1。
|
||||
|
||||
个人释义可为空（允许只记录状态），最长 2000 字符，可含换行与制表符；例句最多 5 条、每条最长 500 字符，不能为空行或含换行。保存失败保留学习者已输入的内容，成功后显示「已保存 · 状态」并立即更新正文高亮。
|
||||
|
||||
保存幂等由唯一键承担：重复提交同一词形只更新同一行，不产生第二条冲突记录；同一账号多端同时编辑为最后写入生效，本版不引入版本冲突拒绝。所有读写都属于会话本人：章节必须本人且已就绪，篡改 id、owner、language、term 或携带未知字段返回 400，他人编号与本人不可见编号统一 404。跨账号不共享任何数据与前端缓存，退出或切换账号后不保留前一账号的词条与高亮。
|
||||
|
||||
例句只保存学习者手输内容，不自动关联原文句子：当前分词只有词/空白/标点边界，没有句子切分规则。原型 v1 中「原文例句已关联到词条」是演示文案，不作为契约。等级选择器属于 #8/#12 的编辑界面，阅读器面板只提供四个状态。
|
||||
|
||||
## #8 复习规则（2026-09-11）
|
||||
|
||||
**间隔表**：答对后按新等级排期，等级上限 7。这是固定表，不是 FSRS，也不是上游「按等级选复习量少的日期」的算法。
|
||||
|
||||
| 等级 | 1 | 2 | 3 | 4 | 5 | 6 | 7（上限） |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 下次复习 | 1 天 | 2 天 | 4 天 | 7 天 | 15 天 | 30 天 | 60 天 |
|
||||
|
||||
**入队范围**：只有状态 `新词` 或 `学习中` 且 `due_at ≤ now` 的词条进入队列；`已知` 与 `忽略` 不入队。新保存的词立即到期（`due_at` = 保存时刻），保存后就能复习；手动把词条改为「学习中 level N」时下次复习为 `now + 间隔[N]`，避免刚标等级就被当成到期。只有新建词条或状态/等级实际变化才移动复习时间：只修改释义、例句或原词形时保留原有排期，否则编辑文本会把逾期词条挤出当天队列。保存未提及等级时保留已获得的等级，只有进入 `学习中` 才从 1 开始，因此阅读器面板的保存不会把等级重置为 1。已在队列中的词条被改成 `已知`/`忽略` 后再次作答会被拒绝（409），排队信息由服务端裁决而不是前端过滤。
|
||||
|
||||
**三个评分动作**：`correct` 认识/答对 → 等级 +1（封顶 7）、按新等级排期、离开本轮；`wrong` 不认识/答错 → `学习中` 降一级（最低 1）、`新词` 保持 `新词`、`due_at = now` 立即回到本轮；`again` 再学一次 → 等级与状态不变、`due_at = now` 立即回到本轮。计数上 `correct_count` 只统计 `correct`，`wrong_count` 统计 `wrong` 与 `again`，`review_count` 统计全部已应用作答。词条的原文、个人释义与例句不因复习改变。
|
||||
|
||||
**时区与到期边界**：`due_at` 以 UTC 绝对时刻存储，到期判定是 `due_at ≤ now`，不引入本地日边界。理由：MVP 没有用户时区设置（属 X 系列边界），绝对时刻在多账号自托管下语义一致、没有夏令时陷阱；代价是复习时刻会随首次作答时间漂移，例如 22:00 答对的 1 天间隔词条要到次日 22:00 才到期。原型界面的「到期复习 N / M」指当前到期队列的位置与本轮卡片数，不是自然日统计。
|
||||
|
||||
**幂等与并发**：客户端每次作答生成一个 `answerId`，服务端以 `UNIQUE(owner_id, answer_id)` 去重。同一个 `answerId` 再次提交返回首次结果并把 `duplicate` 标记为 true（`result` 仍是首次的 `applied` 或 `stale`），不改变等级、间隔和次数，因此客户端重试可以按首次结果计数；同一词条在别处已经被推进（提交回传的 `expectedDueAt` 与服务端当前 `due_at` 不一致）时记为 `result=stale`，同样不改变任何状态。因此网络重发、双击、双标签页作答都只记账一次。每次尝试都会落一条 `lexgo_review_answers`（含 `result`），这是「只记账一次」的证据，也供 #13 统计使用。
|
||||
|
||||
**归属与错误**：词条归属由服务端按会话裁决，`owner` 不接受客户端输入；他人词条与不存在的词条统一 404，未登录 401，未知评分/缺少 `expectedDueAt`/未知字段 400，词条已变成 `已知`/`忽略` 409。复习接口不写审计日志：答题属于私人学习内容。
|
||||
|
||||
**范围边界**：本单只做单词复习。短语复习归 #11，到期范围筛选与策略配置界面（X11）不做，练习模式（X08）不做，进度与「已知词/待复习」计数归 #13。#8 不改变 #7 定下的身份口径：复习状态按规范化词形归属，仍不按 WordNet lemma 候选合并。
|
||||
|
||||
## #9 TXT 上传规则（2026-09-11）
|
||||
|
||||
**支持的编码**：只接受 UTF-8，允许带可选 UTF-8 BOM。BOM 在解码时剥离，不进入原文；其余字节必须整体合法，任何非法序列**直接拒绝**，绝不使用替换字符，因此章节里不会出现学习者没有写过的乱码。UTF-16（含记事本「另存为 Unicode」产生的大小端 BOM）单独识别并提示「请另存为 UTF-8 后重试」；GB18030、Latin-1 等其他编码按非法 UTF-8 拒绝。UTF-16 支持不在本单范围。
|
||||
|
||||
**换行与空白**：不做任何归一化，CRLF、LF、制表符、行尾空格与空行按原字节保存，阅读器以 `pre-wrap` 原样呈现——与粘贴路径一致。
|
||||
|
||||
**大小上限**：文件字节上限 2 MiB；解码后再套用单章上限（非空、≤100000 码点）。超限返回 400 并明确提示，不截断、不部分导入。2 MiB 对 100000 码点的 UTF-8 文本有足够余量。
|
||||
|
||||
**文件生命周期与路径**：上传内容只存在于内存中，解码后直接进入导入事务；服务端**不创建临时文件**，所以没有需要清理或可能泄漏的文件；**客户端文件名不参与任何文件系统路径、也不写入数据库**，它只在选择文件时用于显示（并可预填标题）。因此文件名即使写成 `..\..\windows\system32\evil.txt` 也不会影响任何存储位置。上传并发按单槽限制，忙时返回 429。
|
||||
|
||||
**导入与幂等**：上传与粘贴共用同一套规则——一次提交一章，`requestId` + 内容 SHA 保证重复上传同一文件只产生一章（返回第一次的章节并标记 `duplicate`），同一 `requestId` 换成其他内容返回 409。任务状态、失败重试与崩溃恢复沿用 #5 的任务机制，不新增状态。标题规则与粘贴完全相同（去空白后非空、≤120 字符）；省略 `language` 时默认英语，与粘贴一致；追加章节不接受 `language`。
|
||||
|
||||
**范围边界**：不包含 EPUB、PDF、字幕与其他文件格式；不做按空行自动分章；不做 UTF-16/GB18030 转码；不做断点续传；不把来源文件名持久化（若将来需要「导入来源」溯源，另立范围）。
|
||||
|
||||
## #10 编辑、版本与删除规则（2026-09-11）
|
||||
|
||||
**可编辑内容**：书名、章节标题、章节正文。只有正文变化才重新处理；只改标题不改变处理状态，也不新建任务。
|
||||
|
||||
**版本规则**：`chapters.content_sha256` 是章节的版本键，`jobs.content_sha256` 是任务被创建时对应的版本。任务只能在版本匹配时影响章节：
|
||||
- 过期版本的任务被标为 `error_reason=superseded`（「章节内容已更新为新版本，本次处理已作废」），**不会**把章节标成失败、也不会发布旧文本；界面上失败章节的重试按钮只出现于当前版本的任务。
|
||||
- 任务重试要求版本匹配，否则 409，避免把当前章节拉回旧版本再失败一次。
|
||||
- 重复保存同一正文不是新版本：不新建任务、不改变状态。改回原内容（内容相同）同样不触发处理。
|
||||
- 编辑章节正文会保留章节编号与阅读入口（URL 不变），章节状态回到待处理，处理完成后原文即新版本。
|
||||
|
||||
**存储文本与版本一致性**：章节存储的正文重新计算出的 SHA 必须等于存储的 SHA；出现不一致（绕过 API 的直接写入）时按 `content_changed` 失败，不发布不确定内容。
|
||||
|
||||
**删除规则**：删除在事务内**硬删除**，并沿用现有外键级联清理：书籍 → 章节 → 任务。删除章节后剩余章节序号重排为连续（原型显示「剩余 N 章」，导航按序号取值）；并发删除同一章由书籍行锁序列化，结果是一个成功、一个 404。重复删除返回 404，不把「已经不存在」当成成功。恢复路径是数据库备份与完整恢复（#15 演练范围），产品不提供回收站或撤销。
|
||||
|
||||
**个人学习记录保留**：删除书籍或章节**不删除**个人词条、复习排期与作答记录，因为它们按学习者归属、不引用章节；界面也没有「来自某章节」的引用（例句是学习者输入的副本）。所以删除只影响书籍、章节与处理任务，这一点在原型确认文案中就写明：「本章正文将被删除，已保存词条保留」「删除这本书及其章节？已保存的生词和短语将保留」。
|
||||
|
||||
**归属与边界**：改名、编辑、删除、读取编辑用原文都严格按会话归属；他人资源与不存在资源统一 404，未登录 401，空标题/空正文/超长文本/未知字段 400。学习端正文编辑框的行尾会统一为 LF（浏览器 textarea 行为），粘贴与 TXT 导入路径仍然保留原始 CRLF 与空白。
|
||||
|
||||
**范围边界**：不做封面与音频附件（#21）、不做回收站/撤销、不做批量操作、不做章节跨书移动、不做语言变更。
|
||||
|
||||
## #11 短语规则（2026-09-11）
|
||||
|
||||
**身份**：短语身份＝按阅读顺序的规范化词形以单个空格连接（`a small, step` 与 `a small step` 都是 `a small step`），归属 `(owner, language)`；显示片段保存最近一次保存时的原文（内部标点、换行、多余空格原样保留，仅用于显示）。与单词共用同一张表，因此同一短语在不同章节保存只会得到一条记录，也共用同一幂等键与同一复习排期。
|
||||
|
||||
**范围**：两端对齐整词；首尾若落在空白或标点则跳过；**内部**标点与换行保留但不参与身份比较；不切开代理对、ZWJ 与组合字符。选区内少于 2 个词不是短语（单个词走单词面板），超过 12 个词、身份键超过 128 字符或原文片段超过 191 字符都返回 400 并给出可读提示。切进单词中间的范围被拒绝，而不是静默丢弃那个词。
|
||||
|
||||
**跨章节匹配**：在章节的词片段序列中找**连续词**，其规范化词形逐个相等（中间允许任意标点与空白）。因此编辑正文后：短语仍出现则继续高亮；不再出现则该章不高亮，但**词条与复习排期保留**；章节被删除同样保留。短语不存章节锚点，所以不存在悬空引用。
|
||||
|
||||
**重叠与点击**：同一位置多个候选按 (起点升序, 长度降序) 取互不重叠者，即「最左最长」，结果与输入顺序无关。短语高亮覆盖其内部的单词高亮，但**不修改单词数据**（状态、排期、计数都保留）；点击命中规则是「在已保存短语范围内 → 打开短语面板，否则打开单词面板」。保存后的高亮立即出现，不需要重新加载分词。
|
||||
|
||||
**选择交互**：桌面用浏览器原生拖选（`selectionchange` 去抖 + `pointerup`），手机依赖系统选择手柄且**不拦截 touchmove**，面板提供起点/终点四个按钮按词调整，Shift 点击可把范围从一个词扩展到一个词，Escape 取消并保留阅读位置。单击（折叠选区）仍然是单词查询。真实手感属于运行验证范围，真机证据缺口保留。
|
||||
|
||||
**复习**：短语进入同一个到期队列与同一套间隔表；卡片正面显示短语并把**整段短语挖成一个空**（例句里没有该短语时只显示短语本身），答案面显示个人释义；答对/答错/再学与幂等、stale 规则与单词完全一致；计数归属也一致（`correct_count` 只计答对，`wrong_count` 计答错与再学）。
|
||||
|
||||
**范围边界**：不做短语自动合并同义形式、上下文词性消歧、短语跨书移动、批量编辑（#12）、真机手柄精细手感。短语的例句同样是手输内容，不自动关联原文句子。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Local-Development-and-Verification
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Local-Development-and-Verification.-
|
||||
wiki_revision: 8857cf2545a3de6f1efa07d88b920a8301080e56
|
||||
synchronized_at: 2026-09-10T12:43:25Z
|
||||
wiki_revision: 1643f05aa6987196d5dcc82a07e2daa07e2f2b4d
|
||||
synchronized_at: 2026-09-14T14:21:32Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 本地开发与验证
|
||||
@@ -192,3 +192,245 @@ supervisor 直接管理编译后的 Go 进程，运行时不调用 Python。数
|
||||
|
||||
|
||||
#18 用户验收：2026-09-10T20:40:04+08:00 用户确认日志通过验收（工单评论 7591），包含此前待人工检查的交互。未重新运行自动化测试，未更改其历史结果，未合并 PR 或发布生产。
|
||||
|
||||
|
||||
## #3 英语离线验证入口与复现
|
||||
|
||||
从仓库根目录执行（uv 与 Node 已安装，不能使用本机默认 Python 3.8）：
|
||||
|
||||
```powershell
|
||||
uv venv --python 3.12.12 .local/nlp-venv
|
||||
uv pip install --python .local/nlp-venv/Scripts/python.exe -r spikes/english/requirements.lock
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/setup_resources.py
|
||||
uv pip install --python .local/nlp-venv/Scripts/python.exe --no-deps .local/nlp-resources/en_core_web_sm-3.8.0-py3-none-any.whl
|
||||
.local/nlp-venv/Scripts/python.exe -m unittest discover -s spikes/english -v
|
||||
node --test spikes/english/view.test.mjs
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/benchmark.py
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/app.py
|
||||
```
|
||||
|
||||
打开 http://127.0.0.1:5183/，默认虚构样例，分析后点击 went 应出现 go 与“按原形查询”；dog 直接命中，zzzxqvfiction 未找到。`--resources .local/absent-resources` 可验证词典缺失，`--port` 可更换临时端口。模型缺失、词典缺失、非法输入及内部错误不输出路径/正文。仅 loopback，Host/Origin 校验，禁跨域、无缓存、无访问日志、连接读超时 10 秒。未配置 supervisor；停止该临时进程即可回退，既有服务和数据不变。
|
||||
|
||||
首次准备需要联网，失败可重跑；资源文件通过固定 SHA256 校验后使用。模型 3.8.0 MIT，WordNet 3.0 ZIP 完整保留 LICENSE/版权/免责声明，spaCy MIT、NLTK Apache-2.0。固定资源 URL 和摘要见 spikes/english/resources.json，原始许可与来源见该目录 README。词典为英英格式，不是中文翻译库。
|
||||
|
||||
2026-09-10 实测：11 项 Python 测试和 2 项 JavaScript 测试通过。真实模型/词典测试及 benchmark 禁止 socket connect，验证运行期无在线翻译依赖；不是整机断网测试。浏览器已实测展示分词、点击 went→go、手动查询无结果。外部 spaCy/Click 有一条 DeprecationWarning，未影响测试结果。Windows 10 19044，Intel Family 6 Model 140、8 逻辑核，Python 3.12.12；完整环境、UTC 时间和样本保存在 benchmark-result.json。
|
||||
|
||||
| 测量 | 本次样本 |
|
||||
|---|---|
|
||||
| 冷进程 Engine 加载（含 import，文件系统缓存可能已热） | 2355 ms |
|
||||
| 首次分析 / 首次 dog 查询 | 见 benchmark-result.json（各 1 次） |
|
||||
| 100000 code point（106095 UTF-8 字节），spaCy 3 次 | 中位 1433 ms，约 6.98 万 cp/s |
|
||||
| 同文正则＋WordNet morphology，3 次 | 中位约 336 ms；未包含三位置转换，非完全等价负载 |
|
||||
| dog 查询，热进程 100 次 | 中位 0.0149 ms，p95 0.023 ms |
|
||||
| 12 个显式 lemma 样例 | spaCy 11/12、基线 6/12；保留 leaves 错误 |
|
||||
|
||||
推荐 Python NLP，但该样本不代表一般准确率、生产并发能力或延迟保证。尚未验证正式 Go/Python 调用、长任务持久化、移动端划词（#4）、英汉词典及生产部署。
|
||||
|
||||
|
||||
#3 于 2026-09-10T21:50:14+08:00 通过用户验收（工单评论 7609），独立小样验证完成。此前实测结果和限制保持；本次验收未重跑测试。
|
||||
|
||||
|
||||
## #4 阅读选择验证
|
||||
|
||||
启动：`python spikes/selection/serve.py`，本机 http://127.0.0.1:5184/。默认仅 loopback，提供白名单静态文件，无目录列表或写入接口。虚构英文两章，任意词/连续短语可选；模拟释义与保存仅页面内有效。完整操作及真机清单见 spikes/selection/README.md；本单不配置 supervisor。
|
||||
|
||||
自动测试：`node --test spikes/selection/range.test.mjs spikes/selection/ui.test.mjs`。Node 22.22.1；范围测试无外部依赖，DOM 测试复用 learner 锁定的 jsdom（需要时先 `pnpm --dir learner install --frozen-lockfile`）。11 项通过，覆盖原文分区、反选、词边界、跨行/标点、组合字符/emoji/ZWJ、三位置单位、重复/重叠、关闭后键盘续读、标点取消旧选区、DOM Range 映射与显式保存。关闭丢失键盘位置及标点残留均复现旧代码失败，再验证修正通过。JS 语法检查通过。
|
||||
|
||||
桌面 Chrome 实际验证：鼠标拖选 `Mira opened the workshop`，原生选择与正文高亮相等；按钮扩展终点后可包含 before；键盘连续选择及关闭后下一词续读正常；模拟保存与词典不可用提示正常。章节中段直接点击关闭前后 scrollY 同为 909.5999755859375。自动化 locator.click 会在事件之前滚动到按钮，因此用基于可见坐标的直接点击复核；页面“验证设置”记录事件内关闭前后位置，避免将工具预滚动误判为页面变化。
|
||||
|
||||
桌面窄屏约 390×844 CSS 像素检查无横向溢出，底部面板正常；键盘选中词底部 464.4、面板顶部 489.5，未被遮挡。此为桌面浏览器宽度检查，不是移动系统或触摸模拟结果。视口已恢复。
|
||||
|
||||
**真机项未通过：尚未取得实际手机浏览器结果。** 需要用户记录设备/系统/浏览器版本、长按/手柄扩缩跨行范围、普通滚动、底部面板遮挡、虚拟键盘以及关闭位置，回填 #4。可在明确进行测试时以 `--host` 指定电脑实际局域网 IPv4，手机同网访问；不自动放行防火墙或开放公网。用户已明确验收 #4 并关闭工单；该详细测试证据缺口仍保留，不能将其改记为已执行通过。
|
||||
|
||||
独立规格及代码评审已修复已知问题；没有真实账号、生产数据、词典联网或 SRS 测试。停止小样即可回退，现有学习端/管理端/数据库不变。#21 附件不在本次范围。
|
||||
|
||||
|
||||
#4 用户验收记录：2026-09-10T22:51:52+08:00 用户确认“#4通过验收”（评论 7636）。未补充手机型号/浏览器/操作记录，未重跑测试，未自动合并 PR。后续正式移动端集成应补真机回归。
|
||||
|
||||
## #5 粘贴导入与章节阅读（schema v3）
|
||||
|
||||
升级步骤（本机，仓库根执行）：停止 lexgo-api → `python scripts/server.py build` → `python scripts/server.py migrate` → 启动 lexgo-api。lexgo_dev 已从 v2 升到 v3，新增 lexgo_books、lexgo_chapters、lexgo_ingest_jobs；迁移前后 sys_user 4、lexgo_spaces 4、lexgo_sessions 3、lexgo_login_logs 23、lexgo_operation_logs 1 完全一致。托管实例重启后 /healthz 返回 200，两端首页仍为 200，lexgo-admin 与 lexgo-learner 的 PID 未变化。
|
||||
|
||||
回退：停止 API，把 lexgo_schema 中 id=1 的版本从 3 改回 2，并恢复上一二进制；三张新表保留不删除，旧程序不读写它们。重新升级时显式 migrate 重新执行 IF NOT EXISTS 语句即可；集成测试覆盖 v2→v3 的既有数据保留与 v2 标记下的重复迁移。未在开发库演练回退。
|
||||
|
||||
学习端入口：http://127.0.0.1:5173 → 登录 → 我的书库 → 粘贴文本导入 → 章节就绪后进入阅读。
|
||||
|
||||
测试命令与结果（仓库根执行；本单使用专用库 lexgo_test_issue5，不借用其他测试库）：
|
||||
|
||||
| 命令 | 本次结果 |
|
||||
|---|---|
|
||||
| `python scripts/server.py test-integration`（LEXGO_TEST_DB_NAME=lexgo_test_issue5） | 全部通过：18 个顶层用例，其中 #5 新增 8 个（7 个书库/章节/任务/阅读 + 1 个 v2→v3 数据保留），另含 12 个子用例；既有 10 个用例保持通过 |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner test:unit --run` | 3 个文件 31 项通过（session 9、library 16、reading 6） |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner build` | vue-tsc 类型检查与 vite 构建通过，退出码 0 |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner test:e2e` | 3 项通过（既有 auth 2 项 + 新增 reading 1 项，均为虚构 API 响应） |
|
||||
|
||||
### 真实 API + MySQL 实测（2026-09-10，lexgo_dev）
|
||||
|
||||
使用本单新建的虚构账号 issue5_a、issue5_b，口令只保存在忽略的 .local/issue5-accounts.json；未改动 admin、dev、learner_a、learner_b。脚本 .local/verify-issue5-api.ps1 只在本机运行，不输出口令。
|
||||
|
||||
- 粘贴：HTTP 201，章节与任务均为 pending，charCount 99。
|
||||
- 处理：实测状态序列 pending → ready，约 1132 ms（后台任务每秒轮询）；job attempts=1。
|
||||
- 阅读：originalText 与提交正文逐字符相等，CRLF、制表符、弯引号、破折号、省略号、é 加组合重音、emoji、行尾空格与空行全部保留；sha256 前缀 ce7357ea22a3。
|
||||
- 幂等：同一 requestId 重复提交 HTTP 200、duplicate=true、章节与任务编号不变；同一 requestId 换正文 HTTP 409；书库仍为 1 本。
|
||||
- 隔离：issue5_b 读取 issue5_a 的书籍、章节、任务以及追加、重试全部 404；请求体带 ownerId 与查询参数 ownerId 均 400；issue5_b 书库为空。
|
||||
- 追加与阅读导航：新章节 ordinal=2，处理后就绪，前后章节编号互相指向。
|
||||
- 校验：空标题、纯空白正文、非 en 语言、缺少 requestId、超过 100000 code point 分别返回 400 与可读中文提示。
|
||||
- 遗留 fixture：lexgo_dev 中 issue5_a 名下 1 本虚构书、2 个就绪章节（bookId=1，章节 1、2）。
|
||||
|
||||
浏览器实测：真实学习端 + 真实 API + 真实 MySQL 联测（临时 Playwright 用例，运行后删除）：issue5_a 登录 → 书库显示既有虚构书与“导入内容”入口 → 导入页粘贴含空行、制表符、行尾空格与 emoji 的正文 → 书库页由“处理中”变为“已就绪” → 阅读页 article.reader-text 的 textContent 与粘贴正文逐字符相等、computed white-space 为 pre-wrap → “下一章”切换到第二章且正文精确相等 → 390×844 视口下横向溢出 0 px。同一轮还运行了 3 项虚构 API 的既有 e2e，共 4 项通过。
|
||||
|
||||
截图保存在本机 .local/evidence/（library.png、book-after-paste.png、reader-desktop.png、reader-mobile-390.png）并作为工单附件上传，便于人工目视复核；本次会话模型不能读取图片，截图未经 Agent 目视检查，功能断言来自上面的程序化检查。
|
||||
|
||||
未验证：处理失败到重试的用户界面路径只由集成测试覆盖（无法通过 API 主动制造处理失败）；真实手机浏览器长按、手柄、滚动与虚拟键盘仍属 #4 缺口，本次只用桌面浏览器窄屏检查，不能当作真机结果；Python NLP 未接入，token、lemma 与词典仍为 #3 小样范围；生产并发、容量、备份恢复与部署不在本单范围。
|
||||
|
||||
## #5 审核整改验证（R1～R4，2026-09-11）
|
||||
|
||||
整改提交与完整证据见工单 #5 的整改评论。本次复核命令与结果（仓库根执行，专用测试库 lexgo_test_issue5）：
|
||||
|
||||
| 命令 | 本次结果 |
|
||||
|---|---|
|
||||
| `python scripts/server.py test-integration` | 20 个顶层用例全部通过（新增 `TestMySQLIngestRecoveryWithoutRestart`、`TestMySQLIngestAttemptsAreBoundedAndManualRetryRestarts`） |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner test:unit --run` | 38 项通过（library 21、reading 8、session 9） |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner build` | 通过（vue-tsc + vite） |
|
||||
| `npx --yes pnpm@9.15.1 --dir learner test:e2e` | 3 项通过（虚构 API 响应） |
|
||||
|
||||
回归测试的有效性：新增的前端 7 项用例先在整改前的 `library.ts`/`ImportView.vue` 上运行并实际失败（追加发送 language、离页后响应写回、重试后停在失败），改回修复版本后全部通过。
|
||||
|
||||
真实联调（lexgo_dev，虚构账号 issue5_a）：
|
||||
|
||||
- R1 追加：真实学习端从书籍页进入“追加章节”，提交后回到书籍页，新章节就绪后可阅读；抓取到的请求体只有 requestId、title、text，无 language；正文逐字符相等。
|
||||
- R2 恢复：新建章节后用 SQL 把任务与章节置为 processing 且 updated_at 早于阈值（UTC 时间），**不重启服务**，运行期清扫在 544 ms 内把任务重新入队并发布为就绪，章节与任务编号不变，正文逐字符相等。
|
||||
- R4 重试：SQL 制造真实失败任务（content_changed）后，在浏览器点击“重试”并中断其后的第一次刷新请求，页面立即由“处理失败”变为“处理中”，随后自行变为“已就绪”，无需手工刷新。
|
||||
|
||||
注意：MySQL 会话时区为 SYSTEM（本机为 UTC+8），而服务按 UTC 存储 DATETIME；核对任务时间时使用 UTC_TIMESTAMP 而不是 NOW()，否则会出现 8 小时的假偏差。
|
||||
|
||||
未在本轮验证：R3 的真实浏览器时序（离页与响应同时发生）只由单测覆盖；真机手机证据仍属 #4 缺口。
|
||||
|
||||
|
||||
## #6 部署与验证（2026-09-11）
|
||||
|
||||
正式后端 schema v4，只增加 lexgo_dictionaries。显式 migrate 后启动纯Go lexgo.exe；Python scripts/server.py 仍是开发命令封装，产品运行不依赖Python NLP。
|
||||
|
||||
本机已从v3升级v4并重启lexgo-api，升级前后sys_user/lexgo_spaces/lexgo_sessions/lexgo_books/lexgo_chapters/lexgo_ingest_jobs计数一致。旧二进制保存在忽略的 .local/lexgo-pre-issue6.exe。回退：停止API，恢复旧二进制，将已确认v4的schema标记恢复3，保留新增资源表及全部学习数据，再启动旧API；不要删除数据或重新bootstrap。
|
||||
|
||||
管理端 http://127.0.0.1:5174 的“英语词典”页可下载指定包并导入/启停；本机已导入固定WordNet3.0，155287个词形/词性索引项。学习端 http://127.0.0.1:5173 打开本人ready章节，点词或Enter/空格查询，Escape关闭。went/mice应出现go/mouse候选。源包、本机凭据与测试证据仅存在忽略的.local，不进入Git。
|
||||
|
||||
验证：Go全包MySQL集成（专用lexgo_test_issue5）与go vet通过；学习端53单测、类型检查/构建、默认5173 Playwright3项通过；管理端31单测与lint通过，构建含已有Sass弃用与bundle体积提示；治理56测试与strict通过。实际ZIP解析、schema3→4原文/任务保留、权限、失败保留、重复启用、冷缓存路由重建都有覆盖；冷缓存测试不是完整备份恢复演练。
|
||||
|
||||
主审真实API联调使用真实管理端session模块经5174代理上传；经5173代理两测试账号分别创建虚构章节并验证精确/不规则词形、Unicode原文片段、越权404/普通用户管理403、停用/重复导入启用、错误ZIP保留资源。新增测试书籍id3/4、章节id8/9归issue5_a/issue5_b，没有修改其他账号的书籍。
|
||||
|
||||
常驻5173一度返回空白页：Vue模块转换500、代理缺失；同代码隔离服务正常，只重启lexgo-learner加载配置后恢复，默认E2E通过，未改启动配置。桌面交互浏览器工具因旧会话失效未完成手工真实UI联调；已有项目Playwright使用模拟API，真实API验证另列。手机仅窄屏自动测试，真机证据仍未补齐。
|
||||
|
||||
## #7 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue7，不借用其他测试库。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue7 python scripts/server.py test-integration` | 34 个顶层用例全部通过、0 跳过；含 #7 新增 3 个 MySQL 用例、4 个单元用例和 1 个 v4→v5 迁移用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 58 项单测、类型检查、构建、3 项默认 E2E 全部通过 |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：非法状态与等级边界、文本上限、词形身份与大小写合并、词形不按 lemma 合并、重复与并发保存只留一条记录、跨章节同形词状态一致、两账号互不影响、篡改 owner/language/term 被拒绝、非本人章节 404、未登录 401、未就绪章节 409、v4→v5 迁移保留既有数据与检查约束。
|
||||
|
||||
本机开发库 lexgo_dev 已显式从 v4 升级到 v5：升级前后 sys_user 6、lexgo_spaces 6、lexgo_sessions 8、lexgo_books 4、lexgo_chapters 9、lexgo_ingest_jobs 9、lexgo_dictionaries 1 全部不变，新增空的 lexgo_terms。旧二进制备份在忽略的 `.local/lexgo-pre-issue7.exe`。回退：停止 lexgo-api，恢复旧二进制，把 `lexgo_schema` 标记改回 4 后启动；保留 lexgo_terms 与全部既有数据，不删除数据、不重新 bootstrap。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 30 项检查通过（凭据只从本机安全配置读入进程），覆盖两个虚构测试账号 issue5_a/issue5_b 的登录、保存、幂等、状态边界、跨章节一致、跨账号隔离与越权拒绝；随后用临时 Playwright 用例在真实学习端＋真实 API 上以两个账号复核保存、重新加载后的高亮与预填，以及 390×844 窄屏底部面板。截图保存在本机 `.local/evidence/`（issue7-reader-desktop.png、issue7-reader-mobile-390.png、issue7-account-b.png），临时用例运行后删除。
|
||||
|
||||
未验证：真实手机触屏详细证据与完整备份恢复演练仍属既有缺口（#14/#15）；本单只用桌面浏览器窄屏检查，不当作真机结果。
|
||||
|
||||
## #8 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue8，不借用其他测试库。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue8 python scripts/server.py test-integration` | 41 个顶层用例全部通过、0 跳过；含 #8 新增 6 个复习用例与 1 个 v5→v6 迁移用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 71 项单测、类型检查、构建、5 项 E2E 全部通过 |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：间隔表与每个状态转换（含等级上下限）、到期边界（`due_at = now` 到期、早 1 毫秒不到期）、入队范围（已知/忽略不入队）、两账号队列互不可见、重复提交只记一次、同一词条双标签页 second 写入为 stale、作答归属 404、已知词条 409、未知评分/缺少到期时间/未知字段 400、显式等级的排期规则、幂等键唯一约束、v5→v6 保留既有词条与计数。
|
||||
|
||||
本机开发库 lexgo_dev 已显式从 v5 升级到 v6：升级前后 sys_user 6、lexgo_spaces 6、lexgo_sessions 5、lexgo_books 5、lexgo_chapters 10、lexgo_ingest_jobs 10、lexgo_dictionaries 1、lexgo_terms 4 全部不变；新增 `lexgo_term_reviews` 4 行（全部 `due_at = created_at`）与空的 `lexgo_review_answers`。旧二进制备份在忽略的 `.local/lexgo-pre-issue8.exe`。回退：停止 lexgo-api，恢复旧二进制，把 `lexgo_schema` 标记改回 5 后启动；新表不影响旧二进制写入个人词条，保留新表与全部既有数据，不删除数据、不重新 bootstrap。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 38 项检查通过（凭据只从本机安全配置读入进程，脚本可重复运行），覆盖到期即时入队、答对升级与 1 天排期、答错立即回队、重放与 stale 不重复推进、越权与非法输入拒绝、两账号互不影响、已知词条拒绝作答；随后用临时 Playwright 用例在真实学习端＋真实 API 上完成「登录→粘贴导入→点词查义→保存个人释义→到期复习作答→回到阅读器确认状态」的完整闭环，并复核 390×844 窄屏无横向溢出。截图保存在本机 `.local/evidence/`（issue8-reader-saved.png、issue8-review-revealed.png、issue8-review-summary.png、issue8-reader-reviewed.png、issue8-review-mobile-390.png），临时用例运行后删除。
|
||||
|
||||
运维记录：本次只有 lexgo-api（新二进制）与 lexgo-learner 被重启；学习端常驻 Vite 在长时间运行后一度对 `/src/style.css` 返回空样式表，导致 E2E 观察到 `white-space: normal`，重启后恢复，未改动代码或配置。其余实例保持 Running。
|
||||
|
||||
未验证：真实手机触屏详细证据与完整备份恢复演练仍属既有缺口（#14/#15）；本单只用桌面浏览器窄屏检查，不当作真机结果。并发只覆盖到「双标签页同一词条」这一层，没有做多用户压测。
|
||||
|
||||
## #8 审核整改（R1～R3，2026-09-11）
|
||||
|
||||
独立审核（Claude Code）只读审阅提交 `0328505`／`ec5ec2d`，指出三处影响验收标准第 2、3 条的问题。三点均先在 `lexgo_test_issue8` 写复现用例观察到失败，再修复并转绿。
|
||||
|
||||
| 问题 | 现象与根因 | 修复 | 回归用例 |
|
||||
|---|---|---|---|
|
||||
| R2 并发同键返回 500 | 两个同时到达、带同一 `answerId` 的请求都在加锁前查不到记录；后者进入 stale 分支插入相同 `answer_key`，触发唯一键冲突返回 500 | 取得词条行锁后再用加锁读复查一次答案键，命中直接返回首次结果；stale 插入遇到 1062 也转为返回记录 | `TestMySQLReviewConcurrentReplayOfOneAnswer`（两个 goroutine 同键提交，两个都 2xx、`review_count = 1`、只有一条答案记录） |
|
||||
| R3 编辑文本会重排复习 | `saveTerm` 无条件调用 `syncTermReview`，编辑释义/例句也会把 `due_at` 重算，逾期词条被挤出当天队列 | 保存前加锁读取旧行，只有新建或状态/等级实际变化才移动 `due_at`；缺行时补建排期行 | `TestMySQLReviewEditKeepsSchedule`（逾期 3 级词只改释义：`due_at` 与队列不变；改等级则重排） |
|
||||
| R3 附带发现：面板保存把等级重置为 1 | 阅读器面板只提交状态不提交等级，`termLevel` 对缺省等级一律返回 1，于是 4 级词改一个错字会掉到 1 级 | 保存未提及等级时保留已获得的等级；只有进入 `学习中` 才从 1 开始 | `TestMySQLReviewPanelSaveKeepsLevel`（4 级词面板式保存后仍为 4 级，且排期不变；退出再进入学习中则从 1 开始） |
|
||||
|
||||
**答案契约随之明确**：作答响应 `result` 只取 `applied`/`stale`，另加 `duplicate` 布尔标记。重放返回首次结果并把 `duplicate` 置真，客户端因此可以按首次结果计数：网络把响应丢掉后点「重试提交」拿到 `duplicate=true` 的 `applied`，本轮计数正常增加，完成页不会退化成「今天没有到期词条」。已应用的作答返回 201，重放与 stale 返回 200（R1）。
|
||||
|
||||
**提示与注释（R4、R5）**：卡片因 `stale` 离开时页面显示 `role="status"` 提示「该词已在其他页面复习，本次未计分。」，重放且首次为 stale 时显示「该词已按上一次的评分记录，未重复计分。」；本轮只解决卡片而没有新计分时，完成页显示「本轮没有新的计分：N 个词条已在其他页面复习。」。`answerId` 的作用域注释改为「每张卡片一个，失败重试复用」，与 `answerIdFor` 的实现一致。
|
||||
|
||||
**流程记录（R6）**：评论 7769 的方案写的是在 `lexgo_terms` 上增加列，实际实现改为独立表 `lexgo_term_reviews`（加法迁移可重试、不对既有表做 ALTER）。该变更在实施评论 7776 与 Wiki 中说明了原因，但没有按「数据结构变化先更新工单」的要求在实施前追加变更评论；本页与上文契约按实际实现记录，方案评论中的「新增列均有默认值」以独立表为准。
|
||||
|
||||
整改后重跑：Go 单元与集成测试（专用库 `lexgo_test_issue8`）44 个顶层用例全部通过、0 跳过；学习端 73 项单测、类型检查、构建与 5 项 E2E 通过；管理端 31 项与 lint 通过；治理 56 项与严格检查通过；真实 API＋MySQL 42 项检查通过（新增 4 项针对 R3 与重放契约）；真实浏览器复核面板保存与复习闭环通过。截图 `.local/evidence/issue8-fixed-summary.png`。
|
||||
|
||||
## #9 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue9，不借用其他测试库。本单**不改动数据库结构**，所以没有迁移步骤，回退只需换回旧二进制。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue9 python scripts/server.py test-integration` | 50 个顶层用例全部通过、0 跳过；含 #9 新增 6 个上传用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 80 项单测、类型检查、构建、6 项 E2E 全部通过 |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：有效 UTF-8（含 CRLF、制表符、弯引号、em dash、省略号、emoji、组合字符）字节级往返、UTF-8 BOM 剥离且不进原文、只有 BOM、非法 UTF-8、Latin-1、UTF-16 大小端、NUL 字节、空文件、只有空白、超限与恰好边界（2 MiB、100000 码点）、缺 `file`、缺标题、缺或错误 `language`、缺或过短 `requestId`、未知字段、追加路径携带 `language`、非 multipart 请求、未登录、恶意文件名不影响存储、重复上传只产生一章、同编号换内容 409、追加他人书籍 404、两账号隔离、上传与粘贴共用同一任务管线。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 26 项检查通过（凭据只从本机安全配置读入进程），覆盖有效文件与阅读器原文逐字节一致、BOM 不进入原文、五类无效文件、越权与未登录拒绝、两账号隔离、恶意文件名不泄漏；随后用临时 Playwright 用例在真实学习端完成「登录→切换 TXT→选择真实 UTF-8 文件→上传→处理中就绪→阅读器原文逐字符一致」的闭环，并复核 UTF-16 文件在浏览器预检阶段被拒。截图保存在本机 `.local/evidence/`（issue9-invalid-encoding.png、issue9-upload-processing.png、issue9-reader.png），临时用例运行后删除。
|
||||
|
||||
未验证：真实手机触屏详细证据与完整备份恢复演练仍属既有缺口（#14/#15）；本单只用桌面浏览器检查。大文件并发上传只按单槽并发门设计，没有做多用户压力测试。UTF-16/GB18030 转码与按空行自动分章不在本单。
|
||||
|
||||
## #10 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue9（沿用上一单的库），不借用其他测试库。本单**不改动数据库结构**，没有迁移步骤，回退只需换回旧二进制。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue9 python scripts/server.py test-integration` | 59 个顶层用例全部通过、0 跳过；含 #10 新增 9 个用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 94 项单测、类型检查、构建、7 项 E2E 全部通过 |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：改名不改状态也不新建任务、标题/正文校验、正文变化产生新版本且保留章节编号、**处理中改正文后旧任务只标 `superseded` 且不触碰新版本**、旧版本任务重试 409、重复保存与改回原内容不新建任务、内容与版本不一致时按 `content_changed` 失败、恢复扫描作废过期版本且不重排新版本、编辑用原文对任意状态可读、删除章节后序号连续且导航正确、删除书籍级联清理章节与任务、删除期间在途任务不复活内容、并发删除同一章一个成功一个 404、重复删除 404、跨用户改名/编辑/删除/读原文一律 404、未登录 401、个人词条与复习排期在删除后完全保留。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 36 项检查通过（凭据只从本机安全配置读入进程，脚本可重复运行并自行清理 fixture），覆盖改名、编辑产生新版本并重新处理、重复保存幂等、删除章节重排序号、删除书籍级联、个人词条与复习队列在删除后保留、越权与非法输入拒绝；随后用临时 Playwright 用例在真实学习端完成「导入→改名→编辑正文→新版就绪→删除章节→删除书籍→书库提示」的闭环，并核对阅读器原文等于新版本。截图保存在本机 `.local/evidence/`（issue10-book-after-edit.png、issue10-chapter-deleted.png、issue10-book-deleted.png），临时用例运行后删除。
|
||||
|
||||
未验证：真实手机触屏详细证据与完整备份恢复演练仍属既有缺口（#14/#15）；本单只用桌面浏览器检查。并发只覆盖「同一章节并发删除」与「处理中编辑」两类，没有做多用户压力测试。浏览器 textarea 会把该章的 CRLF 归一为 LF，属已知边界，已记入业务规则页。
|
||||
|
||||
## #11 验证与迁移（2026-09-11）
|
||||
|
||||
仓库根执行；Go 工具链由 `python scripts/server.py` 固定 go1.26.5。本单使用专用测试库 lexgo_test_issue9，不借用其他测试库。**本单不新增数据库列或表**，schema 保持 v6，没有迁移步骤，回退只需换回旧二进制。
|
||||
|
||||
| 命令 | 结果 |
|
||||
|---|---|
|
||||
| `go vet ./...` | 通过 |
|
||||
| `LEXGO_TEST_DB_NAME=lexgo_test_issue9 python scripts/server.py test-integration` | 64 个顶层用例全部通过、0 跳过；含 #11 新增 5 个短语用例 |
|
||||
| `cd learner`：`npx vitest --run` / `npx vue-tsc --build` / `npx pnpm run build` / `npx playwright test` | 106 项单测、类型检查、构建、11 项 E2E 全部通过（含 #11 新增 12 单测与 4 项 E2E） |
|
||||
| `cd admin`：`npx pnpm test` / `npx pnpm lint` | 31 项与 lint 通过；管理端本单无代码改动 |
|
||||
| `python -m unittest discover -s tests` / `python dev_scripts/harness.py check --strict` | 56 项与严格检查通过 |
|
||||
|
||||
覆盖内容：身份键的标点/换行/大小写/弯撇号归一、词数上限（12 与 13）、身份键与片段长度上限、切进单词被拒绝（服务端）、单词语义不被当成短语、跨章节匹配（同一短语在不同章节写法不同仍是同一条记录，两章都高亮，第一次出现两次）、最左最长与输入顺序无关、短语覆盖内部单词而不改动单词记录、短语进同一到期队列并作答（201 应用、重复提交回放）、越权 404 与未登录 401、编辑正文后不再出现则不报错且词条与排期保留、删除章节后词条保留；前端纯函数层覆盖整词对齐、内部保留、端点按词调整不反向、命中优先级与区间换算；E2E 覆盖**真实鼠标拖选**（Chromium 真实输入）→ 短语面板 → 保存 → 两次出现同时高亮 → 端点调整 → 点击已保存短语 → 复习整段挖空。
|
||||
|
||||
真实链路验证：真实 Go API＋真实 MySQL 共 30 项检查通过（凭据只从本机安全配置读入进程，脚本自建两章 fixture 并在结束前删除书籍）。覆盖保存与身份键、显示原文保留标点、跨章节同一条记录与两处高亮、span 两端必须是词、短语内的单词仍是独立词条、单词/切词/伪造身份/越权/未登录的拒绝、到期队列与作答（含重复提交回放）、编辑正文后不再高亮但条目与排期保留、删除章节后条目保留。随后用临时 Playwright 用例在真实学习端完成「导入 → 选择范围 → 保存短语 → 两处高亮 → 点击已保存短语 → 复习整段挖空 → 清理 fixture」闭环，并用程序化选区在同一真实页面上验证范围映射与服务端身份一致。
|
||||
|
||||
未验证与已知限制：真实手机手柄与滚动的手感仍是 #4 起的既有缺口，本单只用桌面浏览器检查。**Playwright 的合成鼠标拖拽在真实页面上不会扩展原生选区**（在同一浏览器里对 mock 页面是成功的，程序化选区在真实页面也能唤起面板），因此真实链路的范围构建改用真实点击 + Shift 点击，连续拖选由 mock E2E 与程序化选区覆盖；真人鼠标拖选与真机手柄仍需人工复核。短语只在同一学习者与语言内匹配，不跨账号共享。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Product-Requirements-Overview
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Product-Requirements-Overview.-
|
||||
wiki_revision: db82eeb1d509b8d3c4a2204143cfb0e829b6b083
|
||||
synchronized_at: 2026-09-10T12:43:32Z
|
||||
wiki_revision: 45077556545aceb54ad03ba7871e8d3614fe2e01
|
||||
synchronized_at: 2026-09-14T14:21:33Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 产品需求总览
|
||||
@@ -223,3 +223,49 @@ Anki（U20）、YouTube/Jellyfin 远程字幕（U07/A08）、全量旧实例迁
|
||||
## 试用前管理能力增补
|
||||
|
||||
用户于 2026-09-10 验收 #2，并批准新增 #18 登录日志与操作审计：两个管理员列表、查询筛选分页、必要字段记录、权限及 90 天保留清理。沿用现有 go-admin 管理布局，无需独立高保真原型。已通过用户验收，纳入 #16“邀请用户试用前完成”；不替代 #3 英语分词和 #4 阅读选择验证，也不引入其他 go-admin 默认模块。
|
||||
|
||||
## 实施进度增补（#5）
|
||||
|
||||
2026-09-10：#5“粘贴英语文本，处理后进入本人章节阅读”已实现并待用户验收，覆盖 F01 基础（书库与章节）、F02（粘贴导入）、F04 基础（处理状态与失败重试）与 F05 原文（可读原文与章节切换）。本节取代此前“阅读、导入尚未实现”的表述：粘贴导入与原文阅读已实现；点词查词、词典、个人词语状态、复习与统计仍未实现（#6～#15）。schema 升级为 v3，新增 lexgo_books、lexgo_chapters、lexgo_ingest_jobs。
|
||||
|
||||
范围边界不变：本单按 Go 处理，不接入 Python NLP，因此不产生 token、lemma 或词典索引；Go＋Python NLP 与全 Go 路线仍未确认，正式接入前必须由用户确认。真机手机详细证据仍缺失（#4 缺口保持）。
|
||||
|
||||
|
||||
## 全 Go 正式架构决定（2026-09-11，#6）
|
||||
|
||||
用户已明确选择全 Go：正式英语分词、原文位置映射、本地词典解析和词形候选查询由 Go 后端完成，不运行 Python NLP 服务。此前“Python 建议/全 Go 未决”仅为历史决策记录，由本决定覆盖；spikes/english 保留历史验证，不接入产品。#6 按该方向实施，当前方案见工单最新启动评论；WordNet 3.0 仍为首个资源（英语释义），词形规则候选不等同于 spaCy 上下文消歧，原文及个人学习状态不按候选合并。
|
||||
|
||||
|
||||
## #6 交付范围更新（2026-09-11）
|
||||
|
||||
用户确认全Go后，英语词典配置与阅读器点词查义已实现待验收：共享WordNet3.0英语释义、管理员导入/启停、本人章节点击/键盘查词、加载/无结果/资源不可用/网络失败/关闭状态。词形结果是规则候选，不提供上下文词性消歧。个人释义输入是未保存临时草稿，#7才持久化。沿用已验收v1；手机底部45dvh面板自动测试通过，真机缺口保留。#5已关闭并合入main，#6尚不关闭。
|
||||
|
||||
## #7 交付范围更新（2026-09-11）
|
||||
|
||||
F07 的个人词语记录已于 2026-09-11 通过用户验收：阅读器可以保存与修改个人释义、例句和状态（新词/学习中/已知/忽略），同一词形在本人其他章节显示相同状态与高亮，两个账号的数据互不影响。个人释义与共享 WordNet 词典分开存储，个人释义不进入审计日志。schema 升级为 v5，新增 lexgo_terms。
|
||||
|
||||
仍未实现并留给后续工单：词汇库搜索与编辑（#12）、短语（#11）、到期复习与等级推进（#8）、阅读完成与进度（#13）、TXT 导入（#9）、书籍章节编辑删除（#10）。词形候选仍不提供上下文消歧，个人学习状态不按候选合并；例句不自动关联原文句子。
|
||||
|
||||
## #8 交付范围更新（2026-09-11）
|
||||
|
||||
F10 的单词到期复习已于 2026-09-11 通过用户验收（包含独立审核整改 R1～R3）：按固定间隔表取本人当前语言的到期词条，正面显示词与挖空例句，显示答案后按「认识/答对」「不认识/答错」「再学一次」评分；答对升级并排下次复习，答错降级并立即回到本轮，再学一次不改等级并回到本轮。重复提交、网络重发与双标签页都不会重复更新次数和间隔；已知与忽略的词条不入队。管理端无改动。
|
||||
|
||||
仍未实现并留给后续工单：短语复习（#11）、词汇库搜索与编辑（#12）、阅读完成与进度统计（#13）、TXT 导入（#9）、书籍章节编辑删除（#10）。复习范围筛选与策略配置（X11）、练习模式（X08）、FSRS 仍在范围外。
|
||||
|
||||
## #9 交付范围更新（2026-09-11）
|
||||
|
||||
F03 的 TXT 文件导入已于 2026-09-13 通过用户验收：学习端导入页新增「粘贴文本 / TXT 文件」来源切换，选择 UTF-8 的 .txt 文件后经大小、空文件与编码校验进入与粘贴相同的处理与阅读流程，失败可重试。只支持 UTF-8（允许可选 BOM）且不替换损坏字符；UTF-16 与其他编码会被明确拒绝；文件只在内存中解码、不写临时文件，客户端文件名不参与任何路径也不入库；重复上传同一文件只产生一章。schema 无变化。
|
||||
|
||||
仍未实现并留给后续工单：书籍与章节的编辑删除（#10）、短语选择与保存（#11）、词汇库搜索与编辑（#12）、阅读完成与进度（#13）、桌面与手机体验补齐（#14）、自托管试用交付与完整恢复（#15）。EPUB/PDF/字幕、UTF-16 转码、按空行自动分章与断点续传不在本单范围。
|
||||
|
||||
## #10 交付范围更新（2026-09-11）
|
||||
|
||||
F01 的编辑与删除已于 2026-09-14 通过用户验收：学习端可改书名、改章节标题、编辑章节正文并按新版本重新处理，可用确认弹窗删除章节或整本书。编辑正文产生明确版本，旧处理结果被标为 `superseded` 而不覆盖新版本；删除在事务内完成并重排剩余章节序号，个人词条、复习排期与作答记录一律保留。本次没有数据库结构变化。
|
||||
|
||||
仍未实现并留给后续工单：短语选择与保存（#11）、词汇库搜索与编辑（#12）、阅读完成与进度（#13）、桌面与手机体验补齐（#14）、自托管试用交付与完整恢复（#15）。封面与音频附件（#21）、回收站/撤销、批量操作、章节跨书移动与语言变更不在本单范围。
|
||||
|
||||
## #11 交付范围更新（2026-09-11）
|
||||
|
||||
F08 的短语学习与 F10 的短语复习已实现，待用户验收：阅读器支持连续选择（桌面原生拖选、手机系统手势、面板按词调整端点），保存个人释义、例句与状态；短语与单词共用同一张表、同一到期队列与同一套幂等作答，同一短语跨章节只存一条记录并在出现处高亮；复习卡片把整段短语挖成一个空。范围规则（整词对齐、内部标点保留、最多 12 词、最左最长重叠）与失效引用回退（高亮消失、学习记录保留）已固化。本次没有数据库结构变化。
|
||||
|
||||
仍未实现并留给后续工单：词汇库搜索与编辑（#12，含短语编辑界面）、阅读完成与进度（#13）、桌面与手机体验补齐（#14）、自托管试用交付与完整恢复（#15）。短语自动合并同义形式、上下文词性消歧、短语跨书移动、批量编辑与真机手柄精细手感不在本单范围。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Workload-Estimate
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Workload-Estimate.-
|
||||
wiki_revision: ba995e6788f52fc6f3471d5cf0db7300b9fbc3b4
|
||||
synchronized_at: 2026-09-10T12:43:39Z
|
||||
wiki_revision: f43e3793edfa02d319d5a1d566e082d3f3a1361d
|
||||
synchronized_at: 2026-09-10T14:52:36Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# LexGo MVP 工作量估算
|
||||
@@ -160,3 +160,13 @@ Quant-UX 桌面和手机原型 v1 已建立，待人工审核；这不计为产
|
||||
|
||||
|
||||
#18 已于 2026-09-10T20:40:04+08:00 验收。#2 与 #18 完成后，剩余 #3～#15 的规划参考为 46～75 人日；历史 49～80 人日是 #18 尚未验收时的区间。
|
||||
|
||||
|
||||
## #3 验收后的剩余计划
|
||||
|
||||
2026-09-10T21:50:14+08:00 用户验收 #3。从此前 46～75 人日扣除该单原计划 3～5 人日后，#4～#15 剩余规划参考为 43～70 人日。#2、#3、#18 均已验收；这不是实际消耗工时，后续仍需结合划词验证重估。
|
||||
|
||||
|
||||
## #4 验收与附件增补后的剩余计划
|
||||
|
||||
2026-09-10T22:51:52+08:00 用户验收 #4。原范围从43～70扣除本单3～5后，#5～#15剩余40～65人日；加已建单 #21 书籍音频/封面初估4～6后，当前剩余规划44～71人日。是计划参考，不代表实际耗时；#21仍待原型与限制细化。
|
||||
|
||||
+26
-3
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki，禁止直接编辑本文件)
|
||||
wiki_page: Home
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Home
|
||||
wiki_revision: d113bfc4d8d33feced5b84f098a5c50ba51df0ba
|
||||
synchronized_at: 2026-09-10T12:43:18Z
|
||||
wiki_revision: 08cde00c48a957705ea5b2a9d3055ff3788e64b8
|
||||
synchronized_at: 2026-09-14T14:21:32Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# LexGo 文档入口
|
||||
@@ -56,4 +56,27 @@ Quant-UX 原型 v1 已通过用户验收。[桌面预览](https://qux.ilapage.cn
|
||||
#2 已验收并关闭（记录 7576）。新增 [#18 登录日志与操作审计](https://git.ilapage.cn/OPC/lexgo/issues/18)，已通过用户验收，列入 #16 的试用前准备。管理端提供登录日志、操作日志查询，默认保留 90 天；数据库为显式迁移后的 schema v2。代码分支 feat/18-audit-logs，基于已验收的 feat/2-account-foundation；#17 尚未合并，因此新 PR 以该基础分支为目标，保持差异只含 #18。
|
||||
|
||||
|
||||
日志审计 #18 于 2026-09-10T20:40:04+08:00 获用户验收并关闭；#16 已更新完成索引。下一阶段 #3/#4 尚未开始。
|
||||
日志审计 #18 于 2026-09-10T20:40:04+08:00 获用户验收并关闭；#16 已更新完成索引。该验收时 #3/#4 尚未开始。
|
||||
|
||||
|
||||
英语分词/原文定位/本地词典 #3 独立可运行小样已通过用户验收并关闭。代码与复现命令位于 spikes/english；临时入口 http://127.0.0.1:5183/。推荐 Python spaCy 英语模型与 WordNet 3.0 的离线组合；尚未接入正式学习端。
|
||||
|
||||
|
||||
#4 阅读选择小样已实现桌面鼠标/键盘、范围调整与原文位置验证，入口 http://127.0.0.1:5184/；固定版本 LinguaCafe 四项源码对照已记录。用户已验收并关闭 #4；真实手机详细测试证据仍缺失，详见本地验证页。#21 仍待实施。
|
||||
|
||||
#5 粘贴导入与章节阅读已实现，待用户验收：schema v3 新增 books/chapters/ingest_jobs，学习端具备粘贴导入、书库、处理状态与原文阅读；查词、词典与复习仍未实现。本单按 Go 处理，Go＋Python NLP 与全 Go 路线仍未决，正式接入前须用户确认。
|
||||
|
||||
|
||||
## 当前进度（2026-09-11）
|
||||
|
||||
#5已验收，#17/#19/#20/#22/#23按依赖顺序合入main。#6已按用户确认的全Go方向实现英语词典与阅读点词，2026-09-11通过用户验收，PR #25已合入main；原文/账户隔离保留。正式NLP不使用Python服务。管理端“英语词典”导入指定WordNet3.0，学习端打开本人章节即可查词。个人释义与学习状态已实现持久化（#7，待用户验收）。详见#6工单、架构和本地开发页面。
|
||||
|
||||
#7 个人词条已实现并于 2026-09-11 通过用户验收（schema v5 新增 lexgo_terms）：阅读器可保存释义、例句与状态，同一词形在本人其他章节显示一致高亮，两个账号数据独立；保存幂等，跨账号与篡改身份均被拒绝；PR #26 已 fast-forward-only 合入 main。
|
||||
|
||||
#8 单词到期复习已于 2026-09-11 通过用户验收（schema v6 新增 lexgo_term_reviews 与 lexgo_review_answers）：固定间隔表（1/2/4/7/15/30/60 天）、答对升级、答错或再学立即回队、已知与忽略不入队；重复提交、网络重发与双标签页都只记账一次，到期判定用 UTC 绝对时刻而不引入本地日边界。学习端新增「到期复习」入口，卡片正面显示词与挖空例句、答案面显示个人释义，并有完成页与空队列页。独立审核指出的并发同键 500、编辑文本重排复习与面板保存重置等级三项已整改并复测；PR #27 已 fast-forward-only 合入 main。第 3 阶段「首条学习闭环」#5～#8 全部验收。
|
||||
|
||||
#9 TXT 文件导入已于 2026-09-13 通过用户验收：导入页新增「粘贴文本 / TXT 文件」来源切换，只接受 UTF-8（允许可选 BOM）且不替换损坏字符，UTF-16 与其他编码会被明确拒绝；文件只在内存中解码、不写临时文件，客户端文件名不参与任何路径也不入库；上传与粘贴共用同一分章、任务与幂等规则，重复上传同一文件只产生一章。本次没有数据库结构变化；PR #28 已 fast-forward-only 合入 main。
|
||||
|
||||
#10 编辑与删除书籍章节已于 2026-09-14 通过用户验收：可改书名、改章节标题、编辑章节正文并按新版本重新处理，也可用确认弹窗删除章节或整本书。编辑正文产生明确版本，旧处理结果会被标为 superseded 而不覆盖新版本；删除在事务内完成并重排剩余章节序号，已保存的个人词条、复习排期与作答记录一律保留。本次没有数据库结构变化；旧处理结果不会覆盖新版本，PR #29 已 fast-forward-only 合入 main。
|
||||
|
||||
#11 短语选择、保存与复习已实现，待用户验收：在正文中连续选择一个范围（桌面原生拖选、手机系统手柄、面板端点按钮与 Shift 点击调整），保存个人释义与状态；短语与单词共用同一张表、同一到期队列与同一套幂等作答，因此同一短语在不同章节只存一条记录并在出现的每处高亮，复习卡片把整段短语挖成一个空。短语与单词重叠时短语高亮覆盖、单词数据不变；编辑正文后短语不再出现时不报错，词条与复习排期保留。本次没有数据库结构变化。
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
import { expect, test, type Page } from '@playwright/test'
|
||||
|
||||
// Renaming, editing and deleting through the real dialogs of the accepted prototype, against
|
||||
// a mocked API.
|
||||
test('rename the book, edit a chapter into a new version and delete both', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-editor', role: 'learner' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const first = { id: 9, bookId: 1, ordinal: 1, title: 'First chapter', status: 'ready', charCount: 24, errorReason: '', errorMessage: '', jobId: 5, ...timestamps }
|
||||
const second = { id: 10, bookId: 1, ordinal: 2, title: 'Second chapter', status: 'ready', charCount: 12, errorReason: '', errorMessage: '', jobId: 6, ...timestamps }
|
||||
let bookTitle = 'A small step'
|
||||
let chapters = [first, second]
|
||||
let firstText = 'Mira opened the workshop.\n'
|
||||
let processing = false
|
||||
const requests: string[] = []
|
||||
|
||||
const listBook = () => ({
|
||||
book: { id: 1, title: bookTitle, language: 'en' },
|
||||
chapters: chapters.map(item => (item.id === 9 && processing ? { ...item, status: 'processing' } : item)),
|
||||
})
|
||||
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
requests.push(`${method} ${path}`)
|
||||
let data: unknown = null
|
||||
let status = 200
|
||||
if (path === '/api/v1/login') data = { token: 'fictional-session', user }
|
||||
else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books' && method === 'GET') {
|
||||
data = { items: bookTitle === '' ? [] : [{ id: 1, title: bookTitle, language: 'en', chapterCount: chapters.length, pendingCount: 0, processingCount: 0, readyCount: chapters.length, failedCount: 0, ...timestamps }] }
|
||||
} else if (path === '/api/v1/books/1' && method === 'PATCH') {
|
||||
bookTitle = (route.request().postDataJSON() as { title: string }).title
|
||||
data = { book: { id: 1, title: bookTitle, language: 'en' } }
|
||||
} else if (path === '/api/v1/books/1' && method === 'DELETE') {
|
||||
chapters = []
|
||||
bookTitle = ''
|
||||
data = { deleted: { bookId: 1, chapters: 2, remaining: 0 } }
|
||||
} else if (path === '/api/v1/books/1') data = listBook()
|
||||
else if (path === '/api/v1/chapters/9/source') data = { source: { id: 9, bookId: 1, ordinal: 1, title: first.title, text: firstText, status: 'ready', contentSha256: 'sha-a', charCount: [...firstText].length } }
|
||||
else if (path === '/api/v1/chapters/9' && method === 'PATCH') {
|
||||
const body = route.request().postDataJSON() as { title?: string; text?: string }
|
||||
const changed = body.text !== undefined && body.text !== firstText
|
||||
if (body.title !== undefined) first.title = body.title
|
||||
if (changed) {
|
||||
firstText = body.text as string
|
||||
processing = true
|
||||
first.status = 'pending'
|
||||
setTimeout(() => { processing = false; first.status = 'ready' }, 400)
|
||||
}
|
||||
status = 200
|
||||
data = { chapter: first, job: changed ? { id: 7, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps } : null, versionChanged: changed }
|
||||
} else if (path === '/api/v1/chapters/9' && method === 'DELETE') {
|
||||
chapters = chapters.filter(item => item.id !== 9).map((item, index) => ({ ...item, ordinal: index + 1 }))
|
||||
data = { deleted: { chapterId: 9, bookId: 1, remaining: chapters.length } }
|
||||
}
|
||||
await route.fulfill({ status, json: { code: 200, data } })
|
||||
})
|
||||
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
await page.getByRole('link', { name: 'A small step' }).click()
|
||||
await expect(page.getByRole('heading', { name: 'A small step' })).toBeVisible()
|
||||
await expect(page.getByText('封面使用系统默认样式')).toBeVisible()
|
||||
|
||||
// Rename the book through the dialog.
|
||||
await page.getByTestId('edit-book').click()
|
||||
await expect(page.getByLabel('书名', { exact: true })).toHaveValue('A small step')
|
||||
await page.getByLabel('书名', { exact: true }).fill('A long step')
|
||||
await page.getByTestId('save-book').click()
|
||||
await expect(page.getByTestId('book-notice')).toContainText('书名已更新')
|
||||
await expect(page.getByRole('heading', { name: 'A long step' })).toBeVisible()
|
||||
|
||||
// Edit the chapter text: the new version re-processes and later becomes ready again.
|
||||
await page.getByTestId('edit-chapter-9').click()
|
||||
await expect(page.getByLabel('章节标题', { exact: true })).toHaveValue('First chapter')
|
||||
await expect(page.getByLabel('正文', { exact: true })).toHaveValue(firstText)
|
||||
await page.getByLabel('正文', { exact: true }).fill('A replacement body.\n')
|
||||
await page.getByTestId('save-chapter').click()
|
||||
await expect(page.getByTestId('book-notice')).toContainText('已保存为新版本，正在重新处理')
|
||||
// The saved version is queued first and becomes readable again when the worker finishes.
|
||||
await expect(page.locator('.chapter-row').first()).toContainText('待处理')
|
||||
await expect(page.locator('.chapter-row').first()).toContainText('已就绪', { timeout: 10000 })
|
||||
|
||||
// Deleting a chapter asks first, then reports the remaining count.
|
||||
await page.getByTestId('edit-chapter-9').click()
|
||||
await page.getByTestId('delete-chapter').click()
|
||||
await expect(page.locator('.el-message-box__message').last()).toContainText('本章正文将被删除，已保存词条保留')
|
||||
await page.locator('.el-message-box').last().getByRole('button', { name: '取消' }).click()
|
||||
expect(requests.filter(entry => entry === 'DELETE /api/v1/chapters/9')).toHaveLength(0)
|
||||
await page.getByTestId('delete-chapter').click()
|
||||
await page.locator('.el-message-box').last().getByRole('button', { name: '确认删除章节' }).click()
|
||||
await expect(page.getByTestId('book-notice')).toContainText('章节已删除 · 剩余 1 章')
|
||||
await expect(page.locator('.chapter-row')).toHaveCount(1)
|
||||
|
||||
// Deleting the book asks first and returns to the library without it.
|
||||
await page.getByTestId('delete-book').click()
|
||||
await expect(page.locator('.el-message-box__message').last()).toContainText('已保存的生词和短语将保留')
|
||||
await page.locator('.el-message-box').last().getByRole('button', { name: '确认删除' }).click()
|
||||
await expect(page).toHaveURL(/\?deleted=\d+$/)
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
await expect(page.getByTestId('library-notice')).toContainText('书籍已删除 · 已保存的生词和短语仍保留在生词本')
|
||||
await expect(page.getByRole('link', { name: 'A long step' })).toHaveCount(0)
|
||||
})
|
||||
@@ -0,0 +1,130 @@
|
||||
import { expect, test, type Page } from '@playwright/test'
|
||||
|
||||
// Real mouse selection over the rendered chapter, against a mocked API: the interaction #4
|
||||
// verified is exercised as a continuous drag, not through a preset button.
|
||||
const original = 'Take a small step\nevery day.\n'
|
||||
const fragments: [string, 'word' | 'space' | 'punctuation'][] = [
|
||||
['Take', 'word'], [' ', 'space'], ['a', 'word'], [' ', 'space'], ['small', 'word'], [' ', 'space'],
|
||||
['step', 'word'], ['\n', 'space'], ['every', 'word'], [' ', 'space'], ['day', 'word'], ['.', 'punctuation'], ['\n', 'space'],
|
||||
]
|
||||
let cp = 0
|
||||
let utf16 = 0
|
||||
const tokens = fragments.map(([text, kind]) => {
|
||||
const token = { text, kind, start: cp, end: cp + [...text].length, startUtf16: utf16, endUtf16: utf16 + text.length }
|
||||
cp = token.end
|
||||
utf16 = token.endUtf16
|
||||
return token
|
||||
})
|
||||
const book = { id: 1, title: 'Phrase chapter', language: 'en' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const chapter = { id: 55, bookId: 1, ordinal: 1, title: 'Phrase chapter', status: 'ready', charCount: [...original].length, errorReason: '', errorMessage: '', jobId: 5, contentSha256: 'fictional-sha', originalText: original, ...timestamps }
|
||||
|
||||
async function openChapter(page: Page, options: { phrases?: unknown[]; onPhrase?: (body: Record<string, unknown>) => void } = {}) {
|
||||
const user = { id: 42, username: 'fictional-phrase', role: 'learner' }
|
||||
await page.addInitScript(() => sessionStorage.setItem('lexgo-learner-token', 'fictional-session'))
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
let data: unknown = null
|
||||
let status = 200
|
||||
if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books') data = { items: [] }
|
||||
else if (path === '/api/v1/chapters/55/tokens') data = { textSha256: 'fictional-sha', tokens, phrases: options.phrases ?? [] }
|
||||
else if (path === '/api/v1/chapters/55') data = { book, chapter, navigation: { previousChapterId: null, nextChapterId: null } }
|
||||
else if (path === '/api/v1/lookup') data = { status: 'not_found', query: 'Take', matchedForm: null, candidates: [], entries: [] }
|
||||
else if (path === '/api/v1/phrases' && method === 'POST') {
|
||||
options.onPhrase?.(route.request().postDataJSON() as Record<string, unknown>)
|
||||
status = 201
|
||||
data = {
|
||||
term: { id: 9, term: 'take a small', originalForm: 'Take a small', definition: '拿一小步', examples: [], status: 'new', level: 0, kind: 'phrase', wordCount: 3 },
|
||||
created: true,
|
||||
}
|
||||
} else if (path === '/api/v1/terms/9') data = { term: { id: 9, term: 'take a small', originalForm: 'Take a small', definition: '拿一小步', examples: ['Take a small step.'], status: 'new', level: 0, kind: 'phrase', wordCount: 3 } }
|
||||
await route.fulfill({ status, json: { code: 200, data } })
|
||||
})
|
||||
await page.goto('/chapters/55')
|
||||
await expect(page.locator('.reader-text')).toBeVisible()
|
||||
}
|
||||
|
||||
/** Drags from the centre of one word to the centre of another, like a person selecting text. */
|
||||
async function dragWords(page: Page, from: string, to: string) {
|
||||
const words = page.locator('.reader-word')
|
||||
const start = words.filter({ hasText: new RegExp(`^${from}$`) }).first()
|
||||
const end = words.filter({ hasText: new RegExp(`^${to}$`) }).first()
|
||||
const startBox = (await start.boundingBox())!
|
||||
const endBox = (await end.boundingBox())!
|
||||
await page.mouse.move(startBox.x + startBox.width / 2, startBox.y + startBox.height / 2)
|
||||
await page.mouse.down()
|
||||
await page.mouse.move(endBox.x + endBox.width / 2, endBox.y + endBox.height / 2, { steps: 8 })
|
||||
await page.mouse.up()
|
||||
}
|
||||
|
||||
test('drag a continuous phrase, save it and highlight it', async ({ page }) => {
|
||||
let body: Record<string, unknown> | undefined
|
||||
await openChapter(page, { onPhrase: value => { body = value } })
|
||||
|
||||
await dragWords(page, 'Take', 'small')
|
||||
const range = page.getByTestId('phrase-range')
|
||||
await expect(range).toContainText('短语 · 3 个单词')
|
||||
// A phrase that is not stored yet is labelled as a new entry.
|
||||
await expect(page.locator('.lookup-panel')).toContainText('新词条')
|
||||
await page.getByLabel(/^我的释义/).fill('拿一小步')
|
||||
await page.screenshot({ path: '../.local/evidence/issue11-phrase-selected.png' })
|
||||
await page.getByTestId('term-save').click()
|
||||
await expect(page.getByText('已保存 · 新词')).toBeVisible()
|
||||
// The code point range covers the whole run including its interior separator.
|
||||
expect(body).toMatchObject({ chapterId: 55, start: 0, end: 12, definition: '拿一小步', status: 'new' })
|
||||
// The saved phrase is underlined in the text.
|
||||
await expect(page.locator('.reader-word.is-phrase')).toHaveCount(3)
|
||||
})
|
||||
|
||||
test('adjust the endpoints by whole words from the panel', async ({ page }) => {
|
||||
await openChapter(page)
|
||||
await dragWords(page, 'Take', 'small')
|
||||
await expect(page.getByTestId('phrase-range')).toContainText('3 个单词')
|
||||
await page.getByTestId('range-end-right').click()
|
||||
await expect(page.getByTestId('phrase-range')).toContainText('4 个单词')
|
||||
await page.getByTestId('range-start-right').click()
|
||||
await expect(page.getByTestId('phrase-range')).toContainText('3 个单词')
|
||||
await page.getByTestId('range-start-left').click()
|
||||
await expect(page.getByTestId('phrase-range')).toContainText('4 个单词')
|
||||
})
|
||||
|
||||
test('open a saved phrase from its highlight', async ({ page }) => {
|
||||
await openChapter(page, { phrases: [{ id: 9, status: 'new', wordCount: 3, startToken: 0, endToken: 4 }] })
|
||||
await expect(page.locator('.reader-word.is-phrase')).toHaveCount(3)
|
||||
await page.locator('.reader-word.is-phrase').first().click()
|
||||
await expect(page.getByTestId('phrase-range')).toContainText('已保存')
|
||||
await expect(page.getByLabel(/^我的释义/)).toHaveValue('拿一小步')
|
||||
await page.screenshot({ path: '../.local/evidence/issue11-phrase-saved.png' })
|
||||
})
|
||||
|
||||
test('a phrase review card masks the whole run as one blank', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-phrase', role: 'learner' }
|
||||
await page.addInitScript(() => sessionStorage.setItem('lexgo-learner-token', 'fictional-session'))
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
let data: unknown = null
|
||||
if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/reviews/queue') {
|
||||
data = {
|
||||
items: [{
|
||||
id: 9, term: 'a small step', originalForm: 'a small step', definition: '一小步',
|
||||
examples: ['Take a small step, every day.'], status: 'new', level: 0, kind: 'phrase', wordCount: 3,
|
||||
dueAt: '2026-01-01T00:00:00Z', reviewCount: 0,
|
||||
}],
|
||||
total: 1,
|
||||
}
|
||||
}
|
||||
await route.fulfill({ json: { code: 200, data } })
|
||||
})
|
||||
await page.goto('/review')
|
||||
await expect(page.getByTestId('review-position')).toContainText('到期复习 · 1 / 1')
|
||||
await expect(page.getByText('Take _____, every day.')).toBeVisible()
|
||||
await expect(page.getByText('短语 · 3 个单词')).toBeVisible()
|
||||
await page.getByTestId('review-reveal').click()
|
||||
await expect(page.getByTestId('review-definition')).toHaveText('一小步')
|
||||
await page.screenshot({ path: '../.local/evidence/issue11-phrase-review.png' })
|
||||
})
|
||||
@@ -0,0 +1,157 @@
|
||||
import { expect, test } from '@playwright/test'
|
||||
|
||||
test('paste English text, watch a chapter finish processing, then read it verbatim', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-reader', role: 'learner' }
|
||||
const book = { id: 1, title: '虚构样例书', language: 'en' }
|
||||
const chapterTitle = '虚构样例第一章'
|
||||
// Line breaks, a tab and repeated spaces must survive the whole round trip.
|
||||
const pasted = 'First line of the chapter.\n\tIndented line.\nTwo spaces kept.\n\nLast line.\n'
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
// The worker reports the fresh chapter as processing until the worker settles it.
|
||||
let status: 'processing' | 'ready' = 'processing'
|
||||
// The personal record the learner saves during this run, served back on reload.
|
||||
let savedTerm: { id: number; term: string; originalForm: string; definition: string; examples: string[]; status: string; level: number } | null = null
|
||||
const chapterPayload = () => ({
|
||||
id: 55,
|
||||
bookId: book.id,
|
||||
ordinal: 1,
|
||||
title: chapterTitle,
|
||||
status,
|
||||
charCount: [...pasted].length,
|
||||
errorReason: '',
|
||||
errorMessage: '',
|
||||
jobId: 7,
|
||||
...timestamps,
|
||||
})
|
||||
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
let data: unknown = null
|
||||
let statusCode = 200
|
||||
if (path === '/api/v1/login') {
|
||||
expect(route.request().postDataJSON()).toEqual({ username: user.username, password: 'fictional-password' })
|
||||
data = { token: 'fictional-session', user }
|
||||
} else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books' && method === 'GET') {
|
||||
data = { items: [{ ...book, chapterCount: 0, pendingCount: 0, processingCount: 0, readyCount: 0, failedCount: 0, ...timestamps }] }
|
||||
} else if (path === '/api/v1/books' && method === 'POST') {
|
||||
const body = route.request().postDataJSON() as { requestId: string; title: string; text: string; language: string }
|
||||
expect(body.requestId).toMatch(/^[0-9a-f-]{36}$/)
|
||||
expect(body).toMatchObject({ title: chapterTitle, text: pasted, language: 'en' })
|
||||
statusCode = 201
|
||||
data = {
|
||||
book,
|
||||
chapter: chapterPayload(),
|
||||
job: { id: 7, bookId: book.id, chapterId: 55, status, attempts: 0, errorReason: '', errorMessage: '', ...timestamps },
|
||||
duplicate: false,
|
||||
}
|
||||
} else if (path === '/api/v1/books/1') data = { book, chapters: [chapterPayload()] }
|
||||
else if (path === '/api/v1/chapters/55') {
|
||||
data = {
|
||||
book,
|
||||
chapter: { ...chapterPayload(), contentSha256: 'fictional-sha256', ...(status === 'ready' ? { originalText: pasted } : {}) },
|
||||
navigation: { previousChapterId: null, nextChapterId: null },
|
||||
}
|
||||
} else if (path === '/api/v1/chapters/55/tokens') {
|
||||
let offset = 0
|
||||
const tokens = (pasted.match(/[A-Za-z]+|\s+|[^A-Za-z\s]+/g) ?? []).map(text => {
|
||||
const start = offset
|
||||
offset += text.length
|
||||
return { text, start, end: offset, startUtf16: start, endUtf16: offset, kind: /^[A-Za-z]+$/.test(text) ? 'word' : /^\s+$/.test(text) ? 'space' : 'punctuation' }
|
||||
})
|
||||
data = {
|
||||
textSha256: 'fictional-sha256',
|
||||
tokens: tokens.map(token => savedTerm && token.text === 'First'
|
||||
? { ...token, term: { id: savedTerm.id, status: savedTerm.status, level: savedTerm.level } }
|
||||
: token),
|
||||
}
|
||||
} else if (path === '/api/v1/terms' && method === 'POST') {
|
||||
const body = route.request().postDataJSON() as { chapterId: number; start: number; end: number; definition: string; examples: string[]; status: string }
|
||||
expect(body).toEqual({ chapterId: 55, start: 0, end: 5, definition: '虚构的个人释义', examples: ['A fictional example.'], status: 'new' })
|
||||
savedTerm = { id: 9, term: 'first', originalForm: 'First', definition: body.definition, examples: body.examples, status: body.status, level: 0 }
|
||||
statusCode = 201
|
||||
data = { term: savedTerm, created: true }
|
||||
} else if (path === '/api/v1/terms/9') data = { term: savedTerm }
|
||||
else if (path === '/api/v1/lookup') {
|
||||
expect(route.request().postDataJSON()).toEqual({ chapterId: 55, start: 0, end: 5 })
|
||||
data = { status: 'exact', query: 'first', matchedForm: 'first', candidates: [], entries: [{ lemma: 'first', pos: 'adjective', definition: 'Coming before all others.', examples: ['The first fictional chapter.'] }] }
|
||||
}
|
||||
await route.fulfill({ status: statusCode, json: { code: 200, data } })
|
||||
})
|
||||
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
|
||||
// The library lists the caller's book.
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
await expect(page.getByRole('link', { name: book.title })).toBeVisible()
|
||||
|
||||
// Paste text through the import form.
|
||||
await page.getByRole('button', { name: '导入内容' }).click()
|
||||
await expect(page.getByRole('heading', { name: '导入英文内容' })).toBeVisible()
|
||||
await page.getByLabel('标题').fill(chapterTitle)
|
||||
await page.getByLabel('正文').fill(pasted)
|
||||
await page.getByRole('button', { name: '开始处理' }).click()
|
||||
|
||||
// The new book opens with the chapter still processing…
|
||||
await expect(page).toHaveURL(/\/books\/1$/)
|
||||
await expect(page.getByText('处理中')).toBeVisible()
|
||||
|
||||
// …and the browser poll turns it ready without a page reload.
|
||||
status = 'ready'
|
||||
await expect(page.getByText('已就绪')).toBeVisible({ timeout: 15000 })
|
||||
|
||||
// Open the chapter and check the pasted text survived verbatim.
|
||||
await page.getByRole('link', { name: chapterTitle }).click()
|
||||
await expect(page).toHaveURL(/\/chapters\/55$/)
|
||||
const readerText = page.locator('.reader-text')
|
||||
await expect(readerText).toBeVisible()
|
||||
expect(await readerText.evaluate(element => element.textContent)).toBe(pasted)
|
||||
expect(await readerText.evaluate(element => getComputedStyle(element).whiteSpace)).toBe('pre-wrap')
|
||||
await expect(page.getByRole('button', { name: '上一章' })).toBeDisabled()
|
||||
await expect(page.getByRole('button', { name: '下一章' })).toBeDisabled()
|
||||
const word = page.locator('.reader-word').first()
|
||||
await expect(word).toHaveAttribute('aria-label', '查询 First')
|
||||
await word.focus()
|
||||
await word.press('Enter')
|
||||
await expect(page.getByText('Coming before all others.')).toBeVisible()
|
||||
expect(await readerText.evaluate(element => element.textContent)).toBe(pasted)
|
||||
|
||||
// Browser narrow viewport check only; this is not real-device acceptance.
|
||||
await page.setViewportSize({ width: 390, height: 844 })
|
||||
await word.click()
|
||||
await expect(page.getByText('Coming before all others.')).toBeVisible()
|
||||
const panelBounds = await page.locator('.lookup-panel').boundingBox()
|
||||
expect(panelBounds!.y + panelBounds!.height).toBeLessThanOrEqual(845)
|
||||
expect(panelBounds!.height).toBeLessThanOrEqual(844 * 0.46)
|
||||
expect(await page.locator('.lookup-panel').evaluate(element => getComputedStyle(element).position)).toBe('fixed')
|
||||
const wordBounds = await word.boundingBox()
|
||||
expect(wordBounds!.y + wordBounds!.height).toBeLessThanOrEqual(panelBounds!.y)
|
||||
expect(await page.getByLabel('我的释义 新词条').inputValue()).toBe('')
|
||||
|
||||
// Save a personal record: definition, example and the default status.
|
||||
await page.setViewportSize({ width: 1280, height: 900 })
|
||||
await page.getByLabel('我的释义 新词条').fill('虚构的个人释义')
|
||||
await page.getByLabel('例句 每行一条，最多 5 条').fill('A fictional example.')
|
||||
await page.getByRole('button', { name: '保存到生词本' }).click()
|
||||
await expect(page.getByText('已保存 · 新词')).toBeVisible()
|
||||
await expect(word).toHaveClass(/is-new/)
|
||||
expect(await readerText.evaluate(element => element.textContent)).toBe(pasted)
|
||||
await page.getByRole('button', { name: '关闭释义' }).press('Escape')
|
||||
await expect(page.locator('.lookup-panel')).toHaveCount(0)
|
||||
await expect(word).toBeFocused()
|
||||
|
||||
// Returning to the chapter shows the same state and the stored text.
|
||||
await page.reload()
|
||||
const reloadedWord = page.locator('.reader-word').first()
|
||||
await expect(reloadedWord).toHaveAttribute('aria-label', '查询 First，已保存')
|
||||
await expect(reloadedWord).toHaveClass(/is-new/)
|
||||
await reloadedWord.click()
|
||||
await expect(page.getByLabel('我的释义 已保存')).toHaveValue('虚构的个人释义')
|
||||
await expect(page.getByLabel('例句 每行一条，最多 5 条')).toHaveValue('A fictional example.')
|
||||
await expect(page.getByRole('radio', { name: '新词' })).toBeChecked()
|
||||
})
|
||||
@@ -0,0 +1,120 @@
|
||||
import { expect, test } from '@playwright/test'
|
||||
|
||||
// The review round against a mocked API: queue, reveal, grades, relearn and the summary.
|
||||
test('review the due words, requeue a missed one and finish the round', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-reviewer', role: 'learner' }
|
||||
const first = {
|
||||
id: 7, term: 'curiosity', originalForm: 'curiosity', definition: '好奇心；求知欲',
|
||||
examples: ['Learning begins with curiosity.'], status: 'new', level: 0,
|
||||
dueAt: '2026-09-11T10:00:00Z', reviewCount: 0,
|
||||
}
|
||||
const second = {
|
||||
id: 8, term: 'step', originalForm: 'step', definition: '一步',
|
||||
examples: ['Take a small step, every day.'], status: 'learning', level: 2,
|
||||
dueAt: '2026-09-11T10:00:00Z', reviewCount: 4,
|
||||
}
|
||||
// The wrong answer puts its word back in the same round, exactly as the server does.
|
||||
const submitted: { grade: string; answerId: string }[] = []
|
||||
let requeued = false
|
||||
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
let data: unknown = null
|
||||
let status = 200
|
||||
if (path === '/api/v1/login') data = { token: 'fictional-session', user }
|
||||
else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books') data = { items: [] }
|
||||
else if (path === '/api/v1/reviews/queue') {
|
||||
data = { items: requeued ? [second] : [first, second], total: requeued ? 1 : 2 }
|
||||
} else if (path.endsWith('/answers') && method === 'POST') {
|
||||
const body = route.request().postDataJSON() as { answerId: string; grade: string; expectedDueAt: string }
|
||||
submitted.push({ grade: body.grade, answerId: body.answerId })
|
||||
expect(body.answerId).toMatch(/^[0-9a-f-]{36}$/)
|
||||
const term = path.includes('8') ? second : first
|
||||
// The client answers the card it was shown, so it echoes that card's due time.
|
||||
expect(body.expectedDueAt).toBe(term.dueAt)
|
||||
const wrong = body.grade === 'wrong'
|
||||
requeued = wrong
|
||||
const dueAtAfter = wrong ? '2026-09-11T10:05:00Z' : '2026-09-12T10:00:00Z'
|
||||
term.dueAt = dueAtAfter
|
||||
status = 201
|
||||
data = {
|
||||
result: 'applied', grade: body.grade, requeued: wrong,
|
||||
statusBefore: term.status, statusAfter: wrong ? term.status : 'learning',
|
||||
levelBefore: term.level, levelAfter: wrong ? 1 : term.level + 1,
|
||||
dueAtBefore: body.expectedDueAt, dueAtAfter,
|
||||
item: { ...term, dueAt: dueAtAfter, reviewCount: term.reviewCount + 1 },
|
||||
}
|
||||
}
|
||||
await route.fulfill({ status, json: { code: 200, data } })
|
||||
})
|
||||
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
|
||||
// The library links into the due queue.
|
||||
await page.getByRole('link', { name: '到期复习' }).click()
|
||||
await expect(page).toHaveURL(/\/review$/)
|
||||
await expect(page.getByTestId('review-position')).toHaveText('到期复习 · 1 / 2')
|
||||
await expect(page.getByRole('heading', { name: 'curiosity' })).toBeVisible()
|
||||
// The answer stays hidden, and the example shows a blank instead of the word.
|
||||
await expect(page.getByText('Learning begins with _____.')).toBeVisible()
|
||||
await expect(page.getByTestId('review-definition')).toHaveCount(0)
|
||||
|
||||
await page.getByTestId('review-reveal').click()
|
||||
await expect(page.getByTestId('review-definition')).toHaveText('好奇心；求知欲')
|
||||
await expect(page.getByTestId('review-correct')).toBeFocused()
|
||||
await page.getByTestId('review-correct').click()
|
||||
|
||||
// The second word is learning level 2 and can be sent back into the round.
|
||||
await expect(page.getByTestId('review-position')).toHaveText('到期复习 · 2 / 2')
|
||||
await expect(page.getByText('学习中 · 等级 2')).toBeVisible()
|
||||
await page.getByTestId('review-reveal').click()
|
||||
await expect(page.getByTestId('review-definition')).toHaveText('一步')
|
||||
await page.getByTestId('review-wrong').click()
|
||||
|
||||
// Requeued: the same word asks again and the round grows instead of pretending it ended.
|
||||
await expect(page.getByTestId('review-position')).toHaveText('到期复习 · 3 / 3')
|
||||
await expect(page.getByTestId('review-definition')).toHaveCount(0)
|
||||
await page.getByTestId('review-reveal').click()
|
||||
await page.getByTestId('review-correct').click()
|
||||
|
||||
await expect(page.getByTestId('review-summary')).toContainText('复习了 2 个词条 · 共 3 次作答')
|
||||
await expect(page.getByTestId('review-summary')).toContainText('答对 2 · 答错或再学 1')
|
||||
expect(submitted.map(entry => entry.grade)).toEqual(['correct', 'wrong', 'correct'])
|
||||
expect(new Set(submitted.map(entry => entry.answerId)).size).toBe(3)
|
||||
|
||||
await page.getByTestId('review-finish').click()
|
||||
await expect(page).toHaveURL(/\/$/)
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
})
|
||||
|
||||
test('an empty due queue says so instead of showing a card', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-reviewer', role: 'learner' }
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
let data: unknown = null
|
||||
if (path === '/api/v1/login') data = { token: 'fictional-session', user }
|
||||
else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books') data = { items: [] }
|
||||
else if (path === '/api/v1/reviews/queue') data = { items: [], total: 0 }
|
||||
await route.fulfill({ json: { code: 200, data } })
|
||||
})
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
await page.goto('/review')
|
||||
await expect(page.getByTestId('review-empty')).toContainText('今天没有到期词条')
|
||||
await expect(page.getByTestId('review-card')).toHaveCount(0)
|
||||
// The narrow layout keeps the grading controls reachable without horizontal overflow.
|
||||
await page.setViewportSize({ width: 390, height: 844 })
|
||||
const overflow = await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth)
|
||||
expect(overflow).toBeLessThanOrEqual(0)
|
||||
})
|
||||
@@ -0,0 +1,76 @@
|
||||
import { expect, test } from '@playwright/test'
|
||||
|
||||
// The TXT upload path against a mocked API: pre-checks, multipart body and the hand-off to
|
||||
// the same processing screen the paste path uses.
|
||||
test('upload a UTF-8 TXT file and open the created book', async ({ page }) => {
|
||||
const user = { id: 42, username: 'fictional-uploader', role: 'learner' }
|
||||
const book = { id: 1, title: 'Studio Notes', language: 'en' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const pasted = 'Mira opened the workshop.\r\n\r\n\tThe sign read “A small step…”\n'
|
||||
let uploaded = false
|
||||
|
||||
await page.route('**/api/v1/**', async route => {
|
||||
const path = new URL(route.request().url()).pathname
|
||||
const method = route.request().method()
|
||||
let data: unknown = null
|
||||
let status = 200
|
||||
if (path === '/api/v1/login') data = { token: 'fictional-session', user }
|
||||
else if (path === '/api/v1/me') data = user
|
||||
else if (path === '/api/v1/space') data = { ownerId: user.id, language: 'en' }
|
||||
else if (path === '/api/v1/books' && method === 'GET') data = { items: uploaded ? [{ ...book, chapterCount: 1, pendingCount: 0, processingCount: 0, readyCount: 1, failedCount: 0, ...timestamps }] : [] }
|
||||
else if (path === '/api/v1/books/upload') {
|
||||
// The upload must arrive as multipart: the fields and the file are inspected directly.
|
||||
expect(route.request().headers()['content-type']).toContain('multipart/form-data')
|
||||
const raw = route.request().postData() ?? ''
|
||||
expect(raw).toContain('name="requestId"')
|
||||
expect(raw).toContain('name="language"')
|
||||
expect(raw).toContain('name="title"')
|
||||
expect(raw).toContain('Studio Notes')
|
||||
expect(raw).toContain('filename="notes.txt"')
|
||||
expect(raw).toContain('Mira opened the workshop.')
|
||||
uploaded = true
|
||||
status = 201
|
||||
data = {
|
||||
book,
|
||||
chapter: { id: 9, bookId: 1, ordinal: 1, title: 'Studio Notes', status: 'pending', charCount: [...pasted].length, errorReason: '', errorMessage: '', jobId: 5, ...timestamps },
|
||||
job: { id: 5, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps },
|
||||
duplicate: false,
|
||||
}
|
||||
} else if (path === '/api/v1/books/1') data = { book, chapters: [{ id: 9, bookId: 1, ordinal: 1, title: 'Studio Notes', status: 'ready', charCount: [...pasted].length, errorReason: '', errorMessage: '', jobId: 5, ...timestamps }] }
|
||||
else if (path === '/api/v1/chapters/9') data = { book, chapter: { id: 9, bookId: 1, ordinal: 1, title: 'Studio Notes', status: 'ready', charCount: [...pasted].length, errorReason: '', errorMessage: '', jobId: 5, contentSha256: 'fictional-sha', originalText: pasted, ...timestamps }, navigation: { previousChapterId: null, nextChapterId: null } }
|
||||
await route.fulfill({ status, json: { code: 200, data } })
|
||||
})
|
||||
|
||||
await page.goto('/')
|
||||
await page.getByLabel('账号').fill(user.username)
|
||||
await page.getByLabel('密码', { exact: true }).fill('fictional-password')
|
||||
await page.getByRole('button', { name: '登录', exact: true }).click()
|
||||
await expect(page.getByRole('heading', { name: '我的书库' })).toBeVisible()
|
||||
|
||||
await page.getByRole('button', { name: '导入内容' }).click()
|
||||
// Element Plus hides the native radio behind a styled span, so the label is what a person
|
||||
// clicks; the input still carries the checked state.
|
||||
await page.locator('label.el-radio', { hasText: 'TXT 文件' }).click()
|
||||
await expect(page.getByRole('radio', { name: 'TXT 文件' })).toBeChecked()
|
||||
await expect(page.locator('textarea#text')).toHaveCount(0)
|
||||
await expect(page.getByText('仅支持 UTF-8')).toBeVisible()
|
||||
|
||||
// An unusable file is refused in the browser, before any request is made.
|
||||
await page.setInputFiles('[data-testid="file-input"]', { name: 'notes.md', mimeType: 'text/markdown', buffer: Buffer.from('# heading\n') })
|
||||
await expect(page.getByText('请选择 .txt 文件。')).toBeVisible()
|
||||
|
||||
// A UTF-8 file is accepted and its metadata is shown; the title comes from the file name.
|
||||
await page.setInputFiles('[data-testid="file-input"]', { name: 'notes.txt', mimeType: 'text/plain', buffer: Buffer.from(pasted, 'utf8') })
|
||||
await expect(page.getByTestId('file-info')).toContainText('notes.txt · UTF-8')
|
||||
await expect(page.getByLabel('标题')).toHaveValue('notes')
|
||||
await page.getByLabel('标题').fill('Studio Notes')
|
||||
|
||||
await page.getByRole('button', { name: '上传并处理' }).click()
|
||||
await expect(page).toHaveURL(/\/books\/1$/)
|
||||
await expect(page.getByText('已就绪')).toBeVisible()
|
||||
await page.getByRole('link', { name: 'Studio Notes' }).click()
|
||||
const readerText = page.locator('.reader-text')
|
||||
await expect(readerText).toBeVisible()
|
||||
// The uploaded bytes reached the reader unchanged.
|
||||
expect(await readerText.evaluate(element => element.textContent)).toBe(pasted)
|
||||
})
|
||||
@@ -8,6 +8,9 @@ export default defineConfig({
|
||||
forbidOnly: !!process.env.CI,
|
||||
retries: 0,
|
||||
reporter: 'list',
|
||||
// Every spec loads the whole SPA from the shared dev server, so a slow first module graph
|
||||
// under parallel workers must not fail an assertion that the app itself would pass.
|
||||
expect: { timeout: 15000 },
|
||||
use: { baseURL: 'http://127.0.0.1:5173', headless: true, trace: 'off' },
|
||||
projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'], channel: 'chrome' } }],
|
||||
webServer: { command: 'npm run dev -- --host 127.0.0.1', url: 'http://127.0.0.1:5173', reuseExistingServer: !process.env.CI },
|
||||
|
||||
@@ -0,0 +1,237 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createRouter, createMemoryHistory, type Router } from 'vue-router'
|
||||
import { ElMessageBox } from 'element-plus'
|
||||
import BookView from '../views/BookView.vue'
|
||||
import { useLibraryStore, type ChapterSource, type ChapterSummary } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const user = { id: 42, username: 'fictional-editor', role: 'learner' as const }
|
||||
const book = { id: 1, title: 'A small step', language: 'en' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const chapter = (overrides: Partial<ChapterSummary> = {}): ChapterSummary => ({
|
||||
id: 9, bookId: 1, ordinal: 1, title: 'First chapter', status: 'ready', charCount: 12,
|
||||
errorReason: '', errorMessage: '', jobId: 5, ...timestamps, ...overrides,
|
||||
})
|
||||
const source: ChapterSource = { id: 9, bookId: 1, ordinal: 1, title: 'First chapter', text: 'Mira opened the workshop.\n', status: 'ready', contentSha256: 'sha-a', charCount: 24 }
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
const fail = (msg: string, status = 400) => new Response(JSON.stringify({ code: status, msg }), { status })
|
||||
let wrapper: VueWrapper | undefined
|
||||
|
||||
function stub(name: string) {
|
||||
return { template: `<div>${name}</div>` }
|
||||
}
|
||||
|
||||
async function viewAt(path: string): Promise<Router> {
|
||||
const router = createRouter({
|
||||
history: createMemoryHistory(),
|
||||
routes: [
|
||||
{ path: '/', component: stub('LibraryStub') },
|
||||
{ path: '/books/:id', component: stub('BookStub') },
|
||||
{ path: '/import', component: stub('ImportStub') },
|
||||
],
|
||||
})
|
||||
await router.push(path)
|
||||
await router.isReady()
|
||||
return router
|
||||
}
|
||||
|
||||
/** Routes the book page's own calls; a test can override any of them. */
|
||||
function mockApi(overrides: Record<string, (init?: RequestInit) => Response | Promise<Response>> = {}): MockInstance {
|
||||
return vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
const url = String(input)
|
||||
const method = String((init as RequestInit | undefined)?.method ?? 'GET')
|
||||
for (const [key, handler] of Object.entries(overrides)) {
|
||||
if (url.includes(key)) return handler(init as RequestInit)
|
||||
}
|
||||
if (url.endsWith('/books/1') && method === 'GET') return ok({ book, chapters: [chapter()] })
|
||||
if (url.includes('/chapters/9/source')) return ok({ source })
|
||||
// A rename answers with the stored book, and a delete reports what it removed.
|
||||
if (url.endsWith('/books/1') && method === 'PATCH') {
|
||||
const body = JSON.parse(String((init as RequestInit | undefined)?.body ?? '{}')) as { title?: string }
|
||||
return ok({ book: { ...book, title: body.title ?? book.title } })
|
||||
}
|
||||
if (url.endsWith('/books/1') && method === 'DELETE') return ok({ deleted: { bookId: 1, chapters: 3 } })
|
||||
if (url.endsWith('/books/1')) return ok({ book })
|
||||
if (url.includes('/chapters/9')) return ok({ chapter: chapter(), job: null, versionChanged: false })
|
||||
return ok({})
|
||||
})
|
||||
}
|
||||
|
||||
async function openBook() {
|
||||
useSessionStore().user = { ...user }
|
||||
const router = await viewAt('/books/1')
|
||||
wrapper = mount(BookView, { attachTo: document.body, global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
return { view: wrapper, router }
|
||||
}
|
||||
|
||||
describe('book editing store', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { vi.restoreAllMocks() })
|
||||
|
||||
it('renames a book and keeps the list entry in step', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const library = useLibraryStore()
|
||||
library.books = [{ ...book, chapterCount: 1, pendingCount: 0, processingCount: 0, readyCount: 1, failedCount: 0, ...timestamps }]
|
||||
library.book = { ...book }
|
||||
const renamed = await library.renameBook(1, ' New name ')
|
||||
expect(renamed.title).toBe('New name')
|
||||
expect(library.book?.title).toBe('New name')
|
||||
expect(library.books[0]?.title).toBe('New name')
|
||||
const call = fetchMock.mock.calls.find(([, init]) => (init as RequestInit | undefined)?.method === 'PATCH')!
|
||||
expect(String(call[0])).toBe('/api/v1/books/1')
|
||||
expect(JSON.parse(String((call[1] as RequestInit).body))).toEqual({ title: 'New name' })
|
||||
})
|
||||
|
||||
it('refuses an invalid title before calling the API', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const library = useLibraryStore()
|
||||
await expect(library.renameBook(1, ' ')).rejects.toThrow('请填写标题')
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('applies a new chapter version and drops the stale reader text', async () => {
|
||||
mockApi({ '/chapters/9': () => ok({ chapter: chapter({ status: 'pending', title: 'Edited' }), job: { id: 6, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps }, versionChanged: true }) })
|
||||
const library = useLibraryStore()
|
||||
library.chapters = [chapter()]
|
||||
library.chapter = { ...chapter(), contentSha256: 'sha-a', originalText: 'old text' }
|
||||
const result = await library.updateChapter(9, { title: 'Edited', text: 'new text' })
|
||||
expect(result.versionChanged).toBe(true)
|
||||
expect(library.chapters[0]).toMatchObject({ status: 'pending', title: 'Edited', jobId: 6 })
|
||||
// The reader must not keep showing text that is no longer the stored version.
|
||||
expect(library.chapter?.originalText).toBeUndefined()
|
||||
})
|
||||
|
||||
it('keeps the reader text when only the title changed', async () => {
|
||||
mockApi({ '/chapters/9': () => ok({ chapter: chapter({ title: 'Renamed' }), job: null, versionChanged: false }) })
|
||||
const library = useLibraryStore()
|
||||
library.chapters = [chapter()]
|
||||
library.chapter = { ...chapter(), contentSha256: 'sha-a', originalText: 'kept text' }
|
||||
await library.updateChapter(9, { title: 'Renamed' })
|
||||
expect(library.chapter?.originalText).toBe('kept text')
|
||||
expect(library.chapter?.title).toBe('Renamed')
|
||||
})
|
||||
|
||||
it('deletes a chapter, reloads the book and removes a deleted book from the list', async () => {
|
||||
mockApi({
|
||||
'/chapters/9': () => ok({ deleted: { chapterId: 9, bookId: 1, remaining: 2 } }),
|
||||
'/books/1': (init) => (init?.method === 'DELETE'
|
||||
? ok({ deleted: { bookId: 1, chapters: 3 } })
|
||||
: ok({ book, chapters: [] })),
|
||||
})
|
||||
const library = useLibraryStore()
|
||||
library.books = [{ ...book, chapterCount: 3, pendingCount: 0, processingCount: 0, readyCount: 3, failedCount: 0, ...timestamps }]
|
||||
library.book = { ...book }
|
||||
library.chapters = [chapter(), chapter({ id: 10, ordinal: 2 }), chapter({ id: 11, ordinal: 3 })]
|
||||
const deleted = await library.deleteChapter(9)
|
||||
expect(deleted.remaining).toBe(2)
|
||||
expect(library.chapters.map(item => item.id)).toEqual([])
|
||||
expect(library.book?.id).toBe(1)
|
||||
const removed = await library.deleteBook(1)
|
||||
expect(removed.chapters).toBe(3)
|
||||
expect(library.books).toHaveLength(0)
|
||||
expect(library.book).toBeNull()
|
||||
})
|
||||
|
||||
it('reads the editable source of any chapter state', async () => {
|
||||
mockApi({ '/chapters/9/source': () => ok({ source: { ...source, status: 'failed' } }) })
|
||||
const library = useLibraryStore()
|
||||
const loaded = await library.loadChapterSource(9)
|
||||
expect(loaded.text).toBe(source.text)
|
||||
expect(loaded.status).toBe('failed')
|
||||
})
|
||||
})
|
||||
|
||||
describe('book editing view', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear(); vi.restoreAllMocks() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; useLibraryStore().stopPolling() })
|
||||
|
||||
it('renames the book through the dialog and reports it', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-book"]').trigger('click'); await flushPromises()
|
||||
const input = view.get('#book-title')
|
||||
expect((input.element as HTMLInputElement).value).toBe('A small step')
|
||||
await input.setValue(' Edited name ')
|
||||
await view.get('[data-testid="save-book"]').trigger('click'); await flushPromises()
|
||||
const patch = fetchMock.mock.calls.find(([, init]) => (init as RequestInit | undefined)?.method === 'PATCH')!
|
||||
expect(JSON.parse(String((patch[1] as RequestInit).body))).toEqual({ title: 'Edited name' })
|
||||
expect(view.get('[data-testid="book-notice"]').text()).toContain('书名已更新')
|
||||
})
|
||||
|
||||
it('keeps the dialog open with the server message when renaming fails', async () => {
|
||||
mockApi({ '/books/1': (init) => (init?.method === 'PATCH' ? fail('书名已存在') : ok({ book, chapters: [chapter()] })) })
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-book"]').trigger('click'); await flushPromises()
|
||||
await view.get('#book-title').setValue('Rejected')
|
||||
await view.get('[data-testid="save-book"]').trigger('click'); await flushPromises()
|
||||
expect(view.text()).toContain('书名已存在')
|
||||
expect(view.get('#book-title')).toBeTruthy()
|
||||
})
|
||||
|
||||
it('asks before deleting the book and does nothing when the learner cancels', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const confirm = vi.spyOn(ElMessageBox, 'confirm').mockRejectedValue('cancel')
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="delete-book"]').trigger('click'); await flushPromises()
|
||||
expect(confirm).toHaveBeenCalledWith(expect.stringContaining('已保存的生词和短语将保留'), '删除书籍', expect.anything())
|
||||
expect(fetchMock.mock.calls.some(([, init]) => (init as RequestInit | undefined)?.method === 'DELETE')).toBe(false)
|
||||
})
|
||||
|
||||
it('deletes the book after confirmation and returns to the library', async () => {
|
||||
const fetchMock = mockApi()
|
||||
vi.spyOn(ElMessageBox, 'confirm').mockResolvedValue('confirm' as never)
|
||||
const { view, router } = await openBook()
|
||||
await view.get('[data-testid="delete-book"]').trigger('click'); await flushPromises()
|
||||
expect(fetchMock.mock.calls.some(([url, init]) => String(url).endsWith('/books/1') && (init as RequestInit | undefined)?.method === 'DELETE')).toBe(true)
|
||||
expect(router.currentRoute.value.path).toBe('/')
|
||||
expect(router.currentRoute.value.query.deleted).toBe('3')
|
||||
})
|
||||
|
||||
it('edits a chapter: loads the source, saves title and text and reports the new version', async () => {
|
||||
const fetchMock = mockApi({ '/chapters/9': (init) => (init?.method === 'PATCH'
|
||||
? ok({ chapter: chapter({ status: 'pending', title: 'Edited chapter' }), job: { id: 6, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps }, versionChanged: true })
|
||||
: ok({ source })) })
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-chapter-9"]').trigger('click'); await flushPromises()
|
||||
expect((view.get('#chapter-title').element as HTMLInputElement).value).toBe('First chapter')
|
||||
expect((view.get('#chapter-text').element as HTMLTextAreaElement).value).toBe(source.text)
|
||||
await view.get('#chapter-title').setValue('Edited chapter')
|
||||
await view.get('#chapter-text').setValue('A new body.\n')
|
||||
await view.get('[data-testid="save-chapter"]').trigger('click'); await flushPromises()
|
||||
const patch = fetchMock.mock.calls.find(([url, init]) => String(url).endsWith('/chapters/9') && (init as RequestInit | undefined)?.method === 'PATCH')!
|
||||
expect(JSON.parse(String((patch[1] as RequestInit).body))).toEqual({ title: 'Edited chapter', text: 'A new body.\n' })
|
||||
expect(view.get('[data-testid="book-notice"]').text()).toContain('已保存为新版本，正在重新处理')
|
||||
})
|
||||
|
||||
it('refuses an empty chapter body locally and keeps the dialog', async () => {
|
||||
const fetchMock = mockApi()
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-chapter-9"]').trigger('click'); await flushPromises()
|
||||
await view.get('#chapter-text').setValue(' \n\t ')
|
||||
await view.get('[data-testid="save-chapter"]').trigger('click'); await flushPromises()
|
||||
expect(view.text()).toContain('请粘贴要导入的英文正文。')
|
||||
expect(fetchMock.mock.calls.some(([, init]) => (init as RequestInit | undefined)?.method === 'PATCH')).toBe(false)
|
||||
})
|
||||
|
||||
it('deletes a chapter after confirmation and reports the remaining count', async () => {
|
||||
const fetchMock = mockApi({ '/chapters/9': (init) => (init?.method === 'DELETE' ? ok({ deleted: { chapterId: 9, bookId: 1, remaining: 2 } }) : ok({ source })) })
|
||||
vi.spyOn(ElMessageBox, 'confirm').mockResolvedValue('confirm' as never)
|
||||
const { view } = await openBook()
|
||||
await view.get('[data-testid="edit-chapter-9"]').trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="delete-chapter"]').trigger('click'); await flushPromises()
|
||||
expect(fetchMock.mock.calls.some(([url, init]) => String(url).endsWith('/chapters/9') && (init as RequestInit | undefined)?.method === 'DELETE')).toBe(true)
|
||||
expect(view.get('[data-testid="book-notice"]').text()).toContain('章节已删除 · 剩余 2 章')
|
||||
// Element Plus keeps a closed dialog in the DOM, so the closed state is what matters.
|
||||
expect(view.get('[data-testid="chapter-dialog"]').isVisible()).toBe(false)
|
||||
})
|
||||
|
||||
it('shows an empty chapter list with the import hint', async () => {
|
||||
mockApi({ '/books/1': () => ok({ book, chapters: [] }) })
|
||||
const { view } = await openBook()
|
||||
expect(view.get('[data-testid="empty-chapters"]').text()).toContain('这一本书还没有章节')
|
||||
expect(view.get('[data-testid="delete-book"]')).toBeTruthy()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,445 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import {
|
||||
NOT_FOUND_MESSAGE,
|
||||
POLL_INTERVAL_MS,
|
||||
TEXT_MAX_CODE_POINTS,
|
||||
canRetry,
|
||||
statusSummary,
|
||||
useLibraryStore,
|
||||
type ChapterDetail,
|
||||
} from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
// All accounts, books and texts in these tests are deliberately fictitious.
|
||||
const user = { id: 7, username: 'fictional-reader', role: 'learner' as const }
|
||||
const book = { id: 1, title: '虚构样例书', language: 'en' }
|
||||
const navigation = { previousChapterId: null, nextChapterId: null }
|
||||
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }), { status: 200 })
|
||||
const created = (data: unknown) => new Response(JSON.stringify({ code: 200, data }), { status: 201 })
|
||||
const httpError = (status: number, msg: string) => new Response(JSON.stringify({ code: status, msg }), { status })
|
||||
|
||||
const chapter = (overrides: Partial<ChapterDetail> = {}): ChapterDetail => ({
|
||||
id: 55,
|
||||
bookId: 1,
|
||||
ordinal: 1,
|
||||
title: '第一篇',
|
||||
status: 'pending',
|
||||
charCount: 120,
|
||||
errorReason: '',
|
||||
errorMessage: '',
|
||||
contentSha256: 'fictional-sha256',
|
||||
jobId: 7,
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
updatedAt: '2026-01-01T00:00:00Z',
|
||||
...overrides,
|
||||
})
|
||||
|
||||
const summary = (overrides: Partial<Record<string, number | string>> = {}) => ({
|
||||
...book,
|
||||
chapterCount: 0,
|
||||
pendingCount: 0,
|
||||
processingCount: 0,
|
||||
readyCount: 0,
|
||||
failedCount: 0,
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
updatedAt: '2026-01-01T00:00:00Z',
|
||||
...overrides,
|
||||
})
|
||||
|
||||
const job = { id: 7, bookId: 1, chapterId: 55, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
|
||||
const fetchMock = () => vi.mocked(globalThis.fetch)
|
||||
const paths = () => fetchMock().mock.calls.map(([input]) => String(input))
|
||||
const bodyOf = (index: number): Record<string, unknown> => JSON.parse(String(fetchMock().mock.calls[index]?.[1]?.body)) as Record<string, unknown>
|
||||
|
||||
async function signIn() {
|
||||
fetchMock().mockResolvedValueOnce(ok({ token: 'fictional-token', expiresAt: '2030-01-01', user }))
|
||||
const session = useSessionStore()
|
||||
await session.login(user.username, 'fictional-password')
|
||||
return session
|
||||
}
|
||||
|
||||
describe('learner library store', () => {
|
||||
beforeEach(() => {
|
||||
sessionStorage.clear()
|
||||
setActivePinia(createPinia())
|
||||
vi.restoreAllMocks()
|
||||
// Any request a test did not expect fails loudly instead of hanging.
|
||||
vi.spyOn(globalThis, 'fetch').mockImplementation(input => {
|
||||
throw new Error(`unexpected request: ${String(input)}`)
|
||||
})
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
// Never let a polling timer outlive its test.
|
||||
useLibraryStore().stopPolling()
|
||||
vi.useRealTimers()
|
||||
})
|
||||
|
||||
it('loads the library and summarises the count fields the API reports', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ items: [summary({ chapterCount: 5, readyCount: 2, processingCount: 1, pendingCount: 1, failedCount: 1 })] }))
|
||||
|
||||
await library.loadBooks()
|
||||
|
||||
expect(library.books).toHaveLength(1)
|
||||
expect(library.booksLoading).toBe(false)
|
||||
expect(library.booksError).toBe('')
|
||||
expect(statusSummary(library.books[0]!)).toBe('已就绪 2 · 处理中 1 · 待处理 1 · 失败 1')
|
||||
expect(paths()).toEqual(['/api/v1/login', '/api/v1/books'])
|
||||
expect(fetchMock().mock.calls[1]?.[1]?.headers).toMatchObject({ Authorization: 'Bearer fictional-token' })
|
||||
})
|
||||
|
||||
it('shows pending and processing separately instead of deriving one from a total', async () => {
|
||||
// processingCount is strictly "processing" now, so pendingCount must be read as given.
|
||||
const queued = summary({ chapterCount: 3, readyCount: 1, processingCount: 0, pendingCount: 2, failedCount: 0 })
|
||||
expect(statusSummary(queued)).toBe('已就绪 1 · 待处理 2')
|
||||
expect(statusSummary(queued)).not.toContain('处理中')
|
||||
const done = summary({ chapterCount: 1, readyCount: 1 })
|
||||
expect(statusSummary(done)).toBe('已就绪 1')
|
||||
expect(statusSummary(summary({ chapterCount: 0 }))).toBe('')
|
||||
})
|
||||
|
||||
it('loads a book detail whose chapters carry the job id used for retry', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapters: [chapter({ status: 'failed', jobId: 7 }), chapter({ id: 56, status: 'failed', jobId: null })] }))
|
||||
|
||||
await library.loadBook(1)
|
||||
|
||||
expect(library.book?.title).toBe('虚构样例书')
|
||||
expect(library.chapters).toHaveLength(2)
|
||||
expect(paths()[1]).toBe('/api/v1/books/1')
|
||||
// The chapter itself carries the job id, even for a freshly loaded book.
|
||||
expect(library.chapters[0]?.jobId).toBe(7)
|
||||
expect(canRetry(library.chapters[0]!)).toBe(true)
|
||||
// A null job id means the chapter has nothing to retry yet.
|
||||
expect(canRetry(library.chapters[1]!)).toBe(false)
|
||||
})
|
||||
|
||||
it('retries a failed chapter loaded fresh from the book detail, without any submit in this session', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
// No submit() call: this is a plain reload, the old workaround would hide retry here.
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapters: [chapter({ status: 'failed', errorMessage: '无法解析正文。', jobId: 7 })] }))
|
||||
await library.loadBook(1)
|
||||
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(ok({ job: { ...job, status: 'pending', attempts: 1 }, chapter: { id: 55, bookId: 1, jobId: 7 } }))
|
||||
.mockResolvedValueOnce(ok({ book, chapters: [chapter({ jobId: 7 })] }))
|
||||
await library.retryChapter(55)
|
||||
|
||||
expect(paths()).toContain('/api/v1/jobs/7/retry')
|
||||
expect(library.retryingChapterId).toBeNull()
|
||||
expect(library.chapters[0]?.status).toBe('pending')
|
||||
})
|
||||
|
||||
it('pastes a new book and reads the job id from the created chapter', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(created({ book, chapter: chapter(), job, duplicate: false }))
|
||||
|
||||
const bookId = await library.submit({ title: ' 第一篇 ', text: 'Hello world.\nSecond line.', target: { mode: 'new' } })
|
||||
|
||||
expect(bookId).toBe(1)
|
||||
expect(library.submitting).toBe(false)
|
||||
expect(library.submitError).toBe('')
|
||||
expect(paths()[1]).toBe('/api/v1/books')
|
||||
expect(bodyOf(1)).toEqual({ requestId: expect.any(String), title: '第一篇', text: 'Hello world.\nSecond line.', language: 'en' })
|
||||
})
|
||||
|
||||
it('appends to an existing book through the chapter endpoint', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(created({ chapter: chapter(), job, duplicate: false }))
|
||||
|
||||
const bookId = await library.submit({ title: '第二篇', text: 'Another text.', target: { mode: 'append', bookId: 1 } })
|
||||
|
||||
expect(bookId).toBe(1)
|
||||
expect(paths()[1]).toBe('/api/v1/books/1/chapters')
|
||||
})
|
||||
|
||||
it('reuses one requestId while the same unsent content keeps failing', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
const input = { title: '第一篇', text: 'Hello world.', target: { mode: 'new' as const } }
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(httpError(500, '服务器开小差了'))
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter(), job, duplicate: false }))
|
||||
|
||||
await expect(library.submit(input)).rejects.toThrow('服务器开小差了')
|
||||
expect(library.submitError).toBe('服务器开小差了')
|
||||
await library.submit(input)
|
||||
|
||||
// One chapter, not two: the retry of unchanged content reuses the requestId.
|
||||
expect(bodyOf(2).requestId).toBe(bodyOf(1).requestId)
|
||||
})
|
||||
|
||||
it('uses a fresh requestId after a successful submit and after the content changes', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter(), job, duplicate: false }))
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter({ id: 56 }), job, duplicate: false }))
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter({ id: 57 }), job, duplicate: false }))
|
||||
|
||||
await library.submit({ title: '第一篇', text: 'Hello world.', target: { mode: 'new' } })
|
||||
await library.submit({ title: '第一篇', text: 'Hello world.', target: { mode: 'new' } })
|
||||
await library.submit({ title: '第一篇', text: 'Hello world changed.', target: { mode: 'new' } })
|
||||
|
||||
expect(bodyOf(2).requestId).not.toBe(bodyOf(1).requestId)
|
||||
expect(bodyOf(3).requestId).not.toBe(bodyOf(2).requestId)
|
||||
})
|
||||
|
||||
it('rejects invalid input before sending anything', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
const sent = paths().length
|
||||
|
||||
await expect(library.submit({ title: ' ', text: 'Hello.', target: { mode: 'new' } })).rejects.toThrow('请填写标题。')
|
||||
await expect(library.submit({ title: '标题', text: ' \n\t ', target: { mode: 'new' } })).rejects.toThrow('请粘贴要导入的英文正文。')
|
||||
await expect(library.submit({ title: 'x'.repeat(121), text: 'Hello.', target: { mode: 'new' } })).rejects.toThrow('标题不能超过 120 个字符。')
|
||||
await expect(library.submit({ title: '标题', text: 'a'.repeat(TEXT_MAX_CODE_POINTS + 1), target: { mode: 'new' } })).rejects.toThrow(`正文不能超过 ${TEXT_MAX_CODE_POINTS} 个字符。`)
|
||||
|
||||
expect(paths().length).toBe(sent)
|
||||
expect(library.submitError).toBe(`正文不能超过 ${TEXT_MAX_CODE_POINTS} 个字符。`)
|
||||
})
|
||||
|
||||
it('measures the text limit in Unicode code points', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
// 100000 astral characters are 200000 UTF-16 units but still within the limit.
|
||||
fetchMock().mockResolvedValueOnce(created({ book, chapter: chapter(), job, duplicate: false }))
|
||||
|
||||
await expect(library.submit({ title: '标题', text: '😀'.repeat(TEXT_MAX_CODE_POINTS), target: { mode: 'new' } })).resolves.toBe(1)
|
||||
expect([...('😀'.repeat(TEXT_MAX_CODE_POINTS))].length).toBe(TEXT_MAX_CODE_POINTS)
|
||||
})
|
||||
|
||||
it('polls a pending chapter until it is ready and then stops', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter(), navigation }))
|
||||
|
||||
await library.loadChapter(55)
|
||||
expect(library.chapter?.status).toBe('pending')
|
||||
expect(library.readerText).toBe('')
|
||||
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Hello\nworld.' }), navigation }))
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS)
|
||||
|
||||
expect(library.chapter?.status).toBe('ready')
|
||||
expect(library.readerText).toBe('Hello\nworld.')
|
||||
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 3)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
it('retries a failed chapter through the chapter job id and resumes polling', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'failed', errorReason: 'decode_error', errorMessage: '无法解析正文。', jobId: 7 }), navigation }))
|
||||
|
||||
await library.loadChapter(55)
|
||||
expect(library.chapter?.status).toBe('failed')
|
||||
expect(canRetry(library.chapter!)).toBe(true)
|
||||
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(ok({ job: { ...job, status: 'pending', attempts: 1 }, chapter: { id: 55, bookId: 1, jobId: 7 } }))
|
||||
.mockResolvedValueOnce(ok({ book, chapter: chapter(), navigation }))
|
||||
await library.retryChapter(55)
|
||||
|
||||
expect(paths()).toContain('/api/v1/jobs/7/retry')
|
||||
expect(library.retryingChapterId).toBeNull()
|
||||
expect(library.chapter?.status).toBe('pending')
|
||||
|
||||
// The retry restarts polling for the chapter it re-queued.
|
||||
const before = paths().length
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Hello world.' }), navigation }))
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS)
|
||||
expect(paths().length).toBe(before + 1)
|
||||
expect(library.readerText).toBe('Hello world.')
|
||||
})
|
||||
|
||||
it('refuses to retry a chapter whose job id is null', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'failed', jobId: null }), navigation }))
|
||||
await library.loadChapter(55)
|
||||
|
||||
expect(canRetry(library.chapter!)).toBe(false)
|
||||
await expect(library.retryChapter(55)).rejects.toThrow('这一章暂时没有可重试的任务编号。')
|
||||
expect(paths()).toEqual(['/api/v1/login', '/api/v1/chapters/55'])
|
||||
})
|
||||
|
||||
it('reports another account id as 内容不存在 and stops polling for it', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(httpError(404, 'chapter not found'))
|
||||
|
||||
await library.loadChapter(99)
|
||||
expect(library.chapter).toBeNull()
|
||||
expect(library.chapterError).toBe(NOT_FOUND_MESSAGE)
|
||||
|
||||
fetchMock().mockResolvedValueOnce(httpError(404, 'book not found'))
|
||||
await library.loadBook(99)
|
||||
expect(library.book).toBeNull()
|
||||
expect(library.bookError).toBe(NOT_FOUND_MESSAGE)
|
||||
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 4)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
it('stops polling and ignores a late response once the session is cleared', async () => {
|
||||
vi.useFakeTimers()
|
||||
const session = await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter(), navigation }))
|
||||
await library.loadChapter(55)
|
||||
expect(library.chapter).not.toBeNull()
|
||||
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock().mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
fetchMock().mockResolvedValueOnce(ok(null))
|
||||
const late = library.loadChapter(55, { silent: true })
|
||||
const logout = session.logout()
|
||||
|
||||
finish(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Late text.' }), navigation }))
|
||||
await late
|
||||
await logout
|
||||
|
||||
expect(session.user).toBeNull()
|
||||
expect(library.chapter).toBeNull()
|
||||
expect(library.readerText).toBe('')
|
||||
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 4)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
it('keeps newer state when an older request answers later', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock().mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
fetchMock().mockResolvedValueOnce(ok({ items: [summary({ title: '较新的标题' })] }))
|
||||
|
||||
const stale = library.loadBooks()
|
||||
await library.loadBooks()
|
||||
finish(ok({ items: [summary({ title: '过期的标题' })] }))
|
||||
await stale
|
||||
|
||||
expect(library.books).toHaveLength(1)
|
||||
expect(library.books[0]?.title).toBe('较新的标题')
|
||||
})
|
||||
|
||||
// Regression R1: the append contract has no language field and the server rejects unknown
|
||||
// fields, so a client that sent one could never append.
|
||||
it('sends the language only when creating a book, never when appending', async () => {
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(created({ book, chapter: chapter({ id: 54 }) }))
|
||||
await library.submit({ title: ' 新书 ', text: 'New book text.', target: { mode: 'new' } })
|
||||
expect(bodyOf(1)).toEqual({ requestId: expect.any(String), title: '新书', text: 'New book text.', language: 'en' })
|
||||
|
||||
fetchMock().mockResolvedValueOnce(created({ chapter: chapter({ id: 55, ordinal: 2 }) }))
|
||||
await library.submit({ title: '第二篇', text: 'Appended text.', target: { mode: 'append', bookId: 1 } })
|
||||
expect(String(fetchMock().mock.calls[2]?.[0])).toBe('/api/v1/books/1/chapters')
|
||||
expect(bodyOf(2)).toEqual({ requestId: expect.any(String), title: '第二篇', text: 'Appended text.' })
|
||||
expect(bodyOf(2)).not.toHaveProperty('language')
|
||||
})
|
||||
|
||||
// Regression R3: leaving a view must invalidate its in-flight request.
|
||||
it('ignores a book response that arrives after the book view was closed', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock().mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
|
||||
const pending = library.loadBook(1)
|
||||
library.closeBook()
|
||||
finish(ok({ book, chapters: [chapter()] }))
|
||||
await pending
|
||||
|
||||
expect(library.book).toBeNull()
|
||||
expect(library.chapters).toEqual([])
|
||||
expect(library.bookLoading).toBe(false)
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 3)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
it('ignores a chapter response that arrives after the reader was closed', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock().mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
|
||||
const pending = library.loadChapter(55)
|
||||
library.closeChapter()
|
||||
finish(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Late text.' }), navigation }))
|
||||
await pending
|
||||
|
||||
expect(library.chapter).toBeNull()
|
||||
expect(library.readerText).toBe('')
|
||||
expect(library.chapterLoading).toBe(false)
|
||||
const settled = paths().length
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS * 3)
|
||||
expect(paths().length).toBe(settled)
|
||||
})
|
||||
|
||||
// Regression R4: an accepted retry must be visible and tracked even if the refresh fails.
|
||||
it('keeps tracking a retried chapter when the first refresh fails', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'failed', errorReason: 'content_changed', errorMessage: '内容在处理前发生变化。', jobId: 7 }), navigation }))
|
||||
await library.loadChapter(55)
|
||||
expect(library.chapter?.status).toBe('failed')
|
||||
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(ok({ job: { ...job, status: 'pending' }, chapter: chapter({ status: 'pending' }) }))
|
||||
.mockRejectedValueOnce(new Error('network down'))
|
||||
await library.retryChapter(55)
|
||||
|
||||
expect(paths()).toContain('/api/v1/jobs/7/retry')
|
||||
expect(library.chapter?.status).toBe('pending')
|
||||
expect(library.readerText).toBe('')
|
||||
|
||||
// The next poll still tracks the queued chapter and shows the final result.
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapter: chapter({ status: 'ready', originalText: 'Recovered text.' }), navigation }))
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS)
|
||||
expect(library.chapter?.status).toBe('ready')
|
||||
expect(library.readerText).toBe('Recovered text.')
|
||||
})
|
||||
|
||||
it('keeps tracking a retried chapter from the book page when the first refresh fails', async () => {
|
||||
vi.useFakeTimers()
|
||||
await signIn()
|
||||
const library = useLibraryStore()
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapters: [chapter({ status: 'failed', errorReason: 'content_changed', errorMessage: '内容在处理前发生变化。' })] }))
|
||||
await library.loadBook(1)
|
||||
expect(library.chapters[0]?.status).toBe('failed')
|
||||
|
||||
fetchMock()
|
||||
.mockResolvedValueOnce(ok({ job: { ...job, status: 'pending' }, chapter: chapter({ status: 'pending' }) }))
|
||||
.mockRejectedValueOnce(new Error('network down'))
|
||||
await library.retryChapter(55)
|
||||
|
||||
expect(library.chapters[0]?.status).toBe('pending')
|
||||
|
||||
fetchMock().mockResolvedValueOnce(ok({ book, chapters: [chapter({ status: 'ready' })] }))
|
||||
await vi.advanceTimersByTimeAsync(POLL_INTERVAL_MS)
|
||||
expect(library.chapters[0]?.status).toBe('ready')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,253 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createMemoryHistory, createRouter } from 'vue-router'
|
||||
import ReaderView from '../views/ReaderView.vue'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
import { useReaderLookup, type SavedTerm } from '../composables/useReaderLookup'
|
||||
import { effectScope, ref } from 'vue'
|
||||
import type { ChapterDetail } from '../stores/library'
|
||||
|
||||
// Fictitious text includes astral, combining, CRLF and repeated whitespace.
|
||||
const original = '😀 Cats\r\n café!'
|
||||
const fragments = [['😀', 'punctuation'], [' ', 'space'], ['Cats', 'word'], ['\r\n ', 'space'], ['café', 'word'], ['!', 'punctuation']]
|
||||
let cp = 0, utf16 = 0
|
||||
const tokens = fragments.map(([text = '', kind]) => {
|
||||
const token = { text, kind, start: cp, end: cp + [...text].length, startUtf16: utf16, endUtf16: utf16 + text.length }
|
||||
cp = token.end; utf16 = token.endUtf16
|
||||
return token
|
||||
})
|
||||
const chapter = { id: 55, bookId: 1, title: '虚构章节', status: 'ready', originalText: original, contentSha256: 'same-sha' }
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
const result = (query: string, status = 'exact') => ({ status, query, matchedForm: query, candidates: [], entries: status === 'exact' ? [{ lemma: query, pos: 'noun', definition: `Definition of ${query}`, examples: ['A fictional example.'] }] : [] })
|
||||
const savedTerm = (overrides: Partial<SavedTerm> = {}): SavedTerm => ({
|
||||
id: 7, term: 'cats', originalForm: 'Cats', definition: '猫', examples: ['A fictional example.'], status: 'learning', level: 2,
|
||||
kind: 'word', wordCount: 1, ...overrides,
|
||||
})
|
||||
let wrapper: VueWrapper | undefined
|
||||
interface OpenOptions {
|
||||
lookup?: (body: Record<string, number>) => Promise<Response>
|
||||
tokenData?: unknown
|
||||
termRead?: () => Promise<Response>
|
||||
termWrite?: (body: Record<string, unknown>) => Promise<Response>
|
||||
}
|
||||
async function open(options: OpenOptions = {}) {
|
||||
const { lookup = async () => ok(result('cat')), tokenData = { textSha256: 'same-sha', tokens } } = options
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
const url = String(input)
|
||||
if (url.endsWith('/tokens')) return ok(tokenData)
|
||||
if (url.endsWith('/lookup')) return lookup(JSON.parse(String(init?.body)))
|
||||
if (url.includes('/terms/')) return options.termRead ? options.termRead() : ok({ term: savedTerm() })
|
||||
if (url.endsWith('/terms')) return options.termWrite ? options.termWrite(JSON.parse(String(init?.body))) : ok({ term: savedTerm(), created: true })
|
||||
return ok({ chapter, navigation: { previousChapterId: null, nextChapterId: 56 } })
|
||||
})
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
const router = createRouter({ history: createMemoryHistory(), routes: [{ path: '/chapters/:id', component: ReaderView }, { path: '/', component: { template: '<div />' } }] })
|
||||
await router.push('/chapters/55')
|
||||
wrapper = mount(ReaderView, { attachTo: document.body, global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
return { fetchMock, router, view: wrapper }
|
||||
}
|
||||
const callsTo = (fetchMock: MockInstance, suffix: string) => fetchMock.mock.calls.filter(call => String(call[0]).endsWith(suffix))
|
||||
// Personal state is attached to the word itself, never to a space or punctuation token.
|
||||
const withTerm = (text: string, term: { id: number; status: string; level: number }) => ({
|
||||
textSha256: 'same-sha',
|
||||
tokens: tokens.map(token => token.text === text ? { ...token, term } : token),
|
||||
})
|
||||
describe('reader word lookup', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; vi.restoreAllMocks() })
|
||||
it('keeps Unicode original text exact and sends only chapter and code-point offsets', async () => {
|
||||
const { view, fetchMock } = await open()
|
||||
expect(view.get('.reader-text').element.textContent).toBe(original)
|
||||
const words = view.findAll('.reader-word')
|
||||
expect(words).toHaveLength(2)
|
||||
await words[0]!.trigger('click'); await flushPromises()
|
||||
const call = callsTo(fetchMock, '/lookup')[0]!
|
||||
expect(call[1]?.method).toBe('POST')
|
||||
expect(JSON.parse(String(call[1]?.body))).toEqual({ chapterId: 55, start: 2, end: 6 })
|
||||
expect(view.text()).toContain('Definition of cat')
|
||||
expect(view.text()).toContain('A fictional example.')
|
||||
})
|
||||
it('discards older word responses and clears the selection form', async () => {
|
||||
let finish!: (response: Response) => void
|
||||
const { view } = await open({ lookup: body => body.start === 2 ? new Promise(resolve => { finish = resolve }) : Promise.resolve(ok(result('cafe', 'not_found'))) })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click')
|
||||
expect(view.text()).toContain('正在查询')
|
||||
await view.findAll('.reader-word')[1]!.trigger('click'); await flushPromises()
|
||||
await view.get('#term-definition').setValue('虚构私人草稿')
|
||||
finish(ok(result('OLD'))); await flushPromises()
|
||||
expect(view.text()).not.toContain('Definition of OLD')
|
||||
expect(view.text()).toContain('未找到释义')
|
||||
expect(view.text()).toContain('新词条')
|
||||
await view.findAll('.reader-word')[0]!.trigger('click')
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('')
|
||||
})
|
||||
it.each(['resource_missing', 'not_found', 'error'])('keeps reading and permits a saved word and retry for %s', async status => {
|
||||
const { view, fetchMock } = await open({ lookup: async () => { if (status === 'error') throw new Error('暂时无法查询'); return ok(result('cat', status)) } })
|
||||
await view.get('.reader-word').trigger('click'); await flushPromises()
|
||||
await view.get('#term-definition').setValue('仅当前词的虚构释义')
|
||||
expect(view.get('.reader-text').element.textContent).toBe(original)
|
||||
expect(view.get('[data-testid="term-save"]').attributes('disabled')).toBeUndefined()
|
||||
expect(callsTo(fetchMock, '/lookup')).toHaveLength(1)
|
||||
await view.get('[data-testid="lookup-retry"]').trigger('click'); await flushPromises()
|
||||
expect(callsTo(fetchMock, '/lookup')).toHaveLength(2)
|
||||
})
|
||||
it.each([{ textSha256: 'wrong', tokens }, { textSha256: 'same-sha', tokens: tokens.slice(1) }])('falls back to original when tokens do not match', async data => {
|
||||
const { view } = await open({ tokenData: data })
|
||||
expect(view.get('.reader-text').element.textContent).toBe(original)
|
||||
expect(view.find('.reader-word').exists()).toBe(false)
|
||||
expect(view.find('[data-testid="tokens-retry"]').exists()).toBe(true)
|
||||
})
|
||||
it('closes with Escape and restores the word focus without scrolling', async () => {
|
||||
const { view } = await open()
|
||||
const word = view.get('.reader-word').element as HTMLElement
|
||||
word.focus()
|
||||
await view.get('.reader-word').trigger('keydown', { key: 'Enter' }); await flushPromises()
|
||||
const focus = vi.spyOn(word, 'focus')
|
||||
await view.get('.lookup-panel').trigger('keydown', { key: 'Escape' })
|
||||
await flushPromises()
|
||||
expect(view.find('.lookup-panel').exists()).toBe(false)
|
||||
expect(document.activeElement).toBe(word)
|
||||
expect(focus).toHaveBeenCalledWith({ preventScroll: true })
|
||||
})
|
||||
it('retries a failed token fetch without losing original text', async () => {
|
||||
const { view, fetchMock } = await open()
|
||||
fetchMock.mockRejectedValueOnce(new Error('网络暂不可用'))
|
||||
// A new chapter load invalidates the old token rendering.
|
||||
const { useLibraryStore } = await import('../stores/library')
|
||||
useLibraryStore().chapter = { ...useLibraryStore().chapter!, contentSha256: 'retry-sha' }
|
||||
await flushPromises()
|
||||
expect(view.get('.reader-text').element.textContent).toBe(original)
|
||||
expect(view.find('.reader-word').exists()).toBe(false)
|
||||
fetchMock.mockResolvedValueOnce(ok({ textSha256: 'retry-sha', tokens }))
|
||||
await view.get('[data-testid="tokens-retry"]').trigger('click'); await flushPromises()
|
||||
expect(view.findAll('.reader-word')).toHaveLength(2)
|
||||
})
|
||||
it('does not accept a late token response after identity changes', async () => {
|
||||
const { view, fetchMock } = await open({ tokenData: { textSha256: 'wrong', tokens } })
|
||||
let finish!: (response: Response) => void
|
||||
fetchMock.mockImplementationOnce(() => new Promise(resolve => { finish = resolve }))
|
||||
await view.get('[data-testid="tokens-retry"]').trigger('click')
|
||||
await useSessionStore().logout()
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
finish(ok({ textSha256: 'same-sha', tokens })); await flushPromises()
|
||||
expect(view.find('.reader-word').exists()).toBe(false)
|
||||
})
|
||||
it('moves a covered selected word above the sheet and restores the prior scroll on close', async () => {
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok(result('cat')))
|
||||
const scrollBy = vi.spyOn(window, 'scrollBy').mockImplementation(() => {})
|
||||
const scrollTo = vi.spyOn(window, 'scrollTo').mockImplementation(() => {})
|
||||
const scope = effectScope()
|
||||
const lookup = scope.run(() => useReaderLookup(ref(chapter as ChapterDetail)))!
|
||||
const element = document.createElement('span')
|
||||
vi.spyOn(element, 'getBoundingClientRect').mockReturnValue({ bottom: 700 } as DOMRect)
|
||||
lookup.select(tokens[2] as never, element)
|
||||
lookup.keepSelectionVisible(500)
|
||||
expect(scrollBy).toHaveBeenCalledWith({ top: 216, behavior: 'instant' })
|
||||
lookup.close()
|
||||
expect(scrollTo).toHaveBeenCalledWith({ top: 0, left: 0, behavior: 'instant' })
|
||||
scope.stop()
|
||||
})
|
||||
it('preserves a later manual reading scroll when closing the sheet', async () => {
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok(result('cat')))
|
||||
let scrollY = 100
|
||||
vi.spyOn(window, 'scrollY', 'get').mockImplementation(() => scrollY)
|
||||
vi.spyOn(window, 'scrollBy').mockImplementation((...args: unknown[]) => {
|
||||
const options = args[0] as ScrollToOptions
|
||||
scrollY += options.top ?? 0
|
||||
})
|
||||
const scrollTo = vi.spyOn(window, 'scrollTo').mockImplementation(() => {})
|
||||
const scope = effectScope()
|
||||
const lookup = scope.run(() => useReaderLookup(ref(chapter as ChapterDetail)))!
|
||||
const element = document.createElement('span')
|
||||
vi.spyOn(element, 'getBoundingClientRect').mockReturnValue({ bottom: 700 } as DOMRect)
|
||||
lookup.select(tokens[2] as never, element)
|
||||
lookup.keepSelectionVisible(500)
|
||||
scrollY += 300 // The reader continues down the page while the sheet is open.
|
||||
lookup.close()
|
||||
expect(scrollTo).not.toHaveBeenCalled()
|
||||
expect(scrollY).toBe(616)
|
||||
scope.stop()
|
||||
})
|
||||
it.each(['logout', 'chapter', 'unmount'])('invalidates pending lookup on %s', async action => {
|
||||
let finish!: (response: Response) => void
|
||||
const { view, router } = await open({ lookup: () => new Promise(resolve => { finish = resolve }) })
|
||||
await view.get('.reader-word').trigger('click')
|
||||
if (action === 'logout') await useSessionStore().logout()
|
||||
else if (action === 'chapter') await router.push('/chapters/56')
|
||||
else { view.unmount(); wrapper = undefined }
|
||||
finish(ok(result('OLD'))); await flushPromises()
|
||||
if (action === 'unmount') expect(document.querySelector('.lookup-panel')).toBeNull()
|
||||
else expect(view.find('.lookup-panel').exists()).toBe(false)
|
||||
expect(view.text()).not.toContain('Definition of OLD')
|
||||
})
|
||||
})
|
||||
describe('personal word records', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; vi.restoreAllMocks() })
|
||||
it('saves the definition, examples and status, then marks the word in the text', async () => {
|
||||
let body: Record<string, unknown> | undefined
|
||||
const { view, fetchMock } = await open({ termWrite: async value => { body = value; return ok({ term: savedTerm({ status: 'learning', level: 2 }), created: true }) } })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
await view.get('#term-definition').setValue(' 猫 ')
|
||||
await view.get('#term-examples').setValue('A fictional example.\n\n Second line ')
|
||||
await view.findAll('input[type="radio"]')[1]!.setValue()
|
||||
await view.get('[data-testid="term-save"]').trigger('click'); await flushPromises()
|
||||
expect(body).toEqual({ chapterId: 55, start: 2, end: 6, definition: ' 猫 ', examples: ['A fictional example.', 'Second line'], status: 'learning' })
|
||||
expect(view.text()).toContain('已保存 · 学习中')
|
||||
expect(view.findAll('.reader-word')[0]!.classes()).toContain('is-learning')
|
||||
expect(callsTo(fetchMock, '/terms')).toHaveLength(1)
|
||||
})
|
||||
it('keeps the typed input and reports the failure when saving fails', async () => {
|
||||
const { view } = await open({ termWrite: async () => { throw new Error('保存暂时不可用') } })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
await view.get('#term-definition').setValue('虚构释义')
|
||||
await view.get('[data-testid="term-save"]').trigger('click'); await flushPromises()
|
||||
expect(view.text()).toContain('保存暂时不可用')
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('虚构释义')
|
||||
expect(view.find('.lookup-panel').exists()).toBe(true)
|
||||
expect(view.findAll('.reader-word')[0]!.classes()).not.toContain('is-new')
|
||||
})
|
||||
it('loads the stored record when a saved word is opened and marks it in the text', async () => {
|
||||
const tokenData = withTerm('Cats', { id: 7, status: 'known', level: 0 })
|
||||
const { view, fetchMock } = await open({ tokenData, termRead: async () => ok({ term: savedTerm({ status: 'known', level: 0 }) }) })
|
||||
expect(view.findAll('.reader-word')[0]!.classes()).toContain('is-known')
|
||||
expect(view.findAll('.reader-word')[0]!.attributes('aria-label')).toContain('已保存')
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
expect(callsTo(fetchMock, '/terms/7')).toHaveLength(1)
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('猫')
|
||||
expect((view.get('#term-examples').element as HTMLTextAreaElement).value).toBe('A fictional example.')
|
||||
expect((view.findAll('input[type="radio"]')[2]!.element as HTMLInputElement).checked).toBe(true)
|
||||
expect(view.text()).toContain('已保存')
|
||||
})
|
||||
it('blocks saving until a failed read of the stored record is retried', async () => {
|
||||
const tokenData = withTerm('Cats', { id: 7, status: 'learning', level: 1 })
|
||||
let attempts = 0
|
||||
const { view, fetchMock } = await open({ tokenData, termRead: async () => { attempts++; if (attempts === 1) throw new Error('已保存的内容暂时无法读取'); return ok({ term: savedTerm() }) } })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
expect(view.text()).toContain('已保存的内容暂时无法读取')
|
||||
expect(view.get('[data-testid="term-save"]').attributes('disabled')).toBeDefined()
|
||||
await view.get('.reader-word').trigger('click'); await flushPromises()
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('猫')
|
||||
expect(view.get('[data-testid="term-save"]').attributes('disabled')).toBeUndefined()
|
||||
expect(callsTo(fetchMock, '/terms')).toHaveLength(0)
|
||||
})
|
||||
it('drops the previous account words and highlights after an identity change', async () => {
|
||||
const { view, fetchMock } = await open({ termWrite: async () => ok({ term: savedTerm({ status: 'new', level: 0 }), created: true }) })
|
||||
await view.findAll('.reader-word')[0]!.trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="term-save"]').trigger('click'); await flushPromises()
|
||||
expect(view.findAll('.reader-word')[0]!.classes()).toContain('is-new')
|
||||
fetchMock.mockResolvedValueOnce(ok({ textSha256: 'same-sha', tokens }))
|
||||
useSessionStore().user = { id: 43, username: 'other-fictional', role: 'learner' }
|
||||
await flushPromises()
|
||||
expect(view.find('.lookup-panel').exists()).toBe(false)
|
||||
// The previous account's words and chapter are gone: the reader falls back to
|
||||
// nothing until the new identity loads its own chapter.
|
||||
expect(view.find('.reader-word').exists()).toBe(false)
|
||||
expect(view.find('.reader-text').exists()).toBe(false)
|
||||
expect(view.text()).not.toContain('已保存')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,186 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createRouter, createMemoryHistory } from 'vue-router'
|
||||
import ReaderView from '../views/ReaderView.vue'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
import type { PhraseSpan, ReaderToken } from '../composables/useReaderLookup'
|
||||
|
||||
// The same fixture shape the phrase tests use on the server side.
|
||||
const original = 'Take a small, step\nevery day.\nMira took a small step again.\n'
|
||||
const fragments: [string, ReaderToken['kind']][] = [
|
||||
['Take', 'word'], [' ', 'space'], ['a', 'word'], [' ', 'space'], ['small', 'word'], [',', 'punctuation'],
|
||||
[' ', 'space'], ['step', 'word'], ['\n', 'space'], ['every', 'word'], [' ', 'space'], ['day', 'word'],
|
||||
['.', 'punctuation'], ['\n', 'space'], ['Mira', 'word'], [' ', 'space'], ['took', 'word'], [' ', 'space'],
|
||||
['a', 'word'], [' ', 'space'], ['small', 'word'], [' ', 'space'], ['step', 'word'], [' ', 'space'],
|
||||
['again', 'word'], ['.', 'punctuation'], ['\n', 'space'],
|
||||
]
|
||||
let cp = 0
|
||||
let utf16 = 0
|
||||
const tokens: ReaderToken[] = fragments.map(([text, kind]) => {
|
||||
const token: ReaderToken = { text, kind, start: cp, end: cp + [...text].length, startUtf16: utf16, endUtf16: utf16 + text.length }
|
||||
cp = token.end
|
||||
utf16 = token.endUtf16
|
||||
return token
|
||||
})
|
||||
const chapter = { id: 55, bookId: 1, title: '虚构章节', status: 'ready', originalText: original, contentSha256: 'same-sha' }
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
const savedPhrase = {
|
||||
id: 7, term: 'a small step', originalForm: 'a small, step', definition: '一小步',
|
||||
examples: ['Take a small step, every day.'], status: 'new', level: 0, kind: 'phrase', wordCount: 3,
|
||||
}
|
||||
let wrapper: VueWrapper | undefined
|
||||
|
||||
interface OpenOptions {
|
||||
phrases?: PhraseSpan[]
|
||||
tokens?: ReaderToken[]
|
||||
original?: string
|
||||
save?: (body: Record<string, unknown>) => Promise<Response>
|
||||
termRead?: () => Promise<Response>
|
||||
}
|
||||
|
||||
async function open(options: OpenOptions = {}) {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
const url = String(input)
|
||||
if (url.endsWith('/tokens')) return ok({ textSha256: 'same-sha', tokens: options.tokens ?? tokens, phrases: options.phrases ?? [] })
|
||||
if (url.endsWith('/lookup')) return ok({ status: 'not_found', query: 'a', matchedForm: null, candidates: [], entries: [] })
|
||||
if (url.includes('/terms/')) return options.termRead ? options.termRead() : ok({ term: savedPhrase })
|
||||
if (url.endsWith('/phrases') || url.endsWith('/terms')) {
|
||||
const body = JSON.parse(String(init?.body)) as Record<string, unknown>
|
||||
if (options.save) return options.save(body)
|
||||
return ok({ term: savedPhrase, created: true })
|
||||
}
|
||||
// A test that supplies its own tokens must also supply the matching chapter text.
|
||||
const text2 = options.original ?? original
|
||||
return ok({ chapter: { ...chapter, originalText: text2 }, navigation: { previousChapterId: null, nextChapterId: 56 } })
|
||||
})
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
const router = createRouter({ history: createMemoryHistory(), routes: [{ path: '/chapters/:id', component: ReaderView }, { path: '/', component: { template: '<div />' } }] })
|
||||
await router.push('/chapters/55')
|
||||
wrapper = mount(ReaderView, { attachTo: document.body, global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
return { fetchMock, router, view: wrapper }
|
||||
}
|
||||
|
||||
/** Clicks one word and shift-clicks the next matching word after it to extend the range. */
|
||||
async function shiftSelect(view: VueWrapper, first: string, last: string) {
|
||||
const words = view.findAll('.reader-word')
|
||||
const startIndex = words.findIndex(item => item.text() === first)
|
||||
const endIndex = words.findIndex((item, index) => index > startIndex && item.text() === last)
|
||||
await words[startIndex]!.trigger('click')
|
||||
await flushPromises()
|
||||
await words[endIndex]!.trigger('click', { shiftKey: true })
|
||||
await flushPromises()
|
||||
}
|
||||
|
||||
describe('phrase selection and panel', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; vi.restoreAllMocks() })
|
||||
|
||||
it('builds a phrase from a shift-click range and saves it through the phrases endpoint', async () => {
|
||||
let body: Record<string, unknown> | undefined
|
||||
const { view } = await open({ save: async value => { body = value; return ok({ term: savedPhrase, created: true }) } })
|
||||
await shiftSelect(view, 'a', 'step')
|
||||
// The panel names the phrase and shows how many words it holds.
|
||||
expect(view.get('[data-testid="phrase-range"]').text()).toContain('短语 · 3 个单词')
|
||||
expect(view.text()).toContain('新词条')
|
||||
// A phrase has no dictionary lookup section.
|
||||
expect(view.find('[data-testid="lookup-retry"]').exists()).toBe(false)
|
||||
await view.get('#term-definition').setValue('一小步')
|
||||
await view.get('[data-testid="term-save"]').trigger('click')
|
||||
await flushPromises()
|
||||
// The selection travels as code point offsets of the whole phrase, punctuation included.
|
||||
expect(body).toMatchObject({ chapterId: 55, start: 5, end: 18, definition: '一小步', status: 'new' })
|
||||
expect(view.text()).toContain('已保存')
|
||||
// The saved phrase highlights immediately.
|
||||
expect(view.findAll('.reader-word').some(item => item.classes().includes('is-phrase'))).toBe(true)
|
||||
})
|
||||
|
||||
it('adjusts both ends by whole words from the panel', async () => {
|
||||
const { view } = await open()
|
||||
await shiftSelect(view, 'a', 'step')
|
||||
expect(view.get('[data-testid="phrase-range"]').text()).toContain('3 个单词')
|
||||
await view.get('[data-testid="range-start-right"]').trigger('click')
|
||||
await flushPromises()
|
||||
expect(view.get('[data-testid="phrase-range"]').text()).toContain('2 个单词')
|
||||
// Moving the start back restores the three-word phrase.
|
||||
await view.get('[data-testid="range-start-left"]').trigger('click')
|
||||
await flushPromises()
|
||||
expect(view.get('[data-testid="phrase-range"]').text()).toContain('3 个单词')
|
||||
// A two-word phrase cannot shrink further, so the shrinking buttons are disabled.
|
||||
await view.get('[data-testid="range-start-right"]').trigger('click')
|
||||
await flushPromises()
|
||||
expect(view.get('[data-testid="phrase-range"]').text()).toContain('2 个单词')
|
||||
expect(view.get('[data-testid="range-start-right"]').attributes('disabled')).toBeDefined()
|
||||
expect(view.get('[data-testid="range-end-left"]').attributes('disabled')).toBeDefined()
|
||||
// Growing it is still possible on either side.
|
||||
expect(view.get('[data-testid="range-start-left"]').attributes('disabled')).toBeUndefined()
|
||||
expect(view.get('[data-testid="range-end-right"]').attributes('disabled')).toBeUndefined()
|
||||
})
|
||||
|
||||
it('opens the saved phrase when its highlighted range is clicked', async () => {
|
||||
const span: PhraseSpan = { id: 7, status: 'new', wordCount: 3, startToken: 2, endToken: 7 }
|
||||
const { view, fetchMock } = await open({ phrases: [span] })
|
||||
const inside = view.findAll('.reader-word').find(item => item.text() === 'small')!
|
||||
expect(inside.classes()).toContain('is-phrase')
|
||||
await inside.trigger('click')
|
||||
await flushPromises()
|
||||
// The stored entry is read through the same route a word uses, and the panel edits it.
|
||||
expect(fetchMock.mock.calls.some(([url]) => String(url).includes('/terms/7'))).toBe(true)
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('一小步')
|
||||
expect(view.get('[data-testid="phrase-range"]').text()).toContain('已保存')
|
||||
})
|
||||
|
||||
it('stops claiming a stored phrase once its range is adjusted', async () => {
|
||||
const span: PhraseSpan = { id: 7, status: 'new', wordCount: 3, startToken: 2, endToken: 7 }
|
||||
const { view } = await open({ phrases: [span] })
|
||||
await view.findAll('.reader-word').find(item => item.text() === 'small')!.trigger('click')
|
||||
await flushPromises()
|
||||
expect(view.get('[data-testid="phrase-range"]').text()).toContain('已保存')
|
||||
await view.get('[data-testid="range-end-right"]').trigger('click')
|
||||
await flushPromises()
|
||||
// The adjusted range is another identity, so the panel no longer says it is stored, while
|
||||
// the text the learner already had stays in the form.
|
||||
expect(view.get('.lookup-panel').text()).toContain('新词条')
|
||||
expect(view.get('.lookup-panel').text()).not.toContain('已保存')
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('一小步')
|
||||
})
|
||||
|
||||
it('refuses a range longer than the phrase limit and keeps the panel closed', async () => {
|
||||
// One word more than a phrase may hold.
|
||||
const longTokens: ReaderToken[] = []
|
||||
const words: string[] = []
|
||||
let cursor = 0
|
||||
for (let index = 0; index < 14; index++) {
|
||||
const word = `w${index}`
|
||||
words.push(word)
|
||||
longTokens.push({ text: word, start: cursor, end: cursor + word.length, startUtf16: cursor, endUtf16: cursor + word.length, kind: 'word' })
|
||||
cursor += word.length
|
||||
longTokens.push({ text: ' ', start: cursor, end: cursor + 1, startUtf16: cursor, endUtf16: cursor + 1, kind: 'space' })
|
||||
cursor += 1
|
||||
}
|
||||
const { view } = await open({ tokens: longTokens, original: longTokens.map(token => token.text).join('') })
|
||||
await shiftSelect(view, words[0]!, words[13]!)
|
||||
expect(view.get('[data-testid="range-notice"]').text()).toContain('短语最多 12 个单词')
|
||||
expect(view.find('[data-testid="phrase-range"]').exists()).toBe(false)
|
||||
})
|
||||
|
||||
it('closes the phrase panel with Escape and restores the reading position', async () => {
|
||||
const { view } = await open()
|
||||
await shiftSelect(view, 'a', 'step')
|
||||
expect(view.find('[data-testid="phrase-range"]').exists()).toBe(true)
|
||||
await view.get('.lookup-panel').trigger('keydown', { key: 'Escape' })
|
||||
await flushPromises()
|
||||
expect(view.find('.lookup-panel').exists()).toBe(false)
|
||||
})
|
||||
|
||||
it('keeps the typed phrase text when saving fails', async () => {
|
||||
const { view } = await open({ save: async () => new Response(JSON.stringify({ code: 400, msg: '短语过长，请缩短选择范围' }), { status: 400 }) })
|
||||
await shiftSelect(view, 'a', 'step')
|
||||
await view.get('#term-definition').setValue('虚构释义')
|
||||
await view.get('[data-testid="term-save"]').trigger('click')
|
||||
await flushPromises()
|
||||
expect(view.text()).toContain('短语过长，请缩短选择范围')
|
||||
expect((view.get('#term-definition').element as HTMLTextAreaElement).value).toBe('虚构释义')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,112 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { adjustTokenRange, MAX_PHRASE_WORDS, normalizeTokenRange, phrasesAt, rangeOfSpan, rangeOfWords, wordIndices } from '../composables/readerRange'
|
||||
import type { ReaderToken } from '../composables/useReaderLookup'
|
||||
|
||||
// The same text #4 verified: interior punctuation, a line break inside the phrase, a repeated
|
||||
// phrase and a combining accent.
|
||||
const original = 'Take a small, step\nevery day.\nMira took a small step again.\n'
|
||||
const tokens: ReaderToken[] = [
|
||||
{ text: 'Take', start: 0, end: 4, startUtf16: 0, endUtf16: 4, kind: 'word' },
|
||||
{ text: ' ', start: 4, end: 5, startUtf16: 4, endUtf16: 5, kind: 'space' },
|
||||
{ text: 'a', start: 5, end: 6, startUtf16: 5, endUtf16: 6, kind: 'word' },
|
||||
{ text: ' ', start: 6, end: 7, startUtf16: 6, endUtf16: 7, kind: 'space' },
|
||||
{ text: 'small', start: 7, end: 12, startUtf16: 7, endUtf16: 12, kind: 'word' },
|
||||
{ text: ',', start: 12, end: 13, startUtf16: 12, endUtf16: 13, kind: 'punctuation' },
|
||||
{ text: ' ', start: 13, end: 14, startUtf16: 13, endUtf16: 14, kind: 'space' },
|
||||
{ text: 'step', start: 14, end: 18, startUtf16: 14, endUtf16: 18, kind: 'word' },
|
||||
{ text: '\n', start: 18, end: 19, startUtf16: 18, endUtf16: 19, kind: 'space' },
|
||||
{ text: 'every', start: 19, end: 24, startUtf16: 19, endUtf16: 24, kind: 'word' },
|
||||
{ text: ' ', start: 24, end: 25, startUtf16: 24, endUtf16: 25, kind: 'space' },
|
||||
{ text: 'day', start: 25, end: 28, startUtf16: 25, endUtf16: 28, kind: 'word' },
|
||||
{ text: '.', start: 28, end: 29, startUtf16: 28, endUtf16: 29, kind: 'punctuation' },
|
||||
{ text: '\n', start: 29, end: 30, startUtf16: 29, endUtf16: 30, kind: 'space' },
|
||||
{ text: 'Mira', start: 30, end: 34, startUtf16: 30, endUtf16: 34, kind: 'word' },
|
||||
{ text: ' ', start: 34, end: 35, startUtf16: 34, endUtf16: 35, kind: 'space' },
|
||||
{ text: 'took', start: 35, end: 39, startUtf16: 35, endUtf16: 39, kind: 'word' },
|
||||
{ text: ' ', start: 39, end: 40, startUtf16: 39, endUtf16: 40, kind: 'space' },
|
||||
{ text: 'a', start: 40, end: 41, startUtf16: 40, endUtf16: 41, kind: 'word' },
|
||||
{ text: ' ', start: 41, end: 42, startUtf16: 41, endUtf16: 42, kind: 'space' },
|
||||
{ text: 'small', start: 42, end: 47, startUtf16: 42, endUtf16: 47, kind: 'word' },
|
||||
{ text: ' ', start: 47, end: 48, startUtf16: 47, endUtf16: 48, kind: 'space' },
|
||||
{ text: 'step', start: 48, end: 52, startUtf16: 48, endUtf16: 52, kind: 'word' },
|
||||
{ text: ' ', start: 52, end: 53, startUtf16: 52, endUtf16: 53, kind: 'space' },
|
||||
{ text: 'again', start: 53, end: 58, startUtf16: 53, endUtf16: 58, kind: 'word' },
|
||||
{ text: '.', start: 58, end: 59, startUtf16: 58, endUtf16: 59, kind: 'punctuation' },
|
||||
{ text: '\n', start: 59, end: 60, startUtf16: 59, endUtf16: 60, kind: 'space' },
|
||||
]
|
||||
|
||||
describe('reader phrase ranges', () => {
|
||||
it('keeps interior punctuation and line breaks in the selected text', () => {
|
||||
const range = normalizeTokenRange(tokens, original, 2, 7)
|
||||
expect(range).not.toBeNull()
|
||||
expect(range).toMatchObject({ firstWord: 2, lastWord: 7, start: 5, end: 18, wordCount: 3 })
|
||||
expect(range!.text).toBe('a small, step')
|
||||
// The second occurrence keeps its own spelling and offsets.
|
||||
const second = normalizeTokenRange(tokens, original, 18, 22)
|
||||
expect(second!.text).toBe('a small step')
|
||||
expect(second!.start).toBe(40)
|
||||
})
|
||||
|
||||
it('aligns both ends to whole words and never selects one word as a phrase', () => {
|
||||
// A selection that starts on a separator begins at the next whole word, and one that ends
|
||||
// on a separator ends at the previous one: words outside the selection stay untouched.
|
||||
expect(normalizeTokenRange(tokens, original, 3, 7)!.firstWord).toBe(4)
|
||||
expect(normalizeTokenRange(tokens, original, 2, 5)!.lastWord).toBe(4)
|
||||
expect(normalizeTokenRange(tokens, original, 2, 7)!.text).toBe('a small, step')
|
||||
expect(normalizeTokenRange(tokens, original, 2, 3)).toBeNull()
|
||||
expect(normalizeTokenRange(tokens, original, 5, 5)).toBeNull()
|
||||
expect(normalizeTokenRange(tokens, original, -1, 7)).toBeNull()
|
||||
expect(normalizeTokenRange(tokens, original, 2, tokens.length + 5)).toBeNull()
|
||||
})
|
||||
|
||||
it('reverses a backwards selection and refuses more than twelve words', () => {
|
||||
const range = normalizeTokenRange(tokens, original, 7, 2)
|
||||
expect(range).toMatchObject({ firstWord: 2, lastWord: 7 })
|
||||
// The fixture is exactly twelve words, which is still a phrase.
|
||||
expect(normalizeTokenRange(tokens, original, 0, tokens.length - 1)!.wordCount).toBe(MAX_PHRASE_WORDS)
|
||||
const longer: ReaderToken[] = []
|
||||
let offset = 0
|
||||
for (let index = 0; index < MAX_PHRASE_WORDS + 1; index++) {
|
||||
const word = `word${index}`
|
||||
longer.push({ text: word, start: offset, end: offset + word.length, startUtf16: offset, endUtf16: offset + word.length, kind: 'word' })
|
||||
offset += word.length
|
||||
longer.push({ text: ' ', start: offset, end: offset + 1, startUtf16: offset, endUtf16: offset + 1, kind: 'space' })
|
||||
offset += 1
|
||||
}
|
||||
expect(normalizeTokenRange(longer, longer.map(token => token.text).join(''), 0, longer.length - 1)).toBeNull()
|
||||
})
|
||||
|
||||
it('moves one end by whole words and never inverts the range', () => {
|
||||
const range = normalizeTokenRange(tokens, original, 2, 7)!
|
||||
const shorterStart = adjustTokenRange(tokens, original, range, 'start', 1)
|
||||
expect(shorterStart).toMatchObject({ firstWord: 4, lastWord: 7, wordCount: 2 })
|
||||
expect(shorterStart!.text).toBe('small, step')
|
||||
// The start can move left while a word remains before it, and not past the first word.
|
||||
expect(adjustTokenRange(tokens, original, range, 'start', -1)).toMatchObject({ firstWord: 0, lastWord: 7 })
|
||||
expect(adjustTokenRange(tokens, original, rangeOfWords(tokens, original, 0, 7)!, 'start', -1)).toBeNull()
|
||||
const longerEnd = adjustTokenRange(tokens, original, range, 'end', 1)
|
||||
expect(longerEnd).toMatchObject({ firstWord: 2, lastWord: 9, wordCount: 4 })
|
||||
expect(longerEnd!.text).toBe('a small, step\nevery')
|
||||
const back = adjustTokenRange(tokens, original, longerEnd!, 'end', -1)!
|
||||
expect(back.text).toBe(range.text)
|
||||
// Moving the end before the start is refused even when the direction is legal.
|
||||
const twoWords = rangeOfWords(tokens, original, 2, 4)!
|
||||
expect(adjustTokenRange(tokens, original, twoWords, 'end', -1)).toBeNull()
|
||||
})
|
||||
|
||||
it('finds the phrase covering a word and builds its range again', () => {
|
||||
const phrases = [
|
||||
{ id: 1, startToken: 2, endToken: 7, status: 'new', wordCount: 3 },
|
||||
{ id: 2, startToken: 4, endToken: 7, status: 'learning', wordCount: 2 },
|
||||
]
|
||||
// Overlapping spans: the longer phrase wins for a word both cover.
|
||||
expect(phrasesAt(tokens, phrases, 4)?.id).toBe(1)
|
||||
expect(phrasesAt(tokens, phrases, 7)?.id).toBe(1)
|
||||
expect(phrasesAt(tokens, phrases, 0)).toBeNull()
|
||||
const span = rangeOfSpan(tokens, original, { startToken: 2, endToken: 7 })!
|
||||
expect(span.text).toBe('a small, step')
|
||||
expect(rangeOfWords(tokens, original, 4, 7)!.text).toBe('small, step')
|
||||
// A span that covers a single word is not a phrase.
|
||||
expect(rangeOfSpan(tokens, original, { startToken: 2, endToken: 2 })).toBeNull()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,265 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { flushPromises, mount } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createMemoryHistory, createRouter, type Router } from 'vue-router'
|
||||
import { defineComponent, h } from 'vue'
|
||||
import BookView from '../views/BookView.vue'
|
||||
import ImportView from '../views/ImportView.vue'
|
||||
import ReaderView from '../views/ReaderView.vue'
|
||||
import { useLibraryStore, type ChapterStatus } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
// All accounts, books and texts in these tests are deliberately fictitious.
|
||||
const user = { id: 42, username: 'fictional-reader', role: 'learner' as const }
|
||||
const book = { id: 1, title: '虚构样例书', language: 'en' }
|
||||
const navigation = { previousChapterId: null, nextChapterId: null }
|
||||
const pasted = 'First line.\n\tIndented line.\nTwo spaces kept.\n\nLast line.\n'
|
||||
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }), { status: 200 })
|
||||
const created = (data: unknown) => new Response(JSON.stringify({ code: 200, data }), { status: 201 })
|
||||
|
||||
function chapter(status: ChapterStatus, extra: Record<string, unknown> = {}) {
|
||||
return {
|
||||
id: 55,
|
||||
bookId: 1,
|
||||
ordinal: 1,
|
||||
title: '第一篇',
|
||||
status,
|
||||
charCount: 120,
|
||||
errorReason: '',
|
||||
errorMessage: '',
|
||||
jobId: 7,
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
updatedAt: '2026-01-01T00:00:00Z',
|
||||
...extra,
|
||||
}
|
||||
}
|
||||
|
||||
const stub = (name: string) => defineComponent({ name, render: () => h('div') })
|
||||
|
||||
async function viewAt(path: string): Promise<Router> {
|
||||
const router = createRouter({
|
||||
history: createMemoryHistory(),
|
||||
routes: [
|
||||
{ path: '/', component: stub('LibraryStub') },
|
||||
{ path: '/import', component: stub('ImportStub') },
|
||||
{ path: '/books/:id', component: stub('BookStub') },
|
||||
{ path: '/chapters/:id', component: stub('ChapterStub') },
|
||||
],
|
||||
})
|
||||
await router.push(path)
|
||||
await router.isReady()
|
||||
return router
|
||||
}
|
||||
|
||||
function signIn() {
|
||||
useSessionStore().user = { ...user }
|
||||
}
|
||||
|
||||
describe('learner reading views', () => {
|
||||
beforeEach(() => {
|
||||
sessionStorage.clear()
|
||||
setActivePinia(createPinia())
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
useLibraryStore().stopPolling()
|
||||
})
|
||||
|
||||
it('shows inline validation and sends nothing for an empty import form', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
signIn()
|
||||
const router = await viewAt('/import')
|
||||
const wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
await wrapper.find('form').trigger('submit')
|
||||
expect(wrapper.text()).toContain('请填写标题。')
|
||||
expect(wrapper.text()).toContain('请粘贴要导入的英文正文。')
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
|
||||
await wrapper.find('input#title').setValue(' ')
|
||||
await wrapper.find('textarea#text').setValue(' \n\t ')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
expect(wrapper.text()).toContain('请填写标题。')
|
||||
expect(wrapper.text()).toContain('请粘贴要导入的英文正文。')
|
||||
|
||||
await wrapper.find('input#title').setValue('虚构样例第一章')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
expect(wrapper.text()).not.toContain('请填写标题。')
|
||||
expect(wrapper.text()).toContain('请粘贴要导入的英文正文。')
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('submits valid pasted text and opens the created book', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
.mockResolvedValueOnce(created({ book, chapter: chapter('pending'), job: { id: 7 }, duplicate: false }))
|
||||
signIn()
|
||||
const router = await viewAt('/import')
|
||||
const wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
// Element Plus assigns the input ids on mount, so wait for the first update.
|
||||
await flushPromises()
|
||||
|
||||
await wrapper.find('input#title').setValue('虚构样例第一章')
|
||||
await wrapper.find('textarea#text').setValue(pasted)
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
expect(JSON.parse(String(fetchMock.mock.calls[0]?.[1]?.body))).toMatchObject({ title: '虚构样例第一章', text: pasted, language: 'en' })
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('renders a ready chapter verbatim, keeping line breaks, tabs and repeated spaces', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ book, chapter: chapter('ready', { originalText: pasted, contentSha256: 'fictional' }), navigation }))
|
||||
signIn()
|
||||
const router = await viewAt('/chapters/55')
|
||||
const wrapper = mount(ReaderView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
const article = wrapper.find('.reader-text')
|
||||
expect(article.exists()).toBe(true)
|
||||
expect(article.element.textContent).toBe(pasted)
|
||||
expect(wrapper.text()).toContain('已就绪')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('shows the failure message of a failed chapter and no text', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ book, chapter: chapter('failed', { errorReason: 'decode_error', errorMessage: '无法解析正文，请检查编码。', jobId: null }), navigation }))
|
||||
signIn()
|
||||
const router = await viewAt('/chapters/55')
|
||||
const wrapper = mount(ReaderView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
expect(wrapper.text()).toContain('处理失败')
|
||||
expect(wrapper.text()).toContain('无法解析正文，请检查编码。')
|
||||
expect(wrapper.find('.reader-text').exists()).toBe(false)
|
||||
// A null job id means there is nothing to retry yet.
|
||||
expect(wrapper.text()).toContain('这一章暂时没有可重试的任务编号。')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('offers retry on a failed chapter through its own job id', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ book, chapter: chapter('failed', { errorMessage: '解析失败。', jobId: 7 }), navigation }))
|
||||
signIn()
|
||||
const router = await viewAt('/chapters/55')
|
||||
const wrapper = mount(ReaderView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
expect(wrapper.text()).toContain('解析失败。')
|
||||
const retryButton = wrapper.get('.notice .el-button')
|
||||
expect(retryButton.text()).toContain('重试处理')
|
||||
await retryButton.trigger('click')
|
||||
await flushPromises()
|
||||
|
||||
// No prior submit in this session: the job id comes from the chapter payload.
|
||||
const retryCall = fetchMock.mock.calls.find(([input]) => String(input).endsWith('/jobs/7/retry'))
|
||||
expect(retryCall?.[1]?.method).toBe('POST')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
it('lists chapters with their status labels and links only ready chapters', async () => {
|
||||
const chapters = [
|
||||
chapter('pending', { id: 55, ordinal: 1, title: '第一篇', jobId: 7 }),
|
||||
chapter('processing', { id: 56, ordinal: 2, title: '第二篇', jobId: 8 }),
|
||||
chapter('ready', { id: 57, ordinal: 3, title: '第三篇', jobId: 9 }),
|
||||
chapter('failed', { id: 58, ordinal: 4, title: '第四篇', errorMessage: '解析失败。', jobId: null }),
|
||||
chapter('failed', { id: 59, ordinal: 5, title: '第五篇', errorMessage: '编码错误。', jobId: 10 }),
|
||||
]
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(input => {
|
||||
if (String(input).endsWith('/jobs/10/retry')) {
|
||||
return Promise.resolve(ok({ job: { id: 10, status: 'pending', attempts: 1 }, chapter: { id: 59, bookId: 1, jobId: 10 } }))
|
||||
}
|
||||
return Promise.resolve(ok({ book, chapters }))
|
||||
})
|
||||
signIn()
|
||||
const router = await viewAt('/books/1')
|
||||
const wrapper = mount(BookView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
const text = wrapper.text()
|
||||
expect(text).toContain('待处理')
|
||||
expect(text).toContain('处理中')
|
||||
expect(text).toContain('已就绪')
|
||||
expect(text).toContain('处理失败')
|
||||
expect(text).toContain('解析失败。')
|
||||
expect(text).toContain('编码错误。')
|
||||
expect(wrapper.find('a[href="/chapters/57"]').exists()).toBe(true)
|
||||
expect(wrapper.find('a[href="/chapters/55"]').exists()).toBe(false)
|
||||
|
||||
// Retry is offered only for the failed chapter that carries a job id.
|
||||
const retryButtons = wrapper.findAll('.chapter-row .el-button').filter(button => button.text().includes('重试'))
|
||||
expect(retryButtons).toHaveLength(1)
|
||||
await retryButtons[0]!.trigger('click')
|
||||
await flushPromises()
|
||||
const retryCall = fetchMock.mock.calls.find(([input]) => String(input).endsWith('/jobs/10/retry'))
|
||||
expect(retryCall?.[1]?.method).toBe('POST')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
// Regression R1: appending must not send the language field that the append contract rejects.
|
||||
it('appends to an existing book through the chapter endpoint without a language field', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
.mockResolvedValueOnce(ok({
|
||||
items: [{
|
||||
...book,
|
||||
chapterCount: 1,
|
||||
pendingCount: 0,
|
||||
processingCount: 0,
|
||||
readyCount: 1,
|
||||
failedCount: 0,
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
updatedAt: '2026-01-01T00:00:00Z',
|
||||
}],
|
||||
}))
|
||||
.mockResolvedValueOnce(created({ chapter: chapter('pending', { id: 56, ordinal: 2 }), job: { id: 8 } }))
|
||||
signIn()
|
||||
const router = await viewAt('/import?book=1')
|
||||
const wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
await wrapper.find('input#title').setValue('第二篇')
|
||||
await wrapper.find('textarea#text').setValue(pasted)
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
|
||||
const appendCall = fetchMock.mock.calls.find(([input]) => String(input).endsWith('/books/1/chapters'))
|
||||
expect(appendCall?.[1]?.method).toBe('POST')
|
||||
const body = JSON.parse(String(appendCall?.[1]?.body)) as Record<string, unknown>
|
||||
expect(body).toEqual({ requestId: expect.any(String), title: '第二篇', text: pasted })
|
||||
expect(body).not.toHaveProperty('language')
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
wrapper.unmount()
|
||||
})
|
||||
|
||||
// Regression R3: a submit that finishes after the user left the page must not navigate back.
|
||||
it('does not navigate after the user left the import page during a submit', async () => {
|
||||
let finish!: (response: Response) => void
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
.mockImplementationOnce(() => new Promise<Response>(resolve => { finish = resolve }))
|
||||
signIn()
|
||||
const router = await viewAt('/import')
|
||||
const wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
|
||||
await wrapper.find('input#title').setValue('虚构样例第一章')
|
||||
await wrapper.find('textarea#text').setValue(pasted)
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
|
||||
// The user leaves the page while the request is still open.
|
||||
wrapper.unmount()
|
||||
await router.push('/')
|
||||
await flushPromises()
|
||||
|
||||
finish(created({ book, chapter: chapter('pending'), job: { id: 7 } }))
|
||||
await flushPromises()
|
||||
|
||||
expect(router.currentRoute.value.path).toBe('/')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,255 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createMemoryHistory, createRouter } from 'vue-router'
|
||||
import ReviewView from '../views/ReviewView.vue'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
import { clozeSentence, maskedPrompt, useReviewStore, type ReviewItem } from '../stores/review'
|
||||
|
||||
const item = (overrides: Partial<ReviewItem> = {}): ReviewItem => ({
|
||||
id: 7, term: 'dogs', originalForm: 'Dogs', definition: '狗', examples: ['Dogs went home.'],
|
||||
status: 'new', level: 0, kind: 'word', wordCount: 1, dueAt: '2026-09-11T10:00:00Z', reviewCount: 0, ...overrides,
|
||||
})
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
const answer = (overrides: Record<string, unknown> = {}) => ({
|
||||
result: 'applied', grade: 'correct', requeued: false, statusBefore: 'new', statusAfter: 'learning',
|
||||
levelBefore: 0, levelAfter: 1, dueAtBefore: '2026-09-11T10:00:00Z', dueAtAfter: '2026-09-12T10:00:00Z',
|
||||
item: item({ status: 'learning', level: 1, dueAt: '2026-09-12T10:00:00Z', reviewCount: 1 }), ...overrides,
|
||||
})
|
||||
const answerCalls = (fetchMock: MockInstance) => fetchMock.mock.calls.filter(call => String(call[0]).includes('/answers'))
|
||||
let wrapper: VueWrapper | undefined
|
||||
|
||||
describe('review prompt', () => {
|
||||
it('masks a whole phrase as one blank and keeps the word rule for words', () => {
|
||||
const phrase = item({ kind: 'phrase', wordCount: 3, term: 'a small step', originalForm: 'a small step', examples: ['Take a small step, every day.'] })
|
||||
expect(maskedPrompt(phrase)).toBe('Take _____, every day.')
|
||||
expect(maskedPrompt(item({ ...phrase, examples: ['Take a small,\nstep today.'] }))).toBe('Take _____ today.')
|
||||
// A phrase that is not in the example is left alone, and a word still masks itself.
|
||||
expect(maskedPrompt(item({ ...phrase, examples: ['Nothing to mask here.'] }))).toBe('Nothing to mask here.')
|
||||
expect(maskedPrompt(item({ term: 'curiosity', originalForm: 'curiosity', examples: ['Learning begins with curiosity.'] }))).toBe('Learning begins with _____.')
|
||||
})
|
||||
|
||||
it('masks the word in the first example and keeps other sentences usable', () => {
|
||||
expect(clozeSentence(item())).toBe('_____ went home.')
|
||||
expect(clozeSentence(item({ term: "isn't", originalForm: "Isn't", examples: ["It isn’t over."] }))).toBe('It _____ over.')
|
||||
expect(clozeSentence(item({ examples: ['No matching word here.'] }))).toBe('No matching word here.')
|
||||
expect(clozeSentence(item({ examples: [] }))).toBeNull()
|
||||
expect(clozeSentence(item({ term: 'dog', originalForm: 'Dog', examples: ['Dogs are not the saved word.'] }))).toBe('Dogs are not the saved word.')
|
||||
})
|
||||
})
|
||||
|
||||
describe('review store', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear(); useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' } })
|
||||
afterEach(() => { vi.restoreAllMocks() })
|
||||
|
||||
it('loads the due queue and answers a word correctly', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
if (String(input).endsWith('/reviews/queue')) return ok({ items: [item()], total: 1 })
|
||||
return ok(answer())
|
||||
})
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
expect(review.queue).toHaveLength(1)
|
||||
expect(review.current?.id).toBe(7)
|
||||
expect(review.empty).toBe(false)
|
||||
review.reveal()
|
||||
expect(review.revealed).toBe(true)
|
||||
await review.answer('correct')
|
||||
const body = JSON.parse(String(answerCalls(fetchMock)[0]![1]?.body))
|
||||
expect(body).toEqual({ answerId: expect.stringMatching(/^[0-9a-f-]{36}$/), grade: 'correct', expectedDueAt: '2026-09-11T10:00:00Z' })
|
||||
expect(review.queue).toHaveLength(0)
|
||||
expect(review.answered).toBe(1)
|
||||
expect(review.correctCount).toBe(1)
|
||||
expect(review.finished).toBe(true)
|
||||
expect(review.revealed).toBe(false)
|
||||
})
|
||||
|
||||
it('requeues a word that is not recognised and counts it as wrong', async () => {
|
||||
const wrong = answer({ grade: 'wrong', requeued: true, levelAfter: 0, statusAfter: 'new', dueAtAfter: '2026-09-11T10:05:00Z', item: item({ dueAt: '2026-09-11T10:05:00Z' }) })
|
||||
vi.spyOn(globalThis, 'fetch').mockImplementation(async input => String(input).endsWith('/reviews/queue') ? ok({ items: [item()], total: 1 }) : ok(wrong))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
await review.answer('wrong')
|
||||
expect(review.queue).toHaveLength(1)
|
||||
expect(review.queue[0]!.dueAt).toBe('2026-09-11T10:05:00Z')
|
||||
expect(review.wrongCount).toBe(1)
|
||||
expect(review.correctCount).toBe(0)
|
||||
expect(review.wordsReviewed).toBe(1)
|
||||
expect(review.finished).toBe(false)
|
||||
})
|
||||
|
||||
it('treats a stale answer as no new score but finishes the round instead of reporting an empty queue', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockImplementation(async input => String(input).endsWith('/reviews/queue') ? ok({ items: [item()], total: 1 }) : ok(answer({ result: 'stale', requeued: false })))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
await review.answer('correct')
|
||||
expect(review.queue).toHaveLength(0)
|
||||
expect(review.answered).toBe(0)
|
||||
expect(review.correctCount).toBe(0)
|
||||
expect(review.resolved).toBe(1)
|
||||
expect(review.empty).toBe(false)
|
||||
expect(review.finished).toBe(true)
|
||||
expect(review.notice).toContain('已在其他页面复习')
|
||||
})
|
||||
|
||||
it('counts a replay of this client own answer after a lost response', async () => {
|
||||
let sent = 0
|
||||
vi.spyOn(globalThis, 'fetch').mockImplementation(async input => {
|
||||
if (String(input).endsWith('/reviews/queue')) return ok({ items: [item()], total: 1 })
|
||||
sent += 1
|
||||
// The first response never reaches the client, the retry reports the recorded answer.
|
||||
if (sent === 1) throw new Error('网络中断')
|
||||
return ok(answer({ result: 'applied', duplicate: true }))
|
||||
})
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
await review.answer('correct')
|
||||
expect(review.error).toContain('网络中断')
|
||||
expect(review.answered).toBe(0)
|
||||
await review.answer('correct')
|
||||
expect(review.answered).toBe(1)
|
||||
expect(review.correctCount).toBe(1)
|
||||
expect(review.wordsReviewed).toBe(1)
|
||||
expect(review.empty).toBe(false)
|
||||
expect(review.finished).toBe(true)
|
||||
expect(review.notice).toBe('')
|
||||
})
|
||||
|
||||
it('keeps the card and the same answer id when a submission fails, then retries once', async () => {
|
||||
let fail = true
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async input => {
|
||||
if (String(input).endsWith('/reviews/queue')) return ok({ items: [item()], total: 1 })
|
||||
if (fail) { fail = false; throw new Error('评分暂时无法提交') }
|
||||
return ok(answer())
|
||||
})
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
await review.answer('correct')
|
||||
expect(review.error).toContain('评分暂时无法提交')
|
||||
expect(review.queue).toHaveLength(1)
|
||||
expect(review.answered).toBe(0)
|
||||
await review.answer('correct')
|
||||
const calls = answerCalls(fetchMock)
|
||||
const ids = calls.map(call => JSON.parse(String(call[1]?.body)).answerId)
|
||||
expect(ids).toHaveLength(2)
|
||||
expect(ids[0]).toBe(ids[1])
|
||||
expect(review.queue).toHaveLength(0)
|
||||
expect(review.answered).toBe(1)
|
||||
})
|
||||
|
||||
it('reports a failed queue load without pretending the queue is empty', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockRejectedValue(new Error('网络暂不可用'))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
expect(review.error).toContain('网络暂不可用')
|
||||
expect(review.finished).toBe(false)
|
||||
expect(review.empty).toBe(false)
|
||||
})
|
||||
|
||||
it('drops the previous account queue and counters when the identity changes', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ items: [item()], total: 1 }))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
expect(review.queue).toHaveLength(1)
|
||||
useSessionStore().user = { id: 43, username: 'other-fictional', role: 'learner' }
|
||||
await flushPromises()
|
||||
expect(review.queue).toHaveLength(0)
|
||||
expect(review.current).toBeNull()
|
||||
expect(review.finished).toBe(false)
|
||||
expect(review.empty).toBe(false)
|
||||
})
|
||||
|
||||
it('reports the words still due beyond the fetched page', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ items: [item()], total: 3 }))
|
||||
const review = useReviewStore()
|
||||
await review.load()
|
||||
expect(review.pending).toBe(2)
|
||||
})
|
||||
})
|
||||
|
||||
async function open(queueResult: unknown, onAnswer: (body: Record<string, unknown>) => Response | Promise<Response> = () => ok(answer())) {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
if (String(input).endsWith('/reviews/queue')) {
|
||||
if (queueResult instanceof Error) throw queueResult
|
||||
return ok(queueResult)
|
||||
}
|
||||
return onAnswer(JSON.parse(String(init?.body)))
|
||||
})
|
||||
useSessionStore().user = { id: 42, username: 'fictional', role: 'learner' }
|
||||
const router = createRouter({ history: createMemoryHistory(), routes: [{ path: '/', component: { template: '<div />' } }, { path: '/review', component: ReviewView }] })
|
||||
await router.push('/review')
|
||||
wrapper = mount(ReviewView, { attachTo: document.body, global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
return { view: wrapper, router, fetchMock }
|
||||
}
|
||||
|
||||
describe('review page', () => {
|
||||
beforeEach(() => { setActivePinia(createPinia()); sessionStorage.clear() })
|
||||
afterEach(() => { wrapper?.unmount(); wrapper = undefined; vi.restoreAllMocks() })
|
||||
|
||||
it('shows the masked example, hides the definition until the answer is revealed, then grades it', async () => {
|
||||
const { view, fetchMock } = await open({ items: [item()], total: 2 })
|
||||
expect(view.get('[data-testid="review-position"]').text()).toBe('到期复习 · 1 / 1')
|
||||
expect(view.text()).toContain('Dogs')
|
||||
expect(view.text()).toContain('_____ went home.')
|
||||
expect(view.find('[data-testid="review-definition"]').exists()).toBe(false)
|
||||
expect(document.activeElement).toBe(view.get('[data-testid="review-reveal"]').element)
|
||||
await view.get('[data-testid="review-reveal"]').trigger('click'); await flushPromises()
|
||||
expect(view.get('[data-testid="review-definition"]').text()).toContain('狗')
|
||||
expect(document.activeElement).toBe(view.get('[data-testid="review-correct"]').element)
|
||||
await view.get('[data-testid="review-correct"]').trigger('click'); await flushPromises()
|
||||
expect(answerCalls(fetchMock)).toHaveLength(1)
|
||||
expect(view.get('[data-testid="review-summary"]').text()).toContain('1 个词条')
|
||||
})
|
||||
|
||||
it('keeps the card and offers a retry when a grade fails', async () => {
|
||||
const { view } = await open({ items: [item()], total: 1 }, () => { throw new Error('评分暂时无法提交，请重试。') })
|
||||
await view.get('[data-testid="review-reveal"]').trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="review-wrong"]').trigger('click'); await flushPromises()
|
||||
expect(view.find('[data-testid="review-card"]').exists()).toBe(true)
|
||||
expect(view.text()).toContain('评分暂时无法提交')
|
||||
expect(view.find('[data-testid="review-retry"]').exists()).toBe(true)
|
||||
expect(view.find('[data-testid="review-summary"]').exists()).toBe(false)
|
||||
})
|
||||
|
||||
it('reports an empty queue and a load failure without pretending anything was reviewed', async () => {
|
||||
const emptyRun = await open({ items: [], total: 0 })
|
||||
expect(emptyRun.view.get('[data-testid="review-empty"]').text()).toContain('今天没有到期词条')
|
||||
await emptyRun.view.get('[data-testid="review-finish"]').trigger('click'); await flushPromises()
|
||||
expect(emptyRun.router.currentRoute.value.path).toBe('/')
|
||||
wrapper?.unmount(); wrapper = undefined
|
||||
|
||||
// A failed load is not an empty queue: it shows the error and a retry.
|
||||
const failing = await open(new Error('复习队列暂时无法加载'))
|
||||
expect(failing.view.get('[data-testid="review-reload"]').text()).toContain('重试')
|
||||
expect(failing.view.text()).toContain('复习队列暂时无法加载')
|
||||
expect(failing.view.find('[data-testid="review-empty"]').exists()).toBe(false)
|
||||
vi.restoreAllMocks()
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(ok({ items: [item()], total: 1 }))
|
||||
await failing.view.get('[data-testid="review-reload"]').trigger('click'); await flushPromises()
|
||||
expect(failing.view.get('[data-testid="review-position"]').text()).toBe('到期复习 · 1 / 1')
|
||||
})
|
||||
|
||||
it('offers the remaining due words after a round finishes', async () => {
|
||||
const { view } = await open({ items: [item()], total: 3 })
|
||||
await view.get('[data-testid="review-reveal"]').trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="review-correct"]').trigger('click'); await flushPromises()
|
||||
expect(view.get('[data-testid="review-more"]').text()).toContain('继续复习')
|
||||
expect(view.text()).toContain('还有 2 个词条到期')
|
||||
})
|
||||
|
||||
it('tells the learner when a card was already reviewed elsewhere', async () => {
|
||||
const { view } = await open({ items: [item()], total: 1 }, () => ok(answer({ result: 'stale', duplicate: false })))
|
||||
await view.get('[data-testid="review-reveal"]').trigger('click'); await flushPromises()
|
||||
await view.get('[data-testid="review-correct"]').trigger('click'); await flushPromises()
|
||||
expect(view.get('[data-testid="review-notice"]').text()).toContain('已在其他页面复习')
|
||||
expect(view.get('[data-testid="review-summary"]').text()).toContain('本轮没有新的计分')
|
||||
})
|
||||
|
||||
it('ends the round without submitting anything', async () => {
|
||||
const { view, router, fetchMock } = await open({ items: [item()], total: 1 })
|
||||
await view.get('[data-testid="review-end"]').trigger('click'); await flushPromises()
|
||||
expect(router.currentRoute.value.path).toBe('/')
|
||||
expect(answerCalls(fetchMock)).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,223 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi, type MockInstance } from 'vitest'
|
||||
import { flushPromises, mount, type VueWrapper } from '@vue/test-utils'
|
||||
import { createPinia, setActivePinia } from 'pinia'
|
||||
import { createRouter, createMemoryHistory, type Router } from 'vue-router'
|
||||
import ImportView from '../views/ImportView.vue'
|
||||
import { fileProblem, fileSizeLabel, TXT_MAX_BYTES, useLibraryStore } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const user = { id: 42, username: 'fictional-uploader', role: 'learner' as const }
|
||||
const book = { id: 1, title: 'Uploaded Book', language: 'en' }
|
||||
const chapter = { id: 9, bookId: 1, ordinal: 1, title: 'Uploaded Book', status: 'pending', charCount: 12, errorReason: '', errorMessage: '', jobId: 5, createdAt: '', updatedAt: '' }
|
||||
const timestamps = { createdAt: '2026-01-01T00:00:00Z', updatedAt: '2026-01-01T00:00:00Z' }
|
||||
const ok = (data: unknown) => new Response(JSON.stringify({ code: 200, data }))
|
||||
let wrapper: VueWrapper | undefined
|
||||
|
||||
function stub(name: string) {
|
||||
return { template: `<div>${name}</div>` }
|
||||
}
|
||||
|
||||
async function viewAt(path: string): Promise<Router> {
|
||||
const router = createRouter({
|
||||
history: createMemoryHistory(),
|
||||
routes: [
|
||||
{ path: '/', component: stub('LibraryStub') },
|
||||
{ path: '/import', component: stub('ImportStub') },
|
||||
{ path: '/books/:id', component: stub('BookStub') },
|
||||
],
|
||||
})
|
||||
await router.push(path)
|
||||
await router.isReady()
|
||||
return router
|
||||
}
|
||||
|
||||
/** jsdom has no file picker, so the input's files are set directly before dispatching change. */
|
||||
async function chooseFile(view: VueWrapper, file: File | null): Promise<void> {
|
||||
const input = view.get('[data-testid="file-input"]')
|
||||
Object.defineProperty(input.element, 'files', { value: file ? [file] : [], configurable: true })
|
||||
await input.trigger('change')
|
||||
await flushPromises()
|
||||
}
|
||||
|
||||
/** The TXT toggle is an Element Plus radio group; its hidden input carries the value. */
|
||||
async function useTxt(view: VueWrapper): Promise<void> {
|
||||
for (const input of view.findAll('input[type="radio"]')) {
|
||||
if ((input.element as HTMLInputElement).value === 'txt') {
|
||||
await input.setValue()
|
||||
await flushPromises()
|
||||
return
|
||||
}
|
||||
}
|
||||
throw new Error('TXT 文件 toggle not found')
|
||||
}
|
||||
|
||||
function uploadResponse() {
|
||||
return ok({ book, chapter, job: { id: 5, bookId: 1, chapterId: 9, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps }, duplicate: false })
|
||||
}
|
||||
|
||||
describe('txt upload pre-checks', () => {
|
||||
it('mirrors the server limits for the file that was picked', () => {
|
||||
expect(fileProblem({ name: 'reading.txt', size: 2048 })).toBe('')
|
||||
expect(fileProblem({ name: 'READING.TXT', size: 1 })).toBe('')
|
||||
expect(fileProblem({ name: 'reading.md', size: 2048 })).toContain('.txt')
|
||||
expect(fileProblem({ name: 'empty.txt', size: 0 })).toContain('空的')
|
||||
expect(fileProblem({ name: 'big.txt', size: TXT_MAX_BYTES + 1 })).toContain('2 MiB')
|
||||
expect(fileProblem({ name: 'limit.txt', size: TXT_MAX_BYTES })).toBe('')
|
||||
expect(fileSizeLabel(512)).toBe('512 B')
|
||||
expect(fileSizeLabel(2048)).toBe('2 KB')
|
||||
expect(fileSizeLabel(1.5 * 1024 * 1024)).toBe('1.5 MB')
|
||||
})
|
||||
})
|
||||
|
||||
describe('txt upload view', () => {
|
||||
beforeEach(() => {
|
||||
setActivePinia(createPinia())
|
||||
sessionStorage.clear()
|
||||
vi.restoreAllMocks()
|
||||
useSessionStore().user = { ...user }
|
||||
})
|
||||
afterEach(() => {
|
||||
wrapper?.unmount()
|
||||
wrapper = undefined
|
||||
useLibraryStore().stopPolling()
|
||||
})
|
||||
|
||||
it('shows the file picker instead of the textarea and reports the picked file', async () => {
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
// Paste mode keeps the textarea; TXT mode replaces it with the picker.
|
||||
expect(wrapper.find('textarea#text').exists()).toBe(true)
|
||||
expect(wrapper.find('[data-testid="file-input"]').exists()).toBe(false)
|
||||
await useTxt(wrapper)
|
||||
expect(wrapper.find('textarea#text').exists()).toBe(false)
|
||||
expect(wrapper.find('[data-testid="file-input"]').exists()).toBe(true)
|
||||
expect(wrapper.text()).toContain('仅支持 UTF-8')
|
||||
|
||||
await chooseFile(wrapper, new File(['Mira opened the workshop.\n'], 'reading.txt', { type: 'text/plain' }))
|
||||
expect(wrapper.get('[data-testid="file-info"]').text()).toBe('reading.txt · UTF-8 · 26 B')
|
||||
// The title is prefilled from the file name and stays editable.
|
||||
expect((wrapper.get('input#title').element as HTMLInputElement).value).toBe('reading')
|
||||
})
|
||||
|
||||
it('refuses a file the server would refuse, before anything is sent', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch')
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
|
||||
await chooseFile(wrapper, new File(['# not a txt file\n'], 'notes.md'))
|
||||
expect(wrapper.text()).toContain('请选择 .txt 文件。')
|
||||
// A file that is not valid UTF-8 is rejected by the preview decode.
|
||||
await chooseFile(wrapper, new File([new Uint8Array([0x63, 0x61, 0x66, 0xe9, 0x0a])], 'latin1.txt'))
|
||||
expect(wrapper.text()).toContain('文件不是 UTF-8 编码')
|
||||
await wrapper.find('input#title').setValue('Latin One')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
expect(wrapper.text()).toContain('文件不是 UTF-8 编码')
|
||||
|
||||
// No file at all is also refused locally.
|
||||
await chooseFile(wrapper, null)
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(fetchMock).not.toHaveBeenCalled()
|
||||
expect(wrapper.text()).toContain('请选择要导入的 TXT 文件。')
|
||||
})
|
||||
|
||||
it('uploads the file as multipart and opens the created book', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockResolvedValueOnce(uploadResponse())
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
const file = new File(['Mira opened the workshop.\n'], 'reading.txt', { type: 'text/plain' })
|
||||
await chooseFile(wrapper, file)
|
||||
await wrapper.find('input#title').setValue('上传的虚构章节')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1)
|
||||
const [url, init] = fetchMock.mock.calls[0]!
|
||||
expect(String(url)).toBe('/api/v1/books/upload')
|
||||
expect(init?.method).toBe('POST')
|
||||
// A multipart body must not be replaced by JSON and must keep the browser's boundary.
|
||||
expect(init?.body).toBeInstanceOf(FormData)
|
||||
expect((init?.headers as Record<string, string>)['Content-Type']).toBeUndefined()
|
||||
const body = init?.body as FormData
|
||||
expect(body.get('requestId')).toMatch(/^[0-9a-f-]{36}$/)
|
||||
expect(body.get('title')).toBe('上传的虚构章节')
|
||||
expect(body.get('language')).toBe('en')
|
||||
expect((body.get('file') as File).name).toBe('reading.txt')
|
||||
expect(await (body.get('file') as File).text()).toBe('Mira opened the workshop.\n')
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
// The form starts clean, so the same file is not submitted twice by accident.
|
||||
expect((wrapper.get('[data-testid="file-input"]').element as HTMLInputElement).value).toBe('')
|
||||
})
|
||||
|
||||
it('reuses one request id when the same upload is retried after a failure', async () => {
|
||||
let attempt = 0
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async () => {
|
||||
attempt += 1
|
||||
if (attempt === 1) throw new Error('上传中断')
|
||||
return uploadResponse()
|
||||
})
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
await chooseFile(wrapper, new File(['Body.\n'], 'retry.txt'))
|
||||
await wrapper.find('input#title').setValue('Retry Upload')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(wrapper.text()).toContain('上传中断')
|
||||
// The picked file stays selected so the learner can retry without choosing it again.
|
||||
expect(wrapper.get('[data-testid="file-info"]').text()).toContain('retry.txt')
|
||||
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
const ids = fetchMock.mock.calls.map(call => (call[1]?.body as FormData).get('requestId'))
|
||||
expect(ids).toHaveLength(2)
|
||||
expect(ids[0]).toBe(ids[1])
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
})
|
||||
|
||||
it('appends an uploaded chapter to a chosen book without a language field', async () => {
|
||||
const fetchMock = vi.spyOn(globalThis, 'fetch').mockImplementation(async (input, init) => {
|
||||
const url = String(input)
|
||||
if (url.endsWith('/books')) return ok({ items: [{ ...book, chapterCount: 1, pendingCount: 0, processingCount: 0, readyCount: 1, failedCount: 0, ...timestamps }] })
|
||||
expect(url).toBe('/api/v1/books/1/chapters/upload')
|
||||
const body = init?.body as FormData
|
||||
expect(body.get('language')).toBeNull()
|
||||
return ok({ chapter: { ...chapter, id: 10, ordinal: 2 }, job: { id: 6, bookId: 1, chapterId: 10, status: 'pending', attempts: 0, errorReason: '', errorMessage: '', ...timestamps }, duplicate: false })
|
||||
})
|
||||
const router = await viewAt('/import?book=1')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
await chooseFile(wrapper, new File(['Second chapter.\n'], 'second.txt'))
|
||||
await wrapper.find('input#title').setValue('Appended Chapter')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
|
||||
const posts = fetchMock.mock.calls.filter(call => String(call[0]).includes('/upload'))
|
||||
expect(posts).toHaveLength(1)
|
||||
expect(router.currentRoute.value.path).toBe('/books/1')
|
||||
})
|
||||
|
||||
it('keeps the server message and the form when the upload is rejected', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(new Response(JSON.stringify({ code: 400, msg: '文件不是 UTF-8 编码，请另存为 UTF-8 后重试' }), { status: 400 }))
|
||||
const router = await viewAt('/import')
|
||||
wrapper = mount(ImportView, { global: { plugins: [router] } })
|
||||
await flushPromises()
|
||||
await useTxt(wrapper)
|
||||
await chooseFile(wrapper, new File(['Body.\n'], 'server-rejects.txt'))
|
||||
await wrapper.find('input#title').setValue('Server Rejects')
|
||||
await wrapper.find('form').trigger('submit')
|
||||
await flushPromises()
|
||||
expect(wrapper.text()).toContain('文件不是 UTF-8 编码，请另存为 UTF-8 后重试')
|
||||
expect(wrapper.get('[data-testid="file-info"]').text()).toContain('server-rejects.txt')
|
||||
expect(router.currentRoute.value.path).toBe('/import')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,97 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted, onUnmounted, ref } from 'vue'
|
||||
import { ElButton, ElInput, ElOption, ElRadio, ElRadioGroup, ElSelect } from 'element-plus'
|
||||
import { TERM_STATUSES, type LookupResult, type TermStatus } from '../composables/useReaderLookup'
|
||||
|
||||
defineProps<{
|
||||
word: string
|
||||
// Set when the selection is a phrase: the panel edits that phrase instead of a word.
|
||||
phrase: { wordCount: number; stored: boolean } | null
|
||||
adjust: { startLeft: boolean; startRight: boolean; endLeft: boolean; endRight: boolean } | null
|
||||
result: LookupResult | null
|
||||
loading: boolean
|
||||
error: string
|
||||
saving: boolean
|
||||
saveError: string
|
||||
saved: string
|
||||
savedTermId: number | null
|
||||
prefilling: boolean
|
||||
prefillError: string
|
||||
canSave: boolean
|
||||
}>()
|
||||
const definition = defineModel<string>('definition', { required: true })
|
||||
const examples = defineModel<string>('examples', { required: true })
|
||||
const status = defineModel<TermStatus>('status', { required: true })
|
||||
const emit = defineEmits<{ close: []; retry: []; save: []; resize: [top: number]; adjust: [edge: 'start' | 'end', direction: -1 | 1] }>()
|
||||
const heading = ref<HTMLElement | null>(null)
|
||||
const panel = ref<HTMLElement | null>(null)
|
||||
let observer: ResizeObserver | undefined
|
||||
function resized() {
|
||||
if (window.innerWidth <= 760 && panel.value) emit('resize', panel.value.getBoundingClientRect().top)
|
||||
}
|
||||
onMounted(() => {
|
||||
heading.value?.focus({ preventScroll: true })
|
||||
resized()
|
||||
if (typeof ResizeObserver !== 'undefined' && panel.value) {
|
||||
observer = new ResizeObserver(resized)
|
||||
observer.observe(panel.value)
|
||||
}
|
||||
window.addEventListener('resize', resized)
|
||||
})
|
||||
onUnmounted(() => { observer?.disconnect(); window.removeEventListener('resize', resized) })
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<aside ref="panel" class="lookup-panel" aria-labelledby="lookup-heading" @keydown.esc.stop.prevent="$emit('close')">
|
||||
<header class="lookup-heading">
|
||||
<h2 id="lookup-heading" ref="heading" tabindex="-1">{{ word }}</h2>
|
||||
<ElButton text aria-label="关闭释义" @click="$emit('close')">关闭</ElButton>
|
||||
</header>
|
||||
<div v-if="phrase" class="lookup-range" data-testid="phrase-range">
|
||||
<p class="subtle">短语 · {{ phrase.wordCount }} 个单词{{ phrase.stored ? ' · 已保存' : '' }}</p>
|
||||
<div class="lookup-range-actions">
|
||||
<ElButton size="small" data-testid="range-start-left" :disabled="!adjust?.startLeft" @click="$emit('adjust', 'start', -1)">起点 ←</ElButton>
|
||||
<ElButton size="small" data-testid="range-start-right" :disabled="!adjust?.startRight" @click="$emit('adjust', 'start', 1)">起点 →</ElButton>
|
||||
<ElButton size="small" data-testid="range-end-left" :disabled="!adjust?.endLeft" @click="$emit('adjust', 'end', -1)">终点 ←</ElButton>
|
||||
<ElButton size="small" data-testid="range-end-right" :disabled="!adjust?.endRight" @click="$emit('adjust', 'end', 1)">终点 →</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
<div v-if="!phrase" class="lookup-content" aria-live="polite" :aria-busy="loading">
|
||||
<p v-if="loading" role="status" class="subtle">正在查询…</p>
|
||||
<p v-else-if="error" role="alert" class="lookup-message">{{ error }}</p>
|
||||
<template v-else-if="result">
|
||||
<p v-if="result.status === 'resource_missing'" class="lookup-message">词典资源暂不可用。</p>
|
||||
<p v-else-if="result.status === 'not_found'" class="lookup-message">未找到释义。</p>
|
||||
<p v-if="result.status === 'lemma'" class="subtle">词形匹配：{{ result.matchedForm }}（规则候选）</p>
|
||||
<ol v-if="result.entries.length" class="lookup-senses">
|
||||
<li v-for="(entry, index) in result.entries" :key="index">
|
||||
<p class="sense-heading"><strong>{{ entry.lemma }}</strong> <span>{{ entry.pos }}</span></p>
|
||||
<p lang="en">{{ entry.definition }}</p>
|
||||
<blockquote v-for="(example, exampleIndex) in entry.examples" :key="exampleIndex" lang="en">{{ example }}</blockquote>
|
||||
</li>
|
||||
</ol>
|
||||
<p v-if="result.resource" class="subtle">{{ result.resource.name }} · {{ result.resource.version }}</p>
|
||||
</template>
|
||||
<ElButton v-if="!loading && (error || result?.status === 'resource_missing' || result?.status === 'not_found')" data-testid="lookup-retry" @click="$emit('retry')">重试查询</ElButton>
|
||||
</div>
|
||||
<div class="lookup-term">
|
||||
<p class="lookup-term-title">学习状态</p>
|
||||
<ElRadioGroup v-model="status" :disabled="saving || prefilling || !!prefillError" aria-label="学习状态">
|
||||
<ElRadio v-for="item in TERM_STATUSES" :key="item.value" :value="item.value">{{ item.label }}</ElRadio>
|
||||
</ElRadioGroup>
|
||||
<p v-if="status === 'learning'" class="subtle">学习中按 1～7 级记录，等级由复习推进。</p>
|
||||
<label for="term-definition">我的释义 <span>{{ savedTermId ? '已保存' : '新词条' }}</span></label>
|
||||
<ElInput id="term-definition" v-model="definition" type="textarea" :rows="3" :disabled="saving || prefilling || !!prefillError" placeholder="记下当前词的释义" />
|
||||
<label for="term-examples">例句 <span>每行一条，最多 5 条</span></label>
|
||||
<ElInput id="term-examples" v-model="examples" type="textarea" :rows="2" :disabled="saving || prefilling || !!prefillError" placeholder="可选，每行一条" />
|
||||
<p v-if="prefilling" role="status" class="subtle">正在读取已保存的内容…</p>
|
||||
<p v-else-if="prefillError" role="alert" class="lookup-message">{{ prefillError }}</p>
|
||||
<p v-if="saved" role="status" class="lookup-saved">✓ {{ saved }}</p>
|
||||
<p v-if="saveError" role="alert" class="lookup-message">{{ saveError }}</p>
|
||||
<div class="lookup-actions">
|
||||
<ElButton type="primary" data-testid="term-save" :loading="saving" :disabled="!canSave" @click="$emit('save')">保存到生词本</ElButton>
|
||||
<ElButton text @click="$emit('close')">关闭，继续阅读</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
</aside>
|
||||
</template>
|
||||
@@ -0,0 +1,48 @@
|
||||
<script setup lang="ts">
|
||||
import { termStatusOf, type PhraseSpan, type ReaderToken } from '../composables/useReaderLookup'
|
||||
defineProps<{ tokens: ReaderToken[]; phrases: PhraseSpan[]; original: string; selectedStart?: number; phraseStart?: number | null }>()
|
||||
const emit = defineEmits<{
|
||||
select: [token: ReaderToken, element: HTMLElement, extend: boolean]
|
||||
selectPhrase: [span: PhraseSpan, element: HTMLElement]
|
||||
}>()
|
||||
|
||||
// A token belongs to the phrase that covers it; the spans come from the server match, so the
|
||||
// reader never re-derives which words form a phrase.
|
||||
function spanOf(phrases: PhraseSpan[], index: number): PhraseSpan | null {
|
||||
let best: PhraseSpan | null = null
|
||||
for (const phrase of phrases) {
|
||||
if (index < phrase.startToken || index > phrase.endToken) continue
|
||||
if (best === null || phrase.endToken - phrase.startToken > best.endToken - best.startToken) best = phrase
|
||||
}
|
||||
return best
|
||||
}
|
||||
|
||||
function select(token: ReaderToken, event: Event, phrase: PhraseSpan | null): void {
|
||||
const element = event.currentTarget as HTMLElement
|
||||
if (phrase && !(event as MouseEvent).shiftKey) {
|
||||
emit('selectPhrase', phrase, element)
|
||||
return
|
||||
}
|
||||
emit('select', token, element, (event as MouseEvent).shiftKey === true)
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<article class="reader-text"><template v-if="tokens.length"><template v-for="(token, index) in tokens" :key="token.start"><span
|
||||
class="reader-token"
|
||||
:data-token-index="index"
|
||||
:class="[
|
||||
token.kind === 'word' ? 'reader-word' : 'reader-separator',
|
||||
token.kind === 'word' && termStatusOf(token) ? `is-${termStatusOf(token)}` : '',
|
||||
spanOf(phrases, index) ? `is-phrase is-phrase-${spanOf(phrases, index)!.status}` : '',
|
||||
spanOf(phrases, index)?.startToken === index ? 'is-phrase-start' : '',
|
||||
spanOf(phrases, index)?.endToken === index ? 'is-phrase-end' : '',
|
||||
{ 'is-selected': token.kind === 'word' && selectedStart === token.start, 'is-phrase-selected': spanOf(phrases, index)?.id === phraseStart },
|
||||
]"
|
||||
v-bind="token.kind === 'word' ? { role: 'button', tabindex: '0', 'aria-pressed': selectedStart === token.start } : {}"
|
||||
:aria-label="token.kind === 'word' ? (spanOf(phrases, index) ? `短语中的 ${token.text}` : termStatusOf(token) ? `查询 ${token.text}，已保存` : `查询 ${token.text}`) : undefined"
|
||||
@click="select(token, $event, token.kind === 'word' ? spanOf(phrases, index) : null)"
|
||||
@keydown.enter.prevent="token.kind === 'word' ? select(token, $event, spanOf(phrases, index)) : undefined"
|
||||
@keydown.space.prevent="token.kind === 'word' ? select(token, $event, spanOf(phrases, index)) : undefined"
|
||||
>{{ token.text }}</span></template></template><template v-else>{{ original }}</template></article>
|
||||
</template>
|
||||
@@ -0,0 +1,49 @@
|
||||
<script setup lang="ts">
|
||||
import { nextTick, ref, watch } from 'vue'
|
||||
import { ElButton } from 'element-plus'
|
||||
import { maskedPrompt, type ReviewItem } from '../stores/review'
|
||||
|
||||
const props = defineProps<{ item: ReviewItem; position: number; total: number; revealed: boolean; busy: boolean; error: string }>()
|
||||
const emit = defineEmits<{ reveal: []; grade: [grade: 'correct' | 'wrong' | 'again']; end: []; retry: [] }>()
|
||||
// The word and its masked example are enough to answer; the definition stays hidden.
|
||||
const prompt = () => maskedPrompt(props.item)
|
||||
const revealButton = ref<{ $el?: HTMLElement } | null>(null)
|
||||
const correctButton = ref<{ $el?: HTMLElement } | null>(null)
|
||||
// Keyboard users should reach the next action without tabbing through the page again.
|
||||
watch(() => props.item.id, async () => {
|
||||
await nextTick()
|
||||
if (!props.revealed) revealButton.value?.$el?.focus()
|
||||
}, { immediate: true })
|
||||
watch(() => props.revealed, async value => {
|
||||
if (!value) return
|
||||
await nextTick()
|
||||
correctButton.value?.$el?.focus()
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section class="review-card" aria-labelledby="review-word" data-testid="review-card">
|
||||
<p class="subtle" data-testid="review-position">到期复习 · {{ position }} / {{ total }}</p>
|
||||
<h2 id="review-word" tabindex="-1" class="review-word">{{ item.originalForm }}</h2>
|
||||
<p v-if="prompt()" class="review-example" lang="en">{{ prompt() }}</p>
|
||||
<p class="subtle">{{ item.kind === 'phrase' ? `短语 · ${item.wordCount} 个单词 · ` : '' }}{{ item.status === 'learning' ? `学习中 · 等级 ${item.level}` : '新词 · 尚未复习' }}</p>
|
||||
<p v-if="error" role="alert" class="notice">{{ error }}</p>
|
||||
|
||||
<template v-if="!revealed">
|
||||
<ElButton ref="revealButton" type="primary" data-testid="review-reveal" :disabled="busy || !!error" @click="emit('reveal')">显示答案</ElButton>
|
||||
</template>
|
||||
<template v-else>
|
||||
<div class="review-answer">
|
||||
<p class="review-definition" data-testid="review-definition">{{ item.definition || '（未填写个人释义）' }}</p>
|
||||
<blockquote v-for="(example, index) in item.examples" :key="index" lang="en">{{ example }}</blockquote>
|
||||
</div>
|
||||
<div class="review-grades">
|
||||
<ElButton ref="correctButton" type="primary" data-testid="review-correct" :loading="busy" @click="emit('grade', 'correct')">认识 / 答对</ElButton>
|
||||
<ElButton data-testid="review-wrong" :disabled="busy" @click="emit('grade', 'wrong')">不认识 / 答错</ElButton>
|
||||
<ElButton data-testid="review-again" :disabled="busy" @click="emit('grade', 'again')">再学一次</ElButton>
|
||||
</div>
|
||||
</template>
|
||||
<p v-if="error" class="review-actions"><ElButton data-testid="review-retry" @click="emit('retry')">重试提交</ElButton></p>
|
||||
<p class="review-actions"><ElButton text data-testid="review-end" @click="emit('end')">结束本次复习</ElButton></p>
|
||||
</section>
|
||||
</template>
|
||||
@@ -0,0 +1,98 @@
|
||||
import type { ReaderToken } from './useReaderLookup'
|
||||
|
||||
/** A phrase selection over the server's own tokens, or a candidate the reader can adjust. */
|
||||
export interface TokenRange {
|
||||
/** Token indices of the first and last word of the phrase. */
|
||||
firstWord: number
|
||||
lastWord: number
|
||||
/** Code point range of the whole phrase inside the original text. */
|
||||
start: number
|
||||
end: number
|
||||
/** The original text exactly as the chapter holds it. */
|
||||
text: string
|
||||
wordCount: number
|
||||
}
|
||||
|
||||
export const MAX_PHRASE_WORDS = 12
|
||||
|
||||
/** Indices of the word tokens, in reading order. */
|
||||
export function wordIndices(tokens: ReaderToken[]): number[] {
|
||||
const indices: number[] = []
|
||||
for (let index = 0; index < tokens.length; index++) {
|
||||
if (tokens[index]!.kind === 'word') indices.push(index)
|
||||
}
|
||||
return indices
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a phrase range from two token indices, aligning both ends to whole words and
|
||||
* keeping interior punctuation and line breaks in the text. It mirrors the rules verified in
|
||||
* #4: separators at the edges are skipped, interior ones survive, and a single word is not a
|
||||
* phrase.
|
||||
*/
|
||||
export function normalizeTokenRange(tokens: ReaderToken[], original: string, anchor: number, focus: number): TokenRange | null {
|
||||
if (!Number.isInteger(anchor) || !Number.isInteger(focus)) return null
|
||||
const from = Math.min(anchor, focus)
|
||||
const to = Math.max(anchor, focus)
|
||||
if (from < 0 || to >= tokens.length) return null
|
||||
let firstWord = -1
|
||||
let lastWord = -1
|
||||
for (let index = from; index <= to; index++) {
|
||||
if (tokens[index]!.kind !== 'word') continue
|
||||
if (firstWord < 0) firstWord = index
|
||||
lastWord = index
|
||||
}
|
||||
if (firstWord < 0 || lastWord < 0 || firstWord === lastWord) return null
|
||||
return rangeOfWords(tokens, original, firstWord, lastWord)
|
||||
}
|
||||
|
||||
/** The phrase range for a known first and last word token. */
|
||||
export function rangeOfWords(tokens: ReaderToken[], original: string, firstWord: number, lastWord: number): TokenRange | null {
|
||||
const first = tokens[firstWord]
|
||||
const last = tokens[lastWord]
|
||||
if (!first || !last || first.kind !== 'word' || last.kind !== 'word' || lastWord < firstWord) return null
|
||||
const wordCount = wordIndices(tokens.slice(firstWord, lastWord + 1)).length
|
||||
if (wordCount < 2 || wordCount > MAX_PHRASE_WORDS) return null
|
||||
const text = [...original].slice(first.start, last.end).join('')
|
||||
if (!text) return null
|
||||
return { firstWord, lastWord, start: first.start, end: last.end, text, wordCount }
|
||||
}
|
||||
|
||||
/**
|
||||
* Moves one end of a phrase by whole words, skipping separators and never inverting the range.
|
||||
* Returns the adjusted range, or null when the move is impossible.
|
||||
*/
|
||||
export function adjustTokenRange(tokens: ReaderToken[], original: string, range: TokenRange, edge: 'start' | 'end', direction: -1 | 1): TokenRange | null {
|
||||
const words = wordIndices(tokens)
|
||||
const positionOf = (tokenIndex: number) => words.indexOf(tokenIndex)
|
||||
const firstPosition = positionOf(range.firstWord)
|
||||
const lastPosition = positionOf(range.lastWord)
|
||||
if (firstPosition < 0 || lastPosition < 0) return null
|
||||
if (edge === 'start') {
|
||||
const next = firstPosition + direction
|
||||
if (next < 0 || next > lastPosition - 1) return null
|
||||
return rangeOfWords(tokens, original, words[next]!, range.lastWord)
|
||||
}
|
||||
const next = lastPosition + direction
|
||||
if (next >= words.length || next < firstPosition + 1) return null
|
||||
return rangeOfWords(tokens, original, range.firstWord, words[next]!)
|
||||
}
|
||||
|
||||
/** The saved phrases that cover a word token, longest first, so a click prefers the phrase. */
|
||||
export function phrasesAt(tokens: ReaderToken[], phrases: { id: number; startToken: number; endToken: number }[], tokenIndex: number): { id: number; startToken: number; endToken: number } | null {
|
||||
let best: { id: number; startToken: number; endToken: number } | null = null
|
||||
for (const phrase of phrases) {
|
||||
if (tokenIndex < phrase.startToken || tokenIndex > phrase.endToken) continue
|
||||
if (best === null || phrase.endToken - phrase.startToken > best.endToken - best.startToken) best = phrase
|
||||
}
|
||||
return best
|
||||
}
|
||||
|
||||
/** The phrase range a saved span covers, used when a highlighted phrase is clicked. */
|
||||
export function rangeOfSpan(tokens: ReaderToken[], original: string, span: { startToken: number; endToken: number }): TokenRange | null {
|
||||
const words = wordIndices(tokens.slice(span.startToken, span.endToken + 1))
|
||||
if (words.length < 2) return null
|
||||
const firstWord = span.startToken + words[0]!
|
||||
const lastWord = span.startToken + words[words.length - 1]!
|
||||
return rangeOfWords(tokens, original, firstWord, lastWord)
|
||||
}
|
||||
@@ -0,0 +1,317 @@
|
||||
import { computed, onScopeDispose, ref, watch, type Ref } from 'vue'
|
||||
import type { ChapterDetail } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
import { adjustTokenRange, type TokenRange } from './readerRange'
|
||||
|
||||
export type TermStatus = 'new' | 'learning' | 'known' | 'ignored'
|
||||
|
||||
export interface TokenTerm { id: number; status: TermStatus; level: number }
|
||||
export interface ReaderToken {
|
||||
text: string
|
||||
start: number
|
||||
end: number
|
||||
startUtf16: number
|
||||
endUtf16: number
|
||||
kind: 'word' | 'space' | 'punctuation'
|
||||
term?: TokenTerm | null
|
||||
}
|
||||
/** One saved phrase occurrence inside the chapter, in token indices. */
|
||||
export interface PhraseSpan { id: number; status: TermStatus; wordCount: number; startToken: number; endToken: number }
|
||||
export interface LookupResult {
|
||||
status: 'exact' | 'lemma' | 'not_found' | 'resource_missing'
|
||||
query: string
|
||||
matchedForm: string | null
|
||||
candidates: string[]
|
||||
entries: { lemma: string; pos: string; definition: string; examples: string[] }[]
|
||||
resource?: { name: string; version: string }
|
||||
}
|
||||
export interface SavedTerm {
|
||||
id: number
|
||||
term: string
|
||||
originalForm: string
|
||||
definition: string
|
||||
examples: string[]
|
||||
status: TermStatus
|
||||
level: number
|
||||
kind: 'word' | 'phrase'
|
||||
wordCount: number
|
||||
}
|
||||
interface TermResponse { term: SavedTerm }
|
||||
interface TokenResponse { textSha256: string; tokens: ReaderToken[]; phrases?: PhraseSpan[] }
|
||||
|
||||
// The four learner-visible states and the level rule behind them. Only a learning
|
||||
// entry carries a level, so every other status reports level 0.
|
||||
export const TERM_STATUSES: { value: TermStatus; label: string }[] = [
|
||||
{ value: 'new', label: '新词' },
|
||||
{ value: 'learning', label: '学习中' },
|
||||
{ value: 'known', label: '已知' },
|
||||
{ value: 'ignored', label: '忽略' },
|
||||
]
|
||||
|
||||
export function termStatusLabel(status: TermStatus): string {
|
||||
return TERM_STATUSES.find(item => item.value === status)?.label ?? status
|
||||
}
|
||||
|
||||
/** Unknown or malformed personal state never becomes a highlight class. */
|
||||
export function termStatusOf(token: ReaderToken): TermStatus | null {
|
||||
const term = token.term
|
||||
if (!term || typeof term.id !== 'number' || term.id <= 0) return null
|
||||
return TERM_STATUSES.some(item => item.value === term.status) ? term.status : null
|
||||
}
|
||||
|
||||
// Validate every coordinate before enabling selection; original text is always the fallback.
|
||||
function matchesChapter(data: TokenResponse, chapter: ChapterDetail): boolean {
|
||||
if (!data || data.textSha256 !== chapter.contentSha256 || !Array.isArray(data.tokens)) return false
|
||||
const text = chapter.originalText ?? ''
|
||||
let cp = 0, utf16 = 0
|
||||
for (const token of data.tokens) {
|
||||
if (typeof token.text !== 'string' || !token.text || !['word', 'space', 'punctuation'].includes(token.kind)) return false
|
||||
if (token.start !== cp || token.startUtf16 !== utf16) return false
|
||||
cp += [...token.text].length
|
||||
utf16 += token.text.length
|
||||
if (token.end !== cp || token.endUtf16 !== utf16 || text.slice(token.startUtf16, utf16) !== token.text) return false
|
||||
}
|
||||
return utf16 === text.length && data.tokens.map(token => token.text).join('') === text
|
||||
}
|
||||
|
||||
export function useReaderLookup(chapter: Ref<ChapterDetail | null>) {
|
||||
const session = useSessionStore()
|
||||
const tokens = ref<ReaderToken[]>([])
|
||||
const phrases = ref<PhraseSpan[]>([])
|
||||
const tokensError = ref('')
|
||||
const tokensLoading = ref(false)
|
||||
const selected = ref<ReaderToken | null>(null)
|
||||
// A phrase selection: either a drag over several words or a click on a saved phrase span.
|
||||
const range = ref<TokenRange | null>(null)
|
||||
const rangeTermId = ref<number | null>(null)
|
||||
const result = ref<LookupResult | null>(null)
|
||||
const loading = ref(false)
|
||||
const error = ref('')
|
||||
|
||||
// The learner's own record for the selected word.
|
||||
const definition = ref('')
|
||||
const examples = ref('')
|
||||
const status = ref<TermStatus>('new')
|
||||
const savedTermId = ref<number | null>(null)
|
||||
const prefilling = ref(false)
|
||||
const prefillError = ref('')
|
||||
const saving = ref(false)
|
||||
const saveError = ref('')
|
||||
const saved = ref('')
|
||||
|
||||
let tokenSequence = 0, lookupSequence = 0, termSequence = 0, saveSequence = 0
|
||||
let origin: HTMLElement | null = null
|
||||
let scrollBeforeAdjustment: { top: number; left: number } | null = null
|
||||
let lastAdjustment: { top: number; left: number } | null = null
|
||||
|
||||
// Sending is blocked until the stored text is on screen, so a slow or failed read
|
||||
// can never let an empty form overwrite what the learner already wrote.
|
||||
const canSave = computed(() => (selected.value !== null || range.value !== null) && !saving.value && !prefilling.value && !prefillError.value)
|
||||
|
||||
function exampleLines(): string[] {
|
||||
return examples.value.split('\n').map(line => line.trim()).filter(Boolean)
|
||||
}
|
||||
|
||||
function applyTerm(next: TokenTerm) {
|
||||
savedTermId.value = next.id
|
||||
const index = tokens.value.findIndex(token => token.start === selected.value?.start)
|
||||
const target = index >= 0 ? tokens.value[index] : undefined
|
||||
if (target) tokens.value[index] = { ...target, term: next }
|
||||
}
|
||||
|
||||
function stillAtAdjustment() {
|
||||
return lastAdjustment !== null && Math.abs(window.scrollY - lastAdjustment.top) <= 2
|
||||
&& Math.abs(window.scrollX - lastAdjustment.left) <= 2
|
||||
}
|
||||
|
||||
function keepSelectionVisible(panelTop: number) {
|
||||
if (!origin) return
|
||||
const coveredBy = origin.getBoundingClientRect().bottom - panelTop + 16
|
||||
if (coveredBy <= 0) return
|
||||
// A later manual scroll becomes the new reading position, including when
|
||||
// another resize subsequently needs to reveal the selected word again.
|
||||
if (!scrollBeforeAdjustment || !stillAtAdjustment()) scrollBeforeAdjustment = { top: window.scrollY, left: window.scrollX }
|
||||
window.scrollBy({ top: coveredBy, behavior: 'instant' })
|
||||
lastAdjustment = { top: window.scrollY, left: window.scrollX }
|
||||
}
|
||||
|
||||
function close(restoreFocus = true) {
|
||||
lookupSequence++
|
||||
termSequence++
|
||||
saveSequence++
|
||||
selected.value = null
|
||||
range.value = null
|
||||
rangeTermId.value = null
|
||||
result.value = null
|
||||
loading.value = false
|
||||
error.value = ''
|
||||
definition.value = ''
|
||||
examples.value = ''
|
||||
status.value = 'new'
|
||||
savedTermId.value = null
|
||||
prefilling.value = false
|
||||
prefillError.value = ''
|
||||
saving.value = false
|
||||
saveError.value = ''
|
||||
saved.value = ''
|
||||
if (restoreFocus && origin?.isConnected) origin.focus({ preventScroll: true })
|
||||
if (restoreFocus && scrollBeforeAdjustment && stillAtAdjustment()) window.scrollTo({ ...scrollBeforeAdjustment, behavior: 'instant' })
|
||||
scrollBeforeAdjustment = null
|
||||
lastAdjustment = null
|
||||
origin = null
|
||||
}
|
||||
function reset() {
|
||||
tokenSequence++
|
||||
tokens.value = []
|
||||
phrases.value = []
|
||||
tokensError.value = ''
|
||||
tokensLoading.value = false
|
||||
close(false)
|
||||
}
|
||||
async function loadTokens() {
|
||||
const current = chapter.value
|
||||
if (!session.user || current?.status !== 'ready') return
|
||||
const seq = ++tokenSequence
|
||||
tokensLoading.value = true
|
||||
tokensError.value = ''
|
||||
try {
|
||||
const data = await session.request<TokenResponse>(`chapters/${current.id}/tokens`)
|
||||
if (seq !== tokenSequence) return
|
||||
if (!matchesChapter(data, current)) throw new Error('分词与正文不一致，请重试。')
|
||||
tokens.value = data.tokens
|
||||
phrases.value = Array.isArray(data.phrases) ? data.phrases : []
|
||||
} catch (reason) {
|
||||
if (seq !== tokenSequence) return
|
||||
tokens.value = []
|
||||
phrases.value = []
|
||||
tokensError.value = reason instanceof Error ? reason.message : '单词暂时无法加载。'
|
||||
} finally { if (seq === tokenSequence) tokensLoading.value = false }
|
||||
}
|
||||
async function lookup() {
|
||||
const current = chapter.value
|
||||
const token = selected.value
|
||||
if (!session.user || !current || !token || range.value) return
|
||||
const seq = ++lookupSequence
|
||||
result.value = null
|
||||
error.value = ''
|
||||
loading.value = true
|
||||
try {
|
||||
const data = await session.request<LookupResult>('lookup', 'POST', { chapterId: current.id, start: token.start, end: token.end })
|
||||
if (seq === lookupSequence) result.value = data
|
||||
} catch (reason) {
|
||||
if (seq === lookupSequence) error.value = reason instanceof Error ? reason.message : '暂时无法查询，请重试。'
|
||||
} finally { if (seq === lookupSequence) loading.value = false }
|
||||
}
|
||||
// Opening a word the learner already saved loads that record, so the panel shows
|
||||
// the stored text and the same id other chapters show.
|
||||
async function loadTerm(id: number) {
|
||||
const seq = ++termSequence
|
||||
prefilling.value = true
|
||||
prefillError.value = ''
|
||||
try {
|
||||
const data = await session.request<TermResponse>(`terms/${id}`)
|
||||
if (seq !== termSequence) return
|
||||
definition.value = data.term.definition
|
||||
examples.value = data.term.examples.join('\n')
|
||||
status.value = data.term.status
|
||||
applyTerm({ id: data.term.id, status: data.term.status, level: data.term.level })
|
||||
} catch (reason) {
|
||||
if (seq !== termSequence) return
|
||||
prefillError.value = reason instanceof Error ? reason.message : '已保存的内容暂时无法读取。'
|
||||
} finally { if (seq === termSequence) prefilling.value = false }
|
||||
}
|
||||
async function save() {
|
||||
const current = chapter.value
|
||||
const selection = range.value
|
||||
const token = selected.value
|
||||
if (!session.user || !current || (!selection && !token) || !canSave.value) return
|
||||
const seq = ++saveSequence
|
||||
saving.value = true
|
||||
saveError.value = ''
|
||||
saved.value = ''
|
||||
const span = selection ?? token!
|
||||
const path = selection ? 'phrases' : 'terms'
|
||||
try {
|
||||
const data = await session.request<TermResponse & { created: boolean }>(path, 'POST', {
|
||||
chapterId: current.id, start: span.start, end: span.end,
|
||||
definition: definition.value, examples: exampleLines(), status: status.value,
|
||||
})
|
||||
if (seq !== saveSequence) return
|
||||
definition.value = data.term.definition
|
||||
examples.value = data.term.examples.join('\n')
|
||||
status.value = data.term.status
|
||||
applyTerm({ id: data.term.id, status: data.term.status, level: data.term.level })
|
||||
if (selection) {
|
||||
// The saved phrase must highlight here without waiting for another token load.
|
||||
rangeTermId.value = data.term.id
|
||||
const savedSpan = {
|
||||
id: data.term.id, status: data.term.status, wordCount: selection.wordCount,
|
||||
startToken: selection.firstWord, endToken: selection.lastWord,
|
||||
}
|
||||
const existing = phrases.value.findIndex(item => item.id === data.term.id)
|
||||
phrases.value = existing >= 0
|
||||
? phrases.value.map((item, index) => (index === existing ? savedSpan : item))
|
||||
: [...phrases.value, savedSpan]
|
||||
}
|
||||
saved.value = `已保存 · ${termStatusLabel(data.term.status)}`
|
||||
} catch (reason) {
|
||||
if (seq !== saveSequence) return
|
||||
saveError.value = reason instanceof Error ? reason.message : '保存失败，请稍后重试。'
|
||||
} finally { if (seq === saveSequence) saving.value = false }
|
||||
}
|
||||
function select(token: ReaderToken, element: HTMLElement) {
|
||||
close(false)
|
||||
origin = element
|
||||
selected.value = token
|
||||
const existing = termStatusOf(token)
|
||||
if (existing !== null && token.term) {
|
||||
status.value = existing
|
||||
void loadTerm(token.term.id)
|
||||
}
|
||||
void lookup()
|
||||
}
|
||||
|
||||
/**
|
||||
* Opens the phrase form for a dragged range or for a saved phrase span. A phrase is one
|
||||
* entry in the same table as a word, so an existing phrase is read through the same route.
|
||||
*/
|
||||
function selectRange(next: TokenRange, element: HTMLElement | null, termId: number | null = null) {
|
||||
close(false)
|
||||
origin = element
|
||||
range.value = next
|
||||
rangeTermId.value = termId
|
||||
if (termId !== null) void loadTerm(termId)
|
||||
}
|
||||
|
||||
/** Moves one end of the phrase by whole words; the panel keeps the selection in sync. */
|
||||
function adjustRange(edge: 'start' | 'end', direction: -1 | 1) {
|
||||
const current = range.value
|
||||
const currentChapter = chapter.value
|
||||
if (!current || !currentChapter) return
|
||||
const next = adjustTokenRange(tokens.value, currentChapter.originalText ?? '', current, edge, direction)
|
||||
if (!next) return
|
||||
// The adjusted range is a different identity, so the panel must stop claiming that the
|
||||
// stored entry is what is on screen: both the phrase link and the entry the form was loaded
|
||||
// from are dropped. The typed text stays, and saving it writes the phrase that matches the
|
||||
// current range (which may update another entry, never two).
|
||||
rangeTermId.value = null
|
||||
savedTermId.value = null
|
||||
range.value = next
|
||||
}
|
||||
|
||||
/** True when the panel is editing an entry that is already stored. */
|
||||
const rangeStored = computed(() => rangeTermId.value !== null)
|
||||
watch(() => [chapter.value?.id, chapter.value?.status, chapter.value?.contentSha256, chapter.value?.originalText], () => {
|
||||
reset()
|
||||
void loadTokens()
|
||||
}, { immediate: true, flush: 'sync' })
|
||||
// Watch the identity object, including clear → login for the same account: leaving
|
||||
// or switching an account drops every word saved by the previous one.
|
||||
watch(() => session.user, reset, { flush: 'sync' })
|
||||
onScopeDispose(reset)
|
||||
return {
|
||||
tokens, phrases, tokensError, tokensLoading, selected, range, rangeTermId, rangeStored, result, loading, error,
|
||||
definition, examples, status, savedTermId, prefilling, prefillError, saving, saveError, saved, canSave,
|
||||
loadTokens, lookup, loadTerm, save, select, selectRange, adjustRange, close, reset, keepSelectionVisible,
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
import { onScopeDispose, type Ref } from 'vue'
|
||||
|
||||
/**
|
||||
* Maps the browser's own selection onto the server tokens. Nothing here intercepts touch or
|
||||
* pointer movement: a mouse drag and the phone's system selection handles both produce a
|
||||
* selectionchange, which is the interaction #4 verified.
|
||||
*/
|
||||
export function tokenIndexOf(node: Node | null): number {
|
||||
if (!node) return -1
|
||||
const element = node.nodeType === Node.TEXT_NODE ? node.parentElement : (node as Element)
|
||||
const holder = element?.closest?.('[data-token-index]')
|
||||
if (!holder) return -1
|
||||
const value = Number((holder as HTMLElement).dataset.tokenIndex)
|
||||
return Number.isInteger(value) ? value : -1
|
||||
}
|
||||
|
||||
export interface TextSelectionOptions {
|
||||
/** The element the selection has to start inside. */
|
||||
container: Ref<HTMLElement | null>
|
||||
/** Called with the first and last token index of a non-empty selection inside the reader. */
|
||||
onSelect: (anchor: number, focus: number) => void
|
||||
}
|
||||
|
||||
/**
|
||||
* Watches the native selection and reports token indices. A collapsed selection (a plain
|
||||
* click) reports nothing, so clicking a word keeps opening the word panel.
|
||||
*/
|
||||
export function useTextSelection(options: TextSelectionOptions) {
|
||||
let timer: number | undefined
|
||||
let disposed = false
|
||||
|
||||
function read(): void {
|
||||
if (disposed) return
|
||||
const root = options.container.value
|
||||
const selection = window.getSelection?.()
|
||||
if (!root || !selection || selection.rangeCount === 0 || selection.isCollapsed) return
|
||||
const range = selection.getRangeAt(0)
|
||||
if (!root.contains(range.startContainer) || !root.contains(range.endContainer)) return
|
||||
const anchor = tokenIndexOf(range.startContainer)
|
||||
const focus = tokenIndexOf(range.endContainer)
|
||||
if (anchor < 0 || focus < 0) return
|
||||
options.onSelect(Math.min(anchor, focus), Math.max(anchor, focus))
|
||||
}
|
||||
|
||||
// The debounce matches the verified spike: it lets the browser finish a drag before the
|
||||
// range is read, and it never blocks scrolling on a touch device.
|
||||
function schedule(): void {
|
||||
if (timer !== undefined) window.clearTimeout(timer)
|
||||
timer = window.setTimeout(() => { timer = undefined; read() }, 100)
|
||||
}
|
||||
|
||||
function pointerUp(event: PointerEvent): void {
|
||||
// Only a pointerup inside the reader can finish a selection; a click on the panel must not
|
||||
// re-read a range the learner already adjusted there.
|
||||
const root = options.container.value
|
||||
const target = event.target as Node | null
|
||||
if (!root || !target || !root.contains(target)) return
|
||||
if (timer !== undefined) window.clearTimeout(timer)
|
||||
// A pointerup arrives before the browser finalises the range, so the read waits a tick.
|
||||
timer = window.setTimeout(() => { timer = undefined; read() }, 0)
|
||||
}
|
||||
|
||||
// Both listeners live on the document: the reader body only exists once a chapter is ready,
|
||||
// so a listener bound to the element at setup time would miss every later selection.
|
||||
document.addEventListener('selectionchange', schedule)
|
||||
document.addEventListener('pointerup', pointerUp)
|
||||
|
||||
onScopeDispose(() => {
|
||||
disposed = true
|
||||
if (timer !== undefined) window.clearTimeout(timer)
|
||||
document.removeEventListener('selectionchange', schedule)
|
||||
document.removeEventListener('pointerup', pointerUp)
|
||||
})
|
||||
|
||||
return {
|
||||
/** Drops the native highlight, e.g. when the panel closes. */
|
||||
clear() { window.getSelection?.()?.removeAllRanges() },
|
||||
read,
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,10 @@ import App from './App.vue'
|
||||
import router from './router'
|
||||
import 'element-plus/es/components/button/style/css'
|
||||
import 'element-plus/es/components/input/style/css'
|
||||
import 'element-plus/es/components/radio/style/css'
|
||||
import 'element-plus/es/components/radio-group/style/css'
|
||||
import 'element-plus/es/components/select/style/css'
|
||||
import 'element-plus/es/components/option/style/css'
|
||||
import './style.css'
|
||||
|
||||
createApp(App).use(createPinia()).use(router).mount('#app')
|
||||
|
||||
@@ -6,6 +6,10 @@ const router = createRouter({
|
||||
routes: [
|
||||
{ path: '/login', name: 'login', component: () => import('../views/LoginView.vue') },
|
||||
{ path: '/', name: 'library', meta: { private: true }, component: () => import('../views/LibraryView.vue') },
|
||||
{ path: '/review', name: 'review', meta: { private: true }, component: () => import('../views/ReviewView.vue') },
|
||||
{ path: '/import', name: 'import', meta: { private: true }, component: () => import('../views/ImportView.vue') },
|
||||
{ path: '/books/:id', name: 'book', meta: { private: true }, component: () => import('../views/BookView.vue') },
|
||||
{ path: '/chapters/:id', name: 'chapter', meta: { private: true }, component: () => import('../views/ReaderView.vue') },
|
||||
{ path: '/:pathMatch(.*)*', redirect: '/' },
|
||||
],
|
||||
})
|
||||
|
||||
@@ -0,0 +1,621 @@
|
||||
import { defineStore } from 'pinia'
|
||||
import { computed, ref, watch } from 'vue'
|
||||
import { ApiError, useSessionStore } from './session'
|
||||
|
||||
export type ChapterStatus = 'pending' | 'processing' | 'ready' | 'failed'
|
||||
|
||||
export interface BookRef { id: number; title: string; language: string }
|
||||
|
||||
export interface BookSummary extends BookRef {
|
||||
chapterCount: number
|
||||
pendingCount: number
|
||||
processingCount: number
|
||||
readyCount: number
|
||||
failedCount: number
|
||||
createdAt: string
|
||||
updatedAt: string
|
||||
}
|
||||
|
||||
export interface ChapterSummary {
|
||||
id: number
|
||||
bookId: number
|
||||
ordinal: number
|
||||
title: string
|
||||
status: ChapterStatus
|
||||
charCount: number
|
||||
errorReason: string
|
||||
errorMessage: string
|
||||
// Present wherever a chapter appears; null while the job id is unknown.
|
||||
jobId: number | null
|
||||
createdAt: string
|
||||
updatedAt: string
|
||||
}
|
||||
|
||||
export interface ChapterDetail extends ChapterSummary {
|
||||
contentSha256: string
|
||||
// Present only for ready chapters; never cached or faked for other statuses.
|
||||
originalText?: string
|
||||
}
|
||||
|
||||
export interface Job {
|
||||
id: number
|
||||
bookId: number
|
||||
chapterId: number
|
||||
status: ChapterStatus
|
||||
attempts: number
|
||||
errorReason: string
|
||||
errorMessage: string
|
||||
createdAt: string
|
||||
updatedAt: string
|
||||
}
|
||||
|
||||
export interface ChapterNavigation { previousChapterId: number | null; nextChapterId: number | null }
|
||||
|
||||
/** The editable text of one owned chapter; separate from the ready-only reader payload. */
|
||||
export interface ChapterSource {
|
||||
id: number
|
||||
bookId: number
|
||||
ordinal: number
|
||||
title: string
|
||||
text: string
|
||||
status: ChapterStatus
|
||||
contentSha256: string
|
||||
charCount: number
|
||||
}
|
||||
|
||||
export interface ChapterEdit { chapter: ChapterSummary; job: Job | null; versionChanged: boolean }
|
||||
export interface DeletionResult { bookId?: number; chapterId?: number; chapters?: number; remaining: number }
|
||||
|
||||
export type SubmitTarget = { mode: 'new' } | { mode: 'append'; bookId: number }
|
||||
|
||||
export interface SubmitInput { title: string; text: string; target: SubmitTarget }
|
||||
|
||||
export const POLL_INTERVAL_MS = 1500
|
||||
export const TITLE_MAX_LENGTH = 120
|
||||
export const TEXT_MAX_CODE_POINTS = 100000
|
||||
// Mirrors the server limit for one TXT upload.
|
||||
export const TXT_MAX_BYTES = 2 * 1024 * 1024
|
||||
export const NOT_FOUND_MESSAGE = '内容不存在。'
|
||||
export const LANGUAGE_LABEL = '英语'
|
||||
export const LANGUAGE_CODE = 'en'
|
||||
|
||||
const CHAPTER_STATUS_LABELS: Record<ChapterStatus, string> = {
|
||||
pending: '待处理',
|
||||
processing: '处理中',
|
||||
ready: '已就绪',
|
||||
failed: '处理失败',
|
||||
}
|
||||
|
||||
export function statusLabel(status: ChapterStatus): string {
|
||||
return CHAPTER_STATUS_LABELS[status]
|
||||
}
|
||||
|
||||
function isUnsettled(status: ChapterStatus): boolean {
|
||||
return status === 'pending' || status === 'processing'
|
||||
}
|
||||
|
||||
/** Mirrors the server rule: non-empty after trim and at most 120 characters. */
|
||||
export function titleProblem(title: string): string {
|
||||
const trimmed = title.trim()
|
||||
if (!trimmed) return '请填写标题。'
|
||||
if ([...trimmed].length > TITLE_MAX_LENGTH) return `标题不能超过 ${TITLE_MAX_LENGTH} 个字符。`
|
||||
return ''
|
||||
}
|
||||
|
||||
/** Mirrors the server rule: at least one non-whitespace character, at most 100000 code points. */
|
||||
export function textProblem(text: string): string {
|
||||
if (!text.trim()) return '请粘贴要导入的英文正文。'
|
||||
if ([...text].length > TEXT_MAX_CODE_POINTS) return `正文不能超过 ${TEXT_MAX_CODE_POINTS} 个字符。`
|
||||
return ''
|
||||
}
|
||||
|
||||
/**
|
||||
* Client-side pre-check for a TXT upload. The server validates the file again and stays the
|
||||
* only authority; this only tells the learner about an obviously unusable choice earlier.
|
||||
*/
|
||||
export function fileProblem(file: { name: string; size: number }): string {
|
||||
if (!/\.txt$/i.test(file.name)) return '请选择 .txt 文件。'
|
||||
if (file.size === 0) return '文件是空的，请选择包含英文正文的 UTF-8 TXT。'
|
||||
if (file.size > TXT_MAX_BYTES) return 'TXT 文件不能超过 2 MiB。'
|
||||
return ''
|
||||
}
|
||||
|
||||
/** A readable size for the selected file, e.g. `2 KB` or `1.5 MB`. */
|
||||
export function fileSizeLabel(bytes: number): string {
|
||||
if (bytes < 1024) return `${bytes} B`
|
||||
if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)} KB`
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
|
||||
}
|
||||
|
||||
/** A failed chapter can be retried as soon as the API told us its job id. */
|
||||
export function canRetry(chapter: Pick<ChapterSummary, 'status' | 'jobId'>): boolean {
|
||||
return chapter.status === 'failed' && chapter.jobId !== null
|
||||
}
|
||||
|
||||
/** Compact one-line status summary for a book card, e.g. `已就绪 2 · 处理中 1 · 待处理 1 · 失败 1`. */
|
||||
export function statusSummary(book: BookSummary): string {
|
||||
const parts: string[] = []
|
||||
if (book.readyCount > 0) parts.push(`已就绪 ${book.readyCount}`)
|
||||
if (book.processingCount > 0) parts.push(`处理中 ${book.processingCount}`)
|
||||
if (book.pendingCount > 0) parts.push(`待处理 ${book.pendingCount}`)
|
||||
if (book.failedCount > 0) parts.push(`失败 ${book.failedCount}`)
|
||||
return parts.join(' · ')
|
||||
}
|
||||
|
||||
interface LoadOptions { silent?: boolean }
|
||||
interface SubmitBookBody { requestId: string; title: string; text: string; language: 'en' }
|
||||
// An appended chapter owns the language of its book, so the append contract has no language
|
||||
// field; the server rejects unknown fields, and a client that sends one gets HTTP 400.
|
||||
interface SubmitChapterBody { requestId: string; title: string; text: string }
|
||||
export interface UploadInput { title: string; target: SubmitTarget; file: File }
|
||||
interface Created { bookId: number; chapter: ChapterSummary }
|
||||
|
||||
function emptyNavigation(): ChapterNavigation {
|
||||
return { previousChapterId: null, nextChapterId: null }
|
||||
}
|
||||
|
||||
export const useLibraryStore = defineStore('library', () => {
|
||||
const session = useSessionStore()
|
||||
|
||||
const books = ref<BookSummary[]>([])
|
||||
const booksLoading = ref(false)
|
||||
const booksError = ref('')
|
||||
|
||||
const book = ref<BookRef | null>(null)
|
||||
const chapters = ref<ChapterSummary[]>([])
|
||||
const bookLoading = ref(false)
|
||||
const bookError = ref('')
|
||||
|
||||
const chapter = ref<ChapterDetail | null>(null)
|
||||
const chapterBook = ref<BookRef | null>(null)
|
||||
const navigation = ref<ChapterNavigation>(emptyNavigation())
|
||||
const chapterLoading = ref(false)
|
||||
const chapterError = ref('')
|
||||
|
||||
const submitting = ref(false)
|
||||
const submitError = ref('')
|
||||
const retryingChapterId = ref<number | null>(null)
|
||||
|
||||
/** Reader text exists only for ready chapters and is never taken from a cache. */
|
||||
const readerText = computed(() => (chapter.value?.status === 'ready' ? chapter.value.originalText ?? '' : ''))
|
||||
|
||||
// Every request is tagged with a generation and the owning account so that a
|
||||
// late response can never repopulate the view after logout or an account switch.
|
||||
let generation = 0
|
||||
let booksSeq = 0
|
||||
let bookSeq = 0
|
||||
let chapterSeq = 0
|
||||
let pollTimer: number | undefined
|
||||
// One requestId per unsent form content: a double click or a repeat submit of
|
||||
// unchanged content must create one chapter, not two.
|
||||
let submissionKey = ''
|
||||
let submissionRequestId = ''
|
||||
|
||||
watch(() => session.user?.id ?? null, (next, previous) => {
|
||||
if (next !== previous) reset()
|
||||
}, { flush: 'sync' })
|
||||
|
||||
function ownerId(): number | null {
|
||||
return session.user?.id ?? null
|
||||
}
|
||||
|
||||
function isStale(version: number, owner: number | null): boolean {
|
||||
return version !== generation || ownerId() !== owner
|
||||
}
|
||||
|
||||
function failureMessage(reason: unknown, fallback: string): string {
|
||||
return reason instanceof Error && reason.message ? reason.message : fallback
|
||||
}
|
||||
|
||||
function isNotFound(reason: unknown): boolean {
|
||||
return reason instanceof ApiError && reason.status === 404
|
||||
}
|
||||
|
||||
function stopPolling(): void {
|
||||
if (pollTimer !== undefined) {
|
||||
window.clearTimeout(pollTimer)
|
||||
pollTimer = undefined
|
||||
}
|
||||
}
|
||||
|
||||
function needsPolling(): boolean {
|
||||
return (book.value !== null && chapters.value.some(item => isUnsettled(item.status)))
|
||||
|| (chapter.value !== null && isUnsettled(chapter.value.status))
|
||||
}
|
||||
|
||||
/** Schedules the next refresh, or stops polling when nothing is pending anymore. */
|
||||
function schedulePolling(): void {
|
||||
if (!needsPolling()) {
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
if (pollTimer !== undefined || !session.user) return
|
||||
pollTimer = window.setTimeout(() => {
|
||||
pollTimer = undefined
|
||||
void poll()
|
||||
}, POLL_INTERVAL_MS)
|
||||
}
|
||||
|
||||
async function poll(): Promise<void> {
|
||||
if (!session.user) {
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const bookId = book.value !== null && chapters.value.some(item => isUnsettled(item.status)) ? book.value.id : null
|
||||
const chapterId = chapter.value !== null && isUnsettled(chapter.value.status) ? chapter.value.id : null
|
||||
if (bookId !== null) await loadBook(bookId, { silent: true })
|
||||
if (chapterId !== null) await loadChapter(chapterId, { silent: true })
|
||||
if (isStale(version, owner)) {
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
schedulePolling()
|
||||
}
|
||||
|
||||
async function loadBooks(): Promise<void> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = ++booksSeq
|
||||
booksLoading.value = true
|
||||
booksError.value = ''
|
||||
try {
|
||||
const result = await session.request<{ items?: BookSummary[] } | null>('books')
|
||||
if (seq !== booksSeq || isStale(version, owner)) return
|
||||
const items = result?.items
|
||||
books.value = Array.isArray(items) ? items : []
|
||||
} catch (reason) {
|
||||
if (seq !== booksSeq || isStale(version, owner)) return
|
||||
booksError.value = failureMessage(reason, '书库暂时无法加载，请稍后重试。')
|
||||
} finally {
|
||||
if (seq === booksSeq && !isStale(version, owner)) booksLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function loadBook(id: number, options: LoadOptions = {}): Promise<void> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = ++bookSeq
|
||||
if (!options.silent) {
|
||||
bookLoading.value = true
|
||||
bookError.value = ''
|
||||
}
|
||||
try {
|
||||
const result = await session.request<{ book: BookRef; chapters?: ChapterSummary[] }>(`books/${id}`)
|
||||
if (seq !== bookSeq || isStale(version, owner)) return
|
||||
book.value = result.book
|
||||
chapters.value = Array.isArray(result.chapters) ? result.chapters : []
|
||||
bookError.value = ''
|
||||
schedulePolling()
|
||||
} catch (reason) {
|
||||
if (seq !== bookSeq || isStale(version, owner)) return
|
||||
if (isNotFound(reason)) {
|
||||
// Another account's id never resolves for this caller: report it and stop
|
||||
// instead of polling a resource that will not appear.
|
||||
book.value = null
|
||||
chapters.value = []
|
||||
bookError.value = NOT_FOUND_MESSAGE
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
// A failed background refresh keeps the data already on screen; the next
|
||||
// tick tries again and the user still sees the last known state.
|
||||
if (!options.silent) bookError.value = failureMessage(reason, '书籍暂时无法加载，请稍后重试。')
|
||||
} finally {
|
||||
if (seq === bookSeq && !isStale(version, owner)) bookLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function loadChapter(id: number, options: LoadOptions = {}): Promise<void> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = ++chapterSeq
|
||||
if (!options.silent) {
|
||||
chapterLoading.value = true
|
||||
chapterError.value = ''
|
||||
// Never keep the previous chapter's text under a new chapter id.
|
||||
if (chapter.value !== null && chapter.value.id !== id) {
|
||||
chapter.value = null
|
||||
chapterBook.value = null
|
||||
navigation.value = emptyNavigation()
|
||||
}
|
||||
}
|
||||
try {
|
||||
const result = await session.request<{ book: BookRef; chapter: ChapterDetail; navigation?: ChapterNavigation }>(`chapters/${id}`)
|
||||
if (seq !== chapterSeq || isStale(version, owner)) return
|
||||
chapterBook.value = result.book
|
||||
chapter.value = result.chapter
|
||||
navigation.value = result.navigation ?? emptyNavigation()
|
||||
chapterError.value = ''
|
||||
schedulePolling()
|
||||
} catch (reason) {
|
||||
if (seq !== chapterSeq || isStale(version, owner)) return
|
||||
if (isNotFound(reason)) {
|
||||
chapter.value = null
|
||||
chapterBook.value = null
|
||||
navigation.value = emptyNavigation()
|
||||
chapterError.value = NOT_FOUND_MESSAGE
|
||||
stopPolling()
|
||||
return
|
||||
}
|
||||
if (!options.silent) chapterError.value = failureMessage(reason, '章节暂时无法加载，请稍后重试。')
|
||||
} finally {
|
||||
if (seq === chapterSeq && !isStale(version, owner)) chapterLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function submissionKeyOf(target: SubmitTarget, title: string, text: string): string {
|
||||
return target.mode === 'new' ? `new\n${title}\n${text}` : `append:${target.bookId}\n${title}\n${text}`
|
||||
}
|
||||
|
||||
async function createBook(body: SubmitBookBody): Promise<Created> {
|
||||
const result = await session.request<{ book: BookRef; chapter: ChapterSummary }>('books', 'POST', body)
|
||||
return { bookId: result.book.id, chapter: result.chapter }
|
||||
}
|
||||
|
||||
async function appendChapter(bookId: number, body: SubmitChapterBody): Promise<Created> {
|
||||
const result = await session.request<{ chapter: ChapterSummary }>(`books/${bookId}/chapters`, 'POST', body)
|
||||
return { bookId: result.chapter.bookId, chapter: result.chapter }
|
||||
}
|
||||
|
||||
/**
|
||||
* Submits pasted text. Returns the book id to open on success and throws on
|
||||
* failure; `submitError` always carries the message shown to the user.
|
||||
*/
|
||||
async function submit(input: SubmitInput): Promise<number> {
|
||||
const title = input.title.trim()
|
||||
const text = input.text
|
||||
const problem = titleProblem(title) || textProblem(text)
|
||||
if (problem) {
|
||||
submitError.value = problem
|
||||
throw new Error(problem)
|
||||
}
|
||||
|
||||
const key = submissionKeyOf(input.target, title, text)
|
||||
if (key !== submissionKey || submissionRequestId === '') {
|
||||
submissionKey = key
|
||||
submissionRequestId = crypto.randomUUID()
|
||||
}
|
||||
const requestId = submissionRequestId
|
||||
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
submitting.value = true
|
||||
submitError.value = ''
|
||||
try {
|
||||
// Only the new-book contract carries a language; appending inherits the book's language.
|
||||
const created = input.target.mode === 'new'
|
||||
? await createBook({ requestId, title, text, language: LANGUAGE_CODE })
|
||||
: await appendChapter(input.target.bookId, { requestId, title, text })
|
||||
if (isStale(version, owner)) throw new Error('登录状态已变化，请重新提交。')
|
||||
// The content was accepted; a later submit must use a fresh requestId.
|
||||
submissionKey = ''
|
||||
submissionRequestId = ''
|
||||
return created.bookId
|
||||
} catch (reason) {
|
||||
if (!isStale(version, owner)) submitError.value = failureMessage(reason, '提交失败，请稍后重试。')
|
||||
throw reason instanceof Error ? reason : new Error('提交失败，请稍后重试。')
|
||||
} finally {
|
||||
if (!isStale(version, owner)) submitting.value = false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Uploads one TXT file. The file and the title share the request id discipline of a paste,
|
||||
* so a repeated upload of the same file answers with the chapter it already created.
|
||||
*/
|
||||
async function upload(input: UploadInput): Promise<number> {
|
||||
const title = input.title.trim()
|
||||
const problem = titleProblem(title) || fileProblem(input.file)
|
||||
if (problem) {
|
||||
submitError.value = problem
|
||||
throw new Error(problem)
|
||||
}
|
||||
const key = `upload\n${title}\n${input.file.name}\n${input.file.size}\n${input.file.lastModified}`
|
||||
if (key !== submissionKey || submissionRequestId === '') {
|
||||
submissionKey = key
|
||||
submissionRequestId = crypto.randomUUID()
|
||||
}
|
||||
const requestId = submissionRequestId
|
||||
const form = new FormData()
|
||||
form.append('requestId', requestId)
|
||||
form.append('title', title)
|
||||
// Only the new-book contract carries a language; appending inherits the book's language.
|
||||
const path = input.target.mode === 'new' ? 'books/upload' : `books/${input.target.bookId}/chapters/upload`
|
||||
if (input.target.mode === 'new') form.append('language', LANGUAGE_CODE)
|
||||
form.append('file', input.file, input.file.name)
|
||||
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
submitting.value = true
|
||||
submitError.value = ''
|
||||
try {
|
||||
const result = await session.request<{ book?: BookRef; chapter: ChapterSummary }>(path, 'POST', form)
|
||||
if (isStale(version, owner)) throw new Error('登录状态已变化，请重新提交。')
|
||||
submissionKey = ''
|
||||
submissionRequestId = ''
|
||||
return result.book?.id ?? result.chapter.bookId
|
||||
} catch (reason) {
|
||||
if (!isStale(version, owner)) submitError.value = failureMessage(reason, '上传失败，请稍后重试。')
|
||||
throw reason instanceof Error ? reason : new Error('上传失败，请稍后重试。')
|
||||
} finally {
|
||||
if (!isStale(version, owner)) submitting.value = false
|
||||
}
|
||||
}
|
||||
|
||||
/** Renames one owned book; the reply is the stored book. */
|
||||
async function renameBook(id: number, title: string): Promise<BookRef> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const problem = titleProblem(title)
|
||||
if (problem) throw new Error(problem)
|
||||
const result = await session.request<{ book: BookRef }>(`books/${id}`, 'PATCH', { title: title.trim() })
|
||||
if (!isStale(version, owner)) {
|
||||
book.value = result.book
|
||||
const listed = books.value.find(item => item.id === id)
|
||||
if (listed) listed.title = result.book.title
|
||||
}
|
||||
return result.book
|
||||
}
|
||||
|
||||
/**
|
||||
* Saves a chapter title and/or text. A changed text becomes a new version and re-processes;
|
||||
* the same text answered again changes nothing.
|
||||
*/
|
||||
async function updateChapter(id: number, input: { title?: string; text?: string }): Promise<ChapterEdit> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
if (input.title !== undefined) {
|
||||
const problem = titleProblem(input.title)
|
||||
if (problem) throw new Error(problem)
|
||||
}
|
||||
if (input.text !== undefined) {
|
||||
const problem = textProblem(input.text)
|
||||
if (problem) throw new Error(problem)
|
||||
}
|
||||
const body: { title?: string; text?: string } = {}
|
||||
if (input.title !== undefined) body.title = input.title.trim()
|
||||
if (input.text !== undefined) body.text = input.text
|
||||
const result = await session.request<ChapterEdit>(`chapters/${id}`, 'PATCH', body)
|
||||
if (isStale(version, owner)) return result
|
||||
const merged = { ...result.chapter, jobId: result.job?.id ?? result.chapter.jobId }
|
||||
applyChapterSummary(merged)
|
||||
if (chapter.value !== null && chapter.value.id === id && result.versionChanged) {
|
||||
// The new version is not readable yet, so the reader must drop the previous text.
|
||||
chapter.value = { ...chapter.value, ...merged, originalText: undefined }
|
||||
}
|
||||
schedulePolling()
|
||||
return result
|
||||
}
|
||||
|
||||
/** Reads the editable text of one owned chapter, in any processing state. */
|
||||
async function loadChapterSource(id: number): Promise<ChapterSource> {
|
||||
const result = await session.request<{ source: ChapterSource }>(`chapters/${id}/source`)
|
||||
return result.source
|
||||
}
|
||||
|
||||
/** Deletes one owned book with its chapters and jobs; personal records stay. */
|
||||
async function deleteBook(id: number): Promise<DeletionResult> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const result = await session.request<{ deleted: DeletionResult }>(`books/${id}`, 'DELETE')
|
||||
if (!isStale(version, owner)) {
|
||||
books.value = books.value.filter(item => item.id !== id)
|
||||
if (book.value?.id === id) closeBook()
|
||||
}
|
||||
return result.deleted
|
||||
}
|
||||
|
||||
/** Deletes one owned chapter and closes the gap in the chapter order. */
|
||||
async function deleteChapter(id: number): Promise<DeletionResult> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const result = await session.request<{ deleted: DeletionResult }>(`chapters/${id}`, 'DELETE')
|
||||
if (isStale(version, owner)) return result.deleted
|
||||
chapters.value = chapters.value.filter(item => item.id !== id)
|
||||
if (book.value !== null) await loadBook(book.value.id, { silent: true })
|
||||
return result.deleted
|
||||
}
|
||||
|
||||
/** The job id comes from the chapter itself, wherever that chapter was loaded from. */
|
||||
function jobIdOf(chapterId: number): number | null {
|
||||
const target = chapters.value.find(item => item.id === chapterId)
|
||||
?? (chapter.value?.id === chapterId ? chapter.value : null)
|
||||
return target?.jobId ?? null
|
||||
}
|
||||
|
||||
/**
|
||||
* Applies a chapter summary coming from any response to the chapter list entry and to the
|
||||
* open reader, so a queued chapter is never displayed with the state or text it had before.
|
||||
*/
|
||||
function applyChapterSummary(summary: ChapterSummary): void {
|
||||
const index = chapters.value.findIndex(item => item.id === summary.id)
|
||||
if (index >= 0) chapters.value[index] = { ...chapters.value[index], ...summary }
|
||||
if (chapter.value !== null && chapter.value.id === summary.id) {
|
||||
const originalText = summary.status === 'ready' ? chapter.value.originalText : undefined
|
||||
chapter.value = { ...chapter.value, ...summary, originalText }
|
||||
}
|
||||
}
|
||||
|
||||
async function retryChapter(chapterId: number): Promise<void> {
|
||||
const jobId = jobIdOf(chapterId)
|
||||
if (jobId === null) throw new Error('这一章暂时没有可重试的任务编号。')
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
retryingChapterId.value = chapterId
|
||||
try {
|
||||
const result = await session.request<{ job: Job; chapter: ChapterSummary }>(`jobs/${jobId}/retry`, 'POST')
|
||||
if (isStale(version, owner)) return
|
||||
// The retry is accepted, so show the queued chapter and keep tracking it even if the
|
||||
// refresh below fails: a failed silent refresh must not freeze the view on the old error.
|
||||
applyChapterSummary(result.chapter)
|
||||
schedulePolling()
|
||||
// Refresh whatever is on screen to pick up the newest job state.
|
||||
if (book.value !== null && book.value.id === result.chapter.bookId) await loadBook(result.chapter.bookId, { silent: true })
|
||||
if (chapter.value !== null && chapter.value.id === result.chapter.id) await loadChapter(result.chapter.id, { silent: true })
|
||||
} catch (reason) {
|
||||
if (isStale(version, owner)) return
|
||||
throw reason instanceof Error ? reason : new Error('重试失败，请稍后重试。')
|
||||
} finally {
|
||||
if (!isStale(version, owner)) retryingChapterId.value = null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Releases the book view so polling stops when the page is left. The request sequence is
|
||||
* advanced first, so a response that arrives after this call cannot repopulate the view or
|
||||
* restart polling for a page the user already left.
|
||||
*/
|
||||
function closeBook(): void {
|
||||
bookSeq++
|
||||
book.value = null
|
||||
chapters.value = []
|
||||
bookLoading.value = false
|
||||
bookError.value = ''
|
||||
schedulePolling()
|
||||
}
|
||||
|
||||
/** Releases the reader view, invalidating in-flight loads the same way. */
|
||||
function closeChapter(): void {
|
||||
chapterSeq++
|
||||
chapter.value = null
|
||||
chapterBook.value = null
|
||||
navigation.value = emptyNavigation()
|
||||
chapterLoading.value = false
|
||||
chapterError.value = ''
|
||||
schedulePolling()
|
||||
}
|
||||
|
||||
function reset(): void {
|
||||
generation++
|
||||
stopPolling()
|
||||
books.value = []
|
||||
booksLoading.value = false
|
||||
booksError.value = ''
|
||||
book.value = null
|
||||
chapters.value = []
|
||||
bookLoading.value = false
|
||||
bookError.value = ''
|
||||
chapter.value = null
|
||||
chapterBook.value = null
|
||||
navigation.value = emptyNavigation()
|
||||
chapterLoading.value = false
|
||||
chapterError.value = ''
|
||||
submitting.value = false
|
||||
submitError.value = ''
|
||||
retryingChapterId.value = null
|
||||
submissionKey = ''
|
||||
submissionRequestId = ''
|
||||
}
|
||||
|
||||
return {
|
||||
books, booksLoading, booksError,
|
||||
book, chapters, bookLoading, bookError,
|
||||
chapter, chapterBook, navigation, chapterLoading, chapterError,
|
||||
submitting, submitError, retryingChapterId, readerText,
|
||||
loadBooks, loadBook, loadChapter, submit, upload, retryChapter,
|
||||
renameBook, updateChapter, loadChapterSource, deleteBook, deleteChapter,
|
||||
stopPolling, closeBook, closeChapter, reset,
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,251 @@
|
||||
import { computed, ref, watch } from 'vue'
|
||||
import { defineStore } from 'pinia'
|
||||
import { useSessionStore } from './session'
|
||||
|
||||
export type ReviewGrade = 'correct' | 'wrong' | 'again'
|
||||
// The outcome of an answer: applied when the word moved, stale when another screen had
|
||||
// already reviewed it. A replayed answer repeats the outcome it was given first.
|
||||
export type ReviewResult = 'applied' | 'stale'
|
||||
|
||||
export interface ReviewItem {
|
||||
id: number
|
||||
term: string
|
||||
originalForm: string
|
||||
definition: string
|
||||
examples: string[]
|
||||
status: 'new' | 'learning'
|
||||
level: number
|
||||
kind: 'word' | 'phrase'
|
||||
wordCount: number
|
||||
dueAt: string
|
||||
reviewCount: number
|
||||
}
|
||||
|
||||
export interface ReviewAnswerResult {
|
||||
result: ReviewResult
|
||||
duplicate: boolean
|
||||
grade: ReviewGrade
|
||||
requeued: boolean
|
||||
statusBefore: string
|
||||
statusAfter: string
|
||||
levelBefore: number
|
||||
levelAfter: number
|
||||
dueAtBefore: string
|
||||
dueAtAfter: string
|
||||
item: ReviewItem
|
||||
}
|
||||
|
||||
interface QueueResponse { items: ReviewItem[]; total: number }
|
||||
|
||||
function escapeRegExp(text: string): string {
|
||||
return text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
|
||||
}
|
||||
|
||||
/**
|
||||
* The prompt shown before the answer: the first personal example with the word masked.
|
||||
* A word without examples is shown on its own; the dictionary is never consulted here.
|
||||
*/
|
||||
export function clozeSentence(item: ReviewItem): string | null {
|
||||
const line = item.examples[0]
|
||||
if (!line) return null
|
||||
const forms = [...new Set([item.term, item.originalForm].filter(Boolean))]
|
||||
let masked = line
|
||||
for (const form of forms) {
|
||||
// Whole word, any case, and either apostrophe form, so the mask matches how the
|
||||
// learner typed the sentence.
|
||||
const pattern = new RegExp(`(?<![\\p{L}\\p{M}])${escapeRegExp(form).replace(/'/g, "['’]")}(?![\\p{L}\\p{M}])`, 'giu')
|
||||
masked = masked.replace(pattern, '_____')
|
||||
}
|
||||
return masked
|
||||
}
|
||||
|
||||
/**
|
||||
* A phrase is masked as one blank covering the whole run, which is what the accepted prototype
|
||||
* shows. Interior separators of any non-letter run are allowed, matching the server's phrase
|
||||
* identity rule.
|
||||
*/
|
||||
export function maskedPrompt(item: ReviewItem): string | null {
|
||||
const line = item.examples[0]
|
||||
if (!line) return null
|
||||
if (item.kind !== 'phrase') return clozeSentence(item)
|
||||
const forms = item.term.split(' ').filter(Boolean)
|
||||
if (forms.length < 2) return clozeSentence(item)
|
||||
const parts = forms.map(form => escapeRegExp(form).replace(/'/g, "['’]"))
|
||||
const pattern = new RegExp(parts.join(`[\\p{L}\\p{M}]*?[^\\p{L}\\p{M}]+[\\p{L}\\p{M}]*?`), 'iu')
|
||||
return line.replace(pattern, '_____')
|
||||
}
|
||||
|
||||
export const useReviewStore = defineStore('review', () => {
|
||||
const session = useSessionStore()
|
||||
|
||||
const queue = ref<ReviewItem[]>([])
|
||||
const loading = ref(false)
|
||||
const error = ref('')
|
||||
const busy = ref(false)
|
||||
const revealed = ref(false)
|
||||
const answered = ref(0)
|
||||
const correctCount = ref(0)
|
||||
const wrongCount = ref(0)
|
||||
// Distinct words in this round: a requeued word is answered again but is one word.
|
||||
const wordsReviewed = ref(0)
|
||||
// Cards this round took off the queue, counted for every outcome, and the reason a card
|
||||
// left without a new score.
|
||||
const resolved = ref(0)
|
||||
const notice = ref('')
|
||||
const seen = new Set<number>()
|
||||
// Words due beyond the fetched page, reported by the server for this round.
|
||||
const pending = ref(0)
|
||||
let started = ref(false)
|
||||
// One answer id per card: a retry of a failed submission reuses its key, so the server
|
||||
// answers the retry from the first outcome instead of counting the same action twice.
|
||||
let attempt: { itemId: number; answerId: string } | null = null
|
||||
let sequence = 0
|
||||
let generation = 0
|
||||
|
||||
const current = computed<ReviewItem | null>(() => queue.value[0] ?? null)
|
||||
// A round that resolved cards is finished even when every answer turned out to be a
|
||||
// replay or a stale submission; only a round that never had a card is empty.
|
||||
const finished = computed(() => started.value && !loading.value && queue.value.length === 0 && resolved.value > 0)
|
||||
const empty = computed(() => started.value && !loading.value && queue.value.length === 0 && resolved.value === 0)
|
||||
|
||||
watch(() => session.user?.id ?? null, (next, previous) => {
|
||||
if (next !== previous) reset()
|
||||
}, { flush: 'sync' })
|
||||
|
||||
function ownerId(): number | null {
|
||||
return session.user?.id ?? null
|
||||
}
|
||||
|
||||
function isStale(version: number, owner: number | null): boolean {
|
||||
return version !== generation || ownerId() !== owner
|
||||
}
|
||||
|
||||
function reset(): void {
|
||||
generation++
|
||||
sequence++
|
||||
queue.value = []
|
||||
loading.value = false
|
||||
error.value = ''
|
||||
busy.value = false
|
||||
revealed.value = false
|
||||
answered.value = 0
|
||||
correctCount.value = 0
|
||||
wrongCount.value = 0
|
||||
wordsReviewed.value = 0
|
||||
resolved.value = 0
|
||||
notice.value = ''
|
||||
pending.value = 0
|
||||
started.value = false
|
||||
attempt = null
|
||||
seen.clear()
|
||||
}
|
||||
|
||||
async function load(newRound = true): Promise<void> {
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = ++sequence
|
||||
loading.value = true
|
||||
error.value = ''
|
||||
revealed.value = false
|
||||
notice.value = ''
|
||||
attempt = null
|
||||
try {
|
||||
const result = await session.request<QueueResponse>('reviews/queue')
|
||||
if (seq !== sequence || isStale(version, owner)) return
|
||||
const items = Array.isArray(result?.items) ? result.items : []
|
||||
queue.value = items
|
||||
const total = typeof result?.total === 'number' ? result.total : items.length
|
||||
pending.value = Math.max(0, total - items.length)
|
||||
// A fresh round starts at zero; continuing with the next page of due words keeps
|
||||
// the counters of the round the learner is already in.
|
||||
if (newRound) {
|
||||
answered.value = 0
|
||||
correctCount.value = 0
|
||||
wrongCount.value = 0
|
||||
wordsReviewed.value = 0
|
||||
resolved.value = 0
|
||||
seen.clear()
|
||||
}
|
||||
started.value = true
|
||||
} catch (reason) {
|
||||
if (seq !== sequence || isStale(version, owner)) return
|
||||
error.value = reason instanceof Error ? reason.message : '复习队列暂时无法加载，请稍后重试。'
|
||||
} finally {
|
||||
if (seq === sequence && !isStale(version, owner)) loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function answerIdFor(item: ReviewItem): string {
|
||||
if (!attempt || attempt.itemId !== item.id) attempt = { itemId: item.id, answerId: crypto.randomUUID() }
|
||||
return attempt.answerId
|
||||
}
|
||||
|
||||
function reveal(): void {
|
||||
revealed.value = true
|
||||
}
|
||||
|
||||
/**
|
||||
* Sends one grade for the current card. A failed request keeps the card and its input
|
||||
* so the learner can retry; the same answer id is reused on that retry.
|
||||
*/
|
||||
async function answer(grade: ReviewGrade): Promise<void> {
|
||||
const item = current.value
|
||||
if (!item || busy.value || !session.user) return
|
||||
const version = generation
|
||||
const owner = ownerId()
|
||||
const seq = sequence
|
||||
busy.value = true
|
||||
error.value = ''
|
||||
try {
|
||||
const result = await session.request<ReviewAnswerResult>(`reviews/${item.id}/answers`, 'POST', {
|
||||
answerId: answerIdFor(item), grade, expectedDueAt: item.dueAt,
|
||||
})
|
||||
if (seq !== sequence || isStale(version, owner)) return
|
||||
applyResult(item, result)
|
||||
} catch (reason) {
|
||||
if (seq !== sequence || isStale(version, owner)) return
|
||||
error.value = reason instanceof Error ? reason.message : '评分暂时无法提交，请重试。'
|
||||
} finally {
|
||||
if (seq === sequence && !isStale(version, owner)) busy.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function applyResult(item: ReviewItem, result: ReviewAnswerResult): void {
|
||||
// The card leaves the round in every outcome; a stale or replayed answer is the same
|
||||
// user action seen twice, not a second review.
|
||||
queue.value = queue.value.filter(entry => entry.id !== item.id)
|
||||
attempt = null
|
||||
revealed.value = false
|
||||
resolved.value += 1
|
||||
if (result.result === 'applied') {
|
||||
// A replay repeats the first outcome, and the first attempt may be this client's own
|
||||
// submission whose response was lost, so it counts exactly like that attempt.
|
||||
notice.value = ''
|
||||
answered.value += 1
|
||||
if (!seen.has(item.id)) {
|
||||
seen.add(item.id)
|
||||
wordsReviewed.value += 1
|
||||
}
|
||||
if (result.grade === 'correct') correctCount.value += 1
|
||||
else wrongCount.value += 1
|
||||
} else {
|
||||
notice.value = result.duplicate
|
||||
? '该词已按上一次的评分记录，未重复计分。'
|
||||
: '该词已在其他页面复习，本次未计分。'
|
||||
}
|
||||
if (result.requeued) {
|
||||
const next = result.item ?? { ...item, level: result.levelAfter, status: result.statusAfter as ReviewItem['status'], dueAt: result.dueAtAfter }
|
||||
queue.value = [...queue.value, { ...next, dueAt: result.dueAtAfter, level: result.levelAfter, status: result.statusAfter as ReviewItem['status'] }]
|
||||
}
|
||||
}
|
||||
|
||||
// Load the next page of due words, keeping the counters of the round in progress.
|
||||
async function continueRound(): Promise<void> {
|
||||
await load(false)
|
||||
}
|
||||
|
||||
return {
|
||||
queue, current, loading, error, busy, revealed, answered, correctCount, wrongCount, wordsReviewed, resolved,
|
||||
notice, pending, finished, empty, load, reveal, answer, continueRound, reset,
|
||||
}
|
||||
})
|
||||
@@ -2,6 +2,17 @@ import { defineStore } from 'pinia'
|
||||
import { ref } from 'vue'
|
||||
|
||||
export const TOKEN_KEY = 'lexgo-learner-token'
|
||||
// Carries the real HTTP status alongside the server message so callers can tell
|
||||
// "this id does not exist for me" (404) from a transient failure without
|
||||
// re-parsing the envelope. The message itself is unchanged.
|
||||
export class ApiError extends Error {
|
||||
readonly status: number
|
||||
constructor(message: string, status: number) {
|
||||
super(message)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
}
|
||||
}
|
||||
interface User { id: number; username: string; role: 'admin' | 'learner' }
|
||||
interface Space { ownerId: number; language: 'en' }
|
||||
interface Login { token: string; expiresAt: string; user: User }
|
||||
@@ -25,10 +36,12 @@ export const useSessionStore = defineStore('session', () => {
|
||||
}
|
||||
|
||||
async function request<T>(path: string, method = 'GET', body?: unknown, auth = token, version = generation): Promise<T> {
|
||||
// A multipart body carries its own content type with the boundary, so it is sent as is.
|
||||
const multipart = body instanceof FormData
|
||||
const response = await fetch(`/api/v1/${path}`, {
|
||||
method,
|
||||
headers: { ...(auth ? { Authorization: `Bearer ${auth}` } : {}), ...(body ? { 'Content-Type': 'application/json' } : {}) },
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
headers: { ...(auth ? { Authorization: `Bearer ${auth}` } : {}), ...(body && !multipart ? { 'Content-Type': 'application/json' } : {}) },
|
||||
body: body ? (multipart ? body : JSON.stringify(body)) : undefined,
|
||||
cache: 'no-store',
|
||||
})
|
||||
if (response.status === 401 && version === generation) {
|
||||
@@ -36,7 +49,7 @@ export const useSessionStore = defineStore('session', () => {
|
||||
notice.value = '登录已失效，请重新登录。'
|
||||
}
|
||||
const result = await response.json()
|
||||
if (!response.ok || result.code !== 200) throw new Error(result.msg || '请求失败，请稍后重试。')
|
||||
if (!response.ok || result.code !== 200) throw new ApiError(result.msg || '请求失败，请稍后重试。', response.status)
|
||||
return result.data as T
|
||||
}
|
||||
|
||||
@@ -91,5 +104,5 @@ export const useSessionStore = defineStore('session', () => {
|
||||
if (previousToken) await request<null>('logout', 'POST', undefined, previousToken, -1)
|
||||
}
|
||||
|
||||
return { user, space, notice, login, restore, logout, loadSpace }
|
||||
return { user, space, notice, login, restore, logout, loadSpace, request }
|
||||
})
|
||||
|
||||
@@ -51,6 +51,101 @@ h1 { font-size: 30px; font-weight: 600; margin: 14px 0; letter-spacing: 1px; }
|
||||
.empty-library .book-mark { color: #6e8967; width: 56px; height: 56px; }
|
||||
.empty-library h2 { font-weight: 500; font-size: 21px; margin: 26px 0 4px; }
|
||||
.loading { padding: 80px 24px; text-align: center; color: #748073; }
|
||||
/* Import / book / reader pages share one shell. */
|
||||
.page { max-width: 1120px; margin: 60px auto; padding: 0 28px; }
|
||||
.page-title { display: flex; align-items: center; justify-content: space-between; gap: 18px; flex-wrap: wrap; margin-bottom: 28px; }
|
||||
.page-title h1 { margin: 14px 0 6px; overflow-wrap: anywhere; }
|
||||
.page-actions { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; }
|
||||
.library-actions { display: flex; align-items: center; gap: 14px; flex-wrap: wrap; }
|
||||
.link-button { display: inline-flex; align-items: center; min-height: 42px; padding: 0 8px; color: #315c43; }
|
||||
.breadcrumb { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; margin: 0 0 4px; font-size: 14px; color: #748073; }
|
||||
.breadcrumb a { color: #315c43; }
|
||||
/* Library book cards. */
|
||||
.book-grid { list-style: none; margin: 0; padding: 0; display: grid; gap: 18px; grid-template-columns: repeat(auto-fill, minmax(290px, 1fr)); }
|
||||
.book-card { display: flex; flex-direction: column; gap: 10px; background: #fffdf8; border: 1px solid #e0e3d8; border-radius: 12px; padding: 22px; }
|
||||
.book-card .subtle { margin: 0; }
|
||||
.book-title { font-family: Georgia, 'Microsoft YaHei', serif; font-size: 21px; font-weight: 600; color: #233d31; text-decoration: none; overflow-wrap: anywhere; }
|
||||
.book-title:hover { color: #315c43; text-decoration: underline; }
|
||||
.status-summary { align-self: flex-start; margin: 0; padding: 6px 13px; border-radius: 20px; background: #eef2eb; color: #3d5b48; font-size: 13px; }
|
||||
/* Chapter and job status, identical vocabulary for both. */
|
||||
.status-chip { display: inline-flex; align-items: center; white-space: nowrap; padding: 5px 13px; border: 1px solid transparent; border-radius: 20px; font-size: 13px; }
|
||||
.status-pending { background: #f5f2e4; border-color: #e2dcc2; color: #7a6a35; }
|
||||
.status-processing { background: #eaf1f7; border-color: #c9dcea; color: #35566e; }
|
||||
.status-ready { background: #eef2eb; border-color: #cbd9c9; color: #315c43; }
|
||||
.status-failed { background: #fff0e7; border-color: #ebc3a8; color: #8b4324; }
|
||||
/* Import form. */
|
||||
.import-form { max-width: 720px; background: #fffdf8; border: 1px solid #e0e3d8; border-radius: 12px; padding: 28px; }
|
||||
.field-label { display: block; font-size: 14px; margin-bottom: 10px; }
|
||||
.fixed-value { margin: 0; padding: 12px 15px; border: 1px solid #d6dccf; border-radius: 8px; background: #fffefa; font-size: 15px; }
|
||||
.field-error { margin: 8px 0 0; color: #8b4324; font-size: 13px; line-height: 1.6; }
|
||||
.counter { margin: 8px 0 0; color: #748073; font-size: 13px; }
|
||||
.form-actions { display: flex; align-items: center; gap: 16px; flex-wrap: wrap; margin-top: 14px; }
|
||||
.import-form .el-textarea__inner { min-height: 220px; line-height: 1.9; }
|
||||
.book-select { width: 100%; }
|
||||
/* Book chapters. */
|
||||
.chapter-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 12px; }
|
||||
.chapter-row { display: grid; grid-template-columns: 40px minmax(0, 1fr) auto auto; align-items: center; gap: 14px; background: #fffdf8; border: 1px solid #e0e3d8; border-radius: 12px; padding: 16px 18px; }
|
||||
.chapter-ordinal { font-family: Georgia, serif; font-size: 17px; color: #8b9a8b; text-align: center; }
|
||||
.chapter-info { min-width: 0; }
|
||||
.chapter-name { display: inline-block; font-size: 16px; font-weight: 500; color: #233d31; text-decoration: none; overflow-wrap: anywhere; }
|
||||
a.chapter-name:hover { color: #315c43; text-decoration: underline; }
|
||||
.chapter-meta { margin: 4px 0 0; color: #748073; font-size: 13px; overflow-wrap: anywhere; }
|
||||
/* Reader keeps the pasted text exactly as it was, including line breaks and tabs. */
|
||||
.reader-page { max-width: 820px; }
|
||||
.reader-text { white-space: pre-wrap; overflow-wrap: break-word; margin: 26px 0 0; font-family: Georgia, 'Times New Roman', 'Microsoft YaHei', serif; font-size: 17px; line-height: 2; }
|
||||
.processing-hint { padding: 36px 0; color: #748073; }
|
||||
.reader-nav { display: flex; align-items: center; justify-content: space-between; gap: 14px; flex-wrap: wrap; margin-top: 36px; padding-top: 22px; border-top: 1px solid #e0e3d8; }
|
||||
.reader-page.has-lookup { max-width: 1120px; }
|
||||
.reader-workspace { display: grid; grid-template-columns: minmax(0, 1fr); gap: 32px; align-items: start; }
|
||||
.has-lookup .reader-workspace { grid-template-columns: minmax(0, 1fr) 320px; }
|
||||
.reader-body { min-width: 0; }
|
||||
.reader-word { cursor: pointer; border-radius: 3px; }
|
||||
/* Saved words keep one style per status everywhere they appear. */
|
||||
.reader-word.is-new { border-bottom: 2px dotted #bc803d; }
|
||||
.reader-word.is-learning { background: #f6e7c9; }
|
||||
.reader-word.is-known { background: #dee9d6; }
|
||||
.reader-word.is-ignored { color: #949c94; }
|
||||
.reader-word:hover, .reader-word.is-selected { background: #e3e9d9; color: #264a35; }
|
||||
.reader-word:focus-visible { outline: 2px solid #bc803d; outline-offset: 2px; background: #eef2eb; }
|
||||
.tokens-notice { color: #748073; font-size: 14px; margin-top: 24px; }
|
||||
.lookup-panel { position: sticky; top: 24px; margin-top: 26px; padding: 22px; border: 1px solid #d9decf; border-radius: 12px; background: #fffdf8; max-height: calc(100dvh - 48px); overflow-y: auto; overflow-wrap: anywhere; }
|
||||
.lookup-heading { display: flex; align-items: start; justify-content: space-between; gap: 10px; border-bottom: 1px solid #e0e3d8; padding-bottom: 12px; }
|
||||
.lookup-heading h2 { margin: 6px 0; font-family: Georgia, serif; font-size: 25px; }
|
||||
.lookup-content { font-size: 15px; line-height: 1.7; }
|
||||
.lookup-message { color: #8b4324; }
|
||||
.lookup-senses { padding-left: 22px; }
|
||||
.lookup-senses li { padding-left: 3px; margin-bottom: 20px; }
|
||||
.lookup-senses p { margin: 8px 0; }
|
||||
.sense-heading span { color: #748073; font-size: 13px; }
|
||||
.lookup-senses blockquote { border-left: 2px solid #cbd9c9; margin: 10px 0; padding-left: 12px; color: #687568; font-style: italic; }
|
||||
.lookup-range { border-bottom: 1px solid #e0e3d8; padding-bottom: 14px; }
|
||||
.lookup-range p { margin: 0 0 10px; }
|
||||
.lookup-range-actions { display: flex; gap: 6px; flex-wrap: wrap; }
|
||||
/* A saved phrase is one unit: the run is underlined and keeps its status colour. */
|
||||
.reader-word.is-phrase { text-decoration: underline; text-decoration-style: double; text-underline-offset: 2px; border-radius: 3px; }
|
||||
.reader-word.is-phrase-selected { outline: 2px solid #bc803d; outline-offset: 1px; }
|
||||
.lookup-term { border-top: 1px solid #e0e3d8; padding-top: 18px; margin-top: 20px; }
|
||||
.lookup-term label { display: flex; justify-content: space-between; gap: 10px; font-size: 14px; margin-top: 14px; }
|
||||
.lookup-term-title { display: flex; font-size: 14px; margin: 0; }
|
||||
.lookup-term label span { color: #8b794e; font-size: 12px; }
|
||||
.lookup-term .el-radio-group { margin-top: 8px; flex-wrap: wrap; gap: 4px 12px; }
|
||||
.lookup-term .el-textarea { margin-top: 8px; }
|
||||
.lookup-term .el-textarea textarea { font: inherit; line-height: 1.6; }
|
||||
.lookup-saved { color: #2f6b45; font-size: 14px; margin: 12px 0 0; }
|
||||
.lookup-actions { display: flex; gap: 8px; margin-top: 14px; flex-wrap: wrap; }
|
||||
.review-page { max-width: 680px; }
|
||||
.chapter-notice { margin: 12px 0 0; padding: 10px 14px; border: 1px solid #d9decf; border-radius: 8px; background: #fbf7ee; color: #6b5b3e; }
|
||||
.chapter-list .chapter-row { flex-wrap: wrap; }
|
||||
.review-notice { margin: 10px 0 0; padding: 10px 14px; border: 1px solid #d9decf; border-radius: 8px; background: #fbf7ee; color: #6b5b3e; }
|
||||
.review-card, .review-summary { margin-top: 26px; padding: 28px; border: 1px solid #d9decf; border-radius: 14px; background: #fffdf8; }
|
||||
.review-summary h2 { margin-top: 0; font-family: Georgia, serif; font-size: 24px; }
|
||||
.review-word { margin: 12px 0; font-family: Georgia, serif; font-size: 34px; }
|
||||
.review-example { margin: 12px 0; font-family: Georgia, serif; font-size: 19px; line-height: 1.8; }
|
||||
.review-answer { margin: 18px 0; padding-top: 18px; border-top: 1px solid #e0e3d8; }
|
||||
.review-definition { margin: 0 0 12px; font-size: 18px; }
|
||||
.review-answer blockquote { border-left: 2px solid #cbd9c9; margin: 10px 0; padding-left: 12px; color: #687568; font-style: italic; }
|
||||
.review-grades { display: flex; gap: 10px; flex-wrap: wrap; margin-top: 8px; }
|
||||
.review-actions { margin: 16px 0 0; }
|
||||
@media (max-width: 760px) {
|
||||
.login-page { grid-template-columns: 1fr; }
|
||||
.welcome { padding: 28px; }
|
||||
@@ -62,4 +157,19 @@ h1 { font-size: 30px; font-weight: 600; margin: 14px 0; letter-spacing: 1px; }
|
||||
.site-header nav { order: 3; flex-basis: 100%; padding-top: 8px; }
|
||||
.library { margin-top: 32px; padding: 0 20px; }
|
||||
h1 { font-size: 26px; }
|
||||
/* Single column, tap-friendly controls and no horizontal overflow. */
|
||||
.page { margin-top: 32px; padding: 0 20px; }
|
||||
.page-title { align-items: flex-start; }
|
||||
.library-title { align-items: flex-start; }
|
||||
.library-actions { width: 100%; justify-content: space-between; }
|
||||
.book-grid { grid-template-columns: 1fr; }
|
||||
.import-form { padding: 20px; }
|
||||
.chapter-row { grid-template-columns: 30px minmax(0, 1fr); align-items: start; row-gap: 10px; padding: 15px 16px; }
|
||||
.chapter-row .status-chip, .chapter-row .el-button { grid-column: 2; justify-self: start; }
|
||||
.reader-text { font-size: 16px; line-height: 1.95; }
|
||||
.has-lookup .reader-workspace { grid-template-columns: minmax(0, 1fr); gap: 20px; }
|
||||
.reader-page.has-lookup { padding-bottom: calc(45dvh + 28px); }
|
||||
.lookup-panel { position: fixed; inset: auto 0 0; z-index: 20; max-height: 45dvh; margin-top: 0; padding: 16px 20px max(20px, env(safe-area-inset-bottom)); border-radius: 16px 16px 0 0; box-shadow: 0 -5px 24px #233d3114; }
|
||||
.lookup-heading { position: sticky; top: -16px; z-index: 1; background: #fffdf8; }
|
||||
.reader-nav .el-button { flex: 1; }
|
||||
}
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
<script setup lang="ts">
|
||||
import { computed, onMounted, onUnmounted, ref, watch } from 'vue'
|
||||
import { RouterLink, useRoute, useRouter } from 'vue-router'
|
||||
import { ElButton, ElDialog, ElInput, ElMessageBox } from 'element-plus'
|
||||
import { canRetry, statusLabel, TEXT_MAX_CODE_POINTS, textProblem, titleProblem, useLibraryStore, type ChapterSource } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const session = useSessionStore()
|
||||
const library = useLibraryStore()
|
||||
const route = useRoute()
|
||||
const router = useRouter()
|
||||
const retryError = ref('')
|
||||
const notice = ref('')
|
||||
|
||||
// Renaming the book.
|
||||
const bookDialog = ref(false)
|
||||
const bookTitle = ref('')
|
||||
const bookError = ref('')
|
||||
// Editing one chapter: title and text, loaded from the source endpoint.
|
||||
const chapterDialog = ref(false)
|
||||
const chapterSource = ref<ChapterSource | null>(null)
|
||||
const chapterTitle = ref('')
|
||||
const chapterText = ref('')
|
||||
const chapterError = ref('')
|
||||
const chapterLoading = ref(false)
|
||||
const saving = ref(false)
|
||||
|
||||
const bookId = computed(() => Number(route.params.id))
|
||||
const chapterLength = computed(() => [...chapterText.value].length)
|
||||
|
||||
async function load() {
|
||||
retryError.value = ''
|
||||
await library.loadBook(bookId.value)
|
||||
}
|
||||
|
||||
async function retry(chapterId: number) {
|
||||
retryError.value = ''
|
||||
try { await library.retryChapter(chapterId) }
|
||||
catch (reason) { retryError.value = reason instanceof Error ? reason.message : '重试失败，请稍后重试。' }
|
||||
}
|
||||
|
||||
function openBookDialog() {
|
||||
bookTitle.value = library.book?.title ?? ''
|
||||
bookError.value = ''
|
||||
bookDialog.value = true
|
||||
}
|
||||
|
||||
async function saveBookTitle() {
|
||||
if (saving.value || library.book === null) return
|
||||
bookError.value = titleProblem(bookTitle.value)
|
||||
if (bookError.value) return
|
||||
saving.value = true
|
||||
try {
|
||||
await library.renameBook(library.book.id, bookTitle.value)
|
||||
bookDialog.value = false
|
||||
notice.value = '书名已更新。'
|
||||
} catch (reason) {
|
||||
bookError.value = reason instanceof Error ? reason.message : '保存失败，请稍后重试。'
|
||||
} finally {
|
||||
saving.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function openChapterDialog(chapterId: number) {
|
||||
chapterDialog.value = true
|
||||
chapterLoading.value = true
|
||||
chapterError.value = ''
|
||||
chapterSource.value = null
|
||||
try {
|
||||
const source = await library.loadChapterSource(chapterId)
|
||||
chapterSource.value = source
|
||||
chapterTitle.value = source.title
|
||||
chapterText.value = source.text
|
||||
} catch (reason) {
|
||||
chapterError.value = reason instanceof Error ? reason.message : '章节内容暂时无法加载，请稍后重试。'
|
||||
} finally {
|
||||
chapterLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function saveChapter() {
|
||||
const source = chapterSource.value
|
||||
if (saving.value || source === null) return
|
||||
chapterError.value = titleProblem(chapterTitle.value) || textProblem(chapterText.value)
|
||||
if (chapterError.value) return
|
||||
saving.value = true
|
||||
try {
|
||||
const edited = await library.updateChapter(source.id, { title: chapterTitle.value, text: chapterText.value })
|
||||
chapterDialog.value = false
|
||||
notice.value = edited.versionChanged ? '章节已保存为新版本，正在重新处理。' : '章节已保存。'
|
||||
} catch (reason) {
|
||||
chapterError.value = reason instanceof Error ? reason.message : '保存失败，请稍后重试。'
|
||||
} finally {
|
||||
saving.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function confirmDeleteBook() {
|
||||
if (library.book === null) return
|
||||
const id = library.book.id
|
||||
try {
|
||||
await ElMessageBox.confirm('删除这本书及其章节？已保存的生词和短语将保留。', '删除书籍', {
|
||||
confirmButtonText: '确认删除', cancelButtonText: '取消，保留书籍', type: 'warning',
|
||||
})
|
||||
} catch { return }
|
||||
try {
|
||||
const deleted = await library.deleteBook(id)
|
||||
await router.replace(`/?deleted=${deleted.chapters ?? 0}`)
|
||||
} catch (reason) {
|
||||
retryError.value = reason instanceof Error ? reason.message : '删除失败，请稍后重试。'
|
||||
}
|
||||
}
|
||||
|
||||
async function confirmDeleteChapter(chapterId: number, ordinal: number) {
|
||||
try {
|
||||
await ElMessageBox.confirm(`删除第 ${ordinal} 章？本章正文将被删除，已保存词条保留。`, '删除章节', {
|
||||
confirmButtonText: '确认删除章节', cancelButtonText: '取消', type: 'warning',
|
||||
})
|
||||
} catch { return }
|
||||
try {
|
||||
const deleted = await library.deleteChapter(chapterId)
|
||||
// The chapter no longer exists, so the dialog closes and the list reports the remainder.
|
||||
chapterDialog.value = false
|
||||
chapterSource.value = null
|
||||
notice.value = `章节已删除 · 剩余 ${deleted.remaining} 章`
|
||||
} catch (reason) {
|
||||
retryError.value = reason instanceof Error ? reason.message : '删除失败，请稍后重试。'
|
||||
}
|
||||
}
|
||||
|
||||
async function logout() {
|
||||
try { await session.logout() }
|
||||
catch { session.notice = '已退出此设备。服务器暂时无法连接，请稍后重试。' }
|
||||
finally { await router.replace('/login') }
|
||||
}
|
||||
|
||||
onMounted(load)
|
||||
watch(bookId, () => { void load() })
|
||||
// Leaving the page releases the book so polling stops.
|
||||
onUnmounted(() => library.closeBook())
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/">我的书库</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
<ElButton text @click="logout">退出登录</ElButton>
|
||||
</div>
|
||||
</header>
|
||||
<main class="page">
|
||||
<p class="breadcrumb"><RouterLink to="/">我的书库</RouterLink><span aria-hidden="true">/</span><span>{{ library.book?.title ?? '书籍' }}</span></p>
|
||||
<p v-if="library.bookLoading && !library.book" role="status" class="loading">正在加载…</p>
|
||||
<div v-else-if="library.bookError" class="notice">
|
||||
<p role="alert">{{ library.bookError }}</p>
|
||||
<ElButton @click="load">重试</ElButton>
|
||||
</div>
|
||||
<template v-else-if="library.book">
|
||||
<div class="page-title">
|
||||
<div>
|
||||
<h1>{{ library.book.title }}</h1>
|
||||
<p class="subtle">{{ library.chapters.length }} 个章节 · 语言 英语 · 封面使用系统默认样式</p>
|
||||
</div>
|
||||
<div class="page-actions">
|
||||
<ElButton data-testid="edit-book" @click="openBookDialog">编辑书名</ElButton>
|
||||
<RouterLink :to="`/import?book=${library.book.id}`" class="link-button">追加章节</RouterLink>
|
||||
<ElButton type="danger" plain data-testid="delete-book" @click="confirmDeleteBook">删除书籍</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
<p v-if="notice" role="status" class="chapter-notice" data-testid="book-notice">{{ notice }}</p>
|
||||
<p v-if="retryError" role="alert" class="notice">{{ retryError }}</p>
|
||||
<ul v-if="library.chapters.length" class="chapter-list" aria-label="章节列表">
|
||||
<li v-for="item in library.chapters" :key="item.id" class="chapter-row">
|
||||
<span class="chapter-ordinal">{{ item.ordinal }}</span>
|
||||
<div class="chapter-info">
|
||||
<RouterLink v-if="item.status === 'ready'" :to="`/chapters/${item.id}`" class="chapter-name">{{ item.title }}</RouterLink>
|
||||
<span v-else class="chapter-name">{{ item.title }}</span>
|
||||
<p class="chapter-meta">
|
||||
{{ item.charCount }} 字符
|
||||
<template v-if="item.status === 'failed' && item.errorMessage"> · {{ item.errorMessage }}</template>
|
||||
</p>
|
||||
</div>
|
||||
<span class="status-chip" :class="`status-${item.status}`">{{ statusLabel(item.status) }}</span>
|
||||
<ElButton
|
||||
v-if="canRetry(item)"
|
||||
size="small"
|
||||
:loading="library.retryingChapterId === item.id"
|
||||
@click="retry(item.id)"
|
||||
>重试</ElButton>
|
||||
<ElButton size="small" :data-testid="`edit-chapter-${item.id}`" @click="openChapterDialog(item.id)">编辑</ElButton>
|
||||
</li>
|
||||
</ul>
|
||||
<section v-else class="empty-library" aria-label="章节列表" data-testid="empty-chapters">
|
||||
<h2>这一本书还没有章节</h2>
|
||||
<p class="subtle">粘贴一段英文即可生成第一章。</p>
|
||||
</section>
|
||||
</template>
|
||||
|
||||
<ElDialog v-model="bookDialog" title="编辑书名" width="420" data-testid="book-dialog">
|
||||
<label for="book-title">书名</label>
|
||||
<ElInput id="book-title" v-model="bookTitle" type="text" maxlength="200" :disabled="saving" />
|
||||
<p class="subtle">封面使用系统默认样式。</p>
|
||||
<p v-if="bookError" role="alert" class="field-error">{{ bookError }}</p>
|
||||
<template #footer>
|
||||
<ElButton @click="bookDialog = false">取消</ElButton>
|
||||
<ElButton type="primary" :loading="saving" data-testid="save-book" @click="saveBookTitle">保存修改</ElButton>
|
||||
</template>
|
||||
</ElDialog>
|
||||
|
||||
<ElDialog v-model="chapterDialog" title="编辑章节" width="620" data-testid="chapter-dialog">
|
||||
<p v-if="chapterLoading" role="status" class="loading">正在加载…</p>
|
||||
<template v-else-if="chapterSource">
|
||||
<label for="chapter-title">章节标题</label>
|
||||
<ElInput id="chapter-title" v-model="chapterTitle" type="text" maxlength="200" :disabled="saving" />
|
||||
<label for="chapter-text">正文</label>
|
||||
<ElInput id="chapter-text" v-model="chapterText" type="textarea" :rows="12" :disabled="saving" />
|
||||
<p class="counter">{{ chapterLength }} / {{ TEXT_MAX_CODE_POINTS }} 字符</p>
|
||||
<p class="subtle">保存后会重新处理这一章；只改标题不会重新处理。</p>
|
||||
</template>
|
||||
<p v-if="chapterError" role="alert" class="field-error">{{ chapterError }}</p>
|
||||
<template #footer>
|
||||
<ElButton data-testid="cancel-chapter" @click="chapterDialog = false">取消</ElButton>
|
||||
<ElButton
|
||||
v-if="chapterSource"
|
||||
type="danger"
|
||||
plain
|
||||
:disabled="saving"
|
||||
data-testid="delete-chapter"
|
||||
@click="confirmDeleteChapter(chapterSource.id, chapterSource.ordinal)"
|
||||
>删除章节</ElButton>
|
||||
<ElButton type="primary" :loading="saving" :disabled="chapterLoading || !chapterSource" data-testid="save-chapter" @click="saveChapter">保存章节</ElButton>
|
||||
</template>
|
||||
</ElDialog>
|
||||
</main>
|
||||
</div>
|
||||
</template>
|
||||
@@ -0,0 +1,196 @@
|
||||
<script setup lang="ts">
|
||||
import { computed, onMounted, onUnmounted, ref, watch } from 'vue'
|
||||
import { RouterLink, useRoute, useRouter } from 'vue-router'
|
||||
import { ElButton, ElInput, ElOption, ElRadio, ElRadioGroup, ElSelect } from 'element-plus'
|
||||
import { LANGUAGE_LABEL, TEXT_MAX_CODE_POINTS, fileProblem, fileSizeLabel, textProblem, titleProblem, useLibraryStore, type SubmitTarget } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const session = useSessionStore()
|
||||
const library = useLibraryStore()
|
||||
const route = useRoute()
|
||||
const router = useRouter()
|
||||
|
||||
const title = ref('')
|
||||
const text = ref('')
|
||||
const mode = ref<'new' | 'append'>('new')
|
||||
const bookId = ref<number | undefined>(undefined)
|
||||
// The accepted prototype offers both sources on one screen; the server treats them the same.
|
||||
const source = ref<'paste' | 'txt'>('paste')
|
||||
const file = ref<File | null>(null)
|
||||
const fileInfo = ref<{ name: string; encoding: string; size: string } | null>(null)
|
||||
const titleError = ref('')
|
||||
const textError = ref('')
|
||||
const fileError = ref('')
|
||||
const bookError = ref('')
|
||||
|
||||
const length = computed(() => [...text.value].length)
|
||||
const busy = computed(() => library.submitting)
|
||||
|
||||
/** The browser pre-check only replaces the server, it never replaces its verdict. */
|
||||
async function readFile(selected: File): Promise<void> {
|
||||
fileError.value = ''
|
||||
fileInfo.value = null
|
||||
const problem = fileProblem(selected)
|
||||
if (problem) {
|
||||
file.value = null
|
||||
fileError.value = problem
|
||||
return
|
||||
}
|
||||
try {
|
||||
const decoder = new TextDecoder('utf-8', { fatal: true })
|
||||
decoder.decode(await selected.arrayBuffer())
|
||||
} catch {
|
||||
file.value = null
|
||||
fileError.value = '文件不是 UTF-8 编码，请另存为 UTF-8 后重试。'
|
||||
return
|
||||
}
|
||||
file.value = selected
|
||||
fileInfo.value = { name: selected.name, encoding: 'UTF-8', size: fileSizeLabel(selected.size) }
|
||||
// A file usually defines the title; the learner can still change it before submitting.
|
||||
if (!title.value.trim()) title.value = selected.name.replace(/\.txt$/i, '').slice(0, 120)
|
||||
}
|
||||
|
||||
function onFile(event: Event): void {
|
||||
const selected = (event.target as HTMLInputElement).files?.[0]
|
||||
file.value = null
|
||||
fileInfo.value = null
|
||||
fileError.value = ''
|
||||
if (selected) void readFile(selected)
|
||||
}
|
||||
|
||||
function requestedBookId(): number | undefined {
|
||||
const raw = Array.isArray(route.query.book) ? route.query.book[0] : route.query.book
|
||||
if (typeof raw !== 'string' || !/^\d+$/.test(raw)) return undefined
|
||||
const value = Number(raw)
|
||||
return Number.isSafeInteger(value) && value > 0 ? value : undefined
|
||||
}
|
||||
|
||||
onMounted(() => {
|
||||
// `?book=<id>` preselects "append to an existing book".
|
||||
const preselect = requestedBookId()
|
||||
if (preselect === undefined) return
|
||||
mode.value = 'append'
|
||||
bookId.value = preselect
|
||||
})
|
||||
|
||||
watch(mode, value => {
|
||||
// The select can only list the caller's own books.
|
||||
if (value === 'append') void library.loadBooks()
|
||||
})
|
||||
|
||||
watch(title, () => { titleError.value = '' })
|
||||
watch(text, () => { textError.value = '' })
|
||||
// Switching source clears the other source's complaints and its result.
|
||||
watch(source, () => {
|
||||
titleError.value = ''
|
||||
textError.value = ''
|
||||
fileError.value = ''
|
||||
library.submitError = ''
|
||||
})
|
||||
|
||||
async function submit() {
|
||||
if (busy.value) return
|
||||
titleError.value = titleProblem(title.value)
|
||||
textError.value = source.value === 'paste' ? textProblem(text.value) : ''
|
||||
fileError.value = source.value === 'txt' && !file.value ? (fileError.value || '请选择要导入的 TXT 文件。') : fileError.value
|
||||
bookError.value = mode.value === 'append' && bookId.value === undefined ? '请选择要追加的书籍。' : ''
|
||||
if (titleError.value || textError.value || fileError.value || bookError.value) return
|
||||
const target: SubmitTarget = mode.value === 'append' && bookId.value !== undefined
|
||||
? { mode: 'append', bookId: bookId.value }
|
||||
: { mode: 'new' }
|
||||
try {
|
||||
const createdBookId = source.value === 'txt' && file.value
|
||||
? await library.upload({ title: title.value, target, file: file.value })
|
||||
: await library.submit({ title: title.value, text: text.value, target })
|
||||
// A response that arrives after the user left this page must not navigate them back.
|
||||
if (disposed) return
|
||||
// The requestId was consumed by this submission, so the form starts clean.
|
||||
title.value = ''
|
||||
text.value = ''
|
||||
file.value = null
|
||||
fileInfo.value = null
|
||||
await router.replace(`/books/${createdBookId}`)
|
||||
} catch {
|
||||
// library.submitError already carries the server message for the template.
|
||||
}
|
||||
}
|
||||
|
||||
// The form may still be awaiting its submit when the user navigates away.
|
||||
let disposed = false
|
||||
onUnmounted(() => {
|
||||
disposed = true
|
||||
library.submitError = ''
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/">我的书库</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
</div>
|
||||
</header>
|
||||
<main class="page">
|
||||
<p class="breadcrumb"><RouterLink to="/">我的书库</RouterLink><span aria-hidden="true">/</span><span>导入内容</span></p>
|
||||
<div class="page-title">
|
||||
<div>
|
||||
<h1>导入英文内容</h1>
|
||||
<p class="subtle">粘贴英文正文，提交后系统会自动切分并处理章节。</p>
|
||||
</div>
|
||||
</div>
|
||||
<form class="import-form" novalidate @submit.prevent="submit">
|
||||
<div class="field">
|
||||
<span class="field-label">导入方式</span>
|
||||
<ElRadioGroup v-model="source" :disabled="busy" aria-label="导入方式">
|
||||
<ElRadio value="paste">粘贴文本</ElRadio>
|
||||
<ElRadio value="txt">TXT 文件</ElRadio>
|
||||
</ElRadioGroup>
|
||||
</div>
|
||||
<div class="field">
|
||||
<span class="field-label">语言</span>
|
||||
<p class="fixed-value">{{ LANGUAGE_LABEL }}</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="title">标题</label>
|
||||
<ElInput id="title" v-model="title" type="text" maxlength="200" placeholder="例如：虚构样例第一章" :disabled="busy" />
|
||||
<p v-if="titleError" role="alert" class="field-error">{{ titleError }}</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<span class="field-label">导入到</span>
|
||||
<ElRadioGroup v-model="mode" :disabled="busy" aria-label="导入目标">
|
||||
<ElRadio value="new">新建书籍</ElRadio>
|
||||
<ElRadio value="append">追加到已有书籍</ElRadio>
|
||||
</ElRadioGroup>
|
||||
</div>
|
||||
<div v-if="mode === 'append'" class="field">
|
||||
<label for="book">选择书籍</label>
|
||||
<ElSelect id="book" v-model="bookId" placeholder="请选择要追加的书籍" :disabled="busy" class="book-select">
|
||||
<ElOption v-for="item in library.books" :key="item.id" :label="item.title" :value="item.id" />
|
||||
</ElSelect>
|
||||
<p v-if="library.booksError" role="alert" class="field-error">{{ library.booksError }}</p>
|
||||
<p v-if="bookError" role="alert" class="field-error">{{ bookError }}</p>
|
||||
</div>
|
||||
<div v-if="source === 'paste'" class="field">
|
||||
<label for="text">正文</label>
|
||||
<ElInput id="text" v-model="text" type="textarea" :rows="12" placeholder="在此粘贴英文正文…" :disabled="busy" />
|
||||
<p class="counter">{{ length }} / {{ TEXT_MAX_CODE_POINTS }} 字符</p>
|
||||
<p v-if="textError" role="alert" class="field-error">{{ textError }}</p>
|
||||
</div>
|
||||
<div v-else class="field">
|
||||
<label for="file">TXT 文件</label>
|
||||
<input id="file" data-testid="file-input" type="file" accept=".txt,text/plain" :disabled="busy" @change="onFile" />
|
||||
<p v-if="fileInfo" class="counter" data-testid="file-info">{{ fileInfo.name }} · {{ fileInfo.encoding }} · {{ fileInfo.size }}</p>
|
||||
<p v-else class="subtle">仅支持 UTF-8 的 .txt 文件，最多 2 MiB；文件不会保存在服务器上。</p>
|
||||
<p v-if="fileError" role="alert" class="field-error">{{ fileError }}</p>
|
||||
</div>
|
||||
<p v-if="library.submitError" role="alert" class="notice">{{ library.submitError }}</p>
|
||||
<div class="form-actions">
|
||||
<ElButton type="primary" native-type="submit" :loading="busy" :disabled="busy">{{ source === 'txt' ? '上传并处理' : '开始处理' }}</ElButton>
|
||||
<RouterLink to="/" class="subtle">返回书库</RouterLink>
|
||||
</div>
|
||||
</form>
|
||||
</main>
|
||||
</div>
|
||||
</template>
|
||||
@@ -1,19 +1,30 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted, ref } from 'vue'
|
||||
import { RouterLink, useRouter } from 'vue-router'
|
||||
import { computed, onMounted, ref } from 'vue'
|
||||
import { RouterLink, useRoute, useRouter } from 'vue-router'
|
||||
import { ElButton } from 'element-plus'
|
||||
import BookMark from '../components/BookMark.vue'
|
||||
import { statusSummary, useLibraryStore } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
const session = useSessionStore()
|
||||
const library = useLibraryStore()
|
||||
const router = useRouter()
|
||||
const route = useRoute()
|
||||
const loading = ref(true)
|
||||
const error = ref('')
|
||||
// A deletion returns here with the count it removed, so the learner sees what happened.
|
||||
const deletedNotice = computed(() => {
|
||||
const raw = Array.isArray(route.query.deleted) ? route.query.deleted[0] : route.query.deleted
|
||||
if (typeof raw !== 'string' || !/^\d+$/.test(raw)) return ''
|
||||
return '书籍已删除 · 已保存的生词和短语仍保留在生词本。'
|
||||
})
|
||||
async function load() {
|
||||
loading.value = true
|
||||
error.value = ''
|
||||
try { await session.loadSpace() }
|
||||
catch (reason) { error.value = reason instanceof Error ? reason.message : '暂时无法加载，请重试。' }
|
||||
finally { loading.value = false }
|
||||
// The book list keeps its own error so one failing call still shows a retry.
|
||||
if (!error.value) await library.loadBooks()
|
||||
loading.value = false
|
||||
}
|
||||
async function logout() {
|
||||
try { await session.logout() }
|
||||
@@ -27,18 +38,36 @@ onMounted(load)
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/" class="active-nav">我的书库</RouterLink></nav>
|
||||
<nav aria-label="学习导航"><RouterLink to="/" class="active-nav">我的书库</RouterLink> · <RouterLink to="/review">到期复习</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
<ElButton text @click="logout">退出登录</ElButton>
|
||||
</div>
|
||||
</header>
|
||||
<main class="library">
|
||||
<div class="library-title"><div><h1>我的书库</h1><p class="subtle">你的阅读与学习，从这里开始。</p></div><span class="language">英语</span></div>
|
||||
<div class="library-title">
|
||||
<div><h1>我的书库</h1><p class="subtle">你的阅读与学习，从这里开始。</p></div>
|
||||
<div class="library-actions">
|
||||
<span class="language">英语</span>
|
||||
<ElButton type="primary" @click="router.push('/import')">导入内容</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
<p v-if="deletedNotice" role="status" class="chapter-notice" data-testid="library-notice">{{ deletedNotice }}</p>
|
||||
<p v-if="loading" role="status" class="loading">正在加载…</p>
|
||||
<div v-else-if="error" class="notice"><p role="alert">{{ error }}</p><ElButton @click="load">重试</ElButton></div>
|
||||
<section v-else-if="session.space" class="empty-library" aria-label="书库内容">
|
||||
<BookMark /><h2>书库还是空的</h2><p class="subtle">这里将收纳你的阅读内容。</p>
|
||||
<div v-else-if="error || library.booksError" class="notice">
|
||||
<p role="alert">{{ error || library.booksError }}</p>
|
||||
<ElButton @click="load">重试</ElButton>
|
||||
</div>
|
||||
<ul v-else-if="library.books.length" class="book-grid" aria-label="书籍列表">
|
||||
<li v-for="item in library.books" :key="item.id" class="book-card">
|
||||
<RouterLink :to="`/books/${item.id}`" class="book-title">{{ item.title }}</RouterLink>
|
||||
<p class="subtle">{{ item.chapterCount }} 个章节</p>
|
||||
<p v-if="statusSummary(item)" class="status-summary">{{ statusSummary(item) }}</p>
|
||||
<p v-else class="subtle">尚未导入章节</p>
|
||||
</li>
|
||||
</ul>
|
||||
<section v-else class="empty-library" aria-label="书库内容">
|
||||
<BookMark /><h2>书库还是空的</h2><p class="subtle">粘贴一段英文，开始你的第一篇阅读。</p>
|
||||
</section>
|
||||
</main>
|
||||
</div>
|
||||
|
||||
@@ -0,0 +1,201 @@
|
||||
<script setup lang="ts">
|
||||
import { computed, onMounted, onUnmounted, ref, watch } from 'vue'
|
||||
import { RouterLink, useRoute, useRouter } from 'vue-router'
|
||||
import { ElButton } from 'element-plus'
|
||||
import { canRetry, statusLabel, useLibraryStore } from '../stores/library'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
import { useReaderLookup, type PhraseSpan, type ReaderToken } from '../composables/useReaderLookup'
|
||||
import { adjustTokenRange, MAX_PHRASE_WORDS, normalizeTokenRange, phrasesAt, rangeOfSpan, wordIndices } from '../composables/readerRange'
|
||||
import { useTextSelection } from '../composables/useTextSelection'
|
||||
import ReaderTokens from '../components/ReaderTokens.vue'
|
||||
import LookupPanel from '../components/LookupPanel.vue'
|
||||
|
||||
const session = useSessionStore()
|
||||
const library = useLibraryStore()
|
||||
const route = useRoute()
|
||||
const router = useRouter()
|
||||
const retryError = ref('')
|
||||
const rangeNotice = ref('')
|
||||
const readerBody = ref<HTMLElement | null>(null)
|
||||
|
||||
const chapterId = computed(() => Number(route.params.id))
|
||||
const chapter = computed(() => library.chapter)
|
||||
const lookup = useReaderLookup(chapter)
|
||||
// Retry uses the job id the chapter carries, no matter where it was loaded from.
|
||||
const retryable = computed(() => chapter.value !== null && canRetry(chapter.value))
|
||||
|
||||
/** A dragged selection becomes a phrase; a single word stays a plain word lookup. */
|
||||
function pickRange(anchor: number, focus: number, element: HTMLElement | null): void {
|
||||
const text = library.readerText
|
||||
const count = wordIndices(lookup.tokens.value.slice(anchor, focus + 1)).length
|
||||
if (count < 2) return
|
||||
if (count > MAX_PHRASE_WORDS) {
|
||||
rangeNotice.value = `短语最多 ${MAX_PHRASE_WORDS} 个单词，请缩短选择范围。`
|
||||
return
|
||||
}
|
||||
const range = normalizeTokenRange(lookup.tokens.value, text, anchor, focus)
|
||||
if (!range) return
|
||||
rangeNotice.value = ''
|
||||
lookup.selectRange(range, element)
|
||||
}
|
||||
|
||||
const selection = useTextSelection({ container: readerBody, onSelect: (anchor, focus) => pickRange(anchor, focus, readerBody.value) })
|
||||
|
||||
/** Shift-clicking a word extends the phrase from the word that was picked first. */
|
||||
function selectToken(token: ReaderToken, element: HTMLElement, extend: boolean): void {
|
||||
rangeNotice.value = ''
|
||||
const anchor = lookup.selected.value
|
||||
if (extend && anchor) {
|
||||
const from = Number(element.dataset.tokenIndex)
|
||||
const first = lookup.tokens.value.findIndex(item => item.start === anchor.start)
|
||||
if (from >= 0 && first >= 0) {
|
||||
pickRange(Math.min(from, first), Math.max(from, first), element)
|
||||
return
|
||||
}
|
||||
}
|
||||
lookup.select(token, element)
|
||||
}
|
||||
|
||||
/** Clicking inside a saved phrase edits the phrase instead of the word under the cursor. */
|
||||
function selectPhrase(span: PhraseSpan, element: HTMLElement): void {
|
||||
const range = rangeOfSpan(lookup.tokens.value, library.readerText, span)
|
||||
if (!range) return
|
||||
rangeNotice.value = ''
|
||||
lookup.selectRange(range, element, span.id)
|
||||
}
|
||||
|
||||
const adjustOptions = computed(() => {
|
||||
const current = lookup.range.value
|
||||
if (!current) return null
|
||||
const text = library.readerText
|
||||
return {
|
||||
startLeft: adjustTokenRange(lookup.tokens.value, text, current, 'start', -1) !== null,
|
||||
startRight: adjustTokenRange(lookup.tokens.value, text, current, 'start', 1) !== null,
|
||||
endLeft: adjustTokenRange(lookup.tokens.value, text, current, 'end', -1) !== null,
|
||||
endRight: adjustTokenRange(lookup.tokens.value, text, current, 'end', 1) !== null,
|
||||
}
|
||||
})
|
||||
|
||||
const panelWord = computed(() => lookup.range.value?.text ?? lookup.selected.value?.text ?? '')
|
||||
|
||||
function closePanel(): void {
|
||||
lookup.close()
|
||||
selection.clear()
|
||||
}
|
||||
|
||||
async function load() {
|
||||
lookup.reset()
|
||||
retryError.value = ''
|
||||
rangeNotice.value = ''
|
||||
await library.loadChapter(chapterId.value)
|
||||
}
|
||||
|
||||
async function retry() {
|
||||
if (chapter.value === null) return
|
||||
retryError.value = ''
|
||||
try { await library.retryChapter(chapter.value.id) }
|
||||
catch (reason) { retryError.value = reason instanceof Error ? reason.message : '重试失败，请稍后重试。' }
|
||||
}
|
||||
|
||||
// Switching chapters reuses this component; only the route param changes.
|
||||
function goTo(id: number | null) {
|
||||
if (id === null) return
|
||||
void router.push(`/chapters/${id}`)
|
||||
}
|
||||
|
||||
async function logout() {
|
||||
try { await session.logout() }
|
||||
catch { session.notice = '已退出此设备。服务器暂时无法连接，请稍后重试。' }
|
||||
finally { await router.replace('/login') }
|
||||
}
|
||||
|
||||
onMounted(load)
|
||||
watch(chapterId, () => { void load() })
|
||||
// Leaving the page releases the chapter so polling stops.
|
||||
onUnmounted(() => library.closeChapter())
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/">我的书库</RouterLink> · <RouterLink to="/review">到期复习</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
<ElButton text @click="logout">退出登录</ElButton>
|
||||
</div>
|
||||
</header>
|
||||
<main class="page reader-page" :class="{ 'has-lookup': lookup.selected.value || lookup.range.value }" @keydown.esc="closePanel">
|
||||
<p v-if="library.chapterLoading && !chapter" role="status" class="loading">正在加载…</p>
|
||||
<div v-else-if="library.chapterError" class="notice">
|
||||
<p role="alert">{{ library.chapterError }}</p>
|
||||
<ElButton @click="load">重试</ElButton>
|
||||
</div>
|
||||
<template v-else-if="chapter">
|
||||
<p class="breadcrumb">
|
||||
<RouterLink to="/">我的书库</RouterLink>
|
||||
<span aria-hidden="true">/</span>
|
||||
<RouterLink v-if="library.chapterBook" :to="`/books/${library.chapterBook.id}`">{{ library.chapterBook.title }}</RouterLink>
|
||||
<span v-else>章节</span>
|
||||
</p>
|
||||
<div class="page-title">
|
||||
<h1>{{ chapter.title }}</h1>
|
||||
<span class="status-chip" :class="`status-${chapter.status}`">{{ statusLabel(chapter.status) }}</span>
|
||||
</div>
|
||||
<div v-if="chapter.status === 'failed'" class="notice">
|
||||
<p role="alert">{{ chapter.errorMessage || '这一章处理失败。' }}</p>
|
||||
<ElButton v-if="retryable" :loading="library.retryingChapterId === chapter.id" @click="retry">重试处理</ElButton>
|
||||
<p v-else class="subtle">这一章暂时没有可重试的任务编号。</p>
|
||||
</div>
|
||||
<p v-else-if="chapter.status !== 'ready'" role="status" class="processing-hint">这一章还在{{ statusLabel(chapter.status) }}，页面会自动刷新。</p>
|
||||
<p v-if="retryError" role="alert" class="notice">{{ retryError }}</p>
|
||||
<p v-if="rangeNotice" role="alert" class="notice" data-testid="range-notice">{{ rangeNotice }}</p>
|
||||
<div v-if="chapter.status === 'ready'" class="reader-workspace">
|
||||
<div class="reader-body" ref="readerBody">
|
||||
<ReaderTokens
|
||||
:tokens="lookup.tokens.value"
|
||||
:phrases="lookup.phrases.value"
|
||||
:original="library.readerText"
|
||||
:selected-start="lookup.selected.value?.start"
|
||||
:phrase-start="lookup.rangeTermId.value"
|
||||
@select="selectToken"
|
||||
@select-phrase="selectPhrase"
|
||||
/>
|
||||
<div v-if="lookup.tokensError.value" class="tokens-notice">
|
||||
<p role="status">{{ lookup.tokensError.value }}</p>
|
||||
<ElButton data-testid="tokens-retry" :loading="lookup.tokensLoading.value" @click="lookup.loadTokens">重试加载单词</ElButton>
|
||||
</div>
|
||||
</div>
|
||||
<LookupPanel
|
||||
v-if="lookup.selected.value || lookup.range.value"
|
||||
:word="panelWord"
|
||||
:phrase="lookup.range.value ? { wordCount: lookup.range.value.wordCount, stored: lookup.rangeStored.value } : null"
|
||||
:adjust="adjustOptions"
|
||||
:result="lookup.result.value"
|
||||
:loading="lookup.loading.value"
|
||||
:error="lookup.error.value"
|
||||
:saving="lookup.saving.value"
|
||||
:save-error="lookup.saveError.value"
|
||||
:saved="lookup.saved.value"
|
||||
:saved-term-id="lookup.savedTermId.value"
|
||||
:prefilling="lookup.prefilling.value"
|
||||
:prefill-error="lookup.prefillError.value"
|
||||
:can-save="lookup.canSave.value"
|
||||
v-model:definition="lookup.definition.value"
|
||||
v-model:examples="lookup.examples.value"
|
||||
v-model:status="lookup.status.value"
|
||||
@close="closePanel"
|
||||
@retry="lookup.lookup"
|
||||
@save="lookup.save"
|
||||
@adjust="(edge, direction) => { lookup.adjustRange(edge, direction); selection.clear() }"
|
||||
@resize="lookup.keepSelectionVisible"
|
||||
/>
|
||||
</div>
|
||||
<nav class="reader-nav" aria-label="章节切换">
|
||||
<ElButton :disabled="library.navigation.previousChapterId === null" @click="goTo(library.navigation.previousChapterId)">上一章</ElButton>
|
||||
<ElButton :disabled="library.navigation.nextChapterId === null" @click="goTo(library.navigation.nextChapterId)">下一章</ElButton>
|
||||
</nav>
|
||||
</template>
|
||||
</main>
|
||||
</div>
|
||||
</template>
|
||||
@@ -0,0 +1,78 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted } from 'vue'
|
||||
import { RouterLink, useRouter } from 'vue-router'
|
||||
import { ElButton } from 'element-plus'
|
||||
import ReviewCard from '../components/ReviewCard.vue'
|
||||
import { useReviewStore, type ReviewItem } from '../stores/review'
|
||||
import { useSessionStore } from '../stores/session'
|
||||
|
||||
const session = useSessionStore()
|
||||
const review = useReviewStore()
|
||||
const router = useRouter()
|
||||
|
||||
function endReview() { void router.push('/') }
|
||||
// Retrying reuses the grade of the failed attempt; the store keeps the answer id.
|
||||
let lastGrade: 'correct' | 'wrong' | 'again' = 'correct'
|
||||
function grade(value: 'correct' | 'wrong' | 'again') { lastGrade = value; void review.answer(value) }
|
||||
function retry() { void review.answer(lastGrade) }
|
||||
|
||||
onMounted(() => { void review.load() })
|
||||
async function logout() {
|
||||
try { await session.logout() }
|
||||
catch { session.notice = '已退出此设备。服务器暂时无法连接，请稍后重试。' }
|
||||
finally { await router.replace('/login') }
|
||||
}
|
||||
// The template guards for a card before rendering it; this keeps the prop type strict.
|
||||
const requireItem = (value: ReviewItem | null): ReviewItem => value as ReviewItem
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="session.user">
|
||||
<header class="site-header">
|
||||
<RouterLink to="/" class="brand">LexGo<span class="brand-dot">.</span></RouterLink>
|
||||
<nav aria-label="学习导航"><RouterLink to="/">我的书库</RouterLink></nav>
|
||||
<div class="account">
|
||||
<span class="account-name">{{ session.user.username }}</span>
|
||||
<ElButton text @click="logout">退出登录</ElButton>
|
||||
</div>
|
||||
</header>
|
||||
<main class="page review-page">
|
||||
<h1>到期复习</h1>
|
||||
<p v-if="review.notice" role="status" class="review-notice" data-testid="review-notice">{{ review.notice }}</p>
|
||||
<p v-if="review.loading" role="status" class="loading">正在加载…</p>
|
||||
<div v-else-if="review.error && !review.current" class="notice">
|
||||
<p role="alert">{{ review.error }}</p>
|
||||
<ElButton data-testid="review-reload" @click="review.load()">重试</ElButton>
|
||||
</div>
|
||||
<section v-else-if="review.finished" class="review-summary" data-testid="review-summary">
|
||||
<h2>本次复习完成</h2>
|
||||
<p v-if="review.answered">复习了 {{ review.wordsReviewed }} 个词条 · 共 {{ review.answered }} 次作答</p>
|
||||
<p v-else class="subtle">本轮没有新的计分：{{ review.resolved }} 个词条已在其他页面复习。</p>
|
||||
<p class="subtle">答对 {{ review.correctCount }} · 答错或再学 {{ review.wrongCount }} · 已更新复习计划</p>
|
||||
<p v-if="review.pending" role="status" class="notice">还有 {{ review.pending }} 个词条到期。</p>
|
||||
<div class="review-grades">
|
||||
<ElButton v-if="review.pending" type="primary" data-testid="review-more" @click="review.continueRound()">继续复习</ElButton>
|
||||
<ElButton data-testid="review-finish" @click="endReview">继续阅读</ElButton>
|
||||
</div>
|
||||
</section>
|
||||
<section v-else-if="review.empty" class="review-summary" data-testid="review-empty">
|
||||
<h2>今天没有到期词条</h2>
|
||||
<p class="subtle">读一篇文章，积累下一次的词汇。</p>
|
||||
<ElButton data-testid="review-finish" @click="endReview">返回书库</ElButton>
|
||||
</section>
|
||||
<ReviewCard
|
||||
v-else-if="review.current"
|
||||
:item="requireItem(review.current)"
|
||||
:position="review.answered + 1"
|
||||
:total="review.answered + review.queue.length"
|
||||
:revealed="review.revealed"
|
||||
:busy="review.busy"
|
||||
:error="review.error"
|
||||
@reveal="review.reveal()"
|
||||
@grade="grade"
|
||||
@end="endReview"
|
||||
@retry="retry"
|
||||
/>
|
||||
</main>
|
||||
</div>
|
||||
</template>
|
||||
@@ -0,0 +1,31 @@
|
||||
WordNet Release 3.0
|
||||
|
||||
This software and database is being provided to you, the LICENSEE, by
|
||||
Princeton University under the following license. By obtaining, using
|
||||
and/or copying this software and database, you agree that you have
|
||||
read, understood, and will comply with these terms and conditions.:
|
||||
|
||||
Permission to use, copy, modify and distribute this software and
|
||||
database and its documentation for any purpose and without fee or
|
||||
royalty is hereby granted, provided that you agree to comply with
|
||||
the following copyright notice and statements, including the disclaimer,
|
||||
and that the same appear on ALL copies of the software, database and
|
||||
documentation, including modifications that you make for internal
|
||||
use or for distribution.
|
||||
|
||||
WordNet 3.0 Copyright 2006 by Princeton University. All rights reserved.
|
||||
|
||||
THIS SOFTWARE AND DATABASE IS PROVIDED "AS IS" AND PRINCETON
|
||||
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
|
||||
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PRINCETON
|
||||
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES OF MERCHANT-
|
||||
ABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE
|
||||
OF THE LICENSED SOFTWARE, DATABASE OR DOCUMENTATION WILL NOT
|
||||
INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR
|
||||
OTHER RIGHTS.
|
||||
|
||||
The name of Princeton University or Princeton may not be used in
|
||||
advertising or publicity pertaining to distribution of the software
|
||||
and/or database. Title to copyright in this software, database and
|
||||
any associated documentation shall at all times remain with
|
||||
Princeton University and LICENSEE agrees to preserve same.
|
||||
+159
-39
@@ -4,10 +4,11 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Migrate takes a connection-scoped lock. Only an empty or LexGo-owned schema is accepted.
|
||||
@@ -59,44 +60,26 @@ func Migrate(db *gorm.DB) error {
|
||||
if err = conn.QueryRowContext(ctx, "SELECT version,product FROM lexgo_schema WHERE id=1").Scan(¤t, &product); err != nil {
|
||||
return err
|
||||
}
|
||||
if product != "lexgo" || current < 0 || current > 2 {
|
||||
if product != "lexgo" || current < 0 || current > SchemaVersion {
|
||||
return errors.New("unknown schema version")
|
||||
}
|
||||
if current == 2 {
|
||||
return nil
|
||||
// Each known version contributes its own statements; the version row advances only
|
||||
// after every statement succeeded, so a partially applied migration can be retried.
|
||||
statements := make([]string, 0, 16)
|
||||
if current < 2 {
|
||||
statements = append(statements, schemaV2Statements...)
|
||||
}
|
||||
statements := []string{
|
||||
`CREATE TABLE IF NOT EXISTS sys_user (
|
||||
user_id BIGINT PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL UNIQUE,
|
||||
password VARCHAR(128) NOT NULL, nick_name VARCHAR(128) NOT NULL DEFAULT '', phone VARCHAR(11) NOT NULL DEFAULT '',
|
||||
role_id INT NOT NULL, salt VARCHAR(255) NOT NULL DEFAULT '', avatar VARCHAR(255) NOT NULL DEFAULT '',
|
||||
sex VARCHAR(255) NOT NULL DEFAULT '', email VARCHAR(128) NOT NULL DEFAULT '', dept_id BIGINT NOT NULL DEFAULT 0,
|
||||
post_id BIGINT NOT NULL DEFAULT 0, remark VARCHAR(255) NOT NULL DEFAULT '', status VARCHAR(4) NOT NULL DEFAULT '2',
|
||||
create_by BIGINT NOT NULL DEFAULT 0, update_by BIGINT NOT NULL DEFAULT 0,
|
||||
created_at DATETIME(3) NULL, updated_at DATETIME(3) NULL, deleted_at DATETIME(3) NULL,
|
||||
CHECK (role_id IN (1,2)), CHECK (status IN ('1','2'))
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_spaces (
|
||||
owner_id BIGINT PRIMARY KEY, language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_sessions (
|
||||
token_hash CHAR(64) CHARACTER SET ascii COLLATE ascii_bin PRIMARY KEY,
|
||||
owner_id BIGINT NOT NULL, expires_at DATETIME(3) NOT NULL,
|
||||
INDEX (owner_id), INDEX (expires_at),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_login_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL, ip VARCHAR(45) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_operation_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, actor_id BIGINT NOT NULL, actor_username VARCHAR(32) NOT NULL,
|
||||
target_id BIGINT NOT NULL DEFAULT 0, target_username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
action VARCHAR(32) NOT NULL, result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL,
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(actor_username,created_at), INDEX(target_username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
if current < 3 {
|
||||
statements = append(statements, schemaV3Statements...)
|
||||
}
|
||||
if current < 4 {
|
||||
statements = append(statements, schemaV4Statements...)
|
||||
}
|
||||
if current < 5 {
|
||||
statements = append(statements, schemaV5Statements...)
|
||||
}
|
||||
if current < 6 {
|
||||
statements = append(statements, schemaV6Statements...)
|
||||
}
|
||||
for i, s := range statements {
|
||||
if _, err = conn.ExecContext(ctx, s); err != nil {
|
||||
@@ -107,16 +90,153 @@ func Migrate(db *gorm.DB) error {
|
||||
return fmt.Errorf("migration statement %d failed", i+1)
|
||||
}
|
||||
}
|
||||
_, err = conn.ExecContext(ctx, "UPDATE lexgo_schema SET version=2 WHERE id=1")
|
||||
_, err = conn.ExecContext(ctx, fmt.Sprintf("UPDATE lexgo_schema SET version=%d WHERE id=1", SchemaVersion))
|
||||
return err
|
||||
}
|
||||
|
||||
// SchemaVersion is the version an explicit migration leaves behind, and the
|
||||
// version the server requires before it starts.
|
||||
const SchemaVersion = 6
|
||||
|
||||
var schemaV6Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_term_reviews (
|
||||
term_id BIGINT UNSIGNED PRIMARY KEY,
|
||||
owner_id BIGINT NOT NULL, language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
due_at DATETIME(3) NOT NULL, review_count INT NOT NULL DEFAULT 0,
|
||||
correct_count INT NOT NULL DEFAULT 0, wrong_count INT NOT NULL DEFAULT 0,
|
||||
last_reviewed_at DATETIME(3) NULL,
|
||||
INDEX idx_term_review_due (owner_id, language, due_at, term_id),
|
||||
FOREIGN KEY (term_id) REFERENCES lexgo_terms(id) ON DELETE CASCADE,
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`INSERT IGNORE INTO lexgo_term_reviews (term_id, owner_id, language, due_at)
|
||||
SELECT id, owner_id, language, created_at FROM lexgo_terms`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_review_answers (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT,
|
||||
owner_id BIGINT NOT NULL,
|
||||
answer_key CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
|
||||
term_id BIGINT UNSIGNED NOT NULL, grade VARCHAR(16) NOT NULL, result VARCHAR(16) NOT NULL,
|
||||
status_before VARCHAR(16) NOT NULL, status_after VARCHAR(16) NOT NULL,
|
||||
level_before TINYINT NOT NULL, level_after TINYINT NOT NULL,
|
||||
due_at_before DATETIME(3) NOT NULL, due_at_after DATETIME(3) NOT NULL,
|
||||
requeued BOOLEAN NOT NULL DEFAULT FALSE, created_at DATETIME(3) NOT NULL,
|
||||
UNIQUE KEY uq_review_answer (owner_id, answer_key),
|
||||
INDEX idx_review_answer_owner (owner_id, created_at, id), INDEX idx_review_answer_term (term_id, created_at),
|
||||
CHECK (grade IN ('correct','wrong','again')), CHECK (result IN ('applied','stale')),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY (term_id) REFERENCES lexgo_terms(id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
// v5 adds one learner's own records for a word form. Identity is the normalized
|
||||
// form under the owner's language, so re-saving the same word updates one row
|
||||
// instead of creating a second, conflicting record. utf8mb4_bin keeps the key
|
||||
// byte-exact: the Go side normalizes, and an accent-insensitive collation must
|
||||
// not fold "resume" and "résumé" into the same entry.
|
||||
var schemaV5Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_terms (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, owner_id BIGINT NOT NULL,
|
||||
language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
term VARCHAR(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
|
||||
original_form VARCHAR(191) NOT NULL,
|
||||
definition TEXT NOT NULL, examples TEXT NOT NULL,
|
||||
status VARCHAR(16) NOT NULL, level TINYINT NOT NULL DEFAULT 0,
|
||||
created_at DATETIME(3) NOT NULL, updated_at DATETIME(3) NOT NULL,
|
||||
UNIQUE KEY uq_term_identity (owner_id, language, term),
|
||||
CONSTRAINT ck_term_status CHECK (status IN ('new','learning','known','ignored')),
|
||||
CONSTRAINT ck_term_level CHECK ((status = 'learning' AND level BETWEEN 1 AND 7) OR (status <> 'learning' AND level = 0)),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
var schemaV4Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_dictionaries (
|
||||
id BIGINT PRIMARY KEY, name VARCHAR(120) NOT NULL, language VARCHAR(16) NOT NULL,
|
||||
version VARCHAR(32) NOT NULL, source VARCHAR(512) NOT NULL, format VARCHAR(32) NOT NULL,
|
||||
sha256 CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
|
||||
entry_count INT NOT NULL, enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
archive LONGBLOB NOT NULL, updated_at DATETIME(3) NOT NULL,
|
||||
CHECK (id = 1), CHECK (language = 'en')
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
var schemaV2Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS sys_user (
|
||||
user_id BIGINT PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL UNIQUE,
|
||||
password VARCHAR(128) NOT NULL, nick_name VARCHAR(128) NOT NULL DEFAULT '', phone VARCHAR(11) NOT NULL DEFAULT '',
|
||||
role_id INT NOT NULL, salt VARCHAR(255) NOT NULL DEFAULT '', avatar VARCHAR(255) NOT NULL DEFAULT '',
|
||||
sex VARCHAR(255) NOT NULL DEFAULT '', email VARCHAR(128) NOT NULL DEFAULT '', dept_id BIGINT NOT NULL DEFAULT 0,
|
||||
post_id BIGINT NOT NULL DEFAULT 0, remark VARCHAR(255) NOT NULL DEFAULT '', status VARCHAR(4) NOT NULL DEFAULT '2',
|
||||
create_by BIGINT NOT NULL DEFAULT 0, update_by BIGINT NOT NULL DEFAULT 0,
|
||||
created_at DATETIME(3) NULL, updated_at DATETIME(3) NULL, deleted_at DATETIME(3) NULL,
|
||||
CHECK (role_id IN (1,2)), CHECK (status IN ('1','2'))
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_spaces (
|
||||
owner_id BIGINT PRIMARY KEY, language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_sessions (
|
||||
token_hash CHAR(64) CHARACTER SET ascii COLLATE ascii_bin PRIMARY KEY,
|
||||
owner_id BIGINT NOT NULL, expires_at DATETIME(3) NOT NULL,
|
||||
INDEX (owner_id), INDEX (expires_at),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_login_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL, ip VARCHAR(45) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_operation_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, actor_id BIGINT NOT NULL, actor_username VARCHAR(32) NOT NULL,
|
||||
target_id BIGINT NOT NULL DEFAULT 0, target_username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
action VARCHAR(32) NOT NULL, result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL,
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(actor_username,created_at), INDEX(target_username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
// v3 adds the private library: books, chapters with byte-exact original text, and
|
||||
// persistent ingestion jobs. owner_id is denormalized onto chapters and jobs so every
|
||||
// query can filter by the authenticated identity without joining.
|
||||
var schemaV3Statements = []string{
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_books (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, owner_id BIGINT NOT NULL,
|
||||
title VARCHAR(120) NOT NULL, language VARCHAR(16) NOT NULL DEFAULT 'en',
|
||||
created_at DATETIME(3) NOT NULL, updated_at DATETIME(3) NOT NULL,
|
||||
INDEX (owner_id, updated_at, id),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_chapters (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, book_id BIGINT UNSIGNED NOT NULL, owner_id BIGINT NOT NULL,
|
||||
ordinal INT NOT NULL, title VARCHAR(120) NOT NULL,
|
||||
original_text MEDIUMTEXT NOT NULL, char_count INT NOT NULL DEFAULT 0,
|
||||
content_sha256 CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL DEFAULT '',
|
||||
status VARCHAR(16) NOT NULL DEFAULT 'pending', error_reason VARCHAR(32) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, updated_at DATETIME(3) NOT NULL,
|
||||
UNIQUE KEY uq_chapter_ordinal (book_id, ordinal), INDEX (owner_id, id),
|
||||
CHECK (status IN ('pending','processing','ready','failed')),
|
||||
FOREIGN KEY (book_id) REFERENCES lexgo_books(id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_ingest_jobs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, owner_id BIGINT NOT NULL,
|
||||
book_id BIGINT UNSIGNED NOT NULL, chapter_id BIGINT UNSIGNED NOT NULL,
|
||||
request_key CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
|
||||
content_sha256 CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
|
||||
status VARCHAR(16) NOT NULL DEFAULT 'pending', attempts INT NOT NULL DEFAULT 0,
|
||||
error_reason VARCHAR(32) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, updated_at DATETIME(3) NOT NULL, finished_at DATETIME(3) NULL,
|
||||
UNIQUE KEY uq_job_request (owner_id, request_key), INDEX (status, id),
|
||||
CHECK (status IN ('pending','processing','ready','failed')),
|
||||
FOREIGN KEY (book_id) REFERENCES lexgo_books(id) ON DELETE CASCADE,
|
||||
FOREIGN KEY (chapter_id) REFERENCES lexgo_chapters(id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
|
||||
func CheckSchema(db *gorm.DB) error {
|
||||
var r struct {
|
||||
Version int
|
||||
Product string
|
||||
}
|
||||
if err := db.Table("lexgo_schema").Where("id=1").First(&r).Error; err != nil || r.Version != 2 || r.Product != "lexgo" {
|
||||
if err := db.Table("lexgo_schema").Where("id=1").First(&r).Error; err != nil || r.Version != SchemaVersion || r.Product != "lexgo" {
|
||||
return errors.New("run the explicit migration before starting")
|
||||
}
|
||||
return nil
|
||||
|
||||
@@ -0,0 +1,329 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// ID 1 is the sole English corpus slot. Keeping the validated ZIP in MySQL
|
||||
// makes ordinary database backups include the resource needed after a restart.
|
||||
type DictionaryResource struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement:false"`
|
||||
Name string
|
||||
Language string
|
||||
Version string
|
||||
Source string
|
||||
Format string
|
||||
SHA256 string `gorm:"column:sha256"`
|
||||
EntryCount int
|
||||
Enabled bool
|
||||
Archive []byte
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (DictionaryResource) TableName() string { return "lexgo_dictionaries" }
|
||||
|
||||
type DictionaryView struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Language string `json:"language"`
|
||||
Version string `json:"version"`
|
||||
Source string `json:"source"`
|
||||
Format string `json:"format"`
|
||||
Status string `json:"status"`
|
||||
Enabled bool `json:"enabled"`
|
||||
SHA256 string `json:"sha256"`
|
||||
EntryCount int `json:"entryCount"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func dictionaryView(r DictionaryResource, status string) DictionaryView {
|
||||
return DictionaryView{r.ID, r.Name, r.Language, r.Version, r.Source, r.Format, status, r.Enabled, r.SHA256, r.EntryCount, r.UpdatedAt}
|
||||
}
|
||||
|
||||
type DictionaryImportResult struct {
|
||||
Resource DictionaryView `json:"resource"`
|
||||
Duplicate bool `json:"duplicate"`
|
||||
}
|
||||
type ChapterTokens struct {
|
||||
TextSHA256 string `json:"textSha256"`
|
||||
Tokens []TextToken `json:"tokens"`
|
||||
Phrases []PhraseSpan `json:"phrases"`
|
||||
}
|
||||
|
||||
// Each router keeps at most one immutable parsed corpus; no private chapter or
|
||||
// lookup data enters the cache. The mutex also coalesces simultaneous cold loads.
|
||||
type dictionaryCache struct {
|
||||
mu sync.Mutex
|
||||
sha string
|
||||
engine *WordNet
|
||||
}
|
||||
|
||||
func (cache *dictionaryCache) load(tx *gorm.DB, r DictionaryResource) (*WordNet, error) {
|
||||
cache.mu.Lock()
|
||||
defer cache.mu.Unlock()
|
||||
if r.SHA256 != WordNetSHA {
|
||||
return nil, errors.New("unsupported resource checksum")
|
||||
}
|
||||
if cache.sha == r.SHA256 && cache.engine != nil {
|
||||
return cache.engine, nil
|
||||
}
|
||||
var stored DictionaryResource
|
||||
if err := tx.Select("id", "archive").Where("id = ? AND sha256 = ?", r.ID, r.SHA256).First(&stored).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
engine, err := ParseWordNet(stored.Archive)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
cache.sha = r.SHA256
|
||||
cache.engine = engine
|
||||
return engine, nil
|
||||
}
|
||||
|
||||
func readyOwnedChapter(tx *gorm.DB, owner int, id int64) (Chapter, error) {
|
||||
var chapter Chapter
|
||||
err := tx.Where("id = ? AND owner_id = ?", id, owner).First(&chapter).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return chapter, failure(404, "章节不存在")
|
||||
}
|
||||
if err != nil {
|
||||
return chapter, err
|
||||
}
|
||||
if chapter.Status != statusReady {
|
||||
return chapter, failure(409, "章节尚未就绪，请稍后重试")
|
||||
}
|
||||
return chapter, nil
|
||||
}
|
||||
|
||||
func readDictionaryUpload(c *gin.Context) (DictionaryResource, error) {
|
||||
bad := failure(400, "词典上传无效，请使用指定的 WordNet 3.0 ZIP 和完整资源信息")
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, maxDictionaryZip+(64<<10))
|
||||
reader, err := c.Request.MultipartReader()
|
||||
if err != nil {
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
fields := map[string]string{}
|
||||
var archive []byte
|
||||
for {
|
||||
part, err := reader.NextPart()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
name := part.FormName()
|
||||
if name == "file" {
|
||||
if archive != nil || part.FileName() == "" {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
archive, err = io.ReadAll(io.LimitReader(part, maxDictionaryZip+1))
|
||||
if err != nil || len(archive) == 0 || len(archive) > maxDictionaryZip {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
} else {
|
||||
if part.FileName() != "" || (name != "name" && name != "language" && name != "version" && name != "source" && name != "format") {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
if _, exists := fields[name]; exists {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
value, e := io.ReadAll(io.LimitReader(part, 1025))
|
||||
if e != nil || len(value) > 1024 || !utf8.Valid(value) {
|
||||
part.Close()
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
fields[name] = string(value)
|
||||
}
|
||||
part.Close()
|
||||
}
|
||||
name := strings.TrimSpace(fields["name"])
|
||||
if name == "" || utf8.RuneCountInString(name) > 120 || strings.IndexFunc(name, unicode.IsControl) >= 0 || fields["language"] != "en" || fields["version"] != "3.0" || fields["format"] != "wordnet-3.0-zip" || fields["source"] != WordNetSource || len(archive) == 0 {
|
||||
return DictionaryResource{}, bad
|
||||
}
|
||||
return DictionaryResource{ID: 1, Name: name, Language: "en", Version: "3.0", Source: WordNetSource, Format: "wordnet-3.0-zip", SHA256: WordNetSHA, Enabled: true, Archive: archive}, nil
|
||||
}
|
||||
|
||||
func registerDictionaryRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
cache := &dictionaryCache{}
|
||||
uploadGate := make(chan struct{}, 1)
|
||||
v.GET("/dictionaries", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "词典列表不接受查询参数")
|
||||
}
|
||||
var rows []DictionaryResource
|
||||
if err := tx.Omit("archive").Order("id").Find(&rows).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items := make([]DictionaryView, 0, len(rows))
|
||||
for _, row := range rows {
|
||||
status := "disabled"
|
||||
if row.Enabled {
|
||||
status = "ready"
|
||||
if _, err := cache.load(tx, row); err != nil {
|
||||
status = "unavailable"
|
||||
}
|
||||
}
|
||||
items = append(items, dictionaryView(row, status))
|
||||
}
|
||||
return gin.H{"items": items, "supported": gin.H{"name": "Princeton WordNet", "language": "en", "version": "3.0", "format": "wordnet-3.0-zip", "source": WordNetSource, "sha256": WordNetSHA}}, nil
|
||||
}))
|
||||
v.POST("/dictionaries/import", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
select {
|
||||
case uploadGate <- struct{}{}:
|
||||
defer func() { <-uploadGate }()
|
||||
default:
|
||||
return nil, failure(429, "已有词典正在导入，请稍后重试")
|
||||
}
|
||||
resource, err := readDictionaryUpload(c)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
engine, err := ParseWordNet(resource.Archive)
|
||||
if err != nil {
|
||||
return nil, failure(400, "词典文件校验失败，请选择指定的 WordNet 3.0 ZIP")
|
||||
}
|
||||
resource.EntryCount = engine.EntryCount
|
||||
resource.UpdatedAt = stamp(now())
|
||||
// INSERT ... ON CONFLICT followed by a locking read serializes even the first
|
||||
// concurrent import. A validated replacement and its metadata commit together.
|
||||
created := tx.Clauses(clause.OnConflict{DoNothing: true}).Create(&resource)
|
||||
if created.Error != nil {
|
||||
return nil, created.Error
|
||||
}
|
||||
var existing DictionaryResource
|
||||
if err = tx.Omit("archive").Clauses(clause.Locking{Strength: "UPDATE"}).Where("id = 1").First(&existing).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
duplicate := created.RowsAffected == 0 && existing.SHA256 == resource.SHA256
|
||||
if created.RowsAffected == 0 {
|
||||
if err = tx.Model(&DictionaryResource{}).Where("id = 1").Select("name", "language", "version", "source", "format", "sha256", "entry_count", "enabled", "archive", "updated_at").Updates(&resource).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
existing = resource
|
||||
}
|
||||
status := "ready"
|
||||
if !existing.Enabled {
|
||||
status = "disabled"
|
||||
}
|
||||
return DictionaryImportResult{dictionaryView(existing, status), duplicate}, nil
|
||||
}))
|
||||
v.PATCH("/dictionaries/:id", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
|
||||
if err != nil || id != 1 {
|
||||
return nil, failure(404, "词典不存在")
|
||||
}
|
||||
var input struct {
|
||||
Enabled *bool `json:"enabled"`
|
||||
}
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if input.Enabled == nil {
|
||||
return nil, failure(400, "请指定词典启用状态")
|
||||
}
|
||||
var resource DictionaryResource
|
||||
if err = tx.Omit("archive").Clauses(clause.Locking{Strength: "UPDATE"}).Where("id = ?", id).First(&resource).Error; errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil, failure(404, "词典不存在")
|
||||
} else if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if *input.Enabled {
|
||||
if _, err = cache.load(tx, resource); err != nil {
|
||||
return nil, failure(409, "词典资源不可用，请重新导入")
|
||||
}
|
||||
}
|
||||
resource.Enabled = *input.Enabled
|
||||
resource.UpdatedAt = stamp(now())
|
||||
if err = tx.Model(&resource).Updates(map[string]any{"enabled": resource.Enabled, "updated_at": resource.UpdatedAt}).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
status := "disabled"
|
||||
if resource.Enabled {
|
||||
status = "ready"
|
||||
}
|
||||
return gin.H{"resource": dictionaryView(resource, status)}, nil
|
||||
}))
|
||||
v.GET("/chapters/:id/tokens", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "分词接口不接受查询参数")
|
||||
}
|
||||
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
return nil, failure(404, "章节不存在")
|
||||
}
|
||||
chapter, err := readyOwnedChapter(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tokens := Tokenize(chapter.OriginalText)
|
||||
language, err := languageOf(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// The learner's own saved words and phrases are part of the chapter view, so the
|
||||
// reader shows the same status here as everywhere else.
|
||||
if err = attachTerms(tx, u.UserId, language, tokens); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
phrases, err := phrasesForChapter(tx, u.UserId, language, tokens)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ChapterTokens{chapter.ContentSHA256, tokens, phrases}, nil
|
||||
}))
|
||||
v.POST("/lookup", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
var input struct {
|
||||
ChapterID int64 `json:"chapterId"`
|
||||
Start *int `json:"start"`
|
||||
End *int `json:"end"`
|
||||
}
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
chapter, err := readyOwnedChapter(tx, u.UserId, input.ChapterID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if input.Start == nil || input.End == nil {
|
||||
return nil, failure(400, "请选择完整单词")
|
||||
}
|
||||
query, err := wordAtRange(chapter, *input.Start, *input.End)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
missing := LookupResult{Status: "resource_missing", Query: query, Candidates: []string{}, Entries: []DictionaryEntry{}}
|
||||
var resource DictionaryResource
|
||||
err = tx.Omit("archive").Where("id = 1 AND enabled = ?", true).First(&resource).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return missing, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
engine, err := cache.load(tx, resource)
|
||||
if err != nil {
|
||||
return missing, nil
|
||||
}
|
||||
result := engine.Lookup(query)
|
||||
result.Resource = &LookupResource{Name: resource.Name, Version: resource.Version}
|
||||
return result, nil
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"mime/multipart"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
)
|
||||
|
||||
func importDictionaryAPI(t *testing.T, r *gin.Engine, token string, raw []byte) (int, DictionaryImportResult) {
|
||||
t.Helper()
|
||||
var body bytes.Buffer
|
||||
w := multipart.NewWriter(&body)
|
||||
for k, v := range map[string]string{"name": "Princeton WordNet", "language": "en", "version": "3.0", "source": WordNetSource, "format": "wordnet-3.0-zip"} {
|
||||
if err := w.WriteField(k, v); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
p, err := w.CreateFormFile("file", "wordnet.zip")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
p.Write(raw)
|
||||
w.Close()
|
||||
req := httptest.NewRequest("POST", "/api/v1/dictionaries/import", &body)
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
response := httptest.NewRecorder()
|
||||
r.ServeHTTP(response, req)
|
||||
var e envelope
|
||||
json.Unmarshal(response.Body.Bytes(), &e)
|
||||
var result DictionaryImportResult
|
||||
json.Unmarshal(e.Data, &result)
|
||||
return response.Code, result
|
||||
}
|
||||
|
||||
func TestDictionaryAPIResourcesAndOwnership(t *testing.T) {
|
||||
db := testDB(t)
|
||||
raw, err := os.ReadFile("../../../.local/nlp-resources/wordnet.zip")
|
||||
if os.IsNotExist(err) {
|
||||
t.Skip("prepare WordNet fixture")
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Test-only dedicated database slot; no development/production resources touched.
|
||||
if err := db.Exec("DELETE FROM lexgo_dictionaries").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { db.Exec("DELETE FROM lexgo_dictionaries") })
|
||||
r := Router(db, time.Now)
|
||||
users := []admin.SysUser{}
|
||||
tokens := []string{}
|
||||
for i, role := range []int{1, 2, 2} {
|
||||
u := admin.SysUser{Username: randomName(fmt.Sprintf("dict%d", i)), Password: fixturePassword, RoleId: role, Status: "2"}
|
||||
if err := db.Create(&u).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
users = append(users, u)
|
||||
tokens = append(tokens, loginToken(t, r, u.Username, fixturePassword))
|
||||
}
|
||||
book := Book{OwnerID: users[1].UserId, Title: "Fictional", Language: "en", CreatedAt: stamp(time.Now()), UpdatedAt: stamp(time.Now())}
|
||||
if err := db.Create(&book).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
original := "😀 Dogs went. Cafe\u0301 123"
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: users[1].UserId, Ordinal: 1, Title: "Fictional", OriginalText: original, ContentSHA256: contentSHA(original), CharCount: len([]rune(original)), Status: statusReady, CreatedAt: stamp(time.Now()), UpdatedAt: stamp(time.Now())}
|
||||
if err := db.Create(&chapter).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lookup := func(token string, start, end int) (int, LookupResult) {
|
||||
code, data := callAPI(t, r, "POST", "/api/v1/lookup", token, map[string]any{"chapterId": chapter.ID, "start": start, "end": end})
|
||||
var got LookupResult
|
||||
json.Unmarshal(data, &got)
|
||||
return code, got
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", "/api/v1/dictionaries", tokens[1], nil); code != 403 {
|
||||
t.Fatal("learner resource list", code)
|
||||
}
|
||||
if code, _ := importDictionaryAPI(t, r, tokens[1], []byte("bad")); code != 403 {
|
||||
t.Fatal("learner import", code)
|
||||
}
|
||||
if code, _ := lookup(tokens[2], 2, 6); code != 404 {
|
||||
t.Fatal("foreign lookup", code)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", chapter.ID), tokens[2], nil); code != 404 {
|
||||
t.Fatal("foreign tokens", code)
|
||||
}
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "resource_missing" {
|
||||
t.Fatal("missing", code, got)
|
||||
}
|
||||
code, data := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", chapter.ID), tokens[1], nil)
|
||||
var analyzed ChapterTokens
|
||||
json.Unmarshal(data, &analyzed)
|
||||
if code != 200 || analyzed.TextSHA256 != contentSHA(original) || analyzed.Tokens[2].StartUtf16 != 3 {
|
||||
t.Fatal("tokens", code, string(data))
|
||||
}
|
||||
for _, span := range [][2]int{{-1, 3}, {2, 5}, {0, 1}, {2, 11}, {18, 21}, {1, 2}, {6, 2}} {
|
||||
if code, _ := lookup(tokens[1], span[0], span[1]); code != 400 {
|
||||
t.Fatal("invalid interval", span, code)
|
||||
}
|
||||
}
|
||||
code, imported := importDictionaryAPI(t, r, tokens[0], raw)
|
||||
if code != 200 || imported.Duplicate || imported.Resource.Status != "ready" {
|
||||
t.Fatal("import", code, imported)
|
||||
}
|
||||
if code, got := lookup(tokens[1], 7, 11); code != 200 || got.Status != "lemma" || *got.MatchedForm != "go" {
|
||||
t.Fatal("went", code, got)
|
||||
}
|
||||
code, duplicate := importDictionaryAPI(t, r, tokens[0], raw)
|
||||
if code != 200 || !duplicate.Duplicate || duplicate.Resource.ID != imported.Resource.ID {
|
||||
t.Fatal("duplicate", code, duplicate)
|
||||
}
|
||||
if code, _ := importDictionaryAPI(t, r, tokens[0], []byte("broken")); code != 400 {
|
||||
t.Fatal("bad import", code)
|
||||
}
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "lemma" {
|
||||
t.Fatal("failed import lost old resource", code, got)
|
||||
}
|
||||
endpoint := fmt.Sprintf("/api/v1/dictionaries/%d", imported.Resource.ID)
|
||||
if code, _ := callAPI(t, r, "PATCH", endpoint, tokens[1], map[string]bool{"enabled": false}); code != 403 {
|
||||
t.Fatal("learner toggle", code)
|
||||
}
|
||||
for _, enabled := range []bool{false, true} {
|
||||
if code, _ := callAPI(t, r, "PATCH", endpoint, tokens[0], map[string]bool{"enabled": enabled}); code != 200 {
|
||||
t.Fatal("toggle", code)
|
||||
}
|
||||
code, got := lookup(tokens[1], 2, 6)
|
||||
if code != 200 || (enabled && got.Status != "lemma") || (!enabled && got.Status != "resource_missing") {
|
||||
t.Fatal("enabled state", enabled, code, got)
|
||||
}
|
||||
}
|
||||
r = Router(db, time.Now) // A new router has an empty cache and reloads the persisted ZIP.
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "lemma" {
|
||||
t.Fatal("cold restart", code, got)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "PATCH", endpoint, tokens[0], map[string]bool{"enabled": false}); code != 200 {
|
||||
t.Fatal("disable before duplicate", code)
|
||||
}
|
||||
if code, result := importDictionaryAPI(t, r, tokens[0], raw); code != 200 || !result.Duplicate || !result.Resource.Enabled {
|
||||
t.Fatal("duplicate must re-enable", code, result)
|
||||
}
|
||||
var count int64
|
||||
if err := db.Model(&DictionaryResource{}).Count(&count).Error; err != nil || count != 1 {
|
||||
t.Fatal("duplicate created extra resource", count, err)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "PATCH", endpoint, tokens[0], map[string]any{"enabled": nil}); code != 400 {
|
||||
t.Fatal("null toggle", code)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "POST", "/api/v1/lookup", tokens[1], map[string]any{"chapterId": chapter.ID, "start": 2, "end": 6, "ownerId": users[2].UserId}); code != 400 {
|
||||
t.Fatal("unknown lookup input", code)
|
||||
}
|
||||
if err := db.Model(&DictionaryResource{}).Where("id=1").Update("archive", []byte("corrupt fixture")).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r = Router(db, time.Now)
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "resource_missing" {
|
||||
t.Fatal("corrupt cold resource", code, got)
|
||||
}
|
||||
if code, got := callAPI(t, r, "GET", "/api/v1/dictionaries", tokens[0], nil); code != 200 || !bytes.Contains(got, []byte(`"status":"unavailable"`)) {
|
||||
t.Fatal("corrupt resource state", code, string(got))
|
||||
}
|
||||
if code, result := importDictionaryAPI(t, r, tokens[0], raw); code != 200 || !result.Duplicate {
|
||||
t.Fatal("repair reimport", code, result)
|
||||
}
|
||||
if code, got := lookup(tokens[1], 2, 6); code != 200 || got.Status != "lemma" {
|
||||
t.Fatal("repair lookup", code, got)
|
||||
}
|
||||
if err := db.Model(&chapter).Update("status", statusPending).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code, _ := lookup(tokens[1], 2, 6); code != 409 {
|
||||
t.Fatal("pending lookup", code)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,286 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// Editing and deleting are strictly owner scoped. Nothing here touches personal terms,
|
||||
// review rows or answers: those belong to the learner, not to a book or a chapter.
|
||||
|
||||
type BookUpdateInput struct {
|
||||
Title string `json:"title"`
|
||||
}
|
||||
|
||||
type ChapterUpdateInput struct {
|
||||
Title *string `json:"title"`
|
||||
Text *string `json:"text"`
|
||||
}
|
||||
|
||||
// ChapterSource is the editable text of one owned chapter. It is separate from the reader
|
||||
// contract, which only exposes text once a chapter is ready, so a failed chapter can be
|
||||
// corrected and submitted again.
|
||||
type ChapterSource struct {
|
||||
ID int64 `json:"id"`
|
||||
BookID int64 `json:"bookId"`
|
||||
Ordinal int `json:"ordinal"`
|
||||
Title string `json:"title"`
|
||||
Text string `json:"text"`
|
||||
Status string `json:"status"`
|
||||
ContentSHA256 string `json:"contentSha256"`
|
||||
CharCount int `json:"charCount"`
|
||||
}
|
||||
|
||||
type ChapterEdit struct {
|
||||
Chapter ChapterSummary `json:"chapter"`
|
||||
Job *JobView `json:"job"`
|
||||
VersionChanged bool `json:"versionChanged"`
|
||||
}
|
||||
|
||||
type DeletionResult struct {
|
||||
BookID int64 `json:"bookId,omitempty"`
|
||||
ChapterID int64 `json:"chapterId,omitempty"`
|
||||
Chapters int `json:"chapters,omitempty"`
|
||||
Remaining int `json:"remaining"`
|
||||
}
|
||||
|
||||
// editTitle validates a title with the same rules a paste uses, so a renamed book or chapter
|
||||
// stays within the limits the list and reader already rely on.
|
||||
func editTitle(raw string) (string, error) {
|
||||
title := strings.TrimSpace(raw)
|
||||
if title == "" {
|
||||
return "", failure(400, "请填写标题")
|
||||
}
|
||||
if utf8.RuneCountInString(title) > maxTitleRunes {
|
||||
return "", failure(400, "标题最多 120 个字符")
|
||||
}
|
||||
return title, nil
|
||||
}
|
||||
|
||||
// lockOwnedChapter returns the caller's chapter or reports it as missing, so another
|
||||
// account's chapter id is never confirmed to exist.
|
||||
func lockOwnedChapter(tx *gorm.DB, owner int, chapterID int64, chapter *Chapter) error {
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", chapterID, owner).First(chapter).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return failure(404, "章节不存在")
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func RenameBook(tx *gorm.DB, owner int, bookID int64, input BookUpdateInput, now time.Time) (BookRef, error) {
|
||||
title, err := editTitle(input.Title)
|
||||
if err != nil {
|
||||
return BookRef{}, err
|
||||
}
|
||||
var book Book
|
||||
if err = lockOwnedBook(tx, owner, bookID, &book); err != nil {
|
||||
return BookRef{}, err
|
||||
}
|
||||
book.Title = title
|
||||
book.UpdatedAt = stamp(now)
|
||||
if err = tx.Model(&Book{}).Where("id = ? AND owner_id = ?", book.ID, owner).
|
||||
Updates(map[string]any{"title": title, "updated_at": book.UpdatedAt}).Error; err != nil {
|
||||
return BookRef{}, err
|
||||
}
|
||||
return bookRef(book), nil
|
||||
}
|
||||
|
||||
func ChapterEditSource(tx *gorm.DB, owner int, chapterID int64) (ChapterSource, error) {
|
||||
var chapter Chapter
|
||||
if err := tx.Where("id = ? AND owner_id = ?", chapterID, owner).First(&chapter).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ChapterSource{}, failure(404, "章节不存在")
|
||||
}
|
||||
return ChapterSource{}, err
|
||||
}
|
||||
return ChapterSource{
|
||||
ID: chapter.ID, BookID: chapter.BookID, Ordinal: chapter.Ordinal, Title: chapter.Title,
|
||||
Text: chapter.OriginalText, Status: chapter.Status, ContentSHA256: chapter.ContentSHA256,
|
||||
CharCount: chapter.CharCount,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// UpdateChapter renames a chapter and, when the text really changed, stores it as a new
|
||||
// version and queues a job for it. Only a changed version re-processes: a repeated save of
|
||||
// the same text is idempotent, and a title-only edit never touches the processing state.
|
||||
func UpdateChapter(tx *gorm.DB, owner int, chapterID int64, input ChapterUpdateInput, now time.Time) (ChapterEdit, error) {
|
||||
if input.Title == nil && input.Text == nil {
|
||||
return ChapterEdit{}, failure(400, "请选择要修改的内容")
|
||||
}
|
||||
var chapter Chapter
|
||||
if err := lockOwnedChapter(tx, owner, chapterID, &chapter); err != nil {
|
||||
return ChapterEdit{}, err
|
||||
}
|
||||
ts := stamp(now)
|
||||
updates := map[string]any{"updated_at": ts}
|
||||
if input.Title != nil {
|
||||
title, err := editTitle(*input.Title)
|
||||
if err != nil {
|
||||
return ChapterEdit{}, err
|
||||
}
|
||||
chapter.Title = title
|
||||
updates["title"] = title
|
||||
}
|
||||
var job *IngestJob
|
||||
if input.Text != nil {
|
||||
text := *input.Text
|
||||
if _, sha, count, err := validatePaste(chapter.Title, text); err != nil {
|
||||
return ChapterEdit{}, err
|
||||
} else if sha != chapter.ContentSHA256 {
|
||||
// A new version replaces the text and owns the chapter's state from here on.
|
||||
chapter.OriginalText, chapter.ContentSHA256, chapter.CharCount = text, sha, count
|
||||
chapter.Status, chapter.ErrorReason = statusPending, ""
|
||||
updates["original_text"] = text
|
||||
updates["content_sha256"] = sha
|
||||
updates["char_count"] = count
|
||||
updates["status"] = statusPending
|
||||
updates["error_reason"] = ""
|
||||
// The request key is derived from chapter and version, so one version has one job.
|
||||
key := contentSHA(fmt.Sprintf("edit:%d:%s", chapter.ID, sha))
|
||||
created := IngestJob{OwnerID: owner, BookID: chapter.BookID, ChapterID: chapter.ID,
|
||||
RequestKey: key, ContentSHA256: sha, Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err := tx.Create(&created).Error; err != nil {
|
||||
return ChapterEdit{}, err
|
||||
}
|
||||
job = &created
|
||||
}
|
||||
}
|
||||
if err := tx.Model(&Chapter{}).Where("id = ? AND owner_id = ?", chapter.ID, owner).Updates(updates).Error; err != nil {
|
||||
return ChapterEdit{}, err
|
||||
}
|
||||
result := ChapterEdit{VersionChanged: job != nil}
|
||||
if job != nil {
|
||||
view := jobView(*job)
|
||||
result.Job = &view
|
||||
}
|
||||
jobID := int64(0)
|
||||
if job != nil {
|
||||
jobID = job.ID
|
||||
}
|
||||
result.Chapter = chapterSummaryWithJob(chapter, &jobID)
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// DeleteBook removes the caller's book with its chapters and their jobs in one transaction.
|
||||
// Personal terms, review schedules and answers are not touched: they belong to the learner.
|
||||
func DeleteBook(tx *gorm.DB, owner int, bookID int64) (DeletionResult, error) {
|
||||
var book Book
|
||||
if err := lockOwnedBook(tx, owner, bookID, &book); err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
var chapters int64
|
||||
if err := tx.Model(&Chapter{}).Where("book_id = ? AND owner_id = ?", book.ID, owner).Count(&chapters).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
// Chapters and their jobs go with the book through the foreign keys.
|
||||
if err := tx.Where("id = ? AND owner_id = ?", book.ID, owner).Delete(&Book{}).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
return DeletionResult{BookID: book.ID, Chapters: int(chapters)}, nil
|
||||
}
|
||||
|
||||
// DeleteChapter removes one owned chapter with its jobs and closes the gap in the chapter
|
||||
// order, so "剩余 N 章" and the reader navigation stay contiguous.
|
||||
func DeleteChapter(tx *gorm.DB, owner int, chapterID int64) (DeletionResult, error) {
|
||||
var chapter Chapter
|
||||
if err := lockOwnedChapter(tx, owner, chapterID, &chapter); err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
// Lock the book too: two concurrent deletions in one book must not renumber each other.
|
||||
var book Book
|
||||
if err := lockOwnedBook(tx, owner, chapter.BookID, &book); err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
if err := tx.Where("id = ? AND owner_id = ?", chapter.ID, owner).Delete(&Chapter{}).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
// Ordering ascending decrements each row into a slot the previous row just freed, which
|
||||
// keeps the unique (book_id, ordinal) key satisfied throughout.
|
||||
if err := tx.Exec("UPDATE lexgo_chapters SET ordinal = ordinal - 1 WHERE book_id = ? AND owner_id = ? AND ordinal > ? ORDER BY ordinal ASC",
|
||||
chapter.BookID, owner, chapter.Ordinal).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
var remaining int64
|
||||
if err := tx.Model(&Chapter{}).Where("book_id = ? AND owner_id = ?", chapter.BookID, owner).Count(&remaining).Error; err != nil {
|
||||
return DeletionResult{}, err
|
||||
}
|
||||
return DeletionResult{ChapterID: chapter.ID, BookID: chapter.BookID, Remaining: int(remaining)}, nil
|
||||
}
|
||||
|
||||
func registerEditRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
v.PATCH("/books/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var input BookUpdateInput
|
||||
if err = decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
book, err := RenameBook(tx, u.UserId, id, input, now())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"book": book}, nil
|
||||
}))
|
||||
v.DELETE("/books/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result, err := DeleteBook(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"deleted": result}, nil
|
||||
}))
|
||||
v.GET("/chapters/:id/source", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "正文接口不接受查询参数")
|
||||
}
|
||||
id, err := pathID(c, "章节不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
source, err := ChapterEditSource(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"source": source}, nil
|
||||
}))
|
||||
v.PATCH("/chapters/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "章节不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var input ChapterUpdateInput
|
||||
if err = decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
edited, err := UpdateChapter(tx, u.UserId, id, input, now())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return edited, nil
|
||||
}))
|
||||
v.DELETE("/chapters/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "章节不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result, err := DeleteChapter(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"deleted": result}, nil
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,543 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
const editFixtureText = "Mira opened the workshop.\nThe sign read “A small step…”\n"
|
||||
|
||||
func TestEditTitleRules(t *testing.T) {
|
||||
if title, err := editTitle(" A small step "); err != nil || title != "A small step" {
|
||||
t.Fatalf("trimmed title: %q %v", title, err)
|
||||
}
|
||||
if _, err := editTitle(" "); err == nil {
|
||||
t.Fatal("an empty title must be rejected")
|
||||
}
|
||||
if _, err := editTitle(strings.Repeat("a", maxTitleRunes+1)); err == nil {
|
||||
t.Fatal("a title over the limit must be rejected")
|
||||
}
|
||||
if _, err := editTitle(strings.Repeat("a", maxTitleRunes)); err != nil {
|
||||
t.Fatalf("the exact title limit must be accepted: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func patchResource(t *testing.T, r *gin.Engine, token, path string, body any) (int, string, json.RawMessage) {
|
||||
t.Helper()
|
||||
return callRaw(t, r, "PATCH", path, token, body)
|
||||
}
|
||||
|
||||
func existingTitle(t *testing.T, r *gin.Engine, token string, bookID int64) string {
|
||||
t.Helper()
|
||||
code, detail := bookDetail(t, r, token, bookID)
|
||||
if code != 200 {
|
||||
t.Fatalf("book detail status %d", code)
|
||||
}
|
||||
return detail.Book.Title
|
||||
}
|
||||
|
||||
// TestMySQLRenameBookAndChapter covers renaming only: the text and the processing state stay
|
||||
// exactly as they were.
|
||||
func TestMySQLRenameBookAndChapter(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "rename-fixture-0001", "title": "Before Rename", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
before := chapterRow(t, db, pasted.Chapter.ID)
|
||||
|
||||
code, msg, data := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]string{"title": " After Rename "})
|
||||
if code != 200 {
|
||||
t.Fatalf("rename book status %d (%s)", code, msg)
|
||||
}
|
||||
var renamed struct {
|
||||
Book struct {
|
||||
ID int64
|
||||
Title string
|
||||
}
|
||||
}
|
||||
json.Unmarshal(data, &renamed)
|
||||
if renamed.Book.Title != "After Rename" || existingTitle(t, r, learner.Token, pasted.Book.ID) != "After Rename" {
|
||||
t.Fatalf("renamed book %+v", renamed.Book)
|
||||
}
|
||||
|
||||
code, msg, data = patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"title": "Chapter Two"})
|
||||
if code != 200 {
|
||||
t.Fatalf("rename chapter status %d (%s)", code, msg)
|
||||
}
|
||||
var edited ChapterEdit
|
||||
json.Unmarshal(data, &edited)
|
||||
if edited.Chapter.Title != "Chapter Two" || edited.VersionChanged || edited.Job != nil {
|
||||
t.Fatalf("rename must not re-process: %+v", edited)
|
||||
}
|
||||
after := chapterRow(t, db, pasted.Chapter.ID)
|
||||
if after.Title != "Chapter Two" || after.Status != statusReady || after.ContentSHA256 != before.ContentSHA256 || after.OriginalText != before.OriginalText {
|
||||
t.Fatalf("rename changed the content state: %+v", after)
|
||||
}
|
||||
var jobs int64
|
||||
db.Model(&IngestJob{}).Where("chapter_id = ?", pasted.Chapter.ID).Count(&jobs)
|
||||
if jobs != 1 {
|
||||
t.Fatalf("a rename created %d jobs", jobs)
|
||||
}
|
||||
|
||||
// Validation and ownership.
|
||||
for _, body := range []map[string]string{{"title": " "}, {"title": strings.Repeat("a", maxTitleRunes+1)}} {
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), body); code != 400 {
|
||||
t.Fatalf("invalid rename %v accepted", body)
|
||||
}
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]any{"title": "x", "ownerId": 9}); code != 400 {
|
||||
t.Fatal("an unknown field must be rejected")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]any{"name": "x"}); code != 400 {
|
||||
t.Fatal("a missing title must be rejected")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, other.Token, fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]string{"title": "Stolen"}); code != 404 {
|
||||
t.Fatal("another account must not rename this book")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, other.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"title": "Stolen"}); code != 404 {
|
||||
t.Fatal("another account must not rename this chapter")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, "", fmt.Sprintf("/api/v1/books/%d", pasted.Book.ID), map[string]string{"title": "Anonymous"}); code != 401 {
|
||||
t.Fatal("renaming requires a session")
|
||||
}
|
||||
if title := existingTitle(t, r, learner.Token, pasted.Book.ID); title != "After Rename" {
|
||||
t.Fatalf("a rejected rename changed the book: %q", title)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLChapterEditVersioning is the core of the ticket: an edit creates a new version, and
|
||||
// the older run must not fail or publish over it.
|
||||
func TestMySQLChapterEditVersioning(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "edit-fixture-0001", "title": "Editable", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// The first run is claimed but not finished, so the edit lands while it is in flight.
|
||||
clock := time.Now().UTC().Truncate(time.Millisecond)
|
||||
job, claimed, err := ClaimNextIngestJob(db, clock)
|
||||
if err != nil || !claimed || job.ChapterID != pasted.Chapter.ID {
|
||||
t.Fatalf("claim (claimed=%v): %v", claimed, err)
|
||||
}
|
||||
|
||||
newText := "Mira reopened the workshop.\r\n\r\nA newer version of the text.\n"
|
||||
code, msg, data := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"text": newText})
|
||||
if code != 200 {
|
||||
t.Fatalf("edit status %d (%s)", code, msg)
|
||||
}
|
||||
var edited ChapterEdit
|
||||
json.Unmarshal(data, &edited)
|
||||
if !edited.VersionChanged || edited.Job == nil || edited.Chapter.Status != statusPending {
|
||||
t.Fatalf("edit must queue a new version: %+v", edited)
|
||||
}
|
||||
row := chapterRow(t, db, pasted.Chapter.ID)
|
||||
if row.OriginalText != newText || row.ContentSHA256 != contentSHA(newText) || row.Status != statusPending || row.CharCount != len([]rune(newText)) {
|
||||
t.Fatalf("stored version: %+v", row)
|
||||
}
|
||||
if row.ID != pasted.Chapter.ID {
|
||||
t.Fatal("an edit must keep the chapter id, so the reading entry stays the same")
|
||||
}
|
||||
|
||||
// The in-flight run belongs to the previous version: it must not touch the chapter.
|
||||
if err := FinishIngestJob(t.Context(), db, job, clock.Add(time.Second)); err != nil {
|
||||
t.Fatalf("finishing the older run must not fail: %v", err)
|
||||
}
|
||||
var stale IngestJob
|
||||
if err := db.First(&stale, job.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if stale.Status != statusFailed || stale.ErrorReason != reasonSuperseded {
|
||||
t.Fatalf("the older run must be marked superseded: %+v", stale)
|
||||
}
|
||||
if row = chapterRow(t, db, pasted.Chapter.ID); row.Status != statusPending || row.ErrorReason != "" || row.OriginalText != newText {
|
||||
t.Fatalf("the older run changed the newer version: %+v", row)
|
||||
}
|
||||
// Retrying the superseded job is refused instead of reprocessing old text.
|
||||
code, _, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", stale.ID), learner.Token, nil)
|
||||
if code != 409 {
|
||||
t.Fatalf("retrying a superseded job: %d", code)
|
||||
}
|
||||
|
||||
// The new version publishes normally and the reader shows the new text.
|
||||
drainIngest(t, db)
|
||||
code, reader := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || reader.Chapter.Status != statusReady || reader.Chapter.OriginalText != newText {
|
||||
t.Fatalf("new version: status %d, %+v", code, reader.Chapter)
|
||||
}
|
||||
|
||||
// Saving the same text again is not a new version.
|
||||
var jobsBefore int64
|
||||
db.Model(&IngestJob{}).Where("chapter_id = ?", pasted.Chapter.ID).Count(&jobsBefore)
|
||||
code, _, data = patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"text": newText})
|
||||
json.Unmarshal(data, &edited)
|
||||
if code != 200 || edited.VersionChanged || edited.Job != nil {
|
||||
t.Fatalf("an unchanged text must not create a version: %+v", edited)
|
||||
}
|
||||
var jobsAfter int64
|
||||
db.Model(&IngestJob{}).Where("chapter_id = ?", pasted.Chapter.ID).Count(&jobsAfter)
|
||||
if jobsAfter != jobsBefore {
|
||||
t.Fatalf("an unchanged text created a job: %d -> %d", jobsBefore, jobsAfter)
|
||||
}
|
||||
if row = chapterRow(t, db, pasted.Chapter.ID); row.Status != statusReady {
|
||||
t.Fatalf("an unchanged save changed the status: %+v", row)
|
||||
}
|
||||
|
||||
// Text rules match a paste, and an empty body is refused.
|
||||
for _, body := range []map[string]string{{"text": " \n\t "}, {"text": strings.Repeat("a", maxChapterRunes+1)}} {
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), body); code != 400 {
|
||||
t.Fatalf("invalid text %v accepted", body)
|
||||
}
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{}); code != 400 {
|
||||
t.Fatal("an empty edit must be rejected")
|
||||
}
|
||||
if code, _, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]any{"text": "Body.\n", "ownerId": 9}); code != 400 {
|
||||
t.Fatal("an unknown field must be rejected")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLChapterSourceAnyStatus(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "source-fixture-0001", "title": "Source", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
readSource := func(token string, chapterID int64) (int, ChapterSource) {
|
||||
code, _, data := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/source", chapterID), token, nil)
|
||||
var payload struct {
|
||||
Source ChapterSource
|
||||
}
|
||||
if len(data) > 0 {
|
||||
json.Unmarshal(data, &payload)
|
||||
}
|
||||
return code, payload.Source
|
||||
}
|
||||
// A pending chapter has no reading text, but its edit source is available to its owner.
|
||||
if code, source := readSource(learner.Token, pasted.Chapter.ID); code != 200 || source.Text != editFixtureText || source.Status != statusPending || source.ContentSHA256 != contentSHA(editFixtureText) {
|
||||
t.Fatalf("pending source: %d %+v", code, source)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
if code, source := readSource(learner.Token, pasted.Chapter.ID); code != 200 || source.Status != statusReady || source.CharCount != len([]rune(editFixtureText)) {
|
||||
t.Fatalf("ready source: %d %+v", code, source)
|
||||
}
|
||||
if code, _ := readSource(other.Token, pasted.Chapter.ID); code != 404 {
|
||||
t.Fatal("another account must not read this source")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/source?id=1", pasted.Chapter.ID), learner.Token, nil); code != 400 {
|
||||
t.Fatal("the source endpoint must reject query parameters")
|
||||
}
|
||||
if code, _ := readSource(learner.Token, 999999); code != 404 {
|
||||
t.Fatal("an unknown chapter must be 404")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLDeleteChapterRenumbers closes the gap in the order and keeps personal records.
|
||||
func TestMySQLDeleteChapterRenumbers(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "delete-fixture-0001", "title": "Three Chapters", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("first paste %d", code)
|
||||
}
|
||||
second := pasteInto(t, r, learner.Token, first.Book.ID, "delete-fixture-0002", "Second", "Second chapter body.\n")
|
||||
third := pasteInto(t, r, learner.Token, first.Book.ID, "delete-fixture-0003", "Third", "Third chapter body.\n")
|
||||
drainIngest(t, db)
|
||||
// One personal word in the chapter that will be deleted, plus one in a surviving chapter.
|
||||
saved := saveWord(t, r, learner.Token, second.Chapter.ID, 0, 6, termStatusNew, nil)
|
||||
survivor := saveWord(t, r, learner.Token, first.Chapter.ID, 0, 4, termStatusNew, nil)
|
||||
before := termReviewRow(t, db, saved.Term.ID)
|
||||
|
||||
code, msg, data := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", second.Chapter.ID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("delete chapter status %d (%s)", code, msg)
|
||||
}
|
||||
var deleted struct {
|
||||
Deleted DeletionResult
|
||||
}
|
||||
json.Unmarshal(data, &deleted)
|
||||
if deleted.Deleted.ChapterID != second.Chapter.ID || deleted.Deleted.Remaining != 2 || deleted.Deleted.BookID != first.Book.ID {
|
||||
t.Fatalf("delete result: %+v", deleted.Deleted)
|
||||
}
|
||||
// The remaining chapters are contiguous and in the original order.
|
||||
code, detail := bookDetail(t, r, learner.Token, first.Book.ID)
|
||||
if code != 200 || len(detail.Chapters) != 2 {
|
||||
t.Fatalf("book detail after delete: %d %+v", code, detail.Chapters)
|
||||
}
|
||||
if detail.Chapters[0].ID != first.Chapter.ID || detail.Chapters[0].Ordinal != 1 || detail.Chapters[1].ID != third.Chapter.ID || detail.Chapters[1].Ordinal != 2 {
|
||||
t.Fatalf("renumbered chapters: %+v", detail.Chapters)
|
||||
}
|
||||
// Navigation follows the new order, and the deleted chapter is gone with its job.
|
||||
code, reader := readChapter(t, r, learner.Token, first.Chapter.ID)
|
||||
if code != 200 || reader.Navigation.NextChapterID == nil || *reader.Navigation.NextChapterID != third.Chapter.ID {
|
||||
t.Fatalf("navigation after delete: %d %+v", code, reader.Navigation)
|
||||
}
|
||||
if code, _ := readChapter(t, r, learner.Token, second.Chapter.ID); code != 404 {
|
||||
t.Fatal("a deleted chapter must be gone")
|
||||
}
|
||||
var jobs int64
|
||||
db.Model(&IngestJob{}).Where("chapter_id = ?", second.Chapter.ID).Count(&jobs)
|
||||
if jobs != 0 {
|
||||
t.Fatalf("the deleted chapter kept %d jobs", jobs)
|
||||
}
|
||||
// Personal records survive the deletion, including the schedule.
|
||||
var terms int64
|
||||
db.Model(&Term{}).Where("id IN ?", []int64{saved.Term.ID, survivor.Term.ID}).Count(&terms)
|
||||
var reviews int64
|
||||
db.Model(&TermReview{}).Where("term_id = ?", saved.Term.ID).Count(&reviews)
|
||||
if terms != 2 || reviews != 1 {
|
||||
t.Fatalf("deleting a chapter changed personal records: terms=%d reviews=%d", terms, reviews)
|
||||
}
|
||||
if after := termReviewRow(t, db, saved.Term.ID); !after.DueAt.Equal(before.DueAt) || after.ReviewCount != before.ReviewCount {
|
||||
t.Fatalf("the review schedule changed: %+v -> %+v", before, after)
|
||||
}
|
||||
// Repeated and foreign deletions.
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", second.Chapter.ID), learner.Token, nil); code != 404 {
|
||||
t.Fatal("a repeated delete must be 404")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", first.Chapter.ID), other.Token, nil); code != 404 {
|
||||
t.Fatal("another account must not delete this chapter")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", first.Chapter.ID), "", nil); code != 401 {
|
||||
t.Fatal("deleting requires a session")
|
||||
}
|
||||
if code, detail := bookDetail(t, r, learner.Token, first.Book.ID); code != 200 || len(detail.Chapters) != 2 {
|
||||
t.Fatalf("a rejected delete changed the book: %+v", detail.Chapters)
|
||||
}
|
||||
}
|
||||
|
||||
// pasteInto appends one chapter to an owned book.
|
||||
func pasteInto(t *testing.T, r *gin.Engine, token string, bookID int64, requestID, title, text string) pasteResponse {
|
||||
t.Helper()
|
||||
code, pasted := pasteChapter(t, r, token, bookID, map[string]string{"requestId": requestID, "title": title, "text": text})
|
||||
if code != 201 {
|
||||
t.Fatalf("append %s status %d", title, code)
|
||||
}
|
||||
return pasted
|
||||
}
|
||||
|
||||
func TestMySQLDeleteBookKeepsPersonalRecords(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "delete-book-0001", "title": "Doomed Book", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
pasteInto(t, r, learner.Token, first.Book.ID, "delete-book-0002", "Second", "Second chapter body.\n")
|
||||
drainIngest(t, db)
|
||||
saved := saveWord(t, r, learner.Token, first.Chapter.ID, 0, 4, termStatusNew, nil)
|
||||
survivorBook := pasteInto(t, r, learner.Token, first.Book.ID, "delete-book-0003", "Third", "Third body.\n")
|
||||
otherBook := func() int64 {
|
||||
code, kept := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "delete-book-keep", "title": "Kept Book", "text": "Kept body.\n", "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("kept book %d", code)
|
||||
}
|
||||
return kept.Book.ID
|
||||
}()
|
||||
drainIngest(t, db)
|
||||
|
||||
code, msg, data := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/books/%d", first.Book.ID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("delete book status %d (%s)", code, msg)
|
||||
}
|
||||
var deleted struct {
|
||||
Deleted DeletionResult
|
||||
}
|
||||
json.Unmarshal(data, &deleted)
|
||||
if deleted.Deleted.BookID != first.Book.ID || deleted.Deleted.Chapters != 3 {
|
||||
t.Fatalf("delete result: %+v", deleted.Deleted)
|
||||
}
|
||||
var books, chapters, jobs int64
|
||||
db.Model(&Book{}).Where("id = ?", first.Book.ID).Count(&books)
|
||||
db.Model(&Chapter{}).Where("book_id = ?", first.Book.ID).Count(&chapters)
|
||||
db.Model(&IngestJob{}).Where("book_id = ?", first.Book.ID).Count(&jobs)
|
||||
if books != 0 || chapters != 0 || jobs != 0 {
|
||||
t.Fatalf("the deleted book left rows: books=%d chapters=%d jobs=%d", books, chapters, jobs)
|
||||
}
|
||||
// The other book of the same account is untouched.
|
||||
if code, detail := bookDetail(t, r, learner.Token, otherBook); code != 200 || len(detail.Chapters) != 1 {
|
||||
t.Fatalf("the other book changed: %d %+v", code, detail.Chapters)
|
||||
}
|
||||
// Personal records and their schedule survive.
|
||||
var terms, reviews int64
|
||||
db.Model(&Term{}).Where("owner_id = ?", learner.ID).Count(&terms)
|
||||
db.Model(&TermReview{}).Where("term_id = ?", saved.Term.ID).Count(&reviews)
|
||||
if terms != 1 || reviews != 1 {
|
||||
t.Fatalf("deleting a book changed personal records: terms=%d reviews=%d", terms, reviews)
|
||||
}
|
||||
code, queue := reviewQueue(t, r, learner.Token)
|
||||
if code != 200 || len(queue.Items) != 1 || queue.Items[0].ID != saved.Term.ID {
|
||||
t.Fatalf("the saved word left the review queue: %d %+v", code, queue.Items)
|
||||
}
|
||||
// The deleted chapter and book are no longer readable, and repeats are 404.
|
||||
if code, _ := readChapter(t, r, learner.Token, survivorBook.Chapter.ID); code != 404 {
|
||||
t.Fatal("a chapter of the deleted book must be gone")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/books/%d", first.Book.ID), learner.Token, nil); code != 404 {
|
||||
t.Fatal("a repeated delete must be 404")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/books/%d", otherBook), other.Token, nil); code != 404 {
|
||||
t.Fatal("another account must not delete this book")
|
||||
}
|
||||
_, list := bookList(t, r, other.Token)
|
||||
if len(list.Items) != 0 {
|
||||
t.Fatalf("the other account must not see this library: %+v", list.Items)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLDeleteDuringProcessing proves a deleted chapter cannot come back through a run
|
||||
// that was already in flight.
|
||||
func TestMySQLDeleteDuringProcessing(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "delete-processing-01", "title": "In Flight", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
clock := time.Now().UTC().Truncate(time.Millisecond)
|
||||
job, claimed, err := ClaimNextIngestJob(db, clock)
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("claim: %v", err)
|
||||
}
|
||||
if code, msg, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), learner.Token, nil); code != 200 {
|
||||
t.Fatalf("deleting a processing chapter: %d (%s)", code, msg)
|
||||
}
|
||||
// The in-flight run finds nothing to publish and reports no error.
|
||||
if err := FinishIngestJob(t.Context(), db, job, clock.Add(time.Second)); err != nil {
|
||||
t.Fatalf("finishing a run for a deleted chapter: %v", err)
|
||||
}
|
||||
var chapters, jobs int64
|
||||
db.Model(&Chapter{}).Where("id = ?", pasted.Chapter.ID).Count(&chapters)
|
||||
db.Model(&IngestJob{}).Where("id = ?", job.ID).Count(&jobs)
|
||||
if chapters != 0 || jobs != 0 {
|
||||
t.Fatalf("a deleted chapter came back: chapters=%d jobs=%d", chapters, jobs)
|
||||
}
|
||||
// The book stays, with no chapters and a clear empty state for the client.
|
||||
code, detail := bookDetail(t, r, learner.Token, pasted.Book.ID)
|
||||
if code != 200 || len(detail.Chapters) != 0 {
|
||||
t.Fatalf("book after deleting its only chapter: %d %+v", code, detail.Chapters)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLConcurrentChapterDelete checks two deletions of one chapter in one book.
|
||||
func TestMySQLConcurrentChapterDelete(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "race-delete-0001", "title": "Race", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
pasteInto(t, r, learner.Token, first.Book.ID, "race-delete-0002", "Second", "Second body.\n")
|
||||
drainIngest(t, db)
|
||||
|
||||
var wg sync.WaitGroup
|
||||
codes := make(chan int, 2)
|
||||
for i := 0; i < 2; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
c, _, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", first.Chapter.ID), learner.Token, nil)
|
||||
codes <- c
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(codes)
|
||||
ok, missing := 0, 0
|
||||
for c := range codes {
|
||||
switch c {
|
||||
case 200:
|
||||
ok++
|
||||
case 404:
|
||||
missing++
|
||||
}
|
||||
}
|
||||
if ok != 1 || missing != 1 {
|
||||
t.Fatalf("concurrent deletes: ok=%d missing=%d", ok, missing)
|
||||
}
|
||||
// The single surviving chapter keeps ordinal 1.
|
||||
var remaining []Chapter
|
||||
db.Where("book_id = ?", first.Book.ID).Order("ordinal ASC").Find(&remaining)
|
||||
if len(remaining) != 1 || remaining[0].Ordinal != 1 {
|
||||
t.Fatalf("ordinals after concurrent delete: %+v", remaining)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLRecoverySkipsSupersededJobs proves the recovery sweep leaves a newer version alone.
|
||||
func TestMySQLRecoverySkipsSupersededJobs(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "recovery-fix-0001", "title": "Recovery", "text": editFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
clock := time.Now().UTC().Truncate(time.Millisecond)
|
||||
job, claimed, err := ClaimNextIngestJob(db, clock)
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("claim: %v", err)
|
||||
}
|
||||
newText := "A replacement body.\n"
|
||||
if code, msg, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"text": newText}); code != 200 {
|
||||
t.Fatalf("edit status %d (%s)", code, msg)
|
||||
}
|
||||
// The old job looks stale to the sweep; it must be abandoned, not requeued.
|
||||
requeued, err := RequeueStaleIngestJobs(db, clock.Add(ingestStaleAfter+time.Second))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if requeued != 0 {
|
||||
t.Fatalf("the sweep requeued %d superseded job(s)", requeued)
|
||||
}
|
||||
var stale IngestJob
|
||||
if err := db.First(&stale, job.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if stale.Status != statusFailed || stale.ErrorReason != reasonSuperseded {
|
||||
t.Fatalf("the sweep left the old job %+v", stale)
|
||||
}
|
||||
if row := chapterRow(t, db, pasted.Chapter.ID); row.Status != statusPending || row.OriginalText != newText {
|
||||
t.Fatalf("the sweep changed the new version: %+v", row)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
if row := chapterRow(t, db, pasted.Chapter.ID); row.Status != statusReady || row.OriginalText != newText {
|
||||
t.Fatalf("the new version did not publish: %+v", row)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,263 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// The ingestion worker is deliberately small: the fixed paste rule stores one chapter per
|
||||
// submit, so "processing" only validates the persisted content and publishes the chapter.
|
||||
// Claiming and finishing are separate transactions on purpose. A durable claim means a
|
||||
// chapter observed as processing stays recoverable, whether the process stops or only the
|
||||
// finishing transaction fails.
|
||||
|
||||
// Recovery is one mechanism used from two places: the startup pass treats every processing row
|
||||
// as abandoned, while the running worker sweeps rows that have been processing longer than any
|
||||
// legitimate claim-to-finish window. A job that keeps failing ends in a readable failure
|
||||
// instead of looping forever, and a retried job never creates a second chapter.
|
||||
const (
|
||||
maxIngestAttempts = 5
|
||||
// ingestStaleAfter must stay longer than the longest legitimate claim-to-finish window,
|
||||
// otherwise a healthy job could be processed twice. Reprocessing is harmless for content
|
||||
// because a job never creates a chapter, only publishes the one it was created with.
|
||||
ingestStaleAfter = 15 * time.Second
|
||||
)
|
||||
|
||||
// RecoverIngestJobs requeues jobs and chapters left in processing by an unclean stop. It runs
|
||||
// once before the worker starts, for a single-instance deployment.
|
||||
func RecoverIngestJobs(db *gorm.DB, now time.Time) (int64, error) {
|
||||
return requeueStaleIngestJobs(db, now, 0)
|
||||
}
|
||||
|
||||
// RequeueStaleIngestJobs recovers jobs whose finishing transaction did not complete, so a
|
||||
// running service does not depend on a restart to make progress again.
|
||||
func RequeueStaleIngestJobs(db *gorm.DB, now time.Time) (int64, error) {
|
||||
return requeueStaleIngestJobs(db, now, ingestStaleAfter)
|
||||
}
|
||||
|
||||
func requeueStaleIngestJobs(db *gorm.DB, now time.Time, staleAfter time.Duration) (int64, error) {
|
||||
ts := stamp(now)
|
||||
cutoff := stamp(now.Add(-staleAfter))
|
||||
var requeued int64
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := abandonSupersededJobs(tx, ts); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := exhaustIngestJobs(tx, ts); err != nil {
|
||||
return err
|
||||
}
|
||||
stale := []int64{}
|
||||
// Only jobs that still describe the chapter's current version may be requeued: an
|
||||
// interrupted run of an older version must not pull the newer text back into processing.
|
||||
if err := tx.Table("lexgo_ingest_jobs AS j").
|
||||
Joins("JOIN lexgo_chapters AS c ON c.id = j.chapter_id AND c.content_sha256 = j.content_sha256").
|
||||
Where("j.status = ? AND j.attempts < ? AND j.updated_at <= ?", statusProcessing, maxIngestAttempts, cutoff).
|
||||
Pluck("j.id", &stale).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(stale) == 0 {
|
||||
return nil
|
||||
}
|
||||
if err := setIngestChapterStatus(tx, stale, statusPending, "", ts); err != nil {
|
||||
return err
|
||||
}
|
||||
result := tx.Model(&IngestJob{}).Where("id IN ?", stale).
|
||||
Updates(map[string]any{"status": statusPending, "updated_at": ts})
|
||||
if result.Error != nil {
|
||||
return result.Error
|
||||
}
|
||||
requeued = result.RowsAffected
|
||||
return nil
|
||||
})
|
||||
return requeued, err
|
||||
}
|
||||
|
||||
// exhaustIngestJobs fails jobs that used up the attempt budget, so nothing can stay queued or
|
||||
// claimed forever. The recorded reason is readable and a manual retry is still allowed.
|
||||
func exhaustIngestJobs(tx *gorm.DB, ts time.Time) error {
|
||||
var exhausted []int64
|
||||
if err := tx.Model(&IngestJob{}).
|
||||
Where("status IN ? AND attempts >= ?", []string{statusPending, statusProcessing}, maxIngestAttempts).
|
||||
Pluck("id", &exhausted).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(exhausted) == 0 {
|
||||
return nil
|
||||
}
|
||||
if err := setIngestChapterStatus(tx, exhausted, statusFailed, reasonAttemptsExhausted, ts); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Model(&IngestJob{}).Where("id IN ?", exhausted).Updates(map[string]any{
|
||||
"status": statusFailed, "error_reason": reasonAttemptsExhausted, "updated_at": ts, "finished_at": ts}).Error
|
||||
}
|
||||
|
||||
// setIngestChapterStatus mirrors a job outcome onto the chapters it owns.
|
||||
func setIngestChapterStatus(tx *gorm.DB, jobIDs []int64, status, reason string, ts time.Time) error {
|
||||
var chapterIDs []int64
|
||||
if err := tx.Model(&IngestJob{}).Where("id IN ?", jobIDs).Pluck("chapter_id", &chapterIDs).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if len(chapterIDs) == 0 {
|
||||
return nil
|
||||
}
|
||||
return tx.Model(&Chapter{}).Where("id IN ?", chapterIDs).
|
||||
Updates(map[string]any{"status": status, "error_reason": reason, "updated_at": ts}).Error
|
||||
}
|
||||
|
||||
// ClaimNextIngestJob takes the oldest pending job with attempts left and marks it processing in
|
||||
// its own transaction. The guarded update means only one worker can own a job.
|
||||
func ClaimNextIngestJob(db *gorm.DB, now time.Time) (IngestJob, bool, error) {
|
||||
ts := stamp(now)
|
||||
var job IngestJob
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := abandonSupersededJobs(tx, ts); err != nil {
|
||||
return err
|
||||
}
|
||||
// A job may only process the version it was created for, so the chapter join is part
|
||||
// of the claim and an edited chapter is never dragged back to processing.
|
||||
if err := tx.Table("lexgo_ingest_jobs AS j").
|
||||
Select("j.id, j.owner_id, j.book_id, j.chapter_id, j.request_key, j.content_sha256, j.status, j.attempts, j.error_reason, j.created_at, j.updated_at, j.finished_at").
|
||||
Joins("JOIN lexgo_chapters AS c ON c.id = j.chapter_id AND c.content_sha256 = j.content_sha256").
|
||||
Where("j.status = ? AND j.attempts < ?", statusPending, maxIngestAttempts).
|
||||
Order("j.id ASC").Limit(1).Find(&job).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if job.ID == 0 {
|
||||
return errNoIngestJob
|
||||
}
|
||||
claim := tx.Model(&IngestJob{}).Where("id = ? AND status = ?", job.ID, statusPending).
|
||||
Updates(map[string]any{"status": statusProcessing, "attempts": gorm.Expr("attempts + 1"), "updated_at": ts})
|
||||
if claim.Error != nil {
|
||||
return claim.Error
|
||||
}
|
||||
if claim.RowsAffected != 1 {
|
||||
return errJobTaken
|
||||
}
|
||||
if err := tx.Model(&Chapter{}).Where("id = ? AND owner_id = ? AND content_sha256 = ?", job.ChapterID, job.OwnerID, job.ContentSHA256).
|
||||
Updates(map[string]any{"status": statusProcessing, "updated_at": ts}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
job.Status = statusProcessing
|
||||
job.Attempts++
|
||||
job.UpdatedAt = ts
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) || errors.Is(err, errJobTaken) || errors.Is(err, errNoIngestJob) {
|
||||
return IngestJob{}, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return IngestJob{}, false, err
|
||||
}
|
||||
return job, true, nil
|
||||
}
|
||||
|
||||
var errJobTaken = errors.New("ingestion job already claimed")
|
||||
|
||||
// errNoIngestJob reports an empty queue, which is not a failure.
|
||||
var errNoIngestJob = errors.New("no ingestion job to claim")
|
||||
|
||||
// FinishIngestJob validates the persisted chapter and publishes it, or records a fixed
|
||||
// failure reason. The check runs again here because a worker must not trust that content
|
||||
// reached the table through the paste API.
|
||||
func FinishIngestJob(ctx context.Context, db *gorm.DB, job IngestJob, now time.Time) error {
|
||||
ts := stamp(now)
|
||||
return db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
var chapter Chapter
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", job.ChapterID, job.OwnerID).First(&chapter).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
// The chapter was deleted while this run was in flight; the cascade removed its
|
||||
// jobs too, so there is nothing left to publish.
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if chapter.ContentSHA256 != job.ContentSHA256 {
|
||||
// The chapter moved to a newer version: publish nothing and leave its state, which
|
||||
// belongs to the newer job, untouched.
|
||||
return tx.Model(&IngestJob{}).Where("id = ?", job.ID).Updates(map[string]any{
|
||||
"status": statusFailed, "error_reason": reasonSuperseded, "updated_at": ts, "finished_at": ts}).Error
|
||||
}
|
||||
var book Book
|
||||
if err := tx.Where("id = ? AND owner_id = ?", job.BookID, job.OwnerID).First(&book).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
if reason := unprocessableReason(book, chapter, job); reason != "" {
|
||||
if err := tx.Model(&IngestJob{}).Where("id = ?", job.ID).Updates(map[string]any{
|
||||
"status": statusFailed, "error_reason": reason, "updated_at": ts, "finished_at": ts}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Model(&Chapter{}).Where("id = ?", chapter.ID).Updates(map[string]any{
|
||||
"status": statusFailed, "error_reason": reason, "updated_at": ts}).Error
|
||||
}
|
||||
if err := tx.Model(&Chapter{}).Where("id = ?", chapter.ID).Updates(map[string]any{
|
||||
"status": statusReady, "char_count": utf8.RuneCountInString(chapter.OriginalText), "updated_at": ts}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Model(&IngestJob{}).Where("id = ?", job.ID).Updates(map[string]any{
|
||||
"status": statusReady, "error_reason": "", "updated_at": ts, "finished_at": ts}).Error
|
||||
})
|
||||
}
|
||||
|
||||
func unprocessableReason(book Book, chapter Chapter, job IngestJob) string {
|
||||
if book.Language != "en" {
|
||||
return reasonUnsupportedLanguage
|
||||
}
|
||||
if strings.TrimFunc(chapter.OriginalText, unicode.IsSpace) == "" {
|
||||
return reasonEmptyText
|
||||
}
|
||||
if utf8.RuneCountInString(chapter.OriginalText) > maxChapterRunes {
|
||||
return reasonTooLong
|
||||
}
|
||||
// The stored text and its stored version must agree. The job-versus-chapter version check
|
||||
// ran before this function, so a direct write that changed the text without updating its
|
||||
// version is the remaining case, and it is a different paste rather than this version.
|
||||
if contentSHA(chapter.OriginalText) != chapter.ContentSHA256 {
|
||||
return reasonContentChanged
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// abandonSupersededJobs fails jobs whose version is no longer the chapter's version. They
|
||||
// must never publish or fail the chapter, because another job owns its current state. It is
|
||||
// one multi-table statement: GORM's Updates does not carry a Joins clause into an UPDATE.
|
||||
func abandonSupersededJobs(tx *gorm.DB, ts time.Time) error {
|
||||
return tx.Exec(`UPDATE lexgo_ingest_jobs j JOIN lexgo_chapters c ON c.id = j.chapter_id
|
||||
SET j.status = ?, j.error_reason = ?, j.updated_at = ?, j.finished_at = ?
|
||||
WHERE j.status IN (?, ?) AND j.content_sha256 <> c.content_sha256`,
|
||||
statusFailed, reasonSuperseded, ts, ts, statusPending, statusProcessing).Error
|
||||
}
|
||||
|
||||
// ProcessIngestJobs drains up to limit pending jobs. Claiming and finishing each use their
|
||||
// own transaction, so an interrupted run simply leaves a job for recovery.
|
||||
func ProcessIngestJobs(ctx context.Context, db *gorm.DB, now func() time.Time, limit int) (int, error) {
|
||||
processed := 0
|
||||
for i := 0; i < limit; i++ {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return processed, err
|
||||
}
|
||||
job, claimed, err := ClaimNextIngestJob(db.WithContext(ctx), now())
|
||||
if err != nil {
|
||||
return processed, err
|
||||
}
|
||||
if !claimed {
|
||||
return processed, nil
|
||||
}
|
||||
if err = FinishIngestJob(ctx, db, job, now()); err != nil {
|
||||
return processed, err
|
||||
}
|
||||
processed++
|
||||
}
|
||||
return processed, nil
|
||||
}
|
||||
@@ -0,0 +1,675 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// Chapter and job share one status vocabulary so the client renders either without mapping.
|
||||
const (
|
||||
statusPending = "pending"
|
||||
statusProcessing = "processing"
|
||||
statusReady = "ready"
|
||||
statusFailed = "failed"
|
||||
)
|
||||
|
||||
// Fixed worker-side failure reasons. Only these codes are stored; the readable text is
|
||||
// produced at the API boundary so no user content can leak into an error field.
|
||||
const (
|
||||
reasonUnsupportedLanguage = "unsupported_language"
|
||||
reasonTooLong = "too_long"
|
||||
reasonEmptyText = "empty_text"
|
||||
reasonContentChanged = "content_changed"
|
||||
// reasonSuperseded marks a job whose chapter already moved to a newer content version.
|
||||
reasonSuperseded = "superseded"
|
||||
reasonAttemptsExhausted = "attempts_exhausted"
|
||||
)
|
||||
|
||||
const (
|
||||
maxChapterRunes = 100000
|
||||
maxTitleRunes = 120
|
||||
maxBooksPerList = 200
|
||||
// A 100000 code point paste stays well inside this even with JSON escaping; the limit
|
||||
// exists only so an oversized body is rejected before it is decoded.
|
||||
maxPasteBodyBytes = 4 << 20
|
||||
maxJSONBodyBytes = 16 * 1024
|
||||
)
|
||||
|
||||
var requestIDPattern = regexp.MustCompile(`^[A-Za-z0-9_-]{8,64}$`)
|
||||
|
||||
func reasonMessage(reason string) string {
|
||||
switch reason {
|
||||
case reasonUnsupportedLanguage:
|
||||
return "当前版本只支持英语内容"
|
||||
case reasonTooLong:
|
||||
return "内容超过单章上限（100000 个字符）"
|
||||
case reasonEmptyText:
|
||||
return "章节内容为空"
|
||||
case reasonSuperseded:
|
||||
return "章节内容已更新为新版本，本次处理已作废"
|
||||
case reasonContentChanged:
|
||||
return "内容在处理前发生变化，请重新提交"
|
||||
case reasonAttemptsExhausted:
|
||||
return "处理多次失败，请重试或重新提交"
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
type Book struct {
|
||||
ID int64 `gorm:"primaryKey"`
|
||||
OwnerID int
|
||||
Title string
|
||||
Language string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (Book) TableName() string { return "lexgo_books" }
|
||||
|
||||
type Chapter struct {
|
||||
ID int64 `gorm:"primaryKey"`
|
||||
BookID int64
|
||||
OwnerID int
|
||||
Ordinal int
|
||||
Title string
|
||||
OriginalText string
|
||||
CharCount int
|
||||
ContentSHA256 string
|
||||
Status string
|
||||
ErrorReason string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (Chapter) TableName() string { return "lexgo_chapters" }
|
||||
|
||||
type IngestJob struct {
|
||||
ID int64 `gorm:"primaryKey"`
|
||||
OwnerID int
|
||||
BookID int64
|
||||
ChapterID int64
|
||||
RequestKey string
|
||||
ContentSHA256 string
|
||||
Status string
|
||||
Attempts int
|
||||
ErrorReason string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
FinishedAt *time.Time
|
||||
}
|
||||
|
||||
func (IngestJob) TableName() string { return "lexgo_ingest_jobs" }
|
||||
|
||||
type BookSummary struct {
|
||||
ID int64 `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Language string `json:"language"`
|
||||
ChapterCount int `json:"chapterCount"`
|
||||
PendingCount int `json:"pendingCount"`
|
||||
ProcessingCount int `json:"processingCount"`
|
||||
ReadyCount int `json:"readyCount"`
|
||||
FailedCount int `json:"failedCount"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
type BookRef struct {
|
||||
ID int64 `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Language string `json:"language"`
|
||||
}
|
||||
|
||||
func bookRef(book Book) BookRef { return BookRef{book.ID, book.Title, book.Language} }
|
||||
|
||||
type ChapterSummary struct {
|
||||
ID int64 `json:"id"`
|
||||
BookID int64 `json:"bookId"`
|
||||
Ordinal int `json:"ordinal"`
|
||||
Title string `json:"title"`
|
||||
Status string `json:"status"`
|
||||
CharCount int `json:"charCount"`
|
||||
// JobID lets a client retry a failed chapter without keeping the submit response.
|
||||
JobID *int64 `json:"jobId"`
|
||||
ErrorReason string `json:"errorReason"`
|
||||
ErrorMessage string `json:"errorMessage"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
type ChapterView struct {
|
||||
ChapterSummary
|
||||
ContentSHA256 string `json:"contentSha256"`
|
||||
// OriginalText is returned only for a ready chapter, so unprocessed content cannot be
|
||||
// rendered as readable text by the client.
|
||||
OriginalText string `json:"originalText,omitempty"`
|
||||
}
|
||||
|
||||
func chapterSummary(c Chapter) ChapterSummary { return chapterSummaryWithJob(c, nil) }
|
||||
|
||||
func chapterSummaryWithJob(c Chapter, jobID *int64) ChapterSummary {
|
||||
return ChapterSummary{ID: c.ID, BookID: c.BookID, Ordinal: c.Ordinal, Title: c.Title,
|
||||
Status: c.Status, CharCount: c.CharCount, JobID: jobID, ErrorReason: c.ErrorReason,
|
||||
ErrorMessage: reasonMessage(c.ErrorReason), CreatedAt: c.CreatedAt, UpdatedAt: c.UpdatedAt}
|
||||
}
|
||||
|
||||
func chapterView(c Chapter, jobID *int64) ChapterView {
|
||||
view := ChapterView{ChapterSummary: chapterSummaryWithJob(c, jobID), ContentSHA256: c.ContentSHA256}
|
||||
if c.Status == statusReady {
|
||||
view.OriginalText = c.OriginalText
|
||||
}
|
||||
return view
|
||||
}
|
||||
|
||||
type JobView struct {
|
||||
ID int64 `json:"id"`
|
||||
BookID int64 `json:"bookId"`
|
||||
ChapterID int64 `json:"chapterId"`
|
||||
Status string `json:"status"`
|
||||
Attempts int `json:"attempts"`
|
||||
ErrorReason string `json:"errorReason"`
|
||||
ErrorMessage string `json:"errorMessage"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func jobView(j IngestJob) JobView {
|
||||
return JobView{j.ID, j.BookID, j.ChapterID, j.Status, j.Attempts, j.ErrorReason, reasonMessage(j.ErrorReason), j.CreatedAt, j.UpdatedAt}
|
||||
}
|
||||
|
||||
type Navigation struct {
|
||||
PreviousChapterID *int64 `json:"previousChapterId"`
|
||||
NextChapterID *int64 `json:"nextChapterId"`
|
||||
}
|
||||
|
||||
// ReaderResponse carries everything the reader needs for one chapter of the caller's own book.
|
||||
type ReaderResponse struct {
|
||||
Book BookRef `json:"book"`
|
||||
Chapter ChapterView `json:"chapter"`
|
||||
Navigation Navigation `json:"navigation"`
|
||||
}
|
||||
|
||||
type PasteResult struct {
|
||||
Book *BookRef `json:"book,omitempty"`
|
||||
Chapter ChapterSummary `json:"chapter"`
|
||||
Job JobView `json:"job"`
|
||||
Duplicate bool `json:"duplicate"`
|
||||
}
|
||||
|
||||
type PasteBookInput struct {
|
||||
RequestID string `json:"requestId"`
|
||||
Title string `json:"title"`
|
||||
Text string `json:"text"`
|
||||
Language string `json:"language"`
|
||||
}
|
||||
|
||||
type PasteChapterInput struct {
|
||||
RequestID string `json:"requestId"`
|
||||
Title string `json:"title"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
func stamp(now time.Time) time.Time { return now.UTC().Truncate(time.Millisecond) }
|
||||
|
||||
func contentSHA(text string) string {
|
||||
v := sha256.Sum256([]byte(text))
|
||||
return hex.EncodeToString(v[:])
|
||||
}
|
||||
|
||||
func requestKey(requestID string) (string, error) {
|
||||
if !requestIDPattern.MatchString(requestID) {
|
||||
return "", failure(400, "请求编号须为 8～64 位字母、数字、下划线或连字符")
|
||||
}
|
||||
return contentSHA(requestID), nil
|
||||
}
|
||||
|
||||
// validatePaste applies the fixed paste rules: a title within the cap, at least one
|
||||
// non-space character, and at most maxChapterRunes code points. The text itself is stored
|
||||
// exactly as received, so whitespace, punctuation and line breaks survive unchanged.
|
||||
func validatePaste(title, text string) (string, string, int, error) {
|
||||
name := strings.TrimSpace(title)
|
||||
if name == "" {
|
||||
return "", "", 0, failure(400, "请填写标题")
|
||||
}
|
||||
if utf8.RuneCountInString(name) > maxTitleRunes {
|
||||
return "", "", 0, failure(400, "标题最多 120 个字符")
|
||||
}
|
||||
if strings.TrimFunc(text, unicode.IsSpace) == "" {
|
||||
return "", "", 0, failure(400, "请输入正文内容")
|
||||
}
|
||||
count := utf8.RuneCountInString(text)
|
||||
if count > maxChapterRunes {
|
||||
return "", "", 0, failure(400, "正文超过单章上限（100000 个字符）")
|
||||
}
|
||||
return name, contentSHA(text), count, nil
|
||||
}
|
||||
|
||||
// PasteBook creates one book with its first chapter and the ingestion job. The paste and the
|
||||
// job are written in one transaction, so a rejected submit leaves no book behind.
|
||||
func PasteBook(db *gorm.DB, owner int, now time.Time, input PasteBookInput) (PasteResult, error) {
|
||||
title, sha, count, err := validatePaste(input.Title, input.Text)
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
if input.Language != "" && input.Language != "en" {
|
||||
return PasteResult{}, failure(400, "当前版本只支持英语内容")
|
||||
}
|
||||
key, err := requestKey(input.RequestID)
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
ts := stamp(now)
|
||||
var result PasteResult
|
||||
err = db.Transaction(func(tx *gorm.DB) error {
|
||||
existing, found, err := jobByRequest(tx, owner, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if found {
|
||||
reused, err := reusePaste(tx, existing, title, sha)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result = reused
|
||||
return nil
|
||||
}
|
||||
book := Book{OwnerID: owner, Title: title, Language: "en", CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&book).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: owner, Ordinal: 1, Title: title,
|
||||
OriginalText: input.Text, CharCount: count, ContentSHA256: sha,
|
||||
Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&chapter).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
job := IngestJob{OwnerID: owner, BookID: book.ID, ChapterID: chapter.ID, RequestKey: key,
|
||||
ContentSHA256: sha, Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&job).Error; err != nil {
|
||||
return pasteInsertError(err)
|
||||
}
|
||||
ref := bookRef(book)
|
||||
result = PasteResult{Book: &ref, Chapter: chapterSummaryWithJob(chapter, &job.ID), Job: jobView(job)}
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, errRequestReuse) {
|
||||
return reusePasteByRequest(db, owner, key, title, sha)
|
||||
}
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// PasteChapter appends one chapter to a book the caller already owns. The book row is locked
|
||||
// so two appends cannot claim the same ordinal.
|
||||
func PasteChapter(db *gorm.DB, owner int, bookID int64, now time.Time, input PasteChapterInput) (PasteResult, error) {
|
||||
title, sha, count, err := validatePaste(input.Title, input.Text)
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
key, err := requestKey(input.RequestID)
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
ts := stamp(now)
|
||||
var result PasteResult
|
||||
err = db.Transaction(func(tx *gorm.DB) error {
|
||||
var book Book
|
||||
if err = lockOwnedBook(tx, owner, bookID, &book); err != nil {
|
||||
return err
|
||||
}
|
||||
existing, found, err := jobByRequest(tx, owner, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if found {
|
||||
if existing.BookID != book.ID {
|
||||
return failure(409, "该请求编号已用于其他内容")
|
||||
}
|
||||
reused, err := reusePaste(tx, existing, title, sha)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result = reused
|
||||
return nil
|
||||
}
|
||||
var last int
|
||||
row := tx.Model(&Chapter{}).Where("book_id = ?", book.ID).Select("COALESCE(MAX(ordinal),0)").Row()
|
||||
if err = row.Scan(&last); err != nil {
|
||||
return err
|
||||
}
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: owner, Ordinal: last + 1, Title: title,
|
||||
OriginalText: input.Text, CharCount: count, ContentSHA256: sha,
|
||||
Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&chapter).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
job := IngestJob{OwnerID: owner, BookID: book.ID, ChapterID: chapter.ID, RequestKey: key,
|
||||
ContentSHA256: sha, Status: statusPending, CreatedAt: ts, UpdatedAt: ts}
|
||||
if err = tx.Create(&job).Error; err != nil {
|
||||
return pasteInsertError(err)
|
||||
}
|
||||
if err = tx.Model(&Book{}).Where("id = ?", book.ID).Update("updated_at", ts).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
result = PasteResult{Chapter: chapterSummaryWithJob(chapter, &job.ID), Job: jobView(job)}
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, errRequestReuse) {
|
||||
return reusePasteByRequest(db, owner, key, title, sha)
|
||||
}
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
var errRequestReuse = errors.New("ingestion request already accepted")
|
||||
|
||||
// pasteInsertError turns a unique-key conflict on the job insert into a request reuse. The
|
||||
// transaction must be abandoned: a concurrent submit that already committed is invisible to
|
||||
// this transaction's snapshot, and its rows are re-read with locking reads below.
|
||||
func pasteInsertError(err error) error {
|
||||
var dup *driver.MySQLError
|
||||
if errors.As(err, &dup) && dup.Number == 1062 {
|
||||
return errRequestReuse
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func reusePasteByRequest(db *gorm.DB, owner int, key, title, sha string) (PasteResult, error) {
|
||||
var result PasteResult
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
job, found, err := jobByRequestLatest(tx, owner, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !found {
|
||||
// The conflict came from another unique key, not from a repeated request id.
|
||||
return failure(409, "提交冲突，请重试")
|
||||
}
|
||||
result, err = reusePaste(tx, job, title, sha)
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func jobByRequest(tx *gorm.DB, owner int, key string) (IngestJob, bool, error) {
|
||||
return jobQuery(tx, owner, key, false)
|
||||
}
|
||||
|
||||
// jobByRequestLatest uses a locking read, which sees the latest committed row instead of this
|
||||
// transaction's older snapshot. It is required after a duplicate-key conflict: only the
|
||||
// competing transaction's commit can cause that conflict, and its rows are newer than the
|
||||
// snapshot this request already took.
|
||||
func jobByRequestLatest(tx *gorm.DB, owner int, key string) (IngestJob, bool, error) {
|
||||
return jobQuery(tx, owner, key, true)
|
||||
}
|
||||
|
||||
func jobQuery(tx *gorm.DB, owner int, key string, latest bool) (IngestJob, bool, error) {
|
||||
var job IngestJob
|
||||
query := tx
|
||||
if latest {
|
||||
query = tx.Clauses(clause.Locking{Strength: "UPDATE"})
|
||||
}
|
||||
err := query.Where("owner_id = ? AND request_key = ?", owner, key).First(&job).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return job, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return job, false, err
|
||||
}
|
||||
return job, true, nil
|
||||
}
|
||||
|
||||
// reusePaste answers a repeated submit with the first result instead of creating a second
|
||||
// chapter. A reused request id with different content or title is a conflict, not a retry.
|
||||
// Its reads are locking reads so the same answer works right after a duplicate-key conflict.
|
||||
func reusePaste(tx *gorm.DB, job IngestJob, title, sha string) (PasteResult, error) {
|
||||
var chapter Chapter
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", job.ChapterID, job.OwnerID).First(&chapter).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return PasteResult{}, failure(409, "该请求已提交过，请刷新后查看结果")
|
||||
}
|
||||
return PasteResult{}, err
|
||||
}
|
||||
if chapter.ContentSHA256 != sha || chapter.Title != title {
|
||||
return PasteResult{}, failure(409, "该请求编号已用于其他内容")
|
||||
}
|
||||
var book Book
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", job.BookID, job.OwnerID).First(&book).Error; err != nil {
|
||||
return PasteResult{}, err
|
||||
}
|
||||
ref := bookRef(book)
|
||||
result := PasteResult{Chapter: chapterSummaryWithJob(chapter, &job.ID), Job: jobView(job), Duplicate: true}
|
||||
if chapter.Ordinal == 1 {
|
||||
result.Book = &ref
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func lockOwnedBook(tx *gorm.DB, owner int, bookID int64, book *Book) error {
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", bookID, owner).First(book).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
// Another user's book and a missing book are indistinguishable to the caller.
|
||||
return failure(404, "书籍不存在")
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func ListBooks(db *gorm.DB, owner int) ([]BookSummary, error) {
|
||||
var books []Book
|
||||
if err := db.Where("owner_id = ?", owner).Order("updated_at DESC, id DESC").Limit(maxBooksPerList).Find(&books).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items := make([]BookSummary, 0, len(books))
|
||||
ids := make([]int64, 0, len(books))
|
||||
for _, b := range books {
|
||||
items = append(items, BookSummary{ID: b.ID, Title: b.Title, Language: b.Language, CreatedAt: b.CreatedAt, UpdatedAt: b.UpdatedAt})
|
||||
ids = append(ids, b.ID)
|
||||
}
|
||||
if len(ids) == 0 {
|
||||
return items, nil
|
||||
}
|
||||
type row struct {
|
||||
BookID int64
|
||||
Status string
|
||||
Total int
|
||||
}
|
||||
var rows []row
|
||||
if err := db.Model(&Chapter{}).Select("book_id, status, COUNT(*) AS total").
|
||||
Where("owner_id = ? AND book_id IN ?", owner, ids).Group("book_id, status").Scan(&rows).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
index := make(map[int64]int, len(items))
|
||||
for i, item := range items {
|
||||
index[item.ID] = i
|
||||
}
|
||||
for _, r := range rows {
|
||||
i, ok := index[r.BookID]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
items[i].ChapterCount += r.Total
|
||||
switch r.Status {
|
||||
case statusReady:
|
||||
items[i].ReadyCount += r.Total
|
||||
case statusProcessing:
|
||||
items[i].ProcessingCount += r.Total
|
||||
case statusFailed:
|
||||
items[i].FailedCount += r.Total
|
||||
default:
|
||||
items[i].PendingCount += r.Total
|
||||
}
|
||||
}
|
||||
return items, nil
|
||||
}
|
||||
|
||||
func BookDetail(db *gorm.DB, owner int, bookID int64) (BookRef, []ChapterSummary, error) {
|
||||
var book Book
|
||||
if err := db.Where("id = ? AND owner_id = ?", bookID, owner).First(&book).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return BookRef{}, nil, failure(404, "书籍不存在")
|
||||
}
|
||||
return BookRef{}, nil, err
|
||||
}
|
||||
var chapters []Chapter
|
||||
if err := db.Where("book_id = ? AND owner_id = ?", book.ID, owner).Order("ordinal ASC").Find(&chapters).Error; err != nil {
|
||||
return BookRef{}, nil, err
|
||||
}
|
||||
items := make([]ChapterSummary, 0, len(chapters))
|
||||
ids := make([]int64, 0, len(chapters))
|
||||
for _, c := range chapters {
|
||||
ids = append(ids, c.ID)
|
||||
}
|
||||
jobs, err := jobIDsByChapter(db, owner, ids)
|
||||
if err != nil {
|
||||
return BookRef{}, nil, err
|
||||
}
|
||||
for _, c := range chapters {
|
||||
var jobID *int64
|
||||
if id, ok := jobs[c.ID]; ok {
|
||||
jobID = &id
|
||||
}
|
||||
items = append(items, chapterSummaryWithJob(c, jobID))
|
||||
}
|
||||
return bookRef(book), items, nil
|
||||
}
|
||||
|
||||
// jobIDsByChapter maps chapters to their ingestion job so a client can retry a failed chapter
|
||||
// without having kept the original submit response.
|
||||
func jobIDsByChapter(db *gorm.DB, owner int, chapterIDs []int64) (map[int64]int64, error) {
|
||||
ids := make(map[int64]int64, len(chapterIDs))
|
||||
if len(chapterIDs) == 0 {
|
||||
return ids, nil
|
||||
}
|
||||
var jobs []IngestJob
|
||||
if err := db.Select("id", "chapter_id").Where("owner_id = ? AND chapter_id IN ?", owner, chapterIDs).
|
||||
Order("id ASC").Find(&jobs).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Ascending order keeps the newest id if a chapter somehow has more than one job.
|
||||
for _, j := range jobs {
|
||||
ids[j.ChapterID] = j.ID
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
// ChapterDetail resolves a chapter strictly inside the caller's own books and returns the
|
||||
// original text only once the chapter is ready.
|
||||
func ChapterDetail(db *gorm.DB, owner int, chapterID int64) (ReaderResponse, error) {
|
||||
var chapter Chapter
|
||||
if err := db.Where("id = ? AND owner_id = ?", chapterID, owner).First(&chapter).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReaderResponse{}, failure(404, "章节不存在")
|
||||
}
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
var book Book
|
||||
if err := db.Where("id = ? AND owner_id = ?", chapter.BookID, owner).First(&book).Error; err != nil {
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
navigation := Navigation{}
|
||||
var previous, next Chapter
|
||||
if err := db.Where("book_id = ? AND owner_id = ? AND ordinal < ?", book.ID, owner, chapter.Ordinal).
|
||||
Order("ordinal DESC").First(&previous).Error; err == nil {
|
||||
navigation.PreviousChapterID = &previous.ID
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
if err := db.Where("book_id = ? AND owner_id = ? AND ordinal > ?", book.ID, owner, chapter.Ordinal).
|
||||
Order("ordinal ASC").First(&next).Error; err == nil {
|
||||
navigation.NextChapterID = &next.ID
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
jobs, err := jobIDsByChapter(db, owner, []int64{chapter.ID})
|
||||
if err != nil {
|
||||
return ReaderResponse{}, err
|
||||
}
|
||||
var jobID *int64
|
||||
if id, ok := jobs[chapter.ID]; ok {
|
||||
jobID = &id
|
||||
}
|
||||
return ReaderResponse{Book: bookRef(book), Chapter: chapterView(chapter, jobID), Navigation: navigation}, nil
|
||||
}
|
||||
|
||||
func JobDetail(db *gorm.DB, owner int, jobID int64) (JobView, error) {
|
||||
var job IngestJob
|
||||
if err := db.Where("id = ? AND owner_id = ?", jobID, owner).First(&job).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return JobView{}, failure(404, "任务不存在")
|
||||
}
|
||||
return JobView{}, err
|
||||
}
|
||||
return jobView(job), nil
|
||||
}
|
||||
|
||||
// RetryIngestJob requeues a failed job on the same chapter, so a retry can never create a
|
||||
// second chapter for one paste. An explicit retry also restarts the attempt budget, because a
|
||||
// person asking again should not be blocked by the bound that stops automatic loops.
|
||||
func RetryIngestJob(db *gorm.DB, owner int, jobID int64, now time.Time) (JobView, ChapterSummary, error) {
|
||||
ts := stamp(now)
|
||||
var job IngestJob
|
||||
var chapter Chapter
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", jobID, owner).First(&job).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return failure(404, "任务不存在")
|
||||
}
|
||||
return err
|
||||
}
|
||||
if job.Status != statusFailed {
|
||||
return failure(409, "只有失败的任务可以重试")
|
||||
}
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("id = ? AND owner_id = ?", job.ChapterID, owner).First(&chapter).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
// A job of an older content version is gone for good: retrying it would process text
|
||||
// the chapter no longer holds, so the newer job owns the chapter instead.
|
||||
if chapter.ContentSHA256 != job.ContentSHA256 {
|
||||
return failure(409, "该任务对应的是旧版本，请刷新后重试当前版本")
|
||||
}
|
||||
if err := tx.Model(&IngestJob{}).Where("id = ?", job.ID).
|
||||
Updates(map[string]any{"status": statusPending, "error_reason": "", "attempts": 0,
|
||||
"updated_at": ts, "finished_at": nil}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if err := tx.Model(&Chapter{}).Where("id = ?", chapter.ID).
|
||||
Updates(map[string]any{"status": statusPending, "error_reason": "", "updated_at": ts}).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
job.Status = statusPending
|
||||
job.ErrorReason = ""
|
||||
job.Attempts = 0
|
||||
job.FinishedAt = nil
|
||||
job.UpdatedAt = ts
|
||||
chapter.Status = statusPending
|
||||
chapter.ErrorReason = ""
|
||||
chapter.UpdatedAt = ts
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return JobView{}, ChapterSummary{}, err
|
||||
}
|
||||
return jobView(job), chapterSummaryWithJob(chapter, &job.ID), nil
|
||||
}
|
||||
@@ -0,0 +1,914 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Fictional English fixture with the characters the paste contract must preserve exactly:
|
||||
// CRLF and LF, a tab, curly quotes, an em dash, an ellipsis, an emoji, a combining acute
|
||||
// accent, a trailing space run and an empty final line.
|
||||
const fixturePastedText = "Mira opened the workshop.\r\n\r\n\tThe sign read “A small step…” — café e\u0301 🙂\r\nTrailing spaces here: \n\n"
|
||||
|
||||
type libraryAccount struct {
|
||||
ID int
|
||||
Username string
|
||||
Token string
|
||||
}
|
||||
|
||||
type pasteResponse struct {
|
||||
Book *struct {
|
||||
ID int64
|
||||
Title string
|
||||
Language string
|
||||
}
|
||||
Chapter struct {
|
||||
ID int64
|
||||
BookID int64
|
||||
Ordinal int
|
||||
Title string
|
||||
Status string
|
||||
CharCount int
|
||||
ErrorReason string
|
||||
ErrorMessage string
|
||||
}
|
||||
Job struct {
|
||||
ID int64
|
||||
BookID int64
|
||||
ChapterID int64
|
||||
Status string
|
||||
Attempts int
|
||||
ErrorReason string
|
||||
ErrorMessage string
|
||||
}
|
||||
Duplicate bool
|
||||
}
|
||||
|
||||
type readerResponse struct {
|
||||
Book struct {
|
||||
ID int64
|
||||
Title string
|
||||
Language string
|
||||
}
|
||||
Chapter struct {
|
||||
ID int64
|
||||
BookID int64
|
||||
Ordinal int
|
||||
Title string
|
||||
Status string
|
||||
CharCount int
|
||||
ErrorReason string
|
||||
ErrorMessage string
|
||||
ContentSHA256 string
|
||||
OriginalText string
|
||||
JobID *int64
|
||||
}
|
||||
Navigation struct {
|
||||
PreviousChapterID *int64
|
||||
NextChapterID *int64
|
||||
}
|
||||
}
|
||||
|
||||
type bookDetailResponse struct {
|
||||
Book struct {
|
||||
ID int64
|
||||
Title string
|
||||
Language string
|
||||
}
|
||||
Chapters []struct {
|
||||
ID int64
|
||||
Ordinal int
|
||||
Title string
|
||||
Status string
|
||||
CharCount int
|
||||
JobID *int64
|
||||
ErrorMessage string
|
||||
}
|
||||
}
|
||||
|
||||
type bookListResponse struct {
|
||||
Items []struct {
|
||||
ID int64
|
||||
Title string
|
||||
ChapterCount int
|
||||
PendingCount int
|
||||
ProcessingCount int
|
||||
ReadyCount int
|
||||
FailedCount int
|
||||
}
|
||||
}
|
||||
|
||||
// callRaw keeps the API message, which is how a readable failure reason is asserted.
|
||||
func callRaw(t *testing.T, r *gin.Engine, method, path, token string, body any) (int, string, json.RawMessage) {
|
||||
t.Helper()
|
||||
b, _ := json.Marshal(body)
|
||||
q := httptest.NewRequest(method, path, bytes.NewReader(b))
|
||||
q.Header.Set("Content-Type", "application/json")
|
||||
if token != "" {
|
||||
q.Header.Set("Authorization", "Bearer "+token)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, q)
|
||||
var e struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data json.RawMessage `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &e); err != nil {
|
||||
t.Fatalf("invalid JSON for %s %s (status %d)", method, path, w.Code)
|
||||
}
|
||||
return w.Code, e.Msg, e.Data
|
||||
}
|
||||
|
||||
func libraryFixture(t *testing.T) (*gorm.DB, *gin.Engine, libraryAccount) {
|
||||
t.Helper()
|
||||
db := testDB(t)
|
||||
owner := admin.SysUser{Username: randomName("admin"), Password: fixturePassword, RoleId: 1, Status: "2"}
|
||||
if err := db.Create(&owner).Error; err != nil {
|
||||
t.Fatal("fixture admin creation failed")
|
||||
}
|
||||
r := Router(db, time.Now)
|
||||
return db, r, libraryAccount{owner.UserId, owner.Username, loginToken(t, r, owner.Username, fixturePassword)}
|
||||
}
|
||||
|
||||
func newLearner(t *testing.T, r *gin.Engine, adminToken string) libraryAccount {
|
||||
t.Helper()
|
||||
name := randomName("lib")
|
||||
code, msg, data := callRaw(t, r, "POST", "/api/v1/accounts", adminToken, map[string]string{"username": name, "password": fixturePassword})
|
||||
if code != 201 {
|
||||
t.Fatalf("create learner status %d (%s)", code, msg)
|
||||
}
|
||||
var created struct {
|
||||
ID int
|
||||
Username string
|
||||
}
|
||||
json.Unmarshal(data, &created)
|
||||
return libraryAccount{created.ID, created.Username, loginToken(t, r, name, fixturePassword)}
|
||||
}
|
||||
|
||||
func pasteBook(t *testing.T, r *gin.Engine, token string, body any) (int, pasteResponse) {
|
||||
t.Helper()
|
||||
code, msg, data := callRaw(t, r, "POST", "/api/v1/books", token, body)
|
||||
var out pasteResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("paste response: %v", err)
|
||||
}
|
||||
}
|
||||
if code >= 400 && msg == "" {
|
||||
t.Fatalf("paste failed with status %d and no message", code)
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func pasteChapter(t *testing.T, r *gin.Engine, token string, bookID int64, body any) (int, pasteResponse) {
|
||||
t.Helper()
|
||||
code, msg, data := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/books/%d/chapters", bookID), token, body)
|
||||
var out pasteResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("paste chapter response: %v", err)
|
||||
}
|
||||
}
|
||||
if code >= 400 && msg == "" {
|
||||
t.Fatalf("paste chapter failed with status %d and no message", code)
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func readChapter(t *testing.T, r *gin.Engine, token string, chapterID int64) (int, readerResponse) {
|
||||
t.Helper()
|
||||
code, _, data := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d", chapterID), token, nil)
|
||||
var out readerResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("reader response: %v", err)
|
||||
}
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func bookDetail(t *testing.T, r *gin.Engine, token string, bookID int64) (int, bookDetailResponse) {
|
||||
t.Helper()
|
||||
code, _, data := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/books/%d", bookID), token, nil)
|
||||
var out bookDetailResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("book response: %v", err)
|
||||
}
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func bookList(t *testing.T, r *gin.Engine, token string) (int, bookListResponse) {
|
||||
t.Helper()
|
||||
code, _, data := callRaw(t, r, "GET", "/api/v1/books", token, nil)
|
||||
var out bookListResponse
|
||||
if len(data) > 0 {
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
t.Fatalf("book list response: %v", err)
|
||||
}
|
||||
}
|
||||
return code, out
|
||||
}
|
||||
|
||||
func drainIngest(t *testing.T, db *gorm.DB) {
|
||||
t.Helper()
|
||||
if _, err := ProcessIngestJobs(t.Context(), db, time.Now, 50); err != nil {
|
||||
t.Fatalf("ingestion failed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func chapterRow(t *testing.T, db *gorm.DB, id int64) Chapter {
|
||||
t.Helper()
|
||||
var c Chapter
|
||||
if err := db.Where("id = ?", id).First(&c).Error; err != nil {
|
||||
t.Fatalf("chapter row: %v", err)
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func jobRow(t *testing.T, db *gorm.DB, id int64) IngestJob {
|
||||
t.Helper()
|
||||
var j IngestJob
|
||||
if err := db.Where("id = ?", id).First(&j).Error; err != nil {
|
||||
t.Fatalf("job row: %v", err)
|
||||
}
|
||||
return j
|
||||
}
|
||||
|
||||
func TestMySQLPasteToReaderFullPath(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
// Empty the queue so the claim below takes this test's own job.
|
||||
drainIngest(t, db)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-request-full-path", "title": "The Workshop", "text": fixturePastedText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d, want 201", code)
|
||||
}
|
||||
if pasted.Book == nil || pasted.Book.ID == 0 || pasted.Book.Title != "The Workshop" || pasted.Book.Language != "en" {
|
||||
t.Fatalf("unexpected book %+v", pasted.Book)
|
||||
}
|
||||
if pasted.Chapter.Ordinal != 1 || pasted.Chapter.BookID != pasted.Book.ID || pasted.Chapter.Title != "The Workshop" {
|
||||
t.Fatalf("unexpected chapter %+v", pasted.Chapter)
|
||||
}
|
||||
if pasted.Chapter.Status != statusPending || pasted.Job.Status != statusPending || pasted.Duplicate {
|
||||
t.Fatalf("paste must queue a pending job, got chapter %q job %q", pasted.Chapter.Status, pasted.Job.Status)
|
||||
}
|
||||
if want := utf8.RuneCountInString(fixturePastedText); pasted.Chapter.CharCount != want {
|
||||
t.Fatalf("charCount %d, want %d", pasted.Chapter.CharCount, want)
|
||||
}
|
||||
|
||||
// A queued chapter has no readable text yet.
|
||||
code, pending := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || pending.Chapter.Status != statusPending || pending.Chapter.OriginalText != "" {
|
||||
t.Fatalf("pending chapter must not expose text: status %d, %+v", code, pending.Chapter)
|
||||
}
|
||||
_, queued := bookList(t, r, learner.Token)
|
||||
if len(queued.Items) != 1 || queued.Items[0].PendingCount != 1 || queued.Items[0].ProcessingCount != 0 || queued.Items[0].ReadyCount != 0 {
|
||||
t.Fatalf("queued book counts %+v", queued.Items)
|
||||
}
|
||||
|
||||
// Processing is a durable state: a claim survives a crash and is observable in between.
|
||||
job, claimed, err := ClaimNextIngestJob(db, time.Now())
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("claim failed (claimed=%v): %v", claimed, err)
|
||||
}
|
||||
if job.ID != pasted.Job.ID || job.Attempts != 1 {
|
||||
t.Fatalf("claimed job %+v, want id %d with 1 attempt", job, pasted.Job.ID)
|
||||
}
|
||||
code, processing := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || processing.Chapter.Status != statusProcessing || processing.Chapter.OriginalText != "" {
|
||||
t.Fatalf("processing chapter must not expose text: status %d, %+v", code, processing.Chapter)
|
||||
}
|
||||
if err = FinishIngestJob(t.Context(), db, job, time.Now()); err != nil {
|
||||
t.Fatalf("finish failed: %v", err)
|
||||
}
|
||||
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady {
|
||||
t.Fatalf("ready chapter status %d %q", code, ready.Chapter.Status)
|
||||
}
|
||||
if ready.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("original text changed:\n got %q\nwant %q", ready.Chapter.OriginalText, fixturePastedText)
|
||||
}
|
||||
if ready.Chapter.ContentSHA256 != contentSHA(fixturePastedText) {
|
||||
t.Fatal("content hash mismatch")
|
||||
}
|
||||
if ready.Chapter.CharCount != utf8.RuneCountInString(fixturePastedText) {
|
||||
t.Fatalf("charCount %d after processing", ready.Chapter.CharCount)
|
||||
}
|
||||
if ready.Navigation.PreviousChapterID != nil || ready.Navigation.NextChapterID != nil {
|
||||
t.Fatal("single chapter must not navigate")
|
||||
}
|
||||
if ready.Book.ID != pasted.Book.ID || ready.Book.Title != "The Workshop" {
|
||||
t.Fatalf("unexpected reader book %+v", ready.Book)
|
||||
}
|
||||
|
||||
code, _, data := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/jobs/%d", pasted.Job.ID), learner.Token, nil)
|
||||
var jobView struct {
|
||||
Job struct {
|
||||
Status string
|
||||
Attempts int
|
||||
ChapterID int64
|
||||
}
|
||||
}
|
||||
json.Unmarshal(data, &jobView)
|
||||
if code != 200 || jobView.Job.Status != statusReady || jobView.Job.Attempts != 1 || jobView.Job.ChapterID != pasted.Chapter.ID {
|
||||
t.Fatalf("job view %+v (status %d)", jobView.Job, code)
|
||||
}
|
||||
|
||||
// Appending keeps the fixed rule: one paste, one more chapter.
|
||||
code, appended := pasteChapter(t, r, learner.Token, pasted.Book.ID, map[string]string{
|
||||
"requestId": "fixture-request-append", "title": "Second Chapter", "text": "A single plain paragraph.\n"})
|
||||
if code != 201 || appended.Chapter.Ordinal != 2 || appended.Book != nil {
|
||||
t.Fatalf("append status %d chapter %+v book %+v", code, appended.Chapter, appended.Book)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, second := readChapter(t, r, learner.Token, appended.Chapter.ID)
|
||||
if code != 200 || second.Chapter.Status != statusReady || second.Chapter.OriginalText != "A single plain paragraph.\n" {
|
||||
t.Fatalf("appended chapter %+v", second.Chapter)
|
||||
}
|
||||
if second.Navigation.PreviousChapterID == nil || *second.Navigation.PreviousChapterID != pasted.Chapter.ID || second.Navigation.NextChapterID != nil {
|
||||
t.Fatalf("appended navigation %+v", second.Navigation)
|
||||
}
|
||||
code, first := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || first.Navigation.NextChapterID == nil || *first.Navigation.NextChapterID != appended.Chapter.ID {
|
||||
t.Fatalf("first chapter navigation %+v", first.Navigation)
|
||||
}
|
||||
|
||||
code, detail := bookDetail(t, r, learner.Token, pasted.Book.ID)
|
||||
if code != 200 || len(detail.Chapters) != 2 || detail.Chapters[0].Status != statusReady || detail.Chapters[1].Ordinal != 2 {
|
||||
t.Fatalf("book detail %+v", detail)
|
||||
}
|
||||
// The chapter list carries the job id, which is what a client needs to retry a failure.
|
||||
if detail.Chapters[0].JobID == nil || *detail.Chapters[0].JobID != pasted.Job.ID {
|
||||
t.Fatalf("book detail chapter job id %+v, want %d", detail.Chapters[0].JobID, pasted.Job.ID)
|
||||
}
|
||||
if ready.Chapter.JobID == nil || *ready.Chapter.JobID != pasted.Job.ID {
|
||||
t.Fatalf("reader chapter job id %+v, want %d", ready.Chapter.JobID, pasted.Job.ID)
|
||||
}
|
||||
code, list := bookList(t, r, learner.Token)
|
||||
if code != 200 || len(list.Items) != 1 || list.Items[0].ChapterCount != 2 || list.Items[0].ReadyCount != 2 || list.Items[0].FailedCount != 0 {
|
||||
t.Fatalf("book list %+v", list)
|
||||
}
|
||||
if list.Items[0].PendingCount != 0 || list.Items[0].ProcessingCount != 0 {
|
||||
t.Fatalf("published book must have no queued chapter: %+v", list.Items[0])
|
||||
}
|
||||
if list.Items[0].ID != pasted.Book.ID {
|
||||
t.Fatal("book list must only contain the caller's own book")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLIngestFailureReasonsAndRetry(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
reason string
|
||||
mutate func(book *Book, chapter *Chapter, job *IngestJob)
|
||||
}{
|
||||
{"valid", "", func(*Book, *Chapter, *IngestJob) {}},
|
||||
{"unsupported_language", reasonUnsupportedLanguage, func(book *Book, _ *Chapter, _ *IngestJob) {
|
||||
book.Language = "de"
|
||||
}},
|
||||
{"empty_text", reasonEmptyText, func(_ *Book, chapter *Chapter, job *IngestJob) {
|
||||
chapter.OriginalText = " \r\n\t "
|
||||
chapter.ContentSHA256 = contentSHA(chapter.OriginalText)
|
||||
job.ContentSHA256 = chapter.ContentSHA256
|
||||
}},
|
||||
{"too_long", reasonTooLong, func(_ *Book, chapter *Chapter, job *IngestJob) {
|
||||
chapter.OriginalText = strings.Repeat("a", maxChapterRunes+1)
|
||||
chapter.ContentSHA256 = contentSHA(chapter.OriginalText)
|
||||
job.ContentSHA256 = chapter.ContentSHA256
|
||||
}},
|
||||
{"content_changed", reasonContentChanged, func(_ *Book, chapter *Chapter, _ *IngestJob) {
|
||||
chapter.OriginalText = "Mira opened"
|
||||
}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": randomName("case"), "title": "Case " + tc.name, "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// These are the states another writing path could leave behind; the worker must
|
||||
// re-validate persisted content instead of trusting the paste API.
|
||||
book := Book{ID: pasted.Book.ID, OwnerID: learner.ID, Language: "en"}
|
||||
chapter := chapterRow(t, db, pasted.Chapter.ID)
|
||||
job := jobRow(t, db, pasted.Job.ID)
|
||||
tc.mutate(&book, &chapter, &job)
|
||||
if err := db.Model(&Book{}).Where("id = ?", book.ID).Update("language", book.Language).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", chapter.ID).
|
||||
Updates(map[string]any{"original_text": chapter.OriginalText, "content_sha256": chapter.ContentSHA256}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Model(&IngestJob{}).Where("id = ?", job.ID).
|
||||
Update("content_sha256", job.ContentSHA256).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, read := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 {
|
||||
t.Fatalf("read status %d", code)
|
||||
}
|
||||
if tc.reason == "" {
|
||||
if read.Chapter.Status != statusReady || read.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("valid content must reach ready, got %+v", read.Chapter)
|
||||
}
|
||||
return
|
||||
}
|
||||
if read.Chapter.Status != statusFailed || read.Chapter.ErrorReason != tc.reason {
|
||||
t.Fatalf("chapter %q with reason %q, want failed/%s", read.Chapter.Status, read.Chapter.ErrorReason, tc.reason)
|
||||
}
|
||||
// The readable reason is produced by the API and never stores user content.
|
||||
if read.Chapter.ErrorMessage == "" || strings.ContainsAny(read.Chapter.ErrorMessage, "\r\n") {
|
||||
t.Fatalf("unreadable failure message %q", read.Chapter.ErrorMessage)
|
||||
}
|
||||
if read.Chapter.OriginalText != "" {
|
||||
t.Fatal("failed chapter must not expose text")
|
||||
}
|
||||
var detail bookDetailResponse
|
||||
_, detail = bookDetail(t, r, learner.Token, pasted.Book.ID)
|
||||
if detail.Chapters[0].Status != statusFailed || detail.Chapters[0].ErrorMessage == "" {
|
||||
t.Fatalf("book detail must show the failure: %+v", detail.Chapters[0])
|
||||
}
|
||||
// Retry through the job id the chapter list exposes, which is the client's only path.
|
||||
if detail.Chapters[0].JobID == nil || *detail.Chapters[0].JobID != pasted.Job.ID {
|
||||
t.Fatalf("failed chapter must expose its job id: %+v", detail.Chapters[0])
|
||||
}
|
||||
code, _, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", *detail.Chapters[0].JobID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("retry status %d, want 200", code)
|
||||
}
|
||||
// Retrying reuses the same chapter: no second chapter for one paste.
|
||||
var count int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatalf("retry created %d chapters, want 1", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLRetryAfterContentRestoredPublishesSameChapter(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-retry", "title": "Retry Book", "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// Simulate a chapter whose stored text was replaced before processing.
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", pasted.Chapter.ID).
|
||||
Update("original_text", "Mira opened").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, failed := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || failed.Chapter.Status != statusFailed || failed.Chapter.ErrorReason != reasonContentChanged {
|
||||
t.Fatalf("expected content_changed failure, got %+v", failed.Chapter)
|
||||
}
|
||||
code, msg, _ := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", pasted.Job.ID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("retry status %d (%s)", code, msg)
|
||||
}
|
||||
// A pending retry exposes no text and does not create a new chapter.
|
||||
code, pending := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || pending.Chapter.Status != statusPending || pending.Chapter.OriginalText != "" {
|
||||
t.Fatalf("retry must return the chapter to pending, got %+v", pending.Chapter)
|
||||
}
|
||||
var count int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatalf("retry created %d chapters, want 1", count)
|
||||
}
|
||||
// Simulate the content being restored to what was submitted, then retry to completion.
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", pasted.Chapter.ID).
|
||||
Update("original_text", fixturePastedText).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("retry did not publish the same chapter: %+v", ready.Chapter)
|
||||
}
|
||||
job := jobRow(t, db, pasted.Job.ID)
|
||||
// A manual retry restarts the attempt budget, so this processing is attempt 1 again.
|
||||
if job.Attempts != 1 {
|
||||
t.Fatalf("attempts %d after a manual retry, want 1", job.Attempts)
|
||||
}
|
||||
code, msg, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", pasted.Job.ID), learner.Token, nil)
|
||||
if code != 409 || msg == "" {
|
||||
t.Fatalf("retry of a ready job status %d (%s), want 409 with a message", code, msg)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLRepeatedPasteIsIdempotent(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
body := map[string]string{"requestId": "fixture-request-repeat", "title": "Repeat", "text": fixturePastedText}
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, body)
|
||||
if code != 201 || first.Duplicate {
|
||||
t.Fatalf("first paste status %d duplicate %v", code, first.Duplicate)
|
||||
}
|
||||
code, second := pasteBook(t, r, learner.Token, body)
|
||||
if code != 200 || !second.Duplicate {
|
||||
t.Fatalf("repeat paste status %d duplicate %v, want 200 with duplicate", code, second.Duplicate)
|
||||
}
|
||||
if second.Chapter.ID != first.Chapter.ID || second.Job.ID != first.Job.ID || second.Book.ID != first.Book.ID {
|
||||
t.Fatalf("repeat paste returned different rows: %+v vs %+v", second, first)
|
||||
}
|
||||
var books, chapters int64
|
||||
db.Model(&Book{}).Where("owner_id = ?", learner.ID).Count(&books)
|
||||
db.Model(&Chapter{}).Where("owner_id = ?", learner.ID).Count(&chapters)
|
||||
if books != 1 || chapters != 1 {
|
||||
t.Fatalf("repeat paste created books=%d chapters=%d, want 1/1", books, chapters)
|
||||
}
|
||||
|
||||
// The same request id with different content is a conflict, not a silent reuse.
|
||||
for name, changed := range map[string]map[string]string{
|
||||
"different text": {"requestId": "fixture-request-repeat", "title": "Repeat", "text": "Mira opened the workshop."},
|
||||
"different title": {"requestId": "fixture-request-repeat", "title": "Another", "text": fixturePastedText},
|
||||
} {
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", learner.Token, changed)
|
||||
if code != 409 || msg == "" {
|
||||
t.Fatalf("%s: status %d (%s), want 409", name, code, msg)
|
||||
}
|
||||
}
|
||||
|
||||
// Two concurrent submits of one request id create exactly one chapter.
|
||||
name := randomName("race")
|
||||
body = map[string]string{"requestId": name, "title": "Race", "text": "A small step; a small step"}
|
||||
var wg sync.WaitGroup
|
||||
type attempt struct {
|
||||
code int
|
||||
msg string
|
||||
}
|
||||
results := make(chan attempt, 2)
|
||||
for i := 0; i < 2; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", learner.Token, body)
|
||||
results <- attempt{code, msg}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(results)
|
||||
created, reused := 0, 0
|
||||
for result := range results {
|
||||
switch result.code {
|
||||
case 201:
|
||||
created++
|
||||
case 200:
|
||||
reused++
|
||||
default:
|
||||
t.Fatalf("concurrent paste status %d (%s)", result.code, result.msg)
|
||||
}
|
||||
}
|
||||
if created != 1 || reused != 1 {
|
||||
t.Fatalf("concurrent paste created=%d reused=%d", created, reused)
|
||||
}
|
||||
db.Model(&Chapter{}).Where("owner_id = ?", learner.ID).Count(&chapters)
|
||||
if chapters != 2 {
|
||||
t.Fatalf("concurrent paste left %d chapters, want 2", chapters)
|
||||
}
|
||||
|
||||
// A request id already used by another book cannot be replayed by appending, while a
|
||||
// fresh request id appends normally.
|
||||
var raceBook Book
|
||||
if err := db.Where("owner_id = ? AND title = ?", learner.ID, "Race").First(&raceBook).Error; err != nil {
|
||||
t.Fatalf("race book: %v", err)
|
||||
}
|
||||
code, msg, _ := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/books/%d/chapters", first.Book.ID), learner.Token,
|
||||
map[string]string{"requestId": name, "title": "Append", "text": "A second chapter."})
|
||||
if code != 409 || msg == "" {
|
||||
t.Fatalf("cross-book request id status %d (%s), want 409", code, msg)
|
||||
}
|
||||
code, appended := pasteChapter(t, r, learner.Token, first.Book.ID, map[string]string{
|
||||
"requestId": randomName("append"), "title": "Append", "text": "A second chapter."})
|
||||
if code != 201 || appended.Chapter.Ordinal != 2 {
|
||||
t.Fatalf("append status %d chapter %+v", code, appended.Chapter)
|
||||
}
|
||||
if raceBook.ID == first.Book.ID {
|
||||
t.Fatal("idempotency fixtures must use different books")
|
||||
}
|
||||
// Append owns the language of its book, so the field is not part of that contract and the
|
||||
// strict decoder rejects it. The learner client must therefore not send it (regression R1).
|
||||
code, msg, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/books/%d/chapters", first.Book.ID), learner.Token,
|
||||
map[string]string{"requestId": randomName("append"), "title": "Strict", "text": "Strict contract.", "language": "en"})
|
||||
if code != 400 || msg == "" {
|
||||
t.Fatalf("append with language status %d (%s), want 400", code, msg)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLLibraryIsolationAndOwnership(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
a := newLearner(t, r, owner.Token)
|
||||
b := newLearner(t, r, owner.Token)
|
||||
|
||||
code, pasted := pasteBook(t, r, a.Token, map[string]string{
|
||||
"requestId": "fixture-request-isolation", "title": "Private Book", "text": "Only A may read this."})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// Force a failed job so the retry path is checked for another account too.
|
||||
if err := db.Model(&IngestJob{}).Where("id = ?", pasted.Job.ID).Updates(map[string]any{"status": statusFailed, "error_reason": reasonContentChanged}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var book Book
|
||||
var chapter Chapter
|
||||
var job IngestJob
|
||||
db.Where("id = ?", pasted.Book.ID).First(&book)
|
||||
db.Where("id = ?", pasted.Chapter.ID).First(&chapter)
|
||||
db.Where("id = ?", pasted.Job.ID).First(&job)
|
||||
if book.OwnerID != a.ID || chapter.OwnerID != a.ID || job.OwnerID != a.ID {
|
||||
t.Fatal("stored rows must belong to the authenticated account")
|
||||
}
|
||||
if chapter.BookID != book.ID || job.ChapterID != chapter.ID {
|
||||
t.Fatal("job and chapter must stay linked to the book")
|
||||
}
|
||||
|
||||
for name, token := range map[string]string{"other learner": b.Token, "administrator": owner.Token} {
|
||||
for _, path := range []string{
|
||||
fmt.Sprintf("/api/v1/books/%d", book.ID),
|
||||
fmt.Sprintf("/api/v1/chapters/%d", chapter.ID),
|
||||
fmt.Sprintf("/api/v1/jobs/%d", job.ID),
|
||||
} {
|
||||
code, _, _ := callRaw(t, r, "GET", path, token, nil)
|
||||
if code != 404 {
|
||||
t.Fatalf("%s GET %s status %d, want 404", name, path, code)
|
||||
}
|
||||
}
|
||||
code, _, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/books/%d/chapters", book.ID), token,
|
||||
map[string]string{"requestId": randomName("intruder"), "title": "Intruder", "text": "Intruder text."})
|
||||
if code != 404 {
|
||||
t.Fatalf("%s append status %d, want 404", name, code)
|
||||
}
|
||||
code, _, _ = callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", job.ID), token, nil)
|
||||
if code != 404 {
|
||||
t.Fatalf("%s retry status %d, want 404", name, code)
|
||||
}
|
||||
_, list := bookList(t, r, token)
|
||||
if len(list.Items) != 0 {
|
||||
t.Fatalf("%s sees %d books", name, len(list.Items))
|
||||
}
|
||||
}
|
||||
|
||||
// The caller's identity comes from the session, never from the request body or query.
|
||||
for _, payload := range []map[string]any{
|
||||
{"requestId": randomName("owner"), "title": "Spoof", "text": "Spoofed owner.", "ownerId": b.ID},
|
||||
{"requestId": randomName("owner"), "title": "Spoof", "text": "Spoofed owner.", "userId": b.ID},
|
||||
} {
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", a.Token, payload)
|
||||
if code != 400 || msg == "" {
|
||||
t.Fatalf("client-supplied owner status %d (%s), want 400", code, msg)
|
||||
}
|
||||
}
|
||||
code, msg, _ := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/books?ownerId=%d", b.ID), a.Token, nil)
|
||||
if code != 400 || msg == "" {
|
||||
t.Fatalf("query owner override status %d (%s), want 400", code, msg)
|
||||
}
|
||||
var books int64
|
||||
db.Model(&Book{}).Where("owner_id = ?", a.ID).Count(&books)
|
||||
if books != 1 {
|
||||
t.Fatalf("rejected requests created %d books", books)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLIngestRecoveryWithoutRestart(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-in-service-recovery", "title": "In-service recovery", "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
job, claimed, err := ClaimNextIngestJob(db, time.Now())
|
||||
if err != nil || !claimed || job.ID != pasted.Job.ID {
|
||||
t.Fatalf("claim failed (claimed=%v, job=%d): %v", claimed, job.ID, err)
|
||||
}
|
||||
|
||||
// The finishing transaction fails (context cancellation stands in for a timeout or a
|
||||
// database error). The claim is already committed, so the job stays processing.
|
||||
canceled, cancel := context.WithCancel(t.Context())
|
||||
cancel()
|
||||
if err = FinishIngestJob(canceled, db, job, time.Now()); err == nil {
|
||||
t.Fatal("a canceled finishing transaction must report an error")
|
||||
}
|
||||
var stuck IngestJob
|
||||
if err = db.Where("id = ?", pasted.Job.ID).First(&stuck).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if stuck.Status != statusProcessing {
|
||||
t.Fatalf("job status %q after a failed finish, want processing", stuck.Status)
|
||||
}
|
||||
// A manual retry cannot rescue it: only failed jobs are accepted.
|
||||
code, msg, _ := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", pasted.Job.ID), learner.Token, nil)
|
||||
if code != 409 || msg == "" {
|
||||
t.Fatalf("retry of a processing job status %d (%s), want 409", code, msg)
|
||||
}
|
||||
|
||||
// A sweep that is too early must leave a healthy claim alone.
|
||||
claimedAt := stuck.UpdatedAt
|
||||
if _, err = RequeueStaleIngestJobs(db, claimedAt.Add(time.Second)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if jobRow(t, db, pasted.Job.ID).Status != statusProcessing {
|
||||
t.Fatal("a fresh claim must not be requeued")
|
||||
}
|
||||
// Once the claim is older than the stale window, the running service recovers it.
|
||||
if _, err = RequeueStaleIngestJobs(db, claimedAt.Add(ingestStaleAfter+time.Second)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
recovered := jobRow(t, db, pasted.Job.ID)
|
||||
if recovered.Status != statusPending || recovered.Attempts != 1 || recovered.FinishedAt != nil {
|
||||
t.Fatalf("recovered job %+v", recovered)
|
||||
}
|
||||
if chapterRow(t, db, pasted.Chapter.ID).Status != statusPending {
|
||||
t.Fatal("recovered chapter must be pending")
|
||||
}
|
||||
// Recovery reuses the same rows: no second chapter, same task id.
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("recovery left %d chapters, want 1", chapters)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("recovered chapter %+v", ready.Chapter)
|
||||
}
|
||||
if ready.Chapter.ID != pasted.Chapter.ID || ready.Chapter.JobID == nil || *ready.Chapter.JobID != pasted.Job.ID {
|
||||
t.Fatal("recovery must keep the original chapter and task ids")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLIngestAttemptsAreBoundedAndManualRetryRestarts(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-attempt-budget", "title": "Attempt budget", "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// Spend the whole budget without any worker running.
|
||||
if err := db.Model(&IngestJob{}).Where("id = ?", pasted.Job.ID).Update("attempts", maxIngestAttempts).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, claimed, err := ClaimNextIngestJob(db, time.Now()); err != nil || claimed {
|
||||
t.Fatalf("claim claimed=%v (%v), want no claim once the budget is used", claimed, err)
|
||||
}
|
||||
if _, err := RequeueStaleIngestJobs(db, time.Now()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
exhausted := jobRow(t, db, pasted.Job.ID)
|
||||
if exhausted.Status != statusFailed || exhausted.ErrorReason != reasonAttemptsExhausted {
|
||||
t.Fatalf("exhausted job %+v", exhausted)
|
||||
}
|
||||
code, read := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || read.Chapter.Status != statusFailed || read.Chapter.ErrorReason != reasonAttemptsExhausted {
|
||||
t.Fatalf("exhausted chapter %+v", read.Chapter)
|
||||
}
|
||||
if read.Chapter.ErrorMessage == "" || read.Chapter.OriginalText != "" {
|
||||
t.Fatalf("exhausted chapter must fail readably without text: %+v", read.Chapter)
|
||||
}
|
||||
// The manual retry is still available and restarts the attempt budget.
|
||||
code, msg, _ := callRaw(t, r, "POST", fmt.Sprintf("/api/v1/jobs/%d/retry", pasted.Job.ID), learner.Token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("manual retry status %d (%s), want 200", code, msg)
|
||||
}
|
||||
retried := jobRow(t, db, pasted.Job.ID)
|
||||
if retried.Status != statusPending || retried.Attempts != 0 || retried.ErrorReason != "" {
|
||||
t.Fatalf("retried job %+v", retried)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.ErrorReason != "" {
|
||||
t.Fatalf("chapter after manual retry %+v", ready.Chapter)
|
||||
}
|
||||
if ready.Chapter.ID != pasted.Chapter.ID {
|
||||
t.Fatal("manual retry must reuse the same chapter")
|
||||
}
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("attempt recovery left %d chapters, want 1", chapters)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLIngestRecoveryAfterRestart(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
// The test database is shared with other cases, so empty the queue first: the claim below
|
||||
// must take this test's own job, not a leftover one.
|
||||
drainIngest(t, db)
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-recovery", "title": "Recovery", "text": fixturePastedText})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste status %d", code)
|
||||
}
|
||||
// A crash between claim and finish leaves rows in processing.
|
||||
job, claimed, err := ClaimNextIngestJob(db, time.Now())
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("claim failed (claimed=%v): %v", claimed, err)
|
||||
}
|
||||
if job.ID != pasted.Job.ID {
|
||||
t.Fatalf("claimed job %d, want this test's job %d", job.ID, pasted.Job.ID)
|
||||
}
|
||||
if chapterRow(t, db, pasted.Chapter.ID).Status != statusProcessing || jobRow(t, db, pasted.Job.ID).Status != statusProcessing {
|
||||
t.Fatal("claim must persist the processing state")
|
||||
}
|
||||
requeued, err := RecoverIngestJobs(db, time.Now())
|
||||
if err != nil || requeued < 1 {
|
||||
t.Fatalf("recovery requeued %d (%v), want at least this test's job", requeued, err)
|
||||
}
|
||||
pending := jobRow(t, db, pasted.Job.ID)
|
||||
if pending.Status != statusPending || pending.Attempts != 1 || pending.FinishedAt != nil {
|
||||
t.Fatalf("recovered job %+v", pending)
|
||||
}
|
||||
if chapterRow(t, db, pasted.Chapter.ID).Status != statusPending {
|
||||
t.Fatal("recovered chapter must be pending")
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, pasted.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.OriginalText != fixturePastedText {
|
||||
t.Fatalf("recovered chapter %+v", ready.Chapter)
|
||||
}
|
||||
if jobRow(t, db, pasted.Job.ID).Attempts != 2 {
|
||||
t.Fatal("the recovered job must be processed as a second attempt")
|
||||
}
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", pasted.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("recovery left %d chapters, want 1", chapters)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLPasteRejectsInvalidInputAndLimits(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
|
||||
longTitle := strings.Repeat("T", maxTitleRunes+1)
|
||||
overLimit := strings.Repeat("a", maxChapterRunes+1)
|
||||
cases := []struct {
|
||||
name string
|
||||
payload map[string]string
|
||||
}{
|
||||
{"missing_request_id", map[string]string{"title": "T", "text": "Text."}},
|
||||
{"short_request_id", map[string]string{"requestId": "short", "title": "T", "text": "Text."}},
|
||||
{"empty_title", map[string]string{"requestId": randomName("invalid"), "title": " ", "text": "Text."}},
|
||||
{"long_title", map[string]string{"requestId": randomName("invalid"), "title": longTitle, "text": "Text."}},
|
||||
{"empty_text", map[string]string{"requestId": randomName("invalid"), "title": "T", "text": " \r\n\t "}},
|
||||
{"unsupported_language", map[string]string{"requestId": randomName("invalid"), "title": "T", "text": "Text.", "language": "zh"}},
|
||||
{"over_limit", map[string]string{"requestId": randomName("invalid"), "title": "T", "text": overLimit}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", learner.Token, tc.payload)
|
||||
if code != 400 || msg == "" {
|
||||
t.Fatalf("status %d (%s), want 400 with a message", code, msg)
|
||||
}
|
||||
})
|
||||
}
|
||||
// The boundary itself is accepted, and an oversized body is refused before decoding.
|
||||
code, accepted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "fixture-boundary-limit", "title": "At the limit", "text": strings.Repeat("a", maxChapterRunes)})
|
||||
if code != 201 || accepted.Chapter.CharCount != maxChapterRunes {
|
||||
t.Fatalf("boundary paste status %d charCount %d", code, accepted.Chapter.CharCount)
|
||||
}
|
||||
code, msg, _ := callRaw(t, r, "POST", "/api/v1/books", learner.Token, map[string]string{
|
||||
"requestId": "fixture-body-limit", "title": "Too large", "text": strings.Repeat("a", maxPasteBodyBytes)})
|
||||
if code != 400 || !strings.Contains(msg, "过大") {
|
||||
t.Fatalf("oversized body status %d (%s), want 400 with 过大", code, msg)
|
||||
}
|
||||
var books, chapters int64
|
||||
db.Model(&Book{}).Where("owner_id = ?", learner.ID).Count(&books)
|
||||
db.Model(&Chapter{}).Where("owner_id = ?", learner.ID).Count(&chapters)
|
||||
if books != 1 || chapters != 1 {
|
||||
t.Fatalf("rejected input created books=%d chapters=%d", books, chapters)
|
||||
}
|
||||
}
|
||||
@@ -88,11 +88,216 @@ func emptyMigrationDB(t *testing.T) *gorm.DB {
|
||||
return db
|
||||
}
|
||||
|
||||
func TestMigrationFromV2PreservesExistingData(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
// Build a v2 database by hand: this is the state a deployed instance is in before #5.
|
||||
if err := db.Exec("CREATE TABLE lexgo_schema (id INT PRIMARY KEY,version INT,product VARCHAR(32))").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Exec("INSERT INTO lexgo_schema VALUES (1,2,'lexgo')").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, statement := range schemaV2Statements {
|
||||
if err := db.Exec(statement).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.Exec("INSERT INTO sys_user (user_id,username,password,role_id) VALUES (7,'fixture_v2','x',2)").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Exec("INSERT INTO lexgo_spaces (owner_id,language) VALUES (7,'en')").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatalf("v2 to v3 migration failed: %v", err)
|
||||
}
|
||||
if err := CheckSchema(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, table := range []string{"lexgo_books", "lexgo_chapters", "lexgo_ingest_jobs"} {
|
||||
var count int64
|
||||
db.Raw("SELECT COUNT(*) FROM information_schema.tables WHERE table_schema=DATABASE() AND table_name=?", table).Scan(&count)
|
||||
if count != 1 {
|
||||
t.Fatalf("migration did not create %s", table)
|
||||
}
|
||||
}
|
||||
var users, spaces int64
|
||||
db.Table("sys_user").Where("user_id = 7").Count(&users)
|
||||
db.Table("lexgo_spaces").Where("owner_id = 7").Count(&spaces)
|
||||
if users != 1 || spaces != 1 {
|
||||
t.Fatalf("migration changed existing rows: users=%d spaces=%d", users, spaces)
|
||||
}
|
||||
// A rollback marker set back to 2 can be upgraded again without touching data.
|
||||
if err := db.Exec("UPDATE lexgo_schema SET version=2 WHERE id=1").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatalf("re-upgrade failed: %v", err)
|
||||
}
|
||||
var version int
|
||||
db.Raw("SELECT version FROM lexgo_schema WHERE id=1").Scan(&version)
|
||||
if version != SchemaVersion {
|
||||
t.Fatalf("schema version %d after re-upgrade, want %d", version, SchemaVersion)
|
||||
}
|
||||
db.Table("sys_user").Where("user_id = 7").Count(&users)
|
||||
if users != 1 {
|
||||
t.Fatal("re-upgrade changed existing rows")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationFromV4KeepsLibraryAndAddsPersonalTerms(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
statements := []string{"CREATE TABLE lexgo_schema (id INT PRIMARY KEY,version INT,product VARCHAR(32))", "INSERT INTO lexgo_schema VALUES (1,4,'lexgo')"}
|
||||
statements = append(statements, schemaV2Statements...)
|
||||
statements = append(statements, schemaV3Statements...)
|
||||
statements = append(statements, schemaV4Statements...)
|
||||
for _, statement := range statements {
|
||||
if err := db.Exec(statement).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.Exec("INSERT INTO sys_user (user_id,username,password,role_id) VALUES (9,'fixture_v4','fictional-not-a-real-hash',2)").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := stamp(time.Now())
|
||||
book := Book{OwnerID: 9, Title: "Fictional migration", Language: "en", CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&book).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
original := "😀 Original e\u0301\r\n"
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: 9, Ordinal: 1, Title: "Fixture", OriginalText: original, ContentSHA256: contentSHA(original), Status: statusReady, CharCount: len([]rune(original)), CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&chapter).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resource := DictionaryResource{ID: 1, Name: "Fixture WordNet", Language: "en", Version: "3.0", Source: WordNetSource, Format: "wordnet-3.0-zip", SHA256: strings.Repeat("a", 64), EntryCount: 5, Enabled: true, Archive: []byte("fixture archive"), UpdatedAt: now}
|
||||
if err := db.Create(&resource).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err == nil {
|
||||
t.Fatal("old schema accepted before explicit migration")
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var restored Chapter
|
||||
if err := db.First(&restored, chapter.ID).Error; err != nil || restored.OriginalText != original || restored.ContentSHA256 != chapter.ContentSHA256 {
|
||||
t.Fatal("migration changed original chapter", err)
|
||||
}
|
||||
var restoredResource DictionaryResource
|
||||
if err := db.Omit("archive").First(&restoredResource, 1).Error; err != nil || restoredResource.SHA256 != resource.SHA256 || restoredResource.EntryCount != 5 {
|
||||
t.Fatal("migration changed the shared dictionary resource", err)
|
||||
}
|
||||
var terms int64
|
||||
if err := db.Model(&Term{}).Count(&terms).Error; err != nil || terms != 0 {
|
||||
t.Fatal("migration must create an empty personal term table", terms, err)
|
||||
}
|
||||
// The new table is usable and enforces the documented status boundary.
|
||||
valid := Term{OwnerID: 9, Language: "en", Term: "curiosity", OriginalForm: "Curiosity", Status: termStatusNew, CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&valid).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Create(&Term{OwnerID: 9, Language: "en", Term: "forged", OriginalForm: "forged", Status: "deleted", CreatedAt: now, UpdatedAt: now}).Error; err == nil {
|
||||
t.Fatal("an unknown status must be rejected by the schema check")
|
||||
}
|
||||
if err := db.Create(&Term{OwnerID: 9, Language: "en", Term: "leveled", OriginalForm: "leveled", Status: termStatusKnown, Level: 5, CreatedAt: now, UpdatedAt: now}).Error; err == nil {
|
||||
t.Fatal("a level outside 0～7 must be rejected by the schema check")
|
||||
}
|
||||
// Rolling the marker back for a binary rollback and upgrading again keeps rows.
|
||||
if err := db.Exec("UPDATE lexgo_schema SET version=4 WHERE id=1").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Model(&Term{}).Count(&terms).Error; err != nil || terms != 1 {
|
||||
t.Fatal("re-upgrade changed existing personal terms", terms, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationFromV5AddsReviewScheduling(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
statements := []string{"CREATE TABLE lexgo_schema (id INT PRIMARY KEY,version INT,product VARCHAR(32))", "INSERT INTO lexgo_schema VALUES (1,5,'lexgo')"}
|
||||
statements = append(statements, schemaV2Statements...)
|
||||
statements = append(statements, schemaV3Statements...)
|
||||
statements = append(statements, schemaV4Statements...)
|
||||
statements = append(statements, schemaV5Statements...)
|
||||
for _, statement := range statements {
|
||||
if err := db.Exec(statement).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.Exec("INSERT INTO sys_user (user_id,username,password,role_id) VALUES (11,'fixture_v5','fictional-not-a-real-hash',2)").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
saved := stamp(time.Now())
|
||||
term := Term{OwnerID: 11, Language: "en", Term: "curiosity", OriginalForm: "Curiosity", Definition: "虚构释义", Examples: "Fictional example.", Status: termStatusLearning, Level: 2, CreatedAt: saved, UpdatedAt: saved}
|
||||
if err := db.Create(&term).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err == nil {
|
||||
t.Fatal("old schema accepted before explicit migration")
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// The learner's own term is untouched and has never been reviewed.
|
||||
var restored Term
|
||||
if err := db.First(&restored, term.ID).Error; err != nil || restored.Definition != term.Definition || restored.Level != 2 || restored.Status != termStatusLearning {
|
||||
t.Fatal("migration changed the personal term", err)
|
||||
}
|
||||
// An existing saved word enters the queue immediately: due at the moment it was saved.
|
||||
var review TermReview
|
||||
if err := db.Where("term_id = ?", term.ID).First(&review).Error; err != nil {
|
||||
t.Fatal("existing terms need a schedule row", err)
|
||||
}
|
||||
if !review.DueAt.Equal(saved) || review.ReviewCount != 0 || review.CorrectCount != 0 || review.WrongCount != 0 || review.Language != "en" {
|
||||
t.Fatalf("schedule row: %#v", review)
|
||||
}
|
||||
var answers int64
|
||||
if err := db.Model(&ReviewAnswer{}).Count(&answers).Error; err != nil || answers != 0 {
|
||||
t.Fatal("migration must create an empty attempt log", answers, err)
|
||||
}
|
||||
// A second identical answer row is rejected by the unique answer key.
|
||||
digest, err := requestKey("migration-fixture-answer")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
record := ReviewAnswer{OwnerID: 11, AnswerKey: digest, TermID: term.ID, Grade: reviewGradeCorrect, Result: "applied", StatusBefore: termStatusNew, StatusAfter: termStatusLearning, LevelBefore: 0, LevelAfter: 1, DueAtBefore: saved, DueAtAfter: saved, CreatedAt: saved}
|
||||
if err := db.Create(&record).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
duplicate := record
|
||||
duplicate.ID = 0
|
||||
if err := db.Create(&duplicate).Error; err == nil {
|
||||
t.Fatal("the answer key must be unique per account")
|
||||
}
|
||||
// Rolling the marker back for a binary rollback and upgrading again keeps the rows.
|
||||
if err := db.Exec("UPDATE lexgo_schema SET version=5 WHERE id=1").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var reviews, attempts int64
|
||||
if err := db.Model(&TermReview{}).Count(&reviews).Error; err != nil || reviews != 1 {
|
||||
t.Fatalf("re-upgrade must not duplicate schedule rows: %d %v", reviews, err)
|
||||
}
|
||||
if err := db.Model(&ReviewAnswer{}).Count(&attempts).Error; err != nil || attempts != 1 {
|
||||
t.Fatalf("re-upgrade must keep the attempt log: %d %v", attempts, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationRefusesUnownedOrUnsupportedSchema(t *testing.T) {
|
||||
for _, tc := range []struct{ name, marker string }{
|
||||
{"empty_marker", ""},
|
||||
{"negative_version", "INSERT INTO lexgo_schema VALUES (1,-1,'lexgo')"},
|
||||
{"future_version", "INSERT INTO lexgo_schema VALUES (1,3,'lexgo')"},
|
||||
{"future_version", "INSERT INTO lexgo_schema VALUES (1,99,'lexgo')"},
|
||||
{"wrong_product", "INSERT INTO lexgo_schema VALUES (1,0,'another-app')"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
@@ -116,3 +321,53 @@ func TestMigrationRefusesUnownedOrUnsupportedSchema(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationFromV3PreservesLibraryAndJobs(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
statements := []string{"CREATE TABLE lexgo_schema (id INT PRIMARY KEY,version INT,product VARCHAR(32))", "INSERT INTO lexgo_schema VALUES (1,3,'lexgo')"}
|
||||
statements = append(statements, schemaV2Statements...)
|
||||
statements = append(statements, schemaV3Statements...)
|
||||
for _, statement := range statements {
|
||||
if err := db.Exec(statement).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := db.Exec("INSERT INTO sys_user (user_id,username,password,role_id) VALUES (8,'fixture_v3','fictional-not-a-real-hash',2)").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := stamp(time.Now())
|
||||
book := Book{OwnerID: 8, Title: "Fictional migration", Language: "en", CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&book).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
original := "😀 Original e\u0301\r\n"
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: 8, Ordinal: 1, Title: "Fixture", OriginalText: original, ContentSHA256: contentSHA(original), Status: statusReady, CharCount: len([]rune(original)), CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&chapter).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
job := IngestJob{OwnerID: 8, BookID: book.ID, ChapterID: chapter.ID, RequestKey: contentSHA("fixture-v3"), ContentSHA256: chapter.ContentSHA256, Status: statusReady, CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&job).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err == nil {
|
||||
t.Fatal("old schema accepted before explicit migration")
|
||||
}
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var restored Chapter
|
||||
if err := db.First(&restored, chapter.ID).Error; err != nil || restored.OriginalText != original || restored.ContentSHA256 != chapter.ContentSHA256 || restored.Status != statusReady {
|
||||
t.Fatal("migration changed original chapter", err)
|
||||
}
|
||||
var restoredJob IngestJob
|
||||
if err := db.First(&restoredJob, job.ID).Error; err != nil || restoredJob.RequestKey != job.RequestKey || restoredJob.Status != statusReady {
|
||||
t.Fatal("migration changed job", err)
|
||||
}
|
||||
var resources int64
|
||||
if err := db.Model(&DictionaryResource{}).Count(&resources).Error; err != nil || resources != 0 {
|
||||
t.Fatal("migration must create empty resource table", resources, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,246 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// A phrase is a run of consecutive words. Its identity is the ordered normalized word forms
|
||||
// joined by single spaces, so interior punctuation, line breaks and extra whitespace never
|
||||
// split one phrase into two records, and a phrase key can never collide with a word key.
|
||||
const (
|
||||
maxPhraseWords = 12
|
||||
minPhraseWords = 2
|
||||
maxPhraseKey = 128
|
||||
maxPhraseSource = 191
|
||||
)
|
||||
|
||||
// PhraseSpan is one saved phrase occurrence inside a chapter, expressed in token indices so
|
||||
// the reader can style a run without re-deriving the match.
|
||||
type PhraseSpan struct {
|
||||
ID int64 `json:"id"`
|
||||
Status string `json:"status"`
|
||||
WordCount int `json:"wordCount"`
|
||||
StartToken int `json:"startToken"`
|
||||
EndToken int `json:"endToken"`
|
||||
}
|
||||
|
||||
// phraseWords returns the word tokens fully inside a requested code point range. A range that
|
||||
// cuts into a word is rejected instead of silently dropping it, so the client always learns
|
||||
// that its selection was not a whole-word range.
|
||||
func phraseWords(tokens []TextToken, start, end int) ([]TextToken, error) {
|
||||
bad := failure(400, "请选择至少两个连续的完整单词")
|
||||
if start < 0 || end <= start {
|
||||
return nil, bad
|
||||
}
|
||||
words := make([]TextToken, 0, 4)
|
||||
for _, token := range tokens {
|
||||
if token.End <= start || token.Start >= end {
|
||||
continue
|
||||
}
|
||||
if token.Kind != "word" {
|
||||
continue
|
||||
}
|
||||
if token.Start < start || token.End > end {
|
||||
return nil, failure(400, "请选择完整的单词，不要切在单词中间")
|
||||
}
|
||||
words = append(words, token)
|
||||
}
|
||||
if len(words) < minPhraseWords {
|
||||
return nil, bad
|
||||
}
|
||||
if len(words) > maxPhraseWords {
|
||||
return nil, failure(400, "短语最多 12 个单词")
|
||||
}
|
||||
return words, nil
|
||||
}
|
||||
|
||||
// phraseKey is the identity of a phrase: the ordered normalized word forms joined by spaces.
|
||||
func phraseKey(words []TextToken) string {
|
||||
forms := make([]string, 0, len(words))
|
||||
for _, word := range words {
|
||||
forms = append(forms, normalizeWord(word.Text))
|
||||
}
|
||||
return strings.Join(forms, " ")
|
||||
}
|
||||
|
||||
// phraseSource is the exact original text of the selection, punctuation and line breaks
|
||||
// included, used for display only.
|
||||
func phraseSource(text string, words []TextToken) string {
|
||||
if len(words) == 0 {
|
||||
return ""
|
||||
}
|
||||
runes := []rune(text)
|
||||
start, end := words[0].Start, words[len(words)-1].End
|
||||
if start < 0 || end > len(runes) || start >= end {
|
||||
return ""
|
||||
}
|
||||
return string(runes[start:end])
|
||||
}
|
||||
|
||||
// phraseMatches finds every saved phrase occurrence in a chapter. Candidates are ordered by
|
||||
// their start and then longest first, and the leftmost-longest non-overlapping set is kept,
|
||||
// which is the display rule #4 verified; nothing stored is modified by matching.
|
||||
func phraseMatches(tokens []TextToken, phrases []Term) []PhraseSpan {
|
||||
if len(phrases) == 0 {
|
||||
return []PhraseSpan{}
|
||||
}
|
||||
// Index the phrases by their first word so a chapter only compares what can match.
|
||||
words := make([]int, 0, len(tokens))
|
||||
for index, token := range tokens {
|
||||
if token.Kind == "word" {
|
||||
words = append(words, index)
|
||||
}
|
||||
}
|
||||
byFirst := map[string][]Term{}
|
||||
for _, phrase := range phrases {
|
||||
forms := strings.Split(phrase.Term, " ")
|
||||
if len(forms) < minPhraseWords {
|
||||
continue
|
||||
}
|
||||
byFirst[forms[0]] = append(byFirst[forms[0]], phrase)
|
||||
}
|
||||
type candidate struct {
|
||||
startToken int
|
||||
endToken int
|
||||
phrase Term
|
||||
}
|
||||
found := make([]candidate, 0, 8)
|
||||
for position, tokenIndex := range words {
|
||||
first := normalizeWord(tokens[tokenIndex].Text)
|
||||
for _, phrase := range byFirst[first] {
|
||||
forms := strings.Split(phrase.Term, " ")
|
||||
if position+len(forms) > len(words) {
|
||||
continue
|
||||
}
|
||||
matched := true
|
||||
for offset, form := range forms {
|
||||
if normalizeWord(tokens[words[position+offset]].Text) != form {
|
||||
matched = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if matched {
|
||||
found = append(found, candidate{words[position], words[position+len(forms)-1], phrase})
|
||||
}
|
||||
}
|
||||
}
|
||||
sort.SliceStable(found, func(i, j int) bool {
|
||||
if found[i].startToken != found[j].startToken {
|
||||
return found[i].startToken < found[j].startToken
|
||||
}
|
||||
if found[i].endToken != found[j].endToken {
|
||||
return found[i].endToken > found[j].endToken
|
||||
}
|
||||
return found[i].phrase.ID < found[j].phrase.ID
|
||||
})
|
||||
spans := make([]PhraseSpan, 0, len(found))
|
||||
lastEnd := -1
|
||||
for _, item := range found {
|
||||
if item.startToken <= lastEnd {
|
||||
continue
|
||||
}
|
||||
spans = append(spans, PhraseSpan{
|
||||
ID: item.phrase.ID, Status: item.phrase.Status, WordCount: termWordCount(item.phrase.Term),
|
||||
StartToken: item.startToken, EndToken: item.endToken,
|
||||
})
|
||||
lastEnd = item.endToken
|
||||
}
|
||||
return spans
|
||||
}
|
||||
|
||||
// phrasesForChapter loads the caller's phrases and matches them against one chapter.
|
||||
func phrasesForChapter(tx *gorm.DB, owner int, language string, tokens []TextToken) ([]PhraseSpan, error) {
|
||||
var phrases []Term
|
||||
// A phrase key always contains a space and a word key never does, so this is the whole
|
||||
// filter; it needs no column.
|
||||
if err := tx.Select("id", "term", "status").
|
||||
Where("owner_id = ? AND language = ? AND term LIKE ?", owner, language, "% %").
|
||||
Find(&phrases).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return phraseMatches(tokens, phrases), nil
|
||||
}
|
||||
|
||||
type PhraseInput struct {
|
||||
ChapterID int64 `json:"chapterId"`
|
||||
Start *int `json:"start"`
|
||||
End *int `json:"end"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
Status string `json:"status"`
|
||||
Level *int `json:"level"`
|
||||
}
|
||||
|
||||
// SavePhrase derives the identity from the server's own tokens of an owned ready chapter, so a
|
||||
// client can neither name a phrase it did not select nor forge an owner or language. Saving
|
||||
// reuses the term table and its idempotent upsert, so one phrase stays one record with one
|
||||
// review schedule.
|
||||
func SavePhrase(tx *gorm.DB, owner int, language string, input PhraseInput, now time.Time) (TermSave, error) {
|
||||
if input.Start == nil || input.End == nil {
|
||||
return TermSave{}, failure(400, "请选择至少两个连续的完整单词")
|
||||
}
|
||||
chapter, err := readyOwnedChapter(tx, owner, input.ChapterID)
|
||||
if err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
words, err := phraseWords(Tokenize(chapter.OriginalText), *input.Start, *input.End)
|
||||
if err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
key := phraseKey(words)
|
||||
if utf8.RuneCountInString(key) > maxPhraseKey {
|
||||
return TermSave{}, failure(400, "短语过长，请缩短选择范围")
|
||||
}
|
||||
source := phraseSource(chapter.OriginalText, words)
|
||||
if source == "" || utf8.RuneCountInString(source) > maxPhraseSource {
|
||||
return TermSave{}, failure(400, "短语原文过长，请缩短选择范围")
|
||||
}
|
||||
level, err := termLevel(input.Status, input.Level, Term{}, false)
|
||||
if err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
definition, examples, err := termContent(input.Definition, input.Examples)
|
||||
if err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
previous, exists, err := previousTerm(tx, owner, language, key)
|
||||
if err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
if exists && termKind(previous.Term) != termKindPhrase {
|
||||
return TermSave{}, failure(409, "该内容已被记录为单词")
|
||||
}
|
||||
if exists {
|
||||
level, err = termLevel(input.Status, input.Level, previous, true)
|
||||
if err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
}
|
||||
fields := TermFields{
|
||||
Definition: definition, Examples: examples, Status: input.Status, Level: level,
|
||||
PreviousStatus: previous.Status, PreviousLevel: previous.Level, Exists: exists,
|
||||
}
|
||||
// The identity is the word-form key; the original selection is only the display text.
|
||||
return saveTerm(tx, owner, language, key, source, fields, now)
|
||||
}
|
||||
|
||||
func registerPhraseRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
v.POST("/phrases", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
var input PhraseInput
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
language, err := languageOf(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return SavePhrase(tx, u.UserId, language, input, now())
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// phraseFixtureText holds the cases #4 verified: interior punctuation, a line break inside a
|
||||
// phrase, a repeated phrase and a combining accent.
|
||||
const phraseFixtureText = "Take a small, step\nevery day.\nMira took a small step again.\n"
|
||||
|
||||
func phraseTokens(text string) []TextToken {
|
||||
return Tokenize(text)
|
||||
}
|
||||
|
||||
func spanOf(t *testing.T, tokens []TextToken, words ...string) (int, int) {
|
||||
t.Helper()
|
||||
var first, last TextToken
|
||||
found := 0
|
||||
for _, token := range tokens {
|
||||
if token.Kind != "word" {
|
||||
continue
|
||||
}
|
||||
if found < len(words) && normalizeWord(token.Text) == normalizeWord(words[found]) {
|
||||
if found == 0 {
|
||||
first = token
|
||||
}
|
||||
last = token
|
||||
found++
|
||||
}
|
||||
}
|
||||
if found != len(words) {
|
||||
t.Fatalf("could not find %v in %q", words, text(tokens))
|
||||
}
|
||||
return first.Start, last.End
|
||||
}
|
||||
|
||||
func text(tokens []TextToken) string {
|
||||
var builder strings.Builder
|
||||
for _, token := range tokens {
|
||||
builder.WriteString(token.Text)
|
||||
}
|
||||
return builder.String()
|
||||
}
|
||||
|
||||
// TestPhraseIdentityRules pins the range and identity rules of a phrase.
|
||||
func TestPhraseIdentityRules(t *testing.T) {
|
||||
tokens := phraseTokens(phraseFixtureText)
|
||||
|
||||
// Interior punctuation and a line break stay in the display text but not in the key.
|
||||
start, end := spanOf(t, tokens, "a", "small", "step")
|
||||
words, err := phraseWords(tokens, start, end)
|
||||
if err != nil || len(words) != 3 {
|
||||
t.Fatalf("phrase words: %v %v", words, err)
|
||||
}
|
||||
if key := phraseKey(words); key != "a small step" {
|
||||
t.Fatalf("key %q, want %q", key, "a small step")
|
||||
}
|
||||
if source := phraseSource(phraseFixtureText, words); source != "a small, step\nevery day."[:0]+"a small, step" {
|
||||
t.Fatalf("source %q", source)
|
||||
}
|
||||
// The same words separated by different punctuation are one identity.
|
||||
secondTokens := phraseTokens("Mira took a small step again.")
|
||||
secondStart, secondEnd := spanOf(t, secondTokens, "a", "small", "step")
|
||||
other, err := phraseWords(secondTokens, secondStart, secondEnd)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if phraseKey(other) != "a small step" {
|
||||
t.Fatalf("a differently spaced phrase must share the key: %q", phraseKey(other))
|
||||
}
|
||||
// Case and curly apostrophes fold, exactly like word identity.
|
||||
if key := phraseKey(mustWords(t, "Don’t Look Back")); key != "don't look back" {
|
||||
t.Fatalf("normalized phrase key: %q", key)
|
||||
}
|
||||
|
||||
// Endpoint alignment: a range that cuts into a word is refused instead of dropping it.
|
||||
if _, err := phraseWords(tokens, start+3, end); err == nil {
|
||||
t.Fatal("a range cutting into a word must be rejected")
|
||||
}
|
||||
if _, err := phraseWords(tokens, start, end-2); err == nil {
|
||||
t.Fatal("a range cutting into the last word must be rejected")
|
||||
}
|
||||
// A range that starts on a separator simply begins at the next whole word.
|
||||
trimmed, err := phraseWords(tokens, start+1, end)
|
||||
if err != nil || len(trimmed) != 2 || normalizeWord(trimmed[0].Text) != "small" {
|
||||
t.Fatalf("a separator boundary must select whole words: %v %v", trimmed, err)
|
||||
}
|
||||
// Surrounding whitespace and punctuation are allowed on both ends.
|
||||
if _, err := phraseWords(tokens, start-1, end+1); err != nil {
|
||||
t.Fatalf("surrounding separators must be accepted: %v", err)
|
||||
}
|
||||
// A single word, an empty range and a punctuation-only range are not phrases.
|
||||
if _, err := phraseWords(tokens, words[0].Start, words[0].End); err == nil {
|
||||
t.Fatal("one word is not a phrase")
|
||||
}
|
||||
if _, err := phraseWords(tokens, 0, 0); err == nil {
|
||||
t.Fatal("an empty range is not a phrase")
|
||||
}
|
||||
if _, err := phraseWords(tokens, -1, 4); err == nil {
|
||||
t.Fatal("a negative range is not a phrase")
|
||||
}
|
||||
// Limits: at most 12 words, and a key within the stored column.
|
||||
long := make([]TextToken, 0, maxPhraseWords+1)
|
||||
position := 0
|
||||
for index := 0; index < maxPhraseWords+1; index++ {
|
||||
word := "word"
|
||||
long = append(long, TextToken{Text: word, Start: position, End: position + len(word), Kind: "word"})
|
||||
position += len(word) + 1
|
||||
}
|
||||
if _, err := phraseWords(long, 0, position); err == nil {
|
||||
t.Fatal("more than 12 words must be rejected")
|
||||
}
|
||||
if _, err := phraseWords(long[:maxPhraseWords], 0, len(long[maxPhraseWords-1].Text)+long[maxPhraseWords-1].Start); err != nil {
|
||||
t.Fatalf("exactly 12 words must be accepted: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestWordKeysNeverContainSpaces locks the premise the derived kind relies on: a word token
|
||||
// never contains a space, so the identity key alone tells a word from a phrase and no column is
|
||||
// needed for it.
|
||||
func TestWordKeysNeverContainSpaces(t *testing.T) {
|
||||
samples := []string{phraseFixtureText, "a b\tc\r\nd", "don’t stop-BELIEVING.", "🙂 café e\u0301 42"}
|
||||
checked := 0
|
||||
for _, sample := range samples {
|
||||
for _, token := range Tokenize(sample) {
|
||||
if token.Kind != "word" {
|
||||
continue
|
||||
}
|
||||
checked++
|
||||
if strings.ContainsAny(normalizeWord(token.Text), " \t\r\n") {
|
||||
t.Fatalf("word key %q contains whitespace", normalizeWord(token.Text))
|
||||
}
|
||||
}
|
||||
}
|
||||
if checked < 8 {
|
||||
t.Fatalf("expected several word tokens, checked %d", checked)
|
||||
}
|
||||
// The derived kind therefore agrees with the shape of the key in both directions.
|
||||
if termKind("a small step") != termKindPhrase || termWordCount("a small step") != 3 {
|
||||
t.Fatal("a key with spaces must be a three-word phrase")
|
||||
}
|
||||
if termKind("curiosity") != termKindWord || termWordCount("curiosity") != 1 {
|
||||
t.Fatal("a key without spaces must be a word")
|
||||
}
|
||||
}
|
||||
|
||||
func mustWords(t *testing.T, text string) []TextToken {
|
||||
t.Helper()
|
||||
words, err := phraseWords(phraseTokens(text), 0, len([]rune(text)))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return words
|
||||
}
|
||||
|
||||
// TestPhraseMatchingAndOverlap pins the cross-chapter match and the leftmost-longest display.
|
||||
func TestPhraseMatchingAndOverlap(t *testing.T) {
|
||||
tokens := phraseTokens(phraseFixtureText)
|
||||
step := Term{ID: 1, Term: "a small step", Status: termStatusNew}
|
||||
small := Term{ID: 2, Term: "small step", Status: termStatusLearning}
|
||||
|
||||
spans := phraseMatches(tokens, []Term{step, small})
|
||||
if len(spans) != 2 {
|
||||
t.Fatalf("expected one occurrence of each phrase, got %+v", spans)
|
||||
}
|
||||
// The longer phrase starts first, so it wins and the shorter one is dropped where they
|
||||
// overlap; the later occurrence of "a small step" is a separate match.
|
||||
if spans[0].ID != step.ID || spans[1].ID != step.ID {
|
||||
t.Fatalf("leftmost-longest rule: %+v", spans)
|
||||
}
|
||||
if spans[0].StartToken >= spans[1].StartToken {
|
||||
t.Fatalf("spans must be ordered: %+v", spans)
|
||||
}
|
||||
// A phrase that no longer occurs is simply absent, and nothing is mutated.
|
||||
if len(phraseMatches(phraseTokens("Nothing to see here."), []Term{step})) != 0 {
|
||||
t.Fatal("a phrase that does not occur must not match")
|
||||
}
|
||||
// Two different phrases that both match at the same place keep a stable choice.
|
||||
spans = phraseMatches(tokens, []Term{small, step})
|
||||
if len(spans) != 2 || spans[0].ID != step.ID {
|
||||
t.Fatalf("order of input must not change the display: %+v", spans)
|
||||
}
|
||||
// A single-word entry never participates as a phrase.
|
||||
if len(phraseMatches(tokens, []Term{{ID: 9, Term: "step"}})) != 0 {
|
||||
t.Fatal("a single-word key is not a phrase match")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLPhraseSaveAndCrossChapterHighlight is the core path: saving a phrase, finding it
|
||||
// again in another chapter, and keeping one record per identity.
|
||||
func TestMySQLPhraseSaveAndCrossChapterHighlight(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
code, first := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "phrase-fixture-0001", "title": "Phrase chapter", "text": phraseFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
// A second chapter with the same phrase in a different form.
|
||||
code, second := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "phrase-fixture-0002", "title": "Another chapter", "text": "Mira took a small step again.\n", "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("second paste %d", code)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
|
||||
tokens := phraseTokens(phraseFixtureText)
|
||||
start, end := spanOf(t, tokens, "a", "small", "step")
|
||||
body := map[string]any{
|
||||
"chapterId": first.Chapter.ID, "start": start, "end": end,
|
||||
"definition": "一小步", "examples": []string{"Take a small step, every day."}, "status": termStatusNew,
|
||||
}
|
||||
code, msg, data := callRaw(t, r, "POST", "/api/v1/phrases", learner.Token, body)
|
||||
if code != 201 {
|
||||
t.Fatalf("save phrase %d (%s)", code, msg)
|
||||
}
|
||||
var saved struct {
|
||||
Term TermView
|
||||
Created bool
|
||||
}
|
||||
json.Unmarshal(data, &saved)
|
||||
if saved.Term.Kind != "phrase" || saved.Term.WordCount != 3 || saved.Term.Term != "a small step" {
|
||||
t.Fatalf("saved phrase %+v", saved.Term)
|
||||
}
|
||||
if saved.Term.OriginalForm != "a small, step" {
|
||||
t.Fatalf("the display text keeps the original punctuation: %q", saved.Term.OriginalForm)
|
||||
}
|
||||
|
||||
// The same phrase saved from the other chapter, written differently, stays one record.
|
||||
otherTokens := phraseTokens("Mira took a small step again.\n")
|
||||
otherStart, otherEnd := spanOf(t, otherTokens, "a", "small", "step")
|
||||
code, msg, data = callRaw(t, r, "POST", "/api/v1/phrases", learner.Token, map[string]any{
|
||||
"chapterId": second.Chapter.ID, "start": otherStart, "end": otherEnd, "definition": "一小步（更新）", "status": termStatusNew,
|
||||
})
|
||||
if code != 200 {
|
||||
t.Fatalf("repeat phrase %d (%s)", code, msg)
|
||||
}
|
||||
var again struct {
|
||||
Term TermView
|
||||
Created bool
|
||||
}
|
||||
json.Unmarshal(data, &again)
|
||||
if again.Created || again.Term.ID != saved.Term.ID || again.Term.OriginalForm != "a small step" {
|
||||
t.Fatalf("one phrase must stay one record: %+v", again)
|
||||
}
|
||||
var count int64
|
||||
// A phrase key always contains a space, so the key shape is the whole filter.
|
||||
db.Model(&Term{}).Where("owner_id = ? AND term LIKE ?", learner.ID, "% %").Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatalf("phrase records: %d", count)
|
||||
}
|
||||
|
||||
// Both chapters highlight the phrase, including the one where it was not saved. The first
|
||||
// chapter holds the phrase twice with different interior punctuation.
|
||||
wantOccurrences := map[int64]int{first.Chapter.ID: 2, second.Chapter.ID: 1}
|
||||
for chapterID, want := range wantOccurrences {
|
||||
code, _, data = callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", chapterID), learner.Token, nil)
|
||||
var analyzed ChapterTokens
|
||||
json.Unmarshal(data, &analyzed)
|
||||
if code != 200 || len(analyzed.Phrases) != want {
|
||||
t.Fatalf("chapter %d phrases: %d %+v", chapterID, code, analyzed.Phrases)
|
||||
}
|
||||
for _, span := range analyzed.Phrases {
|
||||
if span.ID != saved.Term.ID || span.WordCount != 3 || span.EndToken <= span.StartToken {
|
||||
t.Fatalf("chapter %d span %+v", chapterID, span)
|
||||
}
|
||||
if analyzed.Tokens[span.StartToken].Kind != "word" || analyzed.Tokens[span.EndToken].Kind != "word" {
|
||||
t.Fatalf("a phrase must span words: %+v", span)
|
||||
}
|
||||
}
|
||||
}
|
||||
// Another account sees no phrase at all.
|
||||
code, _, data = callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", first.Chapter.ID), other.Token, nil)
|
||||
if code != 404 {
|
||||
t.Fatalf("another account reading this chapter: %d", code)
|
||||
}
|
||||
// The phrase joins the review queue with its kind, and answers like a word.
|
||||
code, queue := reviewQueue(t, r, learner.Token)
|
||||
if code != 200 || len(queue.Items) != 1 || queue.Items[0].Kind != "phrase" || queue.Items[0].WordCount != 3 {
|
||||
t.Fatalf("phrase queue: %d %+v", code, queue.Items)
|
||||
}
|
||||
code, applied := answerReview(t, r, learner.Token, saved.Term.ID, answerBody("phrase-answer-0001", reviewGradeCorrect, queue.Items[0].DueAt))
|
||||
if code != 201 || applied.Result != "applied" || applied.LevelAfter != 1 || applied.Item.Kind != "phrase" {
|
||||
t.Fatalf("phrase answer: %d %+v", code, applied)
|
||||
}
|
||||
// The word-level highlight is unaffected: the same words are still ordinary words.
|
||||
code, _, data = callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", first.Chapter.ID), learner.Token, nil)
|
||||
var withPhrase ChapterTokens
|
||||
json.Unmarshal(data, &withPhrase)
|
||||
for _, index := range []int{withPhrase.Phrases[0].StartToken, withPhrase.Phrases[0].EndToken} {
|
||||
if withPhrase.Tokens[index].Term != nil {
|
||||
t.Fatal("a phrase must not create word-level records")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLPhraseRulesAndIsolation covers the rejected shapes and ownership.
|
||||
func TestMySQLPhraseRulesAndIsolation(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "phrase-rules-0001", "title": "Phrase rules", "text": phraseFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
tokens := phraseTokens(phraseFixtureText)
|
||||
start, end := spanOf(t, tokens, "a", "small", "step")
|
||||
singleStart, singleEnd := wordsAt(t, tokens, "a")
|
||||
base := map[string]any{"chapterId": pasted.Chapter.ID, "start": start, "end": end, "status": termStatusNew}
|
||||
|
||||
if code, _, _ := callRaw(t, r, "POST", "/api/v1/phrases", other.Token, base); code != 404 {
|
||||
t.Fatal("another account must not save a phrase in this chapter")
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "POST", "/api/v1/phrases", "", base); code != 401 {
|
||||
t.Fatal("saving a phrase needs a session")
|
||||
}
|
||||
invalid := []map[string]any{
|
||||
{"chapterId": pasted.Chapter.ID, "start": singleStart, "end": singleEnd, "status": termStatusNew},
|
||||
{"chapterId": pasted.Chapter.ID, "end": end, "status": termStatusNew},
|
||||
{"chapterId": pasted.Chapter.ID, "start": start + 3, "end": end, "status": termStatusNew},
|
||||
{"chapterId": pasted.Chapter.ID, "start": start, "end": end, "status": "deleted"},
|
||||
{"chapterId": pasted.Chapter.ID, "start": start, "end": end, "status": termStatusKnown, "level": 3},
|
||||
{"chapterId": pasted.Chapter.ID, "start": start, "end": end, "status": termStatusNew, "definition": strings.Repeat("a", termDefinitionLimit+1)},
|
||||
{"chapterId": pasted.Chapter.ID, "start": start, "end": end, "status": termStatusNew, "term": "forged"},
|
||||
{"chapterId": pasted.Chapter.ID, "start": start, "end": end, "status": termStatusNew, "ownerId": 9},
|
||||
{"chapterId": 999999, "start": start, "end": end, "status": termStatusNew},
|
||||
}
|
||||
for _, body := range invalid {
|
||||
if code, msg, _ := callRaw(t, r, "POST", "/api/v1/phrases", learner.Token, body); code != 400 && code != 404 {
|
||||
t.Fatalf("invalid phrase %v -> %d (%s)", body, code, msg)
|
||||
}
|
||||
}
|
||||
var count int64
|
||||
db.Model(&Term{}).Where("owner_id = ?", learner.ID).Count(&count)
|
||||
if count != 0 {
|
||||
t.Fatalf("a rejected phrase was stored: %d", count)
|
||||
}
|
||||
|
||||
// A phrase in a chapter that is not ready is refused, exactly like a word.
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", pasted.Chapter.ID).Update("status", statusPending).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "POST", "/api/v1/phrases", learner.Token, base); code != 409 {
|
||||
t.Fatal("a pending chapter must refuse a phrase")
|
||||
}
|
||||
}
|
||||
|
||||
func wordsAt(t *testing.T, tokens []TextToken, word string) (int, int) {
|
||||
t.Helper()
|
||||
start, end := spanOf(t, tokens, word)
|
||||
return start, end
|
||||
}
|
||||
|
||||
// TestMySQLPhraseSurvivesTextEditsAndDeletion locks the fallback rule for stale references.
|
||||
func TestMySQLPhraseSurvivesTextEditsAndDeletion(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "phrase-survive-0001", "title": "Phrase survival", "text": phraseFixtureText, "language": "en"})
|
||||
if code != 201 {
|
||||
t.Fatalf("paste %d", code)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
tokens := phraseTokens(phraseFixtureText)
|
||||
start, end := spanOf(t, tokens, "a", "small", "step")
|
||||
code, msg, data := callRaw(t, r, "POST", "/api/v1/phrases", learner.Token, map[string]any{
|
||||
"chapterId": pasted.Chapter.ID, "start": start, "end": end, "definition": "一小步", "status": termStatusNew})
|
||||
if code != 201 {
|
||||
t.Fatalf("save phrase %d (%s)", code, msg)
|
||||
}
|
||||
var saved struct {
|
||||
Term TermView
|
||||
}
|
||||
json.Unmarshal(data, &saved)
|
||||
|
||||
// Editing the text so the phrase no longer occurs: the entry and its schedule stay.
|
||||
if code, msg, _ := patchResource(t, r, learner.Token, fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), map[string]string{"text": "Nothing left of it.\n"}); code != 200 {
|
||||
t.Fatalf("edit chapter %d (%s)", code, msg)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
code, _, data = callRaw(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", pasted.Chapter.ID), learner.Token, nil)
|
||||
var analyzed ChapterTokens
|
||||
json.Unmarshal(data, &analyzed)
|
||||
if code != 200 || len(analyzed.Phrases) != 0 {
|
||||
t.Fatalf("a phrase that no longer occurs must not highlight: %d %+v", code, analyzed.Phrases)
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/terms/%d", saved.Term.ID), learner.Token, nil); code != 200 {
|
||||
t.Fatal("the entry must survive a text edit")
|
||||
}
|
||||
code, queue := reviewQueue(t, r, learner.Token)
|
||||
if code != 200 || len(queue.Items) != 1 {
|
||||
t.Fatalf("the schedule must survive a text edit: %d %+v", code, queue.Items)
|
||||
}
|
||||
// Deleting the chapter keeps the entry and its schedule too.
|
||||
if code, msg, _ := callRaw(t, r, "DELETE", fmt.Sprintf("/api/v1/chapters/%d", pasted.Chapter.ID), learner.Token, nil); code != 200 {
|
||||
t.Fatalf("delete chapter %d (%s)", code, msg)
|
||||
}
|
||||
if code, _, _ := callRaw(t, r, "GET", fmt.Sprintf("/api/v1/terms/%d", saved.Term.ID), learner.Token, nil); code != 200 {
|
||||
t.Fatal("the entry must survive a chapter deletion")
|
||||
}
|
||||
code, queue = reviewQueue(t, r, learner.Token)
|
||||
if code != 200 || len(queue.Items) != 1 || queue.Items[0].Kind != "phrase" {
|
||||
t.Fatalf("the schedule must survive a chapter deletion: %d %+v", code, queue.Items)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,419 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// The three answers the accepted review prototype offers.
|
||||
const (
|
||||
reviewGradeCorrect = "correct"
|
||||
reviewGradeWrong = "wrong"
|
||||
reviewGradeAgain = "again"
|
||||
)
|
||||
|
||||
const (
|
||||
maxReviewLevel = 7
|
||||
// One queue page is enough for a daily round; the rest is fetched after finishing it.
|
||||
reviewQueueLimit = 50
|
||||
)
|
||||
|
||||
// reviewIntervals is the fixed schedule applied after a correct answer, indexed by
|
||||
// level-1. This is a fixed table, not FSRS: the decision table and its samples live in
|
||||
// the business rules page.
|
||||
var reviewIntervals = [maxReviewLevel]int{1, 2, 4, 7, 15, 30, 60}
|
||||
|
||||
var reviewGrades = map[string]bool{
|
||||
reviewGradeCorrect: true,
|
||||
reviewGradeWrong: true,
|
||||
reviewGradeAgain: true,
|
||||
}
|
||||
|
||||
// TermReview is the scheduling state of one personal term, kept in its own table so a
|
||||
// migration never alters the table that holds the learner's own text.
|
||||
type TermReview struct {
|
||||
TermID int64 `gorm:"primaryKey"`
|
||||
OwnerID int
|
||||
Language string
|
||||
DueAt time.Time
|
||||
ReviewCount int
|
||||
CorrectCount int
|
||||
WrongCount int
|
||||
LastReviewedAt *time.Time
|
||||
}
|
||||
|
||||
func (TermReview) TableName() string { return "lexgo_term_reviews" }
|
||||
|
||||
// ReviewAnswer records one attempt. The unique answer key is what makes a repeated
|
||||
// request, a double click or a network resend update nothing a second time.
|
||||
type ReviewAnswer struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
OwnerID int
|
||||
AnswerKey string `gorm:"column:answer_key"`
|
||||
TermID int64
|
||||
Grade string
|
||||
Result string
|
||||
StatusBefore string `gorm:"column:status_before"`
|
||||
StatusAfter string `gorm:"column:status_after"`
|
||||
LevelBefore int `gorm:"column:level_before"`
|
||||
LevelAfter int `gorm:"column:level_after"`
|
||||
DueAtBefore time.Time `gorm:"column:due_at_before"`
|
||||
DueAtAfter time.Time `gorm:"column:due_at_after"`
|
||||
Requeued bool
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
func (ReviewAnswer) TableName() string { return "lexgo_review_answers" }
|
||||
|
||||
type ReviewItem struct {
|
||||
ID int64 `json:"id"`
|
||||
Term string `json:"term"`
|
||||
OriginalForm string `json:"originalForm"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
Status string `json:"status"`
|
||||
Level int `json:"level"`
|
||||
Kind string `json:"kind"`
|
||||
WordCount int `json:"wordCount"`
|
||||
DueAt time.Time `json:"dueAt"`
|
||||
ReviewCount int `json:"reviewCount"`
|
||||
}
|
||||
|
||||
type ReviewQueue struct {
|
||||
Items []ReviewItem `json:"items"`
|
||||
Total int `json:"total"`
|
||||
}
|
||||
|
||||
type ReviewAnswerInput struct {
|
||||
AnswerID string `json:"answerId"`
|
||||
Grade string `json:"grade"`
|
||||
ExpectedDueAt time.Time `json:"expectedDueAt"`
|
||||
}
|
||||
|
||||
type ReviewAnswerResult struct {
|
||||
// Result is the outcome of this answer: applied when the term moved, stale when another
|
||||
// screen already advanced it. A replayed answer repeats the outcome it was given first.
|
||||
Result string `json:"result"`
|
||||
// Duplicate reports that this answer key was already recorded and nothing changed now.
|
||||
Duplicate bool `json:"duplicate"`
|
||||
Grade string `json:"grade"`
|
||||
Requeued bool `json:"requeued"`
|
||||
StatusBefore string `json:"statusBefore"`
|
||||
StatusAfter string `json:"statusAfter"`
|
||||
LevelBefore int `json:"levelBefore"`
|
||||
LevelAfter int `json:"levelAfter"`
|
||||
DueAtBefore time.Time `json:"dueAtBefore"`
|
||||
DueAtAfter time.Time `json:"dueAtAfter"`
|
||||
Item ReviewItem `json:"item"`
|
||||
}
|
||||
|
||||
// reviewState is the part of a term a grade moves.
|
||||
type reviewState struct {
|
||||
Status string
|
||||
Level int
|
||||
DueAt time.Time
|
||||
}
|
||||
|
||||
func nextDue(level int, now time.Time) time.Time {
|
||||
if level < 1 {
|
||||
level = 1
|
||||
}
|
||||
if level > maxReviewLevel {
|
||||
level = maxReviewLevel
|
||||
}
|
||||
return stamp(now.AddDate(0, 0, reviewIntervals[level-1]))
|
||||
}
|
||||
|
||||
// applyGrade advances one term by exactly one answer. It returns the next state and
|
||||
// whether the item belongs to the current round again.
|
||||
func applyGrade(before reviewState, grade string, now time.Time) (reviewState, bool, error) {
|
||||
switch grade {
|
||||
case reviewGradeCorrect:
|
||||
level := before.Level + 1
|
||||
if level > maxReviewLevel {
|
||||
level = maxReviewLevel
|
||||
}
|
||||
return reviewState{Status: termStatusLearning, Level: level, DueAt: nextDue(level, now)}, false, nil
|
||||
case reviewGradeWrong:
|
||||
next := reviewState{Status: before.Status, Level: before.Level}
|
||||
// Only a learning word loses a level; a new word stays new until it is answered
|
||||
// correctly for the first time.
|
||||
if before.Status == termStatusLearning {
|
||||
next.Level = before.Level - 1
|
||||
if next.Level < 1 {
|
||||
next.Level = 1
|
||||
}
|
||||
}
|
||||
next.DueAt = stamp(now)
|
||||
return next, true, nil
|
||||
case reviewGradeAgain:
|
||||
return reviewState{Status: before.Status, Level: before.Level, DueAt: stamp(now)}, true, nil
|
||||
default:
|
||||
return before, false, failure(400, "评分无效")
|
||||
}
|
||||
}
|
||||
|
||||
func reviewItem(term Term, review TermReview) ReviewItem {
|
||||
return ReviewItem{
|
||||
ID: term.ID, Term: term.Term, OriginalForm: term.OriginalForm, Definition: term.Definition,
|
||||
Examples: splitExamples(term.Examples), Status: term.Status, Level: term.Level,
|
||||
Kind: termKind(term.Term), WordCount: termWordCount(term.Term),
|
||||
DueAt: review.DueAt, ReviewCount: review.ReviewCount,
|
||||
}
|
||||
}
|
||||
|
||||
// ReviewQueueFor returns the caller's due words for the current language. Known and
|
||||
// ignored words stay out of the queue, and the queue is an absolute-instant comparison:
|
||||
// an item is due as soon as due_at is not in the future.
|
||||
func ReviewQueueFor(tx *gorm.DB, owner int, language string, now time.Time) (ReviewQueue, error) {
|
||||
queue := ReviewQueue{Items: []ReviewItem{}}
|
||||
due := func() *gorm.DB {
|
||||
return tx.Table("lexgo_terms AS t").
|
||||
Joins("JOIN lexgo_term_reviews AS r ON r.term_id = t.id").
|
||||
Where("t.owner_id = ? AND r.owner_id = ? AND t.language = ? AND r.language = ?", owner, owner, language, language).
|
||||
Where("t.status IN ?", []string{termStatusNew, termStatusLearning}).
|
||||
Where("r.due_at <= ?", stamp(now))
|
||||
}
|
||||
var total int64
|
||||
if err := due().Count(&total).Error; err != nil {
|
||||
return queue, err
|
||||
}
|
||||
queue.Total = int(total)
|
||||
var rows []struct {
|
||||
ID int64
|
||||
Term string
|
||||
OriginalForm string
|
||||
Definition string
|
||||
Examples string
|
||||
Status string
|
||||
Level int
|
||||
DueAt time.Time
|
||||
ReviewCount int
|
||||
}
|
||||
if err := due().
|
||||
Select("t.id, t.term, t.original_form, t.definition, t.examples, t.status, t.level, r.due_at, r.review_count").
|
||||
Order("r.due_at, t.id").Limit(reviewQueueLimit).Scan(&rows).Error; err != nil {
|
||||
return queue, err
|
||||
}
|
||||
for _, row := range rows {
|
||||
queue.Items = append(queue.Items, ReviewItem{
|
||||
ID: row.ID, Term: row.Term, OriginalForm: row.OriginalForm, Definition: row.Definition,
|
||||
Examples: splitExamples(row.Examples), Status: row.Status, Level: row.Level,
|
||||
Kind: termKind(row.Term), WordCount: termWordCount(row.Term),
|
||||
DueAt: row.DueAt, ReviewCount: row.ReviewCount,
|
||||
})
|
||||
}
|
||||
return queue, nil
|
||||
}
|
||||
|
||||
// syncTermReview keeps the scheduling row in step with a saved term. A new word is due
|
||||
// immediately so saving it starts the loop; a word saved straight into a level waits for
|
||||
// that level's interval, so a manual level is not silently re-queued today. When the caller
|
||||
// does not ask for a reschedule, only a missing row is created and the existing date and
|
||||
// counters stay untouched.
|
||||
func syncTermReview(tx *gorm.DB, term Term, reschedule bool, now time.Time) error {
|
||||
due := stamp(now)
|
||||
if term.Status == termStatusLearning && term.Level >= 1 {
|
||||
due = nextDue(term.Level, now)
|
||||
}
|
||||
var existing TermReview
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("term_id = ?", term.ID).First(&existing).Error
|
||||
switch {
|
||||
case errors.Is(err, gorm.ErrRecordNotFound):
|
||||
return tx.Create(&TermReview{TermID: term.ID, OwnerID: term.OwnerID, Language: term.Language, DueAt: due}).Error
|
||||
case err != nil:
|
||||
return err
|
||||
case reschedule:
|
||||
return tx.Model(&TermReview{}).Where("term_id = ?", term.ID).Update("due_at", due).Error
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func lockedReview(tx *gorm.DB, owner int, term Term) (TermReview, error) {
|
||||
var review TermReview
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("term_id = ? AND owner_id = ?", term.ID, owner).First(&review).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
// A term written before this table existed is due at the moment it was saved.
|
||||
review = TermReview{TermID: term.ID, OwnerID: owner, Language: term.Language, DueAt: stamp(term.CreatedAt)}
|
||||
if err = tx.Create(&review).Error; err != nil {
|
||||
return review, err
|
||||
}
|
||||
return review, nil
|
||||
}
|
||||
return review, err
|
||||
}
|
||||
|
||||
// lockedAnswerKey reads one recorded answer under a lock. It is called again after the term
|
||||
// row is locked, because a locking read sees the newest committed row while the plain read
|
||||
// before it may still see this transaction's snapshot.
|
||||
func lockedAnswerKey(tx *gorm.DB, owner int, key string) (ReviewAnswer, error) {
|
||||
var stored ReviewAnswer
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("owner_id = ? AND answer_key = ?", owner, key).First(&stored).Error
|
||||
return stored, err
|
||||
}
|
||||
|
||||
// AnswerReview applies one grade to one owned term exactly once. A repeated answer key
|
||||
// returns the first outcome, and an answer whose due time no longer matches (another tab
|
||||
// already advanced the term) is recorded as stale without moving anything.
|
||||
func AnswerReview(tx *gorm.DB, owner int, termID int64, input ReviewAnswerInput, now time.Time) (ReviewAnswerResult, error) {
|
||||
if !reviewGrades[input.Grade] {
|
||||
return ReviewAnswerResult{}, failure(400, "评分无效")
|
||||
}
|
||||
if input.ExpectedDueAt.IsZero() {
|
||||
return ReviewAnswerResult{}, failure(400, "请提交队列中的到期时间")
|
||||
}
|
||||
key, err := requestKey(input.AnswerID)
|
||||
if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
if stored, err := lockedAnswerKey(tx, owner, key); err == nil {
|
||||
return replayedAnswer(tx, stored, termID)
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
// Ownership first: another account's term and a missing term answer identically.
|
||||
var term Term
|
||||
if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("id = ? AND owner_id = ?", termID, owner).First(&term).Error; errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReviewAnswerResult{}, failure(404, "词条不存在")
|
||||
} else if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
// Simultaneous submissions of one answer id are serialized by the term lock above; the
|
||||
// second one now sees the recorded answer and reports it instead of counting again.
|
||||
if stored, err := lockedAnswerKey(tx, owner, key); err == nil {
|
||||
return replayedAnswer(tx, stored, termID)
|
||||
} else if !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
review, err := lockedReview(tx, owner, term)
|
||||
if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
if term.Status != termStatusNew && term.Status != termStatusLearning {
|
||||
return ReviewAnswerResult{}, failure(409, "该词条已不在复习队列，请刷新队列")
|
||||
}
|
||||
before := reviewState{Status: term.Status, Level: term.Level, DueAt: review.DueAt}
|
||||
if !input.ExpectedDueAt.Equal(review.DueAt) {
|
||||
return staleAnswer(tx, owner, term, review, input, now)
|
||||
}
|
||||
next, requeued, err := applyGrade(before, input.Grade, now)
|
||||
if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
when := stamp(now)
|
||||
if err := tx.Model(&Term{}).Where("id = ? AND owner_id = ?", term.ID, owner).
|
||||
Updates(map[string]any{"status": next.Status, "level": next.Level, "updated_at": when}).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
counters := map[string]any{
|
||||
"due_at": next.DueAt, "review_count": gorm.Expr("review_count + 1"),
|
||||
"last_reviewed_at": when,
|
||||
}
|
||||
// "again" is not a correct answer either: the word was not recognised this time.
|
||||
if input.Grade == reviewGradeCorrect {
|
||||
counters["correct_count"] = gorm.Expr("correct_count + 1")
|
||||
} else {
|
||||
counters["wrong_count"] = gorm.Expr("wrong_count + 1")
|
||||
}
|
||||
if err := tx.Model(&TermReview{}).Where("term_id = ? AND owner_id = ?", term.ID, owner).Updates(counters).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
term.Status, term.Level, term.UpdatedAt = next.Status, next.Level, when
|
||||
review.DueAt, review.ReviewCount, review.LastReviewedAt = next.DueAt, review.ReviewCount+1, &when
|
||||
record := ReviewAnswer{
|
||||
OwnerID: owner, AnswerKey: key, TermID: term.ID, Grade: input.Grade, Result: "applied",
|
||||
StatusBefore: before.Status, StatusAfter: next.Status, LevelBefore: before.Level, LevelAfter: next.Level,
|
||||
DueAtBefore: before.DueAt, DueAtAfter: next.DueAt, Requeued: requeued, CreatedAt: when,
|
||||
}
|
||||
if err := tx.Create(&record).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
return ReviewAnswerResult{
|
||||
Result: "applied", Grade: input.Grade, Requeued: requeued,
|
||||
StatusBefore: before.Status, StatusAfter: next.Status, LevelBefore: before.Level, LevelAfter: next.Level,
|
||||
DueAtBefore: before.DueAt, DueAtAfter: next.DueAt, Item: reviewItem(term, review),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// staleAnswer records that this attempt changed nothing because the term had already
|
||||
// moved on. It is a normal outcome of two open tabs, not an error.
|
||||
func staleAnswer(tx *gorm.DB, owner int, term Term, review TermReview, input ReviewAnswerInput, now time.Time) (ReviewAnswerResult, error) {
|
||||
key, err := requestKey(input.AnswerID)
|
||||
if err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
record := ReviewAnswer{
|
||||
OwnerID: owner, AnswerKey: key, TermID: term.ID, Grade: input.Grade, Result: "stale",
|
||||
StatusBefore: term.Status, StatusAfter: term.Status, LevelBefore: term.Level, LevelAfter: term.Level,
|
||||
DueAtBefore: review.DueAt, DueAtAfter: review.DueAt, CreatedAt: stamp(now),
|
||||
}
|
||||
if err := tx.Create(&record).Error; err != nil {
|
||||
// Losing a race to another transaction that recorded this very answer id is not an
|
||||
// error: report the outcome that was stored first.
|
||||
var duplicate *driver.MySQLError
|
||||
if errors.As(err, &duplicate) && duplicate.Number == 1062 {
|
||||
if stored, readErr := lockedAnswerKey(tx, owner, key); readErr == nil {
|
||||
return replayedAnswer(tx, stored, term.ID)
|
||||
}
|
||||
}
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
return ReviewAnswerResult{
|
||||
Result: "stale", Grade: input.Grade, Requeued: false,
|
||||
StatusBefore: term.Status, StatusAfter: term.Status, LevelBefore: term.Level, LevelAfter: term.Level,
|
||||
DueAtBefore: review.DueAt, DueAtAfter: review.DueAt, Item: reviewItem(term, review),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// replayedAnswer reports the outcome already recorded for this answer key. Nothing is
|
||||
// advanced a second time, so a resend cannot change counts or intervals.
|
||||
func replayedAnswer(tx *gorm.DB, stored ReviewAnswer, termID int64) (ReviewAnswerResult, error) {
|
||||
if stored.TermID != termID {
|
||||
return ReviewAnswerResult{}, failure(400, "请求编号已用于其他词条")
|
||||
}
|
||||
var term Term
|
||||
if err := tx.Where("id = ? AND owner_id = ?", stored.TermID, stored.OwnerID).First(&term).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
var review TermReview
|
||||
if err := tx.Where("term_id = ? AND owner_id = ?", stored.TermID, stored.OwnerID).First(&review).Error; err != nil {
|
||||
return ReviewAnswerResult{}, err
|
||||
}
|
||||
return ReviewAnswerResult{
|
||||
Result: stored.Result, Duplicate: true, Grade: stored.Grade, Requeued: stored.Requeued,
|
||||
StatusBefore: stored.StatusBefore, StatusAfter: stored.StatusAfter,
|
||||
LevelBefore: stored.LevelBefore, LevelAfter: stored.LevelAfter,
|
||||
DueAtBefore: stored.DueAtBefore, DueAtAfter: stored.DueAtAfter, Item: reviewItem(term, review),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func registerReviewRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
v.GET("/reviews/queue", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "复习队列不接受查询参数")
|
||||
}
|
||||
language, err := languageOf(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ReviewQueueFor(tx, u.UserId, language, now())
|
||||
}))
|
||||
v.POST("/reviews/:termId/answers", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
termID, err := strconv.ParseInt(c.Param("termId"), 10, 64)
|
||||
if err != nil || termID <= 0 {
|
||||
return nil, failure(404, "词条不存在")
|
||||
}
|
||||
var input ReviewAnswerInput
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return AnswerReview(tx, u.UserId, termID, input, now())
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,501 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// TestReviewDecisionTable pins the fixed schedule and every state transition. It is the
|
||||
// design evidence the ticket asks for, so a change to the table must fail here first.
|
||||
func TestReviewDecisionTable(t *testing.T) {
|
||||
levels := []int{1, 2, 3, 4, 5, 6, 7}
|
||||
days := []int{1, 2, 4, 7, 15, 30, 60}
|
||||
if len(reviewIntervals) != len(levels) {
|
||||
t.Fatalf("interval table has %d entries, want %d", len(reviewIntervals), len(levels))
|
||||
}
|
||||
now := time.Date(2026, 9, 11, 10, 30, 0, 0, time.UTC)
|
||||
for index, level := range levels {
|
||||
if reviewIntervals[level-1] != days[index] {
|
||||
t.Fatalf("level %d interval %d, want %d", level, reviewIntervals[level-1], days[index])
|
||||
}
|
||||
next, requeued, err := applyGrade(reviewState{Status: termStatusLearning, Level: level, DueAt: now}, reviewGradeCorrect, now)
|
||||
want := level + 1
|
||||
if want > maxReviewLevel {
|
||||
want = maxReviewLevel
|
||||
}
|
||||
if err != nil || requeued || next.Level != want || !next.DueAt.Equal(now.AddDate(0, 0, reviewIntervals[want-1])) || next.Status != termStatusLearning {
|
||||
t.Fatalf("level %d correct: %#v requeued=%v err=%v", level, next, requeued, err)
|
||||
}
|
||||
}
|
||||
cases := []struct {
|
||||
name string
|
||||
before reviewState
|
||||
grade string
|
||||
status string
|
||||
level int
|
||||
dueAt time.Time
|
||||
requeued bool
|
||||
}{
|
||||
{"new correct becomes learning level 1", reviewState{Status: termStatusNew}, reviewGradeCorrect, termStatusLearning, 1, now.AddDate(0, 0, 1), false},
|
||||
{"new wrong stays new and requeues", reviewState{Status: termStatusNew}, reviewGradeWrong, termStatusNew, 0, now, true},
|
||||
{"new again stays new and requeues", reviewState{Status: termStatusNew}, reviewGradeAgain, termStatusNew, 0, now, true},
|
||||
{"learning 4 wrong drops to 3", reviewState{Status: termStatusLearning, Level: 4}, reviewGradeWrong, termStatusLearning, 3, now, true},
|
||||
{"learning 1 wrong floors at 1", reviewState{Status: termStatusLearning, Level: 1}, reviewGradeWrong, termStatusLearning, 1, now, true},
|
||||
{"learning 7 wrong drops to 6", reviewState{Status: termStatusLearning, Level: 7}, reviewGradeWrong, termStatusLearning, 6, now, true},
|
||||
{"again keeps the level", reviewState{Status: termStatusLearning, Level: 3}, reviewGradeAgain, termStatusLearning, 3, now, true},
|
||||
{"level 7 correct stays at 7", reviewState{Status: termStatusLearning, Level: 7}, reviewGradeCorrect, termStatusLearning, 7, now.AddDate(0, 0, 60), false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
before := tc.before
|
||||
before.DueAt = now
|
||||
next, requeued, err := applyGrade(before, tc.grade, now)
|
||||
if err != nil || requeued != tc.requeued || next.Status != tc.status || next.Level != tc.level || !next.DueAt.Equal(tc.dueAt) {
|
||||
t.Fatalf("%s: %#v requeued=%v err=%v", tc.name, next, requeued, err)
|
||||
}
|
||||
}
|
||||
if _, _, err := applyGrade(reviewState{Status: termStatusNew, DueAt: now}, "maybe", now); err == nil {
|
||||
t.Fatal("an unknown grade must be rejected")
|
||||
}
|
||||
}
|
||||
|
||||
// reviewFixture builds one learner, one book and one ready chapter under a clock the
|
||||
// test can move, so due boundaries are exact.
|
||||
func reviewFixture(t *testing.T, db *gorm.DB, text string) (*gin.Engine, *time.Time, admin.SysUser, string, Chapter) {
|
||||
t.Helper()
|
||||
clock := time.Now().UTC().Truncate(time.Millisecond)
|
||||
r := Router(db, func() time.Time { return clock })
|
||||
user, token, chapters := termFixture(t, db, r, text)
|
||||
return r, &clock, user, token, chapters[0]
|
||||
}
|
||||
|
||||
func reviewQueue(t *testing.T, r *gin.Engine, token string) (int, ReviewQueue) {
|
||||
t.Helper()
|
||||
code, data := callAPI(t, r, "GET", "/api/v1/reviews/queue", token, nil)
|
||||
var queue ReviewQueue
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &queue)
|
||||
}
|
||||
return code, queue
|
||||
}
|
||||
|
||||
func answerReview(t *testing.T, r *gin.Engine, token string, termID int64, body map[string]any) (int, ReviewAnswerResult) {
|
||||
t.Helper()
|
||||
code, data := callAPI(t, r, "POST", fmt.Sprintf("/api/v1/reviews/%d/answers", termID), token, body)
|
||||
var result ReviewAnswerResult
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &result)
|
||||
}
|
||||
return code, result
|
||||
}
|
||||
|
||||
func answerBody(answerID, grade string, dueAt time.Time) map[string]any {
|
||||
return map[string]any{"answerId": answerID, "grade": grade, "expectedDueAt": dueAt.UTC().Format(time.RFC3339Nano)}
|
||||
}
|
||||
|
||||
// saveWord saves one word span of an owned chapter and returns the stored record.
|
||||
func saveWord(t *testing.T, r *gin.Engine, token string, chapterID int64, start, end int, status string, level *int) TermSave {
|
||||
t.Helper()
|
||||
body := map[string]any{"chapterId": chapterID, "start": start, "end": end, "definition": "虚构释义", "status": status}
|
||||
if level != nil {
|
||||
body["level"] = *level
|
||||
}
|
||||
code, saved := saveTermAPI(t, r, token, body)
|
||||
if code != 201 && code != 200 {
|
||||
t.Fatalf("save word %d-%d: %d %#v", start, end, code, saved)
|
||||
}
|
||||
return saved
|
||||
}
|
||||
|
||||
func termReviewRow(t *testing.T, db *gorm.DB, termID int64) TermReview {
|
||||
t.Helper()
|
||||
var review TermReview
|
||||
if err := db.Where("term_id = ?", termID).First(&review).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return review
|
||||
}
|
||||
|
||||
// TestMySQLReviewQueueDueBoundaryAndScope covers the due boundary, the queue scope and
|
||||
// per-account isolation.
|
||||
func TestMySQLReviewQueueDueBoundaryAndScope(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
_, otherToken, otherChapters := termFixture(t, db, r, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
|
||||
// A saved word is due immediately: its due time is exactly the save time.
|
||||
code, queue := reviewQueue(t, r, token)
|
||||
if code != 200 || queue.Total != 1 || len(queue.Items) != 1 || queue.Items[0].ID != dogs.Term.ID {
|
||||
t.Fatalf("a new word must be due at once: %d %#v", code, queue)
|
||||
}
|
||||
item := queue.Items[0]
|
||||
if !item.DueAt.Equal(*clock) || item.Status != termStatusNew || item.Level != 0 || item.ReviewCount != 0 || item.Examples == nil {
|
||||
t.Fatalf("queue item: %#v", item)
|
||||
}
|
||||
if !termReviewRow(t, db, dogs.Term.ID).DueAt.Equal(*clock) {
|
||||
t.Fatal("the schedule row must hold the save time")
|
||||
}
|
||||
|
||||
// due_at == now is due; one millisecond earlier is not.
|
||||
*clock = item.DueAt.Add(-time.Millisecond)
|
||||
if _, early := reviewQueue(t, r, token); early.Total != 0 {
|
||||
t.Fatalf("an item due later must stay out of the queue: %#v", early)
|
||||
}
|
||||
*clock = item.DueAt.Add(time.Millisecond)
|
||||
if _, late := reviewQueue(t, r, token); late.Total != 1 {
|
||||
t.Fatal("an overdue item must be due")
|
||||
}
|
||||
|
||||
// Another account saves the same word in its own chapter without touching this queue.
|
||||
otherDogs := saveWord(t, r, otherToken, otherChapters[0].ID, 0, 4, termStatusNew, nil)
|
||||
if otherDogs.Term.ID == dogs.Term.ID {
|
||||
t.Fatal("two accounts must own separate records for the same word")
|
||||
}
|
||||
code, mine := reviewQueue(t, r, token)
|
||||
if code != 200 || len(mine.Items) != 1 || mine.Items[0].ID != dogs.Term.ID {
|
||||
t.Fatalf("queues must not mix accounts: %d %#v", code, mine)
|
||||
}
|
||||
code, theirs := reviewQueue(t, r, otherToken)
|
||||
if code != 200 || len(theirs.Items) != 1 || theirs.Items[0].ID != otherDogs.Term.ID {
|
||||
t.Fatalf("the other account must see its own queue: %d %#v", code, theirs)
|
||||
}
|
||||
|
||||
// Known and ignored words never enter the queue; returning to new makes it due again.
|
||||
for _, status := range []string{termStatusKnown, termStatusIgnored} {
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": status}); code != 200 {
|
||||
t.Fatalf("marking %s failed", status)
|
||||
}
|
||||
if _, scoped := reviewQueue(t, r, token); scoped.Total != 0 {
|
||||
t.Fatalf("%s must stay out of the queue: %#v", status, scoped)
|
||||
}
|
||||
}
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 200 {
|
||||
t.Fatal("returning to a new word failed")
|
||||
}
|
||||
if _, again := reviewQueue(t, r, token); again.Total != 1 {
|
||||
t.Fatalf("a new word must be due again: %#v", again)
|
||||
}
|
||||
|
||||
if code, _ := callAPI(t, r, "GET", "/api/v1/reviews/queue?status=new", token, nil); code != 400 {
|
||||
t.Fatal("the queue must reject query parameters")
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", "/api/v1/reviews/queue", "", nil); code != 401 {
|
||||
t.Fatal("the queue requires a session")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewAnswerTransitions answers one word three times and checks the term, the
|
||||
// counters and the recorded attempt after each grade.
|
||||
func TestMySQLReviewAnswerTransitions(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, user, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
termID := dogs.Term.ID
|
||||
|
||||
// Correct: a new word becomes learning level 1 and leaves the round.
|
||||
_, queue := reviewQueue(t, r, token)
|
||||
code, first := answerReview(t, r, token, termID, answerBody("answer-correct-0001", reviewGradeCorrect, queue.Items[0].DueAt))
|
||||
if code != 201 || first.Result != "applied" || first.Requeued {
|
||||
t.Fatalf("correct answer: %d %#v", code, first)
|
||||
}
|
||||
if first.StatusAfter != termStatusLearning || first.LevelAfter != 1 || first.Item.Level != 1 || first.Item.ReviewCount != 1 {
|
||||
t.Fatalf("correct transition: %#v", first)
|
||||
}
|
||||
if !first.DueAtAfter.Equal(clock.AddDate(0, 0, 1)) || !first.DueAtBefore.Equal(*clock) {
|
||||
t.Fatalf("correct schedule: %#v", first)
|
||||
}
|
||||
if _, empty := reviewQueue(t, r, token); empty.Total != 0 {
|
||||
t.Fatal("a correctly answered word must leave the queue until it is due again")
|
||||
}
|
||||
// One interval later the word is due again; a wrong answer requeues it immediately.
|
||||
// Sessions last eight hours of the injected clock, so the jump forward needs a login.
|
||||
*clock = first.DueAtAfter
|
||||
token = loginToken(t, r, user.Username, fixturePassword)
|
||||
_, queue = reviewQueue(t, r, token)
|
||||
if queue.Total != 1 || queue.Items[0].Level != 1 || queue.Items[0].ReviewCount != 1 {
|
||||
t.Fatalf("second round queue: %#v", queue)
|
||||
}
|
||||
code, wrong := answerReview(t, r, token, termID, answerBody("answer-wrong-000002", reviewGradeWrong, queue.Items[0].DueAt))
|
||||
if code != 201 || !wrong.Requeued || wrong.LevelAfter != 1 || wrong.StatusAfter != termStatusLearning || !wrong.DueAtAfter.Equal(*clock) {
|
||||
t.Fatalf("wrong answer: %d %#v", code, wrong)
|
||||
}
|
||||
_, queue = reviewQueue(t, r, token)
|
||||
if queue.Total != 1 || queue.Items[0].Level != 1 {
|
||||
t.Fatalf("requeued item: %#v", queue)
|
||||
}
|
||||
code, again := answerReview(t, r, token, termID, answerBody("answer-again-000003", reviewGradeAgain, queue.Items[0].DueAt))
|
||||
if code != 201 || !again.Requeued || again.LevelBefore != 1 || again.LevelAfter != 1 {
|
||||
t.Fatalf("again answer: %d %#v", code, again)
|
||||
}
|
||||
|
||||
// Counters and the attempt log accumulated exactly three answers.
|
||||
review := termReviewRow(t, db, termID)
|
||||
if review.ReviewCount != 3 || review.CorrectCount != 1 || review.WrongCount != 2 || review.LastReviewedAt == nil {
|
||||
t.Fatalf("counters: %#v", review)
|
||||
}
|
||||
var term Term
|
||||
if err := db.First(&term, termID).Error; err != nil || term.Status != termStatusLearning || term.Level != 1 || term.Definition != "虚构释义" {
|
||||
t.Fatalf("the learner's own text must not change: %#v %v", term, err)
|
||||
}
|
||||
var answers int64
|
||||
if err := db.Model(&ReviewAnswer{}).Where("owner_id = ?", user.UserId).Count(&answers).Error; err != nil || answers != 3 {
|
||||
t.Fatalf("attempt log: %d %v", answers, err)
|
||||
}
|
||||
var logged ReviewAnswer
|
||||
if err := db.Where("term_id = ? AND grade = ?", termID, reviewGradeCorrect).First(&logged).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if logged.Result != "applied" || logged.StatusBefore != termStatusNew || logged.LevelAfter != 1 || logged.Requeued {
|
||||
t.Fatalf("logged attempt: %#v", logged)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewAnswerIdempotencyAndStaleTabs is the core of the acceptance: a resend
|
||||
// must not count twice, and a second tab must not advance a term that already moved.
|
||||
func TestMySQLReviewAnswerIdempotencyAndStaleTabs(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
termID := dogs.Term.ID
|
||||
_, queue := reviewQueue(t, r, token)
|
||||
seen := queue.Items[0].DueAt
|
||||
|
||||
// Two tabs answer the same word with different answer ids but the same seen due time.
|
||||
code, applied := answerReview(t, r, token, termID, answerBody("tab-one-answer-0001", reviewGradeCorrect, seen))
|
||||
if code != 201 || applied.Result != "applied" {
|
||||
t.Fatalf("first tab: %d %#v", code, applied)
|
||||
}
|
||||
code, stale := answerReview(t, r, token, termID, answerBody("tab-two-answer-0002", reviewGradeCorrect, seen))
|
||||
if code != 200 || stale.Result != "stale" || stale.Duplicate || stale.Requeued {
|
||||
t.Fatalf("second tab must not advance the term: %d %#v", code, stale)
|
||||
}
|
||||
if stale.LevelAfter != applied.LevelAfter || !stale.DueAtAfter.Equal(applied.DueAtAfter) {
|
||||
t.Fatalf("a stale answer changed state: %#v", stale)
|
||||
}
|
||||
|
||||
// A resend of the first answer returns the first outcome, flagged as a replay, without
|
||||
// another update. The client can therefore count it exactly like the original answer.
|
||||
code, duplicate := answerReview(t, r, token, termID, answerBody("tab-one-answer-0001", reviewGradeCorrect, seen))
|
||||
if code != 200 || duplicate.Result != "applied" || !duplicate.Duplicate {
|
||||
t.Fatalf("resend: %d %#v", code, duplicate)
|
||||
}
|
||||
if duplicate.LevelAfter != applied.LevelAfter || !duplicate.DueAtAfter.Equal(applied.DueAtAfter) || duplicate.Item.ReviewCount != 1 {
|
||||
t.Fatalf("a resend changed state: %#v", duplicate)
|
||||
}
|
||||
// Counters moved exactly once, and the replay is not a new attempt row: the applied
|
||||
// answer and the stale one are two rows, the duplicate returns the first row.
|
||||
review := termReviewRow(t, db, termID)
|
||||
if review.ReviewCount != 1 || review.CorrectCount != 1 || review.WrongCount != 0 {
|
||||
t.Fatalf("counters after three attempts: %#v", review)
|
||||
}
|
||||
var attempts int64
|
||||
if err := db.Model(&ReviewAnswer{}).Where("term_id = ?", termID).Count(&attempts).Error; err != nil || attempts != 2 {
|
||||
t.Fatalf("each distinct attempt must be recorded once: %d %v", attempts, err)
|
||||
}
|
||||
|
||||
// Reusing an answer id for another word is a client error, not a new answer.
|
||||
went := saveWord(t, r, token, chapter.ID, 5, 9, termStatusNew, nil)
|
||||
code, _ = answerReview(t, r, token, went.Term.ID, answerBody("tab-one-answer-0001", reviewGradeCorrect, *clock))
|
||||
if code != 400 {
|
||||
t.Fatalf("a reused answer id must be rejected: %d", code)
|
||||
}
|
||||
// A genuinely new answer for the other word still works.
|
||||
code, fresh := answerReview(t, r, token, went.Term.ID, answerBody("went-answer-000004", reviewGradeCorrect, *clock))
|
||||
if code != 201 || fresh.LevelAfter != 1 {
|
||||
t.Fatalf("a later answer must work: %d %#v", code, fresh)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewAnswerErrorsAndOwnership covers the rejected shapes of the answer
|
||||
// endpoint.
|
||||
func TestMySQLReviewAnswerErrorsAndOwnership(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
_, otherToken, _ := termFixture(t, db, r, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
termID := dogs.Term.ID
|
||||
seen := *clock
|
||||
|
||||
if code, _ := answerReview(t, r, otherToken, termID, answerBody("foreign-answer-0001", reviewGradeCorrect, seen)); code != 404 {
|
||||
t.Fatal("another account's term must be 404")
|
||||
}
|
||||
if code, _ := answerReview(t, r, token, 999999, answerBody("missing-answer-0002", reviewGradeCorrect, seen)); code != 404 {
|
||||
t.Fatal("an unknown term must be 404")
|
||||
}
|
||||
for _, id := range []string{"0", "-3", "abc"} {
|
||||
if code, _ := callAPI(t, r, "POST", "/api/v1/reviews/"+id+"/answers", token, answerBody("bad-id-000003", reviewGradeCorrect, seen)); code != 404 {
|
||||
t.Fatalf("invalid term id %s: %d", id, code)
|
||||
}
|
||||
}
|
||||
for _, body := range []map[string]any{
|
||||
{"answerId": "grade-missing-0001", "expectedDueAt": seen.Format(time.RFC3339Nano)},
|
||||
{"answerId": "grade-unknown-0002", "grade": "maybe", "expectedDueAt": seen.Format(time.RFC3339Nano)},
|
||||
answerBody("short", reviewGradeCorrect, seen),
|
||||
{"answerId": "due-missing-0003", "grade": reviewGradeCorrect},
|
||||
{"answerId": "extra-field-0004", "grade": reviewGradeCorrect, "expectedDueAt": seen.Format(time.RFC3339Nano), "level": 7},
|
||||
} {
|
||||
if code, _ := answerReview(t, r, token, termID, body); code != 400 {
|
||||
t.Fatalf("invalid body %v must be 400: %d", body, code)
|
||||
}
|
||||
}
|
||||
if code, _ := callAPI(t, r, "POST", fmt.Sprintf("/api/v1/reviews/%d/answers", termID), "", answerBody("anonymous-000001", reviewGradeCorrect, seen)); code != 401 {
|
||||
t.Fatal("answering requires a session")
|
||||
}
|
||||
// A word marked as known in another screen is no longer reviewable.
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusKnown}); code != 200 {
|
||||
t.Fatal("marking the word known failed")
|
||||
}
|
||||
code, result := answerReview(t, r, token, termID, answerBody("known-answer-0005", reviewGradeCorrect, seen))
|
||||
if code != 409 || result.Result != "" {
|
||||
t.Fatalf("a known word must refuse an answer: %d %#v", code, result)
|
||||
}
|
||||
if review := termReviewRow(t, db, termID); review.ReviewCount != 0 {
|
||||
t.Fatalf("a refused answer must not count: %#v", review)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewScheduleOnManualStatus checks the documented rule for words saved with
|
||||
// an explicit status: new is due at once, learning waits for its level's interval.
|
||||
func TestMySQLReviewScheduleOnManualStatus(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
level := 3
|
||||
learning := saveWord(t, r, token, chapter.ID, 0, 4, termStatusLearning, &level)
|
||||
if review := termReviewRow(t, db, learning.Term.ID); !review.DueAt.Equal(clock.AddDate(0, 0, 4)) {
|
||||
t.Fatalf("a manual level must wait for its interval: %v", review.DueAt)
|
||||
}
|
||||
if _, queue := reviewQueue(t, r, token); queue.Total != 0 {
|
||||
t.Fatalf("a manually leveled word is not due today: %#v", queue)
|
||||
}
|
||||
// Changing it back to a new word makes it due immediately again.
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 200 {
|
||||
t.Fatal("resetting the status failed")
|
||||
}
|
||||
if _, queue := reviewQueue(t, r, token); queue.Total != 1 || queue.Items[0].Level != 0 {
|
||||
t.Fatalf("a new word is due at once: %#v", queue)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewConcurrentReplayOfOneAnswer reproduces the review finding that two
|
||||
// simultaneous submissions of one answer id must both get the recorded outcome.
|
||||
func TestMySQLReviewConcurrentReplayOfOneAnswer(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
dogs := saveWord(t, r, token, chapter.ID, 0, 4, termStatusNew, nil)
|
||||
_, queue := reviewQueue(t, r, token)
|
||||
seen := queue.Items[0].DueAt
|
||||
body := answerBody("concurrent-answer-01", reviewGradeCorrect, seen)
|
||||
|
||||
var wg sync.WaitGroup
|
||||
codes := make(chan int, 2)
|
||||
results := make(chan string, 2)
|
||||
for i := 0; i < 2; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
code, data := callAPI(t, r, "POST", fmt.Sprintf("/api/v1/reviews/%d/answers", dogs.Term.ID), token, body)
|
||||
var result ReviewAnswerResult
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &result)
|
||||
}
|
||||
codes <- code
|
||||
results <- result.Result
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(codes)
|
||||
close(results)
|
||||
ok, bad := 0, 0
|
||||
for code := range codes {
|
||||
if code == 200 || code == 201 {
|
||||
ok++
|
||||
} else {
|
||||
bad++
|
||||
t.Logf("status %d", code)
|
||||
}
|
||||
}
|
||||
kinds := []string{}
|
||||
for kind := range results {
|
||||
kinds = append(kinds, kind)
|
||||
}
|
||||
if ok != 2 || bad != 0 {
|
||||
t.Fatalf("both submissions must succeed: ok=%d bad=%d kinds=%v clock=%v", ok, bad, kinds, *clock)
|
||||
}
|
||||
review := termReviewRow(t, db, dogs.Term.ID)
|
||||
if review.ReviewCount != 1 {
|
||||
t.Fatalf("concurrent replay counted %d times", review.ReviewCount)
|
||||
}
|
||||
var attempts int64
|
||||
if err := db.Model(&ReviewAnswer{}).Where("term_id = ?", dogs.Term.ID).Count(&attempts).Error; err != nil || attempts != 1 {
|
||||
t.Fatalf("concurrent replay logged %d attempts", attempts)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewEditKeepsSchedule reproduces the review finding that editing only the
|
||||
// learner's own text must not move a review date (decision D2).
|
||||
func TestMySQLReviewEditKeepsSchedule(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
level := 3
|
||||
learning := saveWord(t, r, token, chapter.ID, 0, 4, termStatusLearning, &level)
|
||||
// Make the word overdue, as it would be after a missed day.
|
||||
overdue := clock.AddDate(0, 0, -5)
|
||||
if err := db.Model(&TermReview{}).Where("term_id = ?", learning.Term.ID).Update("due_at", overdue).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Editing only the personal text must keep the schedule exactly as it was.
|
||||
code, saved := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "改过的释义", "status": termStatusLearning, "level": 3})
|
||||
if code != 200 || saved.Term.Level != 3 {
|
||||
t.Fatalf("text edit: %d %#v", code, saved)
|
||||
}
|
||||
if review := termReviewRow(t, db, learning.Term.ID); !review.DueAt.Equal(overdue) {
|
||||
t.Fatalf("editing the text moved the due time: %v want %v", review.DueAt, overdue)
|
||||
}
|
||||
if _, queue := reviewQueue(t, r, token); queue.Total != 1 {
|
||||
t.Fatalf("an overdue word must stay due after a text edit: %#v", queue)
|
||||
}
|
||||
// Changing the level is a scheduling action and does move the due time.
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusLearning, "level": 5}); code != 200 {
|
||||
t.Fatal("level change failed")
|
||||
}
|
||||
if review := termReviewRow(t, db, learning.Term.ID); !review.DueAt.Equal(clock.AddDate(0, 0, 15)) {
|
||||
t.Fatalf("a level change must reschedule: %v", review.DueAt)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLReviewPanelSaveKeepsLevel reproduces the related defect found while checking the
|
||||
// review: the reader panel saves a status without a level, and that must not reset a level
|
||||
// the learner already earned.
|
||||
func TestMySQLReviewPanelSaveKeepsLevel(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r, clock, _, token, chapter := reviewFixture(t, db, "Dogs went home.")
|
||||
level := 4
|
||||
learning := saveWord(t, r, token, chapter.ID, 0, 4, termStatusLearning, &level)
|
||||
overdue := clock.AddDate(0, 0, -2)
|
||||
if err := db.Model(&TermReview{}).Where("term_id = ?", learning.Term.ID).Update("due_at", overdue).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// This is exactly what the reader panel sends: status only, no level.
|
||||
code, saved := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "改过的释义", "status": termStatusLearning})
|
||||
if code != 200 || saved.Term.Level != 4 {
|
||||
t.Fatalf("a panel save must keep the learned level: %d %#v", code, saved)
|
||||
}
|
||||
if review := termReviewRow(t, db, learning.Term.ID); !review.DueAt.Equal(overdue) {
|
||||
t.Fatalf("a panel save moved the due time: %v want %v", review.DueAt, overdue)
|
||||
}
|
||||
// Leaving learning and re-entering it is a real transition and starts at level 1.
|
||||
if code, _ := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 200 {
|
||||
t.Fatal("reset to new failed")
|
||||
}
|
||||
code, again := saveTermAPI(t, r, token, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusLearning})
|
||||
if code != 200 || again.Term.Level != 1 {
|
||||
t.Fatalf("re-entering learning starts at 1: %d %#v", code, again)
|
||||
}
|
||||
}
|
||||
+112
-3
@@ -129,6 +129,26 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
if c.Request.Method == "POST" && c.FullPath() == "/api/v1/accounts" {
|
||||
status = 201
|
||||
}
|
||||
// A repeated paste is answered from the first result, so it is not a new resource.
|
||||
// A pasted or uploaded chapter is one resource; a repeated submit is not.
|
||||
if c.Request.Method == "POST" && (c.FullPath() == "/api/v1/books" || c.FullPath() == "/api/v1/books/:id/chapters" ||
|
||||
c.FullPath() == "/api/v1/books/upload" || c.FullPath() == "/api/v1/books/:id/chapters/upload") {
|
||||
if paste, ok := data.(PasteResult); ok && !paste.Duplicate {
|
||||
status = 201
|
||||
}
|
||||
}
|
||||
// Saving the same entry again is an update of one record, not a new resource.
|
||||
if c.Request.Method == "POST" && (c.FullPath() == "/api/v1/terms" || c.FullPath() == "/api/v1/phrases") {
|
||||
if saved, ok := data.(TermSave); ok && saved.Created {
|
||||
status = 201
|
||||
}
|
||||
}
|
||||
// A replayed or stale answer changed nothing, so it is not a new answer.
|
||||
if c.Request.Method == "POST" && c.FullPath() == "/api/v1/reviews/:termId/answers" {
|
||||
if answer, ok := data.(ReviewAnswerResult); ok && answer.Result == "applied" && !answer.Duplicate {
|
||||
status = 201
|
||||
}
|
||||
}
|
||||
respond(c, status, data, err)
|
||||
}
|
||||
}
|
||||
@@ -213,18 +233,107 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
}
|
||||
return updateAccount(tx, id, u.UserId, input)
|
||||
}))
|
||||
v.POST("/books", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
var input PasteBookInput
|
||||
if err := decodeLimit(c, &input, maxPasteBodyBytes); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return PasteBook(tx, u.UserId, now(), input)
|
||||
}))
|
||||
v.GET("/books", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if c.Request.URL.RawQuery != "" {
|
||||
return nil, failure(400, "书库不接受查询参数")
|
||||
}
|
||||
items, err := ListBooks(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"items": items}, nil
|
||||
}))
|
||||
v.GET("/books/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
book, chapters, err := BookDetail(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"book": book, "chapters": chapters}, nil
|
||||
}))
|
||||
v.POST("/books/:id/chapters", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var input PasteChapterInput
|
||||
if err := decodeLimit(c, &input, maxPasteBodyBytes); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return PasteChapter(tx, u.UserId, id, now(), input)
|
||||
}))
|
||||
v.GET("/chapters/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "章节不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ChapterDetail(tx, u.UserId, id)
|
||||
}))
|
||||
v.GET("/jobs/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "任务不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
job, err := JobDetail(tx, u.UserId, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"job": job}, nil
|
||||
}))
|
||||
v.POST("/jobs/:id/retry", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := pathID(c, "任务不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
job, chapter, err := RetryIngestJob(tx, u.UserId, id, now())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"job": job, "chapter": chapter}, nil
|
||||
}))
|
||||
registerDictionaryRoutes(v, protect, now)
|
||||
registerTermRoutes(v, protect, now)
|
||||
registerReviewRoutes(v, protect, now)
|
||||
registerUploadRoutes(v, protect, now)
|
||||
registerEditRoutes(v, protect, now)
|
||||
r.NoRoute(func(c *gin.Context) { respond(c, 404, nil, failure(404, "页面或接口不存在")) })
|
||||
return r
|
||||
}
|
||||
|
||||
func decode(c *gin.Context, value any) error {
|
||||
// pathID reads a positive numeric path parameter; a malformed id is reported like a
|
||||
// missing resource so it cannot be used to probe for other accounts' rows.
|
||||
func pathID(c *gin.Context, message string) (int64, error) {
|
||||
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
return 0, failure(404, message)
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
func decode(c *gin.Context, value any) error { return decodeLimit(c, value, maxJSONBodyBytes) }
|
||||
|
||||
func decodeLimit(c *gin.Context, value any, limit int64) error {
|
||||
if !strings.HasPrefix(c.GetHeader("Content-Type"), "application/json") {
|
||||
return failure(400, "请使用 JSON 请求")
|
||||
}
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, 16*1024)
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, limit)
|
||||
d := json.NewDecoder(c.Request.Body)
|
||||
d.DisallowUnknownFields()
|
||||
if d.Decode(value) != nil {
|
||||
if err := d.Decode(value); err != nil {
|
||||
var tooLarge *http.MaxBytesError
|
||||
if errors.As(err, &tooLarge) {
|
||||
return failure(400, "内容过大，请减少后重试")
|
||||
}
|
||||
return failure(400, "请求内容无效")
|
||||
}
|
||||
if d.Decode(new(any)) != io.EOF {
|
||||
|
||||
@@ -0,0 +1,408 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// The four learner-visible statuses are stored explicitly instead of the upstream
|
||||
// merged status/level code. The mapping stays available for CSV export and for
|
||||
// migrating an existing LinguaCafe instance: new=2, ignored=1, known=0, and
|
||||
// learning level N=-N. A level is only meaningful while learning, so every other
|
||||
// status keeps level 0 and the database check constraints repeat that rule.
|
||||
const (
|
||||
termStatusNew = "new"
|
||||
termStatusLearning = "learning"
|
||||
termStatusKnown = "known"
|
||||
termStatusIgnored = "ignored"
|
||||
)
|
||||
|
||||
// One table holds both entries: a word and a phrase differ in kind and word count, and a
|
||||
// phrase key always contains a space, so the identity keys cannot collide.
|
||||
const (
|
||||
termKindWord = "word"
|
||||
termKindPhrase = "phrase"
|
||||
)
|
||||
|
||||
const (
|
||||
termFormLimit = 128
|
||||
termDefinitionLimit = 2000
|
||||
termExampleLimit = 500
|
||||
termExamplesLimit = 5
|
||||
termLevelMax = 7
|
||||
// One chapter may contain many distinct words, so identity lookups are batched
|
||||
// instead of sending an unbounded IN list.
|
||||
termLookupBatch = 500
|
||||
)
|
||||
|
||||
var termStatuses = map[string]bool{
|
||||
termStatusNew: true,
|
||||
termStatusLearning: true,
|
||||
termStatusKnown: true,
|
||||
termStatusIgnored: true,
|
||||
}
|
||||
|
||||
// Term is one learner's own record for a word form. Identity is the normalized
|
||||
// form: the original spelling is kept for display only.
|
||||
type Term struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
OwnerID int
|
||||
Language string
|
||||
Term string
|
||||
OriginalForm string `gorm:"column:original_form"`
|
||||
Definition string
|
||||
Examples string
|
||||
Status string
|
||||
Level int
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// Kind and word count need no columns: a phrase key is the ordered normalized word forms
|
||||
// joined by single spaces, and a word form can never contain a space, so the identity key
|
||||
// itself carries both facts. One helper keeps that invariant in a single place.
|
||||
func termKind(key string) string {
|
||||
if strings.Contains(key, " ") {
|
||||
return termKindPhrase
|
||||
}
|
||||
return termKindWord
|
||||
}
|
||||
|
||||
func termWordCount(key string) int {
|
||||
if !strings.Contains(key, " ") {
|
||||
return 1
|
||||
}
|
||||
return len(strings.Split(key, " "))
|
||||
}
|
||||
|
||||
func (Term) TableName() string { return "lexgo_terms" }
|
||||
|
||||
type TermView struct {
|
||||
ID int64 `json:"id"`
|
||||
Language string `json:"language"`
|
||||
Term string `json:"term"`
|
||||
OriginalForm string `json:"originalForm"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
Status string `json:"status"`
|
||||
Level int `json:"level"`
|
||||
Kind string `json:"kind"`
|
||||
WordCount int `json:"wordCount"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
// TokenTerm is what a reader token needs to render its own highlight: the entry
|
||||
// id for the follow-up read, and enough state to style the word.
|
||||
type TokenTerm struct {
|
||||
ID int64 `json:"id"`
|
||||
Status string `json:"status"`
|
||||
Level int `json:"level"`
|
||||
}
|
||||
|
||||
type TermSave struct {
|
||||
Term TermView `json:"term"`
|
||||
Created bool `json:"created"`
|
||||
}
|
||||
|
||||
// splitExamples returns the stored examples as a list; the column holds one per line.
|
||||
func splitExamples(text string) []string {
|
||||
if text == "" {
|
||||
return []string{}
|
||||
}
|
||||
return strings.Split(text, "\n")
|
||||
}
|
||||
|
||||
func termView(t Term) TermView {
|
||||
return TermView{
|
||||
ID: t.ID, Language: t.Language, Term: t.Term, OriginalForm: t.OriginalForm, Definition: t.Definition,
|
||||
Examples: splitExamples(t.Examples), Status: t.Status, Level: t.Level,
|
||||
Kind: termKind(t.Term), WordCount: termWordCount(t.Term),
|
||||
CreatedAt: t.CreatedAt, UpdatedAt: t.UpdatedAt,
|
||||
}
|
||||
}
|
||||
|
||||
// termLevel enforces the documented status/level boundary: only a learning entry carries a
|
||||
// level, every other status must leave the level at 0, and entering learning starts at 1.
|
||||
// A save that does not mention a level keeps the level the learner already earned, so
|
||||
// editing a definition can never roll a word back to level 1.
|
||||
func termLevel(status string, level *int, previous Term, exists bool) (int, error) {
|
||||
if !termStatuses[status] {
|
||||
return 0, failure(400, "词语状态无效")
|
||||
}
|
||||
if status == termStatusLearning {
|
||||
if level != nil && *level != 0 {
|
||||
if *level < 1 || *level > termLevelMax {
|
||||
return 0, failure(400, "学习等级须为 1～7")
|
||||
}
|
||||
return *level, nil
|
||||
}
|
||||
if exists && previous.Status == termStatusLearning && previous.Level >= 1 {
|
||||
return previous.Level, nil
|
||||
}
|
||||
return 1, nil
|
||||
}
|
||||
if level != nil && *level != 0 {
|
||||
return 0, failure(400, "只有学习中的词语可以设置等级")
|
||||
}
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
func hasControlRune(text string, allowNewline bool) bool {
|
||||
for _, r := range text {
|
||||
if allowNewline && (r == '\n' || r == '\t') {
|
||||
continue
|
||||
}
|
||||
if unicode.IsControl(r) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// termContent validates the learner's own text and returns it in storage form:
|
||||
// the definition as typed (trimmed) and examples joined by newline.
|
||||
func termContent(definition string, examples []string) (string, string, error) {
|
||||
definition = strings.TrimSpace(definition)
|
||||
if utf8.RuneCountInString(definition) > termDefinitionLimit {
|
||||
return "", "", failure(400, "个人释义不能超过 2000 个字符")
|
||||
}
|
||||
if hasControlRune(definition, true) {
|
||||
return "", "", failure(400, "个人释义包含不支持的字符")
|
||||
}
|
||||
if len(examples) > termExamplesLimit {
|
||||
return "", "", failure(400, "例句不能超过 5 条")
|
||||
}
|
||||
cleaned := make([]string, 0, len(examples))
|
||||
for _, example := range examples {
|
||||
example = strings.TrimSpace(example)
|
||||
if example == "" {
|
||||
return "", "", failure(400, "例句不能为空行")
|
||||
}
|
||||
if utf8.RuneCountInString(example) > termExampleLimit {
|
||||
return "", "", failure(400, "每条例句不能超过 500 个字符")
|
||||
}
|
||||
if hasControlRune(example, false) {
|
||||
return "", "", failure(400, "例句包含不支持的字符")
|
||||
}
|
||||
cleaned = append(cleaned, example)
|
||||
}
|
||||
return definition, strings.Join(cleaned, "\n"), nil
|
||||
}
|
||||
|
||||
// wordAtRange returns the word the learner actually selected. Identity is always
|
||||
// derived from the server's own tokens, so a client cannot name a word, user or
|
||||
// language that it did not read from this chapter.
|
||||
func wordAtRange(chapter Chapter, start, end int) (string, error) {
|
||||
for _, token := range Tokenize(chapter.OriginalText) {
|
||||
if token.Kind == "word" && token.Start == start && token.End == end {
|
||||
if utf8.RuneCountInString(token.Text) <= termFormLimit {
|
||||
return token.Text, nil
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
return "", failure(400, "请选择不超过 128 个字符的完整单词")
|
||||
}
|
||||
|
||||
// languageOf reads the owner's study language. A missing space row falls back to
|
||||
// the English default that account creation and login already establish.
|
||||
func languageOf(tx *gorm.DB, owner int) (string, error) {
|
||||
var space Space
|
||||
err := tx.Where("owner_id = ?", owner).First(&space).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return "en", nil
|
||||
}
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return space.Language, nil
|
||||
}
|
||||
|
||||
// previousTerm reads the row this save is about to change under a lock, so the level and
|
||||
// the review schedule can be compared with what the learner already had.
|
||||
func previousTerm(tx *gorm.DB, owner int, language, term string) (Term, bool, error) {
|
||||
var existing Term
|
||||
err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).
|
||||
Where("owner_id = ? AND language = ? AND term = ?", owner, language, term).First(&existing).Error
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return Term{}, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return Term{}, false, err
|
||||
}
|
||||
return existing, true, nil
|
||||
}
|
||||
|
||||
// saveTerm writes one identity with INSERT ... ON DUPLICATE KEY UPDATE: a repeated
|
||||
// save updates the same row instead of adding a second, conflicting record, and
|
||||
// two concurrent saves of the same word still leave exactly one.
|
||||
// saveTerm stores one identity: key is the canonical form used for lookup, display is what the
|
||||
// learner sees. For a word both are the word form; for a phrase the key joins normalized word
|
||||
// forms and the display keeps the original punctuation.
|
||||
func saveTerm(tx *gorm.DB, owner int, language, key, display string, fields TermFields, now time.Time) (TermSave, error) {
|
||||
when := stamp(now)
|
||||
row := Term{
|
||||
OwnerID: owner, Language: language, Term: key, OriginalForm: display,
|
||||
Definition: fields.Definition, Examples: fields.Examples, Status: fields.Status, Level: fields.Level,
|
||||
CreatedAt: when, UpdatedAt: when,
|
||||
}
|
||||
// MySQL reports affected rows 1 for an insert and 0 or 2 for an update, so the
|
||||
// counter distinguishes "created" from "saved again" without a second read.
|
||||
insert := tx.Clauses(clause.OnConflict{
|
||||
Columns: []clause.Column{{Name: "owner_id"}, {Name: "language"}, {Name: "term"}},
|
||||
DoUpdates: clause.Assignments(map[string]any{
|
||||
"original_form": row.OriginalForm, "definition": row.Definition, "examples": row.Examples,
|
||||
"status": row.Status, "level": row.Level, "updated_at": row.UpdatedAt,
|
||||
}),
|
||||
}).Create(&row)
|
||||
if insert.Error != nil {
|
||||
return TermSave{}, insert.Error
|
||||
}
|
||||
var stored Term
|
||||
if err := tx.Where("owner_id = ? AND language = ? AND term = ?", owner, language, row.Term).First(&stored).Error; err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
// A saved word always owns a schedule row, but the date only moves for a new word or a
|
||||
// real status/level change: editing a definition must not push a word out of today's
|
||||
// queue (decision D2).
|
||||
reschedule := !fields.Exists || fields.PreviousStatus != stored.Status || fields.PreviousLevel != stored.Level
|
||||
if err := syncTermReview(tx, stored, reschedule, now); err != nil {
|
||||
return TermSave{}, err
|
||||
}
|
||||
return TermSave{termView(stored), insert.RowsAffected == 1}, nil
|
||||
}
|
||||
|
||||
// attachTerms marks the tokens this learner already saved. Matching is by
|
||||
// normalized form across the whole vocabulary, so a word saved in another chapter
|
||||
// is highlighted here with the same status.
|
||||
func attachTerms(tx *gorm.DB, owner int, language string, tokens []TextToken) error {
|
||||
keys := make([]string, 0, 16)
|
||||
seen := map[string]bool{}
|
||||
for _, token := range tokens {
|
||||
if token.Kind != "word" || utf8.RuneCountInString(token.Text) > termFormLimit {
|
||||
continue
|
||||
}
|
||||
key := normalizeWord(token.Text)
|
||||
if !seen[key] {
|
||||
seen[key] = true
|
||||
keys = append(keys, key)
|
||||
}
|
||||
}
|
||||
byKey := map[string]TokenTerm{}
|
||||
for start := 0; start < len(keys); start += termLookupBatch {
|
||||
end := min(start+termLookupBatch, len(keys))
|
||||
var rows []Term
|
||||
if err := tx.Select("id", "term", "status", "level").
|
||||
Where("owner_id = ? AND language = ? AND term IN ?", owner, language, keys[start:end]).
|
||||
Find(&rows).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
for _, row := range rows {
|
||||
byKey[row.Term] = TokenTerm{ID: row.ID, Status: row.Status, Level: row.Level}
|
||||
}
|
||||
}
|
||||
for i := range tokens {
|
||||
if tokens[i].Kind != "word" {
|
||||
continue
|
||||
}
|
||||
if term, ok := byKey[normalizeWord(tokens[i].Text)]; ok {
|
||||
value := term
|
||||
tokens[i].Term = &value
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// TermFields carries the validated learner text and state into storage, together with the
|
||||
// status and level the row had before this save.
|
||||
type TermFields struct {
|
||||
Definition string
|
||||
Examples string
|
||||
Status string
|
||||
Level int
|
||||
PreviousStatus string
|
||||
PreviousLevel int
|
||||
Exists bool
|
||||
}
|
||||
|
||||
type TermInput struct {
|
||||
ChapterID int64 `json:"chapterId"`
|
||||
Start *int `json:"start"`
|
||||
End *int `json:"end"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
Status string `json:"status"`
|
||||
Level *int `json:"level"`
|
||||
}
|
||||
|
||||
// registerTermRoutes exposes the learner's own word records. Nothing here is
|
||||
// written to the audit log: personal learning content stays out of it, and the
|
||||
// account-level audit already covers administrative changes.
|
||||
func registerTermRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
v.POST("/terms", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
var input TermInput
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if input.Start == nil || input.End == nil {
|
||||
return nil, failure(400, "请选择完整单词")
|
||||
}
|
||||
chapter, err := readyOwnedChapter(tx, u.UserId, input.ChapterID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
word, err := wordAtRange(chapter, *input.Start, *input.End)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
language, err := languageOf(tx, u.UserId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// The previous row decides whether a missing level keeps the earned one and whether
|
||||
// the review date may move at all.
|
||||
previous, exists, err := previousTerm(tx, u.UserId, language, normalizeWord(word))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
level, err := termLevel(input.Status, input.Level, previous, exists)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
definition, examples, err := termContent(input.Definition, input.Examples)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
fields := TermFields{
|
||||
Definition: definition, Examples: examples, Status: input.Status, Level: level,
|
||||
PreviousStatus: previous.Status, PreviousLevel: previous.Level, Exists: exists,
|
||||
}
|
||||
return saveTerm(tx, u.UserId, language, normalizeWord(word), word, fields, now())
|
||||
}))
|
||||
v.GET("/terms/:id", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
id, err := strconv.ParseInt(c.Param("id"), 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
return nil, failure(404, "词条不存在")
|
||||
}
|
||||
// Another account's id and a missing id answer identically, so the response
|
||||
// never confirms that someone else's entry exists.
|
||||
var term Term
|
||||
if err := tx.Where("id = ? AND owner_id = ?", id, u.UserId).First(&term).Error; errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil, failure(404, "词条不存在")
|
||||
} else if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gin.H{"term": termView(term)}, nil
|
||||
}))
|
||||
registerPhraseRoutes(v, protect, now)
|
||||
}
|
||||
@@ -0,0 +1,378 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func TestTermLevelBoundary(t *testing.T) {
|
||||
level := func(v int) *int { return &v }
|
||||
// The stored state the save is about to change: a level-4 learning word.
|
||||
stored := Term{Status: termStatusLearning, Level: 4}
|
||||
cases := []struct {
|
||||
status string
|
||||
level *int
|
||||
previous Term
|
||||
exists bool
|
||||
want int
|
||||
ok bool
|
||||
}{
|
||||
{termStatusNew, nil, Term{}, false, 0, true},
|
||||
{termStatusNew, level(0), Term{}, false, 0, true},
|
||||
{termStatusNew, level(1), Term{}, false, 0, false},
|
||||
{termStatusKnown, level(0), stored, true, 0, true},
|
||||
{termStatusKnown, level(3), stored, true, 0, false},
|
||||
{termStatusIgnored, level(0), stored, true, 0, true},
|
||||
{termStatusIgnored, level(-1), stored, true, 0, false},
|
||||
{termStatusLearning, nil, Term{}, false, 1, true},
|
||||
{termStatusLearning, level(0), Term{}, false, 1, true},
|
||||
{termStatusLearning, level(1), stored, true, 1, true},
|
||||
{termStatusLearning, level(7), stored, true, 7, true},
|
||||
{termStatusLearning, level(8), stored, true, 0, false},
|
||||
{termStatusLearning, level(-1), stored, true, 0, false},
|
||||
{"", nil, stored, true, 0, false},
|
||||
{"Learning", nil, stored, true, 0, false},
|
||||
{"deleted", nil, stored, true, 0, false},
|
||||
}
|
||||
// A save that does not mention a level keeps the earned one instead of resetting it.
|
||||
cases = append(cases, struct {
|
||||
status string
|
||||
level *int
|
||||
previous Term
|
||||
exists bool
|
||||
want int
|
||||
ok bool
|
||||
}{termStatusLearning, nil, stored, true, 4, true})
|
||||
for _, tc := range cases {
|
||||
got, err := termLevel(tc.status, tc.level, tc.previous, tc.exists)
|
||||
if tc.ok && (err != nil || got != tc.want) {
|
||||
t.Fatalf("%s/%v: got %d, %v; want %d", tc.status, tc.level, got, err, tc.want)
|
||||
}
|
||||
if !tc.ok && err == nil {
|
||||
t.Fatalf("%s/%v: invalid state accepted as %d", tc.status, tc.level, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTermContentRules(t *testing.T) {
|
||||
definition, examples, err := termContent(" 好奇\t心\n求知 ", []string{" A fictional line. ", "Second line"})
|
||||
if err != nil || definition != "好奇\t心\n求知" || examples != "A fictional line.\nSecond line" {
|
||||
t.Fatalf("trim and join: %q %q %v", definition, examples, err)
|
||||
}
|
||||
if definition, examples, err = termContent(" ", nil); err != nil || definition != "" || examples != "" {
|
||||
t.Fatalf("an empty definition is allowed: %q %q %v", definition, examples, err)
|
||||
}
|
||||
rejected := []struct {
|
||||
name string
|
||||
def string
|
||||
examples []string
|
||||
}{
|
||||
{"definition too long", strings.Repeat("字", termDefinitionLimit+1), nil},
|
||||
{"definition control character", "a\u0007b", nil},
|
||||
{"empty example", "ok", []string{"fine", " "}},
|
||||
{"example with newline", "ok", []string{"two\nlines"}},
|
||||
{"example too long", "ok", []string{strings.Repeat("a", termExampleLimit+1)}},
|
||||
{"too many examples", "ok", []string{"1", "2", "3", "4", "5", "6"}},
|
||||
}
|
||||
for _, tc := range rejected {
|
||||
if _, _, err := termContent(tc.def, tc.examples); err == nil {
|
||||
t.Fatalf("%s was accepted", tc.name)
|
||||
}
|
||||
}
|
||||
if _, _, err := termContent(strings.Repeat("字", termDefinitionLimit), []string{strings.Repeat("a", termExampleLimit)}); err != nil {
|
||||
t.Fatalf("boundary values must be accepted: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWordAtRangeUsesServerTokens(t *testing.T) {
|
||||
chapter := Chapter{OriginalText: "Cats went. Dogs"}
|
||||
for _, tc := range []struct {
|
||||
start, end int
|
||||
want string
|
||||
}{
|
||||
{0, 4, "Cats"},
|
||||
{5, 9, "went"},
|
||||
{11, 15, "Dogs"},
|
||||
} {
|
||||
got, err := wordAtRange(chapter, tc.start, tc.end)
|
||||
if err != nil || got != tc.want {
|
||||
t.Fatalf("%d-%d: got %q %v, want %q", tc.start, tc.end, got, err, tc.want)
|
||||
}
|
||||
}
|
||||
for _, span := range [][2]int{{0, 3}, {0, 5}, {4, 5}, {9, 10}, {10, 11}, {0, 15}, {-1, 3}, {12, 15}, {0, 0}} {
|
||||
if _, err := wordAtRange(chapter, span[0], span[1]); err == nil {
|
||||
t.Fatalf("range %v accepted as a word", span)
|
||||
}
|
||||
}
|
||||
long := "a" + strings.Repeat("b", termFormLimit)
|
||||
if _, err := wordAtRange(Chapter{OriginalText: long}, 0, len([]rune(long))); err == nil {
|
||||
t.Fatal("over-long word accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTermViewSplitsExamples(t *testing.T) {
|
||||
base := Term{ID: 3, Language: "en", Term: "curiosity", OriginalForm: "Curiosity", Status: termStatusNew}
|
||||
if got := termView(base); len(got.Examples) != 0 || got.Examples == nil {
|
||||
t.Fatalf("empty examples must serialize as []: %#v", got.Examples)
|
||||
}
|
||||
base.Examples = "One.\nTwo."
|
||||
if got := termView(base); len(got.Examples) != 2 || got.Examples[1] != "Two." {
|
||||
t.Fatalf("examples: %#v", got.Examples)
|
||||
}
|
||||
}
|
||||
|
||||
// termFixture creates one learner with a private book holding one ready chapter per
|
||||
// text. Fictional content only; nothing here touches another account's data.
|
||||
func termFixture(t *testing.T, db *gorm.DB, r *gin.Engine, texts ...string) (admin.SysUser, string, []Chapter) {
|
||||
t.Helper()
|
||||
u := admin.SysUser{Username: randomName("term"), Password: fixturePassword, RoleId: 2, Status: "2"}
|
||||
if err := db.Create(&u).Error; err != nil {
|
||||
t.Fatal("fixture learner creation failed")
|
||||
}
|
||||
now := stamp(time.Now())
|
||||
book := Book{OwnerID: u.UserId, Title: "Fictional terms", Language: "en", CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&book).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var chapters []Chapter
|
||||
for index, text := range texts {
|
||||
chapter := Chapter{BookID: book.ID, OwnerID: u.UserId, Ordinal: index + 1, Title: fmt.Sprintf("Fictional %d", index+1),
|
||||
OriginalText: text, ContentSHA256: contentSHA(text), CharCount: len([]rune(text)), Status: statusReady, CreatedAt: now, UpdatedAt: now}
|
||||
if err := db.Create(&chapter).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
chapters = append(chapters, chapter)
|
||||
}
|
||||
return u, loginToken(t, r, u.Username, fixturePassword), chapters
|
||||
}
|
||||
|
||||
func saveTermAPI(t *testing.T, r *gin.Engine, token string, body map[string]any) (int, TermSave) {
|
||||
t.Helper()
|
||||
code, data := callAPI(t, r, "POST", "/api/v1/terms", token, body)
|
||||
var saved TermSave
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &saved)
|
||||
}
|
||||
return code, saved
|
||||
}
|
||||
|
||||
func chapterTokens(t *testing.T, r *gin.Engine, token string, chapterID int64) (int, ChapterTokens) {
|
||||
t.Helper()
|
||||
code, data := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/chapters/%d/tokens", chapterID), token, nil)
|
||||
var analyzed ChapterTokens
|
||||
if data != nil {
|
||||
json.Unmarshal(data, &analyzed)
|
||||
}
|
||||
return code, analyzed
|
||||
}
|
||||
|
||||
// wordTokens returns only the selectable words, so a test can talk about the third
|
||||
// word instead of the sixth token.
|
||||
func wordTokens(tokens []TextToken) []TextToken {
|
||||
words := make([]TextToken, 0, len(tokens))
|
||||
for _, token := range tokens {
|
||||
if token.Kind == "word" {
|
||||
words = append(words, token)
|
||||
}
|
||||
}
|
||||
return words
|
||||
}
|
||||
|
||||
// TestMySQLTermIdentityAndCrossChapterConsistency covers the acceptance items that
|
||||
// one saved word stays one record, that case is not a second word, that another
|
||||
// inflection is its own entry, and that another chapter shows the same state.
|
||||
func TestMySQLTermIdentityAndCrossChapterConsistency(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r := Router(db, time.Now)
|
||||
learner, token, chapters := termFixture(t, db, r, "Dogs dogs dog.", "Dogs elsewhere")
|
||||
chapter, elsewhere := chapters[0], chapters[1]
|
||||
|
||||
code, saved := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "狗", "examples": []string{"Fictional dogs."}, "status": termStatusNew,
|
||||
})
|
||||
if code != 201 || !saved.Created || saved.Term.ID == 0 {
|
||||
t.Fatalf("first save: %d %#v", code, saved)
|
||||
}
|
||||
if saved.Term.Term != "dogs" || saved.Term.OriginalForm != "Dogs" || saved.Term.Language != "en" || saved.Term.Level != 0 {
|
||||
t.Fatalf("stored identity: %#v", saved.Term)
|
||||
}
|
||||
first := saved.Term.ID
|
||||
|
||||
// The exact same save is idempotent: same record, same status code, no duplicate.
|
||||
code, repeated := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "狗", "examples": []string{"Fictional dogs."}, "status": termStatusNew,
|
||||
})
|
||||
if code != 200 || repeated.Created || repeated.Term.ID != first {
|
||||
t.Fatalf("identical repeat must update one record: %d %#v", code, repeated)
|
||||
}
|
||||
|
||||
// Repeating the save updates the same record instead of adding a second one.
|
||||
code, again := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "狗；犬", "examples": []string{}, "status": termStatusLearning, "level": 3,
|
||||
})
|
||||
if code != 200 || again.Created || again.Term.ID != first || again.Term.Definition != "狗；犬" || again.Term.Status != termStatusLearning || again.Term.Level != 3 {
|
||||
t.Fatalf("repeated save: %d %#v", code, again)
|
||||
}
|
||||
// The same word in another case is the same identity.
|
||||
code, lower := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 5, "end": 9, "definition": "狗", "status": termStatusKnown,
|
||||
})
|
||||
if code != 200 || lower.Created || lower.Term.ID != first || lower.Term.OriginalForm != "dogs" || lower.Term.Level != 0 {
|
||||
t.Fatalf("case folded identity: %d %#v", code, lower)
|
||||
}
|
||||
// A different inflection stays its own entry instead of merging by lemma.
|
||||
code, inflection := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapter.ID, "start": 10, "end": 13, "definition": "狗（单数）", "status": termStatusNew,
|
||||
})
|
||||
if code != 201 || inflection.Term.ID == first {
|
||||
t.Fatalf("inflection must be a separate entry: %d %#v", code, inflection)
|
||||
}
|
||||
var owned int64
|
||||
if err := db.Model(&Term{}).Where("owner_id = ?", learner.UserId).Count(&owned).Error; err != nil || owned != 2 {
|
||||
t.Fatalf("one word must stay one record: count=%d err=%v", owned, err)
|
||||
}
|
||||
|
||||
// The status saved here is the status the reader shows for this chapter.
|
||||
code, tokens := chapterTokens(t, r, token, chapter.ID)
|
||||
words := wordTokens(tokens.Tokens)
|
||||
if code != 200 || len(words) != 3 {
|
||||
t.Fatalf("tokens: %d %#v", code, tokens.Tokens)
|
||||
}
|
||||
if words[0].Term == nil || words[0].Term.ID != first || words[0].Term.Status != termStatusKnown {
|
||||
t.Fatalf("first token term: %#v", words[0])
|
||||
}
|
||||
if words[2].Term == nil || words[2].Term.ID != inflection.Term.ID {
|
||||
t.Fatalf("second entry token term: %#v", words[2])
|
||||
}
|
||||
for _, token := range tokens.Tokens {
|
||||
if token.Kind != "word" && token.Term != nil {
|
||||
t.Fatalf("spaces and punctuation must not carry a term: %#v", token)
|
||||
}
|
||||
}
|
||||
// Another chapter of the same learner shows the same entry without saving again.
|
||||
code, crossTokens := chapterTokens(t, r, token, elsewhere.ID)
|
||||
crossWords := wordTokens(crossTokens.Tokens)
|
||||
if code != 200 || len(crossWords) != 2 || crossWords[0].Term == nil || crossWords[0].Term.ID != first || crossWords[0].Term.Status != termStatusKnown {
|
||||
t.Fatalf("cross-chapter highlight: %d %#v", code, crossTokens.Tokens)
|
||||
}
|
||||
// Another account sees the same text with no personal state at all.
|
||||
_, otherToken, otherChapters := termFixture(t, db, r, "Dogs elsewhere")
|
||||
code, otherTokens := chapterTokens(t, r, otherToken, otherChapters[0].ID)
|
||||
otherWords := wordTokens(otherTokens.Tokens)
|
||||
if code != 200 || len(otherWords) != 2 || otherWords[0].Term != nil {
|
||||
t.Fatalf("another account must not see a term: %d %#v", code, otherTokens.Tokens)
|
||||
}
|
||||
if code, _ := chapterTokens(t, r, otherToken, chapter.ID); code != 404 {
|
||||
t.Fatalf("foreign chapter must be 404: %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTermIsolationAndInputRules covers ownership, tampered input and the
|
||||
// status/level boundary through the real HTTP surface.
|
||||
func TestMySQLTermIsolationAndInputRules(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r := Router(db, time.Now)
|
||||
_, ownerToken, chapters := termFixture(t, db, r, "Cats went.")
|
||||
chapter := chapters[0]
|
||||
otherUser, otherToken, _ := termFixture(t, db, r, "Cats went.")
|
||||
|
||||
code, saved := saveTermAPI(t, r, ownerToken, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "definition": "猫", "status": termStatusNew})
|
||||
if code != 201 {
|
||||
t.Fatalf("owner save: %d", code)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/terms/%d", saved.Term.ID), ownerToken, nil); code != 200 {
|
||||
t.Fatalf("owner read: %d", code)
|
||||
}
|
||||
if code, _ := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/terms/%d", saved.Term.ID), otherToken, nil); code != 404 {
|
||||
t.Fatalf("foreign read must be 404: %d", code)
|
||||
}
|
||||
for _, id := range []string{"0", "-1", "abc", "99999999999999999999"} {
|
||||
if code, _ := callAPI(t, r, "GET", "/api/v1/terms/"+id, ownerToken, nil); code != 404 {
|
||||
t.Fatalf("invalid id %s: %d", id, code)
|
||||
}
|
||||
}
|
||||
if code, _ := callAPI(t, r, "POST", "/api/v1/terms", "", map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 401 {
|
||||
t.Fatal("anonymous save must be 401")
|
||||
}
|
||||
if code, _ := saveTermAPI(t, r, otherToken, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 404 {
|
||||
t.Fatal("saving into a foreign chapter must be 404")
|
||||
}
|
||||
// A client cannot name the owner, the language, the word or the record.
|
||||
for _, body := range []map[string]any{
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "ownerId": otherUser.UserId},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "language": "fr"},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "term": "forged"},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "id": 1},
|
||||
} {
|
||||
if code, _ := saveTermAPI(t, r, ownerToken, body); code != 400 {
|
||||
t.Fatalf("tampered input %v must be 400: %d", body, code)
|
||||
}
|
||||
}
|
||||
// Invalid or partial ranges, an unknown status and a forbidden level.
|
||||
for _, body := range []map[string]any{
|
||||
{"chapterId": chapter.ID, "start": 0, "status": termStatusNew},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 3, "status": termStatusNew},
|
||||
{"chapterId": chapter.ID, "start": 1, "end": 4, "status": termStatusNew},
|
||||
{"chapterId": chapter.ID, "start": 4, "end": 5, "status": termStatusNew},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": "deleted"},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusKnown, "level": 4},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusLearning, "level": 8},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "definition": strings.Repeat("a", termDefinitionLimit+1)},
|
||||
{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew, "examples": []string{"1", "2", "3", "4", "5", "6"}},
|
||||
} {
|
||||
if code, _ := saveTermAPI(t, r, ownerToken, body); code != 400 {
|
||||
t.Fatalf("invalid input %v must be 400: %d", body, code)
|
||||
}
|
||||
}
|
||||
// Entering learning without a level starts at 1 and still updates the same entry.
|
||||
code, learning := saveTermAPI(t, r, ownerToken, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusLearning})
|
||||
if code != 200 || learning.Term.ID != saved.Term.ID || learning.Term.Level != 1 {
|
||||
t.Fatalf("learning default level: %d %#v", code, learning)
|
||||
}
|
||||
// A chapter that is not ready refuses the save instead of storing a word.
|
||||
if err := db.Model(&Chapter{}).Where("id = ?", chapter.ID).Update("status", statusPending).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code, _ := saveTermAPI(t, r, ownerToken, map[string]any{"chapterId": chapter.ID, "start": 0, "end": 4, "status": termStatusNew}); code != 409 {
|
||||
t.Fatal("pending chapter must be 409")
|
||||
}
|
||||
// An unknown chapter id never confirms whether it belongs to someone else.
|
||||
if code, _ := saveTermAPI(t, r, ownerToken, map[string]any{"chapterId": 999999, "start": 0, "end": 4, "status": termStatusNew}); code != 404 {
|
||||
t.Fatal("unknown chapter must be 404")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTermTextRoundTrip checks the learner's own text, including characters
|
||||
// that are never normalized away.
|
||||
func TestMySQLTermTextRoundTrip(t *testing.T) {
|
||||
db := testDB(t)
|
||||
r := Router(db, time.Now)
|
||||
_, token, chapters := termFixture(t, db, r, "Café Dogs")
|
||||
code, saved := saveTermAPI(t, r, token, map[string]any{
|
||||
"chapterId": chapters[0].ID, "start": 0, "end": 4,
|
||||
"definition": "咖啡\n附带说明", "examples": []string{"A fictional example.", "😀 second"},
|
||||
"status": termStatusLearning, "level": 7,
|
||||
})
|
||||
if code != 201 {
|
||||
t.Fatalf("save: %d", code)
|
||||
}
|
||||
code, data := callAPI(t, r, "GET", fmt.Sprintf("/api/v1/terms/%d", saved.Term.ID), token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("read: %d", code)
|
||||
}
|
||||
var response struct {
|
||||
Term TermView `json:"term"`
|
||||
}
|
||||
json.Unmarshal(data, &response)
|
||||
if response.Term.Definition != "咖啡\n附带说明" || len(response.Term.Examples) != 2 || response.Term.Examples[1] != "😀 second" || response.Term.Level != 7 {
|
||||
t.Fatalf("round trip: %#v", response.Term)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,183 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
admin "go-admin/app/admin/models"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// A TXT upload is decoded in memory and handed to the same paste pipeline. Nothing is
|
||||
// written to disk: there is no temporary file to leak, and the client file name never
|
||||
// becomes a path, so it cannot reach outside the request.
|
||||
const maxTextUploadBytes = 2 << 20
|
||||
|
||||
var (
|
||||
utf8BOM = []byte{0xEF, 0xBB, 0xBF}
|
||||
utf16BOMBig = []byte{0xFE, 0xFF}
|
||||
utf16BOMSmall = []byte{0xFF, 0xFE}
|
||||
)
|
||||
|
||||
// decodeTextUpload turns an uploaded TXT into the exact text the reader will show. Only
|
||||
// UTF-8 is accepted: an optional BOM is removed before the text is validated, and any byte
|
||||
// that is not valid UTF-8 rejects the file instead of being replaced, so a chapter never
|
||||
// contains a substitute character the learner did not write.
|
||||
func decodeTextUpload(raw []byte) (string, error) {
|
||||
if len(raw) > maxTextUploadBytes {
|
||||
return "", failure(400, "TXT 文件不能超过 2 MiB")
|
||||
}
|
||||
if bytes.HasPrefix(raw, utf16BOMBig) || bytes.HasPrefix(raw, utf16BOMSmall) {
|
||||
return "", failure(400, "文件是 UTF-16 编码，请另存为 UTF-8 后重试")
|
||||
}
|
||||
raw = bytes.TrimPrefix(raw, utf8BOM)
|
||||
if !utf8.Valid(raw) {
|
||||
return "", failure(400, "文件不是 UTF-8 编码，请另存为 UTF-8 后重试")
|
||||
}
|
||||
if bytes.IndexByte(raw, 0) >= 0 {
|
||||
return "", failure(400, "文件包含无法处理的字符，请另存为纯文本后重试")
|
||||
}
|
||||
return string(raw), nil
|
||||
}
|
||||
|
||||
type textUpload struct {
|
||||
RequestID string
|
||||
Title string
|
||||
Language string
|
||||
Text string
|
||||
}
|
||||
|
||||
// readTextUpload parses the multipart submission: one file part plus a whitelist of text
|
||||
// fields. The uploaded name is never read, not even for validation, because it only serves
|
||||
// display in the browser.
|
||||
func readTextUpload(c *gin.Context, allowLanguage bool) (textUpload, error) {
|
||||
bad := failure(400, "TXT 上传无效，请选择 UTF-8 的 .txt 文件并填写标题")
|
||||
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, maxTextUploadBytes+(64<<10))
|
||||
reader, err := c.Request.MultipartReader()
|
||||
if err != nil {
|
||||
return textUpload{}, bad
|
||||
}
|
||||
fields := map[string]string{}
|
||||
var content []byte
|
||||
for {
|
||||
part, err := reader.NextPart()
|
||||
if errors.Is(err, io.EOF) {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return textUpload{}, uploadBodyError(err, bad)
|
||||
}
|
||||
name := part.FormName()
|
||||
if name == "file" {
|
||||
if content != nil {
|
||||
part.Close()
|
||||
return textUpload{}, bad
|
||||
}
|
||||
content, err = io.ReadAll(io.LimitReader(part, maxTextUploadBytes+1))
|
||||
part.Close()
|
||||
if err != nil {
|
||||
return textUpload{}, uploadBodyError(err, bad)
|
||||
}
|
||||
if len(content) == 0 || len(content) > maxTextUploadBytes {
|
||||
return textUpload{}, failure(400, "TXT 文件不能为空且不能超过 2 MiB")
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !textUploadField(name, allowLanguage) {
|
||||
part.Close()
|
||||
return textUpload{}, bad
|
||||
}
|
||||
if _, exists := fields[name]; exists {
|
||||
part.Close()
|
||||
return textUpload{}, bad
|
||||
}
|
||||
value, readErr := io.ReadAll(io.LimitReader(part, 1025))
|
||||
part.Close()
|
||||
if readErr != nil {
|
||||
return textUpload{}, uploadBodyError(readErr, bad)
|
||||
}
|
||||
if len(value) > 1024 || !utf8.Valid(value) {
|
||||
return textUpload{}, bad
|
||||
}
|
||||
fields[name] = string(value)
|
||||
}
|
||||
if content == nil {
|
||||
return textUpload{}, bad
|
||||
}
|
||||
text, err := decodeTextUpload(content)
|
||||
if err != nil {
|
||||
return textUpload{}, err
|
||||
}
|
||||
return textUpload{RequestID: fields["requestId"], Title: fields["title"], Language: fields["language"], Text: text}, nil
|
||||
}
|
||||
|
||||
func textUploadField(name string, allowLanguage bool) bool {
|
||||
switch name {
|
||||
case "requestId", "title":
|
||||
return true
|
||||
case "language":
|
||||
return allowLanguage
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// An oversized body is reported as a size problem, not as an invalid upload.
|
||||
func uploadBodyError(err error, bad error) error {
|
||||
var tooLarge *http.MaxBytesError
|
||||
if errors.As(err, &tooLarge) {
|
||||
return failure(400, "TXT 文件不能超过 2 MiB")
|
||||
}
|
||||
return bad
|
||||
}
|
||||
|
||||
// registerUploadRoutes reuses the paste pipeline: the decoded file becomes the same
|
||||
// PasteBook/PasteChapter input, so idempotency, ownership and the ingest job behave exactly
|
||||
// as they do for pasted text. One upload at a time keeps concurrent large decodes bounded.
|
||||
func registerUploadRoutes(v *gin.RouterGroup, protect func(bool, func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc, now func() time.Time) {
|
||||
gate := make(chan struct{}, 1)
|
||||
enter := func() error {
|
||||
select {
|
||||
case gate <- struct{}{}:
|
||||
return nil
|
||||
default:
|
||||
return failure(429, "已有文件正在上传，请稍后重试")
|
||||
}
|
||||
}
|
||||
leave := func() { <-gate }
|
||||
v.POST("/books/upload", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
if err := enter(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer leave()
|
||||
upload, err := readTextUpload(c, true)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return PasteBook(tx, u.UserId, now(), PasteBookInput{
|
||||
RequestID: upload.RequestID, Title: upload.Title, Text: upload.Text, Language: upload.Language,
|
||||
})
|
||||
}))
|
||||
v.POST("/books/:id/chapters/upload", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
bookID, err := pathID(c, "书籍不存在")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := enter(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer leave()
|
||||
upload, err := readTextUpload(c, false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return PasteChapter(tx, u.UserId, bookID, now(), PasteChapterInput{
|
||||
RequestID: upload.RequestID, Title: upload.Title, Text: upload.Text,
|
||||
})
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,376 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"mime/multipart"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
// uploadFixture is the same character set the paste contract preserves, written as the bytes
|
||||
// a file would hold: CRLF and LF, a tab, curly quotes, an em dash, an ellipsis, an emoji, a
|
||||
// combining acute accent, a trailing space run and an empty final line.
|
||||
const uploadFixture = "Mira opened the workshop.\r\n\r\n\tThe sign read “A small step…” — café e\u0301 🙂\r\nTrailing spaces here: \n\n"
|
||||
|
||||
// uploadFile posts one multipart TXT submission. A nil file means "no file part", and an
|
||||
// empty name means "no file name", so the rejection paths stay testable.
|
||||
func uploadFile(t *testing.T, r *gin.Engine, token, path, fileName string, content []byte, fields map[string]string) (int, string, pasteResponse) {
|
||||
t.Helper()
|
||||
var body bytes.Buffer
|
||||
writer := multipart.NewWriter(&body)
|
||||
names := make([]string, 0, len(fields))
|
||||
for name := range fields {
|
||||
names = append(names, name)
|
||||
}
|
||||
// Field order must be stable so a failure message is reproducible.
|
||||
for _, name := range []string{"requestId", "title", "language"} {
|
||||
if value, ok := fields[name]; ok {
|
||||
if err := writer.WriteField(name, value); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
names = removeString(names, name)
|
||||
}
|
||||
}
|
||||
for _, name := range names {
|
||||
if err := writer.WriteField(name, fields[name]); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if content != nil {
|
||||
part, err := writer.CreateFormFile("file", fileName)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = part.Write(content); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
writer.Close()
|
||||
request := httptest.NewRequest("POST", path, &body)
|
||||
request.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
if token != "" {
|
||||
request.Header.Set("Authorization", "Bearer "+token)
|
||||
}
|
||||
response := httptest.NewRecorder()
|
||||
r.ServeHTTP(response, request)
|
||||
var envelope struct {
|
||||
Msg string `json:"msg"`
|
||||
Data json.RawMessage `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(response.Body.Bytes(), &envelope); err != nil {
|
||||
t.Fatalf("upload response for %s: %v", path, err)
|
||||
}
|
||||
var out pasteResponse
|
||||
if len(envelope.Data) > 0 {
|
||||
json.Unmarshal(envelope.Data, &out)
|
||||
}
|
||||
return response.Code, envelope.Msg, out
|
||||
}
|
||||
|
||||
func removeString(values []string, target string) []string {
|
||||
result := values[:0]
|
||||
for _, value := range values {
|
||||
if value != target {
|
||||
result = append(result, value)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func TestDecodeTextUploadRules(t *testing.T) {
|
||||
// Valid UTF-8 is returned exactly as received, including unusual but legal characters.
|
||||
if text, err := decodeTextUpload([]byte(uploadFixture)); err != nil || text != uploadFixture {
|
||||
t.Fatalf("valid file: %q %v", text, err)
|
||||
}
|
||||
// A UTF-8 BOM is removed and never becomes part of the original text.
|
||||
withBOM := append(append([]byte{}, utf8BOM...), []byte("BOM before text\n")...)
|
||||
if text, err := decodeTextUpload(withBOM); err != nil || text != "BOM before text\n" {
|
||||
t.Fatalf("BOM file: %q %v", text, err)
|
||||
}
|
||||
// Only a BOM leaves an empty text, which the paste rules then reject.
|
||||
if text, err := decodeTextUpload(append([]byte{}, utf8BOM...)); err != nil || text != "" {
|
||||
t.Fatalf("BOM only: %q %v", text, err)
|
||||
}
|
||||
rejected := []struct {
|
||||
name string
|
||||
content []byte
|
||||
message string
|
||||
}{
|
||||
{"invalid UTF-8", []byte{0x41, 0x80, 0x42}, "UTF-8"},
|
||||
{"latin-1 text", []byte("caf\xe9 plain\n"), "UTF-8"},
|
||||
{"UTF-16 little endian", []byte{0xFF, 0xFE, 0x41, 0x00}, "UTF-16"},
|
||||
{"UTF-16 big endian", []byte{0xFE, 0xFF, 0x00, 0x41}, "UTF-16"},
|
||||
{"NUL byte", []byte("text\x00more"), "无法处理"},
|
||||
}
|
||||
for _, tc := range rejected {
|
||||
text, err := decodeTextUpload(tc.content)
|
||||
if err == nil || text != "" {
|
||||
t.Fatalf("%s was accepted as %q", tc.name, text)
|
||||
}
|
||||
api, ok := err.(*apiError)
|
||||
if !ok || api.status != 400 || !strings.Contains(api.message, tc.message) {
|
||||
t.Fatalf("%s message: %v", tc.name, err)
|
||||
}
|
||||
}
|
||||
// The size boundary is exact on both sides, and the byte limit is checked before decoding.
|
||||
if _, err := decodeTextUpload(bytes.Repeat([]byte("a"), maxTextUploadBytes)); err != nil {
|
||||
t.Fatalf("a file at the size limit must be accepted: %v", err)
|
||||
}
|
||||
if _, err := decodeTextUpload(bytes.Repeat([]byte("a"), maxTextUploadBytes+1)); err == nil {
|
||||
t.Fatal("a file over the size limit must be rejected")
|
||||
}
|
||||
// The paste rules still bound one chapter, so the byte limit cannot smuggle in more text.
|
||||
if _, _, _, err := validatePaste("title", strings.Repeat("a", maxChapterRunes)); err != nil {
|
||||
t.Fatalf("the exact chapter limit must be accepted: %v", err)
|
||||
}
|
||||
if _, _, _, err := validatePaste("title", strings.Repeat("a", maxChapterRunes+1)); err == nil {
|
||||
t.Fatal("the chapter code point limit must still apply to uploaded text")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadImportPath covers the accepted file: it creates the same book, chapter
|
||||
// and job as a paste, and the reader text equals the file byte for byte.
|
||||
func TestMySQLTextUploadImportPath(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
fields := map[string]string{"requestId": "upload-fixture-0001", "title": "The Workshop Upload", "language": "en"}
|
||||
code, msg, uploaded := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "workshop.txt", []byte(uploadFixture), fields)
|
||||
if code != 201 {
|
||||
t.Fatalf("upload status %d (%s)", code, msg)
|
||||
}
|
||||
if uploaded.Book == nil || uploaded.Book.Title != "The Workshop Upload" || uploaded.Book.Language != "en" {
|
||||
t.Fatalf("unexpected book %+v", uploaded.Book)
|
||||
}
|
||||
if uploaded.Chapter.Ordinal != 1 || uploaded.Chapter.Status != statusPending || uploaded.Duplicate {
|
||||
t.Fatalf("unexpected chapter %+v", uploaded.Chapter)
|
||||
}
|
||||
|
||||
// The worker publishes the chapter, and the reader shows exactly the file content.
|
||||
drainIngest(t, db)
|
||||
code, ready := readChapter(t, r, learner.Token, uploaded.Chapter.ID)
|
||||
if code != 200 || ready.Chapter.Status != statusReady || ready.Chapter.OriginalText != uploadFixture {
|
||||
t.Fatalf("reader text: status %d, %+v", code, ready.Chapter)
|
||||
}
|
||||
if want := len([]rune(uploadFixture)); ready.Chapter.CharCount != want {
|
||||
t.Fatalf("charCount %d, want %d", ready.Chapter.CharCount, want)
|
||||
}
|
||||
|
||||
// A UTF-8 BOM is stripped, so the reader never shows it.
|
||||
bomFields := map[string]string{"requestId": "upload-bom-000002", "title": "BOM Upload", "language": "en"}
|
||||
withBOM := append(append([]byte{}, utf8BOM...), []byte("Plain text with BOM.\n")...)
|
||||
code, msg, bom := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "bom.txt", withBOM, bomFields)
|
||||
if code != 201 {
|
||||
t.Fatalf("BOM upload status %d (%s)", code, msg)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
if code, read := readChapter(t, r, learner.Token, bom.Chapter.ID); code != 200 || read.Chapter.OriginalText != "Plain text with BOM.\n" {
|
||||
t.Fatalf("BOM text: status %d, %q", code, read.Chapter.OriginalText)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadIdempotencyAndAppend reuses the paste job rules: one file yields one
|
||||
// chapter, a repeated upload answers with that chapter, and the same request id with other
|
||||
// content is a conflict.
|
||||
func TestMySQLTextUploadIdempotencyAndAppend(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
other := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
fields := map[string]string{"requestId": "upload-repeat-0001", "title": "Repeat Upload", "language": "en"}
|
||||
code, msg, first := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "repeat.txt", []byte("First upload body.\n"), fields)
|
||||
if code != 201 {
|
||||
t.Fatalf("first upload %d (%s)", code, msg)
|
||||
}
|
||||
code, msg, again := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "repeat.txt", []byte("First upload body.\n"), fields)
|
||||
if code != 200 || !again.Duplicate || again.Chapter.ID != first.Chapter.ID {
|
||||
t.Fatalf("repeat upload %d (%s): %+v", code, msg, again)
|
||||
}
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", first.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("a repeated upload created %d chapters", chapters)
|
||||
}
|
||||
// The same request id with other content is a conflict, not a second chapter.
|
||||
code, msg, _ = uploadFile(t, r, learner.Token, "/api/v1/books/upload", "repeat.txt", []byte("Different body.\n"), fields)
|
||||
if code != 409 {
|
||||
t.Fatalf("changed content status %d (%s)", code, msg)
|
||||
}
|
||||
|
||||
// Appending uses the same rules and the same job pipeline.
|
||||
appendFields := map[string]string{"requestId": "upload-append-0002", "title": "Second Chapter"}
|
||||
path := fmt.Sprintf("/api/v1/books/%d/chapters/upload", first.Book.ID)
|
||||
code, msg, appended := uploadFile(t, r, learner.Token, path, "second.txt", []byte("A single plain paragraph.\n"), appendFields)
|
||||
if code != 201 {
|
||||
t.Fatalf("append upload %d (%s)", code, msg)
|
||||
}
|
||||
if appended.Chapter.Ordinal != 2 || appended.Chapter.BookID != first.Book.ID {
|
||||
t.Fatalf("unexpected appended chapter %+v", appended.Chapter)
|
||||
}
|
||||
code, msg, againAppend := uploadFile(t, r, learner.Token, path, "second.txt", []byte("A single plain paragraph.\n"), appendFields)
|
||||
if code != 200 || !againAppend.Duplicate || againAppend.Chapter.ID != appended.Chapter.ID {
|
||||
t.Fatalf("repeated append %d (%s): %+v", code, msg, againAppend)
|
||||
}
|
||||
// The append endpoint does not accept a language field: the book owns the language.
|
||||
code, msg, _ = uploadFile(t, r, learner.Token, path, "second.txt", []byte("Another body.\n"),
|
||||
map[string]string{"requestId": "upload-append-lang-0004", "title": "Second Chapter", "language": "en"})
|
||||
if code != 400 {
|
||||
t.Fatalf("append with a language field: status %d (%s)", code, msg)
|
||||
}
|
||||
|
||||
// Another account cannot append into this book, and never learns whether it exists.
|
||||
code, foreignMsg, _ := uploadFile(t, r, other.Token, path, "second.txt", []byte("Foreign body.\n"), map[string]string{"requestId": "upload-foreign-0003", "title": "Foreign"})
|
||||
if code != 404 {
|
||||
t.Fatalf("foreign append status %d (%s)", code, foreignMsg)
|
||||
}
|
||||
// The other account's own library stays empty.
|
||||
_, list := bookList(t, r, other.Token)
|
||||
if len(list.Items) != 0 {
|
||||
t.Fatalf("the other account must not see this book: %+v", list.Items)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadRejectsInvalidSubmissions covers the validation surface, including the
|
||||
// client file name, which is never used as a path.
|
||||
func TestMySQLTextUploadRejectsInvalidSubmissions(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
base := map[string]string{"requestId": "upload-invalid-0001", "title": "Invalid Upload", "language": "en"}
|
||||
cases := []struct {
|
||||
name string
|
||||
fileName string
|
||||
content []byte
|
||||
fields map[string]string
|
||||
status int
|
||||
}{
|
||||
{"no file part", "", nil, base, 400},
|
||||
{"empty file", "empty.txt", []byte{}, base, 400},
|
||||
{"empty file part", "empty.txt", []byte{}, base, 400},
|
||||
{"whitespace only", "blank.txt", []byte(" \n\t\n"), base, 400},
|
||||
{"BOM only", "bom.txt", append([]byte{}, utf8BOM...), base, 400},
|
||||
{"invalid UTF-8", "latin1.txt", []byte("caf\xe9\n"), base, 400},
|
||||
{"UTF-16 file", "unicode.txt", []byte{0xFF, 0xFE, 0x41, 0x00}, base, 400},
|
||||
{"oversized file", "big.txt", bytes.Repeat([]byte("a"), maxTextUploadBytes+1), base, 400},
|
||||
{"missing request id", "text.txt", []byte("Body.\n"), map[string]string{"title": "Invalid Upload", "language": "en"}, 400},
|
||||
{"missing title", "text.txt", []byte("Body.\n"), map[string]string{"requestId": "upload-invalid-0002", "language": "en"}, 400},
|
||||
{"unsupported language", "text.txt", []byte("Body.\n"), map[string]string{"requestId": "upload-invalid-0004", "title": "Invalid Upload", "language": "fr"}, 400},
|
||||
{"short request id", "text.txt", []byte("Body.\n"), map[string]string{"requestId": "short", "title": "Invalid Upload", "language": "en"}, 400},
|
||||
{"unknown field", "text.txt", []byte("Body.\n"), map[string]string{"requestId": "upload-invalid-0005", "title": "Invalid Upload", "language": "en", "ownerId": "9"}, 400},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
code, msg, _ := uploadFile(t, r, learner.Token, "/api/v1/books/upload", tc.fileName, tc.content, tc.fields)
|
||||
if code != tc.status {
|
||||
t.Fatalf("%s: status %d (%s), want %d", tc.name, code, msg, tc.status)
|
||||
}
|
||||
if msg == "" {
|
||||
t.Fatalf("%s: rejection without a readable message", tc.name)
|
||||
}
|
||||
}
|
||||
// No rejected submission left a book behind.
|
||||
_, list := bookList(t, r, learner.Token)
|
||||
if len(list.Items) != 0 {
|
||||
t.Fatalf("rejected uploads created books: %+v", list.Items)
|
||||
}
|
||||
|
||||
// A missing language field follows the paste rule and defaults to English.
|
||||
code, msg, defaulted := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "default.txt", []byte("Body without language.\n"),
|
||||
map[string]string{"requestId": "upload-default-lang-0007", "title": "Default Language"})
|
||||
if code != 201 || defaulted.Book == nil || defaulted.Book.Language != "en" {
|
||||
t.Fatalf("missing language must default to English: status %d (%s) book %+v", code, msg, defaulted.Book)
|
||||
}
|
||||
|
||||
// The uploaded name is only a display string: a traversal-shaped name changes nothing.
|
||||
hostile := map[string]string{"requestId": "upload-hostile-0006", "title": "Hostile Name", "language": "en"}
|
||||
code, msg, uploaded := uploadFile(t, r, learner.Token, "/api/v1/books/upload", `..\..\windows\system32\evil.txt`, []byte("Hostile but harmless.\n"), hostile)
|
||||
if code != 201 {
|
||||
t.Fatalf("hostile name status %d (%s)", code, msg)
|
||||
}
|
||||
var chapter Chapter
|
||||
if err := db.First(&chapter, uploaded.Chapter.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var book Book
|
||||
if err := db.First(&book, uploaded.Book.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, field := range []string{chapter.Title, chapter.OriginalText, book.Title} {
|
||||
if strings.Contains(field, "evil") || strings.Contains(field, "system32") || strings.Contains(field, `..`) {
|
||||
t.Fatalf("the uploaded name leaked into stored data: %q", field)
|
||||
}
|
||||
}
|
||||
|
||||
// An unrelated content type is not a multipart upload.
|
||||
request := httptest.NewRequest("POST", "/api/v1/books/upload", strings.NewReader(`{"requestId":"json-upload-0001"}`))
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
request.Header.Set("Authorization", "Bearer "+learner.Token)
|
||||
response := httptest.NewRecorder()
|
||||
r.ServeHTTP(response, request)
|
||||
if response.Code != 400 {
|
||||
t.Fatalf("JSON body accepted as an upload: %d", response.Code)
|
||||
}
|
||||
// Uploading without a session is rejected before any parsing.
|
||||
code, _, _ = uploadFile(t, r, "", "/api/v1/books/upload", "text.txt", []byte("Body.\n"), base)
|
||||
if code != 401 {
|
||||
t.Fatalf("anonymous upload status %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadKeepsChapterLimit proves the byte cap cannot bypass the one-chapter
|
||||
// code point rule, and that a large but legal file is stored completely.
|
||||
func TestMySQLTextUploadKeepsChapterLimit(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
// Exactly at the chapter limit: accepted, and the stored text keeps its full length.
|
||||
atLimit := strings.Repeat("a", maxChapterRunes-1) + "\n"
|
||||
fields := map[string]string{"requestId": "upload-limit-0001", "title": "At The Limit", "language": "en"}
|
||||
code, msg, uploaded := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "limit.txt", []byte(atLimit), fields)
|
||||
if code != 201 {
|
||||
t.Fatalf("upload at the chapter limit %d (%s)", code, msg)
|
||||
}
|
||||
drainIngest(t, db)
|
||||
if code, read := readChapter(t, r, learner.Token, uploaded.Chapter.ID); code != 200 || read.Chapter.OriginalText != atLimit {
|
||||
t.Fatalf("chapter at the limit: status %d, length %d", code, len([]rune(read.Chapter.OriginalText)))
|
||||
}
|
||||
// One code point more is rejected by the same rule that already applies to a paste.
|
||||
overLimit := strings.Repeat("a", maxChapterRunes+1)
|
||||
code, msg, _ = uploadFile(t, r, learner.Token, "/api/v1/books/upload", "over.txt", []byte(overLimit), map[string]string{"requestId": "upload-limit-0002", "title": "Over The Limit", "language": "en"})
|
||||
if code != 400 || !strings.Contains(msg, "100000") {
|
||||
t.Fatalf("upload over the chapter limit: status %d (%s)", code, msg)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMySQLTextUploadAndPasteShareOnePipeline checks the two entry points cannot produce a
|
||||
// second chapter for the same submitted content when the client stays on one request id.
|
||||
func TestMySQLTextUploadAndPasteShareOnePipeline(t *testing.T) {
|
||||
db, r, owner := libraryFixture(t)
|
||||
learner := newLearner(t, r, owner.Token)
|
||||
drainIngest(t, db)
|
||||
|
||||
fields := map[string]string{"requestId": "upload-shared-0001", "title": "Shared Pipeline", "language": "en"}
|
||||
code, msg, uploaded := uploadFile(t, r, learner.Token, "/api/v1/books/upload", "shared.txt", []byte("Shared body.\n"), fields)
|
||||
if code != 201 {
|
||||
t.Fatalf("upload %d (%s)", code, msg)
|
||||
}
|
||||
// The same request id through the paste endpoint answers with the uploaded chapter.
|
||||
code, pasted := pasteBook(t, r, learner.Token, map[string]string{
|
||||
"requestId": "upload-shared-0001", "title": "Shared Pipeline", "text": "Shared body.\n", "language": "en"})
|
||||
if code != 200 || !pasted.Duplicate || pasted.Chapter.ID != uploaded.Chapter.ID {
|
||||
t.Fatalf("paste after upload %d: %+v", code, pasted)
|
||||
}
|
||||
var chapters int64
|
||||
db.Model(&Chapter{}).Where("book_id = ?", uploaded.Book.ID).Count(&chapters)
|
||||
if chapters != 1 {
|
||||
t.Fatalf("the two entry points created %d chapters", chapters)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,338 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bufio"
|
||||
"bytes"
|
||||
"errors"
|
||||
"io"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode"
|
||||
|
||||
"golang.org/x/text/unicode/norm"
|
||||
)
|
||||
|
||||
const WordNetSHA = "cbda5ea6eef7f36a97a43d4a75f85e07fccbb4f23657d27b4ccbc93e2646ab59"
|
||||
const WordNetSource = "https://raw.githubusercontent.com/nltk/nltk_data/96f9b3252457a2b97e52aec64c3dfceeb5c312d5/packages/corpora/wordnet.zip"
|
||||
const maxDictionaryZip = 32 << 20
|
||||
const maxDictionaryInflated = 128 << 20
|
||||
|
||||
type TextToken struct {
|
||||
Text string `json:"text"`
|
||||
Start int `json:"start"`
|
||||
End int `json:"end"`
|
||||
StartUtf16 int `json:"startUtf16"`
|
||||
EndUtf16 int `json:"endUtf16"`
|
||||
Kind string `json:"kind"`
|
||||
Term *TokenTerm `json:"term,omitempty"`
|
||||
}
|
||||
|
||||
// Tokenize never normalizes text. Letters begin words; marks continue them and
|
||||
// apostrophes join letters. Numbers and symbols are non-selectable punctuation.
|
||||
// All offsets are half-open, in code points and UTF-16 code units respectively.
|
||||
func Tokenize(text string) []TextToken {
|
||||
runes := []rune(text)
|
||||
result := make([]TextToken, 0)
|
||||
utf16Offset := 0
|
||||
for i := 0; i < len(runes); {
|
||||
start, start16 := i, utf16Offset
|
||||
kind := "punctuation"
|
||||
if unicode.IsLetter(runes[i]) {
|
||||
kind = "word"
|
||||
} else if unicode.IsSpace(runes[i]) {
|
||||
kind = "space"
|
||||
}
|
||||
i++
|
||||
for i < len(runes) {
|
||||
if kind == "word" && (unicode.IsLetter(runes[i]) || unicode.IsMark(runes[i]) || ((runes[i] == '\'' || runes[i] == '’') && i+1 < len(runes) && unicode.IsLetter(runes[i+1]))) {
|
||||
i++
|
||||
continue
|
||||
}
|
||||
if kind == "space" && unicode.IsSpace(runes[i]) {
|
||||
i++
|
||||
continue
|
||||
}
|
||||
break
|
||||
}
|
||||
for _, r := range runes[start:i] {
|
||||
utf16Offset++
|
||||
if r > 0xffff {
|
||||
utf16Offset++
|
||||
}
|
||||
}
|
||||
result = append(result, TextToken{Text: string(runes[start:i]), Start: start, End: i, StartUtf16: start16, EndUtf16: utf16Offset, Kind: kind})
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func normalizeWord(word string) string {
|
||||
return norm.NFC.String(strings.ToLower(strings.ReplaceAll(word, "’", "'")))
|
||||
}
|
||||
|
||||
type DictionaryEntry struct {
|
||||
Lemma string `json:"lemma"`
|
||||
POS string `json:"pos"`
|
||||
Definition string `json:"definition"`
|
||||
Examples []string `json:"examples"`
|
||||
}
|
||||
type LookupResult struct {
|
||||
Status string `json:"status"`
|
||||
Query string `json:"query"`
|
||||
MatchedForm *string `json:"matchedForm"`
|
||||
Candidates []string `json:"candidates"`
|
||||
Entries []DictionaryEntry `json:"entries"`
|
||||
Resource *LookupResource `json:"resource,omitempty"`
|
||||
}
|
||||
type LookupResource struct {
|
||||
Name string `json:"name"`
|
||||
Version string `json:"version"`
|
||||
}
|
||||
type wordNetPOS struct {
|
||||
pos string
|
||||
index map[string][]int
|
||||
data map[int]DictionaryEntry
|
||||
exceptions map[string][]string
|
||||
}
|
||||
type WordNet struct {
|
||||
parts []wordNetPOS
|
||||
EntryCount int
|
||||
}
|
||||
|
||||
// ParseWordNet accepts only the pinned official corpus. No archive member is ever
|
||||
// extracted to disk; both declared and actual inflated sizes are bounded.
|
||||
func ParseWordNet(raw []byte) (*WordNet, error) {
|
||||
if len(raw) > maxDictionaryZip || contentSHA(string(raw)) != WordNetSHA {
|
||||
return nil, errors.New("WordNet ZIP checksum mismatch")
|
||||
}
|
||||
return parseWordNetArchive(raw)
|
||||
}
|
||||
|
||||
func readWordNetArchive(raw []byte) (map[string][]byte, error) {
|
||||
if len(raw) > maxDictionaryZip {
|
||||
return nil, errors.New("archive too large")
|
||||
}
|
||||
reader, err := zip.NewReader(bytes.NewReader(raw), int64(len(raw)))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(reader.File) > 64 {
|
||||
return nil, errors.New("too many archive members")
|
||||
}
|
||||
files := map[string][]byte{}
|
||||
seen := map[string]bool{}
|
||||
total := int64(0)
|
||||
for _, f := range reader.File {
|
||||
name := f.Name
|
||||
if strings.Contains(name, "\\") || path.IsAbs(name) || strings.Contains(name, ":") || path.Clean(name) != strings.TrimSuffix(name, "/") || !strings.HasPrefix(name, "wordnet/") || seen[name] {
|
||||
return nil, errors.New("unsafe archive member")
|
||||
}
|
||||
seen[name] = true
|
||||
if !f.FileInfo().IsDir() && !f.Mode().IsRegular() {
|
||||
return nil, errors.New("unsupported archive member")
|
||||
}
|
||||
if f.FileInfo().IsDir() {
|
||||
continue
|
||||
}
|
||||
if f.UncompressedSize64 > maxDictionaryInflated || total+int64(f.UncompressedSize64) > maxDictionaryInflated {
|
||||
return nil, errors.New("inflated archive too large")
|
||||
}
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(rc, maxDictionaryInflated-total+1))
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
total += int64(len(data))
|
||||
if total > maxDictionaryInflated {
|
||||
return nil, errors.New("inflated archive too large")
|
||||
}
|
||||
files[strings.TrimPrefix(name, "wordnet/")] = data
|
||||
}
|
||||
for _, name := range []string{"LICENSE", "README", "lexnames", "index.noun", "index.verb", "index.adj", "index.adv", "data.noun", "data.verb", "data.adj", "data.adv", "noun.exc", "verb.exc", "adj.exc", "adv.exc"} {
|
||||
if len(files[name]) == 0 {
|
||||
return nil, errors.New("missing required WordNet member")
|
||||
}
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func parseWordNetArchive(raw []byte) (*WordNet, error) {
|
||||
files, err := readWordNetArchive(raw)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := &WordNet{}
|
||||
for i, suffix := range []string{"noun", "verb", "adj", "adv"} {
|
||||
part := wordNetPOS{pos: []string{"n", "v", "a", "r"}[i], index: map[string][]int{}, data: map[int]DictionaryEntry{}, exceptions: map[string][]string{}}
|
||||
scanner := bufio.NewScanner(bytes.NewReader(files["data."+suffix]))
|
||||
scanner.Buffer(make([]byte, 4096), 1<<20)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if line == "" || line[0] == ' ' {
|
||||
continue
|
||||
}
|
||||
header, gloss, ok := strings.Cut(line, "|")
|
||||
fields := strings.Fields(header)
|
||||
if !ok || len(fields) < 6 {
|
||||
return nil, errors.New("invalid WordNet data record")
|
||||
}
|
||||
offset, e := strconv.Atoi(fields[0])
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
if _, exists := part.data[offset]; exists {
|
||||
return nil, errors.New("duplicate synset offset")
|
||||
}
|
||||
definition, examples := wordNetGloss(gloss)
|
||||
part.data[offset] = DictionaryEntry{POS: fields[2], Definition: definition, Examples: examples}
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
scanner = bufio.NewScanner(bytes.NewReader(files["index."+suffix]))
|
||||
scanner.Buffer(make([]byte, 4096), 1<<20)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if line == "" || line[0] == ' ' {
|
||||
continue
|
||||
}
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) < 7 {
|
||||
return nil, errors.New("invalid WordNet index record")
|
||||
}
|
||||
count, e := strconv.Atoi(fields[2])
|
||||
if e != nil || count < 1 || count > 1000 {
|
||||
return nil, errors.New("invalid sense count")
|
||||
}
|
||||
pointers, e := strconv.Atoi(fields[3])
|
||||
if e != nil || pointers < 0 || len(fields) != 6+pointers+count {
|
||||
return nil, errors.New("invalid index offsets")
|
||||
}
|
||||
offsets := make([]int, 0, count)
|
||||
for _, rawOffset := range fields[6+pointers:] {
|
||||
offset, e := strconv.Atoi(rawOffset)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
if _, ok := part.data[offset]; !ok {
|
||||
return nil, errors.New("missing synset")
|
||||
}
|
||||
offsets = append(offsets, offset)
|
||||
}
|
||||
if _, exists := part.index[fields[0]]; exists {
|
||||
return nil, errors.New("duplicate index key")
|
||||
}
|
||||
part.index[fields[0]] = offsets
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
scanner = bufio.NewScanner(bytes.NewReader(files[suffix+".exc"]))
|
||||
for scanner.Scan() {
|
||||
fields := strings.Fields(scanner.Text())
|
||||
if len(fields) < 2 {
|
||||
return nil, errors.New("invalid exception record")
|
||||
}
|
||||
part.exceptions[fields[0]] = fields[1:]
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result.EntryCount += len(part.index)
|
||||
result.parts = append(result.parts, part)
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func wordNetGloss(gloss string) (string, []string) {
|
||||
examples := []string{}
|
||||
definition := strings.Builder{}
|
||||
for {
|
||||
before, after, ok := strings.Cut(gloss, "\"")
|
||||
definition.WriteString(before)
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
example, rest, closed := strings.Cut(after, "\"")
|
||||
if !closed {
|
||||
definition.WriteString(after)
|
||||
break
|
||||
}
|
||||
examples = append(examples, example)
|
||||
gloss = rest
|
||||
}
|
||||
return strings.Trim(strings.TrimSpace(definition.String()), "; "), examples
|
||||
}
|
||||
|
||||
var morphyRules = map[string][][2]string{
|
||||
"n": {{"s", ""}, {"ses", "s"}, {"ves", "f"}, {"xes", "x"}, {"zes", "z"}, {"ches", "ch"}, {"shes", "sh"}, {"men", "man"}, {"ies", "y"}},
|
||||
"v": {{"s", ""}, {"ies", "y"}, {"es", "e"}, {"es", ""}, {"ed", "e"}, {"ed", ""}, {"ing", "e"}, {"ing", ""}},
|
||||
"a": {{"er", ""}, {"est", ""}, {"er", "e"}, {"est", "e"}},
|
||||
}
|
||||
|
||||
// Morphy yields possible dictionary forms, never contextual POS or a merged
|
||||
// learning identity. Exact forms win before exceptions and suffix detachment.
|
||||
func (wn *WordNet) Lookup(query string) LookupResult {
|
||||
result := LookupResult{Status: "not_found", Query: query, Candidates: []string{}, Entries: []DictionaryEntry{}}
|
||||
key := normalizeWord(query)
|
||||
add := func(form string, part wordNetPOS) {
|
||||
if len(part.index[form]) == 0 {
|
||||
return
|
||||
}
|
||||
found := false
|
||||
for _, candidate := range result.Candidates {
|
||||
if candidate == form {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
result.Candidates = append(result.Candidates, form)
|
||||
}
|
||||
for _, offset := range part.index[form] {
|
||||
if len(result.Entries) >= 12 {
|
||||
break
|
||||
}
|
||||
entry := part.data[offset]
|
||||
entry.Lemma = strings.ReplaceAll(form, "_", " ")
|
||||
result.Entries = append(result.Entries, entry)
|
||||
}
|
||||
}
|
||||
for _, part := range wn.parts {
|
||||
add(key, part)
|
||||
}
|
||||
if len(result.Entries) > 0 {
|
||||
result.Status = "exact"
|
||||
} else {
|
||||
for _, part := range wn.parts {
|
||||
forms := part.exceptions[key]
|
||||
if len(forms) == 0 {
|
||||
for _, rule := range morphyRules[part.pos] {
|
||||
if strings.HasSuffix(key, rule[0]) && len(key) > len(rule[0]) {
|
||||
forms = append(forms, strings.TrimSuffix(key, rule[0])+rule[1])
|
||||
}
|
||||
}
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, form := range forms {
|
||||
if !seen[form] {
|
||||
add(form, part)
|
||||
seen[form] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(result.Entries) > 0 {
|
||||
result.Status = "lemma"
|
||||
}
|
||||
}
|
||||
if len(result.Candidates) > 0 {
|
||||
form := result.Candidates[0]
|
||||
result.MatchedForm = &form
|
||||
}
|
||||
return result
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode/utf16"
|
||||
)
|
||||
|
||||
func TestArchiveRejectsUnsafeMembersAndMissingLicense(t *testing.T) {
|
||||
for _, name := range []string{"../data.noun", "wordnet/../data.noun", "/wordnet/data.noun", "wordnet\\data.noun", "wordnet/C:data.noun", "wordnet/LICENSE"} {
|
||||
var raw bytes.Buffer
|
||||
w := zip.NewWriter(&raw)
|
||||
entry, err := w.Create(name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entry.Write([]byte("fictional fixture"))
|
||||
w.Close()
|
||||
if _, err := readWordNetArchive(raw.Bytes()); err == nil {
|
||||
t.Fatalf("accepted unsafe/incomplete archive %s", name)
|
||||
}
|
||||
}
|
||||
var raw bytes.Buffer
|
||||
w := zip.NewWriter(&raw)
|
||||
for i := 0; i < 2; i++ {
|
||||
entry, _ := w.Create("wordnet/LICENSE")
|
||||
entry.Write([]byte("fixture"))
|
||||
}
|
||||
w.Close()
|
||||
if _, err := readWordNetArchive(raw.Bytes()); err == nil {
|
||||
t.Fatal("accepted duplicate member")
|
||||
}
|
||||
if _, err := ParseWordNet(make([]byte, maxDictionaryZip+1)); err == nil {
|
||||
t.Fatal("accepted oversized archive")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMorphyCandidatesAndExactPrecedence(t *testing.T) {
|
||||
wn := &WordNet{parts: []wordNetPOS{{pos: "v", index: map[string][]int{"axes": {1}, "ax": {1}, "axe": {2}}, data: map[int]DictionaryEntry{1: {POS: "v", Definition: "fixture one", Examples: []string{}}, 2: {POS: "v", Definition: "fixture two", Examples: []string{}}}, exceptions: map[string][]string{}}}}
|
||||
if got := wn.Lookup("AXES"); got.Status != "exact" || len(got.Candidates) != 1 || *got.MatchedForm != "axes" {
|
||||
t.Fatal("exact must precede lemma", got)
|
||||
}
|
||||
delete(wn.parts[0].index, "axes")
|
||||
got := wn.Lookup("axes")
|
||||
if got.Status != "lemma" || strings.Join(got.Candidates, ",") != "axe,ax" {
|
||||
t.Fatal("deterministic candidate forms", got)
|
||||
}
|
||||
definition, examples := wordNetGloss(`a fictional gloss; "one example"; "second example"`)
|
||||
if definition != "a fictional gloss" || len(examples) != 2 {
|
||||
t.Fatal("gloss", definition, examples)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTokensPreserveUnicodeOriginal(t *testing.T) {
|
||||
text := "😀 Cafe\u0301 can’t\r\nDogs 123 中文!"
|
||||
tokens := Tokenize(text)
|
||||
runes := []rune(text)
|
||||
cursor, u16 := 0, 0
|
||||
words := []string{}
|
||||
for _, token := range tokens {
|
||||
if token.Start != cursor || token.StartUtf16 != u16 || string(runes[token.Start:token.End]) != token.Text {
|
||||
t.Fatalf("incorrect token %#v", token)
|
||||
}
|
||||
cursor = token.End
|
||||
u16 += len(utf16.Encode([]rune(token.Text)))
|
||||
if token.EndUtf16 != u16 {
|
||||
t.Fatal("incorrect UTF-16 end")
|
||||
}
|
||||
if token.Kind == "word" {
|
||||
words = append(words, token.Text)
|
||||
}
|
||||
}
|
||||
if cursor != len(runes) || strings.Join(words, "|") != "Cafe\u0301|can’t|Dogs|中文" {
|
||||
t.Fatalf("coverage/words %v", words)
|
||||
}
|
||||
if normalizeWord("CAFE\u0301") != "café" || normalizeWord("CAN’T") != "can't" {
|
||||
t.Fatal("normalization")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOfficialWordNet(t *testing.T) {
|
||||
raw, err := os.ReadFile("../../../.local/nlp-resources/wordnet.zip")
|
||||
if os.IsNotExist(err) {
|
||||
t.Skip("explicitly prepared official WordNet fixture unavailable")
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
engine, err := ParseWordNet(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if engine.EntryCount < 150000 {
|
||||
t.Fatalf("entry count %d", engine.EntryCount)
|
||||
}
|
||||
for _, tc := range []struct{ query, status, lemma string }{{"dog", "exact", "dog"}, {"went", "lemma", "go"}, {"mice", "lemma", "mouse"}, {"zzzznonword", "not_found", ""}} {
|
||||
got := engine.Lookup(tc.query)
|
||||
if got.Status != tc.status || (tc.lemma != "" && (got.MatchedForm == nil || *got.MatchedForm != tc.lemma)) {
|
||||
t.Fatalf("%s: %#v", tc.query, got)
|
||||
}
|
||||
if tc.lemma != "" && (len(got.Entries) == 0 || got.Entries[0].Definition == "") {
|
||||
t.Fatal("missing definition")
|
||||
}
|
||||
}
|
||||
raw[100] ^= 1
|
||||
if _, err := ParseWordNet(raw); err == nil {
|
||||
t.Fatal("accepted corrupt checksum")
|
||||
}
|
||||
}
|
||||
@@ -82,7 +82,7 @@ func run() error {
|
||||
if err = lexgo.Migrate(db); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println("LexGo schema version 2 ready")
|
||||
fmt.Println("LexGo schema version", lexgo.SchemaVersion, "ready")
|
||||
return nil
|
||||
}
|
||||
if err = lexgo.CheckSchema(db); err != nil {
|
||||
@@ -112,6 +112,36 @@ func run() error {
|
||||
srv := &http.Server{Addr: addr, Handler: lexgo.Router(db, time.Now), ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 15 * time.Second, WriteTimeout: 15 * time.Second, IdleTimeout: 60 * time.Second, MaxHeaderBytes: 1 << 20}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
// A single instance owns the worker. Startup recovery returns chapters left in processing
|
||||
// by an unclean stop, and the running loop sweeps jobs whose finishing transaction failed,
|
||||
// so a transient database problem does not need a restart.
|
||||
if _, err = lexgo.RecoverIngestJobs(db, time.Now()); err != nil {
|
||||
return errors.New("ingestion recovery failed")
|
||||
}
|
||||
go func() {
|
||||
ticker := time.NewTicker(time.Second)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
jobCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||
requeued, recoverErr := lexgo.RequeueStaleIngestJobs(db, time.Now())
|
||||
if recoverErr != nil {
|
||||
if ctx.Err() == nil {
|
||||
log.Print("ingestion recovery failed; the next sweep retries it")
|
||||
}
|
||||
} else if requeued > 0 && ctx.Err() == nil {
|
||||
log.Printf("requeued %d interrupted ingestion job(s)", requeued)
|
||||
}
|
||||
if _, err := lexgo.ProcessIngestJobs(jobCtx, db, time.Now, 20); err != nil && ctx.Err() == nil {
|
||||
log.Print("ingestion batch stopped before finishing; the claimed job stays processing until the next sweep requeues it")
|
||||
}
|
||||
cancel()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
}
|
||||
}()
|
||||
go func() {
|
||||
ticker := time.NewTicker(time.Hour)
|
||||
defer ticker.Stop()
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"name": "Princeton WordNet",
|
||||
"language": "en",
|
||||
"version": "3.0",
|
||||
"format": "wordnet-3.0-zip",
|
||||
"source": "https://raw.githubusercontent.com/nltk/nltk_data/96f9b3252457a2b97e52aec64c3dfceeb5c312d5/packages/corpora/wordnet.zip",
|
||||
"sha256": "cbda5ea6eef7f36a97a43d4a75f85e07fccbb4f23657d27b4ccbc93e2646ab59",
|
||||
"license": "WORDNET-LICENSE.txt",
|
||||
"licenseUrl": "https://wordnet.princeton.edu/license-and-commercial-use",
|
||||
"documentation": [
|
||||
"https://wordnet.princeton.edu/documentation/wndb5wn",
|
||||
"https://wordnet.princeton.edu/documentation/morphy7wn"
|
||||
],
|
||||
"runtime": "Go only; explicitly imported ZIP persisted in MySQL; no runtime download or Python process"
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
# #3 英语技术验证
|
||||
|
||||
独立小样，不是学习端正式功能；不连接 MySQL、不读取账号、不保存输入。仅本机、单进程串行运行，默认英语。长期契约与实测结论见 [开发验证 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Local-Development-and-Verification)。
|
||||
|
||||
从仓库根目录运行（Windows PowerShell，需要 uv、Node）：
|
||||
|
||||
```powershell
|
||||
uv venv --python 3.12.12 .local/nlp-venv
|
||||
uv pip install --python .local/nlp-venv/Scripts/python.exe -r spikes/english/requirements.lock
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/setup_resources.py
|
||||
uv pip install --python .local/nlp-venv/Scripts/python.exe --no-deps .local/nlp-resources/en_core_web_sm-3.8.0-py3-none-any.whl
|
||||
.local/nlp-venv/Scripts/python.exe -m unittest discover -s spikes/english -v
|
||||
node --test spikes/english/view.test.mjs
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/benchmark.py
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/app.py
|
||||
```
|
||||
|
||||
打开 <http://127.0.0.1:5183/>。点击“分析文本”后点单词；`went` 应以 `go` 查询。手动查询仅查输入形式，不猜测词性：`dog` 直接命中，`went` 无结果。关闭进程即停止小样；端口占用时使用 `--port 5185`。用 `--resources .local/absent-resources` 启动可验证词典缺失，模型与词典独立加载。
|
||||
|
||||
准备依赖和资源时需要联网；安装完成后运行不依赖在线翻译或下载服务。`test_engine.py` 与 `benchmark.py` 禁止 socket connect，用真实模型与词典验证离线运行。网络下载失败可重新执行准备命令，已有资源先校验再复用。默认系统 Python 3.8 不适用，命令必须使用上述独立环境。
|
||||
|
||||
资源版本、固定下载 URL 和 SHA256 见 `resources.json`，Python 依赖固定于 `requirements.lock`。大文件只保存在忽略的 `.local/nlp-resources`。
|
||||
|
||||
许可与来源：
|
||||
|
||||
- [spaCy 3.8.7](https://pypi.org/pypi/spacy/3.8.7/json) 与 [en_core_web_sm 3.8.0](https://github.com/explosion/spacy-models/releases/tag/en_core_web_sm-3.8.0)：MIT；安装包保留其许可证。模型的 POS/lemma 组件保留，parser/NER 在本小样中停用；不输出句界。
|
||||
- [NLTK](https://github.com/nltk/nltk/blob/3.9.2/LICENSE.txt)：Apache-2.0；只读取本地词典文件，无隐式 downloader。
|
||||
- [Princeton WordNet 3.0](https://wordnet.princeton.edu/license-and-commercial-use)：WordNet 3.0 许可证；下载 ZIP 完整保留 `wordnet/LICENSE`、版权及免责声明。英英释义，不提供中文翻译。再分发必须保留许可声明。
|
||||
- [WordNet 原生格式](https://wordnet.princeton.edu/documentation/wndb5wn) 是 index/data/exception 文件，本小样直接读取 ZIP 中原始文件，不使用 go-admin 的系统枚举字典。
|
||||
|
||||
`benchmark-result.json` 是本机虚构语料的测量样本，不代表一般准确率或生产性能承诺。
|
||||
@@ -0,0 +1,69 @@
|
||||
import {validateTokens,latestOnly} from './view.mjs'
|
||||
const source=document.querySelector('#source'), reading=document.querySelector('#reading'), definition=document.querySelector('#definition'), notice=document.querySelector('#notice'), position=document.querySelector('#position'), query=document.querySelector('#query'), analyzeButton=document.querySelector('#analyze')
|
||||
const analysis=latestOnly(), lookup=latestOnly()
|
||||
const labels={exact:'直接命中',lemma:'按原形查询',not_found:'未找到释义',resource_missing:'本地资源缺失',invalid_input:'输入格式无效或文本超过限制',internal_error:'暂时无法处理，请重试',too_large:'文本超过限制'}
|
||||
async function post(path,payload) {
|
||||
const response=await fetch(path,{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(payload)})
|
||||
const data=await response.json()
|
||||
if(!response.ok) throw new Error(labels[data.status] || '请求失败，请重试')
|
||||
return data
|
||||
}
|
||||
function element(tag,text) {const node=document.createElement(tag);node.textContent=text;return node}
|
||||
async function showWord(surface,lemma='',token=null) {
|
||||
const version=lookup.next()
|
||||
query.value=surface
|
||||
definition.replaceChildren(element('p','查询中…'))
|
||||
position.textContent=token ? JSON.stringify({text:token.text,lemma:token.lemma,code_point:[token.start_cp,token.end_cp],utf8:[token.start_utf8,token.end_utf8],utf16:[token.start_utf16,token.end_utf16]},null,2) : '手动查询，无原文位置'
|
||||
try {
|
||||
const data=await post('/lookup',{surface,lemma})
|
||||
if(!lookup.current(version)) return
|
||||
definition.replaceChildren(element('h3',data.matched_form || surface),element('small',labels[data.status] || data.status))
|
||||
if(data.entries.length) {
|
||||
const list=document.createElement('ol')
|
||||
for(const entry of data.entries) {
|
||||
const item=element('li',entry.definition)
|
||||
item.prepend(element('small',entry.pos+' · '))
|
||||
for(const example of entry.examples) item.append(element('p',example))
|
||||
list.append(item)
|
||||
}
|
||||
definition.append(list)
|
||||
}
|
||||
} catch(error) {if(lookup.current(version)) definition.textContent=error.message}
|
||||
}
|
||||
function resetReading() {
|
||||
lookup.next()
|
||||
reading.replaceChildren()
|
||||
definition.textContent='点击文中的单词。'
|
||||
position.textContent='尚未选择单词'
|
||||
}
|
||||
async function analyze() {
|
||||
const version=analysis.next(), text=source.value
|
||||
resetReading()
|
||||
analyzeButton.disabled=true
|
||||
notice.textContent='正在分析…'
|
||||
try {
|
||||
const result=await post('/analyze',{text})
|
||||
if(!analysis.current(version)) return
|
||||
validateTokens(text,result.tokens)
|
||||
const fragment=document.createDocumentFragment()
|
||||
for(const token of result.tokens) {
|
||||
if(token.kind==='word') {
|
||||
const button=element('button',token.text)
|
||||
button.type='button'
|
||||
button.addEventListener('click',()=>{
|
||||
reading.querySelector('.selected')?.classList.remove('selected')
|
||||
button.classList.add('selected')
|
||||
showWord(token.text,token.lemma,token)
|
||||
})
|
||||
fragment.append(button)
|
||||
} else fragment.append(document.createTextNode(token.text))
|
||||
}
|
||||
reading.replaceChildren(fragment)
|
||||
notice.textContent=text ? '已分析 · 点击单词查词' : '请输入阅读文本'
|
||||
} catch(error) {if(analysis.current(version)) notice.textContent=error.message}
|
||||
finally {if(analysis.current(version)) analyzeButton.disabled=false}
|
||||
}
|
||||
source.addEventListener('input',()=>{analysis.next();resetReading();analyzeButton.disabled=false;notice.textContent='文本已修改，请重新分析'})
|
||||
analyzeButton.addEventListener('click',analyze)
|
||||
document.querySelector('#lookup-form').addEventListener('submit',event=>{event.preventDefault();showWord(query.value)})
|
||||
analyze()
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Loopback-only, ephemeral English experiment. Not a production API."""
|
||||
import argparse
|
||||
import json
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
STATIC = Path(__file__).parent
|
||||
ROOT = STATIC.parent.parent
|
||||
|
||||
|
||||
def create_server(engine, port=5183):
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def setup(self):
|
||||
super().setup()
|
||||
self.connection.settimeout(10)
|
||||
|
||||
def log_message(self, *_args):
|
||||
pass # Do not retain input or query text in access logs.
|
||||
|
||||
def reply(self, code, body, mime='application/json; charset=utf-8'):
|
||||
if not isinstance(body, bytes):
|
||||
body = json.dumps(body, ensure_ascii=True).encode('utf-8')
|
||||
self.send_response(code)
|
||||
self.send_header('Content-Type', mime)
|
||||
self.send_header('Content-Length', str(len(body)))
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.send_header('X-Content-Type-Options', 'nosniff')
|
||||
self.send_header('Content-Security-Policy', "default-src 'self'; style-src 'self'; script-src 'self'; connect-src 'self'; frame-ancestors 'none'; base-uri 'none'")
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def local_request(self):
|
||||
host = self.headers.get('Host')
|
||||
allowed = {f'127.0.0.1:{self.server.server_port}', f'localhost:{self.server.server_port}'}
|
||||
origin = self.headers.get('Origin')
|
||||
if host not in allowed or (origin is not None and origin not in {'http://' + h for h in allowed}):
|
||||
self.reply(403, {'status': 'forbidden'})
|
||||
return False
|
||||
return True
|
||||
|
||||
def do_GET(self):
|
||||
if not self.local_request():
|
||||
return
|
||||
files = {'/': ('index.html', 'text/html'), '/app.mjs': ('app.mjs', 'text/javascript'), '/view.mjs': ('view.mjs', 'text/javascript'), '/style.css': ('style.css', 'text/css')}
|
||||
if self.path not in files:
|
||||
self.reply(404, {'status': 'not_found'})
|
||||
return
|
||||
name, mime = files[self.path]
|
||||
self.reply(200, (STATIC / name).read_bytes(), mime + '; charset=utf-8')
|
||||
|
||||
def do_POST(self):
|
||||
if not self.local_request():
|
||||
return
|
||||
if self.path not in {'/analyze', '/lookup'}:
|
||||
self.reply(404, {'status': 'not_found'})
|
||||
return
|
||||
try:
|
||||
length = int(self.headers.get('Content-Length', '0'))
|
||||
except ValueError:
|
||||
self.reply(400, {'status': 'invalid_input'})
|
||||
return
|
||||
if length > 1_000_000:
|
||||
self.reply(413, {'status': 'too_large'})
|
||||
return
|
||||
if length <= 0:
|
||||
self.reply(400, {'status': 'invalid_input'})
|
||||
return
|
||||
if self.headers.get_content_type() != 'application/json':
|
||||
self.reply(415, {'status': 'invalid_content_type'})
|
||||
return
|
||||
try:
|
||||
payload = json.loads(self.rfile.read(length))
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError('object required')
|
||||
key = 'text' if self.path == '/analyze' else 'surface'
|
||||
if not isinstance(payload.get(key), str):
|
||||
raise ValueError('string required')
|
||||
if self.path == '/analyze':
|
||||
result = engine.analyze(payload['text'])
|
||||
else:
|
||||
lemma = payload.get('lemma', '')
|
||||
if not isinstance(lemma, str) or len(lemma) > 200 or len(payload['surface']) > 200:
|
||||
raise ValueError('invalid query')
|
||||
result = engine.lookup(payload['surface'], lemma)
|
||||
self.reply(200, result)
|
||||
except (ValueError, TypeError, UnicodeError):
|
||||
self.reply(400, {'status': 'invalid_input'})
|
||||
except Exception as exc:
|
||||
# ResourceMissing is deliberately exposed without filesystem paths.
|
||||
from engine import ResourceMissing
|
||||
if isinstance(exc, ResourceMissing):
|
||||
self.reply(503, {'status': 'resource_missing', 'resource': exc.resource})
|
||||
else:
|
||||
self.reply(500, {'status': 'internal_error'})
|
||||
|
||||
server = HTTPServer(('127.0.0.1', port), Handler)
|
||||
return server
|
||||
|
||||
|
||||
def main():
|
||||
from engine import Engine
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--port', type=int, default=5183)
|
||||
parser.add_argument('--resources', type=Path, default=ROOT / '.local/nlp-resources')
|
||||
args = parser.parse_args()
|
||||
server = create_server(Engine(args.resources), args.port)
|
||||
print(f'LexGo English experiment: http://127.0.0.1:{server.server_port}/', flush=True)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
server.server_close()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,181 @@
|
||||
{
|
||||
"measured_at_utc": "2026-09-10T12:55:12.016850+00:00",
|
||||
"environment": {
|
||||
"python": "3.12.12",
|
||||
"platform": "Windows-10-10.0.19044-SP0",
|
||||
"processor": "Intel64 Family 6 Model 140 Stepping 1, GenuineIntel",
|
||||
"logical_cpus": 8,
|
||||
"packages": {
|
||||
"spacy": "3.8.7",
|
||||
"nltk": "3.9.2",
|
||||
"en-core-web-sm": "3.8.0"
|
||||
}
|
||||
},
|
||||
"offline": true,
|
||||
"cold_engine_load_ms": 2355.1940000616014,
|
||||
"text_codepoints": 100000,
|
||||
"text_utf8_bytes": 106095,
|
||||
"quality": {
|
||||
"cases": [
|
||||
{
|
||||
"sentence": "She went home.",
|
||||
"surface": "went",
|
||||
"expected": "go",
|
||||
"spacy": "go",
|
||||
"baseline": "go",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "The children ate apples.",
|
||||
"surface": "children",
|
||||
"expected": "child",
|
||||
"spacy": "child",
|
||||
"baseline": "child",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "The children ate apples.",
|
||||
"surface": "ate",
|
||||
"expected": "eat",
|
||||
"spacy": "eat",
|
||||
"baseline": "ate",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "The dogs ran quickly.",
|
||||
"surface": "dogs",
|
||||
"expected": "dog",
|
||||
"spacy": "dog",
|
||||
"baseline": "dog",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "The dogs ran quickly.",
|
||||
"surface": "ran",
|
||||
"expected": "run",
|
||||
"spacy": "run",
|
||||
"baseline": "run",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "I saw a bird.",
|
||||
"surface": "saw",
|
||||
"expected": "see",
|
||||
"spacy": "see",
|
||||
"baseline": "saw",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "The saw is sharp.",
|
||||
"surface": "saw",
|
||||
"expected": "saw",
|
||||
"spacy": "saw",
|
||||
"baseline": "saw",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "She leaves today.",
|
||||
"surface": "leaves",
|
||||
"expected": "leave",
|
||||
"spacy": "leave",
|
||||
"baseline": "leaf",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "The leaves fell.",
|
||||
"surface": "leaves",
|
||||
"expected": "leaf",
|
||||
"spacy": "leave",
|
||||
"baseline": "leaf",
|
||||
"spacy_correct": false,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "They are reading books.",
|
||||
"surface": "reading",
|
||||
"expected": "read",
|
||||
"spacy": "read",
|
||||
"baseline": "reading",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "He was better yesterday.",
|
||||
"surface": "was",
|
||||
"expected": "be",
|
||||
"spacy": "be",
|
||||
"baseline": "wa",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "She can't go.",
|
||||
"surface": "n't",
|
||||
"expected": "not",
|
||||
"spacy": "not",
|
||||
"baseline": null,
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
}
|
||||
],
|
||||
"total": 12,
|
||||
"spacy_correct": 11,
|
||||
"baseline_correct": 6
|
||||
},
|
||||
"measurements": {
|
||||
"first_analysis": {
|
||||
"repeats": 1,
|
||||
"median_ms": 2.9011000879108906,
|
||||
"p95_ms": 2.9011000879108906
|
||||
},
|
||||
"first_lookup": {
|
||||
"repeats": 1,
|
||||
"median_ms": 78.619199921377,
|
||||
"p95_ms": 78.619199921377
|
||||
},
|
||||
"spacy_100k": {
|
||||
"repeats": 3,
|
||||
"median_ms": 1432.92090005707,
|
||||
"p95_ms": 1499.0439999382943,
|
||||
"codepoints_per_second": 69787.52281163407
|
||||
},
|
||||
"baseline_100k": {
|
||||
"repeats": 3,
|
||||
"median_ms": 336.1744999419898,
|
||||
"p95_ms": 343.62249996047467,
|
||||
"codepoints_per_second": 297464.56086721626
|
||||
},
|
||||
"lookup": {
|
||||
"dog": {
|
||||
"repeats": 100,
|
||||
"median_ms": 0.014899997040629387,
|
||||
"p95_ms": 0.022999942302703857
|
||||
},
|
||||
"went": {
|
||||
"repeats": 100,
|
||||
"median_ms": 0.021250045392662287,
|
||||
"p95_ms": 0.02929999027401209
|
||||
},
|
||||
"zzzxqvfiction": {
|
||||
"repeats": 100,
|
||||
"median_ms": 0.003600027412176132,
|
||||
"p95_ms": 0.004999921657145023
|
||||
}
|
||||
}
|
||||
},
|
||||
"limitations": [
|
||||
"Synthetic microbenchmark; 12 selected cases do not establish general accuracy.",
|
||||
"Baseline excludes UTF-8/UTF-16 conversion; spaCy timing includes full analyze contract.",
|
||||
"Repeated queries are warm-process; no claim about production concurrency.",
|
||||
"Cold engine load includes dependency imports but OS filesystem caches may be warm.",
|
||||
"POS and sense disambiguation are not provided by dictionary lookup."
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Offline synthetic measurements; stdout is one JSON document, no text input log."""
|
||||
import argparse
|
||||
from datetime import datetime, timezone
|
||||
import importlib.metadata
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import re
|
||||
import statistics
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
from engine import Engine
|
||||
|
||||
|
||||
CASES = [
|
||||
('She went home.', 'went', 'go'),
|
||||
('The children ate apples.', 'children', 'child'),
|
||||
('The children ate apples.', 'ate', 'eat'),
|
||||
('The dogs ran quickly.', 'dogs', 'dog'),
|
||||
('The dogs ran quickly.', 'ran', 'run'),
|
||||
('I saw a bird.', 'saw', 'see'),
|
||||
('The saw is sharp.', 'saw', 'saw'),
|
||||
('She leaves today.', 'leaves', 'leave'),
|
||||
('The leaves fell.', 'leaves', 'leaf'),
|
||||
('They are reading books.', 'reading', 'read'),
|
||||
('He was better yesterday.', 'was', 'be'),
|
||||
("She can't go.", "n't", 'not'),
|
||||
]
|
||||
|
||||
|
||||
def summary(samples):
|
||||
values = sorted(samples)
|
||||
return {'repeats': len(values), 'median_ms': statistics.median(values) * 1000,
|
||||
'p95_ms': values[max(0, math.ceil(len(values) * .95) - 1)] * 1000}
|
||||
|
||||
|
||||
def measure(function, repeats):
|
||||
samples = []
|
||||
for _ in range(repeats):
|
||||
start = time.perf_counter()
|
||||
function()
|
||||
samples.append(time.perf_counter() - start)
|
||||
return summary(samples)
|
||||
|
||||
|
||||
def run(args):
|
||||
start = time.perf_counter()
|
||||
engine = Engine(args.resources)
|
||||
load_seconds = time.perf_counter() - start
|
||||
if engine.nlp is None or engine.wordnet is None:
|
||||
raise RuntimeError(f'Resources unavailable: model={engine.model_error}, wordnet={engine.wordnet_error}')
|
||||
measured = {}
|
||||
measured['first_analysis'] = measure(lambda: engine.analyze('She went home.'), 1)
|
||||
measured['first_lookup'] = measure(lambda: engine.lookup('dog'), 1)
|
||||
baseline_pattern = re.compile(r"\w+(?:['’]\w+)*|\s+|[^\w\s]", re.UNICODE)
|
||||
# Lower-cost baseline: regex spans and context-free WordNet morphology.
|
||||
def baseline(text):
|
||||
return [(match.group(), engine.wordnet.morphy(match.group().lower()) or match.group().lower(),
|
||||
match.start(), match.end()) for match in baseline_pattern.finditer(text)]
|
||||
|
||||
quality = []
|
||||
for sentence, surface, expected in CASES:
|
||||
contextual = next((token['lemma'] for token in engine.analyze(sentence)['tokens']
|
||||
if token['text'] == surface), None)
|
||||
simple = next((lemma for token, lemma, _, _ in baseline(sentence) if token == surface), None)
|
||||
quality.append(dict(sentence=sentence, surface=surface, expected=expected,
|
||||
spacy=contextual, baseline=simple,
|
||||
spacy_correct=contextual == expected, baseline_correct=simple == expected))
|
||||
seed = "She went home. The children ate apples. I saw a bird. The leaves fell. Café 😀 e\u0301\r\n"
|
||||
text = (seed * (100000 // len(seed) + 1))[:100000]
|
||||
for name, function in [('spacy', engine.analyze), ('baseline', baseline)]:
|
||||
metrics = measure(lambda: function(text), args.text_repeats)
|
||||
metrics['codepoints_per_second'] = len(text) / (metrics['median_ms'] / 1000)
|
||||
measured[name + '_100k'] = metrics
|
||||
queries = [('dog', ''), ('went', 'go'), ('zzzxqvfiction', '')]
|
||||
measured['lookup'] = {surface: measure(lambda: engine.lookup(surface, lemma), args.query_repeats)
|
||||
for surface, lemma in queries}
|
||||
return {
|
||||
'measured_at_utc': datetime.now(timezone.utc).isoformat(),
|
||||
'environment': {'python': platform.python_version(), 'platform': platform.platform(),
|
||||
'processor': platform.processor(), 'logical_cpus': os.cpu_count(),
|
||||
'packages': {name: importlib.metadata.version(name)
|
||||
for name in ('spacy', 'nltk', 'en-core-web-sm')}},
|
||||
'offline': True, 'cold_engine_load_ms': load_seconds * 1000,
|
||||
'text_codepoints': len(text), 'text_utf8_bytes': len(text.encode()),
|
||||
'quality': {'cases': quality, 'total': len(quality),
|
||||
'spacy_correct': sum(case['spacy_correct'] for case in quality),
|
||||
'baseline_correct': sum(case['baseline_correct'] for case in quality)},
|
||||
'measurements': measured,
|
||||
'limitations': [
|
||||
'Synthetic microbenchmark; 12 selected cases do not establish general accuracy.',
|
||||
'Baseline excludes UTF-8/UTF-16 conversion; spaCy timing includes full analyze contract.',
|
||||
'Repeated queries are warm-process; no claim about production concurrency.',
|
||||
'Cold engine load includes dependency imports but OS filesystem caches may be warm.',
|
||||
'POS and sense disambiguation are not provided by dictionary lookup.',
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--resources', type=Path, default=Path(__file__).resolve().parents[2] / '.local/nlp-resources')
|
||||
parser.add_argument('--text-repeats', type=int, default=3)
|
||||
parser.add_argument('--query-repeats', type=int, default=100)
|
||||
args = parser.parse_args()
|
||||
if args.text_repeats < 1 or args.query_repeats < 1:
|
||||
parser.error('repeat counts must be positive')
|
||||
with patch('socket.socket.connect', side_effect=AssertionError('network forbidden')):
|
||||
print(json.dumps(run(args), ensure_ascii=True, indent=2))
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Isolated English experiment. Resources are local; no downloader is used."""
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import unicodedata
|
||||
import warnings
|
||||
import zipfile
|
||||
|
||||
|
||||
class ResourceMissing(RuntimeError):
|
||||
def __init__(self, resource):
|
||||
self.resource = resource
|
||||
super().__init__(f'Local {resource} resource is unavailable')
|
||||
|
||||
|
||||
def validate_text(text):
|
||||
if not isinstance(text, str):
|
||||
raise TypeError('text must be a string')
|
||||
if len(text) > 100000:
|
||||
raise ValueError('text exceeds 100000 code points')
|
||||
if any(0xD800 <= ord(char) <= 0xDFFF for char in text):
|
||||
raise ValueError('text contains an unpaired surrogate')
|
||||
|
||||
|
||||
def lookup_form(text):
|
||||
validate_text(text)
|
||||
return unicodedata.normalize('NFC', text.casefold()).replace('’', "'").replace('‘', "'")
|
||||
|
||||
|
||||
class Engine:
|
||||
def __init__(self, resource_dir: Path, model_name='en_core_web_sm'):
|
||||
self.nlp = None
|
||||
self.wordnet = None
|
||||
self.model_error = None
|
||||
self.wordnet_error = None
|
||||
try:
|
||||
import spacy
|
||||
self.nlp = spacy.load(model_name, disable=['parser', 'ner'])
|
||||
except (ImportError, OSError, ValueError) as error:
|
||||
self.model_error = type(error).__name__
|
||||
try:
|
||||
from nltk.corpus.reader import WordNetCorpusReader
|
||||
from nltk.data import ZipFilePathPointer
|
||||
|
||||
class EnglishWordNet30Reader(WordNetCorpusReader):
|
||||
def map_wn(self, version='wordnet'):
|
||||
# NLTK's default cross-version OMW mapping loads a global
|
||||
# corpus. English-only WordNet 3.0 needs no such mapping.
|
||||
if self.get_version() != '3.0':
|
||||
raise ValueError('This experiment requires WordNet 3.0')
|
||||
return None
|
||||
|
||||
root = ZipFilePathPointer(str(Path(resource_dir) / 'wordnet.zip'), 'wordnet/')
|
||||
with warnings.catch_warnings():
|
||||
warnings.filterwarnings('ignore', message='The multilingual functions are not available with this Wordnet version', category=UserWarning)
|
||||
self.wordnet = EnglishWordNet30Reader(root, None)
|
||||
except (ImportError, OSError, LookupError, ValueError, zipfile.BadZipFile) as error:
|
||||
self.wordnet_error = type(error).__name__
|
||||
|
||||
def analyze(self, text):
|
||||
validate_text(text)
|
||||
if self.nlp is None:
|
||||
raise ResourceMissing('model')
|
||||
# Prefix tables make conversion linear even for long Unicode documents.
|
||||
utf8 = [0]
|
||||
utf16 = [0]
|
||||
for char in text:
|
||||
utf8.append(utf8[-1] + len(char.encode('utf-8')))
|
||||
utf16.append(utf16[-1] + (2 if ord(char) > 0xFFFF else 1))
|
||||
tokens = []
|
||||
|
||||
def append(start, end, lemma, kind):
|
||||
tokens.append(dict(text=text[start:end], lemma=lemma, kind=kind,
|
||||
start_cp=start, end_cp=end,
|
||||
start_utf8=utf8[start], end_utf8=utf8[end],
|
||||
start_utf16=utf16[start], end_utf16=utf16[end]))
|
||||
|
||||
cursor = 0
|
||||
for token in self.nlp(text):
|
||||
if token.idx > cursor:
|
||||
append(cursor, token.idx, '', 'space')
|
||||
end = token.idx + len(token.text)
|
||||
kind = 'space' if token.is_space else 'punctuation' if token.is_punct else 'word'
|
||||
append(token.idx, end, token.lemma_ if kind == 'word' else '', kind)
|
||||
cursor = end
|
||||
if cursor < len(text):
|
||||
append(cursor, len(text), '', 'space')
|
||||
return dict(status='ok', contract_version='english-spike-v1', original_text=text,
|
||||
text_sha256=hashlib.sha256(text.encode('utf-8')).hexdigest(), tokens=tokens)
|
||||
|
||||
def _exact_entries(self, form):
|
||||
# NLTK 3.9.2 synsets() applies morphy even with check_exceptions=False.
|
||||
# Read its loaded index directly to keep exact and explicit lemma distinct.
|
||||
index = self.wordnet._lemma_pos_offset_map.get(form, {})
|
||||
entries = []
|
||||
for pos in ('n', 'v', 'a', 'r'):
|
||||
for offset in index.get(pos, []):
|
||||
synset = self.wordnet.synset_from_pos_and_offset(pos, offset)
|
||||
entries.append(dict(lemma=form, pos=synset.pos(),
|
||||
definition=synset.definition(), examples=synset.examples()))
|
||||
if len(entries) == 12:
|
||||
return entries
|
||||
return entries
|
||||
|
||||
def lookup(self, surface, lemma=''):
|
||||
form = lookup_form(surface)
|
||||
fallback = lookup_form(lemma)
|
||||
result = dict(query=surface, matched_form=None, entries=[])
|
||||
if self.wordnet is None:
|
||||
return dict(result, status='resource_missing', resource='wordnet')
|
||||
for candidate, status in ((form, 'exact'), (fallback, 'lemma')):
|
||||
if not candidate:
|
||||
continue
|
||||
entries = self._exact_entries(candidate)
|
||||
if entries:
|
||||
return dict(result, status=status, matched_form=candidate, entries=entries)
|
||||
return dict(result, status='not_found')
|
||||
@@ -0,0 +1,12 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>LexGo · 英语验证</title><link rel="stylesheet" href="/style.css"></head>
|
||||
<body>
|
||||
<header><a class="brand" href="/">LexGo<span>.</span></a><span class="badge">英语 · 技术验证</span></header>
|
||||
<main>
|
||||
<section class="input-section"><label for="source">阅读文本</label><textarea id="source" spellcheck="false" maxlength="100000">Mira’s well-known dogs went home. She can't wait.
|
||||
The children were running beside a café. 🙂 Café!</textarea><div class="actions"><span>仅本机处理 · 不保存</span><button id="analyze">分析文本</button></div></section>
|
||||
<p id="notice" role="status" aria-live="polite"></p>
|
||||
<div class="workspace"><section class="paper" aria-label="阅读结果"><h1>阅读</h1><div id="reading"></div></section><aside aria-label="词典"><div class="dictionary-header"><h2>本地词典</h2><span>WordNet 3.0</span></div><form id="lookup-form"><label class="sr-only" for="query">查询单词</label><input id="query" maxlength="200" placeholder="输入英语单词" autocomplete="off"><button>查询</button></form><div id="definition" aria-live="polite">点击文中的单词。</div><details><summary>原文位置</summary><pre id="position">尚未选择单词</pre></details></aside></div>
|
||||
</main><footer>独立验证小样 · 英语释义</footer><script type="module" src="/app.mjs"></script>
|
||||
</body></html>
|
||||
@@ -0,0 +1,47 @@
|
||||
annotated-doc==0.0.5
|
||||
annotated-types==0.8.0
|
||||
blis==1.3.3
|
||||
catalogue==2.0.10
|
||||
certifi==2026.7.22
|
||||
charset-normalizer==3.5.1
|
||||
click==8.5.0
|
||||
cloudpathlib==0.25.0
|
||||
cloudpickle==3.1.2
|
||||
colorama==0.4.6
|
||||
confection==0.1.5
|
||||
cymem==2.0.13
|
||||
idna==3.19
|
||||
jinja2==3.1.6
|
||||
joblib==1.6.0
|
||||
langcodes==3.5.1
|
||||
markdown-it-py==4.2.0
|
||||
markupsafe==3.0.3
|
||||
mdurl==0.1.2
|
||||
murmurhash==1.0.15
|
||||
nltk==3.9.2
|
||||
numpy==2.5.3
|
||||
packaging==26.3
|
||||
preshed==3.0.13
|
||||
pydantic==2.13.5
|
||||
pydantic-core==2.46.5
|
||||
pygments==2.21.0
|
||||
regex==2026.9.10
|
||||
requests==2.34.2
|
||||
rich==15.0.0
|
||||
setuptools==84.0.0
|
||||
shellingham==1.5.4
|
||||
smart-open==7.7.1
|
||||
spacy==3.8.7
|
||||
spacy-legacy==3.0.12
|
||||
spacy-loggers==1.0.5
|
||||
srsly==2.5.3
|
||||
thinc==8.3.11
|
||||
tqdm==4.70.0
|
||||
typer==0.27.2
|
||||
typer-slim==0.24.0
|
||||
typing-extensions==4.16.0
|
||||
typing-inspection==0.4.4
|
||||
urllib3==2.7.0
|
||||
wasabi==1.1.3
|
||||
weasel==0.4.3
|
||||
wrapt==2.4.0
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"python": "3.12.12",
|
||||
"spacy": "3.8.7",
|
||||
"nltk": "3.9.2",
|
||||
"model": {
|
||||
"name": "en_core_web_sm",
|
||||
"version": "3.8.0",
|
||||
"file": "en_core_web_sm-3.8.0-py3-none-any.whl",
|
||||
"url": "https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl",
|
||||
"sha256": "1932429db727d4bff3deed6b34cfc05df17794f4a52eeb26cf8928f7c1a0fb85",
|
||||
"license": "MIT"
|
||||
},
|
||||
"dictionary": {
|
||||
"name": "Princeton WordNet",
|
||||
"version": "3.0",
|
||||
"file": "wordnet.zip",
|
||||
"url": "https://raw.githubusercontent.com/nltk/nltk_data/96f9b3252457a2b97e52aec64c3dfceeb5c312d5/packages/corpora/wordnet.zip",
|
||||
"sha256": "cbda5ea6eef7f36a97a43d4a75f85e07fccbb4f23657d27b4ccbc93e2646ab59",
|
||||
"license": "WordNet 3.0 License (included in ZIP: wordnet/LICENSE)"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Explicit network-only preparation; runtime never calls this script."""
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import urllib.request
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
DEST = HERE.parent.parent / '.local/nlp-resources'
|
||||
|
||||
|
||||
def main():
|
||||
manifest = json.loads((HERE / 'resources.json').read_text(encoding='utf-8'))
|
||||
DEST.mkdir(parents=True, exist_ok=True)
|
||||
for key in ('model', 'dictionary'):
|
||||
item = manifest[key]
|
||||
target = DEST / item['file']
|
||||
if target.exists() and hashlib.sha256(target.read_bytes()).hexdigest() == item['sha256']:
|
||||
print(key + ': checksum verified', flush=True)
|
||||
continue
|
||||
req = urllib.request.Request(item['url'], headers={'User-Agent': 'LexGo-English-Spike/1'})
|
||||
with urllib.request.urlopen(req, timeout=120) as response:
|
||||
data = response.read(32 * 1024 * 1024 + 1)
|
||||
if hashlib.sha256(data).hexdigest() != item['sha256']:
|
||||
raise RuntimeError(key + ': checksum mismatch; resource not installed')
|
||||
target.write_bytes(data)
|
||||
print(key + ': downloaded and checksum verified', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
:root{color:#253b32;background:#f5f4ef;font-family:"Segoe UI","Microsoft YaHei",sans-serif;font-synthesis:none}*{box-sizing:border-box}body{margin:0}header{height:78px;border-bottom:1px solid #dfe3da;padding:0 5%;display:flex;align-items:center;justify-content:space-between;background:#fff}.brand{font-size:29px;font-weight:700;text-decoration:none;color:inherit;letter-spacing:-1px}.brand span{color:#3b8060}.badge{font-size:13px;color:#66776a}main{max-width:1260px;margin:35px auto;padding:0 28px}.input-section{border-bottom:1px solid #d5ddd4;padding-bottom:25px}label{display:block;font-weight:600;margin-bottom:12px}textarea{display:block;width:100%;min-height:135px;resize:vertical;border:1px solid #cbd5cb;border-radius:8px;background:#fff;padding:16px;color:#263b30;font:18px/1.7 Georgia,serif}textarea:focus,input:focus,button:focus-visible{outline:2px solid #538967;outline-offset:3px}.actions{display:flex;justify-content:space-between;align-items:center;margin-top:12px}.actions span,footer{font-size:12px;color:#738075}button{background:#2c6347;color:white;border:0;border-radius:5px;padding:10px 20px;cursor:pointer;font:inherit}button:disabled{opacity:.55;cursor:wait}#notice{font-size:14px;min-height:20px}.workspace{display:grid;grid-template-columns:minmax(0,1fr) 330px;gap:25px}.paper,aside{background:#fff;border:1px solid #e0e5dc;border-radius:8px}.paper{padding:28px 32px;min-height:340px}h1{font-size:13px;letter-spacing:2px;color:#6f7e72;margin:0 0 26px}#reading{font:23px/1.95 Georgia,"Times New Roman",serif;white-space:pre-wrap;overflow-wrap:anywhere}#reading button{font:inherit;color:inherit;padding:0;border-radius:2px;background:transparent;text-align:left}#reading button:hover,#reading button.selected{background:#e1edcf;box-shadow:0 2px #658447}aside{padding:24px}.dictionary-header{display:flex;justify-content:space-between;align-items:center;margin-bottom:20px}.dictionary-header h2{font-size:17px;margin:0}.dictionary-header span{font-size:11px;color:#7e887f}form{display:flex;gap:7px;margin-bottom:22px}input{min-width:0;width:100%;padding:9px;border:1px solid #cbd5cb;border-radius:4px;font:inherit}form button{padding:9px 12px;white-space:nowrap}#definition{font-size:14px;line-height:1.7;overflow-wrap:anywhere}#definition h3{font:27px Georgia,serif;margin:0 0 8px}#definition ol{padding-left:21px}#definition li{margin-bottom:13px}#definition small{color:#637567}details{margin-top:25px;border-top:1px solid #e1e6de;padding-top:15px;color:#7a847d;font-size:12px}summary{cursor:pointer}pre{white-space:pre-wrap;overflow-wrap:anywhere;font-size:11px}footer{text-align:center;padding:35px}.sr-only{position:absolute;width:1px;height:1px;overflow:hidden;clip-path:inset(50%)}@media(max-width:750px){main{padding:0 16px;margin-top:20px}.workspace{grid-template-columns:1fr}.paper{padding:24px;min-height:230px}#reading{font-size:21px}.actions span{font-size:11px}}
|
||||
@@ -0,0 +1,72 @@
|
||||
import http.client
|
||||
import json
|
||||
import threading
|
||||
import unittest
|
||||
from unittest.mock import Mock
|
||||
|
||||
from app import create_server
|
||||
|
||||
|
||||
class HTTPTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.engine = Mock()
|
||||
self.engine.analyze.return_value = {'status': 'ok', 'tokens': []}
|
||||
self.engine.lookup.return_value = {'status': 'not_found', 'entries': []}
|
||||
self.server = create_server(self.engine, 0)
|
||||
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
self.port = self.server.server_port
|
||||
|
||||
def tearDown(self):
|
||||
self.server.shutdown()
|
||||
self.server.server_close()
|
||||
self.thread.join()
|
||||
|
||||
def call(self, method, path, body=None, headers=None):
|
||||
c = http.client.HTTPConnection('127.0.0.1', self.port, timeout=3)
|
||||
c.request(method, path, body, headers or {})
|
||||
r = c.getresponse()
|
||||
result = r.status, r.read(), dict(r.getheaders())
|
||||
c.close()
|
||||
return result
|
||||
|
||||
def test_local_page_and_no_arbitrary_file_access(self):
|
||||
status, body, headers = self.call('GET', '/')
|
||||
self.assertEqual(status, 200)
|
||||
self.assertIn(b'LexGo', body)
|
||||
self.assertIn('Content-Security-Policy', headers)
|
||||
self.assertEqual(self.call('GET', '/../../.env.local')[0], 404)
|
||||
|
||||
def test_json_analyze_and_lookup(self):
|
||||
self.assertEqual(self.call('POST', '/analyze', json.dumps({'text': 'Hello'}), {'Content-Type': 'application/json'})[0], 200)
|
||||
self.engine.analyze.assert_called_once_with('Hello')
|
||||
self.assertEqual(self.call('POST', '/lookup', json.dumps({'surface': 'went', 'lemma': 'go'}), {'Content-Type': 'application/json'})[0], 200)
|
||||
self.engine.lookup.assert_called_once_with('went', 'go')
|
||||
|
||||
def test_reject_cross_origin_and_rebinding(self):
|
||||
for headers in ({'Origin': 'https://evil.example'}, {'Host': 'evil.example'}):
|
||||
self.assertEqual(self.call('POST', '/analyze', '{}', headers)[0], 403)
|
||||
self.engine.analyze.assert_not_called()
|
||||
|
||||
def test_invalid_payload_and_size(self):
|
||||
for payload in ('[]', '{}', '{', '{"text":42}'):
|
||||
self.assertEqual(self.call('POST', '/analyze', payload, {'Content-Type': 'application/json'})[0], 400)
|
||||
self.assertEqual(self.call('POST', '/analyze', '{}', {'Content-Type': 'text/plain'})[0], 415)
|
||||
self.assertEqual(self.call('POST', '/analyze', '{}', {'Content-Type': 'application/json', 'Content-Length': '1000001'})[0], 413)
|
||||
|
||||
def test_internal_errors_do_not_echo_input(self):
|
||||
self.engine.analyze.side_effect = RuntimeError('private sample')
|
||||
status, body, _ = self.call('POST', '/analyze', '{"text":"x"}', {'Content-Type': 'application/json'})
|
||||
self.assertEqual(status, 500)
|
||||
self.assertNotIn(b'private sample', body)
|
||||
|
||||
def test_missing_model_is_distinct_from_invalid_input(self):
|
||||
from engine import ResourceMissing
|
||||
self.engine.analyze.side_effect = ResourceMissing('model')
|
||||
status, body, _ = self.call('POST', '/analyze', '{"text":"x"}', {'Content-Type': 'application/json'})
|
||||
self.assertEqual(status, 503)
|
||||
self.assertEqual(json.loads(body), {'status': 'resource_missing', 'resource': 'model'})
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,92 @@
|
||||
"""Run with the isolated Python: -m unittest discover -s spikes/english -v."""
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
try:
|
||||
from engine import Engine, ResourceMissing
|
||||
except ImportError:
|
||||
Engine = None
|
||||
|
||||
RESOURCES = Path(__file__).resolve().parents[2] / '.local/nlp-resources'
|
||||
|
||||
|
||||
class EngineTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.network = patch('socket.socket.connect', side_effect=AssertionError('network forbidden'))
|
||||
cls.network.start()
|
||||
cls.addClassCleanup(cls.network.stop)
|
||||
if Engine:
|
||||
cls.engine = Engine(RESOURCES)
|
||||
|
||||
def setUp(self):
|
||||
self.assertIsNotNone(Engine, 'English engine has not been implemented')
|
||||
|
||||
def test_unicode_partition_and_three_offsets(self):
|
||||
for text in ['', ' \t\r\n', " She went!\r\nDogs’ paws\tcan't. e\u0301 café 😀 中文\u00a0\u200bend ",
|
||||
'well-known mother-in-law 👩💻 👨👩👧👦 🏳️🌈']:
|
||||
with self.subTest(text=text):
|
||||
result = self.engine.analyze(text)
|
||||
self.assertEqual(result['status'], 'ok')
|
||||
self.assertEqual(result['contract_version'], 'english-spike-v1')
|
||||
self.assertEqual(result['original_text'], text)
|
||||
self.assertEqual(result['text_sha256'], hashlib.sha256(text.encode()).hexdigest())
|
||||
tokens = result['tokens']
|
||||
self.assertEqual(''.join(t['text'] for t in tokens), text)
|
||||
cursor = 0
|
||||
for token in tokens:
|
||||
self.assertEqual(token['start_cp'], cursor)
|
||||
cursor = token['end_cp']
|
||||
self.assertGreater(cursor, token['start_cp'])
|
||||
self.assertEqual(text[token['start_cp']:cursor], token['text'])
|
||||
for encoding, unit, suffix in [('utf-8', 1, 'utf8'), ('utf-16-le', 2, 'utf16')]:
|
||||
start, end = token['start_' + suffix], token['end_' + suffix]
|
||||
self.assertEqual(text.encode(encoding)[start*unit:end*unit].decode(encoding), token['text'])
|
||||
self.assertEqual(len(text[:token['start_cp']].encode(encoding)) // unit, start)
|
||||
self.assertIn(token['kind'], ['word', 'space', 'punctuation'])
|
||||
self.assertEqual(cursor, len(text))
|
||||
|
||||
def test_contextual_irregular_lemma(self):
|
||||
tokens = self.engine.analyze('She went home. The children ate apples.')['tokens']
|
||||
lemmas = {t['text']: t['lemma'] for t in tokens}
|
||||
self.assertEqual(lemmas['went'], 'go')
|
||||
self.assertEqual(lemmas['children'], 'child')
|
||||
self.assertEqual(lemmas['ate'], 'eat')
|
||||
|
||||
def test_exact_then_explicit_lemma(self):
|
||||
exact = self.engine.lookup('DOG')
|
||||
self.assertEqual(exact['status'], 'exact')
|
||||
self.assertEqual(exact['matched_form'], 'dog')
|
||||
self.assertTrue(exact['entries'])
|
||||
self.assertLessEqual(len(exact['entries']), 12)
|
||||
self.assertEqual(exact, self.engine.lookup('DOG'))
|
||||
self.assertEqual(self.engine.lookup('went')['status'], 'not_found')
|
||||
lemma = self.engine.lookup('went', 'go')
|
||||
self.assertEqual(lemma['status'], 'lemma')
|
||||
self.assertEqual(lemma['matched_form'], 'go')
|
||||
self.assertEqual(self.engine.lookup('zzzxqvfiction')['status'], 'not_found')
|
||||
|
||||
def test_validation(self):
|
||||
for text in ['x' * 100001, '\ud800']:
|
||||
with self.assertRaises(ValueError):
|
||||
self.engine.analyze(text)
|
||||
with self.assertRaises(TypeError):
|
||||
self.engine.analyze(None)
|
||||
with self.assertRaises(ValueError):
|
||||
self.engine.lookup('\udfff')
|
||||
|
||||
def test_missing_resources_are_not_misses(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
engine = Engine(Path(directory), model_name='nonexistent_english_spike_model')
|
||||
self.assertEqual(engine.lookup('dog')['status'], 'resource_missing')
|
||||
self.assertEqual(engine.lookup('dog')['resource'], 'wordnet')
|
||||
with self.assertRaises(ResourceMissing) as error:
|
||||
engine.analyze('dog')
|
||||
self.assertEqual(error.exception.resource, 'model')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,14 @@
|
||||
export function validateTokens(text, tokens) {
|
||||
let end=0
|
||||
for (const token of tokens) {
|
||||
if (!Number.isInteger(token.start_utf16) || !Number.isInteger(token.end_utf16) || token.start_utf16!==end || token.end_utf16<=end || text.slice(token.start_utf16,token.end_utf16)!==token.text) throw new Error('原文位置校验失败')
|
||||
end=token.end_utf16
|
||||
}
|
||||
if(end!==text.length) throw new Error('原文还原失败')
|
||||
return true
|
||||
}
|
||||
|
||||
export function latestOnly() {
|
||||
let version=0
|
||||
return {next:()=>++version,current:value=>value===version}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import { validateTokens, latestOnly } from './view.mjs'
|
||||
|
||||
test('UTF-16 positions reconstruct emoji and combining characters without normalization', () => {
|
||||
const text = '🙂 Café'
|
||||
const tokens = [{text:'🙂',start_utf16:0,end_utf16:2},{text:' ',start_utf16:2,end_utf16:3},{text:'Café',start_utf16:3,end_utf16:8}]
|
||||
assert.equal(validateTokens(text,tokens), true)
|
||||
assert.throws(() => validateTokens(text,[{text:'🙂',start_utf16:0,end_utf16:1}]))
|
||||
assert.throws(() => validateTokens(text,[]))
|
||||
})
|
||||
test('old analysis and lookup responses cannot replace newer text or selection', () => {
|
||||
const gate=latestOnly()
|
||||
const first=gate.next(), second=gate.next()
|
||||
assert.equal(gate.current(first),false)
|
||||
assert.equal(gate.current(second),true)
|
||||
gate.next()
|
||||
assert.equal(gate.current(second),false)
|
||||
})
|
||||
@@ -0,0 +1,39 @@
|
||||
# #4 阅读选择技术验证
|
||||
|
||||
独立小样，默认英语；虚构章节与释义，保存仅在当前页面内有效，刷新即清空。不是生产阅读器，不连接账号、数据库、词典服务或复习调度。
|
||||
|
||||
从仓库根目录运行：
|
||||
|
||||
```powershell
|
||||
python spikes/selection/serve.py
|
||||
```
|
||||
|
||||
打开 <http://127.0.0.1:5184/>。点词或用鼠标拖选连续文本；使用面板起点/终点按钮调整。正文获得焦点后，←/→ 选择相邻词，Shift+←/→ 调整连续范围，Escape 取消；关闭面板保留阅读位置和键盘续读位置。修改释义后必须点击保存；关闭不会自动保存。
|
||||
|
||||
测试（Node 22.22.1；DOM 测试复用学习端锁定的 jsdom 开发依赖）：
|
||||
|
||||
```powershell
|
||||
pnpm --dir learner install --frozen-lockfile
|
||||
node --test spikes/selection/range.test.mjs spikes/selection/ui.test.mjs
|
||||
node --check spikes/selection/app.mjs
|
||||
```
|
||||
|
||||
`range.mjs` 的 Intl.Segmenter 仅用于验证 UI 选择边界，不替代 #3 的 spaCy 分词。正式集成时必须使用章节原文、内容版本和 NLP token 映射。代码没有复制 LinguaCafe 实现；固定上游版本与四项对照见 [架构 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Architecture-and-Code-Map)，范围/重叠策略见 [业务规则 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Business-Rules-and-Glossary)。
|
||||
|
||||
## 手机测试证据（待补充）
|
||||
|
||||
用户已确认 #4 验收通过，工单已关闭；没有提供具体手机测试记录，以下证据缺口保留。390 CSS 像素的桌面窄屏检查不是真机结果。需要使用实际手机浏览器记录设备、系统、浏览器版本和以下结果：
|
||||
|
||||
1. 普通上下滑动正文，不应意外选词或阻止滚动。
|
||||
2. 长按单词，用系统选择手柄扩展/缩短跨行短语；正文反馈、面板文字和“选中范围”一致。特别检查靠近底部面板的位置，系统菜单/手柄是否被遮挡。
|
||||
3. 使用面板起点/终点按钮调整，验证 emoji、组合字符、连字符及两次出现的 `a small step` 有不同位置。
|
||||
4. 关闭面板继续阅读，打开“验证设置与快捷键”核对关闭前后的滚动位置；输入释义时检查虚拟键盘与面板滚动。
|
||||
5. 测试词典无结果/资源不可用，仍可填写虚构释义、显式保存并继续阅读。
|
||||
|
||||
默认服务器只监听本机。手机需与电脑处于同一可信局域网，在明确进行手机测试时，用电脑的实际局域网 IPv4 地址启动：
|
||||
|
||||
```powershell
|
||||
python spikes/selection/serve.py --host <电脑局域网IPv4地址>
|
||||
```
|
||||
|
||||
手机打开 `http://<电脑局域网IPv4地址>:5184/`。只提供本目录白名单静态文件，不提供目录列表、私人内容或写入接口。若系统防火墙阻止访问，先由用户确认网络放行范围；不自动修改防火墙或发布公网。测试完停止进程。后续真机回归结果回填相关实施工单，并引用 #4 的证据缺口。
|
||||
@@ -0,0 +1,120 @@
|
||||
import {tokenize,normalizeRange,pointSelection,adjustRange,occurrences,resolveOverlaps} from './range.mjs'
|
||||
import {chapters} from './fixtures.mjs'
|
||||
|
||||
const $=id=>document.getElementById(id)
|
||||
const reader=$('reader'), panel=$('panel'), saved=new Map(), positions=new Map()
|
||||
const meanings={'curiosity':'好奇心；求知欲','a small step':'一小步','small step':'小小的一步','learning':'学习','café':'咖啡馆'}
|
||||
let chapterIndex=0,tokens=[],nodes=[],selected=null,anchor=null,focus=null,changingChapter=false,timer
|
||||
|
||||
function chapter(){return chapters[chapterIndex]}
|
||||
function clearNative(){window.getSelection()?.removeAllRanges()}
|
||||
function paint(){
|
||||
const candidates=[]
|
||||
for(const [text,item] of saved) for(const range of occurrences(chapter().text,text)) candidates.push({...range,state:item.state})
|
||||
const visible=resolveOverlaps(candidates)
|
||||
nodes.forEach((node,index)=>{
|
||||
const token=tokens[index]
|
||||
const match=visible.find(range=>token.start>=range.start && token.end<=range.end)
|
||||
node.className=(token.word?'word ':'')+(match?.state || '')
|
||||
node.classList.toggle('selected',!!selected && token.start>=selected.start && token.end<=selected.end)
|
||||
})
|
||||
}
|
||||
function dictionary(){
|
||||
if(!selected)return
|
||||
const mode=$('dictionary-mode').value
|
||||
$('dictionary').textContent=mode==='missing'?'该语言词典暂不可用，仍可填写释义。':mode==='empty'?'未找到释义，可手动填写。':meanings[selected.text.toLocaleLowerCase('en')] || '未找到释义，可手动填写。'
|
||||
}
|
||||
function choose(range,{resetAnchor=true}={}){
|
||||
if(!range)return
|
||||
if(selected?.start===range.start && selected?.end===range.end)return
|
||||
selected=range
|
||||
if(resetAnchor){anchor=range.first;focus=range.last}
|
||||
panel.hidden=false;$('empty').hidden=true
|
||||
$('term').textContent=range.text
|
||||
$('kind').textContent=range.first===range.last?'单词':'短语'
|
||||
const item=saved.get(range.text)
|
||||
$('meaning').value=item?.meaning || ''
|
||||
$('word-state').value=item?.state || 'learning'
|
||||
$('feedback').textContent=''
|
||||
$('range-output').textContent=JSON.stringify({chapter:chapter().id,text:range.text,utf16:[range.start,range.end],code_point:[range.start_cp,range.end_cp],utf8:[range.start_utf8,range.end_utf8]},null,2)
|
||||
dictionary();paint()
|
||||
document.querySelectorAll('[data-edge]').forEach(button=>{
|
||||
const next=adjustRange(chapter().text,tokens,range,button.dataset.edge,Number(button.dataset.direction))
|
||||
button.disabled=next.start===range.start && next.end===range.end
|
||||
})
|
||||
}
|
||||
function close(){
|
||||
const scroll=window.scrollY
|
||||
selected=null;anchor=focus
|
||||
clearTimeout(timer);clearNative();panel.hidden=true;$('empty').hidden=false
|
||||
paint();reader.focus({preventScroll:true});window.scrollTo({top:scroll,behavior:'instant'})
|
||||
$('event-status').textContent=`关闭前 ${scroll}，关闭后 ${window.scrollY}`
|
||||
}
|
||||
function readNativeSelection(){
|
||||
const selection=window.getSelection()
|
||||
if(changingChapter || !selection || selection.isCollapsed || !selection.rangeCount)return false
|
||||
const range=selection.getRangeAt(0)
|
||||
if(!reader.contains(range.startContainer)||!reader.contains(range.endContainer))return false
|
||||
const prefix=document.createRange();prefix.selectNodeContents(reader);prefix.setEnd(range.startContainer,range.startOffset)
|
||||
const start=prefix.toString().length
|
||||
prefix.setEnd(range.endContainer,range.endOffset)
|
||||
const end=prefix.toString().length
|
||||
const normalized=normalizeRange(chapter().text,tokens,start,end)
|
||||
if(!normalized){
|
||||
selected=null;panel.hidden=true;$('empty').hidden=false;paint()
|
||||
return false
|
||||
}
|
||||
choose(normalized)
|
||||
return true
|
||||
}
|
||||
function renderChapter(){
|
||||
changingChapter=true;clearTimeout(timer);clearNative();selected=null;anchor=null;focus=null
|
||||
$('title').textContent=chapter().title;$('chapter-number').textContent=`${chapterIndex+1} / ${chapters.length}`
|
||||
$('previous').disabled=chapterIndex===0;$('next').disabled=chapterIndex===chapters.length-1
|
||||
tokens=tokenize(chapter().text)
|
||||
nodes=tokens.map((token,index)=>{const span=document.createElement('span');span.textContent=token.text;span.dataset.index=String(index);return span})
|
||||
reader.replaceChildren(...nodes)
|
||||
if(reader.textContent!==chapter().text)throw new Error('Original text reconstruction failed')
|
||||
panel.hidden=true;$('empty').hidden=false;paint();changingChapter=false
|
||||
}
|
||||
reader.addEventListener('click',event=>{
|
||||
if(readNativeSelection())return
|
||||
const node=event.target.closest('[data-index]')
|
||||
if(node)choose(pointSelection(chapter().text,tokens,Number(node.dataset.index)))
|
||||
})
|
||||
// Native selection remains in charge of mouse dragging and mobile long-press handles.
|
||||
// No touchmove preventDefault, pointer capture or text-node replacement on selection.
|
||||
document.addEventListener('selectionchange',()=>{clearTimeout(timer);timer=setTimeout(readNativeSelection,100)})
|
||||
reader.addEventListener('pointerup',()=>{clearTimeout(timer);timer=setTimeout(readNativeSelection,0)})
|
||||
reader.addEventListener('keydown',event=>{
|
||||
if(event.key==='Escape'){event.preventDefault();close();return}
|
||||
if(!['ArrowLeft','ArrowRight'].includes(event.key)||event.ctrlKey||event.metaKey||event.altKey)return
|
||||
event.preventDefault();clearNative()
|
||||
const direction=event.key==='ArrowLeft'?-1:1
|
||||
let next=focus===null?(direction===1?0:tokens.length-1):focus+direction
|
||||
while(next>=0 && next<tokens.length && !tokens[next].word)next+=direction
|
||||
if(next<0||next>=tokens.length)return
|
||||
if(!event.shiftKey||anchor===null){anchor=next;focus=next;choose(pointSelection(chapter().text,tokens,next),{resetAnchor:false})}
|
||||
else {focus=next;choose(normalizeRange(chapter().text,tokens,Math.min(tokens[anchor].start,tokens[focus].start),Math.max(tokens[anchor].end,tokens[focus].end)),{resetAnchor:false})}
|
||||
// Keyboard users can navigate a long chapter without losing the active word.
|
||||
nodes[next].scrollIntoView({block:'nearest',inline:'nearest',behavior:'instant'})
|
||||
})
|
||||
document.addEventListener('keydown',event=>{if(event.key==='Escape' && !panel.hidden && event.target!==reader){event.preventDefault();close()}})
|
||||
document.querySelectorAll('[data-edge]').forEach(button=>button.addEventListener('click',()=>{
|
||||
if(!selected)return
|
||||
clearNative();choose(adjustRange(chapter().text,tokens,selected,button.dataset.edge,Number(button.dataset.direction)))
|
||||
}))
|
||||
$('close').addEventListener('click',close)
|
||||
$('save').addEventListener('click',()=>{
|
||||
if(!selected)return
|
||||
const meaning=$('meaning').value.trim()
|
||||
if(!meaning){$('feedback').textContent='请填写释义。';return}
|
||||
saved.set(selected.text,{meaning,state:$('word-state').value})
|
||||
$('feedback').textContent='已保存 · 本次页面内有效';paint()
|
||||
})
|
||||
$('dictionary-mode').addEventListener('change',dictionary)
|
||||
for(const [id,delta] of [['previous',-1],['next',1]])$(id).addEventListener('click',()=>{
|
||||
positions.set(chapter().id,window.scrollY);chapterIndex+=delta;renderChapter()
|
||||
reader.focus({preventScroll:true});window.scrollTo({top:positions.get(chapter().id)||0,behavior:'instant'})
|
||||
})
|
||||
renderChapter()
|
||||
@@ -0,0 +1,54 @@
|
||||
// Fictional public test material, never copied from a learner's library.
|
||||
export const chapters = [
|
||||
{
|
||||
id: 'lantern-1',
|
||||
title: 'The Lantern Workshop',
|
||||
text: `Mira opened the workshop before sunrise. A folded map lay beside the window, and a small step separated the quiet room from the garden. Her curiosity had brought her here. Today she would build a lantern that could travel across the valley.
|
||||
|
||||
“Begin with a small step,” her friend Rowan said. He placed a paper frame on the table. Mira examined the thin edges, turned the frame around, and found a tiny mark near the corner. The mark looked like a star, but its five points were not quite equal.
|
||||
|
||||
Outside, rain tapped against the glass. Inside, the workshop smelled of cedar and warm tea. They measured the paper twice. Then they measured it again, because a hurried answer could turn an ordinary mistake into a long afternoon of repairs.
|
||||
|
||||
Mira wrote a note: keep the frame light, keep the corners strong, and leave enough room for the light. Rowan read the note aloud. Neither of them knew whether the lantern would fly, but both wanted to understand why it might.
|
||||
|
||||
Their first attempt leaned to the left. The second leaned to the right. At the third attempt, Mira laughed. “A well-known problem,” she said, “can still offer an unexpected lesson.” She did not throw the old frames away. Each one showed a different part of the problem.
|
||||
|
||||
By noon the rain had stopped. A narrow line of sunlight crossed the wooden floor. Mira moved the lantern into the light, and the paper shone like a pale orange leaf. Her curiosity was no smaller than it had been in the morning.
|
||||
|
||||
On the shelf was a postcard from an imaginary traveler: “Hello 🙂! The café has a sign that reads café. Our paper astronaut 👩🚀 is ready for the journey. A second sign says 中文.” Mira kept every character exactly as it was written, including the accent that followed its letter.
|
||||
|
||||
The last instruction covered two lines:
|
||||
take a small step
|
||||
and watch what changes.
|
||||
Rowan pointed at the break between them. A pause was useful, he thought; it gave the reader time to notice the words on either side. The paper itself had not lost a single letter.
|
||||
|
||||
In the garden, a light breeze lifted the grass. Mira carried the lantern to the gate and waited. She could hear a bird in the old pear tree, a cart on the distant road, and Rowan counting softly behind her. She held the frame with both hands until the counting stopped.
|
||||
|
||||
The lantern rose a little, turned slowly, and settled beside the gate. It had traveled less than a meter. Mira smiled anyway. For a first journey, a small step was enough. Tomorrow they would return to the workshop, read their notes, and try again.`,
|
||||
},
|
||||
{
|
||||
id: 'lantern-2',
|
||||
title: 'Across the Quiet Valley',
|
||||
text: `The next morning, Rowan arrived with a blue notebook. He had drawn the valley as a series of gentle curves. A small bridge crossed the river near the mill, and a winding path connected the bridge to the workshop. Mira studied the drawing before she touched the lantern.
|
||||
|
||||
“Yesterday was a small step,” she said. “Today we should learn what the wind is doing.” They tied a ribbon to the gate. It pointed toward the river, fell still, and then lifted toward the hills. The wind had more than one answer.
|
||||
|
||||
They waited beneath the pear tree. Waiting felt different when there was something to observe. Rowan noted the direction of each gust. Mira counted the seconds between them. After a while, the blank page held a pattern that neither of them had expected.
|
||||
|
||||
A neighbor passed with an empty basket. She asked why they were staring at a ribbon. Mira explained the experiment, and the neighbor offered a practical suggestion: walk to the bridge before making another lantern. The river often changed the air near its banks.
|
||||
|
||||
At the bridge, the breeze was cooler. Water slipped around the stones with a soft, uneven sound. Rowan opened the notebook to a fresh page. Mira held the ribbon above the railing and watched its end flicker. Their curiosity made an ordinary walk feel like a journey.
|
||||
|
||||
There was no single perfect direction. Near the bank, the ribbon pointed downstream. At the center of the bridge, it twisted back toward the trees. They wrote down both observations. An answer that left out the difficult part would not help them tomorrow.
|
||||
|
||||
On their way home, they stopped beside a weather-beaten sign. Its painted words had faded, but the letters were still clear: a small step can begin a long journey. Underneath, someone had drawn a smiling face 🙂 and a tiny astronaut 👩🚀 beside an imaginary moon.
|
||||
|
||||
Back at the workshop, Mira compared the old paper frames. One was too heavy at its base. Another had a loose corner. She chose the simplest frame and changed only one thing. Rowan recorded the change before they carried the lantern outside.
|
||||
|
||||
This time it floated beyond the gate. For a moment, the orange paper rested against the blue sky like a leaf on clear water. Then the breeze shifted, and the lantern drifted gently into the grass. Nothing had broken. They could try again with the same frame.
|
||||
|
||||
Mira did not call the experiment a failure. The notebook now held a direction, a time, and a careful description of the flight. They had learned something they could test. That was enough to make the next question smaller and more useful.
|
||||
|
||||
When evening came, they closed the window and cleared the table. The map remained beside the notebook. Tomorrow might bring a longer flight, or another surprising turn. Either way, they would begin with a small step, pay attention, and preserve the story exactly as it happened.`,
|
||||
},
|
||||
];
|
||||
@@ -0,0 +1,8 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>LexGo · 阅读选择验证</title><link rel="stylesheet" href="/style.css"></head>
|
||||
<body><header><a class="brand" href="/">LexGo<span>.</span></a><span>英语 · 阅读选择验证</span></header>
|
||||
<main><div class="book"><div><p class="eyebrow">A SMALL STEP, EVERY DAY</p><h1 id="title"></h1></div><nav aria-label="章节"><button id="previous" class="quiet">上一章</button><span id="chapter-number"></span><button id="next" class="quiet">下一章</button></nav></div>
|
||||
<div class="layout"><section class="paper"><p class="hint" id="reader-help">点词查词，划选连续短语。</p><article id="reader" tabindex="0" aria-label="章节正文" aria-describedby="reader-help"></article></section>
|
||||
<aside class="side"><p id="empty">选择一个词或短语</p><section id="panel" aria-label="查词面板" hidden><div class="panel-top"><span id="kind"></span><button id="close" class="quiet" aria-label="关闭，继续阅读">关闭</button></div><h2 id="term"></h2><div class="range-controls" aria-label="调整选择范围"><span>起点</span><button data-edge="start" data-direction="-1" aria-label="起点前移">−</button><button data-edge="start" data-direction="1" aria-label="起点后移">＋</button><span>终点</span><button data-edge="end" data-direction="-1" aria-label="终点前移">−</button><button data-edge="end" data-direction="1" aria-label="终点后移">＋</button></div><p id="dictionary"></p><label for="meaning">我的释义</label><textarea id="meaning" rows="2" maxlength="500"></textarea><label for="word-state">状态</label><select id="word-state"><option value="learning">学习中</option><option value="known">已认识</option><option value="ignored">忽略</option></select><button id="save">保存到生词本</button><p id="feedback" role="status" aria-live="polite"></p><details><summary>选中范围</summary><output id="range-output"></output></details></section></aside></div>
|
||||
<details class="settings"><summary>验证设置与快捷键</summary><p>正文获得焦点后：← / → 选择相邻词；Shift + ← / → 调整连续范围；Escape 取消。范围按钮也可用 Tab、Enter 操作。</p><label for="dictionary-mode">词典状态</label><select id="dictionary-mode"><option value="ready">正常</option><option value="empty">无结果</option><option value="missing">资源不可用</option></select><p>虚构文本与释义，保存仅在本次页面内有效。刷新即清空。</p><p id="event-status" role="status"></p></details>
|
||||
</main><footer>交互验证小样 · 不保存私人数据</footer><script type="module" src="/app.mjs"></script></body></html>
|
||||
@@ -0,0 +1,85 @@
|
||||
// UI selection boundaries only: this does not replace the #3 NLP tokenizer.
|
||||
const words = new Intl.Segmenter('en', { granularity: 'word' });
|
||||
const graphemes = new Intl.Segmenter('en', { granularity: 'grapheme' });
|
||||
const encoder = new TextEncoder();
|
||||
|
||||
export function tokenize(text) {
|
||||
const tokens = [];
|
||||
for (const part of words.segment(text)) {
|
||||
if (part.isWordLike) {
|
||||
tokens.push({ text: part.segment, start: part.index, end: part.index + part.segment.length, word: true });
|
||||
} else {
|
||||
for (const item of graphemes.segment(part.segment)) {
|
||||
const start = part.index + item.index;
|
||||
tokens.push({ text: item.segment, start, end: start + item.segment.length, word: /\p{Extended_Pictographic}/u.test(item.segment) });
|
||||
}
|
||||
}
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
function selection(text, tokens, first, last) {
|
||||
const start = tokens[first].start;
|
||||
const end = tokens[last].end;
|
||||
return {
|
||||
start, end, text: text.slice(start, end), first, last,
|
||||
start_cp: [...text.slice(0, start)].length,
|
||||
end_cp: [...text.slice(0, end)].length,
|
||||
start_utf8: encoder.encode(text.slice(0, start)).length,
|
||||
end_utf8: encoder.encode(text.slice(0, end)).length,
|
||||
start_utf16: start, end_utf16: end,
|
||||
};
|
||||
}
|
||||
|
||||
// Inputs and outputs use half-open UTF-16 offsets into the unchanged original.
|
||||
export function normalizeRange(text, tokens, start, end) {
|
||||
if (![start, end].every(n => Number.isInteger(n) && n >= 0 && n <= text.length) || start === end) return null;
|
||||
const low = Math.min(start, end);
|
||||
const high = Math.max(start, end);
|
||||
let first = -1;
|
||||
let last = -1;
|
||||
tokens.forEach((token, index) => {
|
||||
if (token.word && token.start < high && token.end > low) {
|
||||
if (first === -1) first = index;
|
||||
last = index;
|
||||
}
|
||||
});
|
||||
return first === -1 ? null : selection(text, tokens, first, last);
|
||||
}
|
||||
|
||||
export function pointSelection(text, tokens, index) {
|
||||
return Number.isInteger(index) && tokens[index]?.word ? selection(text, tokens, index, index) : null;
|
||||
}
|
||||
|
||||
// direction -1/+1 moves a boundary left/right; it never collapses a selection.
|
||||
export function adjustRange(text, tokens, range, edge, direction) {
|
||||
if (!range || !['start', 'end'].includes(edge) || ![-1, 1].includes(direction)) return range;
|
||||
let next = (edge === 'start' ? range.first : range.last) + direction;
|
||||
while (next >= 0 && next < tokens.length && !tokens[next].word) next += direction;
|
||||
if (next < 0 || next >= tokens.length) return range;
|
||||
const first = edge === 'start' ? next : range.first;
|
||||
const last = edge === 'end' ? next : range.last;
|
||||
return first > last ? range : selection(text, tokens, first, last);
|
||||
}
|
||||
|
||||
export function occurrences(text, needle) {
|
||||
if (!needle) return [];
|
||||
const tokens = tokenize(text);
|
||||
const result = [];
|
||||
for (let start = text.indexOf(needle); start !== -1; start = text.indexOf(needle, start + 1)) {
|
||||
const end = start + needle.length;
|
||||
const range = normalizeRange(text, tokens, start, end);
|
||||
if (range?.start === start && range.end === end) result.push(range);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// Display precedence only. Saved occurrence identities remain separate.
|
||||
export function resolveOverlaps(ranges) {
|
||||
const sorted = [...ranges].sort((a, b) => a.start - b.start || b.end - a.end);
|
||||
const result = [];
|
||||
for (const range of sorted) {
|
||||
if (range.end > range.start && (!result.length || range.start >= result.at(-1).end)) result.push(range);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { tokenize, normalizeRange, pointSelection, adjustRange, occurrences, resolveOverlaps } from './range.mjs';
|
||||
|
||||
test('partition preserves original Unicode, whitespace and punctuation exactly', () => {
|
||||
const text = 'Hi, cafe\u0301!\n🙂 👩🚀 well-known 中文';
|
||||
const tokens = tokenize(text);
|
||||
assert.equal(tokens.map(t => t.text).join(''), text);
|
||||
let end = 0;
|
||||
for (const t of tokens) {
|
||||
assert.equal(t.start, end);
|
||||
assert.equal(text.slice(t.start, t.end), t.text);
|
||||
end = t.end;
|
||||
}
|
||||
assert.deepEqual(tokens.filter(t => t.word).map(t => t.text), ['Hi', 'cafe\u0301', '🙂', '👩🚀', 'well', 'known', '中文']);
|
||||
});
|
||||
|
||||
test('reverse partial selection aligns words and preserves interior punctuation/newlines', () => {
|
||||
const text = ' bright,\nsmall step! ';
|
||||
const tokens = tokenize(text);
|
||||
const range = normalizeRange(text, tokens, 18, 4);
|
||||
assert.equal(range?.text, 'bright,\nsmall step');
|
||||
assert.equal(range.start, 2);
|
||||
assert.equal(range.end, 20);
|
||||
assert.deepEqual(normalizeRange(text, tokens, 0, text.length), range);
|
||||
});
|
||||
|
||||
test('collapsed, whitespace, punctuation and invalid bounds do not select neighbors', () => {
|
||||
const text = 'a, b';
|
||||
const tokens = tokenize(text);
|
||||
for (const [a, b] of [[1, 1], [1, 3], [-1, 2], [0, 5], [NaN, 3], [0.5, 3]]) {
|
||||
assert.equal(normalizeRange(text, tokens, a, b), null);
|
||||
}
|
||||
});
|
||||
|
||||
test('selection cannot cut a surrogate pair, ZWJ or combining grapheme', () => {
|
||||
const text = '🙂 e\u0301 👩🚀';
|
||||
const tokens = tokenize(text);
|
||||
const first = normalizeRange(text, tokens, 1, 2);
|
||||
assert.equal(first?.text, '🙂');
|
||||
const accent = normalizeRange(text, tokens, 4, 5);
|
||||
assert.equal(accent?.text, 'e\u0301');
|
||||
const astronaut = normalizeRange(text, tokens, 8, 9);
|
||||
assert.equal(astronaut?.text, '👩🚀');
|
||||
assert.deepEqual([astronaut.start_cp, astronaut.end_cp, astronaut.start_utf8, astronaut.end_utf8, astronaut.start_utf16, astronaut.end_utf16], [5, 8, 9, 20, 6, 11]);
|
||||
});
|
||||
|
||||
test('point selection and boundary adjustment skip separators and never invert', () => {
|
||||
const text = 'one, two\nthree';
|
||||
const tokens = tokenize(text);
|
||||
const middle = tokens.findIndex(t => t.text === 'two');
|
||||
const one = pointSelection(text, tokens, middle);
|
||||
assert.equal(one?.text, 'two');
|
||||
assert.equal(pointSelection(text, tokens, middle - 1), null);
|
||||
const left = adjustRange(text, tokens, one, 'start', -1);
|
||||
assert.equal(left?.text, 'one, two');
|
||||
const all = adjustRange(text, tokens, left, 'end', 1);
|
||||
assert.equal(all?.text, text);
|
||||
assert.equal(adjustRange(text, tokens, all, 'start', 1)?.text, 'two\nthree');
|
||||
assert.equal(adjustRange(text, tokens, all, 'end', -1)?.text, 'one, two');
|
||||
assert.deepEqual(adjustRange(text, tokens, one, 'start', 1), one);
|
||||
assert.deepEqual(adjustRange(text, tokens, one, 'end', -1), one);
|
||||
assert.deepEqual(adjustRange(text, tokens, all, 'start', -1), all);
|
||||
assert.deepEqual(adjustRange(text, tokens, all, 'end', 1), all);
|
||||
});
|
||||
|
||||
test('occurrences preserve distinct offsets, case and exact combining form', () => {
|
||||
const text = 'a small step; a small step. A small step. steps e\u0301 é';
|
||||
assert.deepEqual(occurrences(text, 'a small step').map(r => [r.start, r.end]), [[0, 12], [14, 26]]);
|
||||
assert.equal(occurrences(text, 'step').length, 3);
|
||||
assert.equal(occurrences(text, 'e\u0301').length, 1);
|
||||
assert.equal(occurrences(text, 'é').length, 1);
|
||||
assert.equal(occurrences(text, 'small st').length, 0);
|
||||
assert.deepEqual(occurrences(text, ''), []);
|
||||
});
|
||||
|
||||
test('overlapping occurrences remain available; display picks leftmost longest without mutation', () => {
|
||||
assert.deepEqual(occurrences('a a a', 'a a').map(r => r.start), [0, 2]);
|
||||
const ranges = [{ start: 2, end: 9, id: 'later' }, { start: 0, end: 3, id: 'short' }, { start: 0, end: 5, id: 'long' }, { start: 5, end: 7, id: 'next' }];
|
||||
const before = structuredClone(ranges);
|
||||
assert.deepEqual(resolveOverlaps(ranges).map(r => r.id), ['long', 'next']);
|
||||
assert.deepEqual(ranges, before);
|
||||
});
|
||||
@@ -0,0 +1,43 @@
|
||||
"""Serve only the fictional reading experiment; default is loopback."""
|
||||
import argparse
|
||||
import ipaddress
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
FILES = {'/': ('index.html', 'text/html'), '/style.css': ('style.css', 'text/css'),
|
||||
'/app.mjs': ('app.mjs', 'text/javascript'), '/range.mjs': ('range.mjs', 'text/javascript'),
|
||||
'/fixtures.mjs': ('fixtures.mjs', 'text/javascript')}
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
timeout = 10
|
||||
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def do_GET(self):
|
||||
item = FILES.get(self.path)
|
||||
data = (ROOT / item[0]).read_bytes() if item else b'Not found'
|
||||
self.send_response(200 if item else 404)
|
||||
self.send_header('Content-Type', (item[1] if item else 'text/plain') + '; charset=utf-8')
|
||||
self.send_header('Content-Length', str(len(data)))
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.send_header('X-Content-Type-Options', 'nosniff')
|
||||
self.send_header('Content-Security-Policy', "default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'none'; frame-ancestors 'none'; base-uri 'none'")
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--host', default='127.0.0.1', help='Numeric local address; LAN testing is opt-in')
|
||||
parser.add_argument('--port', type=int, default=5184)
|
||||
args = parser.parse_args()
|
||||
ipaddress.IPv4Address(args.host)
|
||||
with HTTPServer((args.host, args.port), Handler) as server:
|
||||
print(f'LexGo selection experiment: http://{args.host}:{server.server_port}/', flush=True)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
@@ -0,0 +1 @@
|
||||
:root{font-family:"Segoe UI","Microsoft YaHei",sans-serif;color:#203d33;background:#f6f4ed;--green:#285c49;--line:#dbe1d8}*{box-sizing:border-box}body{margin:0}header{height:76px;background:#fff;border-bottom:1px solid var(--line);padding:0 5%;display:flex;align-items:center;justify-content:space-between}header>span{color:#6c7971;font-size:13px}.brand{color:inherit;text-decoration:none;font-size:28px;font-weight:750;letter-spacing:-1px}.brand span{color:#538461}main{max-width:1220px;margin:36px auto;padding:0 24px}.book{display:flex;align-items:center;justify-content:space-between;gap:20px;margin-bottom:25px}.eyebrow{font-size:11px;letter-spacing:2px;color:#738276}h1{font-size:26px;font-weight:600;margin:10px 0}nav{display:flex;gap:12px;align-items:center;white-space:nowrap}button,input,textarea,select{font:inherit}button{cursor:pointer;border:1px solid var(--line);border-radius:5px;padding:9px 14px;background:var(--green);color:#fff;touch-action:manipulation}button:disabled{opacity:.4;cursor:default}.quiet{background:#fff;color:var(--green)}button:focus-visible,select:focus-visible,textarea:focus-visible,article:focus-visible{outline:2px solid #588364;outline-offset:3px}.layout{display:grid;grid-template-columns:minmax(0,1fr) 310px;gap:24px;align-items:start}.paper{background:#fff;border:1px solid var(--line);border-radius:8px;padding:32px 38px}.hint{font-size:12px;color:#738276;margin:0 0 22px}article{font:22px/1.95 Georgia,"Times New Roman",serif;white-space:pre-wrap;overflow-wrap:anywhere;user-select:text;-webkit-user-select:text;touch-action:auto;outline-offset:10px}article span.word{cursor:text}article .learning{background:#f5e7b7}article .known{box-shadow:inset 0 -2px #a2c1ab}article .ignored{color:#849086}article .selected{background:#d7e9b3;box-shadow:inset 0 -2px #56743d}article ::selection{background:#bad59a;color:#18321e}.side{position:sticky;top:18px;min-height:100px}#empty{font-size:14px;color:#859186;padding:20px;text-align:center}#panel{background:#fff;border:1px solid var(--line);border-radius:8px;padding:22px}.panel-top{display:flex;align-items:center;justify-content:space-between;color:#7b897f;font-size:12px}h2{font:27px/1.4 Georgia,serif;margin:14px 0;overflow-wrap:anywhere;max-height:120px;overflow:auto}.range-controls{display:flex;align-items:center;gap:5px;font-size:12px;flex-wrap:wrap}.range-controls button{min-width:32px;padding:6px;background:#f5f7ef;color:var(--green)}.range-controls span:nth-of-type(2){margin-left:8px}#dictionary{font-size:14px;line-height:1.6;border-top:1px solid var(--line);padding-top:16px}label{display:block;font-size:12px;color:#667b6d;margin:16px 0 7px}textarea,select{width:100%;border:1px solid var(--line);border-radius:4px;padding:9px;background:white;color:inherit}textarea{resize:vertical}#save{width:100%;margin-top:18px}#feedback{min-height:20px;font-size:13px;color:#467345}details{font-size:12px;color:#748276}summary{cursor:pointer}output{display:block;white-space:pre-wrap;overflow-wrap:anywhere;font:11px/1.5 Consolas,monospace;margin-top:10px}.settings{margin-top:28px;max-width:720px;line-height:1.8}.settings select{max-width:240px}footer{text-align:center;font-size:12px;color:#839083;padding:30px}[hidden]{display:none!important}@media(max-width:760px){header{height:60px}.book{display:block}nav{margin-top:18px;justify-content:space-between}main{padding:0 14px;margin-top:22px}.layout{display:block}.paper{padding:24px 20px}article{font-size:20px;line-height:1.95}article .word{scroll-margin-bottom:45dvh}body{padding-bottom:360px}.side{position:static;min-height:0}#empty{display:none}#panel{position:fixed;bottom:0;left:0;right:0;z-index:2;max-height:42dvh;overflow:auto;overscroll-behavior:contain;border-radius:14px 14px 0 0;padding:16px 20px;box-shadow:0 -4px 20px #233c3314}.panel-top{position:sticky;top:-16px;background:#fff;padding:5px 0}h2{font-size:23px;max-height:65px}.range-controls button{min-height:40px;min-width:40px}}
|
||||
@@ -0,0 +1,53 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import {readFile} from 'node:fs/promises'
|
||||
import {JSDOM} from '../../learner/node_modules/jsdom/lib/api.js'
|
||||
|
||||
let serial=0
|
||||
async function page(run){
|
||||
const html=await readFile(new URL('./index.html',import.meta.url),'utf8')
|
||||
const dom=new JSDOM(html,{url:'http://127.0.0.1:5184/'})
|
||||
globalThis.window=dom.window;globalThis.document=dom.window.document
|
||||
dom.window.scrollTo=()=>{}
|
||||
dom.window.HTMLElement.prototype.scrollIntoView=()=>{}
|
||||
await import(`./app.mjs?test=${++serial}`)
|
||||
const $=id=>document.getElementById(id)
|
||||
try{await run($,dom.window)}finally{await new Promise(resolve=>setTimeout(resolve,120));dom.window.close()}
|
||||
}
|
||||
|
||||
test('close preserves keyboard cursor: next arrow continues at the adjacent word',()=>page(($,w)=>{
|
||||
const span=[...$('reader').children].find(n=>n.textContent==='workshop')
|
||||
span.click();assert.equal($('term').textContent,'workshop')
|
||||
$('close').click();assert.equal($('panel').hidden,true)
|
||||
$('reader').dispatchEvent(new w.KeyboardEvent('keydown',{key:'ArrowRight',bubbles:true}))
|
||||
assert.equal($('term').textContent,'before')
|
||||
}))
|
||||
test('native punctuation-only selection clears stale word and cannot save it',()=>page(async($,w)=>{
|
||||
[...$('reader').children].find(n=>n.textContent==='curiosity').click()
|
||||
const dot=[...$('reader').children].find(n=>n.textContent==='.')
|
||||
const range=document.createRange();range.selectNodeContents(dot)
|
||||
w.getSelection().removeAllRanges();w.getSelection().addRange(range)
|
||||
document.dispatchEvent(new w.Event('selectionchange'))
|
||||
await new Promise(resolve=>setTimeout(resolve,130))
|
||||
assert.equal($('panel').hidden,true)
|
||||
assert.equal($('reader').querySelectorAll('.selected').length,0)
|
||||
}))
|
||||
test('native selection maps DOM text through spans and expands to full token endpoints',()=>page(async($,w)=>{
|
||||
const spans=[...$('reader').children]
|
||||
const start=spans.find(n=>n.textContent==='Mira'),end=spans.find(n=>n.textContent==='workshop')
|
||||
const range=document.createRange();range.setStart(start.firstChild,2);range.setEnd(end.firstChild,3)
|
||||
w.getSelection().removeAllRanges();w.getSelection().addRange(range)
|
||||
document.dispatchEvent(new w.Event('selectionchange'))
|
||||
await new Promise(resolve=>setTimeout(resolve,130))
|
||||
assert.equal($('term').textContent,'Mira opened the workshop')
|
||||
const result=JSON.parse($('range-output').textContent)
|
||||
assert.equal($('reader').textContent.slice(...result.utf16),result.text)
|
||||
}))
|
||||
test('explicit save updates repeated words; unsaved edits are discarded on close',()=>page(($)=>{
|
||||
const words=[...$('reader').children].filter(n=>n.textContent==='curiosity')
|
||||
words[0].click();$('meaning').value='虚构释义';$('save').click()
|
||||
assert.ok(words.every(n=>n.classList.contains('learning')))
|
||||
$('meaning').value='未保存修改';$('close').click();words[1].click()
|
||||
assert.equal($('meaning').value,'虚构释义')
|
||||
$('next').click();assert.equal($('panel').hidden,true)
|
||||
}))
|
||||
Reference in New Issue
Block a user