Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ed5da31888 | ||
|
|
8d4a095590 | ||
|
|
2b6aa90813 | ||
|
|
b7c976eb75 | ||
|
|
96ac5eb618 | ||
|
|
850c78e2aa |
@@ -258,7 +258,7 @@ MVP 内所有单元任务通过后才能做 MVP 集成验收;MVP 通过后才
|
||||
|
||||
- 治理模式:轻量。数据库:MySQL 8(用户于 2026-09-10 确认);具体小版本在工程验证后锁定。
|
||||
- 远端:https://git.ilapage.cn/OPC/lexgo.git;分支 main。不得把邻接 dev_harness 工作区当成本项目工作区。
|
||||
- 工程基础 #2 已实现待用户验收:server 基于指定 go-admin 选用模型扩展账号/会话 API,admin 复用 go-admin-ui,learner 为独立 Vue 3 + TypeScript + Vite 工程。默认英语;阅读、导入、词典、复习及 Python NLP 尚未实现或验证。
|
||||
- 工程基础 #2 已通过用户验收:server 基于指定 go-admin 选用模型扩展账号/会话 API,admin 复用 go-admin-ui,learner 为独立 Vue 3 + TypeScript + Vite 工程。默认英语;阅读、导入、词典与复习尚未接入产品;#3 独立 Python NLP/词典验证小样已通过用户验收。
|
||||
- 原四份研究保留为历史参考;PostgreSQL 建议被 MySQL 8 决策覆盖,U/A/N 索引用于追踪而不是批准所有范围。
|
||||
- 用户/语言数据所有权、Unicode 原文位置、任务和复习幂等、完整备份恢复是后续方案的必要验收边界。
|
||||
- 当前 MCP 连接其他 Gitea 站点,需使用目标站点 API 时记录原因;凭据仅从安全配置进入进程。
|
||||
@@ -273,7 +273,10 @@ MVP 内所有单元任务通过后才能做 MVP 集成验收;MVP 通过后才
|
||||
|
||||
- F01~F12 已确认进入 MVP,X 系列本轮不纳入;用户指定 Quant-UX 原型,原型审核后才拆实施工单。默认学习语言已确认英语,原型语料为虚构样例;不导出本地 HTML。
|
||||
- 原型尽量减少说明文字,与目标页面一致:产品页只保留实际字段、操作与必要反馈;功能编号、模拟边界和审核说明放在独立导览或工单。
|
||||
- 当前 Quant-UX v1 已获用户验收(工单 #1 评论 7498),实施总览为 #16、单元工单为 #2~#15;LinguaCafe 对照与真实划词验证由 #4 承接,不能记作已完成。
|
||||
- 当前 Quant-UX v1 已获用户验收(工单 #1 评论 7498),实施总览为 #16、单元工单为 #2~#15;LinguaCafe 源码对照与桌面划词验证已由 #4 交付并获用户验收;真机详细测试证据缺口仍保留。
|
||||
- 学习端和管理端均使用账号(用户名)+密码登录,账号不要求邮箱格式,邮箱不作为必填登录标识;后端独立校验管理权限与本人学习数据归属。
|
||||
- 已验证 MySQL 8.4.3,本机 127.0.0.1:3308;开发库 lexgo_dev、测试库 lexgo_test_issue2。密码只从环境或忽略的 .env.local 读取。迁移测试只能使用 lexgo_test_ 前缀专用库,不能借用其他数据库。
|
||||
- 后端命令使用 `python scripts/server.py migrate|bootstrap|serve|build|test|test-integration`;仅显式 migrate 修改表。bootstrap 只接受尚无账号的 LexGo 库,不覆盖已有管理员。Go 1.26.5、Node 22.22.1、pnpm 9.15.1;两端分别构建。
|
||||
- #18 登录日志与操作审计已通过用户验收:schema v2 显式迁移;日志只保存白名单字段,禁止保存凭据、请求/响应正文及私人学习内容。仅管理员查询,默认保留 90 天;启动/每小时及 `python scripts/server.py audit-cleanup` 仅清理两张审计表的过期记录。
|
||||
- #3 独立小样位于 `spikes/english/`,使用 `.local/nlp-venv/Scripts/python.exe`(3.12.12)运行;固定 spaCy 3.8.7、英语模型 3.8.0、NLTK 3.9.2、WordNet 3.0。资源仅显式准备时下载,摘要见 resources.json。不得把本机无账号的实验接口用于正式学习端;后续集成仍需 Go 授权、数据归属和任务设计。原文不归一化,位置区分 cp/UTF-8/UTF-16,lemma 不自动合并学习状态。
|
||||
- #4 独立阅读选择小样位于 `spikes/selection/`,`python spikes/selection/serve.py` 默认仅本机 5184。桌面鼠标/键盘与 11 项测试已验证,#4 已获用户验收并关闭;真实手机长按/手柄/滚动详细证据仍未提供;禁止把窄屏桌面当作真机验收。Intl.Segmenter 只用于 UI 范围验证,不替代 #3 NLP;释义保存只在内存。固定 LinguaCafe 源码对照和与 v1 的差异记录见架构 Wiki。
|
||||
|
||||
@@ -2,11 +2,13 @@
|
||||
|
||||
面向自托管场景的阅读式语言学习项目,规划提供内容导入、阅读查词、词汇与短语、复习和实例管理。
|
||||
|
||||
已确认:**DevHarness 轻量模式、MySQL 8、go-admin 管理端**。工程基础 #2 已实现待验收:两端用户名登录、学习账号管理、可撤销会话和本人英语空空间。管理端基于指定 go-admin/go-admin-ui 选用模块,学习端为独立 Vue 3 + TypeScript + Vite 工程,共用 Go 后端和 MySQL 8.4.3。阅读、导入、词典与复习尚未实现。MVP 定位为“支持多账号、数据独立的自托管学习工具”,先邀请少量用户使用;F01~F12 已确认,X 系列后置。
|
||||
已确认:**DevHarness 轻量模式、MySQL 8、go-admin 管理端**。工程基础 #2 已通过验收:两端用户名登录、学习账号管理、可撤销会话和本人英语空空间。管理端基于指定 go-admin/go-admin-ui 选用模块,学习端为独立 Vue 3 + TypeScript + Vite 工程,共用 Go 后端和 MySQL 8.4.3。#18 登录日志与操作审计已通过用户验收,支持管理员查询和 90 天保留清理。阅读、导入、词典与复习尚未实现。MVP 定位为“支持多账号、数据独立的自托管学习工具”,先邀请少量用户使用;F01~F12 已确认,X 系列后置。
|
||||
|
||||
- [文档入口](docs/README.md) · [线上 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Home)
|
||||
- [英语分词与离线词典验证小样](spikes/english/README.md)(#3 已验收,独立本机入口)
|
||||
- [阅读选择验证小样](spikes/selection/README.md)(#4 已验收,真机详细测试证据缺口保留)
|
||||
- [项目档案](docs/00-project-profile.md) · [需求总览](docs/09-product-requirements-overview.md)
|
||||
- [工作量估算](docs/10-workload-estimate.md):F01~F12 剩余产品工作量初估 52~84 人日,技术验证后重估;旧全量研究仅供参考。
|
||||
- [工作量估算](docs/10-workload-estimate.md):#2、#3、#4、#18 已验收,剩余 #5~#15 与新增 #21 规划参考 44~71 人日;后续结合集成结果重估,旧全量研究仅供参考。
|
||||
- [四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16):14 张单元工单,工程基础 → 技术验证 → 首条学习闭环 → 补齐 MVP;原型 v1 已获用户验收。两端使用账号(用户名)+密码登录,不要求邮箱。
|
||||
- [原型工单 #1](https://git.ilapage.cn/OPC/lexgo/issues/1):Quant-UX 桌面/手机原型 v1,预览入口与审核记录见工单及需求总览。
|
||||
- [工作流](docs/01-workflow.md) · [开发与验证](docs/04-local-development-and-verification.md)
|
||||
@@ -17,6 +19,8 @@
|
||||
|
||||
本地入口:学习端 http://127.0.0.1:5173,管理端 http://127.0.0.1:5174。完整安装与测试命令见[开发与验证](docs/04-local-development-and-verification.md)。账号使用用户名,无需邮箱;没有随代码交付的默认密码。
|
||||
|
||||
本机三个服务已由 `D:/supervisord/supervisord.conf` 中的 lexgo-learner、lexgo-admin、lexgo-api 托管,使用时不要重复手动启动同端口。当前 schema v2;从 #2 升级时停止 API,执行 build、migrate 后再启动。审计过期清理可执行 `python scripts/server.py audit-cleanup`,只影响超过 90 天的日志。
|
||||
|
||||
## 文档与治理
|
||||
|
||||
远端为 `https://git.ilapage.cn/OPC/lexgo.git`,默认分支 main。需要工单的任务以 Gitea 工单为单次任务事实来源;轻量直接实施项通过 Git 提交和结果报告留痕。长期规则以 Wiki 为主源,docs 为单向镜像;默认不创建任务归档。
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
export const actionLabels = { create: '创建账号', enable: '启用账号', disable: '停用账号', reset_password: '重置密码', update: '更新账号' }
|
||||
export const reasonLabels = { ok: '完成', invalid_input: '输入无效', invalid_credentials: '账号或密码不正确', rate_limited: '请求过于频繁', forbidden: '无操作权限', not_found: '账号不存在', conflict: '账号冲突', internal_error: '服务异常' }
|
||||
|
||||
export function auditQuery(kind, filters = {}) {
|
||||
if (!['login', 'operation'].includes(kind)) throw new Error('日志类型无效')
|
||||
const { page = 1, limit = 20, result, action, from, to } = filters
|
||||
if (!Number.isInteger(page) || page < 1 || !Number.isInteger(limit) || limit < 1 || limit > 100) throw new Error('分页参数无效')
|
||||
const params = new URLSearchParams({ page, limit })
|
||||
const username = (filters.username || '').trim().toLowerCase()
|
||||
if (username) params.set('username', username)
|
||||
if (result) {
|
||||
if (!['success', 'failure'].includes(result)) throw new Error('结果筛选无效')
|
||||
params.set('result', result)
|
||||
}
|
||||
if (action) {
|
||||
if (kind !== 'operation' || !Object.hasOwn(actionLabels, action)) throw new Error('操作筛选无效')
|
||||
params.set('action', action)
|
||||
}
|
||||
for (const [key, value] of [['from', from], ['to', to]]) {
|
||||
if (!value) continue
|
||||
const date = new Date(value)
|
||||
if (Number.isNaN(date.getTime())) throw new Error('请选择有效时间')
|
||||
params.set(key, date.toISOString())
|
||||
}
|
||||
if (from && to && new Date(from) > new Date(to)) throw new Error('开始时间不能晚于结束时间')
|
||||
return params.toString()
|
||||
}
|
||||
|
||||
// Every page/filter change and session invalidation retires pending responses.
|
||||
export function createAuditLogLoader(session, state) {
|
||||
let sequence = 0
|
||||
function invalidate() {
|
||||
sequence++
|
||||
Object.assign(state, { items: [], total: 0, loading: false, error: '' })
|
||||
}
|
||||
return {
|
||||
invalidate,
|
||||
async load(kind, filters) {
|
||||
// Keep the page count stable while fetching another page; resetting total
|
||||
// makes Element Plus clamp the current page to one and issue a new query.
|
||||
sequence++
|
||||
Object.assign(state, { items: [], loading: true, error: '' })
|
||||
const current = sequence
|
||||
const generation = session.state.generation
|
||||
const isCurrent = () => current === sequence && generation === session.state.generation
|
||||
try {
|
||||
const data = await session.queryAuditLogs(kind, filters)
|
||||
if (isCurrent()) Object.assign(state, { items: data.items, total: data.total })
|
||||
} catch (error) {
|
||||
if (isCurrent()) state.error = error.message
|
||||
} finally {
|
||||
if (current === sequence) state.loading = false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -3,14 +3,16 @@
|
||||
<div v-if="device === 'mobile' && sidebar.opened" class="drawer-bg" @click="handleClickOutside" />
|
||||
<aside class="sidebar-container">
|
||||
<div class="brand">{{ sidebar.opened ? 'LexGo 管理' : 'LG' }}</div>
|
||||
<el-menu default-active="/accounts" :collapse="!sidebar.opened" router>
|
||||
<el-menu :default-active="$route.path" :collapse="!sidebar.opened" router>
|
||||
<el-menu-item index="/accounts"><span>账号管理</span></el-menu-item>
|
||||
<el-menu-item index="/login-logs"><span>登录日志</span></el-menu-item>
|
||||
<el-menu-item index="/operation-logs"><span>操作日志</span></el-menu-item>
|
||||
</el-menu>
|
||||
</aside>
|
||||
<div class="main-container">
|
||||
<header class="navbar">
|
||||
<button class="menu-toggle" aria-label="切换导航" @click="$store.dispatch('app/toggleSideBar')"><hamburger :is-active="sidebar.opened" /></button>
|
||||
<span>账号管理</span>
|
||||
<span>{{ $route.meta.title }}</span>
|
||||
<div class="right-menu"><span>{{ $store.state.user?.username }}</span><el-button @click="logout">退出登录</el-button></div>
|
||||
</header>
|
||||
<app-main />
|
||||
|
||||
@@ -3,9 +3,15 @@ import { session } from '../store'
|
||||
import Layout from '../layout/index.vue'
|
||||
import Login from '../views/Login.vue'
|
||||
import Accounts from '../views/Accounts.vue'
|
||||
import AuditLogs from '../views/AuditLogs.vue'
|
||||
const router = createRouter({ history: createWebHashHistory(), routes: [
|
||||
{ path: '/login', component: Login },
|
||||
{ path: '/', component: Layout, children: [{ path: '', redirect: '/accounts' }, { path: 'accounts', component: Accounts }] },
|
||||
{ path: '/', component: Layout, children: [
|
||||
{ path: '', redirect: '/accounts' },
|
||||
{ path: 'accounts', component: Accounts, meta: { title: '账号管理' } },
|
||||
{ path: 'login-logs', component: AuditLogs, props: { kind: 'login' }, meta: { title: '登录日志' } },
|
||||
{ path: 'operation-logs', component: AuditLogs, props: { kind: 'operation' }, meta: { title: '操作日志' } }
|
||||
] },
|
||||
{ path: '/:pathMatch(.*)*', redirect: '/accounts' }
|
||||
] })
|
||||
router.beforeEach(async to => {
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { auditQuery } from './audit-logs.mjs'
|
||||
export function normalizeUsername(value) { return value.trim().toLowerCase() }
|
||||
export function validPassword(value) {
|
||||
const bytes = new TextEncoder().encode(value).length
|
||||
@@ -83,6 +84,14 @@ export function createSession({ fetch, storage, changed = () => {} }) {
|
||||
} catch (error) { if (generation === state.generation) clear(); throw error }
|
||||
},
|
||||
async logout() { const token = state.token; clear(); await revoke(token) },
|
||||
async queryAuditLogs(kind, filters) {
|
||||
authorized()
|
||||
const generation = state.generation
|
||||
const query = auditQuery(kind, filters)
|
||||
const data = await request('/' + kind + '-logs?' + query)
|
||||
assertCurrent(generation)
|
||||
return data
|
||||
},
|
||||
async loadAccounts() {
|
||||
authorized()
|
||||
const generation = state.generation
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
<template>
|
||||
<basic-layout><template #wrapper><el-card>
|
||||
<div class="toolbar"><h1>{{ kind === 'login' ? '登录日志' : '操作日志' }}</h1><span class="retention">保留最近 90 天记录</span></div>
|
||||
<el-form :model="filters" class="filters" label-position="top" @submit.prevent="search">
|
||||
<el-form-item label="账号"><el-input v-model="filters.username" :placeholder="kind === 'login' ? '精确匹配账号' : '操作人或目标账号'" clearable /></el-form-item>
|
||||
<el-form-item label="结果"><el-select v-model="filters.result"><el-option label="全部" value="" /><el-option label="成功" value="success" /><el-option label="失败" value="failure" /></el-select></el-form-item>
|
||||
<el-form-item v-if="kind === 'operation'" label="操作"><el-select v-model="filters.action"><el-option label="全部" value="" /><el-option v-for="(label, action) in actionLabels" :key="action" :label="label" :value="action" /></el-select></el-form-item>
|
||||
<el-form-item label="开始时间"><el-date-picker v-model="filters.from" type="datetime" placeholder="选择开始时间" /></el-form-item>
|
||||
<el-form-item label="结束时间"><el-date-picker v-model="filters.to" type="datetime" placeholder="选择结束时间" /></el-form-item>
|
||||
<div class="filter-actions"><el-button type="primary" native-type="submit" :loading="logs.loading">查询</el-button><el-button @click="reset">重置</el-button></div>
|
||||
</el-form>
|
||||
<el-alert v-if="logs.error" :title="logs.error" type="error" :closable="false" show-icon />
|
||||
<el-table v-loading="logs.loading" :data="logs.items" border :empty-text="logs.error ? '加载失败,请重试' : '暂无符合条件的记录'">
|
||||
<el-table-column label="时间" min-width="180"><template #default="scope">{{ formatDate(scope.row.createdAt) }}</template></el-table-column>
|
||||
<el-table-column v-if="kind === 'login'" label="账号" prop="username" min-width="150" show-overflow-tooltip />
|
||||
<template v-else>
|
||||
<el-table-column label="操作人" prop="actorUsername" min-width="150" show-overflow-tooltip />
|
||||
<el-table-column label="目标账号" min-width="150" show-overflow-tooltip><template #default="scope">{{ scope.row.targetUsername || '—' }}</template></el-table-column>
|
||||
<el-table-column label="操作" min-width="120"><template #default="scope">{{ actionLabels[scope.row.action] || '未知操作' }}</template></el-table-column>
|
||||
</template>
|
||||
<el-table-column label="结果" width="90"><template #default="scope"><el-tag :type="scope.row.result === 'success' ? 'success' : 'danger'">{{ scope.row.result === 'success' ? '成功' : '失败' }}</el-tag></template></el-table-column>
|
||||
<el-table-column label="说明" min-width="170"><template #default="scope">{{ reasonLabels[scope.row.reason] || '未知原因' }}</template></el-table-column>
|
||||
<el-table-column v-if="kind === 'login'" label="IP 地址" prop="ip" min-width="150" show-overflow-tooltip />
|
||||
</el-table>
|
||||
<pagination v-show="logs.total" v-model:page="page" v-model:limit="limit" :total="logs.total" :page-sizes="[20, 50, 100]" :auto-scroll="false" layout="total, sizes, prev, pager, next" @pagination="paginate" />
|
||||
</el-card></template></basic-layout>
|
||||
</template>
|
||||
<script>
|
||||
import BasicLayout from '../layout/BasicLayout.vue'
|
||||
import Pagination from '../components/Pagination/index.vue'
|
||||
import { session } from '../store'
|
||||
import { actionLabels, reasonLabels, createAuditLogLoader } from '../audit-logs.mjs'
|
||||
const emptyFilters = () => ({ username: '', result: '', action: '', from: null, to: null })
|
||||
export default {
|
||||
name: 'AuditLogs', components: { BasicLayout, Pagination },
|
||||
props: { kind: { type: String, required: true } },
|
||||
data: () => ({ filters: emptyFilters(), applied: emptyFilters(), page: 1, limit: 20, logs: { items: [], total: 0, loading: false, error: '' }, actionLabels, reasonLabels }),
|
||||
watch: {
|
||||
kind() { this.loader.invalidate(); this.reset() },
|
||||
'$store.state.generation': { flush: 'sync', handler() { this.loader.invalidate(); this.filters = emptyFilters(); this.applied = emptyFilters() } }
|
||||
},
|
||||
created() { this.loader = createAuditLogLoader(session, this.logs) },
|
||||
mounted() { this.load() },
|
||||
beforeUnmount() { this.loader.invalidate() },
|
||||
methods: {
|
||||
load() { return this.loader.load(this.kind, { ...this.applied, page: this.page, limit: this.limit }) },
|
||||
search() { this.page = 1; this.applied = { ...this.filters }; this.load() },
|
||||
reset() { this.filters = emptyFilters(); this.search() },
|
||||
paginate({ page, limit }) { this.page = page; this.limit = limit; this.load() },
|
||||
formatDate(value) { const date = new Date(value); return Number.isNaN(date.getTime()) ? '—' : date.toLocaleString('zh-CN', { hour12: false }) }
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<style scoped>
|
||||
.toolbar { display: flex; align-items: baseline; gap: 16px; margin-bottom: 20px; flex-wrap: wrap; }
|
||||
h1 { font-size: 20px; margin: 0; }
|
||||
.retention { color: #909399; font-size: 13px; }
|
||||
.filters { display: flex; align-items: flex-end; gap: 0 16px; flex-wrap: wrap; }
|
||||
.filters .el-form-item { width: 200px; }
|
||||
.filters :deep(.el-date-editor) { width: 100%; }
|
||||
.filter-actions { display: flex; margin-bottom: 18px; }
|
||||
.el-alert { margin-bottom: 16px; }
|
||||
@media (max-width: 600px) { .filters .el-form-item { width: 100%; } }
|
||||
</style>
|
||||
@@ -0,0 +1,104 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import { createSession } from '../src/session.mjs'
|
||||
import * as audit from '../src/audit-logs.mjs'
|
||||
|
||||
function setup() {
|
||||
const pending = []
|
||||
const session = createSession({
|
||||
storage: { getItem() {}, setItem() {}, removeItem() {} },
|
||||
fetch: (url, options) => new Promise(resolve => pending.push({ url, options, resolve }))
|
||||
})
|
||||
session.state.user = { id: 1, role: 'admin', username: 'fixture.admin' }
|
||||
session.state.token = 'fictional-token'
|
||||
return { session, pending }
|
||||
}
|
||||
const response = data => ({ ok: true, status: 200, json: async () => ({ code: 200, data }) })
|
||||
|
||||
test('unclassified rejected account updates have a readable action and filter', () => {
|
||||
assert.equal(audit.actionLabels.update, '更新账号')
|
||||
assert.equal(new URLSearchParams(audit.auditQuery('operation', { action: 'update' })).get('action'), 'update')
|
||||
})
|
||||
|
||||
test('audit query normalizes exact username, preserves RFC3339 bounds and bounds pagination', async () => {
|
||||
const { session, pending } = setup()
|
||||
assert.equal(typeof session.queryAuditLogs, 'function')
|
||||
const request = session.queryAuditLogs('operation', { username: ' Fixture.Admin ', result: 'failure', action: 'reset_password', page: 2, limit: 100, from: new Date('2026-09-10T00:00:00+08:00'), to: new Date('2026-09-11T00:00:00+08:00') })
|
||||
const url = new URL(pending[0].url, 'https://fixture.invalid')
|
||||
assert.equal(url.pathname, '/api/v1/operation-logs')
|
||||
assert.deepEqual(Object.fromEntries(url.searchParams), { page: '2', limit: '100', username: 'fixture.admin', result: 'failure', action: 'reset_password', from: '2026-09-09T16:00:00.000Z', to: '2026-09-10T16:00:00.000Z' })
|
||||
assert.equal(pending[0].options.cache, 'no-store')
|
||||
pending[0].resolve(response({ items: [], total: 0, page: 2, limit: 100 }))
|
||||
assert.equal((await request).total, 0)
|
||||
for (const filter of [{ limit: 101 }, { page: 0 }, { result: 'unknown' }, { action: 'delete' }, { from: 'bad' }, { from: '2026-09-11', to: '2026-09-10' }]) {
|
||||
await assert.rejects(session.queryAuditLogs('operation', filter))
|
||||
}
|
||||
assert.equal(pending.length, 1)
|
||||
})
|
||||
|
||||
test('login query omits empty filters and rejects operation-only actions and unauthenticated access', async () => {
|
||||
const { session, pending } = setup()
|
||||
assert.equal(typeof session.queryAuditLogs, 'function')
|
||||
const request = session.queryAuditLogs('login', { username: ' ', result: '', from: null, to: null })
|
||||
assert.equal(pending[0].url, '/api/v1/login-logs?page=1&limit=20')
|
||||
pending[0].resolve(response({ items: [], total: 0, page: 1, limit: 20 }))
|
||||
await request
|
||||
await assert.rejects(session.queryAuditLogs('login', { action: 'create' }))
|
||||
session.clear()
|
||||
await assert.rejects(session.queryAuditLogs('login'), /请先登录/)
|
||||
assert.equal(pending.length, 1)
|
||||
})
|
||||
|
||||
test('late audit response cannot survive session change', async () => {
|
||||
const { session, pending } = setup()
|
||||
assert.equal(typeof session.queryAuditLogs, 'function')
|
||||
const request = session.queryAuditLogs('login')
|
||||
session.clear()
|
||||
session.state.user = { id: 3, role: 'admin', username: 'fixture.otheradmin' }
|
||||
pending[0].resolve(response({ items: [{ username: 'fixture.private' }], total: 1 }))
|
||||
await assert.rejects(request, /会话已变化/)
|
||||
})
|
||||
|
||||
test('route/filter changes discard out-of-order rows and invalidation clears loaded private data', async () => {
|
||||
assert.equal(typeof audit.createAuditLogLoader, 'function')
|
||||
const { session, pending } = setup()
|
||||
const state = { items: [], total: 0, loading: false, error: '' }
|
||||
const loader = audit.createAuditLogLoader(session, state)
|
||||
const old = loader.load('login', {})
|
||||
const current = loader.load('operation', {})
|
||||
pending[1].resolve(response({ items: [{ id: 2 }], total: 1 }))
|
||||
await current
|
||||
pending[0].resolve(response({ items: [{ id: 1 }], total: 1 }))
|
||||
await old
|
||||
assert.deepEqual(state.items, [{ id: 2 }])
|
||||
loader.invalidate()
|
||||
assert.deepEqual(state.items, [])
|
||||
assert.equal(state.total, 0)
|
||||
const late = loader.load('login', {})
|
||||
session.clear()
|
||||
pending[2].resolve(response({ items: [{ id: 3 }], total: 1 }))
|
||||
await late
|
||||
assert.deepEqual(state.items, [])
|
||||
assert.equal(state.error, '')
|
||||
})
|
||||
|
||||
test('loading page two preserves total so pagination cannot clamp the page back to one', async () => {
|
||||
const { session, pending } = setup()
|
||||
const state = { items: [], total: 0, loading: false, error: '' }
|
||||
const loader = audit.createAuditLogLoader(session, state)
|
||||
const first = loader.load('login', { page: 1 })
|
||||
pending[0].resolve(response({ items: [{ id: 50 }], total: 50 }))
|
||||
await first
|
||||
const second = loader.load('login', { page: 2 })
|
||||
assert.equal(state.total, 50)
|
||||
assert.equal(state.loading, true)
|
||||
assert.deepEqual(state.items, [])
|
||||
assert.equal(new URL(pending[1].url, 'https://fixture.invalid').searchParams.get('page'), '2')
|
||||
pending[1].resolve(response({ items: [{ id: 30 }], total: 50 }))
|
||||
await second
|
||||
assert.deepEqual(state.items, [{ id: 30 }])
|
||||
assert.equal(state.total, 50)
|
||||
loader.invalidate()
|
||||
assert.equal(state.total, 0)
|
||||
assert.deepEqual(state.items, [])
|
||||
})
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Project-Profile
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Project-Profile.-
|
||||
wiki_revision: 80cd91be7e64552b0205f45c59a2234934760b0e
|
||||
synchronized_at: 2026-09-10T09:38:10Z
|
||||
wiki_revision: 40ae5e3a0521195da4f1a9cd9f42beec15da20d4
|
||||
synchronized_at: 2026-09-10T12:43:19Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# LexGo 项目档案
|
||||
@@ -106,7 +106,7 @@ M0 在 LexGo 隔离工程中验证:指定前后端配套构建;MySQL 8 迁
|
||||
|
||||
## MVP 功能范围确认
|
||||
|
||||
2026-09-10 用户确认 F01~F12 全部进入首版,X01~X14 暂不纳入;随后授权使用 Quant-UX 制作原型,并同意先建原型工单,原型审核后再拆实施工单。桌面和手机均覆盖;默认学习语言已确认英语。工程基础已实现待验收,其余范围沿用已验收原型。
|
||||
2026-09-10 用户确认 F01~F12 全部进入首版,X01~X14 暂不纳入;随后授权使用 Quant-UX 制作原型,并同意先建原型工单,原型审核后再拆实施工单。桌面和手机均覆盖;默认学习语言已确认英语。工程基础已通过验收,其余范围沿用已验收原型。
|
||||
|
||||
原型站点为 https://qux.ilapage.cn/;v1 编辑源和预览统一见需求总览及工单 #1。用户要求减少页面说明、按目标页面呈现。账号配置不进入仓库。
|
||||
|
||||
@@ -119,7 +119,7 @@ M0 在 LexGo 隔离工程中验证:指定前后端配套构建;MySQL 8 迁
|
||||
|
||||
学习端与管理端均使用账号(用户名)+密码登录,账号不要求邮箱格式;邮箱不作为必填登录标识。两端共用账号体系,管理权限由服务端单独校验。用户名唯一性和大小写规则在账号工单中细化。
|
||||
|
||||
[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16) 已建立四阶段、14 张单元工单;详见需求总览与工作量估算。工程基础 #2 已实现并完成本地验证,等待用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。
|
||||
[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16) 已建立四阶段、14 张单元工单;详见需求总览与工作量估算。工程基础 #2 已于 2026-09-10 通过用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。
|
||||
|
||||
|
||||
## 工程基础实际基线(#2,2026-09-10)
|
||||
@@ -131,3 +131,8 @@ Go 1.26.5 通过 Go 独立工具链下载并校验,系统 Go 1.24.0 保持不
|
||||
server 是 go-admin 的选用模块接入:原样保留 SysUser、SysDept、必要 common/models 和 MIT 许可,新增 LexGo 路由、账号服务、会话与显式迁移;未接入整套后台路由、代码生成或任意任务执行。server/upstream.json 记录来源和文件摘要。admin/upstream.json 区分原样复用与适配的 go-admin-ui 布局、组件、页面和依赖,原始三个 go-admin 源工作区保持只读。
|
||||
|
||||
当前采用随机 Bearer 会话而非直接启用上游开发模式的超长 JWT。已验证退出、禁用、密码重置和到期拒绝;该差异是为满足当前可撤销会话要求,不宣称上游默认实现满足此要求。
|
||||
|
||||
|
||||
## 日志审计基线(#18)
|
||||
|
||||
管理端新增两个审计列表,后端为 LexGo 自有日志模型与接口,参照 go-admin 模块布局但不复制其原始参数/响应持久化逻辑。schema v2 新增 lexgo_login_logs、lexgo_operation_logs;保留 90 天,启动和每小时分批清理,也可显式 audit-cleanup。新建/重置密码均为 6~72 UTF-8 字节;初始 bootstrap 保留 10 字节下限。#2 已验收,#18 已验收;默认模块中的其他候选未纳入。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Architecture-and-Code-Map
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Architecture-and-Code-Map.-
|
||||
wiki_revision: 51d40a6d4f7d0bcc0882bcd7d9c544aae964afa8
|
||||
synchronized_at: 2026-09-10T09:38:13Z
|
||||
wiki_revision: 5bc7ccf33d1cc1cc28598c90e27a2ab6af4146bd
|
||||
synchronized_at: 2026-09-10T14:33:23Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 架构与代码地图
|
||||
@@ -23,7 +23,7 @@ Go 承担业务与后台任务,浏览器提供阅读学习界面,NLP 保留
|
||||
| `dev_scripts/harness.py`、`dev_scripts/wiki_docs.py` | 原样复制的 DevHarness 工具 | 治理工具入口,无业务 API |
|
||||
| `tests/` | 上游治理工具与文档结构测试 | 不验证阅读、NLP 或 SRS |
|
||||
|
||||
不存在产品入口、数据库迁移或前端页面。拟定职责:identity(身份)、library(书库)、ingestion(导入)、lexicon(全局词典)、vocabulary(个人词语)、review(复习)、progress(统计)、administration(管理)。底座固定后再决定具体目录。
|
||||
工程入口与已实现模块见下方 #2/#18;学习领域拟定职责:identity(身份)、library(书库)、ingestion(导入)、lexicon(全局词典)、vocabulary(个人词语)、review(复习)、progress(统计)、administration(管理)。底座固定后再决定具体目录。
|
||||
|
||||
## 两条主要执行路径
|
||||
|
||||
@@ -47,7 +47,7 @@ Go 承担业务与后台任务,浏览器提供阅读学习界面,NLP 保留
|
||||
|
||||
## 学习端与管理端的目标架构
|
||||
|
||||
技术方向已纳入本轮方案,以下是目标结构,尚无对应产品目录或可运行应用。
|
||||
以下是目标结构;账号、管理端与学习空空间已由 #2 实现。英语 NLP 已完成 #3 独立验证,尚未接入业务 API。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
@@ -56,7 +56,7 @@ flowchart TD
|
||||
API --> B[独立学习业务模块]
|
||||
B --> DB[(MySQL 8)]
|
||||
B --> W[后台任务:具体选型待验证]
|
||||
W --> N[NLP 服务:Python 方案待确认]
|
||||
W --> N[NLP:Python 小样已验证,生产集成待实施]
|
||||
```
|
||||
|
||||
| 交付部分 | 建设方式 | 复用与自建边界 |
|
||||
@@ -122,3 +122,56 @@ flowchart TD
|
||||
schema v1:sys_user 保留选用 go-admin 模型字段,唯一小写用户名;lexgo_spaces 以 owner_id 为主键;lexgo_sessions 保存 token_hash/owner_id/expires_at;lexgo_schema 记录版本与产品所有权。服务启动不自动迁移。迁移仅接受空库或合法已有 LexGo marker,拒绝空 marker、其他产品、负版本与未来版本;版本 0 可重试部分迁移,版本 1 幂等。
|
||||
|
||||
凭据字段只存在本地环境和必要数据库哈希中。后台账号密码更新使用表/字段更新,避免上游 BeforeUpdate hook 对已有哈希再次加密。账号行再会话行的锁顺序用于串行化撤销与请求;API 在事务提交后才返回成功。
|
||||
|
||||
|
||||
## 登录与操作审计(#18)
|
||||
|
||||
server/app/lexgo/audit.go 定义两类白名单字段日志、筛选分页、失败记录和过期清理;router.go 在登录/账号操作边界接入。登录成功时会话与日志同一事务;账号操作成功时业务写入与日志同一事务;失败时先回滚业务,再以独立、有 3 秒超时的事务写失败记录。日志写入失败返回通用 500,不输出数据库原始错误或凭据。
|
||||
|
||||
database.go 显式迁移至 v2,两张新增表均以 created_at/id 建立排序清理索引,账号字段建查询索引,无业务表级联删除。cmd/lexgo/main.go 的服务进程在启动和每小时执行审计清理,每次最多运行一分钟、每批删除 1000 条,仅影响过期审计记录。
|
||||
|
||||
admin/src/views/AuditLogs.vue 通过 kind 复用登录/操作列表;audit-logs.mjs 负责筛选编码和请求序号,session.mjs 继续进行管理员及会话 generation 校验。切换页面/账号清空日志,普通翻页保留总数,防止分页组件跳回第一页。菜单与标题按当前路由显示。
|
||||
|
||||
|
||||
## 英语分词与本地词典验证(#3,已验收)
|
||||
|
||||
`spikes/english/` 是独立可运行验证小样,不是学习端生产功能。推荐后续采用 Python 3.12.12、spaCy 3.8.7、en_core_web_sm 3.8.0(保留 tok2vec/tagger/attribute_ruler/lemmatizer,停用 parser/ner)和 NLTK 3.9.2 读取 WordNet 3.0。Go 继续管理用户、权限、任务和持久数据,后续通过显式契约调用 NLP;本单未新增 Go API、MySQL 表或常驻部署实例。
|
||||
|
||||
| 文件 | 作用 |
|
||||
|---|---|
|
||||
| engine.py | 原文分词、lemma、三个位置单位、直接/lemma 查词;只读本地资源 |
|
||||
| app.py、index.html、app.mjs、view.mjs、style.css | loopback 临时 HTTP 小样、输入/阅读/查词结果;单进程串行,输入不落盘 |
|
||||
| resources.json、setup_resources.py、requirements.lock | 固定版本、来源和 SHA256;显式联网准备,运行期无自动下载 |
|
||||
| test_engine.py、test_app.py、view.test.mjs | 真实模型离线验证、HTTP 边界与浏览器偏移/迟到响应测试 |
|
||||
| benchmark.py、benchmark-result.json | 虚构语料的候选对照、长文/查询实测及环境样本 |
|
||||
|
||||
WordNet 使用 ZIP 内原始 index/data/exception 文件,不使用 SysDict 或新增业务库。NLTK 默认 synsets 会隐式词形还原,本小样直接读取其固定版本索引以区分 exact 和显式 lemma;禁用依赖全局 corpus 的 OMW 跨版本映射,只接受 WordNet 3.0。升级 NLTK 或词典时必须重跑契约测试。
|
||||
|
||||
候选比较:正则分词+WordNet 默认名词 morphology 依赖少、速度快,但不具备上下文判断,缩写和词性歧义处理弱;纯 Go 规则同样需要自行维护这些语言规则。本次 spaCy 在 12 个明确样例中答对 11 个,基线 6 个,因此推荐保留独立 Python NLP 边界。样例量不足以证明总体准确率;不宣称部署或正式阅读功能已完成。
|
||||
|
||||
|
||||
## #4 阅读选择小样与 LinguaCafe 对照
|
||||
|
||||
小样位于 spikes/selection:serve.py 只提供白名单静态文件,app.mjs 负责 DOM 原生选择/键盘/面板状态,range.mjs 负责原文范围和匹配,fixtures.mjs 提供两章虚构文本。保存仅在内存 Map,刷新清空,不接入账号、MySQL、正式词典、复习或 #21 附件。使用 Intl.Segmenter 的词与字形边界验证 UI,不能替代 #3 的 spaCy 结果;正式阅读器必须以章节原文、内容版本与 NLP tokens 为共同基准。
|
||||
|
||||
参考版本固定 LinguaCafe c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8。本次实际读取以下源码;未启动 LinguaCafe、未亲测其浏览器或手机行为。下表“参考行为”是源码证据;LexGo 方案是独立实现与取舍,不应标为上游已经验证的体验。
|
||||
|
||||
| 项目 | 参考行为与来源 | LexGo 采用及保留差异 |
|
||||
|---|---|---|
|
||||
| 阅读器 | TextBlockGroup.vue L56–100 用词/短语 stage、selected、hover 等状态;InteractiveTextStyling.scss L20–126 由主题决定颜色。TextReaderChapterList.vue L15–43 展示章节统计及已处理章节阅读入口;TextReader.vue L195–214 提供阅读完成后的书库/下一章 | 保留正文上下文、词语状态与选中区分、章节切换;小样以两章和学习中/已认识/忽略验证,章节统计及已读持久化仍属后续功能。侧栏/底部面板为 LexGo 布局,不宣称复制上游排版 |
|
||||
| 查词面板 | VocabularyBox.vue L124–169、374–404 区分 Translation 与字典搜索,新短语需 Save phrase;TextBlockGroup.vue L1351–1380 失选时自动保存单词/已有短语。VocabularySearchBox.vue L149–179 及保存 catch 未证明完整错误反馈 | 沿用 v1 的词典释义/我的释义、明确保存、关闭继续阅读。LexGo 关闭不自动保存;保存后可见反馈,无词典/无结果仍可手填。小样用固定虚构释义与故障状态,不把它当成真实词典/网络重试验证 |
|
||||
| 短语与键盘 | TextBlockGroup.vue L399–599 自定义鼠标范围;L374–457 为手机 500ms 长按及后续触摸移动,选区开始后阻止默认滚动;L1071–1202 的 Shift+方向键跳高亮词,而非扩展范围,Esc 失选 | LexGo 采用原生鼠标拖选、手机长按/系统手柄,不拦截 touchmove;起止端点按钮可调整。←/→ 相邻词,Shift+←/→ 扩缩范围,Esc 取消。原型的预设短语按钮被真实正文选区替代,属于 #4 明确要求的验证;手机手柄是否与底部面板冲突仍待真机 |
|
||||
| 复习 | Review.vue L270–345、559–692 为 Reveal→I was correct/Again,正确移除卡、Again 保留并随机抽剩余卡,最后一张正确完成;ReviewHotkeyInformationDialog.vue L14–24 提供快捷键。练习模式不写状态 | 已验收 v1 保留中文显答、答对/答错、再学与完成,单词/短语分别有状态。上游随机下一卡、阶段降级和快捷键不是本单已实现行为;#8 再明确调度、重学和幂等,#4 不新增复习引擎 |
|
||||
|
||||
源码链接:
|
||||
- [resources/js/components/Text/TextBlockGroup.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Text/TextBlockGroup.vue)
|
||||
- [resources/sass/Text/InteractiveTextStyling.scss](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/sass/Text/InteractiveTextStyling.scss)
|
||||
- [resources/js/components/TextReader/TextReaderChapterList.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/TextReader/TextReaderChapterList.vue)
|
||||
- [resources/js/components/TextReader/TextReader.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/TextReader/TextReader.vue)
|
||||
- [resources/js/components/Text/VocabularyBox.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Text/VocabularyBox.vue)
|
||||
- [resources/js/components/Text/VocabularySearchBox.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Text/VocabularySearchBox.vue)
|
||||
- [resources/js/components/TextReader/TextReaderHotkeyInformationDialog.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/TextReader/TextReaderHotkeyInformationDialog.vue)
|
||||
- [resources/js/components/Review/Review.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Review/Review.vue)
|
||||
- [resources/js/components/Review/ReviewHotkeyInformationDialog.vue](https://github.com/simjanos-dev/LinguaCafe/blob/c1ea298ce40c65b9dd33e9b26fd2e52fae66f2c8/resources/js/components/Review/ReviewHotkeyInformationDialog.vue)
|
||||
|
||||
上游 LICENSE 为 GPL v3,本单只核对并描述行为,没有移植源码。#1 评论 7497 的四项对照登记由本节补充;原型 v1 的保存/关闭主要流程保持,不更改其历史验收记录,不声称已经在 Quant-UX 新建修订版。范围调整与原生手柄作为可运行小样验证,若真机结果导致主要流程变化,应先更新关键原型状态并由用户确认。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Business-Rules-and-Glossary
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Business-Rules-and-Glossary.-
|
||||
wiki_revision: c6044b9b3f863c90363563e079dff3e3fcb89bd8
|
||||
synchronized_at: 2026-09-10T12:13:06Z
|
||||
wiki_revision: 227c6cb92d30490f0f1f4b1b9f29cdbed8559ee4
|
||||
synchronized_at: 2026-09-10T14:33:25Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 业务规则与术语
|
||||
@@ -71,3 +71,44 @@ M0 固定首发语言语料、词条身份规则、短语选择与重叠规则
|
||||
|
||||
|
||||
用户进一步确认重置密码下限也为 6 字节。新增和重置共用同一密码校验,前后端均为 6~72 UTF-8 字节;重置密码事务仍撤销该用户全部旧会话。首次管理员 bootstrap 的 10 字节下限保持原样。
|
||||
|
||||
|
||||
## 审计记录、查询和保留规则
|
||||
|
||||
- 登录日志保存时间、规范用户名、成功/失败、固定原因码和连接来源 IP。覆盖成功、凭据错误/禁用、非法请求、限流;非法用户名不保存原文。未进行地理定位或保存 User-Agent。
|
||||
- 操作日志保存管理员编号/账号、目标编号/账号、动作、结果、固定原因码、时间。动作 create、enable、disable、reset_password;无法解析明确动作的非法 PATCH 记 update。一次同时修改状态和密码生成两条动作日志。只有已认证管理员发起的管理操作进入此表;未认证或学习者的越权查询不记录原始身份输入。
|
||||
- 日志绝不保存密码、token、Cookie、请求/响应正文、错误堆栈或私人学习内容。合法账号、IP 属于本功能必要的审计数据,仅管理员可查询。
|
||||
- GET /api/v1/login-logs 与 /operation-logs:未登录 401,学习者 403;page 默认 1,limit 默认 20、最大 100。username 精确匹配;操作日志匹配操作人或目标账号。result 为 success/failure,action 限定枚举,from/to 为 RFC3339。返回 data.items/total/page/limit,按时间和编号倒序,时间按毫秒存储、浏览器按本地时区展示。
|
||||
- 固定保留最近 90 天,查询即排除过期记录;默认起止为保留边界和当前时间。清理只删除 created_at 严格早于边界的两表记录,不影响账号、空间、会话。无清空全部或导出按钮;本期不开放保留时长配置。
|
||||
|
||||
|
||||
## #3 英语位置与查询实验契约 v1
|
||||
|
||||
实验版本 `english-spike-v1`。POST /analyze 接收 {text},返回 status、contract_version、original_text、text_sha256 和 tokens;仅为本机小样接口,不是生产 API。原文以收到的字符串为准,不先做 NFC、大小写、换行或空白归一化;SHA256 对原文 UTF-8 字节计算。tokens 连续覆盖全文,拼接 text 必须逐字符等于原文,空白也有独立区间。空串合法;最多 100000 Unicode code point,拒绝孤立代理项。kind 为 word/space/punctuation;word 是本小样的可点 token 类别,也可能包含数字或 emoji,不保证是自然语言词条。
|
||||
|
||||
每个 token 提供 text、lemma、kind 与半开区间 [start,end):
|
||||
|
||||
| 字段后缀 | 单位与使用方 |
|
||||
|---|---|
|
||||
| cp | Unicode code point,Python 字符串索引;不是用户感知字形 |
|
||||
| utf8 | UTF-8 字节,可供 Go string 字节切片 |
|
||||
| utf16 | UTF-16 code unit,JavaScript String.slice / DOM 文本位置 |
|
||||
|
||||
例:原文 `A🙂é`,emoji 的 cp=[1,2)、utf8=[1,5)、utf16=[1,3);后面的 e 加组合重音共两个 code point,cp=[2,4)、utf8=[5,8)、utf16=[3,5)。不能把这些单位混用,也不能把组合字符或 ZWJ 序列的 code point 数当作可见字符数。浏览器先逐 token 校验 UTF-16 切片并检查完整重建再展示;textarea 会按浏览器规范将换行转为 LF,所以 HTTP 契约保证收到的原文,不承诺还原剪贴板进入 textarea 前的 CRLF。服务端 CRLF 原文测试单独覆盖。
|
||||
|
||||
POST /lookup 接收 {surface,lemma?}。查词键单独 casefold/NFC/弯撇号转 ASCII,不改变原文位置;先精确查询 surface,再尝试调用方提供的 lemma。结果 status 为 exact、lemma、not_found 或 resource_missing,含 matched_form 和最多 12 条 entries(lemma/pos/definition/examples)。`dog` 直接命中;点击 `went` 可用上下文 lemma `go` 回退;手动只输入 `went` 不猜词性而返回未找到。词典缺失和模型缺失分别标识 wordnet/model,不能伪装成查无结果。
|
||||
|
||||
实验查询无用户状态、写入或缓存私人输入;重复查询确定性返回,原文哈希可检测文本版本变化,但尚未定义生产 token ID、任务幂等或个人词语合并规则。lemma 不等于学习状态身份,禁止自动合并原词/词元。WordNet 仅英英释义,按 n/v/a/r 与原生 sense 顺序截取,不做上下文义项排序、翻译或发音;`The leaves fell.` 的 leaves 实测被模型错误还原为 leave,此限制保留供后续用户选择/修正方案参考。
|
||||
|
||||
|
||||
## #4 阅读选择与重叠匹配建议
|
||||
|
||||
位置始终引用当前章节不变的原文,半开区间 [start,end)。浏览器 DOM Range 的 start/end 先按正文文本节点累计为 UTF-16,再扩展至触及的完整可选择词元;反向拖选得到同一规范区间。首尾空白/纯标点不独立成词,选区内部标点、空白和换行原样保留。emoji 字形可选,代理对、组合重音和 ZWJ 不拆开;cp/utf8/utf16 均与 #3 单位约定一致。
|
||||
|
||||
推荐以章节 ID、内容版本、原文区间标识一次出现,不使用短语文字定位所有实例。重复出现的 a small step 保持不同 offsets;查找采用区分大小写的精确原文匹配,并要求完整词元端点,不擅自折叠空白或 NFC。例:`a small step; a small step` 两次区间分别为 [0,12)、[14,26)。跨行 `small\nstep` 不与 `small step` 自动等同。
|
||||
|
||||
重叠匹配保留所有候选;只在显示层采用“起点靠前优先,同起点长范围优先,跳过已占用区间”,不合并或删除词条身份。例如 a small step 与 small step 都可保存,前者优先显示;当前用户选区高亮优先于已保存状态。#11 正式集成前再依据实际语言 tokens 验证候选索引与长度限制,本小样不继承未经实测的上游 14 词上限。
|
||||
|
||||
桌面鼠标/手机长按使用原生 Selection;正文中键盘 ←/→ 选相邻词,Shift+←/→ 从锚点扩缩连续范围,起点/终点按钮可用 Tab/Enter。Esc 或关闭清除选区及未保存编辑,保留滚动和键盘阅读位置;切换章节清除选区,当前页面内分别记住章节滚动位置。纯标点的新选区清除旧面板,防止操作上一个词。个人释义必须显式保存,状态变化也随保存提交;本小样仅写内存。
|
||||
|
||||
手机方案保留浏览器原生长按、选择手柄和滚动,面板最多占底部 42dvh,正文有底部阅读余量;键盘定位用滚动边距避开面板。未获得真实手机结果,不能判定手柄、系统菜单、虚拟键盘或触摸滚动冲突已经解决。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Local-Development-and-Verification
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Local-Development-and-Verification.-
|
||||
wiki_revision: a12e404e317f2f106936b99147f993d7cc6da608
|
||||
synchronized_at: 2026-09-10T12:13:08Z
|
||||
wiki_revision: 694facbb7a861182122c9f6c69d28c58b005cb1a
|
||||
synchronized_at: 2026-09-10T14:52:22Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 本地开发与验证
|
||||
@@ -176,3 +176,73 @@ supervisor 直接管理编译后的 Go 进程,运行时不调用 Python。数
|
||||
|
||||
|
||||
重置密码下限已同步为 6 字节。管理端测试与真实 MySQL 集成测试覆盖 5/6/72/73 字节及多字节字符,验证新密码登录成功、旧密码和旧会话被拒绝。管理员页面的新增/重置提示均为 6–72 字节;本次不修改现有账号密码。
|
||||
|
||||
|
||||
## 审计模块启动与验证(#18 / schema v2)
|
||||
|
||||
首次更新此版本时先在 supervisor 停止 lexgo-api,再从仓库根目录执行 python scripts/server.py build、python scripts/server.py migrate,随后启动 lexgo-api。不要在 Windows 中覆盖正在运行的 exe。新服务要求 schema v2,启动本身不迁移;旧 v1 的账号、学习空间、会话完整保留。
|
||||
|
||||
本机已执行 v1→v2 并验证 sys_user 4 行、lexgo_spaces 4 行、lexgo_sessions 2 行迁移前后的整体指纹相同。原二进制仅在忽略目录 .local/lexgo-pre-audit.exe 留作本机回退,不提交 Git。后续正常启动仍通过 supervisor;登录日志入口 http://127.0.0.1:5174/#/login-logs,操作日志入口 http://127.0.0.1:5174/#/operation-logs。
|
||||
|
||||
手动清理过期审计记录:python scripts/server.py audit-cleanup;仅删除两张审计表中超过 90 天的记录,执行前确认本机 LEXGO_DB_NAME。后台自动清理在服务启动时及每小时运行,每次限制一分钟,失败只记录固定提示并于下一小时重试。
|
||||
|
||||
回退到已验收 #2 时:停止 API,确认没有其他 v2 实例,再恢复 .local/lexgo-pre-audit.exe;将 lexgo_schema 中 id=1、product=lexgo 的版本从 2 改回 1 后启动旧程序。两张日志表保留,不删除,旧程序不继续记录新日志;重新升级时显式 migrate 恢复 v2。本次已验证增量/重复迁移及业务数据保留,没有在开发库演练回退。
|
||||
|
||||
后端 MySQL 集成测试覆盖审计权限、查询边界、创建/启停/重置审计、无敏感输入、事务回滚、90 天边界、增量与重复迁移;管理端 24 项测试、lint 和生产构建通过。浏览器已展示真实登录与重复创建失败记录。为翻页检查生成了 21 条 fixture.audit 虚构账号登录失败记录,不创建该账号;记录按相同 90 天规则保留。浏览器自动化点击未触发页面事件,用户本次已确认日志模块验收通过;自动化点击检查仍保留原始未通过记录。
|
||||
|
||||
|
||||
#18 用户验收:2026-09-10T20:40:04+08:00 用户确认日志通过验收(工单评论 7591),包含此前待人工检查的交互。未重新运行自动化测试,未更改其历史结果,未合并 PR 或发布生产。
|
||||
|
||||
|
||||
## #3 英语离线验证入口与复现
|
||||
|
||||
从仓库根目录执行(uv 与 Node 已安装,不能使用本机默认 Python 3.8):
|
||||
|
||||
```powershell
|
||||
uv venv --python 3.12.12 .local/nlp-venv
|
||||
uv pip install --python .local/nlp-venv/Scripts/python.exe -r spikes/english/requirements.lock
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/setup_resources.py
|
||||
uv pip install --python .local/nlp-venv/Scripts/python.exe --no-deps .local/nlp-resources/en_core_web_sm-3.8.0-py3-none-any.whl
|
||||
.local/nlp-venv/Scripts/python.exe -m unittest discover -s spikes/english -v
|
||||
node --test spikes/english/view.test.mjs
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/benchmark.py
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/app.py
|
||||
```
|
||||
|
||||
打开 http://127.0.0.1:5183/,默认虚构样例,分析后点击 went 应出现 go 与“按原形查询”;dog 直接命中,zzzxqvfiction 未找到。`--resources .local/absent-resources` 可验证词典缺失,`--port` 可更换临时端口。模型缺失、词典缺失、非法输入及内部错误不输出路径/正文。仅 loopback,Host/Origin 校验,禁跨域、无缓存、无访问日志、连接读超时 10 秒。未配置 supervisor;停止该临时进程即可回退,既有服务和数据不变。
|
||||
|
||||
首次准备需要联网,失败可重跑;资源文件通过固定 SHA256 校验后使用。模型 3.8.0 MIT,WordNet 3.0 ZIP 完整保留 LICENSE/版权/免责声明,spaCy MIT、NLTK Apache-2.0。固定资源 URL 和摘要见 spikes/english/resources.json,原始许可与来源见该目录 README。词典为英英格式,不是中文翻译库。
|
||||
|
||||
2026-09-10 实测:11 项 Python 测试和 2 项 JavaScript 测试通过。真实模型/词典测试及 benchmark 禁止 socket connect,验证运行期无在线翻译依赖;不是整机断网测试。浏览器已实测展示分词、点击 went→go、手动查询无结果。外部 spaCy/Click 有一条 DeprecationWarning,未影响测试结果。Windows 10 19044,Intel Family 6 Model 140、8 逻辑核,Python 3.12.12;完整环境、UTC 时间和样本保存在 benchmark-result.json。
|
||||
|
||||
| 测量 | 本次样本 |
|
||||
|---|---|
|
||||
| 冷进程 Engine 加载(含 import,文件系统缓存可能已热) | 2355 ms |
|
||||
| 首次分析 / 首次 dog 查询 | 见 benchmark-result.json(各 1 次) |
|
||||
| 100000 code point(106095 UTF-8 字节),spaCy 3 次 | 中位 1433 ms,约 6.98 万 cp/s |
|
||||
| 同文正则+WordNet morphology,3 次 | 中位约 336 ms;未包含三位置转换,非完全等价负载 |
|
||||
| dog 查询,热进程 100 次 | 中位 0.0149 ms,p95 0.023 ms |
|
||||
| 12 个显式 lemma 样例 | spaCy 11/12、基线 6/12;保留 leaves 错误 |
|
||||
|
||||
推荐 Python NLP,但该样本不代表一般准确率、生产并发能力或延迟保证。尚未验证正式 Go/Python 调用、长任务持久化、移动端划词(#4)、英汉词典及生产部署。
|
||||
|
||||
|
||||
#3 于 2026-09-10T21:50:14+08:00 通过用户验收(工单评论 7609),独立小样验证完成。此前实测结果和限制保持;本次验收未重跑测试。
|
||||
|
||||
|
||||
## #4 阅读选择验证
|
||||
|
||||
启动:`python spikes/selection/serve.py`,本机 http://127.0.0.1:5184/。默认仅 loopback,提供白名单静态文件,无目录列表或写入接口。虚构英文两章,任意词/连续短语可选;模拟释义与保存仅页面内有效。完整操作及真机清单见 spikes/selection/README.md;本单不配置 supervisor。
|
||||
|
||||
自动测试:`node --test spikes/selection/range.test.mjs spikes/selection/ui.test.mjs`。Node 22.22.1;范围测试无外部依赖,DOM 测试复用 learner 锁定的 jsdom(需要时先 `pnpm --dir learner install --frozen-lockfile`)。11 项通过,覆盖原文分区、反选、词边界、跨行/标点、组合字符/emoji/ZWJ、三位置单位、重复/重叠、关闭后键盘续读、标点取消旧选区、DOM Range 映射与显式保存。关闭丢失键盘位置及标点残留均复现旧代码失败,再验证修正通过。JS 语法检查通过。
|
||||
|
||||
桌面 Chrome 实际验证:鼠标拖选 `Mira opened the workshop`,原生选择与正文高亮相等;按钮扩展终点后可包含 before;键盘连续选择及关闭后下一词续读正常;模拟保存与词典不可用提示正常。章节中段直接点击关闭前后 scrollY 同为 909.5999755859375。自动化 locator.click 会在事件之前滚动到按钮,因此用基于可见坐标的直接点击复核;页面“验证设置”记录事件内关闭前后位置,避免将工具预滚动误判为页面变化。
|
||||
|
||||
桌面窄屏约 390×844 CSS 像素检查无横向溢出,底部面板正常;键盘选中词底部 464.4、面板顶部 489.5,未被遮挡。此为桌面浏览器宽度检查,不是移动系统或触摸模拟结果。视口已恢复。
|
||||
|
||||
**真机项未通过:尚未取得实际手机浏览器结果。** 需要用户记录设备/系统/浏览器版本、长按/手柄扩缩跨行范围、普通滚动、底部面板遮挡、虚拟键盘以及关闭位置,回填 #4。可在明确进行测试时以 `--host` 指定电脑实际局域网 IPv4,手机同网访问;不自动放行防火墙或开放公网。用户已明确验收 #4 并关闭工单;该详细测试证据缺口仍保留,不能将其改记为已执行通过。
|
||||
|
||||
独立规格及代码评审已修复已知问题;没有真实账号、生产数据、词典联网或 SRS 测试。停止小样即可回退,现有学习端/管理端/数据库不变。#21 附件不在本次范围。
|
||||
|
||||
|
||||
#4 用户验收记录:2026-09-10T22:51:52+08:00 用户确认“#4通过验收”(评论 7636)。未补充手机型号/浏览器/操作记录,未重跑测试,未自动合并 PR。后续正式移动端集成应补真机回归。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Product-Requirements-Overview
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Product-Requirements-Overview.-
|
||||
wiki_revision: e506edcf2487b3d0faf035bf0a6826332bccd994
|
||||
synchronized_at: 2026-09-10T09:38:24Z
|
||||
wiki_revision: db82eeb1d509b8d3c4a2204143cfb0e829b6b083
|
||||
synchronized_at: 2026-09-10T12:43:32Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# 产品需求总览
|
||||
@@ -15,7 +15,7 @@ synchronized_at: 2026-09-10T09:38:24Z
|
||||
|
||||
## 事实来源边界
|
||||
|
||||
当前产品实现:账号与独立空间工程基础已实现待验收,其余学习业务待实施。用户已确认多账号、数据独立、自托管和 F01~F12;已建立原型工单 #1 及 Quant-UX 桌面/手机设计。下方 U/A/N 与 M0~M4 保留为研究索引,不能扩大已批准 MVP,也不能用原型完成度代替产品实现进度。
|
||||
当前产品实现:账号与独立空间工程基础已通过验收,其余学习业务待实施。用户已确认多账号、数据独立、自托管和 F01~F12;已建立原型工单 #1 及 Quant-UX 桌面/手机设计。下方 U/A/N 与 M0~M4 保留为研究索引,不能扩大已批准 MVP,也不能用原型完成度代替产品实现进度。
|
||||
|
||||
## 当前需求索引
|
||||
|
||||
@@ -193,7 +193,7 @@ Anki(U20)、YouTube/Jellyfin 远程字幕(U07/A08)、全量旧实例迁
|
||||
|
||||
学习端与管理端均使用账号(用户名)+密码登录,账号不要求邮箱格式;邮箱不作为必填登录标识。两端共用账号体系,管理权限由服务端单独校验。用户名唯一性和大小写规则在账号工单中细化。
|
||||
|
||||
[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16) 汇总四阶段与依赖,各单均包含完整路径、参考模块、设计证据、验收和人日估算。工程基础 #2 已实现并完成本地验证,等待用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。
|
||||
[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16) 汇总四阶段与依赖,各单均包含完整路径、参考模块、设计证据、验收和人日估算。工程基础 #2 已于 2026-09-10 通过用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。
|
||||
|
||||
| 阶段 | 工单 | 覆盖 | 预计人日 |
|
||||
|---|---|---|---:|
|
||||
@@ -215,6 +215,11 @@ Anki(U20)、YouTube/Jellyfin 远程字幕(U07/A08)、全量旧实例迁
|
||||
|
||||
## 工程基础交付边界(#2)
|
||||
|
||||
工程基础 #2 已实现并完成本地验证,等待用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。
|
||||
工程基础 #2 已于 2026-09-10 通过用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。
|
||||
|
||||
用户名登录与账号管理覆盖 B01/B02 和 F12 的登录基础;所有 F 学习业务仍需按 #3~#15 验收。#2 的会话和权限测试不能证明尚未存在的书籍/词汇/复习权限已通过。
|
||||
|
||||
|
||||
## 试用前管理能力增补
|
||||
|
||||
用户于 2026-09-10 验收 #2,并批准新增 #18 登录日志与操作审计:两个管理员列表、查询筛选分页、必要字段记录、权限及 90 天保留清理。沿用现有 go-admin 管理布局,无需独立高保真原型。已通过用户验收,纳入 #16“邀请用户试用前完成”;不替代 #3 英语分词和 #4 阅读选择验证,也不引入其他 go-admin 默认模块。
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Workload-Estimate
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Workload-Estimate.-
|
||||
wiki_revision: de8fcf81b07bd7e96df1c68e5a63914139c62b6c
|
||||
synchronized_at: 2026-09-10T09:38:31Z
|
||||
wiki_revision: f43e3793edfa02d319d5a1d566e082d3f3a1361d
|
||||
synchronized_at: 2026-09-10T14:52:36Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# LexGo MVP 工作量估算
|
||||
@@ -152,3 +152,21 @@ Quant-UX 桌面和手机原型 v1 已建立,待人工审核;这不计为产
|
||||
## 工程基础进展(#2)
|
||||
|
||||
账号与独立空间基础已实现并本地验证,待用户验收。原计划 #2 为 6~9 人日,这不是实际消耗;未记录人工有效工时,不按 AI 会话耗时反推。#2 验收后,原拆分中 #3~#15 的规划工作量合计 46~75 人日,仍需在英语资源/划词验证后重估。本页顶部 52~84 为本轮开始时的完整 MVP 基线,不重复计入新增工作。
|
||||
|
||||
|
||||
## #2 验收与 #18 增补后的当前估算
|
||||
|
||||
#2 已验收,原 52~84 人日 MVP 计划扣除基础单元 6~9 人日后,原范围剩余 46~75 人日。新增 #18 日志审计独立计划 3~5 人日,扩展后全范围原始预算 55~89 人日、#18 验收前的剩余计划参考 49~80 人日。#18 已通过用户验收,本次验收后剩余规划为 46~75 人日。人日是规划参考,不把本次代理运行时间折算为已消耗人日。
|
||||
|
||||
|
||||
#18 已于 2026-09-10T20:40:04+08:00 验收。#2 与 #18 完成后,剩余 #3~#15 的规划参考为 46~75 人日;历史 49~80 人日是 #18 尚未验收时的区间。
|
||||
|
||||
|
||||
## #3 验收后的剩余计划
|
||||
|
||||
2026-09-10T21:50:14+08:00 用户验收 #3。从此前 46~75 人日扣除该单原计划 3~5 人日后,#4~#15 剩余规划参考为 43~70 人日。#2、#3、#18 均已验收;这不是实际消耗工时,后续仍需结合划词验证重估。
|
||||
|
||||
|
||||
## #4 验收与附件增补后的剩余计划
|
||||
|
||||
2026-09-10T22:51:52+08:00 用户验收 #4。原范围从43~70扣除本单3~5后,#5~#15剩余40~65人日;加已建单 #21 书籍音频/封面初估4~6后,当前剩余规划44~71人日。是计划参考,不代表实际耗时;#21仍待原型与限制细化。
|
||||
|
||||
+18
-4
@@ -2,8 +2,8 @@
|
||||
generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件)
|
||||
wiki_page: Home
|
||||
wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Home
|
||||
wiki_revision: 5ff3348f14ba1ffa810be3996b9340526a689529
|
||||
synchronized_at: 2026-09-10T09:38:08Z
|
||||
wiki_revision: a645b99632c70734b493ed030ad6d4d2acfd13be
|
||||
synchronized_at: 2026-09-10T14:52:14Z
|
||||
<!-- gitea-wiki-mirror:end -->
|
||||
|
||||
# LexGo 文档入口
|
||||
@@ -27,7 +27,7 @@ synchronized_at: 2026-09-10T09:38:08Z
|
||||
|
||||
## 事实来源
|
||||
|
||||
Wiki 保存长期规则,工单保存单次实施与验收,Git 保存源码和镜像。当前工程基础已实现待验收,已有调研不等于运行验收。数据库选择以用户确认的 MySQL 8 为准,历史 PostgreSQL 建议不再适用。
|
||||
Wiki 保存长期规则,工单保存单次实施与验收,Git 保存源码和镜像。当前工程基础已通过验收,已有调研不等于运行验收。数据库选择以用户确认的 MySQL 8 为准,历史 PostgreSQL 建议不再适用。
|
||||
|
||||
原有四份调研资料作为证据页面保留:[需求提取](https://git.ilapage.cn/OPC/lexgo/wiki/LinguaCafe-Requirements.-)、[Go 分析](https://git.ilapage.cn/OPC/lexgo/wiki/Go-Architecture-Analysis.-)、[另一份需求提取](https://git.ilapage.cn/OPC/lexgo/wiki/LinguaCafe-Requirements-Alternative.-)、[另一份 Go 分析](https://git.ilapage.cn/OPC/lexgo/wiki/Go-Analysis-Alternative.-)。其正文为既有调研记录,不将整份建议视为已批准方案。
|
||||
|
||||
@@ -43,9 +43,23 @@ F01~F12 已获用户确认,X 系列暂不纳入。Quant-UX 桌面与手机
|
||||
|
||||
原型制作进度与证据见[原型工单 #1](https://git.ilapage.cn/OPC/lexgo/issues/1)。
|
||||
|
||||
Quant-UX 原型 v1 已通过用户验收。[桌面预览](https://qux.ilapage.cn/#/simulate.html?h=a2aa10as5WpKpxe9fjDXETLqmigWeGWqsSjGmNmERlfgTpjXambHTDwagN8C&s=true&log=false)、[手机预览](https://qux.ilapage.cn/#/simulate.html?h=a2aa10aFDLzpVy0xPE70ftx5adTPyjWy5bj6nhPk2kDkJEwtIf1GiZjqx1n6&s=true&log=false)。工程基础 #2 已实现并完成本地验证,等待用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。
|
||||
Quant-UX 原型 v1 已通过用户验收。[桌面预览](https://qux.ilapage.cn/#/simulate.html?h=a2aa10as5WpKpxe9fjDXETLqmigWeGWqsSjGmNmERlfgTpjXambHTDwagN8C&s=true&log=false)、[手机预览](https://qux.ilapage.cn/#/simulate.html?h=a2aa10aFDLzpVy0xPE70ftx5adTPyjWy5bj6nhPk2kDkJEwtIf1GiZjqx1n6&s=true&log=false)。工程基础 #2 已于 2026-09-10 通过用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。
|
||||
|
||||
|
||||
## 当前实施入口
|
||||
|
||||
[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16)。14 张单元工单,预计 52~84 人日;实际产品实现未开始。学习端与管理端均使用账号(用户名)+密码登录,不要求邮箱。详见需求总览和工作量表。
|
||||
|
||||
|
||||
## 当前管理端交付
|
||||
|
||||
#2 已验收并关闭(记录 7576)。新增 [#18 登录日志与操作审计](https://git.ilapage.cn/OPC/lexgo/issues/18),已通过用户验收,列入 #16 的试用前准备。管理端提供登录日志、操作日志查询,默认保留 90 天;数据库为显式迁移后的 schema v2。代码分支 feat/18-audit-logs,基于已验收的 feat/2-account-foundation;#17 尚未合并,因此新 PR 以该基础分支为目标,保持差异只含 #18。
|
||||
|
||||
|
||||
日志审计 #18 于 2026-09-10T20:40:04+08:00 获用户验收并关闭;#16 已更新完成索引。该验收时 #3/#4 尚未开始。
|
||||
|
||||
|
||||
英语分词/原文定位/本地词典 #3 独立可运行小样已通过用户验收并关闭。代码与复现命令位于 spikes/english;临时入口 http://127.0.0.1:5183/。推荐 Python spaCy 英语模型与 WordNet 3.0 的离线组合;尚未接入正式学习端。
|
||||
|
||||
|
||||
#4 阅读选择小样已实现桌面鼠标/键盘、范围调整与原文位置验证,入口 http://127.0.0.1:5184/;固定版本 LinguaCafe 四项源码对照已记录。用户已验收并关闭 #4;真实手机详细测试证据仍缺失,详见本地验证页。#21 仍待实施。
|
||||
|
||||
+2
-1
@@ -12,12 +12,13 @@ def main():
|
||||
"migrate": ["go", "run", "./cmd/lexgo", "migrate"],
|
||||
"bootstrap": ["go", "run", "./cmd/lexgo", "bootstrap"],
|
||||
"serve": ["go", "run", "./cmd/lexgo", "serve"],
|
||||
"audit-cleanup": ["go", "run", "./cmd/lexgo", "audit-cleanup"],
|
||||
"build": ["go", "build", "./cmd/lexgo"],
|
||||
"test": ["go", "test", "./...", "-count=1"],
|
||||
"test-integration": ["go", "test", "./...", "-count=1", "-v"],
|
||||
}
|
||||
if len(sys.argv) != 2 or sys.argv[1] not in commands:
|
||||
raise SystemExit("Usage: python scripts/server.py migrate|bootstrap|serve|build|test|test-integration")
|
||||
raise SystemExit("Usage: python scripts/server.py migrate|bootstrap|serve|audit-cleanup|build|test|test-integration")
|
||||
env = dict(os.environ)
|
||||
config = ROOT / ".env.local"
|
||||
if config.exists():
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
const AuditRetention = 90 * 24 * time.Hour
|
||||
|
||||
type LoginLog struct {
|
||||
ID uint64 `json:"id" gorm:"primaryKey"`
|
||||
Username string `json:"username"`
|
||||
Result string `json:"result"`
|
||||
Reason string `json:"reason"`
|
||||
IP string `json:"ip" gorm:"column:ip"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
|
||||
func (LoginLog) TableName() string { return "lexgo_login_logs" }
|
||||
|
||||
type OperationLog struct {
|
||||
ID uint64 `json:"id" gorm:"primaryKey"`
|
||||
ActorID int `json:"actorId"`
|
||||
ActorUsername string `json:"actorUsername"`
|
||||
TargetID int `json:"targetId"`
|
||||
TargetUsername string `json:"targetUsername"`
|
||||
Action string `json:"action"`
|
||||
Result string `json:"result"`
|
||||
Reason string `json:"reason"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
|
||||
func (OperationLog) TableName() string { return "lexgo_operation_logs" }
|
||||
func safeAuditUsername(value string) string {
|
||||
value = strings.ToLower(strings.TrimSpace(value))
|
||||
if !usernamePattern.MatchString(value) {
|
||||
return ""
|
||||
}
|
||||
return value
|
||||
}
|
||||
func auditReason(err error) string {
|
||||
if err == nil {
|
||||
return "ok"
|
||||
}
|
||||
var e *apiError
|
||||
if errors.As(err, &e) {
|
||||
switch e.status {
|
||||
case 400:
|
||||
return "invalid_input"
|
||||
case 401:
|
||||
return "invalid_credentials"
|
||||
case 403:
|
||||
return "forbidden"
|
||||
case 404:
|
||||
return "not_found"
|
||||
case 409:
|
||||
return "conflict"
|
||||
case 429:
|
||||
return "rate_limited"
|
||||
}
|
||||
}
|
||||
return "internal_error"
|
||||
}
|
||||
func loginAudit(c *gin.Context, username string, at time.Time, err error) LoginLog {
|
||||
ip := net.ParseIP(c.ClientIP())
|
||||
value := ""
|
||||
if ip != nil {
|
||||
value = ip.String()
|
||||
}
|
||||
result := "success"
|
||||
if err != nil {
|
||||
result = "failure"
|
||||
}
|
||||
return LoginLog{Username: safeAuditUsername(username), Result: result, Reason: auditReason(err), IP: value, CreatedAt: at.UTC().Truncate(time.Millisecond)}
|
||||
}
|
||||
|
||||
// A bounded independent context records a failure after the business transaction
|
||||
// rolls back, including when the caller disconnects. Only allowlisted fields exist.
|
||||
func saveFailedAudit(db *gorm.DB, value any) error {
|
||||
if db == nil {
|
||||
return nil
|
||||
} // Router(nil) is used only by input-shape unit tests.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
return db.WithContext(ctx).Create(value).Error
|
||||
}
|
||||
|
||||
func saveOperationAudits(db *gorm.DB, c *gin.Context, entry *OperationLog) error {
|
||||
actions := []string{entry.Action}
|
||||
if v, ok := c.Get("operationActions"); ok {
|
||||
actions = v.([]string)
|
||||
}
|
||||
for _, action := range actions {
|
||||
row := *entry
|
||||
row.ID = 0
|
||||
row.Action = action
|
||||
if err := db.Create(&row).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func saveOperationFailure(db *gorm.DB, c *gin.Context, entry *OperationLog) error {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
return db.WithContext(ctx).Transaction(func(tx *gorm.DB) error { return saveOperationAudits(tx, c, entry) })
|
||||
}
|
||||
|
||||
type auditFilter struct {
|
||||
Page, Limit int
|
||||
Username, Result, Action string
|
||||
From, To time.Time
|
||||
}
|
||||
|
||||
func parseAuditFilter(q url.Values, operation bool, now time.Time) (auditFilter, error) {
|
||||
f := auditFilter{Page: 1, Limit: 20, From: now.UTC().Add(-AuditRetention), To: now.UTC()}
|
||||
bad := failure(400, "日志筛选参数无效")
|
||||
for k, v := range q {
|
||||
if len(v) != 1 {
|
||||
return f, bad
|
||||
}
|
||||
switch k {
|
||||
case "page", "limit", "username", "result", "from", "to":
|
||||
case "action":
|
||||
if !operation {
|
||||
return f, bad
|
||||
}
|
||||
default:
|
||||
return f, bad
|
||||
}
|
||||
}
|
||||
for key, dest := range map[string]*int{"page": &f.Page, "limit": &f.Limit} {
|
||||
if v := q.Get(key); v != "" {
|
||||
n, e := strconv.Atoi(v)
|
||||
if e != nil || n < 1 {
|
||||
return f, bad
|
||||
}
|
||||
*dest = n
|
||||
}
|
||||
}
|
||||
if f.Limit > 100 || f.Page > 1000000 {
|
||||
return f, bad
|
||||
}
|
||||
if v := q.Get("username"); v != "" {
|
||||
f.Username = safeAuditUsername(v)
|
||||
if f.Username == "" {
|
||||
return f, bad
|
||||
}
|
||||
}
|
||||
f.Result = q.Get("result")
|
||||
if f.Result != "" && f.Result != "success" && f.Result != "failure" {
|
||||
return f, bad
|
||||
}
|
||||
f.Action = q.Get("action")
|
||||
if f.Action != "" {
|
||||
switch f.Action {
|
||||
case "create", "enable", "disable", "reset_password", "update":
|
||||
default:
|
||||
return f, bad
|
||||
}
|
||||
}
|
||||
for key, dest := range map[string]*time.Time{"from": &f.From, "to": &f.To} {
|
||||
if v := q.Get(key); v != "" {
|
||||
t, e := time.Parse(time.RFC3339, v)
|
||||
if e != nil {
|
||||
return f, bad
|
||||
}
|
||||
*dest = t.UTC()
|
||||
}
|
||||
}
|
||||
if f.To.Before(f.From) {
|
||||
return f, bad
|
||||
}
|
||||
if cutoff := now.UTC().Add(-AuditRetention); f.From.Before(cutoff) {
|
||||
f.From = cutoff
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
func queryAudit(db *gorm.DB, q url.Values, operation bool, now time.Time) (any, error) {
|
||||
f, err := parseAuditFilter(q, operation, now)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
table := "lexgo_login_logs"
|
||||
if operation {
|
||||
table = "lexgo_operation_logs"
|
||||
}
|
||||
query := db.Table(table).Where("created_at >= ? AND created_at <= ?", f.From, f.To)
|
||||
if f.Username != "" {
|
||||
if operation {
|
||||
query = query.Where("actor_username = ? OR target_username = ?", f.Username, f.Username)
|
||||
} else {
|
||||
query = query.Where("username = ?", f.Username)
|
||||
}
|
||||
}
|
||||
if f.Result != "" {
|
||||
query = query.Where("result = ?", f.Result)
|
||||
}
|
||||
if f.Action != "" {
|
||||
query = query.Where("action = ?", f.Action)
|
||||
}
|
||||
var total int64
|
||||
if err = query.Count(&total).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
query = query.Order("created_at DESC, id DESC").Offset((f.Page - 1) * f.Limit).Limit(f.Limit)
|
||||
result := gin.H{"total": total, "page": f.Page, "limit": f.Limit}
|
||||
if operation {
|
||||
items := []OperationLog{}
|
||||
err = query.Find(&items).Error
|
||||
result["items"] = items
|
||||
} else {
|
||||
items := []LoginLog{}
|
||||
err = query.Find(&items).Error
|
||||
result["items"] = items
|
||||
}
|
||||
return result, err
|
||||
}
|
||||
|
||||
// CleanupAudit removes only expired audit rows in bounded batches. No user,
|
||||
// session, or learning table is touched. Cutoff is fixed for the entire run.
|
||||
func CleanupAudit(ctx context.Context, db *gorm.DB, now time.Time) error {
|
||||
for _, table := range []string{"lexgo_login_logs", "lexgo_operation_logs"} {
|
||||
for {
|
||||
result := db.WithContext(ctx).Exec("DELETE FROM "+table+" WHERE created_at < ? ORDER BY created_at LIMIT 1000", now.UTC().Add(-AuditRetention))
|
||||
if result.Error != nil {
|
||||
return result.Error
|
||||
}
|
||||
if result.RowsAffected < 1000 {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,213 @@
|
||||
package lexgo
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
admin "go-admin/app/admin/models"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestAuditRetentionAndV1Migration(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
u := admin.SysUser{Username: randomName("retention"), Password: fixturePassword, RoleId: 1, Status: "2"}
|
||||
if err := db.Create(&u).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
previousToken := loginToken(t, Router(db, time.Now), u.Username, fixturePassword)
|
||||
// Reconstruct v1 in this disposable schema to exercise the incremental path.
|
||||
for _, sql := range []string{"DROP TABLE lexgo_login_logs", "DROP TABLE lexgo_operation_logs", "UPDATE lexgo_schema SET version=1 WHERE id=1"} {
|
||||
if err := db.Exec(sql).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := CheckSchema(db); err == nil {
|
||||
t.Fatal("v1 must require explicit migration")
|
||||
}
|
||||
for i := 0; i < 2; i++ {
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
var count int64
|
||||
db.Table("sys_user").Where("user_id=?", u.UserId).Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatal("migration changed account")
|
||||
}
|
||||
if code, _ := callAPI(t, Router(db, time.Now), "GET", "/api/v1/space", previousToken, nil); code != 200 {
|
||||
t.Fatal("migration changed existing session or learning space")
|
||||
}
|
||||
now := time.Now().UTC().Truncate(time.Millisecond)
|
||||
cutoff := now.Add(-AuditRetention)
|
||||
for _, at := range []time.Time{cutoff.Add(-time.Millisecond), cutoff, now} {
|
||||
if err := db.Create(&LoginLog{Username: u.Username, Result: "success", Reason: "ok", CreatedAt: at}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Create(&OperationLog{ActorID: u.UserId, ActorUsername: u.Username, Action: "create", Result: "success", Reason: "ok", CreatedAt: at}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
r := Router(db, func() time.Time { return now })
|
||||
token := loginToken(t, r, u.Username, fixturePassword)
|
||||
code, data := callAPI(t, r, "GET", "/api/v1/operation-logs?limit=1", token, nil)
|
||||
var page struct {
|
||||
Total int64
|
||||
Items []OperationLog
|
||||
}
|
||||
json.Unmarshal(data, &page)
|
||||
if code != 200 || page.Total != 2 || len(page.Items) != 1 || !page.Items[0].CreatedAt.Equal(now) {
|
||||
t.Fatal("retention/page/order incorrect")
|
||||
}
|
||||
if err := CleanupAudit(context.Background(), db, now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, table := range []string{"lexgo_login_logs", "lexgo_operation_logs"} {
|
||||
db.Table(table).Where("created_at < ?", cutoff).Count(&count)
|
||||
if count != 0 {
|
||||
t.Fatal("expired rows remain")
|
||||
}
|
||||
}
|
||||
db.Table("lexgo_operation_logs").Count(&count)
|
||||
if count != 2 {
|
||||
t.Fatal("cleanup deleted boundary/current rows")
|
||||
}
|
||||
db.Table("sys_user").Count(&count)
|
||||
if count != 1 {
|
||||
t.Fatal("cleanup touched users")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAuditFailureRollsBackAccountAndSession(t *testing.T) {
|
||||
db := emptyMigrationDB(t)
|
||||
if err := Migrate(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
u := admin.SysUser{Username: randomName("atomic"), Password: fixturePassword, RoleId: 1, Status: "2"}
|
||||
if err := db.Create(&u).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := Router(db, time.Now)
|
||||
token := loginToken(t, r, u.Username, fixturePassword)
|
||||
if err := db.Exec("DROP TABLE lexgo_operation_logs").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
name := randomName("rollback")
|
||||
code, _ := callAPI(t, r, "POST", "/api/v1/accounts", token, map[string]string{"username": name, "password": fixturePassword})
|
||||
if code != 500 {
|
||||
t.Fatal("missing audit store must fail operation")
|
||||
}
|
||||
var count int64
|
||||
db.Table("sys_user").Where("username=?", name).Count(&count)
|
||||
if count != 0 {
|
||||
t.Fatal("account committed without audit")
|
||||
}
|
||||
if err := db.Exec("DROP TABLE lexgo_login_logs").Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var before int64
|
||||
db.Table("lexgo_sessions").Count(&before)
|
||||
code, _ = callAPI(t, r, "POST", "/api/v1/login", "", map[string]string{"username": u.Username, "password": fixturePassword})
|
||||
db.Table("lexgo_sessions").Count(&count)
|
||||
if code != 500 || count != before {
|
||||
t.Fatal("session committed without login audit")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAuditQueriesAndPrivacy(t *testing.T) {
|
||||
db := testDB(t)
|
||||
u := admin.SysUser{Username: randomName("audit"), Password: fixturePassword, RoleId: 1, Status: "2"}
|
||||
if err := db.Create(&u).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
r := Router(db, func() time.Time { return now })
|
||||
token := loginToken(t, r, u.Username, fixturePassword)
|
||||
code, data := callAPI(t, r, "GET", "/api/v1/login-logs?username="+u.Username, token, nil)
|
||||
if code != 200 {
|
||||
t.Fatalf("login logs status %d, want 200", code)
|
||||
}
|
||||
var page struct {
|
||||
Items []struct{ Username, Result, Reason string }
|
||||
Total int64
|
||||
}
|
||||
if err := json.Unmarshal(data, &page); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if page.Total != 1 || len(page.Items) != 1 || page.Items[0].Result != "success" {
|
||||
t.Fatal("successful login missing")
|
||||
}
|
||||
callAPI(t, r, "POST", "/api/v1/login", "", map[string]string{"username": u.Username, "password": "fictional-wrong-secret"})
|
||||
name := randomName("auditlearner")
|
||||
code, _ = callAPI(t, r, "POST", "/api/v1/accounts", token, map[string]string{"username": name, "password": fixturePassword})
|
||||
if code != 201 {
|
||||
t.Fatal("create failed")
|
||||
}
|
||||
callAPI(t, r, "POST", "/api/v1/accounts", token, map[string]string{"username": name, "password": fixturePassword})
|
||||
code, data = callAPI(t, r, "GET", "/api/v1/operation-logs?username="+u.Username, token, nil)
|
||||
if code != 200 {
|
||||
t.Fatal("operation query failed")
|
||||
}
|
||||
if err := json.Unmarshal(data, &page); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if page.Total != 2 {
|
||||
t.Fatalf("expected success and failure operation logs, got %d", page.Total)
|
||||
}
|
||||
for _, secret := range []string{fixturePassword, "fictional-wrong-secret", token} {
|
||||
if strings.Contains(string(data), secret) {
|
||||
t.Fatal("secret leaked")
|
||||
}
|
||||
}
|
||||
learnerToken := loginToken(t, r, name, fixturePassword)
|
||||
for _, path := range []string{"/api/v1/login-logs", "/api/v1/operation-logs"} {
|
||||
if code, _ = callAPI(t, r, "GET", path, "", nil); code != 401 {
|
||||
t.Fatal("anonymous allowed")
|
||||
}
|
||||
if code, _ = callAPI(t, r, "GET", path, learnerToken, nil); code != 403 {
|
||||
t.Fatal("learner allowed")
|
||||
}
|
||||
for _, q := range []string{"?limit=101", "?page=0", "?from=invalid", "?result=invalid", "?unexpected=x"} {
|
||||
if code, _ = callAPI(t, r, "GET", path+q, token, nil); code != 400 {
|
||||
t.Fatalf("invalid query accepted %s %d", q, code)
|
||||
}
|
||||
}
|
||||
}
|
||||
code, data = callAPI(t, r, "GET", "/api/v1/login-logs?username="+u.Username+"&result=failure", token, nil)
|
||||
json.Unmarshal(data, &page)
|
||||
if code != 200 || page.Total != 1 || page.Items[0].Reason != "invalid_credentials" {
|
||||
t.Fatal("failure filter incorrect")
|
||||
}
|
||||
var target admin.SysUser
|
||||
db.Where("username=?", name).First(&target)
|
||||
path := "/api/v1/accounts/" + strconv.Itoa(target.UserId)
|
||||
for _, input := range []any{map[string]any{"disabled": true}, map[string]any{"disabled": false}, map[string]any{"password": strings.Repeat("q", 6)}} {
|
||||
if code, _ = callAPI(t, r, "PATCH", path, token, input); code != 200 {
|
||||
t.Fatal("audited update failed")
|
||||
}
|
||||
}
|
||||
callAPI(t, r, "PATCH", path, token, map[string]any{"password": "tiny"})
|
||||
for _, action := range []string{"disable", "enable", "reset_password"} {
|
||||
code, data = callAPI(t, r, "GET", "/api/v1/operation-logs?username="+name+"&action="+action+"&result=success", token, nil)
|
||||
json.Unmarshal(data, &page)
|
||||
if code != 200 || page.Total != 1 {
|
||||
t.Fatalf("missing %s audit", action)
|
||||
}
|
||||
}
|
||||
if code, _ = callAPI(t, r, "GET", "/api/v1/me", learnerToken, nil); code != 401 {
|
||||
t.Fatal("old learner session survived")
|
||||
}
|
||||
callAPI(t, r, "POST", "/api/v1/login", "", map[string]string{"username": "<raw-private-input>", "password": "raw-password-marker"})
|
||||
var logs []LoginLog
|
||||
db.Where("created_at >= ?", now.Truncate(time.Millisecond)).Find(&logs)
|
||||
encoded, _ := json.Marshal(logs)
|
||||
for _, secret := range []string{"<raw-private-input>", "raw-password-marker", fixturePassword, token} {
|
||||
if strings.Contains(string(encoded), secret) {
|
||||
t.Fatal("private input in persisted login records")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -59,10 +59,10 @@ func Migrate(db *gorm.DB) error {
|
||||
if err = conn.QueryRowContext(ctx, "SELECT version,product FROM lexgo_schema WHERE id=1").Scan(¤t, &product); err != nil {
|
||||
return err
|
||||
}
|
||||
if product != "lexgo" || current < 0 || current > 1 {
|
||||
if product != "lexgo" || current < 0 || current > 2 {
|
||||
return errors.New("unknown schema version")
|
||||
}
|
||||
if current == 1 {
|
||||
if current == 2 {
|
||||
return nil
|
||||
}
|
||||
statements := []string{
|
||||
@@ -86,6 +86,17 @@ func Migrate(db *gorm.DB) error {
|
||||
INDEX (owner_id), INDEX (expires_at),
|
||||
FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_login_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL, ip VARCHAR(45) NOT NULL DEFAULT '',
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
`CREATE TABLE IF NOT EXISTS lexgo_operation_logs (
|
||||
id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, actor_id BIGINT NOT NULL, actor_username VARCHAR(32) NOT NULL,
|
||||
target_id BIGINT NOT NULL DEFAULT 0, target_username VARCHAR(32) NOT NULL DEFAULT '',
|
||||
action VARCHAR(32) NOT NULL, result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL,
|
||||
created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(actor_username,created_at), INDEX(target_username,created_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`,
|
||||
}
|
||||
for i, s := range statements {
|
||||
if _, err = conn.ExecContext(ctx, s); err != nil {
|
||||
@@ -96,7 +107,7 @@ func Migrate(db *gorm.DB) error {
|
||||
return fmt.Errorf("migration statement %d failed", i+1)
|
||||
}
|
||||
}
|
||||
_, err = conn.ExecContext(ctx, "UPDATE lexgo_schema SET version=1 WHERE id=1")
|
||||
_, err = conn.ExecContext(ctx, "UPDATE lexgo_schema SET version=2 WHERE id=1")
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -105,7 +116,7 @@ func CheckSchema(db *gorm.DB) error {
|
||||
Version int
|
||||
Product string
|
||||
}
|
||||
if err := db.Table("lexgo_schema").Where("id=1").First(&r).Error; err != nil || r.Version != 1 || r.Product != "lexgo" {
|
||||
if err := db.Table("lexgo_schema").Where("id=1").First(&r).Error; err != nil || r.Version != 2 || r.Product != "lexgo" {
|
||||
return errors.New("run the explicit migration before starting")
|
||||
}
|
||||
return nil
|
||||
|
||||
@@ -92,7 +92,7 @@ func TestMigrationRefusesUnownedOrUnsupportedSchema(t *testing.T) {
|
||||
for _, tc := range []struct{ name, marker string }{
|
||||
{"empty_marker", ""},
|
||||
{"negative_version", "INSERT INTO lexgo_schema VALUES (1,-1,'lexgo')"},
|
||||
{"future_version", "INSERT INTO lexgo_schema VALUES (1,2,'lexgo')"},
|
||||
{"future_version", "INSERT INTO lexgo_schema VALUES (1,3,'lexgo')"},
|
||||
{"wrong_product", "INSERT INTO lexgo_schema VALUES (1,0,'another-app')"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
|
||||
@@ -38,21 +38,41 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
Username string `json:"username"`
|
||||
Password string `json:"password"`
|
||||
}
|
||||
if err := decode(c, &input); err != nil {
|
||||
failed := func(err error) {
|
||||
entry := loginAudit(c, input.Username, now(), err)
|
||||
if logErr := saveFailedAudit(db, &entry); logErr != nil {
|
||||
err = logErr
|
||||
}
|
||||
respond(c, 400, nil, err)
|
||||
}
|
||||
if err := decode(c, &input); err != nil {
|
||||
failed(err)
|
||||
return
|
||||
}
|
||||
name, err := loginCredentials(input.Username, input.Password)
|
||||
if err != nil {
|
||||
respond(c, 400, nil, err)
|
||||
failed(err)
|
||||
return
|
||||
}
|
||||
if !limits.allow(c.ClientIP(), now()) {
|
||||
respond(c, 429, nil, failure(429, "登录尝试过多,请稍后再试"))
|
||||
failed(failure(429, "登录尝试过多,请稍后再试"))
|
||||
return
|
||||
}
|
||||
result, err := login(db.WithContext(c.Request.Context()), now(), name, input.Password)
|
||||
respond(c, 200, result, err)
|
||||
var result LoginResult
|
||||
err = db.WithContext(c.Request.Context()).Transaction(func(tx *gorm.DB) error {
|
||||
var loginErr error
|
||||
result, loginErr = login(tx, now(), name, input.Password)
|
||||
if loginErr != nil {
|
||||
return loginErr
|
||||
}
|
||||
entry := loginAudit(c, name, now(), nil)
|
||||
return tx.Create(&entry).Error
|
||||
})
|
||||
if err != nil {
|
||||
failed(err)
|
||||
return
|
||||
}
|
||||
respond(c, 200, result, nil)
|
||||
})
|
||||
protect := func(adminOnly bool, action func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
@@ -62,6 +82,7 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
return
|
||||
}
|
||||
var data any
|
||||
var audit *OperationLog
|
||||
err := db.WithContext(c.Request.Context()).Transaction(func(tx *gorm.DB) error {
|
||||
u, err := authenticate(tx, token, now())
|
||||
if err != nil {
|
||||
@@ -70,9 +91,40 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
if adminOnly && u.RoleId != 1 {
|
||||
return failure(403, "需要管理员权限")
|
||||
}
|
||||
if c.FullPath() == "/api/v1/accounts" && c.Request.Method == "POST" || c.FullPath() == "/api/v1/accounts/:id" && c.Request.Method == "PATCH" {
|
||||
audit = &OperationLog{ActorID: u.UserId, ActorUsername: u.Username, Action: "create", CreatedAt: now().UTC().Truncate(time.Millisecond)}
|
||||
if c.Request.Method == "PATCH" {
|
||||
audit.Action = "update"
|
||||
if id, e := strconv.Atoi(c.Param("id")); e == nil && id > 0 {
|
||||
audit.TargetID = id
|
||||
var target admin.SysUser
|
||||
if tx.Where("user_id = ?", id).First(&target).Error == nil {
|
||||
audit.TargetUsername = target.Username
|
||||
}
|
||||
}
|
||||
}
|
||||
c.Set("operationAudit", audit)
|
||||
}
|
||||
data, err = action(c, tx, u)
|
||||
if err == nil && audit != nil {
|
||||
if a, ok := data.(Account); ok {
|
||||
audit.TargetID = a.ID
|
||||
audit.TargetUsername = a.Username
|
||||
}
|
||||
audit.Result = "success"
|
||||
audit.Reason = "ok"
|
||||
err = saveOperationAudits(tx, c, audit)
|
||||
}
|
||||
return err
|
||||
})
|
||||
if err != nil && audit != nil {
|
||||
audit.ID = 0
|
||||
audit.Result = "failure"
|
||||
audit.Reason = auditReason(err)
|
||||
if logErr := saveOperationFailure(db, c, audit); logErr != nil {
|
||||
err = logErr
|
||||
}
|
||||
}
|
||||
status := 200
|
||||
if c.Request.Method == "POST" && c.FullPath() == "/api/v1/accounts" {
|
||||
status = 201
|
||||
@@ -81,6 +133,12 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
}
|
||||
}
|
||||
v.GET("/me", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) { return account(u), nil }))
|
||||
v.GET("/login-logs", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
return queryAudit(tx, c.Request.URL.Query(), false, now())
|
||||
}))
|
||||
v.GET("/operation-logs", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
return queryAudit(tx, c.Request.URL.Query(), true, now())
|
||||
}))
|
||||
v.POST("/logout", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
token := strings.TrimPrefix(c.GetHeader("Authorization"), "Bearer ")
|
||||
return nil, tx.Where("token_hash = ? AND owner_id = ?", digest(token), u.UserId).Delete(&Session{}).Error
|
||||
@@ -125,6 +183,9 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
if err := decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if value, ok := c.Get("operationAudit"); ok {
|
||||
value.(*OperationLog).TargetUsername = safeAuditUsername(input.Username)
|
||||
}
|
||||
return createAccount(tx, input.Username, input.Password, 2, u.UserId)
|
||||
}))
|
||||
v.PATCH("/accounts/:id", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) {
|
||||
@@ -136,6 +197,20 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine {
|
||||
if err = decode(c, &input); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
actions := []string{}
|
||||
if input.Disabled != nil {
|
||||
a := "enable"
|
||||
if *input.Disabled {
|
||||
a = "disable"
|
||||
}
|
||||
actions = append(actions, a)
|
||||
}
|
||||
if input.Password != nil {
|
||||
actions = append(actions, "reset_password")
|
||||
}
|
||||
if len(actions) > 0 {
|
||||
c.Set("operationActions", actions)
|
||||
}
|
||||
return updateAccount(tx, id, u.UserId, input)
|
||||
}))
|
||||
r.NoRoute(func(c *gin.Context) { respond(c, 404, nil, failure(404, "页面或接口不存在")) })
|
||||
|
||||
@@ -66,10 +66,10 @@ func openDB() (*gorm.DB, error) {
|
||||
|
||||
func run() error {
|
||||
if len(os.Args) != 2 {
|
||||
return errors.New("usage: lexgo migrate|bootstrap|serve")
|
||||
return errors.New("usage: lexgo migrate|bootstrap|serve|audit-cleanup")
|
||||
}
|
||||
command := os.Args[1]
|
||||
if command != "migrate" && command != "bootstrap" && command != "serve" {
|
||||
if command != "migrate" && command != "bootstrap" && command != "serve" && command != "audit-cleanup" {
|
||||
return errors.New("unknown command")
|
||||
}
|
||||
db, err := openDB()
|
||||
@@ -82,12 +82,21 @@ func run() error {
|
||||
if err = lexgo.Migrate(db); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println("LexGo schema version 1 ready")
|
||||
fmt.Println("LexGo schema version 2 ready")
|
||||
return nil
|
||||
}
|
||||
if err = lexgo.CheckSchema(db); err != nil {
|
||||
return err
|
||||
}
|
||||
if command == "audit-cleanup" {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
|
||||
defer cancel()
|
||||
if err = lexgo.CleanupAudit(ctx, db, time.Now()); err != nil {
|
||||
return errors.New("audit cleanup failed")
|
||||
}
|
||||
fmt.Println("Expired audit records cleaned")
|
||||
return nil
|
||||
}
|
||||
if command == "bootstrap" {
|
||||
if err = lexgo.BootstrapAdmin(db, os.Getenv("LEXGO_BOOTSTRAP_USERNAME"), os.Getenv("LEXGO_BOOTSTRAP_PASSWORD")); err != nil {
|
||||
return err
|
||||
@@ -103,6 +112,22 @@ func run() error {
|
||||
srv := &http.Server{Addr: addr, Handler: lexgo.Router(db, time.Now), ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 15 * time.Second, WriteTimeout: 15 * time.Second, IdleTimeout: 60 * time.Second, MaxHeaderBytes: 1 << 20}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
go func() {
|
||||
ticker := time.NewTicker(time.Hour)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
cleanupCtx, cancel := context.WithTimeout(ctx, time.Minute)
|
||||
if err := lexgo.CleanupAudit(cleanupCtx, db, time.Now()); err != nil && ctx.Err() == nil {
|
||||
log.Print("audit cleanup failed; retrying next hour")
|
||||
}
|
||||
cancel()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
}
|
||||
}()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- srv.ListenAndServe() }()
|
||||
fmt.Println("LexGo API listening on", addr)
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
# #3 英语技术验证
|
||||
|
||||
独立小样,不是学习端正式功能;不连接 MySQL、不读取账号、不保存输入。仅本机、单进程串行运行,默认英语。长期契约与实测结论见 [开发验证 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Local-Development-and-Verification)。
|
||||
|
||||
从仓库根目录运行(Windows PowerShell,需要 uv、Node):
|
||||
|
||||
```powershell
|
||||
uv venv --python 3.12.12 .local/nlp-venv
|
||||
uv pip install --python .local/nlp-venv/Scripts/python.exe -r spikes/english/requirements.lock
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/setup_resources.py
|
||||
uv pip install --python .local/nlp-venv/Scripts/python.exe --no-deps .local/nlp-resources/en_core_web_sm-3.8.0-py3-none-any.whl
|
||||
.local/nlp-venv/Scripts/python.exe -m unittest discover -s spikes/english -v
|
||||
node --test spikes/english/view.test.mjs
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/benchmark.py
|
||||
.local/nlp-venv/Scripts/python.exe spikes/english/app.py
|
||||
```
|
||||
|
||||
打开 <http://127.0.0.1:5183/>。点击“分析文本”后点单词;`went` 应以 `go` 查询。手动查询仅查输入形式,不猜测词性:`dog` 直接命中,`went` 无结果。关闭进程即停止小样;端口占用时使用 `--port 5185`。用 `--resources .local/absent-resources` 启动可验证词典缺失,模型与词典独立加载。
|
||||
|
||||
准备依赖和资源时需要联网;安装完成后运行不依赖在线翻译或下载服务。`test_engine.py` 与 `benchmark.py` 禁止 socket connect,用真实模型与词典验证离线运行。网络下载失败可重新执行准备命令,已有资源先校验再复用。默认系统 Python 3.8 不适用,命令必须使用上述独立环境。
|
||||
|
||||
资源版本、固定下载 URL 和 SHA256 见 `resources.json`,Python 依赖固定于 `requirements.lock`。大文件只保存在忽略的 `.local/nlp-resources`。
|
||||
|
||||
许可与来源:
|
||||
|
||||
- [spaCy 3.8.7](https://pypi.org/pypi/spacy/3.8.7/json) 与 [en_core_web_sm 3.8.0](https://github.com/explosion/spacy-models/releases/tag/en_core_web_sm-3.8.0):MIT;安装包保留其许可证。模型的 POS/lemma 组件保留,parser/NER 在本小样中停用;不输出句界。
|
||||
- [NLTK](https://github.com/nltk/nltk/blob/3.9.2/LICENSE.txt):Apache-2.0;只读取本地词典文件,无隐式 downloader。
|
||||
- [Princeton WordNet 3.0](https://wordnet.princeton.edu/license-and-commercial-use):WordNet 3.0 许可证;下载 ZIP 完整保留 `wordnet/LICENSE`、版权及免责声明。英英释义,不提供中文翻译。再分发必须保留许可声明。
|
||||
- [WordNet 原生格式](https://wordnet.princeton.edu/documentation/wndb5wn) 是 index/data/exception 文件,本小样直接读取 ZIP 中原始文件,不使用 go-admin 的系统枚举字典。
|
||||
|
||||
`benchmark-result.json` 是本机虚构语料的测量样本,不代表一般准确率或生产性能承诺。
|
||||
@@ -0,0 +1,69 @@
|
||||
import {validateTokens,latestOnly} from './view.mjs'
|
||||
const source=document.querySelector('#source'), reading=document.querySelector('#reading'), definition=document.querySelector('#definition'), notice=document.querySelector('#notice'), position=document.querySelector('#position'), query=document.querySelector('#query'), analyzeButton=document.querySelector('#analyze')
|
||||
const analysis=latestOnly(), lookup=latestOnly()
|
||||
const labels={exact:'直接命中',lemma:'按原形查询',not_found:'未找到释义',resource_missing:'本地资源缺失',invalid_input:'输入格式无效或文本超过限制',internal_error:'暂时无法处理,请重试',too_large:'文本超过限制'}
|
||||
async function post(path,payload) {
|
||||
const response=await fetch(path,{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(payload)})
|
||||
const data=await response.json()
|
||||
if(!response.ok) throw new Error(labels[data.status] || '请求失败,请重试')
|
||||
return data
|
||||
}
|
||||
function element(tag,text) {const node=document.createElement(tag);node.textContent=text;return node}
|
||||
async function showWord(surface,lemma='',token=null) {
|
||||
const version=lookup.next()
|
||||
query.value=surface
|
||||
definition.replaceChildren(element('p','查询中…'))
|
||||
position.textContent=token ? JSON.stringify({text:token.text,lemma:token.lemma,code_point:[token.start_cp,token.end_cp],utf8:[token.start_utf8,token.end_utf8],utf16:[token.start_utf16,token.end_utf16]},null,2) : '手动查询,无原文位置'
|
||||
try {
|
||||
const data=await post('/lookup',{surface,lemma})
|
||||
if(!lookup.current(version)) return
|
||||
definition.replaceChildren(element('h3',data.matched_form || surface),element('small',labels[data.status] || data.status))
|
||||
if(data.entries.length) {
|
||||
const list=document.createElement('ol')
|
||||
for(const entry of data.entries) {
|
||||
const item=element('li',entry.definition)
|
||||
item.prepend(element('small',entry.pos+' · '))
|
||||
for(const example of entry.examples) item.append(element('p',example))
|
||||
list.append(item)
|
||||
}
|
||||
definition.append(list)
|
||||
}
|
||||
} catch(error) {if(lookup.current(version)) definition.textContent=error.message}
|
||||
}
|
||||
function resetReading() {
|
||||
lookup.next()
|
||||
reading.replaceChildren()
|
||||
definition.textContent='点击文中的单词。'
|
||||
position.textContent='尚未选择单词'
|
||||
}
|
||||
async function analyze() {
|
||||
const version=analysis.next(), text=source.value
|
||||
resetReading()
|
||||
analyzeButton.disabled=true
|
||||
notice.textContent='正在分析…'
|
||||
try {
|
||||
const result=await post('/analyze',{text})
|
||||
if(!analysis.current(version)) return
|
||||
validateTokens(text,result.tokens)
|
||||
const fragment=document.createDocumentFragment()
|
||||
for(const token of result.tokens) {
|
||||
if(token.kind==='word') {
|
||||
const button=element('button',token.text)
|
||||
button.type='button'
|
||||
button.addEventListener('click',()=>{
|
||||
reading.querySelector('.selected')?.classList.remove('selected')
|
||||
button.classList.add('selected')
|
||||
showWord(token.text,token.lemma,token)
|
||||
})
|
||||
fragment.append(button)
|
||||
} else fragment.append(document.createTextNode(token.text))
|
||||
}
|
||||
reading.replaceChildren(fragment)
|
||||
notice.textContent=text ? '已分析 · 点击单词查词' : '请输入阅读文本'
|
||||
} catch(error) {if(analysis.current(version)) notice.textContent=error.message}
|
||||
finally {if(analysis.current(version)) analyzeButton.disabled=false}
|
||||
}
|
||||
source.addEventListener('input',()=>{analysis.next();resetReading();analyzeButton.disabled=false;notice.textContent='文本已修改,请重新分析'})
|
||||
analyzeButton.addEventListener('click',analyze)
|
||||
document.querySelector('#lookup-form').addEventListener('submit',event=>{event.preventDefault();showWord(query.value)})
|
||||
analyze()
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Loopback-only, ephemeral English experiment. Not a production API."""
|
||||
import argparse
|
||||
import json
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
STATIC = Path(__file__).parent
|
||||
ROOT = STATIC.parent.parent
|
||||
|
||||
|
||||
def create_server(engine, port=5183):
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def setup(self):
|
||||
super().setup()
|
||||
self.connection.settimeout(10)
|
||||
|
||||
def log_message(self, *_args):
|
||||
pass # Do not retain input or query text in access logs.
|
||||
|
||||
def reply(self, code, body, mime='application/json; charset=utf-8'):
|
||||
if not isinstance(body, bytes):
|
||||
body = json.dumps(body, ensure_ascii=True).encode('utf-8')
|
||||
self.send_response(code)
|
||||
self.send_header('Content-Type', mime)
|
||||
self.send_header('Content-Length', str(len(body)))
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.send_header('X-Content-Type-Options', 'nosniff')
|
||||
self.send_header('Content-Security-Policy', "default-src 'self'; style-src 'self'; script-src 'self'; connect-src 'self'; frame-ancestors 'none'; base-uri 'none'")
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def local_request(self):
|
||||
host = self.headers.get('Host')
|
||||
allowed = {f'127.0.0.1:{self.server.server_port}', f'localhost:{self.server.server_port}'}
|
||||
origin = self.headers.get('Origin')
|
||||
if host not in allowed or (origin is not None and origin not in {'http://' + h for h in allowed}):
|
||||
self.reply(403, {'status': 'forbidden'})
|
||||
return False
|
||||
return True
|
||||
|
||||
def do_GET(self):
|
||||
if not self.local_request():
|
||||
return
|
||||
files = {'/': ('index.html', 'text/html'), '/app.mjs': ('app.mjs', 'text/javascript'), '/view.mjs': ('view.mjs', 'text/javascript'), '/style.css': ('style.css', 'text/css')}
|
||||
if self.path not in files:
|
||||
self.reply(404, {'status': 'not_found'})
|
||||
return
|
||||
name, mime = files[self.path]
|
||||
self.reply(200, (STATIC / name).read_bytes(), mime + '; charset=utf-8')
|
||||
|
||||
def do_POST(self):
|
||||
if not self.local_request():
|
||||
return
|
||||
if self.path not in {'/analyze', '/lookup'}:
|
||||
self.reply(404, {'status': 'not_found'})
|
||||
return
|
||||
try:
|
||||
length = int(self.headers.get('Content-Length', '0'))
|
||||
except ValueError:
|
||||
self.reply(400, {'status': 'invalid_input'})
|
||||
return
|
||||
if length > 1_000_000:
|
||||
self.reply(413, {'status': 'too_large'})
|
||||
return
|
||||
if length <= 0:
|
||||
self.reply(400, {'status': 'invalid_input'})
|
||||
return
|
||||
if self.headers.get_content_type() != 'application/json':
|
||||
self.reply(415, {'status': 'invalid_content_type'})
|
||||
return
|
||||
try:
|
||||
payload = json.loads(self.rfile.read(length))
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError('object required')
|
||||
key = 'text' if self.path == '/analyze' else 'surface'
|
||||
if not isinstance(payload.get(key), str):
|
||||
raise ValueError('string required')
|
||||
if self.path == '/analyze':
|
||||
result = engine.analyze(payload['text'])
|
||||
else:
|
||||
lemma = payload.get('lemma', '')
|
||||
if not isinstance(lemma, str) or len(lemma) > 200 or len(payload['surface']) > 200:
|
||||
raise ValueError('invalid query')
|
||||
result = engine.lookup(payload['surface'], lemma)
|
||||
self.reply(200, result)
|
||||
except (ValueError, TypeError, UnicodeError):
|
||||
self.reply(400, {'status': 'invalid_input'})
|
||||
except Exception as exc:
|
||||
# ResourceMissing is deliberately exposed without filesystem paths.
|
||||
from engine import ResourceMissing
|
||||
if isinstance(exc, ResourceMissing):
|
||||
self.reply(503, {'status': 'resource_missing', 'resource': exc.resource})
|
||||
else:
|
||||
self.reply(500, {'status': 'internal_error'})
|
||||
|
||||
server = HTTPServer(('127.0.0.1', port), Handler)
|
||||
return server
|
||||
|
||||
|
||||
def main():
|
||||
from engine import Engine
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--port', type=int, default=5183)
|
||||
parser.add_argument('--resources', type=Path, default=ROOT / '.local/nlp-resources')
|
||||
args = parser.parse_args()
|
||||
server = create_server(Engine(args.resources), args.port)
|
||||
print(f'LexGo English experiment: http://127.0.0.1:{server.server_port}/', flush=True)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
server.server_close()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,181 @@
|
||||
{
|
||||
"measured_at_utc": "2026-09-10T12:55:12.016850+00:00",
|
||||
"environment": {
|
||||
"python": "3.12.12",
|
||||
"platform": "Windows-10-10.0.19044-SP0",
|
||||
"processor": "Intel64 Family 6 Model 140 Stepping 1, GenuineIntel",
|
||||
"logical_cpus": 8,
|
||||
"packages": {
|
||||
"spacy": "3.8.7",
|
||||
"nltk": "3.9.2",
|
||||
"en-core-web-sm": "3.8.0"
|
||||
}
|
||||
},
|
||||
"offline": true,
|
||||
"cold_engine_load_ms": 2355.1940000616014,
|
||||
"text_codepoints": 100000,
|
||||
"text_utf8_bytes": 106095,
|
||||
"quality": {
|
||||
"cases": [
|
||||
{
|
||||
"sentence": "She went home.",
|
||||
"surface": "went",
|
||||
"expected": "go",
|
||||
"spacy": "go",
|
||||
"baseline": "go",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "The children ate apples.",
|
||||
"surface": "children",
|
||||
"expected": "child",
|
||||
"spacy": "child",
|
||||
"baseline": "child",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "The children ate apples.",
|
||||
"surface": "ate",
|
||||
"expected": "eat",
|
||||
"spacy": "eat",
|
||||
"baseline": "ate",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "The dogs ran quickly.",
|
||||
"surface": "dogs",
|
||||
"expected": "dog",
|
||||
"spacy": "dog",
|
||||
"baseline": "dog",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "The dogs ran quickly.",
|
||||
"surface": "ran",
|
||||
"expected": "run",
|
||||
"spacy": "run",
|
||||
"baseline": "run",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "I saw a bird.",
|
||||
"surface": "saw",
|
||||
"expected": "see",
|
||||
"spacy": "see",
|
||||
"baseline": "saw",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "The saw is sharp.",
|
||||
"surface": "saw",
|
||||
"expected": "saw",
|
||||
"spacy": "saw",
|
||||
"baseline": "saw",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "She leaves today.",
|
||||
"surface": "leaves",
|
||||
"expected": "leave",
|
||||
"spacy": "leave",
|
||||
"baseline": "leaf",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "The leaves fell.",
|
||||
"surface": "leaves",
|
||||
"expected": "leaf",
|
||||
"spacy": "leave",
|
||||
"baseline": "leaf",
|
||||
"spacy_correct": false,
|
||||
"baseline_correct": true
|
||||
},
|
||||
{
|
||||
"sentence": "They are reading books.",
|
||||
"surface": "reading",
|
||||
"expected": "read",
|
||||
"spacy": "read",
|
||||
"baseline": "reading",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "He was better yesterday.",
|
||||
"surface": "was",
|
||||
"expected": "be",
|
||||
"spacy": "be",
|
||||
"baseline": "wa",
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
},
|
||||
{
|
||||
"sentence": "She can't go.",
|
||||
"surface": "n't",
|
||||
"expected": "not",
|
||||
"spacy": "not",
|
||||
"baseline": null,
|
||||
"spacy_correct": true,
|
||||
"baseline_correct": false
|
||||
}
|
||||
],
|
||||
"total": 12,
|
||||
"spacy_correct": 11,
|
||||
"baseline_correct": 6
|
||||
},
|
||||
"measurements": {
|
||||
"first_analysis": {
|
||||
"repeats": 1,
|
||||
"median_ms": 2.9011000879108906,
|
||||
"p95_ms": 2.9011000879108906
|
||||
},
|
||||
"first_lookup": {
|
||||
"repeats": 1,
|
||||
"median_ms": 78.619199921377,
|
||||
"p95_ms": 78.619199921377
|
||||
},
|
||||
"spacy_100k": {
|
||||
"repeats": 3,
|
||||
"median_ms": 1432.92090005707,
|
||||
"p95_ms": 1499.0439999382943,
|
||||
"codepoints_per_second": 69787.52281163407
|
||||
},
|
||||
"baseline_100k": {
|
||||
"repeats": 3,
|
||||
"median_ms": 336.1744999419898,
|
||||
"p95_ms": 343.62249996047467,
|
||||
"codepoints_per_second": 297464.56086721626
|
||||
},
|
||||
"lookup": {
|
||||
"dog": {
|
||||
"repeats": 100,
|
||||
"median_ms": 0.014899997040629387,
|
||||
"p95_ms": 0.022999942302703857
|
||||
},
|
||||
"went": {
|
||||
"repeats": 100,
|
||||
"median_ms": 0.021250045392662287,
|
||||
"p95_ms": 0.02929999027401209
|
||||
},
|
||||
"zzzxqvfiction": {
|
||||
"repeats": 100,
|
||||
"median_ms": 0.003600027412176132,
|
||||
"p95_ms": 0.004999921657145023
|
||||
}
|
||||
}
|
||||
},
|
||||
"limitations": [
|
||||
"Synthetic microbenchmark; 12 selected cases do not establish general accuracy.",
|
||||
"Baseline excludes UTF-8/UTF-16 conversion; spaCy timing includes full analyze contract.",
|
||||
"Repeated queries are warm-process; no claim about production concurrency.",
|
||||
"Cold engine load includes dependency imports but OS filesystem caches may be warm.",
|
||||
"POS and sense disambiguation are not provided by dictionary lookup."
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Offline synthetic measurements; stdout is one JSON document, no text input log."""
|
||||
import argparse
|
||||
from datetime import datetime, timezone
|
||||
import importlib.metadata
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import re
|
||||
import statistics
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
from engine import Engine
|
||||
|
||||
|
||||
CASES = [
|
||||
('She went home.', 'went', 'go'),
|
||||
('The children ate apples.', 'children', 'child'),
|
||||
('The children ate apples.', 'ate', 'eat'),
|
||||
('The dogs ran quickly.', 'dogs', 'dog'),
|
||||
('The dogs ran quickly.', 'ran', 'run'),
|
||||
('I saw a bird.', 'saw', 'see'),
|
||||
('The saw is sharp.', 'saw', 'saw'),
|
||||
('She leaves today.', 'leaves', 'leave'),
|
||||
('The leaves fell.', 'leaves', 'leaf'),
|
||||
('They are reading books.', 'reading', 'read'),
|
||||
('He was better yesterday.', 'was', 'be'),
|
||||
("She can't go.", "n't", 'not'),
|
||||
]
|
||||
|
||||
|
||||
def summary(samples):
|
||||
values = sorted(samples)
|
||||
return {'repeats': len(values), 'median_ms': statistics.median(values) * 1000,
|
||||
'p95_ms': values[max(0, math.ceil(len(values) * .95) - 1)] * 1000}
|
||||
|
||||
|
||||
def measure(function, repeats):
|
||||
samples = []
|
||||
for _ in range(repeats):
|
||||
start = time.perf_counter()
|
||||
function()
|
||||
samples.append(time.perf_counter() - start)
|
||||
return summary(samples)
|
||||
|
||||
|
||||
def run(args):
|
||||
start = time.perf_counter()
|
||||
engine = Engine(args.resources)
|
||||
load_seconds = time.perf_counter() - start
|
||||
if engine.nlp is None or engine.wordnet is None:
|
||||
raise RuntimeError(f'Resources unavailable: model={engine.model_error}, wordnet={engine.wordnet_error}')
|
||||
measured = {}
|
||||
measured['first_analysis'] = measure(lambda: engine.analyze('She went home.'), 1)
|
||||
measured['first_lookup'] = measure(lambda: engine.lookup('dog'), 1)
|
||||
baseline_pattern = re.compile(r"\w+(?:['’]\w+)*|\s+|[^\w\s]", re.UNICODE)
|
||||
# Lower-cost baseline: regex spans and context-free WordNet morphology.
|
||||
def baseline(text):
|
||||
return [(match.group(), engine.wordnet.morphy(match.group().lower()) or match.group().lower(),
|
||||
match.start(), match.end()) for match in baseline_pattern.finditer(text)]
|
||||
|
||||
quality = []
|
||||
for sentence, surface, expected in CASES:
|
||||
contextual = next((token['lemma'] for token in engine.analyze(sentence)['tokens']
|
||||
if token['text'] == surface), None)
|
||||
simple = next((lemma for token, lemma, _, _ in baseline(sentence) if token == surface), None)
|
||||
quality.append(dict(sentence=sentence, surface=surface, expected=expected,
|
||||
spacy=contextual, baseline=simple,
|
||||
spacy_correct=contextual == expected, baseline_correct=simple == expected))
|
||||
seed = "She went home. The children ate apples. I saw a bird. The leaves fell. Café 😀 e\u0301\r\n"
|
||||
text = (seed * (100000 // len(seed) + 1))[:100000]
|
||||
for name, function in [('spacy', engine.analyze), ('baseline', baseline)]:
|
||||
metrics = measure(lambda: function(text), args.text_repeats)
|
||||
metrics['codepoints_per_second'] = len(text) / (metrics['median_ms'] / 1000)
|
||||
measured[name + '_100k'] = metrics
|
||||
queries = [('dog', ''), ('went', 'go'), ('zzzxqvfiction', '')]
|
||||
measured['lookup'] = {surface: measure(lambda: engine.lookup(surface, lemma), args.query_repeats)
|
||||
for surface, lemma in queries}
|
||||
return {
|
||||
'measured_at_utc': datetime.now(timezone.utc).isoformat(),
|
||||
'environment': {'python': platform.python_version(), 'platform': platform.platform(),
|
||||
'processor': platform.processor(), 'logical_cpus': os.cpu_count(),
|
||||
'packages': {name: importlib.metadata.version(name)
|
||||
for name in ('spacy', 'nltk', 'en-core-web-sm')}},
|
||||
'offline': True, 'cold_engine_load_ms': load_seconds * 1000,
|
||||
'text_codepoints': len(text), 'text_utf8_bytes': len(text.encode()),
|
||||
'quality': {'cases': quality, 'total': len(quality),
|
||||
'spacy_correct': sum(case['spacy_correct'] for case in quality),
|
||||
'baseline_correct': sum(case['baseline_correct'] for case in quality)},
|
||||
'measurements': measured,
|
||||
'limitations': [
|
||||
'Synthetic microbenchmark; 12 selected cases do not establish general accuracy.',
|
||||
'Baseline excludes UTF-8/UTF-16 conversion; spaCy timing includes full analyze contract.',
|
||||
'Repeated queries are warm-process; no claim about production concurrency.',
|
||||
'Cold engine load includes dependency imports but OS filesystem caches may be warm.',
|
||||
'POS and sense disambiguation are not provided by dictionary lookup.',
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--resources', type=Path, default=Path(__file__).resolve().parents[2] / '.local/nlp-resources')
|
||||
parser.add_argument('--text-repeats', type=int, default=3)
|
||||
parser.add_argument('--query-repeats', type=int, default=100)
|
||||
args = parser.parse_args()
|
||||
if args.text_repeats < 1 or args.query_repeats < 1:
|
||||
parser.error('repeat counts must be positive')
|
||||
with patch('socket.socket.connect', side_effect=AssertionError('network forbidden')):
|
||||
print(json.dumps(run(args), ensure_ascii=True, indent=2))
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Isolated English experiment. Resources are local; no downloader is used."""
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import unicodedata
|
||||
import warnings
|
||||
import zipfile
|
||||
|
||||
|
||||
class ResourceMissing(RuntimeError):
|
||||
def __init__(self, resource):
|
||||
self.resource = resource
|
||||
super().__init__(f'Local {resource} resource is unavailable')
|
||||
|
||||
|
||||
def validate_text(text):
|
||||
if not isinstance(text, str):
|
||||
raise TypeError('text must be a string')
|
||||
if len(text) > 100000:
|
||||
raise ValueError('text exceeds 100000 code points')
|
||||
if any(0xD800 <= ord(char) <= 0xDFFF for char in text):
|
||||
raise ValueError('text contains an unpaired surrogate')
|
||||
|
||||
|
||||
def lookup_form(text):
|
||||
validate_text(text)
|
||||
return unicodedata.normalize('NFC', text.casefold()).replace('’', "'").replace('‘', "'")
|
||||
|
||||
|
||||
class Engine:
|
||||
def __init__(self, resource_dir: Path, model_name='en_core_web_sm'):
|
||||
self.nlp = None
|
||||
self.wordnet = None
|
||||
self.model_error = None
|
||||
self.wordnet_error = None
|
||||
try:
|
||||
import spacy
|
||||
self.nlp = spacy.load(model_name, disable=['parser', 'ner'])
|
||||
except (ImportError, OSError, ValueError) as error:
|
||||
self.model_error = type(error).__name__
|
||||
try:
|
||||
from nltk.corpus.reader import WordNetCorpusReader
|
||||
from nltk.data import ZipFilePathPointer
|
||||
|
||||
class EnglishWordNet30Reader(WordNetCorpusReader):
|
||||
def map_wn(self, version='wordnet'):
|
||||
# NLTK's default cross-version OMW mapping loads a global
|
||||
# corpus. English-only WordNet 3.0 needs no such mapping.
|
||||
if self.get_version() != '3.0':
|
||||
raise ValueError('This experiment requires WordNet 3.0')
|
||||
return None
|
||||
|
||||
root = ZipFilePathPointer(str(Path(resource_dir) / 'wordnet.zip'), 'wordnet/')
|
||||
with warnings.catch_warnings():
|
||||
warnings.filterwarnings('ignore', message='The multilingual functions are not available with this Wordnet version', category=UserWarning)
|
||||
self.wordnet = EnglishWordNet30Reader(root, None)
|
||||
except (ImportError, OSError, LookupError, ValueError, zipfile.BadZipFile) as error:
|
||||
self.wordnet_error = type(error).__name__
|
||||
|
||||
def analyze(self, text):
|
||||
validate_text(text)
|
||||
if self.nlp is None:
|
||||
raise ResourceMissing('model')
|
||||
# Prefix tables make conversion linear even for long Unicode documents.
|
||||
utf8 = [0]
|
||||
utf16 = [0]
|
||||
for char in text:
|
||||
utf8.append(utf8[-1] + len(char.encode('utf-8')))
|
||||
utf16.append(utf16[-1] + (2 if ord(char) > 0xFFFF else 1))
|
||||
tokens = []
|
||||
|
||||
def append(start, end, lemma, kind):
|
||||
tokens.append(dict(text=text[start:end], lemma=lemma, kind=kind,
|
||||
start_cp=start, end_cp=end,
|
||||
start_utf8=utf8[start], end_utf8=utf8[end],
|
||||
start_utf16=utf16[start], end_utf16=utf16[end]))
|
||||
|
||||
cursor = 0
|
||||
for token in self.nlp(text):
|
||||
if token.idx > cursor:
|
||||
append(cursor, token.idx, '', 'space')
|
||||
end = token.idx + len(token.text)
|
||||
kind = 'space' if token.is_space else 'punctuation' if token.is_punct else 'word'
|
||||
append(token.idx, end, token.lemma_ if kind == 'word' else '', kind)
|
||||
cursor = end
|
||||
if cursor < len(text):
|
||||
append(cursor, len(text), '', 'space')
|
||||
return dict(status='ok', contract_version='english-spike-v1', original_text=text,
|
||||
text_sha256=hashlib.sha256(text.encode('utf-8')).hexdigest(), tokens=tokens)
|
||||
|
||||
def _exact_entries(self, form):
|
||||
# NLTK 3.9.2 synsets() applies morphy even with check_exceptions=False.
|
||||
# Read its loaded index directly to keep exact and explicit lemma distinct.
|
||||
index = self.wordnet._lemma_pos_offset_map.get(form, {})
|
||||
entries = []
|
||||
for pos in ('n', 'v', 'a', 'r'):
|
||||
for offset in index.get(pos, []):
|
||||
synset = self.wordnet.synset_from_pos_and_offset(pos, offset)
|
||||
entries.append(dict(lemma=form, pos=synset.pos(),
|
||||
definition=synset.definition(), examples=synset.examples()))
|
||||
if len(entries) == 12:
|
||||
return entries
|
||||
return entries
|
||||
|
||||
def lookup(self, surface, lemma=''):
|
||||
form = lookup_form(surface)
|
||||
fallback = lookup_form(lemma)
|
||||
result = dict(query=surface, matched_form=None, entries=[])
|
||||
if self.wordnet is None:
|
||||
return dict(result, status='resource_missing', resource='wordnet')
|
||||
for candidate, status in ((form, 'exact'), (fallback, 'lemma')):
|
||||
if not candidate:
|
||||
continue
|
||||
entries = self._exact_entries(candidate)
|
||||
if entries:
|
||||
return dict(result, status=status, matched_form=candidate, entries=entries)
|
||||
return dict(result, status='not_found')
|
||||
@@ -0,0 +1,12 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>LexGo · 英语验证</title><link rel="stylesheet" href="/style.css"></head>
|
||||
<body>
|
||||
<header><a class="brand" href="/">LexGo<span>.</span></a><span class="badge">英语 · 技术验证</span></header>
|
||||
<main>
|
||||
<section class="input-section"><label for="source">阅读文本</label><textarea id="source" spellcheck="false" maxlength="100000">Mira’s well-known dogs went home. She can't wait.
|
||||
The children were running beside a café. 🙂 Café!</textarea><div class="actions"><span>仅本机处理 · 不保存</span><button id="analyze">分析文本</button></div></section>
|
||||
<p id="notice" role="status" aria-live="polite"></p>
|
||||
<div class="workspace"><section class="paper" aria-label="阅读结果"><h1>阅读</h1><div id="reading"></div></section><aside aria-label="词典"><div class="dictionary-header"><h2>本地词典</h2><span>WordNet 3.0</span></div><form id="lookup-form"><label class="sr-only" for="query">查询单词</label><input id="query" maxlength="200" placeholder="输入英语单词" autocomplete="off"><button>查询</button></form><div id="definition" aria-live="polite">点击文中的单词。</div><details><summary>原文位置</summary><pre id="position">尚未选择单词</pre></details></aside></div>
|
||||
</main><footer>独立验证小样 · 英语释义</footer><script type="module" src="/app.mjs"></script>
|
||||
</body></html>
|
||||
@@ -0,0 +1,47 @@
|
||||
annotated-doc==0.0.5
|
||||
annotated-types==0.8.0
|
||||
blis==1.3.3
|
||||
catalogue==2.0.10
|
||||
certifi==2026.7.22
|
||||
charset-normalizer==3.5.1
|
||||
click==8.5.0
|
||||
cloudpathlib==0.25.0
|
||||
cloudpickle==3.1.2
|
||||
colorama==0.4.6
|
||||
confection==0.1.5
|
||||
cymem==2.0.13
|
||||
idna==3.19
|
||||
jinja2==3.1.6
|
||||
joblib==1.6.0
|
||||
langcodes==3.5.1
|
||||
markdown-it-py==4.2.0
|
||||
markupsafe==3.0.3
|
||||
mdurl==0.1.2
|
||||
murmurhash==1.0.15
|
||||
nltk==3.9.2
|
||||
numpy==2.5.3
|
||||
packaging==26.3
|
||||
preshed==3.0.13
|
||||
pydantic==2.13.5
|
||||
pydantic-core==2.46.5
|
||||
pygments==2.21.0
|
||||
regex==2026.9.10
|
||||
requests==2.34.2
|
||||
rich==15.0.0
|
||||
setuptools==84.0.0
|
||||
shellingham==1.5.4
|
||||
smart-open==7.7.1
|
||||
spacy==3.8.7
|
||||
spacy-legacy==3.0.12
|
||||
spacy-loggers==1.0.5
|
||||
srsly==2.5.3
|
||||
thinc==8.3.11
|
||||
tqdm==4.70.0
|
||||
typer==0.27.2
|
||||
typer-slim==0.24.0
|
||||
typing-extensions==4.16.0
|
||||
typing-inspection==0.4.4
|
||||
urllib3==2.7.0
|
||||
wasabi==1.1.3
|
||||
weasel==0.4.3
|
||||
wrapt==2.4.0
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"python": "3.12.12",
|
||||
"spacy": "3.8.7",
|
||||
"nltk": "3.9.2",
|
||||
"model": {
|
||||
"name": "en_core_web_sm",
|
||||
"version": "3.8.0",
|
||||
"file": "en_core_web_sm-3.8.0-py3-none-any.whl",
|
||||
"url": "https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl",
|
||||
"sha256": "1932429db727d4bff3deed6b34cfc05df17794f4a52eeb26cf8928f7c1a0fb85",
|
||||
"license": "MIT"
|
||||
},
|
||||
"dictionary": {
|
||||
"name": "Princeton WordNet",
|
||||
"version": "3.0",
|
||||
"file": "wordnet.zip",
|
||||
"url": "https://raw.githubusercontent.com/nltk/nltk_data/96f9b3252457a2b97e52aec64c3dfceeb5c312d5/packages/corpora/wordnet.zip",
|
||||
"sha256": "cbda5ea6eef7f36a97a43d4a75f85e07fccbb4f23657d27b4ccbc93e2646ab59",
|
||||
"license": "WordNet 3.0 License (included in ZIP: wordnet/LICENSE)"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Explicit network-only preparation; runtime never calls this script."""
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import urllib.request
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
DEST = HERE.parent.parent / '.local/nlp-resources'
|
||||
|
||||
|
||||
def main():
|
||||
manifest = json.loads((HERE / 'resources.json').read_text(encoding='utf-8'))
|
||||
DEST.mkdir(parents=True, exist_ok=True)
|
||||
for key in ('model', 'dictionary'):
|
||||
item = manifest[key]
|
||||
target = DEST / item['file']
|
||||
if target.exists() and hashlib.sha256(target.read_bytes()).hexdigest() == item['sha256']:
|
||||
print(key + ': checksum verified', flush=True)
|
||||
continue
|
||||
req = urllib.request.Request(item['url'], headers={'User-Agent': 'LexGo-English-Spike/1'})
|
||||
with urllib.request.urlopen(req, timeout=120) as response:
|
||||
data = response.read(32 * 1024 * 1024 + 1)
|
||||
if hashlib.sha256(data).hexdigest() != item['sha256']:
|
||||
raise RuntimeError(key + ': checksum mismatch; resource not installed')
|
||||
target.write_bytes(data)
|
||||
print(key + ': downloaded and checksum verified', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
:root{color:#253b32;background:#f5f4ef;font-family:"Segoe UI","Microsoft YaHei",sans-serif;font-synthesis:none}*{box-sizing:border-box}body{margin:0}header{height:78px;border-bottom:1px solid #dfe3da;padding:0 5%;display:flex;align-items:center;justify-content:space-between;background:#fff}.brand{font-size:29px;font-weight:700;text-decoration:none;color:inherit;letter-spacing:-1px}.brand span{color:#3b8060}.badge{font-size:13px;color:#66776a}main{max-width:1260px;margin:35px auto;padding:0 28px}.input-section{border-bottom:1px solid #d5ddd4;padding-bottom:25px}label{display:block;font-weight:600;margin-bottom:12px}textarea{display:block;width:100%;min-height:135px;resize:vertical;border:1px solid #cbd5cb;border-radius:8px;background:#fff;padding:16px;color:#263b30;font:18px/1.7 Georgia,serif}textarea:focus,input:focus,button:focus-visible{outline:2px solid #538967;outline-offset:3px}.actions{display:flex;justify-content:space-between;align-items:center;margin-top:12px}.actions span,footer{font-size:12px;color:#738075}button{background:#2c6347;color:white;border:0;border-radius:5px;padding:10px 20px;cursor:pointer;font:inherit}button:disabled{opacity:.55;cursor:wait}#notice{font-size:14px;min-height:20px}.workspace{display:grid;grid-template-columns:minmax(0,1fr) 330px;gap:25px}.paper,aside{background:#fff;border:1px solid #e0e5dc;border-radius:8px}.paper{padding:28px 32px;min-height:340px}h1{font-size:13px;letter-spacing:2px;color:#6f7e72;margin:0 0 26px}#reading{font:23px/1.95 Georgia,"Times New Roman",serif;white-space:pre-wrap;overflow-wrap:anywhere}#reading button{font:inherit;color:inherit;padding:0;border-radius:2px;background:transparent;text-align:left}#reading button:hover,#reading button.selected{background:#e1edcf;box-shadow:0 2px #658447}aside{padding:24px}.dictionary-header{display:flex;justify-content:space-between;align-items:center;margin-bottom:20px}.dictionary-header h2{font-size:17px;margin:0}.dictionary-header span{font-size:11px;color:#7e887f}form{display:flex;gap:7px;margin-bottom:22px}input{min-width:0;width:100%;padding:9px;border:1px solid #cbd5cb;border-radius:4px;font:inherit}form button{padding:9px 12px;white-space:nowrap}#definition{font-size:14px;line-height:1.7;overflow-wrap:anywhere}#definition h3{font:27px Georgia,serif;margin:0 0 8px}#definition ol{padding-left:21px}#definition li{margin-bottom:13px}#definition small{color:#637567}details{margin-top:25px;border-top:1px solid #e1e6de;padding-top:15px;color:#7a847d;font-size:12px}summary{cursor:pointer}pre{white-space:pre-wrap;overflow-wrap:anywhere;font-size:11px}footer{text-align:center;padding:35px}.sr-only{position:absolute;width:1px;height:1px;overflow:hidden;clip-path:inset(50%)}@media(max-width:750px){main{padding:0 16px;margin-top:20px}.workspace{grid-template-columns:1fr}.paper{padding:24px;min-height:230px}#reading{font-size:21px}.actions span{font-size:11px}}
|
||||
@@ -0,0 +1,72 @@
|
||||
import http.client
|
||||
import json
|
||||
import threading
|
||||
import unittest
|
||||
from unittest.mock import Mock
|
||||
|
||||
from app import create_server
|
||||
|
||||
|
||||
class HTTPTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.engine = Mock()
|
||||
self.engine.analyze.return_value = {'status': 'ok', 'tokens': []}
|
||||
self.engine.lookup.return_value = {'status': 'not_found', 'entries': []}
|
||||
self.server = create_server(self.engine, 0)
|
||||
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
self.port = self.server.server_port
|
||||
|
||||
def tearDown(self):
|
||||
self.server.shutdown()
|
||||
self.server.server_close()
|
||||
self.thread.join()
|
||||
|
||||
def call(self, method, path, body=None, headers=None):
|
||||
c = http.client.HTTPConnection('127.0.0.1', self.port, timeout=3)
|
||||
c.request(method, path, body, headers or {})
|
||||
r = c.getresponse()
|
||||
result = r.status, r.read(), dict(r.getheaders())
|
||||
c.close()
|
||||
return result
|
||||
|
||||
def test_local_page_and_no_arbitrary_file_access(self):
|
||||
status, body, headers = self.call('GET', '/')
|
||||
self.assertEqual(status, 200)
|
||||
self.assertIn(b'LexGo', body)
|
||||
self.assertIn('Content-Security-Policy', headers)
|
||||
self.assertEqual(self.call('GET', '/../../.env.local')[0], 404)
|
||||
|
||||
def test_json_analyze_and_lookup(self):
|
||||
self.assertEqual(self.call('POST', '/analyze', json.dumps({'text': 'Hello'}), {'Content-Type': 'application/json'})[0], 200)
|
||||
self.engine.analyze.assert_called_once_with('Hello')
|
||||
self.assertEqual(self.call('POST', '/lookup', json.dumps({'surface': 'went', 'lemma': 'go'}), {'Content-Type': 'application/json'})[0], 200)
|
||||
self.engine.lookup.assert_called_once_with('went', 'go')
|
||||
|
||||
def test_reject_cross_origin_and_rebinding(self):
|
||||
for headers in ({'Origin': 'https://evil.example'}, {'Host': 'evil.example'}):
|
||||
self.assertEqual(self.call('POST', '/analyze', '{}', headers)[0], 403)
|
||||
self.engine.analyze.assert_not_called()
|
||||
|
||||
def test_invalid_payload_and_size(self):
|
||||
for payload in ('[]', '{}', '{', '{"text":42}'):
|
||||
self.assertEqual(self.call('POST', '/analyze', payload, {'Content-Type': 'application/json'})[0], 400)
|
||||
self.assertEqual(self.call('POST', '/analyze', '{}', {'Content-Type': 'text/plain'})[0], 415)
|
||||
self.assertEqual(self.call('POST', '/analyze', '{}', {'Content-Type': 'application/json', 'Content-Length': '1000001'})[0], 413)
|
||||
|
||||
def test_internal_errors_do_not_echo_input(self):
|
||||
self.engine.analyze.side_effect = RuntimeError('private sample')
|
||||
status, body, _ = self.call('POST', '/analyze', '{"text":"x"}', {'Content-Type': 'application/json'})
|
||||
self.assertEqual(status, 500)
|
||||
self.assertNotIn(b'private sample', body)
|
||||
|
||||
def test_missing_model_is_distinct_from_invalid_input(self):
|
||||
from engine import ResourceMissing
|
||||
self.engine.analyze.side_effect = ResourceMissing('model')
|
||||
status, body, _ = self.call('POST', '/analyze', '{"text":"x"}', {'Content-Type': 'application/json'})
|
||||
self.assertEqual(status, 503)
|
||||
self.assertEqual(json.loads(body), {'status': 'resource_missing', 'resource': 'model'})
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,92 @@
|
||||
"""Run with the isolated Python: -m unittest discover -s spikes/english -v."""
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
try:
|
||||
from engine import Engine, ResourceMissing
|
||||
except ImportError:
|
||||
Engine = None
|
||||
|
||||
RESOURCES = Path(__file__).resolve().parents[2] / '.local/nlp-resources'
|
||||
|
||||
|
||||
class EngineTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.network = patch('socket.socket.connect', side_effect=AssertionError('network forbidden'))
|
||||
cls.network.start()
|
||||
cls.addClassCleanup(cls.network.stop)
|
||||
if Engine:
|
||||
cls.engine = Engine(RESOURCES)
|
||||
|
||||
def setUp(self):
|
||||
self.assertIsNotNone(Engine, 'English engine has not been implemented')
|
||||
|
||||
def test_unicode_partition_and_three_offsets(self):
|
||||
for text in ['', ' \t\r\n', " She went!\r\nDogs’ paws\tcan't. e\u0301 café 😀 中文\u00a0\u200bend ",
|
||||
'well-known mother-in-law 👩💻 👨👩👧👦 🏳️🌈']:
|
||||
with self.subTest(text=text):
|
||||
result = self.engine.analyze(text)
|
||||
self.assertEqual(result['status'], 'ok')
|
||||
self.assertEqual(result['contract_version'], 'english-spike-v1')
|
||||
self.assertEqual(result['original_text'], text)
|
||||
self.assertEqual(result['text_sha256'], hashlib.sha256(text.encode()).hexdigest())
|
||||
tokens = result['tokens']
|
||||
self.assertEqual(''.join(t['text'] for t in tokens), text)
|
||||
cursor = 0
|
||||
for token in tokens:
|
||||
self.assertEqual(token['start_cp'], cursor)
|
||||
cursor = token['end_cp']
|
||||
self.assertGreater(cursor, token['start_cp'])
|
||||
self.assertEqual(text[token['start_cp']:cursor], token['text'])
|
||||
for encoding, unit, suffix in [('utf-8', 1, 'utf8'), ('utf-16-le', 2, 'utf16')]:
|
||||
start, end = token['start_' + suffix], token['end_' + suffix]
|
||||
self.assertEqual(text.encode(encoding)[start*unit:end*unit].decode(encoding), token['text'])
|
||||
self.assertEqual(len(text[:token['start_cp']].encode(encoding)) // unit, start)
|
||||
self.assertIn(token['kind'], ['word', 'space', 'punctuation'])
|
||||
self.assertEqual(cursor, len(text))
|
||||
|
||||
def test_contextual_irregular_lemma(self):
|
||||
tokens = self.engine.analyze('She went home. The children ate apples.')['tokens']
|
||||
lemmas = {t['text']: t['lemma'] for t in tokens}
|
||||
self.assertEqual(lemmas['went'], 'go')
|
||||
self.assertEqual(lemmas['children'], 'child')
|
||||
self.assertEqual(lemmas['ate'], 'eat')
|
||||
|
||||
def test_exact_then_explicit_lemma(self):
|
||||
exact = self.engine.lookup('DOG')
|
||||
self.assertEqual(exact['status'], 'exact')
|
||||
self.assertEqual(exact['matched_form'], 'dog')
|
||||
self.assertTrue(exact['entries'])
|
||||
self.assertLessEqual(len(exact['entries']), 12)
|
||||
self.assertEqual(exact, self.engine.lookup('DOG'))
|
||||
self.assertEqual(self.engine.lookup('went')['status'], 'not_found')
|
||||
lemma = self.engine.lookup('went', 'go')
|
||||
self.assertEqual(lemma['status'], 'lemma')
|
||||
self.assertEqual(lemma['matched_form'], 'go')
|
||||
self.assertEqual(self.engine.lookup('zzzxqvfiction')['status'], 'not_found')
|
||||
|
||||
def test_validation(self):
|
||||
for text in ['x' * 100001, '\ud800']:
|
||||
with self.assertRaises(ValueError):
|
||||
self.engine.analyze(text)
|
||||
with self.assertRaises(TypeError):
|
||||
self.engine.analyze(None)
|
||||
with self.assertRaises(ValueError):
|
||||
self.engine.lookup('\udfff')
|
||||
|
||||
def test_missing_resources_are_not_misses(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
engine = Engine(Path(directory), model_name='nonexistent_english_spike_model')
|
||||
self.assertEqual(engine.lookup('dog')['status'], 'resource_missing')
|
||||
self.assertEqual(engine.lookup('dog')['resource'], 'wordnet')
|
||||
with self.assertRaises(ResourceMissing) as error:
|
||||
engine.analyze('dog')
|
||||
self.assertEqual(error.exception.resource, 'model')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,14 @@
|
||||
export function validateTokens(text, tokens) {
|
||||
let end=0
|
||||
for (const token of tokens) {
|
||||
if (!Number.isInteger(token.start_utf16) || !Number.isInteger(token.end_utf16) || token.start_utf16!==end || token.end_utf16<=end || text.slice(token.start_utf16,token.end_utf16)!==token.text) throw new Error('原文位置校验失败')
|
||||
end=token.end_utf16
|
||||
}
|
||||
if(end!==text.length) throw new Error('原文还原失败')
|
||||
return true
|
||||
}
|
||||
|
||||
export function latestOnly() {
|
||||
let version=0
|
||||
return {next:()=>++version,current:value=>value===version}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import { validateTokens, latestOnly } from './view.mjs'
|
||||
|
||||
test('UTF-16 positions reconstruct emoji and combining characters without normalization', () => {
|
||||
const text = '🙂 Café'
|
||||
const tokens = [{text:'🙂',start_utf16:0,end_utf16:2},{text:' ',start_utf16:2,end_utf16:3},{text:'Café',start_utf16:3,end_utf16:8}]
|
||||
assert.equal(validateTokens(text,tokens), true)
|
||||
assert.throws(() => validateTokens(text,[{text:'🙂',start_utf16:0,end_utf16:1}]))
|
||||
assert.throws(() => validateTokens(text,[]))
|
||||
})
|
||||
test('old analysis and lookup responses cannot replace newer text or selection', () => {
|
||||
const gate=latestOnly()
|
||||
const first=gate.next(), second=gate.next()
|
||||
assert.equal(gate.current(first),false)
|
||||
assert.equal(gate.current(second),true)
|
||||
gate.next()
|
||||
assert.equal(gate.current(second),false)
|
||||
})
|
||||
@@ -0,0 +1,39 @@
|
||||
# #4 阅读选择技术验证
|
||||
|
||||
独立小样,默认英语;虚构章节与释义,保存仅在当前页面内有效,刷新即清空。不是生产阅读器,不连接账号、数据库、词典服务或复习调度。
|
||||
|
||||
从仓库根目录运行:
|
||||
|
||||
```powershell
|
||||
python spikes/selection/serve.py
|
||||
```
|
||||
|
||||
打开 <http://127.0.0.1:5184/>。点词或用鼠标拖选连续文本;使用面板起点/终点按钮调整。正文获得焦点后,←/→ 选择相邻词,Shift+←/→ 调整连续范围,Escape 取消;关闭面板保留阅读位置和键盘续读位置。修改释义后必须点击保存;关闭不会自动保存。
|
||||
|
||||
测试(Node 22.22.1;DOM 测试复用学习端锁定的 jsdom 开发依赖):
|
||||
|
||||
```powershell
|
||||
pnpm --dir learner install --frozen-lockfile
|
||||
node --test spikes/selection/range.test.mjs spikes/selection/ui.test.mjs
|
||||
node --check spikes/selection/app.mjs
|
||||
```
|
||||
|
||||
`range.mjs` 的 Intl.Segmenter 仅用于验证 UI 选择边界,不替代 #3 的 spaCy 分词。正式集成时必须使用章节原文、内容版本和 NLP token 映射。代码没有复制 LinguaCafe 实现;固定上游版本与四项对照见 [架构 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Architecture-and-Code-Map),范围/重叠策略见 [业务规则 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Business-Rules-and-Glossary)。
|
||||
|
||||
## 手机测试证据(待补充)
|
||||
|
||||
用户已确认 #4 验收通过,工单已关闭;没有提供具体手机测试记录,以下证据缺口保留。390 CSS 像素的桌面窄屏检查不是真机结果。需要使用实际手机浏览器记录设备、系统、浏览器版本和以下结果:
|
||||
|
||||
1. 普通上下滑动正文,不应意外选词或阻止滚动。
|
||||
2. 长按单词,用系统选择手柄扩展/缩短跨行短语;正文反馈、面板文字和“选中范围”一致。特别检查靠近底部面板的位置,系统菜单/手柄是否被遮挡。
|
||||
3. 使用面板起点/终点按钮调整,验证 emoji、组合字符、连字符及两次出现的 `a small step` 有不同位置。
|
||||
4. 关闭面板继续阅读,打开“验证设置与快捷键”核对关闭前后的滚动位置;输入释义时检查虚拟键盘与面板滚动。
|
||||
5. 测试词典无结果/资源不可用,仍可填写虚构释义、显式保存并继续阅读。
|
||||
|
||||
默认服务器只监听本机。手机需与电脑处于同一可信局域网,在明确进行手机测试时,用电脑的实际局域网 IPv4 地址启动:
|
||||
|
||||
```powershell
|
||||
python spikes/selection/serve.py --host <电脑局域网IPv4地址>
|
||||
```
|
||||
|
||||
手机打开 `http://<电脑局域网IPv4地址>:5184/`。只提供本目录白名单静态文件,不提供目录列表、私人内容或写入接口。若系统防火墙阻止访问,先由用户确认网络放行范围;不自动修改防火墙或发布公网。测试完停止进程。后续真机回归结果回填相关实施工单,并引用 #4 的证据缺口。
|
||||
@@ -0,0 +1,120 @@
|
||||
import {tokenize,normalizeRange,pointSelection,adjustRange,occurrences,resolveOverlaps} from './range.mjs'
|
||||
import {chapters} from './fixtures.mjs'
|
||||
|
||||
const $=id=>document.getElementById(id)
|
||||
const reader=$('reader'), panel=$('panel'), saved=new Map(), positions=new Map()
|
||||
const meanings={'curiosity':'好奇心;求知欲','a small step':'一小步','small step':'小小的一步','learning':'学习','café':'咖啡馆'}
|
||||
let chapterIndex=0,tokens=[],nodes=[],selected=null,anchor=null,focus=null,changingChapter=false,timer
|
||||
|
||||
function chapter(){return chapters[chapterIndex]}
|
||||
function clearNative(){window.getSelection()?.removeAllRanges()}
|
||||
function paint(){
|
||||
const candidates=[]
|
||||
for(const [text,item] of saved) for(const range of occurrences(chapter().text,text)) candidates.push({...range,state:item.state})
|
||||
const visible=resolveOverlaps(candidates)
|
||||
nodes.forEach((node,index)=>{
|
||||
const token=tokens[index]
|
||||
const match=visible.find(range=>token.start>=range.start && token.end<=range.end)
|
||||
node.className=(token.word?'word ':'')+(match?.state || '')
|
||||
node.classList.toggle('selected',!!selected && token.start>=selected.start && token.end<=selected.end)
|
||||
})
|
||||
}
|
||||
function dictionary(){
|
||||
if(!selected)return
|
||||
const mode=$('dictionary-mode').value
|
||||
$('dictionary').textContent=mode==='missing'?'该语言词典暂不可用,仍可填写释义。':mode==='empty'?'未找到释义,可手动填写。':meanings[selected.text.toLocaleLowerCase('en')] || '未找到释义,可手动填写。'
|
||||
}
|
||||
function choose(range,{resetAnchor=true}={}){
|
||||
if(!range)return
|
||||
if(selected?.start===range.start && selected?.end===range.end)return
|
||||
selected=range
|
||||
if(resetAnchor){anchor=range.first;focus=range.last}
|
||||
panel.hidden=false;$('empty').hidden=true
|
||||
$('term').textContent=range.text
|
||||
$('kind').textContent=range.first===range.last?'单词':'短语'
|
||||
const item=saved.get(range.text)
|
||||
$('meaning').value=item?.meaning || ''
|
||||
$('word-state').value=item?.state || 'learning'
|
||||
$('feedback').textContent=''
|
||||
$('range-output').textContent=JSON.stringify({chapter:chapter().id,text:range.text,utf16:[range.start,range.end],code_point:[range.start_cp,range.end_cp],utf8:[range.start_utf8,range.end_utf8]},null,2)
|
||||
dictionary();paint()
|
||||
document.querySelectorAll('[data-edge]').forEach(button=>{
|
||||
const next=adjustRange(chapter().text,tokens,range,button.dataset.edge,Number(button.dataset.direction))
|
||||
button.disabled=next.start===range.start && next.end===range.end
|
||||
})
|
||||
}
|
||||
function close(){
|
||||
const scroll=window.scrollY
|
||||
selected=null;anchor=focus
|
||||
clearTimeout(timer);clearNative();panel.hidden=true;$('empty').hidden=false
|
||||
paint();reader.focus({preventScroll:true});window.scrollTo({top:scroll,behavior:'instant'})
|
||||
$('event-status').textContent=`关闭前 ${scroll},关闭后 ${window.scrollY}`
|
||||
}
|
||||
function readNativeSelection(){
|
||||
const selection=window.getSelection()
|
||||
if(changingChapter || !selection || selection.isCollapsed || !selection.rangeCount)return false
|
||||
const range=selection.getRangeAt(0)
|
||||
if(!reader.contains(range.startContainer)||!reader.contains(range.endContainer))return false
|
||||
const prefix=document.createRange();prefix.selectNodeContents(reader);prefix.setEnd(range.startContainer,range.startOffset)
|
||||
const start=prefix.toString().length
|
||||
prefix.setEnd(range.endContainer,range.endOffset)
|
||||
const end=prefix.toString().length
|
||||
const normalized=normalizeRange(chapter().text,tokens,start,end)
|
||||
if(!normalized){
|
||||
selected=null;panel.hidden=true;$('empty').hidden=false;paint()
|
||||
return false
|
||||
}
|
||||
choose(normalized)
|
||||
return true
|
||||
}
|
||||
function renderChapter(){
|
||||
changingChapter=true;clearTimeout(timer);clearNative();selected=null;anchor=null;focus=null
|
||||
$('title').textContent=chapter().title;$('chapter-number').textContent=`${chapterIndex+1} / ${chapters.length}`
|
||||
$('previous').disabled=chapterIndex===0;$('next').disabled=chapterIndex===chapters.length-1
|
||||
tokens=tokenize(chapter().text)
|
||||
nodes=tokens.map((token,index)=>{const span=document.createElement('span');span.textContent=token.text;span.dataset.index=String(index);return span})
|
||||
reader.replaceChildren(...nodes)
|
||||
if(reader.textContent!==chapter().text)throw new Error('Original text reconstruction failed')
|
||||
panel.hidden=true;$('empty').hidden=false;paint();changingChapter=false
|
||||
}
|
||||
reader.addEventListener('click',event=>{
|
||||
if(readNativeSelection())return
|
||||
const node=event.target.closest('[data-index]')
|
||||
if(node)choose(pointSelection(chapter().text,tokens,Number(node.dataset.index)))
|
||||
})
|
||||
// Native selection remains in charge of mouse dragging and mobile long-press handles.
|
||||
// No touchmove preventDefault, pointer capture or text-node replacement on selection.
|
||||
document.addEventListener('selectionchange',()=>{clearTimeout(timer);timer=setTimeout(readNativeSelection,100)})
|
||||
reader.addEventListener('pointerup',()=>{clearTimeout(timer);timer=setTimeout(readNativeSelection,0)})
|
||||
reader.addEventListener('keydown',event=>{
|
||||
if(event.key==='Escape'){event.preventDefault();close();return}
|
||||
if(!['ArrowLeft','ArrowRight'].includes(event.key)||event.ctrlKey||event.metaKey||event.altKey)return
|
||||
event.preventDefault();clearNative()
|
||||
const direction=event.key==='ArrowLeft'?-1:1
|
||||
let next=focus===null?(direction===1?0:tokens.length-1):focus+direction
|
||||
while(next>=0 && next<tokens.length && !tokens[next].word)next+=direction
|
||||
if(next<0||next>=tokens.length)return
|
||||
if(!event.shiftKey||anchor===null){anchor=next;focus=next;choose(pointSelection(chapter().text,tokens,next),{resetAnchor:false})}
|
||||
else {focus=next;choose(normalizeRange(chapter().text,tokens,Math.min(tokens[anchor].start,tokens[focus].start),Math.max(tokens[anchor].end,tokens[focus].end)),{resetAnchor:false})}
|
||||
// Keyboard users can navigate a long chapter without losing the active word.
|
||||
nodes[next].scrollIntoView({block:'nearest',inline:'nearest',behavior:'instant'})
|
||||
})
|
||||
document.addEventListener('keydown',event=>{if(event.key==='Escape' && !panel.hidden && event.target!==reader){event.preventDefault();close()}})
|
||||
document.querySelectorAll('[data-edge]').forEach(button=>button.addEventListener('click',()=>{
|
||||
if(!selected)return
|
||||
clearNative();choose(adjustRange(chapter().text,tokens,selected,button.dataset.edge,Number(button.dataset.direction)))
|
||||
}))
|
||||
$('close').addEventListener('click',close)
|
||||
$('save').addEventListener('click',()=>{
|
||||
if(!selected)return
|
||||
const meaning=$('meaning').value.trim()
|
||||
if(!meaning){$('feedback').textContent='请填写释义。';return}
|
||||
saved.set(selected.text,{meaning,state:$('word-state').value})
|
||||
$('feedback').textContent='已保存 · 本次页面内有效';paint()
|
||||
})
|
||||
$('dictionary-mode').addEventListener('change',dictionary)
|
||||
for(const [id,delta] of [['previous',-1],['next',1]])$(id).addEventListener('click',()=>{
|
||||
positions.set(chapter().id,window.scrollY);chapterIndex+=delta;renderChapter()
|
||||
reader.focus({preventScroll:true});window.scrollTo({top:positions.get(chapter().id)||0,behavior:'instant'})
|
||||
})
|
||||
renderChapter()
|
||||
@@ -0,0 +1,54 @@
|
||||
// Fictional public test material, never copied from a learner's library.
|
||||
export const chapters = [
|
||||
{
|
||||
id: 'lantern-1',
|
||||
title: 'The Lantern Workshop',
|
||||
text: `Mira opened the workshop before sunrise. A folded map lay beside the window, and a small step separated the quiet room from the garden. Her curiosity had brought her here. Today she would build a lantern that could travel across the valley.
|
||||
|
||||
“Begin with a small step,” her friend Rowan said. He placed a paper frame on the table. Mira examined the thin edges, turned the frame around, and found a tiny mark near the corner. The mark looked like a star, but its five points were not quite equal.
|
||||
|
||||
Outside, rain tapped against the glass. Inside, the workshop smelled of cedar and warm tea. They measured the paper twice. Then they measured it again, because a hurried answer could turn an ordinary mistake into a long afternoon of repairs.
|
||||
|
||||
Mira wrote a note: keep the frame light, keep the corners strong, and leave enough room for the light. Rowan read the note aloud. Neither of them knew whether the lantern would fly, but both wanted to understand why it might.
|
||||
|
||||
Their first attempt leaned to the left. The second leaned to the right. At the third attempt, Mira laughed. “A well-known problem,” she said, “can still offer an unexpected lesson.” She did not throw the old frames away. Each one showed a different part of the problem.
|
||||
|
||||
By noon the rain had stopped. A narrow line of sunlight crossed the wooden floor. Mira moved the lantern into the light, and the paper shone like a pale orange leaf. Her curiosity was no smaller than it had been in the morning.
|
||||
|
||||
On the shelf was a postcard from an imaginary traveler: “Hello 🙂! The café has a sign that reads café. Our paper astronaut 👩🚀 is ready for the journey. A second sign says 中文.” Mira kept every character exactly as it was written, including the accent that followed its letter.
|
||||
|
||||
The last instruction covered two lines:
|
||||
take a small step
|
||||
and watch what changes.
|
||||
Rowan pointed at the break between them. A pause was useful, he thought; it gave the reader time to notice the words on either side. The paper itself had not lost a single letter.
|
||||
|
||||
In the garden, a light breeze lifted the grass. Mira carried the lantern to the gate and waited. She could hear a bird in the old pear tree, a cart on the distant road, and Rowan counting softly behind her. She held the frame with both hands until the counting stopped.
|
||||
|
||||
The lantern rose a little, turned slowly, and settled beside the gate. It had traveled less than a meter. Mira smiled anyway. For a first journey, a small step was enough. Tomorrow they would return to the workshop, read their notes, and try again.`,
|
||||
},
|
||||
{
|
||||
id: 'lantern-2',
|
||||
title: 'Across the Quiet Valley',
|
||||
text: `The next morning, Rowan arrived with a blue notebook. He had drawn the valley as a series of gentle curves. A small bridge crossed the river near the mill, and a winding path connected the bridge to the workshop. Mira studied the drawing before she touched the lantern.
|
||||
|
||||
“Yesterday was a small step,” she said. “Today we should learn what the wind is doing.” They tied a ribbon to the gate. It pointed toward the river, fell still, and then lifted toward the hills. The wind had more than one answer.
|
||||
|
||||
They waited beneath the pear tree. Waiting felt different when there was something to observe. Rowan noted the direction of each gust. Mira counted the seconds between them. After a while, the blank page held a pattern that neither of them had expected.
|
||||
|
||||
A neighbor passed with an empty basket. She asked why they were staring at a ribbon. Mira explained the experiment, and the neighbor offered a practical suggestion: walk to the bridge before making another lantern. The river often changed the air near its banks.
|
||||
|
||||
At the bridge, the breeze was cooler. Water slipped around the stones with a soft, uneven sound. Rowan opened the notebook to a fresh page. Mira held the ribbon above the railing and watched its end flicker. Their curiosity made an ordinary walk feel like a journey.
|
||||
|
||||
There was no single perfect direction. Near the bank, the ribbon pointed downstream. At the center of the bridge, it twisted back toward the trees. They wrote down both observations. An answer that left out the difficult part would not help them tomorrow.
|
||||
|
||||
On their way home, they stopped beside a weather-beaten sign. Its painted words had faded, but the letters were still clear: a small step can begin a long journey. Underneath, someone had drawn a smiling face 🙂 and a tiny astronaut 👩🚀 beside an imaginary moon.
|
||||
|
||||
Back at the workshop, Mira compared the old paper frames. One was too heavy at its base. Another had a loose corner. She chose the simplest frame and changed only one thing. Rowan recorded the change before they carried the lantern outside.
|
||||
|
||||
This time it floated beyond the gate. For a moment, the orange paper rested against the blue sky like a leaf on clear water. Then the breeze shifted, and the lantern drifted gently into the grass. Nothing had broken. They could try again with the same frame.
|
||||
|
||||
Mira did not call the experiment a failure. The notebook now held a direction, a time, and a careful description of the flight. They had learned something they could test. That was enough to make the next question smaller and more useful.
|
||||
|
||||
When evening came, they closed the window and cleared the table. The map remained beside the notebook. Tomorrow might bring a longer flight, or another surprising turn. Either way, they would begin with a small step, pay attention, and preserve the story exactly as it happened.`,
|
||||
},
|
||||
];
|
||||
@@ -0,0 +1,8 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>LexGo · 阅读选择验证</title><link rel="stylesheet" href="/style.css"></head>
|
||||
<body><header><a class="brand" href="/">LexGo<span>.</span></a><span>英语 · 阅读选择验证</span></header>
|
||||
<main><div class="book"><div><p class="eyebrow">A SMALL STEP, EVERY DAY</p><h1 id="title"></h1></div><nav aria-label="章节"><button id="previous" class="quiet">上一章</button><span id="chapter-number"></span><button id="next" class="quiet">下一章</button></nav></div>
|
||||
<div class="layout"><section class="paper"><p class="hint" id="reader-help">点词查词,划选连续短语。</p><article id="reader" tabindex="0" aria-label="章节正文" aria-describedby="reader-help"></article></section>
|
||||
<aside class="side"><p id="empty">选择一个词或短语</p><section id="panel" aria-label="查词面板" hidden><div class="panel-top"><span id="kind"></span><button id="close" class="quiet" aria-label="关闭,继续阅读">关闭</button></div><h2 id="term"></h2><div class="range-controls" aria-label="调整选择范围"><span>起点</span><button data-edge="start" data-direction="-1" aria-label="起点前移">−</button><button data-edge="start" data-direction="1" aria-label="起点后移">+</button><span>终点</span><button data-edge="end" data-direction="-1" aria-label="终点前移">−</button><button data-edge="end" data-direction="1" aria-label="终点后移">+</button></div><p id="dictionary"></p><label for="meaning">我的释义</label><textarea id="meaning" rows="2" maxlength="500"></textarea><label for="word-state">状态</label><select id="word-state"><option value="learning">学习中</option><option value="known">已认识</option><option value="ignored">忽略</option></select><button id="save">保存到生词本</button><p id="feedback" role="status" aria-live="polite"></p><details><summary>选中范围</summary><output id="range-output"></output></details></section></aside></div>
|
||||
<details class="settings"><summary>验证设置与快捷键</summary><p>正文获得焦点后:← / → 选择相邻词;Shift + ← / → 调整连续范围;Escape 取消。范围按钮也可用 Tab、Enter 操作。</p><label for="dictionary-mode">词典状态</label><select id="dictionary-mode"><option value="ready">正常</option><option value="empty">无结果</option><option value="missing">资源不可用</option></select><p>虚构文本与释义,保存仅在本次页面内有效。刷新即清空。</p><p id="event-status" role="status"></p></details>
|
||||
</main><footer>交互验证小样 · 不保存私人数据</footer><script type="module" src="/app.mjs"></script></body></html>
|
||||
@@ -0,0 +1,85 @@
|
||||
// UI selection boundaries only: this does not replace the #3 NLP tokenizer.
|
||||
const words = new Intl.Segmenter('en', { granularity: 'word' });
|
||||
const graphemes = new Intl.Segmenter('en', { granularity: 'grapheme' });
|
||||
const encoder = new TextEncoder();
|
||||
|
||||
export function tokenize(text) {
|
||||
const tokens = [];
|
||||
for (const part of words.segment(text)) {
|
||||
if (part.isWordLike) {
|
||||
tokens.push({ text: part.segment, start: part.index, end: part.index + part.segment.length, word: true });
|
||||
} else {
|
||||
for (const item of graphemes.segment(part.segment)) {
|
||||
const start = part.index + item.index;
|
||||
tokens.push({ text: item.segment, start, end: start + item.segment.length, word: /\p{Extended_Pictographic}/u.test(item.segment) });
|
||||
}
|
||||
}
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
function selection(text, tokens, first, last) {
|
||||
const start = tokens[first].start;
|
||||
const end = tokens[last].end;
|
||||
return {
|
||||
start, end, text: text.slice(start, end), first, last,
|
||||
start_cp: [...text.slice(0, start)].length,
|
||||
end_cp: [...text.slice(0, end)].length,
|
||||
start_utf8: encoder.encode(text.slice(0, start)).length,
|
||||
end_utf8: encoder.encode(text.slice(0, end)).length,
|
||||
start_utf16: start, end_utf16: end,
|
||||
};
|
||||
}
|
||||
|
||||
// Inputs and outputs use half-open UTF-16 offsets into the unchanged original.
|
||||
export function normalizeRange(text, tokens, start, end) {
|
||||
if (![start, end].every(n => Number.isInteger(n) && n >= 0 && n <= text.length) || start === end) return null;
|
||||
const low = Math.min(start, end);
|
||||
const high = Math.max(start, end);
|
||||
let first = -1;
|
||||
let last = -1;
|
||||
tokens.forEach((token, index) => {
|
||||
if (token.word && token.start < high && token.end > low) {
|
||||
if (first === -1) first = index;
|
||||
last = index;
|
||||
}
|
||||
});
|
||||
return first === -1 ? null : selection(text, tokens, first, last);
|
||||
}
|
||||
|
||||
export function pointSelection(text, tokens, index) {
|
||||
return Number.isInteger(index) && tokens[index]?.word ? selection(text, tokens, index, index) : null;
|
||||
}
|
||||
|
||||
// direction -1/+1 moves a boundary left/right; it never collapses a selection.
|
||||
export function adjustRange(text, tokens, range, edge, direction) {
|
||||
if (!range || !['start', 'end'].includes(edge) || ![-1, 1].includes(direction)) return range;
|
||||
let next = (edge === 'start' ? range.first : range.last) + direction;
|
||||
while (next >= 0 && next < tokens.length && !tokens[next].word) next += direction;
|
||||
if (next < 0 || next >= tokens.length) return range;
|
||||
const first = edge === 'start' ? next : range.first;
|
||||
const last = edge === 'end' ? next : range.last;
|
||||
return first > last ? range : selection(text, tokens, first, last);
|
||||
}
|
||||
|
||||
export function occurrences(text, needle) {
|
||||
if (!needle) return [];
|
||||
const tokens = tokenize(text);
|
||||
const result = [];
|
||||
for (let start = text.indexOf(needle); start !== -1; start = text.indexOf(needle, start + 1)) {
|
||||
const end = start + needle.length;
|
||||
const range = normalizeRange(text, tokens, start, end);
|
||||
if (range?.start === start && range.end === end) result.push(range);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// Display precedence only. Saved occurrence identities remain separate.
|
||||
export function resolveOverlaps(ranges) {
|
||||
const sorted = [...ranges].sort((a, b) => a.start - b.start || b.end - a.end);
|
||||
const result = [];
|
||||
for (const range of sorted) {
|
||||
if (range.end > range.start && (!result.length || range.start >= result.at(-1).end)) result.push(range);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { tokenize, normalizeRange, pointSelection, adjustRange, occurrences, resolveOverlaps } from './range.mjs';
|
||||
|
||||
test('partition preserves original Unicode, whitespace and punctuation exactly', () => {
|
||||
const text = 'Hi, cafe\u0301!\n🙂 👩🚀 well-known 中文';
|
||||
const tokens = tokenize(text);
|
||||
assert.equal(tokens.map(t => t.text).join(''), text);
|
||||
let end = 0;
|
||||
for (const t of tokens) {
|
||||
assert.equal(t.start, end);
|
||||
assert.equal(text.slice(t.start, t.end), t.text);
|
||||
end = t.end;
|
||||
}
|
||||
assert.deepEqual(tokens.filter(t => t.word).map(t => t.text), ['Hi', 'cafe\u0301', '🙂', '👩🚀', 'well', 'known', '中文']);
|
||||
});
|
||||
|
||||
test('reverse partial selection aligns words and preserves interior punctuation/newlines', () => {
|
||||
const text = ' bright,\nsmall step! ';
|
||||
const tokens = tokenize(text);
|
||||
const range = normalizeRange(text, tokens, 18, 4);
|
||||
assert.equal(range?.text, 'bright,\nsmall step');
|
||||
assert.equal(range.start, 2);
|
||||
assert.equal(range.end, 20);
|
||||
assert.deepEqual(normalizeRange(text, tokens, 0, text.length), range);
|
||||
});
|
||||
|
||||
test('collapsed, whitespace, punctuation and invalid bounds do not select neighbors', () => {
|
||||
const text = 'a, b';
|
||||
const tokens = tokenize(text);
|
||||
for (const [a, b] of [[1, 1], [1, 3], [-1, 2], [0, 5], [NaN, 3], [0.5, 3]]) {
|
||||
assert.equal(normalizeRange(text, tokens, a, b), null);
|
||||
}
|
||||
});
|
||||
|
||||
test('selection cannot cut a surrogate pair, ZWJ or combining grapheme', () => {
|
||||
const text = '🙂 e\u0301 👩🚀';
|
||||
const tokens = tokenize(text);
|
||||
const first = normalizeRange(text, tokens, 1, 2);
|
||||
assert.equal(first?.text, '🙂');
|
||||
const accent = normalizeRange(text, tokens, 4, 5);
|
||||
assert.equal(accent?.text, 'e\u0301');
|
||||
const astronaut = normalizeRange(text, tokens, 8, 9);
|
||||
assert.equal(astronaut?.text, '👩🚀');
|
||||
assert.deepEqual([astronaut.start_cp, astronaut.end_cp, astronaut.start_utf8, astronaut.end_utf8, astronaut.start_utf16, astronaut.end_utf16], [5, 8, 9, 20, 6, 11]);
|
||||
});
|
||||
|
||||
test('point selection and boundary adjustment skip separators and never invert', () => {
|
||||
const text = 'one, two\nthree';
|
||||
const tokens = tokenize(text);
|
||||
const middle = tokens.findIndex(t => t.text === 'two');
|
||||
const one = pointSelection(text, tokens, middle);
|
||||
assert.equal(one?.text, 'two');
|
||||
assert.equal(pointSelection(text, tokens, middle - 1), null);
|
||||
const left = adjustRange(text, tokens, one, 'start', -1);
|
||||
assert.equal(left?.text, 'one, two');
|
||||
const all = adjustRange(text, tokens, left, 'end', 1);
|
||||
assert.equal(all?.text, text);
|
||||
assert.equal(adjustRange(text, tokens, all, 'start', 1)?.text, 'two\nthree');
|
||||
assert.equal(adjustRange(text, tokens, all, 'end', -1)?.text, 'one, two');
|
||||
assert.deepEqual(adjustRange(text, tokens, one, 'start', 1), one);
|
||||
assert.deepEqual(adjustRange(text, tokens, one, 'end', -1), one);
|
||||
assert.deepEqual(adjustRange(text, tokens, all, 'start', -1), all);
|
||||
assert.deepEqual(adjustRange(text, tokens, all, 'end', 1), all);
|
||||
});
|
||||
|
||||
test('occurrences preserve distinct offsets, case and exact combining form', () => {
|
||||
const text = 'a small step; a small step. A small step. steps e\u0301 é';
|
||||
assert.deepEqual(occurrences(text, 'a small step').map(r => [r.start, r.end]), [[0, 12], [14, 26]]);
|
||||
assert.equal(occurrences(text, 'step').length, 3);
|
||||
assert.equal(occurrences(text, 'e\u0301').length, 1);
|
||||
assert.equal(occurrences(text, 'é').length, 1);
|
||||
assert.equal(occurrences(text, 'small st').length, 0);
|
||||
assert.deepEqual(occurrences(text, ''), []);
|
||||
});
|
||||
|
||||
test('overlapping occurrences remain available; display picks leftmost longest without mutation', () => {
|
||||
assert.deepEqual(occurrences('a a a', 'a a').map(r => r.start), [0, 2]);
|
||||
const ranges = [{ start: 2, end: 9, id: 'later' }, { start: 0, end: 3, id: 'short' }, { start: 0, end: 5, id: 'long' }, { start: 5, end: 7, id: 'next' }];
|
||||
const before = structuredClone(ranges);
|
||||
assert.deepEqual(resolveOverlaps(ranges).map(r => r.id), ['long', 'next']);
|
||||
assert.deepEqual(ranges, before);
|
||||
});
|
||||
@@ -0,0 +1,43 @@
|
||||
"""Serve only the fictional reading experiment; default is loopback."""
|
||||
import argparse
|
||||
import ipaddress
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
FILES = {'/': ('index.html', 'text/html'), '/style.css': ('style.css', 'text/css'),
|
||||
'/app.mjs': ('app.mjs', 'text/javascript'), '/range.mjs': ('range.mjs', 'text/javascript'),
|
||||
'/fixtures.mjs': ('fixtures.mjs', 'text/javascript')}
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
timeout = 10
|
||||
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def do_GET(self):
|
||||
item = FILES.get(self.path)
|
||||
data = (ROOT / item[0]).read_bytes() if item else b'Not found'
|
||||
self.send_response(200 if item else 404)
|
||||
self.send_header('Content-Type', (item[1] if item else 'text/plain') + '; charset=utf-8')
|
||||
self.send_header('Content-Length', str(len(data)))
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.send_header('X-Content-Type-Options', 'nosniff')
|
||||
self.send_header('Content-Security-Policy', "default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'none'; frame-ancestors 'none'; base-uri 'none'")
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--host', default='127.0.0.1', help='Numeric local address; LAN testing is opt-in')
|
||||
parser.add_argument('--port', type=int, default=5184)
|
||||
args = parser.parse_args()
|
||||
ipaddress.IPv4Address(args.host)
|
||||
with HTTPServer((args.host, args.port), Handler) as server:
|
||||
print(f'LexGo selection experiment: http://{args.host}:{server.server_port}/', flush=True)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
@@ -0,0 +1 @@
|
||||
:root{font-family:"Segoe UI","Microsoft YaHei",sans-serif;color:#203d33;background:#f6f4ed;--green:#285c49;--line:#dbe1d8}*{box-sizing:border-box}body{margin:0}header{height:76px;background:#fff;border-bottom:1px solid var(--line);padding:0 5%;display:flex;align-items:center;justify-content:space-between}header>span{color:#6c7971;font-size:13px}.brand{color:inherit;text-decoration:none;font-size:28px;font-weight:750;letter-spacing:-1px}.brand span{color:#538461}main{max-width:1220px;margin:36px auto;padding:0 24px}.book{display:flex;align-items:center;justify-content:space-between;gap:20px;margin-bottom:25px}.eyebrow{font-size:11px;letter-spacing:2px;color:#738276}h1{font-size:26px;font-weight:600;margin:10px 0}nav{display:flex;gap:12px;align-items:center;white-space:nowrap}button,input,textarea,select{font:inherit}button{cursor:pointer;border:1px solid var(--line);border-radius:5px;padding:9px 14px;background:var(--green);color:#fff;touch-action:manipulation}button:disabled{opacity:.4;cursor:default}.quiet{background:#fff;color:var(--green)}button:focus-visible,select:focus-visible,textarea:focus-visible,article:focus-visible{outline:2px solid #588364;outline-offset:3px}.layout{display:grid;grid-template-columns:minmax(0,1fr) 310px;gap:24px;align-items:start}.paper{background:#fff;border:1px solid var(--line);border-radius:8px;padding:32px 38px}.hint{font-size:12px;color:#738276;margin:0 0 22px}article{font:22px/1.95 Georgia,"Times New Roman",serif;white-space:pre-wrap;overflow-wrap:anywhere;user-select:text;-webkit-user-select:text;touch-action:auto;outline-offset:10px}article span.word{cursor:text}article .learning{background:#f5e7b7}article .known{box-shadow:inset 0 -2px #a2c1ab}article .ignored{color:#849086}article .selected{background:#d7e9b3;box-shadow:inset 0 -2px #56743d}article ::selection{background:#bad59a;color:#18321e}.side{position:sticky;top:18px;min-height:100px}#empty{font-size:14px;color:#859186;padding:20px;text-align:center}#panel{background:#fff;border:1px solid var(--line);border-radius:8px;padding:22px}.panel-top{display:flex;align-items:center;justify-content:space-between;color:#7b897f;font-size:12px}h2{font:27px/1.4 Georgia,serif;margin:14px 0;overflow-wrap:anywhere;max-height:120px;overflow:auto}.range-controls{display:flex;align-items:center;gap:5px;font-size:12px;flex-wrap:wrap}.range-controls button{min-width:32px;padding:6px;background:#f5f7ef;color:var(--green)}.range-controls span:nth-of-type(2){margin-left:8px}#dictionary{font-size:14px;line-height:1.6;border-top:1px solid var(--line);padding-top:16px}label{display:block;font-size:12px;color:#667b6d;margin:16px 0 7px}textarea,select{width:100%;border:1px solid var(--line);border-radius:4px;padding:9px;background:white;color:inherit}textarea{resize:vertical}#save{width:100%;margin-top:18px}#feedback{min-height:20px;font-size:13px;color:#467345}details{font-size:12px;color:#748276}summary{cursor:pointer}output{display:block;white-space:pre-wrap;overflow-wrap:anywhere;font:11px/1.5 Consolas,monospace;margin-top:10px}.settings{margin-top:28px;max-width:720px;line-height:1.8}.settings select{max-width:240px}footer{text-align:center;font-size:12px;color:#839083;padding:30px}[hidden]{display:none!important}@media(max-width:760px){header{height:60px}.book{display:block}nav{margin-top:18px;justify-content:space-between}main{padding:0 14px;margin-top:22px}.layout{display:block}.paper{padding:24px 20px}article{font-size:20px;line-height:1.95}article .word{scroll-margin-bottom:45dvh}body{padding-bottom:360px}.side{position:static;min-height:0}#empty{display:none}#panel{position:fixed;bottom:0;left:0;right:0;z-index:2;max-height:42dvh;overflow:auto;overscroll-behavior:contain;border-radius:14px 14px 0 0;padding:16px 20px;box-shadow:0 -4px 20px #233c3314}.panel-top{position:sticky;top:-16px;background:#fff;padding:5px 0}h2{font-size:23px;max-height:65px}.range-controls button{min-height:40px;min-width:40px}}
|
||||
@@ -0,0 +1,53 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import {readFile} from 'node:fs/promises'
|
||||
import {JSDOM} from '../../learner/node_modules/jsdom/lib/api.js'
|
||||
|
||||
let serial=0
|
||||
async function page(run){
|
||||
const html=await readFile(new URL('./index.html',import.meta.url),'utf8')
|
||||
const dom=new JSDOM(html,{url:'http://127.0.0.1:5184/'})
|
||||
globalThis.window=dom.window;globalThis.document=dom.window.document
|
||||
dom.window.scrollTo=()=>{}
|
||||
dom.window.HTMLElement.prototype.scrollIntoView=()=>{}
|
||||
await import(`./app.mjs?test=${++serial}`)
|
||||
const $=id=>document.getElementById(id)
|
||||
try{await run($,dom.window)}finally{await new Promise(resolve=>setTimeout(resolve,120));dom.window.close()}
|
||||
}
|
||||
|
||||
test('close preserves keyboard cursor: next arrow continues at the adjacent word',()=>page(($,w)=>{
|
||||
const span=[...$('reader').children].find(n=>n.textContent==='workshop')
|
||||
span.click();assert.equal($('term').textContent,'workshop')
|
||||
$('close').click();assert.equal($('panel').hidden,true)
|
||||
$('reader').dispatchEvent(new w.KeyboardEvent('keydown',{key:'ArrowRight',bubbles:true}))
|
||||
assert.equal($('term').textContent,'before')
|
||||
}))
|
||||
test('native punctuation-only selection clears stale word and cannot save it',()=>page(async($,w)=>{
|
||||
[...$('reader').children].find(n=>n.textContent==='curiosity').click()
|
||||
const dot=[...$('reader').children].find(n=>n.textContent==='.')
|
||||
const range=document.createRange();range.selectNodeContents(dot)
|
||||
w.getSelection().removeAllRanges();w.getSelection().addRange(range)
|
||||
document.dispatchEvent(new w.Event('selectionchange'))
|
||||
await new Promise(resolve=>setTimeout(resolve,130))
|
||||
assert.equal($('panel').hidden,true)
|
||||
assert.equal($('reader').querySelectorAll('.selected').length,0)
|
||||
}))
|
||||
test('native selection maps DOM text through spans and expands to full token endpoints',()=>page(async($,w)=>{
|
||||
const spans=[...$('reader').children]
|
||||
const start=spans.find(n=>n.textContent==='Mira'),end=spans.find(n=>n.textContent==='workshop')
|
||||
const range=document.createRange();range.setStart(start.firstChild,2);range.setEnd(end.firstChild,3)
|
||||
w.getSelection().removeAllRanges();w.getSelection().addRange(range)
|
||||
document.dispatchEvent(new w.Event('selectionchange'))
|
||||
await new Promise(resolve=>setTimeout(resolve,130))
|
||||
assert.equal($('term').textContent,'Mira opened the workshop')
|
||||
const result=JSON.parse($('range-output').textContent)
|
||||
assert.equal($('reader').textContent.slice(...result.utf16),result.text)
|
||||
}))
|
||||
test('explicit save updates repeated words; unsaved edits are discarded on close',()=>page(($)=>{
|
||||
const words=[...$('reader').children].filter(n=>n.textContent==='curiosity')
|
||||
words[0].click();$('meaning').value='虚构释义';$('save').click()
|
||||
assert.ok(words.every(n=>n.classList.contains('learning')))
|
||||
$('meaning').value='未保存修改';$('close').click();words[1].click()
|
||||
assert.equal($('meaning').value,'虚构释义')
|
||||
$('next').click();assert.equal($('panel').hidden,true)
|
||||
}))
|
||||
Reference in New Issue
Block a user