diff --git a/AGENTS.md b/AGENTS.md index 947edff..75f3057 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -258,7 +258,7 @@ MVP 内所有单元任务通过后才能做 MVP 集成验收;MVP 通过后才 - 治理模式:轻量。数据库:MySQL 8(用户于 2026-09-10 确认);具体小版本在工程验证后锁定。 - 远端:https://git.ilapage.cn/OPC/lexgo.git;分支 main。不得把邻接 dev_harness 工作区当成本项目工作区。 -- 工程基础 #2 已实现待用户验收:server 基于指定 go-admin 选用模型扩展账号/会话 API,admin 复用 go-admin-ui,learner 为独立 Vue 3 + TypeScript + Vite 工程。默认英语;阅读、导入、词典、复习及 Python NLP 尚未实现或验证。 +- 工程基础 #2 已通过用户验收:server 基于指定 go-admin 选用模型扩展账号/会话 API,admin 复用 go-admin-ui,learner 为独立 Vue 3 + TypeScript + Vite 工程。默认英语;阅读、导入、词典、复习及 Python NLP 尚未实现或验证。 - 原四份研究保留为历史参考;PostgreSQL 建议被 MySQL 8 决策覆盖,U/A/N 索引用于追踪而不是批准所有范围。 - 用户/语言数据所有权、Unicode 原文位置、任务和复习幂等、完整备份恢复是后续方案的必要验收边界。 - 当前 MCP 连接其他 Gitea 站点,需使用目标站点 API 时记录原因;凭据仅从安全配置进入进程。 @@ -277,3 +277,4 @@ MVP 内所有单元任务通过后才能做 MVP 集成验收;MVP 通过后才 - 学习端和管理端均使用账号(用户名)+密码登录,账号不要求邮箱格式,邮箱不作为必填登录标识;后端独立校验管理权限与本人学习数据归属。 - 已验证 MySQL 8.4.3,本机 127.0.0.1:3308;开发库 lexgo_dev、测试库 lexgo_test_issue2。密码只从环境或忽略的 .env.local 读取。迁移测试只能使用 lexgo_test_ 前缀专用库,不能借用其他数据库。 - 后端命令使用 `python scripts/server.py migrate|bootstrap|serve|build|test|test-integration`;仅显式 migrate 修改表。bootstrap 只接受尚无账号的 LexGo 库,不覆盖已有管理员。Go 1.26.5、Node 22.22.1、pnpm 9.15.1;两端分别构建。 +- #18 登录日志与操作审计已通过用户验收:schema v2 显式迁移;日志只保存白名单字段,禁止保存凭据、请求/响应正文及私人学习内容。仅管理员查询,默认保留 90 天;启动/每小时及 `python scripts/server.py audit-cleanup` 仅清理两张审计表的过期记录。 diff --git a/README.md b/README.md index 4066292..1e2df7f 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,11 @@ 面向自托管场景的阅读式语言学习项目,规划提供内容导入、阅读查词、词汇与短语、复习和实例管理。 -已确认:**DevHarness 轻量模式、MySQL 8、go-admin 管理端**。工程基础 #2 已实现待验收:两端用户名登录、学习账号管理、可撤销会话和本人英语空空间。管理端基于指定 go-admin/go-admin-ui 选用模块,学习端为独立 Vue 3 + TypeScript + Vite 工程,共用 Go 后端和 MySQL 8.4.3。阅读、导入、词典与复习尚未实现。MVP 定位为“支持多账号、数据独立的自托管学习工具”,先邀请少量用户使用;F01~F12 已确认,X 系列后置。 +已确认:**DevHarness 轻量模式、MySQL 8、go-admin 管理端**。工程基础 #2 已通过验收:两端用户名登录、学习账号管理、可撤销会话和本人英语空空间。管理端基于指定 go-admin/go-admin-ui 选用模块,学习端为独立 Vue 3 + TypeScript + Vite 工程,共用 Go 后端和 MySQL 8.4.3。#18 登录日志与操作审计已通过用户验收,支持管理员查询和 90 天保留清理。阅读、导入、词典与复习尚未实现。MVP 定位为“支持多账号、数据独立的自托管学习工具”,先邀请少量用户使用;F01~F12 已确认,X 系列后置。 - [文档入口](docs/README.md) · [线上 Wiki](https://git.ilapage.cn/OPC/lexgo/wiki/Home) - [项目档案](docs/00-project-profile.md) · [需求总览](docs/09-product-requirements-overview.md) -- [工作量估算](docs/10-workload-estimate.md):F01~F12 剩余产品工作量初估 52~84 人日,技术验证后重估;旧全量研究仅供参考。 +- [工作量估算](docs/10-workload-estimate.md):#2 验收后原范围剩余 46~75 人日,新增 #18 的 3~5 人日计划后为 49~80 人日;技术验证后重估,旧全量研究仅供参考。 - [四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16):14 张单元工单,工程基础 → 技术验证 → 首条学习闭环 → 补齐 MVP;原型 v1 已获用户验收。两端使用账号(用户名)+密码登录,不要求邮箱。 - [原型工单 #1](https://git.ilapage.cn/OPC/lexgo/issues/1):Quant-UX 桌面/手机原型 v1,预览入口与审核记录见工单及需求总览。 - [工作流](docs/01-workflow.md) · [开发与验证](docs/04-local-development-and-verification.md) @@ -17,6 +17,8 @@ 本地入口:学习端 http://127.0.0.1:5173,管理端 http://127.0.0.1:5174。完整安装与测试命令见[开发与验证](docs/04-local-development-and-verification.md)。账号使用用户名,无需邮箱;没有随代码交付的默认密码。 +本机三个服务已由 `D:/supervisord/supervisord.conf` 中的 lexgo-learner、lexgo-admin、lexgo-api 托管,使用时不要重复手动启动同端口。当前 schema v2;从 #2 升级时停止 API,执行 build、migrate 后再启动。审计过期清理可执行 `python scripts/server.py audit-cleanup`,只影响超过 90 天的日志。 + ## 文档与治理 远端为 `https://git.ilapage.cn/OPC/lexgo.git`,默认分支 main。需要工单的任务以 Gitea 工单为单次任务事实来源;轻量直接实施项通过 Git 提交和结果报告留痕。长期规则以 Wiki 为主源,docs 为单向镜像;默认不创建任务归档。 diff --git a/admin/src/audit-logs.mjs b/admin/src/audit-logs.mjs new file mode 100644 index 0000000..faa9b71 --- /dev/null +++ b/admin/src/audit-logs.mjs @@ -0,0 +1,56 @@ +export const actionLabels = { create: '创建账号', enable: '启用账号', disable: '停用账号', reset_password: '重置密码', update: '更新账号' } +export const reasonLabels = { ok: '完成', invalid_input: '输入无效', invalid_credentials: '账号或密码不正确', rate_limited: '请求过于频繁', forbidden: '无操作权限', not_found: '账号不存在', conflict: '账号冲突', internal_error: '服务异常' } + +export function auditQuery(kind, filters = {}) { + if (!['login', 'operation'].includes(kind)) throw new Error('日志类型无效') + const { page = 1, limit = 20, result, action, from, to } = filters + if (!Number.isInteger(page) || page < 1 || !Number.isInteger(limit) || limit < 1 || limit > 100) throw new Error('分页参数无效') + const params = new URLSearchParams({ page, limit }) + const username = (filters.username || '').trim().toLowerCase() + if (username) params.set('username', username) + if (result) { + if (!['success', 'failure'].includes(result)) throw new Error('结果筛选无效') + params.set('result', result) + } + if (action) { + if (kind !== 'operation' || !Object.hasOwn(actionLabels, action)) throw new Error('操作筛选无效') + params.set('action', action) + } + for (const [key, value] of [['from', from], ['to', to]]) { + if (!value) continue + const date = new Date(value) + if (Number.isNaN(date.getTime())) throw new Error('请选择有效时间') + params.set(key, date.toISOString()) + } + if (from && to && new Date(from) > new Date(to)) throw new Error('开始时间不能晚于结束时间') + return params.toString() +} + +// Every page/filter change and session invalidation retires pending responses. +export function createAuditLogLoader(session, state) { + let sequence = 0 + function invalidate() { + sequence++ + Object.assign(state, { items: [], total: 0, loading: false, error: '' }) + } + return { + invalidate, + async load(kind, filters) { + // Keep the page count stable while fetching another page; resetting total + // makes Element Plus clamp the current page to one and issue a new query. + sequence++ + Object.assign(state, { items: [], loading: true, error: '' }) + const current = sequence + const generation = session.state.generation + const isCurrent = () => current === sequence && generation === session.state.generation + try { + const data = await session.queryAuditLogs(kind, filters) + if (isCurrent()) Object.assign(state, { items: data.items, total: data.total }) + } catch (error) { + if (isCurrent()) state.error = error.message + } finally { + if (current === sequence) state.loading = false + } + } + } +} diff --git a/admin/src/layout/index.vue b/admin/src/layout/index.vue index b5bf533..a1600e6 100644 --- a/admin/src/layout/index.vue +++ b/admin/src/layout/index.vue @@ -3,14 +3,16 @@
diff --git a/admin/src/router/index.js b/admin/src/router/index.js index b3a0817..4fec950 100644 --- a/admin/src/router/index.js +++ b/admin/src/router/index.js @@ -3,9 +3,15 @@ import { session } from '../store' import Layout from '../layout/index.vue' import Login from '../views/Login.vue' import Accounts from '../views/Accounts.vue' +import AuditLogs from '../views/AuditLogs.vue' const router = createRouter({ history: createWebHashHistory(), routes: [ { path: '/login', component: Login }, - { path: '/', component: Layout, children: [{ path: '', redirect: '/accounts' }, { path: 'accounts', component: Accounts }] }, + { path: '/', component: Layout, children: [ + { path: '', redirect: '/accounts' }, + { path: 'accounts', component: Accounts, meta: { title: '账号管理' } }, + { path: 'login-logs', component: AuditLogs, props: { kind: 'login' }, meta: { title: '登录日志' } }, + { path: 'operation-logs', component: AuditLogs, props: { kind: 'operation' }, meta: { title: '操作日志' } } + ] }, { path: '/:pathMatch(.*)*', redirect: '/accounts' } ] }) router.beforeEach(async to => { diff --git a/admin/src/session.mjs b/admin/src/session.mjs index 5bd0a2e..e560654 100644 --- a/admin/src/session.mjs +++ b/admin/src/session.mjs @@ -1,3 +1,4 @@ +import { auditQuery } from './audit-logs.mjs' export function normalizeUsername(value) { return value.trim().toLowerCase() } export function validPassword(value) { const bytes = new TextEncoder().encode(value).length @@ -83,6 +84,14 @@ export function createSession({ fetch, storage, changed = () => {} }) { } catch (error) { if (generation === state.generation) clear(); throw error } }, async logout() { const token = state.token; clear(); await revoke(token) }, + async queryAuditLogs(kind, filters) { + authorized() + const generation = state.generation + const query = auditQuery(kind, filters) + const data = await request('/' + kind + '-logs?' + query) + assertCurrent(generation) + return data + }, async loadAccounts() { authorized() const generation = state.generation diff --git a/admin/src/views/AuditLogs.vue b/admin/src/views/AuditLogs.vue new file mode 100644 index 0000000..e9b9a88 --- /dev/null +++ b/admin/src/views/AuditLogs.vue @@ -0,0 +1,64 @@ + + + diff --git a/admin/tests/audit-logs.test.mjs b/admin/tests/audit-logs.test.mjs new file mode 100644 index 0000000..e44b5d8 --- /dev/null +++ b/admin/tests/audit-logs.test.mjs @@ -0,0 +1,104 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { createSession } from '../src/session.mjs' +import * as audit from '../src/audit-logs.mjs' + +function setup() { + const pending = [] + const session = createSession({ + storage: { getItem() {}, setItem() {}, removeItem() {} }, + fetch: (url, options) => new Promise(resolve => pending.push({ url, options, resolve })) + }) + session.state.user = { id: 1, role: 'admin', username: 'fixture.admin' } + session.state.token = 'fictional-token' + return { session, pending } +} +const response = data => ({ ok: true, status: 200, json: async () => ({ code: 200, data }) }) + +test('unclassified rejected account updates have a readable action and filter', () => { + assert.equal(audit.actionLabels.update, '更新账号') + assert.equal(new URLSearchParams(audit.auditQuery('operation', { action: 'update' })).get('action'), 'update') +}) + +test('audit query normalizes exact username, preserves RFC3339 bounds and bounds pagination', async () => { + const { session, pending } = setup() + assert.equal(typeof session.queryAuditLogs, 'function') + const request = session.queryAuditLogs('operation', { username: ' Fixture.Admin ', result: 'failure', action: 'reset_password', page: 2, limit: 100, from: new Date('2026-09-10T00:00:00+08:00'), to: new Date('2026-09-11T00:00:00+08:00') }) + const url = new URL(pending[0].url, 'https://fixture.invalid') + assert.equal(url.pathname, '/api/v1/operation-logs') + assert.deepEqual(Object.fromEntries(url.searchParams), { page: '2', limit: '100', username: 'fixture.admin', result: 'failure', action: 'reset_password', from: '2026-09-09T16:00:00.000Z', to: '2026-09-10T16:00:00.000Z' }) + assert.equal(pending[0].options.cache, 'no-store') + pending[0].resolve(response({ items: [], total: 0, page: 2, limit: 100 })) + assert.equal((await request).total, 0) + for (const filter of [{ limit: 101 }, { page: 0 }, { result: 'unknown' }, { action: 'delete' }, { from: 'bad' }, { from: '2026-09-11', to: '2026-09-10' }]) { + await assert.rejects(session.queryAuditLogs('operation', filter)) + } + assert.equal(pending.length, 1) +}) + +test('login query omits empty filters and rejects operation-only actions and unauthenticated access', async () => { + const { session, pending } = setup() + assert.equal(typeof session.queryAuditLogs, 'function') + const request = session.queryAuditLogs('login', { username: ' ', result: '', from: null, to: null }) + assert.equal(pending[0].url, '/api/v1/login-logs?page=1&limit=20') + pending[0].resolve(response({ items: [], total: 0, page: 1, limit: 20 })) + await request + await assert.rejects(session.queryAuditLogs('login', { action: 'create' })) + session.clear() + await assert.rejects(session.queryAuditLogs('login'), /请先登录/) + assert.equal(pending.length, 1) +}) + +test('late audit response cannot survive session change', async () => { + const { session, pending } = setup() + assert.equal(typeof session.queryAuditLogs, 'function') + const request = session.queryAuditLogs('login') + session.clear() + session.state.user = { id: 3, role: 'admin', username: 'fixture.otheradmin' } + pending[0].resolve(response({ items: [{ username: 'fixture.private' }], total: 1 })) + await assert.rejects(request, /会话已变化/) +}) + +test('route/filter changes discard out-of-order rows and invalidation clears loaded private data', async () => { + assert.equal(typeof audit.createAuditLogLoader, 'function') + const { session, pending } = setup() + const state = { items: [], total: 0, loading: false, error: '' } + const loader = audit.createAuditLogLoader(session, state) + const old = loader.load('login', {}) + const current = loader.load('operation', {}) + pending[1].resolve(response({ items: [{ id: 2 }], total: 1 })) + await current + pending[0].resolve(response({ items: [{ id: 1 }], total: 1 })) + await old + assert.deepEqual(state.items, [{ id: 2 }]) + loader.invalidate() + assert.deepEqual(state.items, []) + assert.equal(state.total, 0) + const late = loader.load('login', {}) + session.clear() + pending[2].resolve(response({ items: [{ id: 3 }], total: 1 })) + await late + assert.deepEqual(state.items, []) + assert.equal(state.error, '') +}) + +test('loading page two preserves total so pagination cannot clamp the page back to one', async () => { + const { session, pending } = setup() + const state = { items: [], total: 0, loading: false, error: '' } + const loader = audit.createAuditLogLoader(session, state) + const first = loader.load('login', { page: 1 }) + pending[0].resolve(response({ items: [{ id: 50 }], total: 50 })) + await first + const second = loader.load('login', { page: 2 }) + assert.equal(state.total, 50) + assert.equal(state.loading, true) + assert.deepEqual(state.items, []) + assert.equal(new URL(pending[1].url, 'https://fixture.invalid').searchParams.get('page'), '2') + pending[1].resolve(response({ items: [{ id: 30 }], total: 50 })) + await second + assert.deepEqual(state.items, [{ id: 30 }]) + assert.equal(state.total, 50) + loader.invalidate() + assert.equal(state.total, 0) + assert.deepEqual(state.items, []) +}) diff --git a/docs/00-project-profile.md b/docs/00-project-profile.md index fa6c108..1433da3 100644 --- a/docs/00-project-profile.md +++ b/docs/00-project-profile.md @@ -2,8 +2,8 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件) wiki_page: Project-Profile wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Project-Profile.- -wiki_revision: 80cd91be7e64552b0205f45c59a2234934760b0e -synchronized_at: 2026-09-10T09:38:10Z +wiki_revision: 40ae5e3a0521195da4f1a9cd9f42beec15da20d4 +synchronized_at: 2026-09-10T12:43:19Z # LexGo 项目档案 @@ -106,7 +106,7 @@ M0 在 LexGo 隔离工程中验证:指定前后端配套构建;MySQL 8 迁 ## MVP 功能范围确认 -2026-09-10 用户确认 F01~F12 全部进入首版,X01~X14 暂不纳入;随后授权使用 Quant-UX 制作原型,并同意先建原型工单,原型审核后再拆实施工单。桌面和手机均覆盖;默认学习语言已确认英语。工程基础已实现待验收,其余范围沿用已验收原型。 +2026-09-10 用户确认 F01~F12 全部进入首版,X01~X14 暂不纳入;随后授权使用 Quant-UX 制作原型,并同意先建原型工单,原型审核后再拆实施工单。桌面和手机均覆盖;默认学习语言已确认英语。工程基础已通过验收,其余范围沿用已验收原型。 原型站点为 https://qux.ilapage.cn/;v1 编辑源和预览统一见需求总览及工单 #1。用户要求减少页面说明、按目标页面呈现。账号配置不进入仓库。 @@ -119,7 +119,7 @@ M0 在 LexGo 隔离工程中验证:指定前后端配套构建;MySQL 8 迁 学习端与管理端均使用账号(用户名)+密码登录,账号不要求邮箱格式;邮箱不作为必填登录标识。两端共用账号体系,管理权限由服务端单独校验。用户名唯一性和大小写规则在账号工单中细化。 -[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16) 已建立四阶段、14 张单元工单;详见需求总览与工作量估算。工程基础 #2 已实现并完成本地验证,等待用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。 +[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16) 已建立四阶段、14 张单元工单;详见需求总览与工作量估算。工程基础 #2 已于 2026-09-10 通过用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。 ## 工程基础实际基线(#2,2026-09-10) @@ -131,3 +131,8 @@ Go 1.26.5 通过 Go 独立工具链下载并校验,系统 Go 1.24.0 保持不 server 是 go-admin 的选用模块接入:原样保留 SysUser、SysDept、必要 common/models 和 MIT 许可,新增 LexGo 路由、账号服务、会话与显式迁移;未接入整套后台路由、代码生成或任意任务执行。server/upstream.json 记录来源和文件摘要。admin/upstream.json 区分原样复用与适配的 go-admin-ui 布局、组件、页面和依赖,原始三个 go-admin 源工作区保持只读。 当前采用随机 Bearer 会话而非直接启用上游开发模式的超长 JWT。已验证退出、禁用、密码重置和到期拒绝;该差异是为满足当前可撤销会话要求,不宣称上游默认实现满足此要求。 + + +## 日志审计基线(#18) + +管理端新增两个审计列表,后端为 LexGo 自有日志模型与接口,参照 go-admin 模块布局但不复制其原始参数/响应持久化逻辑。schema v2 新增 lexgo_login_logs、lexgo_operation_logs;保留 90 天,启动和每小时分批清理,也可显式 audit-cleanup。新建/重置密码均为 6~72 UTF-8 字节;初始 bootstrap 保留 10 字节下限。#2 已验收,#18 已验收;默认模块中的其他候选未纳入。 diff --git a/docs/02-architecture-and-code-map.md b/docs/02-architecture-and-code-map.md index ad6a484..9db749a 100644 --- a/docs/02-architecture-and-code-map.md +++ b/docs/02-architecture-and-code-map.md @@ -2,8 +2,8 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件) wiki_page: Architecture-and-Code-Map wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Architecture-and-Code-Map.- -wiki_revision: 51d40a6d4f7d0bcc0882bcd7d9c544aae964afa8 -synchronized_at: 2026-09-10T09:38:13Z +wiki_revision: 3b6f6db7f0fe1019b5811f70af8d6174cec0b950 +synchronized_at: 2026-09-10T12:36:57Z # 架构与代码地图 @@ -122,3 +122,12 @@ flowchart TD schema v1:sys_user 保留选用 go-admin 模型字段,唯一小写用户名;lexgo_spaces 以 owner_id 为主键;lexgo_sessions 保存 token_hash/owner_id/expires_at;lexgo_schema 记录版本与产品所有权。服务启动不自动迁移。迁移仅接受空库或合法已有 LexGo marker,拒绝空 marker、其他产品、负版本与未来版本;版本 0 可重试部分迁移,版本 1 幂等。 凭据字段只存在本地环境和必要数据库哈希中。后台账号密码更新使用表/字段更新,避免上游 BeforeUpdate hook 对已有哈希再次加密。账号行再会话行的锁顺序用于串行化撤销与请求;API 在事务提交后才返回成功。 + + +## 登录与操作审计(#18) + +server/app/lexgo/audit.go 定义两类白名单字段日志、筛选分页、失败记录和过期清理;router.go 在登录/账号操作边界接入。登录成功时会话与日志同一事务;账号操作成功时业务写入与日志同一事务;失败时先回滚业务,再以独立、有 3 秒超时的事务写失败记录。日志写入失败返回通用 500,不输出数据库原始错误或凭据。 + +database.go 显式迁移至 v2,两张新增表均以 created_at/id 建立排序清理索引,账号字段建查询索引,无业务表级联删除。cmd/lexgo/main.go 的服务进程在启动和每小时执行审计清理,每次最多运行一分钟、每批删除 1000 条,仅影响过期审计记录。 + +admin/src/views/AuditLogs.vue 通过 kind 复用登录/操作列表;audit-logs.mjs 负责筛选编码和请求序号,session.mjs 继续进行管理员及会话 generation 校验。切换页面/账号清空日志,普通翻页保留总数,防止分页组件跳回第一页。菜单与标题按当前路由显示。 diff --git a/docs/03-business-rules-and-glossary.md b/docs/03-business-rules-and-glossary.md index dc7cf03..2526f7d 100644 --- a/docs/03-business-rules-and-glossary.md +++ b/docs/03-business-rules-and-glossary.md @@ -2,8 +2,8 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件) wiki_page: Business-Rules-and-Glossary wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Business-Rules-and-Glossary.- -wiki_revision: c6044b9b3f863c90363563e079dff3e3fcb89bd8 -synchronized_at: 2026-09-10T12:13:06Z +wiki_revision: 7061627fed9b5acd74fbd2369f27acadc0fd2e2a +synchronized_at: 2026-09-10T12:36:59Z # 业务规则与术语 @@ -71,3 +71,12 @@ M0 固定首发语言语料、词条身份规则、短语选择与重叠规则 用户进一步确认重置密码下限也为 6 字节。新增和重置共用同一密码校验,前后端均为 6~72 UTF-8 字节;重置密码事务仍撤销该用户全部旧会话。首次管理员 bootstrap 的 10 字节下限保持原样。 + + +## 审计记录、查询和保留规则 + +- 登录日志保存时间、规范用户名、成功/失败、固定原因码和连接来源 IP。覆盖成功、凭据错误/禁用、非法请求、限流;非法用户名不保存原文。未进行地理定位或保存 User-Agent。 +- 操作日志保存管理员编号/账号、目标编号/账号、动作、结果、固定原因码、时间。动作 create、enable、disable、reset_password;无法解析明确动作的非法 PATCH 记 update。一次同时修改状态和密码生成两条动作日志。只有已认证管理员发起的管理操作进入此表;未认证或学习者的越权查询不记录原始身份输入。 +- 日志绝不保存密码、token、Cookie、请求/响应正文、错误堆栈或私人学习内容。合法账号、IP 属于本功能必要的审计数据,仅管理员可查询。 +- GET /api/v1/login-logs 与 /operation-logs:未登录 401,学习者 403;page 默认 1,limit 默认 20、最大 100。username 精确匹配;操作日志匹配操作人或目标账号。result 为 success/failure,action 限定枚举,from/to 为 RFC3339。返回 data.items/total/page/limit,按时间和编号倒序,时间按毫秒存储、浏览器按本地时区展示。 +- 固定保留最近 90 天,查询即排除过期记录;默认起止为保留边界和当前时间。清理只删除 created_at 严格早于边界的两表记录,不影响账号、空间、会话。无清空全部或导出按钮;本期不开放保留时长配置。 diff --git a/docs/04-local-development-and-verification.md b/docs/04-local-development-and-verification.md index 15ed262..cee64c4 100644 --- a/docs/04-local-development-and-verification.md +++ b/docs/04-local-development-and-verification.md @@ -2,8 +2,8 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件) wiki_page: Local-Development-and-Verification wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Local-Development-and-Verification.- -wiki_revision: a12e404e317f2f106936b99147f993d7cc6da608 -synchronized_at: 2026-09-10T12:13:08Z +wiki_revision: 8857cf2545a3de6f1efa07d88b920a8301080e56 +synchronized_at: 2026-09-10T12:43:25Z # 本地开发与验证 @@ -176,3 +176,19 @@ supervisor 直接管理编译后的 Go 进程,运行时不调用 Python。数 重置密码下限已同步为 6 字节。管理端测试与真实 MySQL 集成测试覆盖 5/6/72/73 字节及多字节字符,验证新密码登录成功、旧密码和旧会话被拒绝。管理员页面的新增/重置提示均为 6–72 字节;本次不修改现有账号密码。 + + +## 审计模块启动与验证(#18 / schema v2) + +首次更新此版本时先在 supervisor 停止 lexgo-api,再从仓库根目录执行 python scripts/server.py build、python scripts/server.py migrate,随后启动 lexgo-api。不要在 Windows 中覆盖正在运行的 exe。新服务要求 schema v2,启动本身不迁移;旧 v1 的账号、学习空间、会话完整保留。 + +本机已执行 v1→v2 并验证 sys_user 4 行、lexgo_spaces 4 行、lexgo_sessions 2 行迁移前后的整体指纹相同。原二进制仅在忽略目录 .local/lexgo-pre-audit.exe 留作本机回退,不提交 Git。后续正常启动仍通过 supervisor;登录日志入口 http://127.0.0.1:5174/#/login-logs,操作日志入口 http://127.0.0.1:5174/#/operation-logs。 + +手动清理过期审计记录:python scripts/server.py audit-cleanup;仅删除两张审计表中超过 90 天的记录,执行前确认本机 LEXGO_DB_NAME。后台自动清理在服务启动时及每小时运行,每次限制一分钟,失败只记录固定提示并于下一小时重试。 + +回退到已验收 #2 时:停止 API,确认没有其他 v2 实例,再恢复 .local/lexgo-pre-audit.exe;将 lexgo_schema 中 id=1、product=lexgo 的版本从 2 改回 1 后启动旧程序。两张日志表保留,不删除,旧程序不继续记录新日志;重新升级时显式 migrate 恢复 v2。本次已验证增量/重复迁移及业务数据保留,没有在开发库演练回退。 + +后端 MySQL 集成测试覆盖审计权限、查询边界、创建/启停/重置审计、无敏感输入、事务回滚、90 天边界、增量与重复迁移;管理端 24 项测试、lint 和生产构建通过。浏览器已展示真实登录与重复创建失败记录。为翻页检查生成了 21 条 fixture.audit 虚构账号登录失败记录,不创建该账号;记录按相同 90 天规则保留。浏览器自动化点击未触发页面事件,用户本次已确认日志模块验收通过;自动化点击检查仍保留原始未通过记录。 + + +#18 用户验收:2026-09-10T20:40:04+08:00 用户确认日志通过验收(工单评论 7591),包含此前待人工检查的交互。未重新运行自动化测试,未更改其历史结果,未合并 PR 或发布生产。 diff --git a/docs/09-product-requirements-overview.md b/docs/09-product-requirements-overview.md index 958e4dd..5abbd45 100644 --- a/docs/09-product-requirements-overview.md +++ b/docs/09-product-requirements-overview.md @@ -2,8 +2,8 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件) wiki_page: Product-Requirements-Overview wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Product-Requirements-Overview.- -wiki_revision: e506edcf2487b3d0faf035bf0a6826332bccd994 -synchronized_at: 2026-09-10T09:38:24Z +wiki_revision: db82eeb1d509b8d3c4a2204143cfb0e829b6b083 +synchronized_at: 2026-09-10T12:43:32Z # 产品需求总览 @@ -15,7 +15,7 @@ synchronized_at: 2026-09-10T09:38:24Z ## 事实来源边界 -当前产品实现:账号与独立空间工程基础已实现待验收,其余学习业务待实施。用户已确认多账号、数据独立、自托管和 F01~F12;已建立原型工单 #1 及 Quant-UX 桌面/手机设计。下方 U/A/N 与 M0~M4 保留为研究索引,不能扩大已批准 MVP,也不能用原型完成度代替产品实现进度。 +当前产品实现:账号与独立空间工程基础已通过验收,其余学习业务待实施。用户已确认多账号、数据独立、自托管和 F01~F12;已建立原型工单 #1 及 Quant-UX 桌面/手机设计。下方 U/A/N 与 M0~M4 保留为研究索引,不能扩大已批准 MVP,也不能用原型完成度代替产品实现进度。 ## 当前需求索引 @@ -193,7 +193,7 @@ Anki(U20)、YouTube/Jellyfin 远程字幕(U07/A08)、全量旧实例迁 学习端与管理端均使用账号(用户名)+密码登录,账号不要求邮箱格式;邮箱不作为必填登录标识。两端共用账号体系,管理权限由服务端单独校验。用户名唯一性和大小写规则在账号工单中细化。 -[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16) 汇总四阶段与依赖,各单均包含完整路径、参考模块、设计证据、验收和人日估算。工程基础 #2 已实现并完成本地验证,等待用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。 +[四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16) 汇总四阶段与依赖,各单均包含完整路径、参考模块、设计证据、验收和人日估算。工程基础 #2 已于 2026-09-10 通过用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。 | 阶段 | 工单 | 覆盖 | 预计人日 | |---|---|---|---:| @@ -215,6 +215,11 @@ Anki(U20)、YouTube/Jellyfin 远程字幕(U07/A08)、全量旧实例迁 ## 工程基础交付边界(#2) -工程基础 #2 已实现并完成本地验证,等待用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。 +工程基础 #2 已于 2026-09-10 通过用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。 用户名登录与账号管理覆盖 B01/B02 和 F12 的登录基础;所有 F 学习业务仍需按 #3~#15 验收。#2 的会话和权限测试不能证明尚未存在的书籍/词汇/复习权限已通过。 + + +## 试用前管理能力增补 + +用户于 2026-09-10 验收 #2,并批准新增 #18 登录日志与操作审计:两个管理员列表、查询筛选分页、必要字段记录、权限及 90 天保留清理。沿用现有 go-admin 管理布局,无需独立高保真原型。已通过用户验收,纳入 #16“邀请用户试用前完成”;不替代 #3 英语分词和 #4 阅读选择验证,也不引入其他 go-admin 默认模块。 diff --git a/docs/10-workload-estimate.md b/docs/10-workload-estimate.md index 50a4fa6..1c20c1d 100644 --- a/docs/10-workload-estimate.md +++ b/docs/10-workload-estimate.md @@ -2,8 +2,8 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件) wiki_page: Workload-Estimate wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Workload-Estimate.- -wiki_revision: de8fcf81b07bd7e96df1c68e5a63914139c62b6c -synchronized_at: 2026-09-10T09:38:31Z +wiki_revision: ba995e6788f52fc6f3471d5cf0db7300b9fbc3b4 +synchronized_at: 2026-09-10T12:43:39Z # LexGo MVP 工作量估算 @@ -152,3 +152,11 @@ Quant-UX 桌面和手机原型 v1 已建立,待人工审核;这不计为产 ## 工程基础进展(#2) 账号与独立空间基础已实现并本地验证,待用户验收。原计划 #2 为 6~9 人日,这不是实际消耗;未记录人工有效工时,不按 AI 会话耗时反推。#2 验收后,原拆分中 #3~#15 的规划工作量合计 46~75 人日,仍需在英语资源/划词验证后重估。本页顶部 52~84 为本轮开始时的完整 MVP 基线,不重复计入新增工作。 + + +## #2 验收与 #18 增补后的当前估算 + +#2 已验收,原 52~84 人日 MVP 计划扣除基础单元 6~9 人日后,原范围剩余 46~75 人日。新增 #18 日志审计独立计划 3~5 人日,扩展后全范围原始预算 55~89 人日、#18 验收前的剩余计划参考 49~80 人日。#18 已通过用户验收,本次验收后剩余规划为 46~75 人日。人日是规划参考,不把本次代理运行时间折算为已消耗人日。 + + +#18 已于 2026-09-10T20:40:04+08:00 验收。#2 与 #18 完成后,剩余 #3~#15 的规划参考为 46~75 人日;历史 49~80 人日是 #18 尚未验收时的区间。 diff --git a/docs/README.md b/docs/README.md index ea11d60..ad4d514 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,8 +2,8 @@ generated: true (请先修改 Gitea Wiki,禁止直接编辑本文件) wiki_page: Home wiki_url: https://git.ilapage.cn/OPC/lexgo/wiki/Home -wiki_revision: 5ff3348f14ba1ffa810be3996b9340526a689529 -synchronized_at: 2026-09-10T09:38:08Z +wiki_revision: d113bfc4d8d33feced5b84f098a5c50ba51df0ba +synchronized_at: 2026-09-10T12:43:18Z # LexGo 文档入口 @@ -27,7 +27,7 @@ synchronized_at: 2026-09-10T09:38:08Z ## 事实来源 -Wiki 保存长期规则,工单保存单次实施与验收,Git 保存源码和镜像。当前工程基础已实现待验收,已有调研不等于运行验收。数据库选择以用户确认的 MySQL 8 为准,历史 PostgreSQL 建议不再适用。 +Wiki 保存长期规则,工单保存单次实施与验收,Git 保存源码和镜像。当前工程基础已通过验收,已有调研不等于运行验收。数据库选择以用户确认的 MySQL 8 为准,历史 PostgreSQL 建议不再适用。 原有四份调研资料作为证据页面保留:[需求提取](https://git.ilapage.cn/OPC/lexgo/wiki/LinguaCafe-Requirements.-)、[Go 分析](https://git.ilapage.cn/OPC/lexgo/wiki/Go-Architecture-Analysis.-)、[另一份需求提取](https://git.ilapage.cn/OPC/lexgo/wiki/LinguaCafe-Requirements-Alternative.-)、[另一份 Go 分析](https://git.ilapage.cn/OPC/lexgo/wiki/Go-Analysis-Alternative.-)。其正文为既有调研记录,不将整份建议视为已批准方案。 @@ -43,9 +43,17 @@ F01~F12 已获用户确认,X 系列暂不纳入。Quant-UX 桌面与手机 原型制作进度与证据见[原型工单 #1](https://git.ilapage.cn/OPC/lexgo/issues/1)。 -Quant-UX 原型 v1 已通过用户验收。[桌面预览](https://qux.ilapage.cn/#/simulate.html?h=a2aa10as5WpKpxe9fjDXETLqmigWeGWqsSjGmNmERlfgTpjXambHTDwagN8C&s=true&log=false)、[手机预览](https://qux.ilapage.cn/#/simulate.html?h=a2aa10aFDLzpVy0xPE70ftx5adTPyjWy5bj6nhPk2kDkJEwtIf1GiZjqx1n6&s=true&log=false)。工程基础 #2 已实现并完成本地验证,等待用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。 +Quant-UX 原型 v1 已通过用户验收。[桌面预览](https://qux.ilapage.cn/#/simulate.html?h=a2aa10as5WpKpxe9fjDXETLqmigWeGWqsSjGmNmERlfgTpjXambHTDwagN8C&s=true&log=false)、[手机预览](https://qux.ilapage.cn/#/simulate.html?h=a2aa10aFDLzpVy0xPE70ftx5adTPyjWy5bj6nhPk2kDkJEwtIf1GiZjqx1n6&s=true&log=false)。工程基础 #2 已于 2026-09-10 通过用户验收:两端用户名登录、管理员创建/启停/重置学习账号、可撤销会话和本人英语空空间。阅读、导入、词典和复习尚未实现。代码在 feat/2-account-foundation 分支。 ## 当前实施入口 [四阶段实施总览 #16](https://git.ilapage.cn/OPC/lexgo/issues/16)。14 张单元工单,预计 52~84 人日;实际产品实现未开始。学习端与管理端均使用账号(用户名)+密码登录,不要求邮箱。详见需求总览和工作量表。 + + +## 当前管理端交付 + +#2 已验收并关闭(记录 7576)。新增 [#18 登录日志与操作审计](https://git.ilapage.cn/OPC/lexgo/issues/18),已通过用户验收,列入 #16 的试用前准备。管理端提供登录日志、操作日志查询,默认保留 90 天;数据库为显式迁移后的 schema v2。代码分支 feat/18-audit-logs,基于已验收的 feat/2-account-foundation;#17 尚未合并,因此新 PR 以该基础分支为目标,保持差异只含 #18。 + + +日志审计 #18 于 2026-09-10T20:40:04+08:00 获用户验收并关闭;#16 已更新完成索引。下一阶段 #3/#4 尚未开始。 diff --git a/scripts/server.py b/scripts/server.py index 5f297a3..49b0c7d 100644 --- a/scripts/server.py +++ b/scripts/server.py @@ -12,12 +12,13 @@ def main(): "migrate": ["go", "run", "./cmd/lexgo", "migrate"], "bootstrap": ["go", "run", "./cmd/lexgo", "bootstrap"], "serve": ["go", "run", "./cmd/lexgo", "serve"], + "audit-cleanup": ["go", "run", "./cmd/lexgo", "audit-cleanup"], "build": ["go", "build", "./cmd/lexgo"], "test": ["go", "test", "./...", "-count=1"], "test-integration": ["go", "test", "./...", "-count=1", "-v"], } if len(sys.argv) != 2 or sys.argv[1] not in commands: - raise SystemExit("Usage: python scripts/server.py migrate|bootstrap|serve|build|test|test-integration") + raise SystemExit("Usage: python scripts/server.py migrate|bootstrap|serve|audit-cleanup|build|test|test-integration") env = dict(os.environ) config = ROOT / ".env.local" if config.exists(): diff --git a/server/app/lexgo/audit.go b/server/app/lexgo/audit.go new file mode 100644 index 0000000..c8ef939 --- /dev/null +++ b/server/app/lexgo/audit.go @@ -0,0 +1,243 @@ +package lexgo + +import ( + "context" + "errors" + "net" + "net/url" + "strconv" + "strings" + "time" + + "github.com/gin-gonic/gin" + "gorm.io/gorm" +) + +const AuditRetention = 90 * 24 * time.Hour + +type LoginLog struct { + ID uint64 `json:"id" gorm:"primaryKey"` + Username string `json:"username"` + Result string `json:"result"` + Reason string `json:"reason"` + IP string `json:"ip" gorm:"column:ip"` + CreatedAt time.Time `json:"createdAt"` +} + +func (LoginLog) TableName() string { return "lexgo_login_logs" } + +type OperationLog struct { + ID uint64 `json:"id" gorm:"primaryKey"` + ActorID int `json:"actorId"` + ActorUsername string `json:"actorUsername"` + TargetID int `json:"targetId"` + TargetUsername string `json:"targetUsername"` + Action string `json:"action"` + Result string `json:"result"` + Reason string `json:"reason"` + CreatedAt time.Time `json:"createdAt"` +} + +func (OperationLog) TableName() string { return "lexgo_operation_logs" } +func safeAuditUsername(value string) string { + value = strings.ToLower(strings.TrimSpace(value)) + if !usernamePattern.MatchString(value) { + return "" + } + return value +} +func auditReason(err error) string { + if err == nil { + return "ok" + } + var e *apiError + if errors.As(err, &e) { + switch e.status { + case 400: + return "invalid_input" + case 401: + return "invalid_credentials" + case 403: + return "forbidden" + case 404: + return "not_found" + case 409: + return "conflict" + case 429: + return "rate_limited" + } + } + return "internal_error" +} +func loginAudit(c *gin.Context, username string, at time.Time, err error) LoginLog { + ip := net.ParseIP(c.ClientIP()) + value := "" + if ip != nil { + value = ip.String() + } + result := "success" + if err != nil { + result = "failure" + } + return LoginLog{Username: safeAuditUsername(username), Result: result, Reason: auditReason(err), IP: value, CreatedAt: at.UTC().Truncate(time.Millisecond)} +} + +// A bounded independent context records a failure after the business transaction +// rolls back, including when the caller disconnects. Only allowlisted fields exist. +func saveFailedAudit(db *gorm.DB, value any) error { + if db == nil { + return nil + } // Router(nil) is used only by input-shape unit tests. + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + return db.WithContext(ctx).Create(value).Error +} + +func saveOperationAudits(db *gorm.DB, c *gin.Context, entry *OperationLog) error { + actions := []string{entry.Action} + if v, ok := c.Get("operationActions"); ok { + actions = v.([]string) + } + for _, action := range actions { + row := *entry + row.ID = 0 + row.Action = action + if err := db.Create(&row).Error; err != nil { + return err + } + } + return nil +} +func saveOperationFailure(db *gorm.DB, c *gin.Context, entry *OperationLog) error { + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + return db.WithContext(ctx).Transaction(func(tx *gorm.DB) error { return saveOperationAudits(tx, c, entry) }) +} + +type auditFilter struct { + Page, Limit int + Username, Result, Action string + From, To time.Time +} + +func parseAuditFilter(q url.Values, operation bool, now time.Time) (auditFilter, error) { + f := auditFilter{Page: 1, Limit: 20, From: now.UTC().Add(-AuditRetention), To: now.UTC()} + bad := failure(400, "日志筛选参数无效") + for k, v := range q { + if len(v) != 1 { + return f, bad + } + switch k { + case "page", "limit", "username", "result", "from", "to": + case "action": + if !operation { + return f, bad + } + default: + return f, bad + } + } + for key, dest := range map[string]*int{"page": &f.Page, "limit": &f.Limit} { + if v := q.Get(key); v != "" { + n, e := strconv.Atoi(v) + if e != nil || n < 1 { + return f, bad + } + *dest = n + } + } + if f.Limit > 100 || f.Page > 1000000 { + return f, bad + } + if v := q.Get("username"); v != "" { + f.Username = safeAuditUsername(v) + if f.Username == "" { + return f, bad + } + } + f.Result = q.Get("result") + if f.Result != "" && f.Result != "success" && f.Result != "failure" { + return f, bad + } + f.Action = q.Get("action") + if f.Action != "" { + switch f.Action { + case "create", "enable", "disable", "reset_password", "update": + default: + return f, bad + } + } + for key, dest := range map[string]*time.Time{"from": &f.From, "to": &f.To} { + if v := q.Get(key); v != "" { + t, e := time.Parse(time.RFC3339, v) + if e != nil { + return f, bad + } + *dest = t.UTC() + } + } + if f.To.Before(f.From) { + return f, bad + } + if cutoff := now.UTC().Add(-AuditRetention); f.From.Before(cutoff) { + f.From = cutoff + } + return f, nil +} +func queryAudit(db *gorm.DB, q url.Values, operation bool, now time.Time) (any, error) { + f, err := parseAuditFilter(q, operation, now) + if err != nil { + return nil, err + } + table := "lexgo_login_logs" + if operation { + table = "lexgo_operation_logs" + } + query := db.Table(table).Where("created_at >= ? AND created_at <= ?", f.From, f.To) + if f.Username != "" { + if operation { + query = query.Where("actor_username = ? OR target_username = ?", f.Username, f.Username) + } else { + query = query.Where("username = ?", f.Username) + } + } + if f.Result != "" { + query = query.Where("result = ?", f.Result) + } + if f.Action != "" { + query = query.Where("action = ?", f.Action) + } + var total int64 + if err = query.Count(&total).Error; err != nil { + return nil, err + } + query = query.Order("created_at DESC, id DESC").Offset((f.Page - 1) * f.Limit).Limit(f.Limit) + result := gin.H{"total": total, "page": f.Page, "limit": f.Limit} + if operation { + items := []OperationLog{} + err = query.Find(&items).Error + result["items"] = items + } else { + items := []LoginLog{} + err = query.Find(&items).Error + result["items"] = items + } + return result, err +} + +// CleanupAudit removes only expired audit rows in bounded batches. No user, +// session, or learning table is touched. Cutoff is fixed for the entire run. +func CleanupAudit(ctx context.Context, db *gorm.DB, now time.Time) error { + for _, table := range []string{"lexgo_login_logs", "lexgo_operation_logs"} { + for { + result := db.WithContext(ctx).Exec("DELETE FROM "+table+" WHERE created_at < ? ORDER BY created_at LIMIT 1000", now.UTC().Add(-AuditRetention)) + if result.Error != nil { + return result.Error + } + if result.RowsAffected < 1000 { + break + } + } + } + return nil +} diff --git a/server/app/lexgo/audit_test.go b/server/app/lexgo/audit_test.go new file mode 100644 index 0000000..7a94802 --- /dev/null +++ b/server/app/lexgo/audit_test.go @@ -0,0 +1,213 @@ +package lexgo + +import ( + "context" + "encoding/json" + admin "go-admin/app/admin/models" + "strconv" + "strings" + "testing" + "time" +) + +func TestAuditRetentionAndV1Migration(t *testing.T) { + db := emptyMigrationDB(t) + if err := Migrate(db); err != nil { + t.Fatal(err) + } + u := admin.SysUser{Username: randomName("retention"), Password: fixturePassword, RoleId: 1, Status: "2"} + if err := db.Create(&u).Error; err != nil { + t.Fatal(err) + } + previousToken := loginToken(t, Router(db, time.Now), u.Username, fixturePassword) + // Reconstruct v1 in this disposable schema to exercise the incremental path. + for _, sql := range []string{"DROP TABLE lexgo_login_logs", "DROP TABLE lexgo_operation_logs", "UPDATE lexgo_schema SET version=1 WHERE id=1"} { + if err := db.Exec(sql).Error; err != nil { + t.Fatal(err) + } + } + if err := CheckSchema(db); err == nil { + t.Fatal("v1 must require explicit migration") + } + for i := 0; i < 2; i++ { + if err := Migrate(db); err != nil { + t.Fatal(err) + } + } + var count int64 + db.Table("sys_user").Where("user_id=?", u.UserId).Count(&count) + if count != 1 { + t.Fatal("migration changed account") + } + if code, _ := callAPI(t, Router(db, time.Now), "GET", "/api/v1/space", previousToken, nil); code != 200 { + t.Fatal("migration changed existing session or learning space") + } + now := time.Now().UTC().Truncate(time.Millisecond) + cutoff := now.Add(-AuditRetention) + for _, at := range []time.Time{cutoff.Add(-time.Millisecond), cutoff, now} { + if err := db.Create(&LoginLog{Username: u.Username, Result: "success", Reason: "ok", CreatedAt: at}).Error; err != nil { + t.Fatal(err) + } + if err := db.Create(&OperationLog{ActorID: u.UserId, ActorUsername: u.Username, Action: "create", Result: "success", Reason: "ok", CreatedAt: at}).Error; err != nil { + t.Fatal(err) + } + } + r := Router(db, func() time.Time { return now }) + token := loginToken(t, r, u.Username, fixturePassword) + code, data := callAPI(t, r, "GET", "/api/v1/operation-logs?limit=1", token, nil) + var page struct { + Total int64 + Items []OperationLog + } + json.Unmarshal(data, &page) + if code != 200 || page.Total != 2 || len(page.Items) != 1 || !page.Items[0].CreatedAt.Equal(now) { + t.Fatal("retention/page/order incorrect") + } + if err := CleanupAudit(context.Background(), db, now); err != nil { + t.Fatal(err) + } + for _, table := range []string{"lexgo_login_logs", "lexgo_operation_logs"} { + db.Table(table).Where("created_at < ?", cutoff).Count(&count) + if count != 0 { + t.Fatal("expired rows remain") + } + } + db.Table("lexgo_operation_logs").Count(&count) + if count != 2 { + t.Fatal("cleanup deleted boundary/current rows") + } + db.Table("sys_user").Count(&count) + if count != 1 { + t.Fatal("cleanup touched users") + } +} + +func TestAuditFailureRollsBackAccountAndSession(t *testing.T) { + db := emptyMigrationDB(t) + if err := Migrate(db); err != nil { + t.Fatal(err) + } + u := admin.SysUser{Username: randomName("atomic"), Password: fixturePassword, RoleId: 1, Status: "2"} + if err := db.Create(&u).Error; err != nil { + t.Fatal(err) + } + r := Router(db, time.Now) + token := loginToken(t, r, u.Username, fixturePassword) + if err := db.Exec("DROP TABLE lexgo_operation_logs").Error; err != nil { + t.Fatal(err) + } + name := randomName("rollback") + code, _ := callAPI(t, r, "POST", "/api/v1/accounts", token, map[string]string{"username": name, "password": fixturePassword}) + if code != 500 { + t.Fatal("missing audit store must fail operation") + } + var count int64 + db.Table("sys_user").Where("username=?", name).Count(&count) + if count != 0 { + t.Fatal("account committed without audit") + } + if err := db.Exec("DROP TABLE lexgo_login_logs").Error; err != nil { + t.Fatal(err) + } + var before int64 + db.Table("lexgo_sessions").Count(&before) + code, _ = callAPI(t, r, "POST", "/api/v1/login", "", map[string]string{"username": u.Username, "password": fixturePassword}) + db.Table("lexgo_sessions").Count(&count) + if code != 500 || count != before { + t.Fatal("session committed without login audit") + } +} + +func TestAuditQueriesAndPrivacy(t *testing.T) { + db := testDB(t) + u := admin.SysUser{Username: randomName("audit"), Password: fixturePassword, RoleId: 1, Status: "2"} + if err := db.Create(&u).Error; err != nil { + t.Fatal(err) + } + now := time.Now().UTC() + r := Router(db, func() time.Time { return now }) + token := loginToken(t, r, u.Username, fixturePassword) + code, data := callAPI(t, r, "GET", "/api/v1/login-logs?username="+u.Username, token, nil) + if code != 200 { + t.Fatalf("login logs status %d, want 200", code) + } + var page struct { + Items []struct{ Username, Result, Reason string } + Total int64 + } + if err := json.Unmarshal(data, &page); err != nil { + t.Fatal(err) + } + if page.Total != 1 || len(page.Items) != 1 || page.Items[0].Result != "success" { + t.Fatal("successful login missing") + } + callAPI(t, r, "POST", "/api/v1/login", "", map[string]string{"username": u.Username, "password": "fictional-wrong-secret"}) + name := randomName("auditlearner") + code, _ = callAPI(t, r, "POST", "/api/v1/accounts", token, map[string]string{"username": name, "password": fixturePassword}) + if code != 201 { + t.Fatal("create failed") + } + callAPI(t, r, "POST", "/api/v1/accounts", token, map[string]string{"username": name, "password": fixturePassword}) + code, data = callAPI(t, r, "GET", "/api/v1/operation-logs?username="+u.Username, token, nil) + if code != 200 { + t.Fatal("operation query failed") + } + if err := json.Unmarshal(data, &page); err != nil { + t.Fatal(err) + } + if page.Total != 2 { + t.Fatalf("expected success and failure operation logs, got %d", page.Total) + } + for _, secret := range []string{fixturePassword, "fictional-wrong-secret", token} { + if strings.Contains(string(data), secret) { + t.Fatal("secret leaked") + } + } + learnerToken := loginToken(t, r, name, fixturePassword) + for _, path := range []string{"/api/v1/login-logs", "/api/v1/operation-logs"} { + if code, _ = callAPI(t, r, "GET", path, "", nil); code != 401 { + t.Fatal("anonymous allowed") + } + if code, _ = callAPI(t, r, "GET", path, learnerToken, nil); code != 403 { + t.Fatal("learner allowed") + } + for _, q := range []string{"?limit=101", "?page=0", "?from=invalid", "?result=invalid", "?unexpected=x"} { + if code, _ = callAPI(t, r, "GET", path+q, token, nil); code != 400 { + t.Fatalf("invalid query accepted %s %d", q, code) + } + } + } + code, data = callAPI(t, r, "GET", "/api/v1/login-logs?username="+u.Username+"&result=failure", token, nil) + json.Unmarshal(data, &page) + if code != 200 || page.Total != 1 || page.Items[0].Reason != "invalid_credentials" { + t.Fatal("failure filter incorrect") + } + var target admin.SysUser + db.Where("username=?", name).First(&target) + path := "/api/v1/accounts/" + strconv.Itoa(target.UserId) + for _, input := range []any{map[string]any{"disabled": true}, map[string]any{"disabled": false}, map[string]any{"password": strings.Repeat("q", 6)}} { + if code, _ = callAPI(t, r, "PATCH", path, token, input); code != 200 { + t.Fatal("audited update failed") + } + } + callAPI(t, r, "PATCH", path, token, map[string]any{"password": "tiny"}) + for _, action := range []string{"disable", "enable", "reset_password"} { + code, data = callAPI(t, r, "GET", "/api/v1/operation-logs?username="+name+"&action="+action+"&result=success", token, nil) + json.Unmarshal(data, &page) + if code != 200 || page.Total != 1 { + t.Fatalf("missing %s audit", action) + } + } + if code, _ = callAPI(t, r, "GET", "/api/v1/me", learnerToken, nil); code != 401 { + t.Fatal("old learner session survived") + } + callAPI(t, r, "POST", "/api/v1/login", "", map[string]string{"username": "", "password": "raw-password-marker"}) + var logs []LoginLog + db.Where("created_at >= ?", now.Truncate(time.Millisecond)).Find(&logs) + encoded, _ := json.Marshal(logs) + for _, secret := range []string{"", "raw-password-marker", fixturePassword, token} { + if strings.Contains(string(encoded), secret) { + t.Fatal("private input in persisted login records") + } + } +} diff --git a/server/app/lexgo/database.go b/server/app/lexgo/database.go index d552b2f..38f0244 100644 --- a/server/app/lexgo/database.go +++ b/server/app/lexgo/database.go @@ -59,10 +59,10 @@ func Migrate(db *gorm.DB) error { if err = conn.QueryRowContext(ctx, "SELECT version,product FROM lexgo_schema WHERE id=1").Scan(¤t, &product); err != nil { return err } - if product != "lexgo" || current < 0 || current > 1 { + if product != "lexgo" || current < 0 || current > 2 { return errors.New("unknown schema version") } - if current == 1 { + if current == 2 { return nil } statements := []string{ @@ -86,6 +86,17 @@ func Migrate(db *gorm.DB) error { INDEX (owner_id), INDEX (expires_at), FOREIGN KEY (owner_id) REFERENCES sys_user(user_id) ON DELETE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`, + `CREATE TABLE IF NOT EXISTS lexgo_login_logs ( + id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, username VARCHAR(32) NOT NULL DEFAULT '', + result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL, ip VARCHAR(45) NOT NULL DEFAULT '', + created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(username,created_at) + ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`, + `CREATE TABLE IF NOT EXISTS lexgo_operation_logs ( + id BIGINT UNSIGNED PRIMARY KEY AUTO_INCREMENT, actor_id BIGINT NOT NULL, actor_username VARCHAR(32) NOT NULL, + target_id BIGINT NOT NULL DEFAULT 0, target_username VARCHAR(32) NOT NULL DEFAULT '', + action VARCHAR(32) NOT NULL, result VARCHAR(16) NOT NULL, reason VARCHAR(32) NOT NULL, + created_at DATETIME(3) NOT NULL, INDEX(created_at,id), INDEX(actor_username,created_at), INDEX(target_username,created_at) + ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4`, } for i, s := range statements { if _, err = conn.ExecContext(ctx, s); err != nil { @@ -96,7 +107,7 @@ func Migrate(db *gorm.DB) error { return fmt.Errorf("migration statement %d failed", i+1) } } - _, err = conn.ExecContext(ctx, "UPDATE lexgo_schema SET version=1 WHERE id=1") + _, err = conn.ExecContext(ctx, "UPDATE lexgo_schema SET version=2 WHERE id=1") return err } @@ -105,7 +116,7 @@ func CheckSchema(db *gorm.DB) error { Version int Product string } - if err := db.Table("lexgo_schema").Where("id=1").First(&r).Error; err != nil || r.Version != 1 || r.Product != "lexgo" { + if err := db.Table("lexgo_schema").Where("id=1").First(&r).Error; err != nil || r.Version != 2 || r.Product != "lexgo" { return errors.New("run the explicit migration before starting") } return nil diff --git a/server/app/lexgo/migration_test.go b/server/app/lexgo/migration_test.go index 26316ac..b474b44 100644 --- a/server/app/lexgo/migration_test.go +++ b/server/app/lexgo/migration_test.go @@ -92,7 +92,7 @@ func TestMigrationRefusesUnownedOrUnsupportedSchema(t *testing.T) { for _, tc := range []struct{ name, marker string }{ {"empty_marker", ""}, {"negative_version", "INSERT INTO lexgo_schema VALUES (1,-1,'lexgo')"}, - {"future_version", "INSERT INTO lexgo_schema VALUES (1,2,'lexgo')"}, + {"future_version", "INSERT INTO lexgo_schema VALUES (1,3,'lexgo')"}, {"wrong_product", "INSERT INTO lexgo_schema VALUES (1,0,'another-app')"}, } { t.Run(tc.name, func(t *testing.T) { diff --git a/server/app/lexgo/router.go b/server/app/lexgo/router.go index 90ed571..8103bd8 100644 --- a/server/app/lexgo/router.go +++ b/server/app/lexgo/router.go @@ -38,21 +38,41 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine { Username string `json:"username"` Password string `json:"password"` } - if err := decode(c, &input); err != nil { + failed := func(err error) { + entry := loginAudit(c, input.Username, now(), err) + if logErr := saveFailedAudit(db, &entry); logErr != nil { + err = logErr + } respond(c, 400, nil, err) + } + if err := decode(c, &input); err != nil { + failed(err) return } name, err := loginCredentials(input.Username, input.Password) if err != nil { - respond(c, 400, nil, err) + failed(err) return } if !limits.allow(c.ClientIP(), now()) { - respond(c, 429, nil, failure(429, "登录尝试过多,请稍后再试")) + failed(failure(429, "登录尝试过多,请稍后再试")) return } - result, err := login(db.WithContext(c.Request.Context()), now(), name, input.Password) - respond(c, 200, result, err) + var result LoginResult + err = db.WithContext(c.Request.Context()).Transaction(func(tx *gorm.DB) error { + var loginErr error + result, loginErr = login(tx, now(), name, input.Password) + if loginErr != nil { + return loginErr + } + entry := loginAudit(c, name, now(), nil) + return tx.Create(&entry).Error + }) + if err != nil { + failed(err) + return + } + respond(c, 200, result, nil) }) protect := func(adminOnly bool, action func(*gin.Context, *gorm.DB, admin.SysUser) (any, error)) gin.HandlerFunc { return func(c *gin.Context) { @@ -62,6 +82,7 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine { return } var data any + var audit *OperationLog err := db.WithContext(c.Request.Context()).Transaction(func(tx *gorm.DB) error { u, err := authenticate(tx, token, now()) if err != nil { @@ -70,9 +91,40 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine { if adminOnly && u.RoleId != 1 { return failure(403, "需要管理员权限") } + if c.FullPath() == "/api/v1/accounts" && c.Request.Method == "POST" || c.FullPath() == "/api/v1/accounts/:id" && c.Request.Method == "PATCH" { + audit = &OperationLog{ActorID: u.UserId, ActorUsername: u.Username, Action: "create", CreatedAt: now().UTC().Truncate(time.Millisecond)} + if c.Request.Method == "PATCH" { + audit.Action = "update" + if id, e := strconv.Atoi(c.Param("id")); e == nil && id > 0 { + audit.TargetID = id + var target admin.SysUser + if tx.Where("user_id = ?", id).First(&target).Error == nil { + audit.TargetUsername = target.Username + } + } + } + c.Set("operationAudit", audit) + } data, err = action(c, tx, u) + if err == nil && audit != nil { + if a, ok := data.(Account); ok { + audit.TargetID = a.ID + audit.TargetUsername = a.Username + } + audit.Result = "success" + audit.Reason = "ok" + err = saveOperationAudits(tx, c, audit) + } return err }) + if err != nil && audit != nil { + audit.ID = 0 + audit.Result = "failure" + audit.Reason = auditReason(err) + if logErr := saveOperationFailure(db, c, audit); logErr != nil { + err = logErr + } + } status := 200 if c.Request.Method == "POST" && c.FullPath() == "/api/v1/accounts" { status = 201 @@ -81,6 +133,12 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine { } } v.GET("/me", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) { return account(u), nil })) + v.GET("/login-logs", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) { + return queryAudit(tx, c.Request.URL.Query(), false, now()) + })) + v.GET("/operation-logs", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) { + return queryAudit(tx, c.Request.URL.Query(), true, now()) + })) v.POST("/logout", protect(false, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) { token := strings.TrimPrefix(c.GetHeader("Authorization"), "Bearer ") return nil, tx.Where("token_hash = ? AND owner_id = ?", digest(token), u.UserId).Delete(&Session{}).Error @@ -125,6 +183,9 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine { if err := decode(c, &input); err != nil { return nil, err } + if value, ok := c.Get("operationAudit"); ok { + value.(*OperationLog).TargetUsername = safeAuditUsername(input.Username) + } return createAccount(tx, input.Username, input.Password, 2, u.UserId) })) v.PATCH("/accounts/:id", protect(true, func(c *gin.Context, tx *gorm.DB, u admin.SysUser) (any, error) { @@ -136,6 +197,20 @@ func Router(db *gorm.DB, now func() time.Time) *gin.Engine { if err = decode(c, &input); err != nil { return nil, err } + actions := []string{} + if input.Disabled != nil { + a := "enable" + if *input.Disabled { + a = "disable" + } + actions = append(actions, a) + } + if input.Password != nil { + actions = append(actions, "reset_password") + } + if len(actions) > 0 { + c.Set("operationActions", actions) + } return updateAccount(tx, id, u.UserId, input) })) r.NoRoute(func(c *gin.Context) { respond(c, 404, nil, failure(404, "页面或接口不存在")) }) diff --git a/server/cmd/lexgo/main.go b/server/cmd/lexgo/main.go index 6d16472..a267461 100644 --- a/server/cmd/lexgo/main.go +++ b/server/cmd/lexgo/main.go @@ -66,10 +66,10 @@ func openDB() (*gorm.DB, error) { func run() error { if len(os.Args) != 2 { - return errors.New("usage: lexgo migrate|bootstrap|serve") + return errors.New("usage: lexgo migrate|bootstrap|serve|audit-cleanup") } command := os.Args[1] - if command != "migrate" && command != "bootstrap" && command != "serve" { + if command != "migrate" && command != "bootstrap" && command != "serve" && command != "audit-cleanup" { return errors.New("unknown command") } db, err := openDB() @@ -82,12 +82,21 @@ func run() error { if err = lexgo.Migrate(db); err != nil { return err } - fmt.Println("LexGo schema version 1 ready") + fmt.Println("LexGo schema version 2 ready") return nil } if err = lexgo.CheckSchema(db); err != nil { return err } + if command == "audit-cleanup" { + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + if err = lexgo.CleanupAudit(ctx, db, time.Now()); err != nil { + return errors.New("audit cleanup failed") + } + fmt.Println("Expired audit records cleaned") + return nil + } if command == "bootstrap" { if err = lexgo.BootstrapAdmin(db, os.Getenv("LEXGO_BOOTSTRAP_USERNAME"), os.Getenv("LEXGO_BOOTSTRAP_PASSWORD")); err != nil { return err @@ -103,6 +112,22 @@ func run() error { srv := &http.Server{Addr: addr, Handler: lexgo.Router(db, time.Now), ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 15 * time.Second, WriteTimeout: 15 * time.Second, IdleTimeout: 60 * time.Second, MaxHeaderBytes: 1 << 20} ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) defer stop() + go func() { + ticker := time.NewTicker(time.Hour) + defer ticker.Stop() + for { + cleanupCtx, cancel := context.WithTimeout(ctx, time.Minute) + if err := lexgo.CleanupAudit(cleanupCtx, db, time.Now()); err != nil && ctx.Err() == nil { + log.Print("audit cleanup failed; retrying next hour") + } + cancel() + select { + case <-ctx.Done(): + return + case <-ticker.C: + } + } + }() done := make(chan error, 1) go func() { done <- srv.ListenAndServe() }() fmt.Println("LexGo API listening on", addr)