Compare commits
@@ -0,0 +1,50 @@
|
||||
# OPC SynapBus 开发规则
|
||||
|
||||
本仓库是 GitHub `synapbus/synapbus` 的 OPC fork,托管在 Gitea `https://git.ilapage.cn/OPC/synapbus`。技术栈、目录结构和编码约定以上游 `CLAUDE.md` 为准;本文件只补充 OPC 本地的分支、运行和安全规则。两者冲突时,安全规则以本文件为准。
|
||||
|
||||
## 1. 远程与分支
|
||||
|
||||
- `origin` = Gitea(OPC 改动推送到这里);`upstream` = GitHub(只拉取,推送地址设为 `DISABLED`,禁止向上游直接推送)。
|
||||
- `main`:只同步上游,不放 OPC 改动。同步方式:`git fetch upstream` 后把 `upstream/main` 快进推到 `origin/main`。
|
||||
- `opc/main`:上游 + OPC 补丁,是本机实际编译运行的版本。
|
||||
- 新功能或修复从 `main` 拉分支(如 `feat/<工单号>-<简述>`),在分支上开发和测试,合入 `opc/main`。这样以后可以把同一分支直接提给上游。
|
||||
- 本地专用、不打算提给上游的补丁(如 Windows 编译修复)直接提交到 `opc/main`,并在提交信息中注明 `OPC local patch`。
|
||||
- 同步上游后把 `main` 合入 `opc/main`,重新编译并跑测试,确认本地补丁仍然需要、仍然有效。
|
||||
|
||||
### worktree 使用与清理
|
||||
|
||||
- 功能分支在独立 worktree 中开发,统一放在 `D:/OPC/synapbus-wt/issue-<N>`;`D:/OPC/synapbus` 是正在运行服务的检出,不在其中切分支开发。
|
||||
- Windows 编译补丁(renameio)在功能 worktree 中以未提交 + `skip-worktree` 的方式放置,不提交到功能分支。
|
||||
- 功能分支合入 `opc/main` 并推送后(需要替换运行中二进制的,在替换完成后),由实施该工单的 Agent 删除对应 worktree,并在工单记录。删除前确认:HEAD 已合入 `opc/main`;除 skip-worktree 文件外 `git status --porcelain` 为空;没有进程引用;目录内指向外部的链接先断开。
|
||||
- 删除前先撤销 skip-worktree 标记并还原 `go.mod`/`go.sum`(`git update-index --no-skip-worktree go.mod go.sum` 后 `git checkout -- go.mod go.sum`),再执行 `git worktree remove <路径>`,不加 `--force`,不删除分支;之后 `git worktree prune`。
|
||||
- 未合并或有未提交改动的 worktree 不得删除,先向用户报告。
|
||||
|
||||
## 2. 工单
|
||||
|
||||
- 功能、缺陷修复,以及鉴权、权限、接口、数据库迁移方面的改动,先在 Gitea `OPC/synapbus` 建工单,写清目标、非目标、方案、验收、风险和测试;方案经用户确认后再写代码。
|
||||
- 涉及鉴权、权限边界、数据库迁移、删除数据的改动属于高风险:实施前和替换运行中的服务前都要等用户确认。
|
||||
- 提交信息引用工单号,例如 `feat(#1): ...`。
|
||||
|
||||
## 3. 本机运行
|
||||
|
||||
- 运行命令:`D:/OPC/synapbus/bin/synapbus.exe serve --host 127.0.0.1 --port 8182 --data D:/OPC/synapbus/data`。
|
||||
- **只监听 127.0.0.1**:不使用 `0.0.0.0` 或局域网地址,不设置 `SYNAPBUS_ALLOW_PRIVATE_NETWORKS=true`;需要对外开放时先建单评估。
|
||||
- 管理命令要显式指定 socket:`bin/synapbus.exe --socket ./data/synapbus.sock <command>`(Windows 上默认的 `/tmp/synapbus.sock` 连不上)。
|
||||
- 编译输出到 `bin/`,不要把 `synapbus.exe` 留在仓库根目录。替换 `bin/synapbus.exe` 意味着重启服务,会断开所有 Agent 的连接,需要先告知用户。
|
||||
- `internal/web/dist/` 下只有 `index.html` 受版本控制,前端资源文件(`_app/`)被忽略。本机对 `index.html` 设置了 `skip-worktree`,不要还原或提交它,否则会和当前构建的资源文件名对不上。
|
||||
|
||||
## 4. 数据与密钥
|
||||
|
||||
- `data/`(`synapbus.db`、`secrets.key`、`vapid_keys.json`、附件)和 `bin/` 已被 `.gitignore` 忽略,**永远不要提交**,也不要复制到工单、日志或消息里。
|
||||
- Agent API Key 只保存在 `C:\Users\ila20\synapbus.env`;创建或重新生成 key 时,用脚本把 key 直接写入该文件,不在屏幕或对话中显示。
|
||||
- 测试使用临时数据目录,不读写正在运行的 `data/`。
|
||||
|
||||
## 5. 验证
|
||||
|
||||
- 改动后至少运行:`go build ./cmd/synapbus`(输出到 `bin/` 或临时目录)和受影响包的 `go test`;修改 Web 时按上游 `Makefile` 构建前端。
|
||||
- 鉴权和权限相关改动必须有越权测试,例如某个 Agent 的 key 不能看到其他 Agent 的私信或事件。
|
||||
- 测试结果必须如实记录;没跑的、跑不了的要写明。
|
||||
|
||||
## 6. 许可证
|
||||
|
||||
- 上游为 Apache-2.0:保留 `LICENSE`;修改过的文件在文件头或改动处注明 OPC 修改。
|
||||
@@ -114,6 +114,8 @@ make lint # Run linters
|
||||
- SQLite via `modernc.org/sqlite` — five new migrations (`021_goals_tasks.sql`, `022_agent_proposals.sql`, `023_agent_trust_model.sql`, `024_secrets.sql`, `025_harness_runs_task_id.sql`); existing content-addressable attachment store reused for encrypted secret blobs (018-dynamic-agent-spawning)
|
||||
- Go 1.25+ (per go.mod) + `mark3labs/mcp-go` (MCP), `go-chi/chi` (HTTP), `spf13/cobra` (CLI), `modernc.org/sqlite` (storage), `jmoiron/sqlx` (query helpers), `cloudflare/tableflip` (graceful restart — NEW), `gopkg.in/yaml.v3` (config), `xeipuuv/gojsonschema` (config-schema validation) (019-plugin-system)
|
||||
- SQLite via `modernc.org/sqlite` (pure Go, zero CGO). New core table `plugin_migrations`. Plugin tables namespaced `plugin_<name>_*`. (019-plugin-system)
|
||||
- Go 1.25+ (per `go.mod`) + `mark3labs/mcp-go` (MCP tools), `go-chi/chi` (HTTP), `modernc.org/sqlite` (storage), `TFMV/hnsw` (vectors via existing `search.Service`), existing `internal/harness` package (dispatch seam). **No new external dependencies.** (020-proactive-memory-dream-worker)
|
||||
- SQLite via `modernc.org/sqlite` — one new migration `028_memory_consolidation.sql`. Memory pool reuses the existing `messages` table on memory-flagged channels. (020-proactive-memory-dream-worker)
|
||||
|
||||
## Recent Changes
|
||||
- 002-mcp-auth-ux-polish: Added Go 1.23+ + ory/fosite (OAuth 2.1), mark3labs/mcp-go (MCP server), go-chi/chi (HTTP), Svelte 5 + Tailwind (Web UI)
|
||||
|
||||
+1
-1
@@ -19,7 +19,7 @@ RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w -X main.version=${VERSION}
|
||||
|
||||
# Stage 3: Runtime
|
||||
FROM alpine:3.19
|
||||
RUN apk add --no-cache ca-certificates tzdata && touch /.dockerenv
|
||||
RUN apk add --no-cache ca-certificates tzdata sqlite && touch /.dockerenv
|
||||
COPY --from=go-builder /synapbus /synapbus
|
||||
EXPOSE 8080
|
||||
VOLUME ["/data"]
|
||||
|
||||
+131
-5
@@ -702,10 +702,10 @@ func addAdminCommands(rootCmd *cobra.Command) {
|
||||
channelsJoinCmd.MarkFlagRequired("agent")
|
||||
|
||||
var (
|
||||
channelsUpdateName string
|
||||
channelsUpdateAutoApprove string
|
||||
channelsUpdateStalemateRemind string
|
||||
channelsUpdateStalemateEscalate string
|
||||
channelsUpdateName string
|
||||
channelsUpdateAutoApprove string
|
||||
channelsUpdateStalemateRemind string
|
||||
channelsUpdateStalemateEscalate string
|
||||
)
|
||||
channelsUpdateCmd := &cobra.Command{
|
||||
Use: "update",
|
||||
@@ -1341,10 +1341,136 @@ Examples:
|
||||
harnessConfigCmd.AddCommand(harnessConfigGetCmd, harnessConfigSetCmd, harnessConfigEditCmd)
|
||||
harnessCmd.AddCommand(harnessConfigCmd)
|
||||
|
||||
// ----- memory commands (feature 020 — proactive memory + dream worker) -----
|
||||
memoryCmd := &cobra.Command{
|
||||
Use: "memory",
|
||||
Short: "Manage proactive memory (feature 020)",
|
||||
}
|
||||
|
||||
memoryCoreCmd := &cobra.Command{
|
||||
Use: "core",
|
||||
Short: "Manage per-(owner, agent) core memory blobs",
|
||||
}
|
||||
|
||||
var (
|
||||
memCoreOwner string
|
||||
memCoreAgent string
|
||||
memCoreBlob string
|
||||
memCoreBlobFile string
|
||||
memCoreUpdater string
|
||||
)
|
||||
|
||||
memoryCoreGetCmd := &cobra.Command{
|
||||
Use: "get",
|
||||
Short: "Print the current core memory blob for an (owner, agent)",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
resp, err := adminRequest("memory.core.get", map[string]string{
|
||||
"owner": memCoreOwner,
|
||||
"agent": memCoreAgent,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
printJSON(resp["data"])
|
||||
return nil
|
||||
},
|
||||
}
|
||||
memoryCoreGetCmd.Flags().StringVar(&memCoreOwner, "owner", "", "Owner username or numeric user ID")
|
||||
memoryCoreGetCmd.Flags().StringVar(&memCoreAgent, "agent", "", "Agent name")
|
||||
_ = memoryCoreGetCmd.MarkFlagRequired("owner")
|
||||
_ = memoryCoreGetCmd.MarkFlagRequired("agent")
|
||||
|
||||
memoryCoreSetCmd := &cobra.Command{
|
||||
Use: "set",
|
||||
Short: "Replace the core memory blob (wholesale, no merge)",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
blob := memCoreBlob
|
||||
if memCoreBlobFile != "" {
|
||||
data, err := os.ReadFile(memCoreBlobFile)
|
||||
if err != nil {
|
||||
return fmt.Errorf("read --blob-file: %w", err)
|
||||
}
|
||||
blob = string(data)
|
||||
}
|
||||
if blob == "" {
|
||||
return fmt.Errorf("either --blob or --blob-file is required (non-empty)")
|
||||
}
|
||||
resp, err := adminRequest("memory.core.set", map[string]string{
|
||||
"owner": memCoreOwner,
|
||||
"agent": memCoreAgent,
|
||||
"blob": blob,
|
||||
"updated_by": memCoreUpdater,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
printJSON(resp["data"])
|
||||
return nil
|
||||
},
|
||||
}
|
||||
memoryCoreSetCmd.Flags().StringVar(&memCoreOwner, "owner", "", "Owner username or numeric user ID")
|
||||
memoryCoreSetCmd.Flags().StringVar(&memCoreAgent, "agent", "", "Agent name")
|
||||
memoryCoreSetCmd.Flags().StringVar(&memCoreBlob, "blob", "", "Core memory blob (inline)")
|
||||
memoryCoreSetCmd.Flags().StringVar(&memCoreBlobFile, "blob-file", "", "Read blob from file path (overrides --blob)")
|
||||
memoryCoreSetCmd.Flags().StringVar(&memCoreUpdater, "updated-by", "human", "updated_by audit field (default: human)")
|
||||
_ = memoryCoreSetCmd.MarkFlagRequired("owner")
|
||||
_ = memoryCoreSetCmd.MarkFlagRequired("agent")
|
||||
|
||||
memoryCoreDeleteCmd := &cobra.Command{
|
||||
Use: "delete",
|
||||
Short: "Remove the core memory blob",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
resp, err := adminRequest("memory.core.delete", map[string]string{
|
||||
"owner": memCoreOwner,
|
||||
"agent": memCoreAgent,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
printJSON(resp["data"])
|
||||
return nil
|
||||
},
|
||||
}
|
||||
memoryCoreDeleteCmd.Flags().StringVar(&memCoreOwner, "owner", "", "Owner username or numeric user ID")
|
||||
memoryCoreDeleteCmd.Flags().StringVar(&memCoreAgent, "agent", "", "Agent name")
|
||||
_ = memoryCoreDeleteCmd.MarkFlagRequired("owner")
|
||||
_ = memoryCoreDeleteCmd.MarkFlagRequired("agent")
|
||||
|
||||
memoryCoreCmd.AddCommand(memoryCoreGetCmd, memoryCoreSetCmd, memoryCoreDeleteCmd)
|
||||
memoryCmd.AddCommand(memoryCoreCmd)
|
||||
|
||||
// ----- memory dream-run (feature 020 — manual dispatch) -----
|
||||
var (
|
||||
dreamRunOwner string
|
||||
dreamRunJobType string
|
||||
dreamRunParallel int
|
||||
)
|
||||
memoryDreamRunCmd := &cobra.Command{
|
||||
Use: "dream-run",
|
||||
Short: "Force a consolidation job dispatch (bypasses trigger checks). With --parallel N spawns N concurrent jobs.",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
resp, err := adminRequest("memory.dream_run", map[string]any{
|
||||
"owner": dreamRunOwner,
|
||||
"job_type": dreamRunJobType,
|
||||
"parallel": dreamRunParallel,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
printJSON(resp["data"])
|
||||
return nil
|
||||
},
|
||||
}
|
||||
memoryDreamRunCmd.Flags().StringVar(&dreamRunOwner, "owner", "", "Owner username or numeric user ID")
|
||||
memoryDreamRunCmd.Flags().StringVar(&dreamRunJobType, "job", "reflection", "Job type (reflection | core_rewrite | dedup_contradiction | link_gen)")
|
||||
memoryDreamRunCmd.Flags().IntVar(&dreamRunParallel, "parallel", 0, "Spawn N concurrent jobs of this type (0 = use server SYNAPBUS_DREAM_PARALLEL; core_rewrite forces 1)")
|
||||
_ = memoryDreamRunCmd.MarkFlagRequired("owner")
|
||||
memoryCmd.AddCommand(memoryDreamRunCmd)
|
||||
|
||||
// ----- add persistent flag and commands to root -----
|
||||
rootCmd.PersistentFlags().StringVar(&adminSocket, "socket", "/tmp/synapbus.sock", "Path to admin Unix socket")
|
||||
|
||||
rootCmd.AddCommand(userCmd, agentCmd, auditCmd, backupCmd, messagesCmd, channelsCmd, conversationsCmd, embeddingsCmd, dbCmd, retentionCmd, webhookCmd, k8sCmd, attachmentsCmd, harnessCmd)
|
||||
rootCmd.AddCommand(userCmd, agentCmd, auditCmd, backupCmd, messagesCmd, channelsCmd, conversationsCmd, embeddingsCmd, dbCmd, retentionCmd, webhookCmd, k8sCmd, attachmentsCmd, harnessCmd, memoryCmd)
|
||||
}
|
||||
|
||||
// toTableRows remaps []map[string]string using a header->key mapping.
|
||||
|
||||
+224
-50
@@ -26,6 +26,7 @@ import (
|
||||
"github.com/synapbus/synapbus/internal/a2a"
|
||||
"github.com/synapbus/synapbus/internal/actions"
|
||||
"github.com/synapbus/synapbus/internal/admin"
|
||||
"github.com/synapbus/synapbus/internal/agentquery"
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/api"
|
||||
"github.com/synapbus/synapbus/internal/apikeys"
|
||||
@@ -34,51 +35,50 @@ import (
|
||||
"github.com/synapbus/synapbus/internal/auth/idp"
|
||||
"github.com/synapbus/synapbus/internal/channels"
|
||||
"github.com/synapbus/synapbus/internal/console"
|
||||
"github.com/synapbus/synapbus/internal/dispatcher"
|
||||
"github.com/synapbus/synapbus/internal/goals"
|
||||
"github.com/synapbus/synapbus/internal/goaltasks"
|
||||
"github.com/synapbus/synapbus/internal/secrets"
|
||||
"github.com/synapbus/synapbus/internal/dispatcher"
|
||||
"github.com/synapbus/synapbus/internal/health"
|
||||
"github.com/synapbus/synapbus/internal/jsruntime"
|
||||
k8spkg "github.com/synapbus/synapbus/internal/k8s"
|
||||
"github.com/synapbus/synapbus/internal/marketplace"
|
||||
mcpserver "github.com/synapbus/synapbus/internal/mcp"
|
||||
"github.com/synapbus/synapbus/internal/agentquery"
|
||||
reactorpkg "github.com/synapbus/synapbus/internal/reactor"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
prommetrics "github.com/synapbus/synapbus/internal/metrics"
|
||||
"github.com/synapbus/synapbus/internal/harness"
|
||||
"github.com/synapbus/synapbus/internal/harness/docker"
|
||||
"github.com/synapbus/synapbus/internal/harness/k8sjob"
|
||||
"github.com/synapbus/synapbus/internal/harness/runs"
|
||||
"github.com/synapbus/synapbus/internal/harness/subprocess"
|
||||
"github.com/synapbus/synapbus/internal/harness/webhook"
|
||||
"github.com/synapbus/synapbus/internal/health"
|
||||
"github.com/synapbus/synapbus/internal/jsruntime"
|
||||
k8spkg "github.com/synapbus/synapbus/internal/k8s"
|
||||
"github.com/synapbus/synapbus/internal/marketplace"
|
||||
mcpserver "github.com/synapbus/synapbus/internal/mcp"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
prommetrics "github.com/synapbus/synapbus/internal/metrics"
|
||||
"github.com/synapbus/synapbus/internal/observability"
|
||||
"github.com/synapbus/synapbus/internal/push"
|
||||
"github.com/synapbus/synapbus/internal/reactions"
|
||||
reactorpkg "github.com/synapbus/synapbus/internal/reactor"
|
||||
"github.com/synapbus/synapbus/internal/search"
|
||||
"github.com/synapbus/synapbus/internal/search/embedding"
|
||||
"github.com/synapbus/synapbus/internal/secrets"
|
||||
"github.com/synapbus/synapbus/internal/storage"
|
||||
"github.com/synapbus/synapbus/internal/push"
|
||||
"github.com/synapbus/synapbus/internal/trace"
|
||||
"github.com/synapbus/synapbus/internal/trust"
|
||||
"github.com/synapbus/synapbus/internal/web"
|
||||
"github.com/synapbus/synapbus/internal/wiki"
|
||||
"github.com/synapbus/synapbus/internal/webhooks"
|
||||
"github.com/synapbus/synapbus/internal/wiki"
|
||||
)
|
||||
|
||||
// version is set at build time via -ldflags "-X main.version=..."
|
||||
var version = "dev"
|
||||
|
||||
var (
|
||||
host string
|
||||
port int
|
||||
dataDir string
|
||||
logLevel string
|
||||
metricsEnabled bool
|
||||
traceRetention string
|
||||
adminSocketPath string
|
||||
webhookWorkers int
|
||||
messageRetention string
|
||||
host string
|
||||
port int
|
||||
dataDir string
|
||||
logLevel string
|
||||
metricsEnabled bool
|
||||
traceRetention string
|
||||
adminSocketPath string
|
||||
webhookWorkers int
|
||||
messageRetention string
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -305,7 +305,7 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
}
|
||||
|
||||
// Create swarm service (task auction + stigmergy)
|
||||
taskStore := channels.NewSQLiteTaskStore(db.DB)
|
||||
taskStore := channels.NewSQLiteTaskStore(db.DB).WithReadDB(db.ReadDB)
|
||||
swarmService := channels.NewSwarmService(taskStore, channelStore, tracer)
|
||||
|
||||
// Create attachment service
|
||||
@@ -512,7 +512,15 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
// going through the existing createJob + poller path; subprocess
|
||||
// and webhook agents go through Registry.Execute.
|
||||
harnessRegistry := harness.NewRegistry()
|
||||
harnessRegistry.Register(k8sjob.New(k8sRunner, nil, slog.Default()))
|
||||
// Build a ClientsetWaiter when we have a real in-cluster runner so
|
||||
// the k8sjob backend can actually wait for Job completion. Without
|
||||
// this the backend errors immediately with "no Waiter configured"
|
||||
// (the failure mode dream-worker jobs were hitting pre-fix).
|
||||
var k8sWaiter k8sjob.Waiter
|
||||
if rr, ok := k8sRunner.(*k8spkg.K8sJobRunner); ok {
|
||||
k8sWaiter = k8sjob.NewClientsetWaiter(rr.GetClientset(), 0)
|
||||
}
|
||||
harnessRegistry.Register(k8sjob.New(k8sRunner, k8sWaiter, slog.Default()))
|
||||
// SYNAPBUS_KEEP_WORKDIR=1 preserves per-run workdirs after successful
|
||||
// runs. Useful when debugging MCP tool traces, gemini stdout, or
|
||||
// materialized config files. Default off to avoid disk growth.
|
||||
@@ -582,6 +590,49 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
|
||||
mcpSrv := mcpserver.NewMCPServer(msgService, agentService, channelService, swarmService, attachmentService, searchService, reactionService, trustService, wikiService, con, jsPool, actionRegistry, actionIndex, db.DB)
|
||||
|
||||
// Feature 020 — proactive memory injection.
|
||||
//
|
||||
// Parse the memory config from env, build the audit-ring store and
|
||||
// the per-(owner, agent) core memory store, and wire both into the
|
||||
// MCP hybrid tool surface. When SYNAPBUS_INJECTION_ENABLED=0 (the
|
||||
// default), SetInjection still runs but WrapInjection returns each
|
||||
// handler unchanged, so tool responses keep their pre-feature shape
|
||||
// bit-for-bit (FR-012, SC-009).
|
||||
memCfg := messaging.ParseMemoryConfig()
|
||||
memoryInjectionStore := messaging.NewMemoryInjections(db.DB)
|
||||
coreMemoryStore := messaging.NewCoreMemoryStore(db.DB, memCfg.CoreMemoryMaxBytes)
|
||||
mcpSrv.SetInjection(memCfg, memoryInjectionStore, messaging.NewCoreProvider(coreMemoryStore))
|
||||
slog.Info("proactive memory injection wired",
|
||||
"enabled", memCfg.InjectionEnabled,
|
||||
"budget_tokens", memCfg.InjectionBudgetTokens,
|
||||
"core_max_bytes", memCfg.CoreMemoryMaxBytes,
|
||||
)
|
||||
|
||||
// Feature 020 — dream worker (US3) stores. These are always
|
||||
// constructed so admin CLI / future REST endpoints can read them
|
||||
// even when SYNAPBUS_DREAM_ENABLED=0. The worker itself starts
|
||||
// only when the flag is on.
|
||||
memoryLinkStore := messaging.NewLinkStore(db.DB)
|
||||
memoryPinStore := messaging.NewPinStore(db.DB)
|
||||
memoryJobsStore := messaging.NewJobsStore(db.DB)
|
||||
dispatchTokens := messaging.NewDispatchTokenStore(db.DB)
|
||||
|
||||
// Wire the auto-link emitter as a message listener (T035).
|
||||
msgService.AddMessageListener(messaging.NewAutoLinkListener(db.DB, memoryLinkStore))
|
||||
|
||||
// Register the six memory_* MCP tools when SYNAPBUS_DREAM_ENABLED=1.
|
||||
mcpSrv.SetDream(mcpserver.MemoryToolDeps{
|
||||
DB: db.DB,
|
||||
Msg: msgService,
|
||||
Agents: agentService,
|
||||
Core: coreMemoryStore,
|
||||
Links: memoryLinkStore,
|
||||
Pins: memoryPinStore,
|
||||
Jobs: memoryJobsStore,
|
||||
Tokens: dispatchTokens,
|
||||
MemConfig: memCfg,
|
||||
})
|
||||
|
||||
// Wire the agent marketplace (spec 016 MVP).
|
||||
marketplaceStore := marketplace.NewStore(db.DB)
|
||||
marketplaceSvc := marketplace.NewService(marketplaceStore, wikiService, swarmService, channelService, msgService, tracer)
|
||||
@@ -599,7 +650,7 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
db.DB,
|
||||
)
|
||||
mcpSrv.WireGoalsTools(goalsToolReg)
|
||||
slog.Info("spec-018 MCP tools wired (create_goal, propose_task_tree, propose_agent, claim_task, request_resource, list_resources)")
|
||||
slog.Info("spec-018 MCP tools wired (create_goal, propose_task_tree, claim_task, request_resource, list_resources, complete_goal)")
|
||||
|
||||
// Set up SQL query executor for agents (uses read pool if available)
|
||||
queryDB := db.QueryDB()
|
||||
@@ -645,18 +696,51 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
slog.Info("stalemate worker disabled by SYNAPBUS_DISABLE_STALEMATE_WORKER=1")
|
||||
} else {
|
||||
stalemateConfig := messaging.ParseStalemateConfig()
|
||||
stalemateWorker = messaging.NewStalemateWorker(db.DB, msgService, &channelLookupAdapter{channelService: channelService}, stalemateConfig)
|
||||
stalemateWorker = messaging.NewStalemateWorker(db.DB, msgService, stalemateConfig)
|
||||
stalemateWorker.SetMemoryInjections(memoryInjectionStore)
|
||||
stalemateWorker.Start()
|
||||
slog.Info("stalemate worker started",
|
||||
"processing_timeout", stalemateConfig.ProcessingTimeout.String(),
|
||||
"reminder_after", stalemateConfig.ReminderAfter.String(),
|
||||
"escalate_after", stalemateConfig.EscalateAfter.String(),
|
||||
"interval", stalemateConfig.Interval.String(),
|
||||
)
|
||||
}
|
||||
|
||||
// Create health checker
|
||||
healthChecker := health.NewChecker(db.DB, version)
|
||||
// Feature 020 — consolidator (dream) worker. Only starts when
|
||||
// SYNAPBUS_DREAM_ENABLED=1.
|
||||
var consolidator *messaging.ConsolidatorWorker
|
||||
if memCfg.DreamEnabled {
|
||||
consolidator = messaging.NewConsolidatorWorker(
|
||||
db.DB,
|
||||
memoryJobsStore,
|
||||
dispatchTokens,
|
||||
&harnessDispatcherAdapter{reg: harnessRegistry},
|
||||
&agentLookupAdapter{svc: agentService},
|
||||
memCfg,
|
||||
)
|
||||
// Wire per-(owner, day) circuit breaker. The gate skips
|
||||
// dispatch (and records a circuit_broken job) once any of
|
||||
// SYNAPBUS_DREAM_DAILY_{TOKEN_LIMIT_IN,TOKEN_LIMIT_OUT,JOB_LIMIT}
|
||||
// is exceeded for the owner.
|
||||
dreamUsageStore := messaging.NewDreamUsageStore(db.DB)
|
||||
consolidator.SetUsageGate(dreamUsageStore, messaging.NewUsageGate(memCfg, dreamUsageStore))
|
||||
consolidator.Start()
|
||||
slog.Info("consolidator (dream) worker started",
|
||||
"interval", memCfg.DreamInterval.String(),
|
||||
"watermark", memCfg.DreamWatermark,
|
||||
"max_concurrent", memCfg.DreamMaxConcurrent,
|
||||
"agent", memCfg.DreamAgent,
|
||||
)
|
||||
} else {
|
||||
slog.Info("consolidator (dream) worker disabled (SYNAPBUS_DREAM_ENABLED=0)")
|
||||
}
|
||||
|
||||
// Create health checker. Use the read pool (8 concurrent conns) so /readyz
|
||||
// can't be starved by a long-running writer holding the serialized write
|
||||
// connection — most notably the consolidator's dream-job dispatch, which
|
||||
// does a K8s Job create + DB writes that can run 30s+. With the write pool
|
||||
// (MaxOpenConns=1) the probe blocked for the entire dispatch, the kubelet
|
||||
// flipped the pod to not-ready, and the watchdog scaled the deploy to 0.
|
||||
healthChecker := health.NewChecker(db.QueryDB(), version)
|
||||
|
||||
// Set up chi router
|
||||
r := chi.NewRouter()
|
||||
@@ -757,6 +841,14 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
// Register broadcaster as a message listener so SSE events fire
|
||||
// for messages sent via MCP (agents) as well as the REST API.
|
||||
msgService.AddMessageListener(sseBroadcaster)
|
||||
sseBroadcaster.SetMessageService(msgService)
|
||||
|
||||
// Per-agent SSE stream (agent API key auth, not session auth).
|
||||
sseHub.SetAgentEventBacklog(msgService)
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Use(agents.RequiredAuthMiddlewareWithOAuth(agentService, apiKeyService, oauthProvider))
|
||||
r.Get("/api/agent-events", sseHub.HandleAgentEvents)
|
||||
})
|
||||
|
||||
// Initialize push notification service
|
||||
pushStore := push.NewSQLiteStore(db.DB)
|
||||
@@ -781,6 +873,7 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
Broadcaster: sseBroadcaster,
|
||||
SessionMiddleware: sessionMiddleware,
|
||||
DB: db.DB,
|
||||
ReadDB: db.QueryDB(),
|
||||
Version: version,
|
||||
PushService: pushService,
|
||||
TrustService: trustService,
|
||||
@@ -791,6 +884,7 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
GoalTasksService: goalTasksService,
|
||||
BaseURL: baseURL,
|
||||
WikiService: wikiService,
|
||||
CoreMemoryStore: coreMemoryStore,
|
||||
})
|
||||
r.Mount("/", apiRouter)
|
||||
|
||||
@@ -799,13 +893,13 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
|
||||
// Start admin socket server
|
||||
adminSvcs := &admin.Services{
|
||||
Users: userStore,
|
||||
Sessions: sessionStore,
|
||||
Agents: agentService,
|
||||
Messages: msgService,
|
||||
Channels: channelService,
|
||||
Traces: traceStore,
|
||||
DataDir: dataDir,
|
||||
Users: userStore,
|
||||
Sessions: sessionStore,
|
||||
Agents: agentService,
|
||||
Messages: msgService,
|
||||
Channels: channelService,
|
||||
Traces: traceStore,
|
||||
DataDir: dataDir,
|
||||
}
|
||||
// Wire optional services into admin (may be nil if not configured)
|
||||
if searchCfg.IsEnabled() {
|
||||
@@ -821,6 +915,19 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
}
|
||||
adminSvcs.WebhookService = webhookService
|
||||
adminSvcs.K8sService = k8sService
|
||||
adminSvcs.CoreMemoryStore = coreMemoryStore
|
||||
if consolidator != nil {
|
||||
// Closure form keeps admin's import graph independent of
|
||||
// messaging.ConsolidatorWorker's full surface.
|
||||
c := consolidator
|
||||
adminSvcs.DreamRun = func(ctx context.Context, ownerID, jobType string) (int64, error) {
|
||||
return c.ForceRun(ctx, ownerID, jobType)
|
||||
}
|
||||
adminSvcs.DreamRunN = func(ctx context.Context, ownerID, jobType string, parallel int) ([]int64, error) {
|
||||
return c.ForceRunN(ctx, ownerID, jobType, parallel)
|
||||
}
|
||||
adminSvcs.DefaultDreamParallel = memCfg.DreamParallel
|
||||
}
|
||||
adminServer := admin.NewServer(adminSocketPath, db.DB, adminSvcs, logger)
|
||||
if err := adminServer.Start(); err != nil {
|
||||
return fmt.Errorf("start admin socket: %w", err)
|
||||
@@ -886,6 +993,11 @@ func runServe(cmd *cobra.Command, args []string) error {
|
||||
stalemateWorker.Stop()
|
||||
}
|
||||
|
||||
// Stop consolidator (dream) worker
|
||||
if consolidator != nil {
|
||||
consolidator.Stop()
|
||||
}
|
||||
|
||||
// Stop embedding pipeline
|
||||
if embPipeline != nil {
|
||||
embPipeline.Stop()
|
||||
@@ -1158,19 +1270,6 @@ func ensureDefaultMCPClient(ctx context.Context, db *sql.DB, bcryptCost int) {
|
||||
)
|
||||
}
|
||||
|
||||
// channelLookupAdapter adapts channels.Service to messaging.ChannelLookup.
|
||||
type channelLookupAdapter struct {
|
||||
channelService *channels.Service
|
||||
}
|
||||
|
||||
func (a *channelLookupAdapter) GetChannelIDByName(ctx context.Context, name string) (int64, error) {
|
||||
ch, err := a.channelService.GetChannelByName(ctx, name)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return ch.ID, nil
|
||||
}
|
||||
|
||||
// trustAdjusterAdapter adapts trust.Service to reactions.TrustAdjuster.
|
||||
type trustAdjusterAdapter struct {
|
||||
svc *trust.Service
|
||||
@@ -1211,3 +1310,78 @@ func (a *messageAuthorResolverAdapter) GetMessageAuthor(ctx context.Context, mes
|
||||
}
|
||||
return msg.FromAgent, nil
|
||||
}
|
||||
|
||||
// agentLookupAdapter adapts agents.AgentService to
|
||||
// messaging.AgentLookup so the dream worker can resolve the dream-agent
|
||||
// record without dragging the full *agents.AgentService into the
|
||||
// messaging package. The returned messaging.DreamAgent is the raw
|
||||
// *agents.Agent itself — DreamAgent's only required method
|
||||
// (AgentName()) is satisfied by agents.Agent.Name via the
|
||||
// agentNameMethod helper below.
|
||||
type agentLookupAdapter struct {
|
||||
svc *agents.AgentService
|
||||
}
|
||||
|
||||
func (a *agentLookupAdapter) GetAgent(ctx context.Context, name string) (messaging.DreamAgent, error) {
|
||||
ag, err := a.svc.GetAgent(ctx, name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return agentDreamWrap{ag: ag}, nil
|
||||
}
|
||||
|
||||
// agentDreamWrap adapts *agents.Agent to messaging.DreamAgent.
|
||||
type agentDreamWrap struct{ ag *agents.Agent }
|
||||
|
||||
func (w agentDreamWrap) AgentName() string {
|
||||
if w.ag == nil {
|
||||
return ""
|
||||
}
|
||||
return w.ag.Name
|
||||
}
|
||||
|
||||
// harnessDispatcherAdapter adapts *harness.Registry to
|
||||
// messaging.HarnessDispatcher so the consolidator worker can dispatch
|
||||
// dream-agent runs without importing the harness package (which would
|
||||
// create an import cycle — harness already imports messaging).
|
||||
type harnessDispatcherAdapter struct {
|
||||
reg *harness.Registry
|
||||
}
|
||||
|
||||
func (a *harnessDispatcherAdapter) Execute(
|
||||
ctx context.Context,
|
||||
agent messaging.DreamAgent,
|
||||
req *messaging.HarnessExecRequest,
|
||||
) (*messaging.HarnessExecResult, error) {
|
||||
// Unbox the agent record. The worker stores a DreamAgent
|
||||
// interface; in production it's an agentDreamWrap holding the
|
||||
// real *agents.Agent. Tests / admin force-runs may pass a bare
|
||||
// DreamAgentNamed which has no underlying record — the harness
|
||||
// fallback chain then resolves the backend by name alone.
|
||||
var realAgent *agents.Agent
|
||||
if wrap, ok := agent.(agentDreamWrap); ok {
|
||||
realAgent = wrap.ag
|
||||
}
|
||||
hreq := &harness.ExecRequest{
|
||||
RunID: req.RunID,
|
||||
AgentName: req.AgentName,
|
||||
Agent: realAgent,
|
||||
Env: req.Env,
|
||||
Budget: harness.Budget{MaxWallClock: req.MaxWallClock},
|
||||
}
|
||||
if req.Body != "" {
|
||||
hreq.Message = &messaging.Message{Body: req.Body}
|
||||
}
|
||||
res, err := a.reg.Execute(ctx, realAgent, hreq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := &messaging.HarnessExecResult{}
|
||||
if res != nil {
|
||||
out.ExitCode = res.ExitCode
|
||||
out.Logs = res.Logs
|
||||
out.TokensIn = res.Usage.TokensIn
|
||||
out.TokensOut = res.Usage.TokensOut
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
apiVersion: v2
|
||||
name: synapbus
|
||||
description: Agent-to-agent messaging for AI swarms
|
||||
type: application
|
||||
version: 0.1.0
|
||||
appVersion: "0.1.0"
|
||||
@@ -1,51 +0,0 @@
|
||||
{{/*
|
||||
Expand the name of the chart.
|
||||
*/}}
|
||||
{{- define "synapbus.name" -}}
|
||||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create a default fully qualified app name.
|
||||
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
|
||||
If release name contains chart name it will be used as a full name.
|
||||
*/}}
|
||||
{{- define "synapbus.fullname" -}}
|
||||
{{- if .Values.fullnameOverride }}
|
||||
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
|
||||
{{- else }}
|
||||
{{- $name := default .Chart.Name .Values.nameOverride }}
|
||||
{{- if contains $name .Release.Name }}
|
||||
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
|
||||
{{- else }}
|
||||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create chart name and version as used by the chart label.
|
||||
*/}}
|
||||
{{- define "synapbus.chart" -}}
|
||||
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Common labels
|
||||
*/}}
|
||||
{{- define "synapbus.labels" -}}
|
||||
helm.sh/chart: {{ include "synapbus.chart" . }}
|
||||
{{ include "synapbus.selectorLabels" . }}
|
||||
{{- if .Chart.AppVersion }}
|
||||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
|
||||
{{- end }}
|
||||
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Selector labels
|
||||
*/}}
|
||||
{{- define "synapbus.selectorLabels" -}}
|
||||
app.kubernetes.io/name: {{ include "synapbus.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
{{- end }}
|
||||
@@ -1,88 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{ include "synapbus.fullname" . }}
|
||||
labels:
|
||||
{{- include "synapbus.labels" . | nindent 4 }}
|
||||
spec:
|
||||
replicas: {{ .Values.replicaCount }}
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "synapbus.selectorLabels" . | nindent 6 }}
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "synapbus.selectorLabels" . | nindent 8 }}
|
||||
spec:
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
containers:
|
||||
- name: {{ .Chart.Name }}
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
args:
|
||||
- serve
|
||||
- --host
|
||||
- "0.0.0.0"
|
||||
- --port
|
||||
- "8080"
|
||||
- --data
|
||||
- /data
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 8080
|
||||
protocol: TCP
|
||||
env:
|
||||
{{- range $key, $value := .Values.env }}
|
||||
- name: {{ $key }}
|
||||
value: {{ $value | quote }}
|
||||
{{- end }}
|
||||
{{- with .Values.envFrom }}
|
||||
envFrom:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /readyz
|
||||
port: http
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 5
|
||||
resources:
|
||||
{{- toYaml .Values.resources | nindent 12 }}
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
- name: run
|
||||
mountPath: /tmp
|
||||
volumes:
|
||||
- name: data
|
||||
{{- if .Values.persistence.enabled }}
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ include "synapbus.fullname" . }}
|
||||
{{- else }}
|
||||
emptyDir: {}
|
||||
{{- end }}
|
||||
- name: run
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 1Mi
|
||||
{{- with .Values.nodeSelector }}
|
||||
nodeSelector:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.affinity }}
|
||||
affinity:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.tolerations }}
|
||||
tolerations:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
@@ -1,41 +0,0 @@
|
||||
{{- if .Values.ingress.enabled }}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {{ include "synapbus.fullname" . }}
|
||||
labels:
|
||||
{{- include "synapbus.labels" . | nindent 4 }}
|
||||
{{- with .Values.ingress.annotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- if .Values.ingress.className }}
|
||||
ingressClassName: {{ .Values.ingress.className }}
|
||||
{{- end }}
|
||||
{{- if .Values.ingress.tls }}
|
||||
tls:
|
||||
{{- range .Values.ingress.tls }}
|
||||
- hosts:
|
||||
{{- range .hosts }}
|
||||
- {{ . | quote }}
|
||||
{{- end }}
|
||||
secretName: {{ .secretName }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
rules:
|
||||
{{- range .Values.ingress.hosts }}
|
||||
- host: {{ .host | quote }}
|
||||
http:
|
||||
paths:
|
||||
{{- range .paths }}
|
||||
- path: {{ .path }}
|
||||
pathType: {{ .pathType }}
|
||||
backend:
|
||||
service:
|
||||
name: {{ include "synapbus.fullname" $ }}
|
||||
port:
|
||||
name: http
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -1,17 +0,0 @@
|
||||
{{- if .Values.persistence.enabled }}
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: {{ include "synapbus.fullname" . }}
|
||||
labels:
|
||||
{{- include "synapbus.labels" . | nindent 4 }}
|
||||
spec:
|
||||
accessModes:
|
||||
{{- toYaml .Values.persistence.accessModes | nindent 4 }}
|
||||
{{- if .Values.persistence.storageClass }}
|
||||
storageClassName: {{ .Values.persistence.storageClass | quote }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.persistence.size }}
|
||||
{{- end }}
|
||||
@@ -1,18 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "synapbus.fullname" . }}
|
||||
labels:
|
||||
{{- include "synapbus.labels" . | nindent 4 }}
|
||||
spec:
|
||||
type: {{ .Values.service.type }}
|
||||
ports:
|
||||
- port: {{ .Values.service.port }}
|
||||
targetPort: http
|
||||
protocol: TCP
|
||||
name: http
|
||||
{{- if and (eq .Values.service.type "NodePort") .Values.service.nodePort }}
|
||||
nodePort: {{ .Values.service.nodePort }}
|
||||
{{- end }}
|
||||
selector:
|
||||
{{- include "synapbus.selectorLabels" . | nindent 4 }}
|
||||
@@ -1,20 +0,0 @@
|
||||
{{- if .Values.metrics.enabled }}
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: ServiceMonitor
|
||||
metadata:
|
||||
name: {{ include "synapbus.fullname" . }}
|
||||
labels:
|
||||
{{- include "synapbus.labels" . | nindent 4 }}
|
||||
{{- with .Values.metrics.serviceMonitor.additionalLabels }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "synapbus.selectorLabels" . | nindent 6 }}
|
||||
endpoints:
|
||||
- port: http
|
||||
path: /metrics
|
||||
interval: {{ .Values.metrics.serviceMonitor.interval }}
|
||||
scrapeTimeout: {{ .Values.metrics.serviceMonitor.scrapeTimeout }}
|
||||
{{- end }}
|
||||
@@ -1,66 +0,0 @@
|
||||
replicaCount: 1
|
||||
|
||||
image:
|
||||
repository: ghcr.io/synapbus/synapbus
|
||||
pullPolicy: IfNotPresent
|
||||
tag: "latest"
|
||||
|
||||
imagePullSecrets: []
|
||||
nameOverride: ""
|
||||
fullnameOverride: ""
|
||||
|
||||
serviceAccount:
|
||||
create: false
|
||||
name: ""
|
||||
|
||||
service:
|
||||
type: ClusterIP
|
||||
port: 8080
|
||||
|
||||
ingress:
|
||||
enabled: false
|
||||
className: ""
|
||||
annotations: {}
|
||||
# kubernetes.io/ingress.class: nginx
|
||||
# cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||
hosts:
|
||||
- host: synapbus.local
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
tls: []
|
||||
# - secretName: synapbus-tls
|
||||
# hosts:
|
||||
# - synapbus.local
|
||||
|
||||
persistence:
|
||||
enabled: true
|
||||
storageClass: ""
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
size: 1Gi
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 256Mi
|
||||
|
||||
env:
|
||||
SYNAPBUS_LOG_LEVEL: info
|
||||
SYNAPBUS_METRICS: "true"
|
||||
|
||||
metrics:
|
||||
enabled: true
|
||||
serviceMonitor:
|
||||
interval: 30s
|
||||
scrapeTimeout: 10s
|
||||
additionalLabels: {}
|
||||
|
||||
nodeSelector: {}
|
||||
|
||||
tolerations: []
|
||||
|
||||
affinity: {}
|
||||
@@ -0,0 +1,71 @@
|
||||
# SynapBus on kubic
|
||||
|
||||
Plain Kubernetes manifests for the kubic single-node MicroK8s cluster
|
||||
(`kubic.home.arpa`). No Helm — the image is built locally, imported directly
|
||||
into MicroK8s containerd, and rolled with `kubectl set image`.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `namespace.yaml` | `synapbus` namespace |
|
||||
| `pvc.yaml` | 2 Gi PVC on `microk8s-hostpath` for `/data` (DB + WAL + attachments + HNSW index) |
|
||||
| `secret.example.yaml` | Template for `synapbus-secrets` (OpenAI/Gemini keys, mounted via `envFrom`) |
|
||||
| `deployment.yaml` | Single replica, `docker.io/library/synapbus:vX.Y.Z-amd64`, `imagePullPolicy: IfNotPresent` (image is pre-loaded into containerd) |
|
||||
| `service.yaml` | NodePort 30088 on port 8080 |
|
||||
| `otel-collector.yaml` | OpenTelemetry collector for traces/metrics |
|
||||
|
||||
## Initial install
|
||||
|
||||
```sh
|
||||
kubectl apply -f deploy/kubic/namespace.yaml
|
||||
kubectl apply -f deploy/kubic/pvc.yaml
|
||||
# Edit secret.example.yaml first — never commit real keys.
|
||||
kubectl apply -f deploy/kubic/secret.example.yaml
|
||||
kubectl apply -f deploy/kubic/service.yaml
|
||||
kubectl apply -f deploy/kubic/deployment.yaml
|
||||
```
|
||||
|
||||
## Releasing a new version
|
||||
|
||||
```sh
|
||||
scripts/deploy-kubic.sh v0.17.0
|
||||
```
|
||||
|
||||
The script:
|
||||
|
||||
1. `docker buildx build --platform linux/amd64` with the version baked in.
|
||||
2. `docker save` to a tarball.
|
||||
3. `scp` to `kubic.home.arpa`.
|
||||
4. `ssh kubic 'sudo microk8s ctr image import …'` (loads the image into the
|
||||
in-cluster containerd registry — the image is *not* pushed to a remote
|
||||
registry).
|
||||
5. `kubectl set image deploy/synapbus synapbus=docker.io/library/synapbus:vX.Y.Z-amd64`.
|
||||
6. `kubectl rollout status …` and a `/healthz` smoke test.
|
||||
|
||||
The `docker.io/library/` prefix is required because that's how containerd
|
||||
resolves image references that don't specify a registry — `synapbus:v…`
|
||||
written into the deployment is normalised to `docker.io/library/synapbus:v…`
|
||||
on the node.
|
||||
|
||||
## Why no Helm?
|
||||
|
||||
The original chart under `deploy/helm/` (since deleted) was used for the very
|
||||
first install (Mar 2026) and then went into a `failed` state when someone
|
||||
ran `kubectl set image` for a hotfix; subsequent `helm upgrade` attempts hit
|
||||
server-side-apply ownership conflicts. Rather than reconcile, we now own the
|
||||
manifests directly. The deploy flow is simple enough that templating buys
|
||||
nothing.
|
||||
|
||||
## Backups
|
||||
|
||||
Before any version that touches schema, snapshot `/data`:
|
||||
|
||||
```sh
|
||||
kubectl exec -n synapbus deploy/synapbus -- \
|
||||
tar -C /data -cf - synapbus.db synapbus.db-shm synapbus.db-wal vapid_keys.json \
|
||||
| tar -xf - -C "$HOME/synapbus-backups/$(date -u +%Y%m%dT%H%M%SZ)/"
|
||||
```
|
||||
|
||||
Then `sqlite3 synapbus.db 'PRAGMA wal_checkpoint(TRUNCATE); PRAGMA integrity_check;'`
|
||||
to fold the WAL into the main file and verify integrity before archiving.
|
||||
@@ -0,0 +1,77 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: synapbus
|
||||
namespace: synapbus
|
||||
labels:
|
||||
app.kubernetes.io/name: synapbus
|
||||
app.kubernetes.io/instance: synapbus
|
||||
spec:
|
||||
replicas: 1
|
||||
revisionHistoryLimit: 10
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxSurge: 25%
|
||||
maxUnavailable: 25%
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: synapbus
|
||||
app.kubernetes.io/instance: synapbus
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: synapbus
|
||||
app.kubernetes.io/instance: synapbus
|
||||
spec:
|
||||
containers:
|
||||
- name: synapbus
|
||||
image: docker.io/library/synapbus:v0.17.0-amd64
|
||||
imagePullPolicy: IfNotPresent
|
||||
args: ["serve", "--host", "0.0.0.0", "--port", "8080", "--data", "/data"]
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 8080
|
||||
protocol: TCP
|
||||
env:
|
||||
- name: SYNAPBUS_BASE_URL
|
||||
value: auto
|
||||
- name: SYNAPBUS_EMBEDDING_PROVIDER
|
||||
value: openai
|
||||
- name: SYNAPBUS_LOG_LEVEL
|
||||
value: info
|
||||
- name: SYNAPBUS_MESSAGE_RETENTION
|
||||
value: "0"
|
||||
- name: SYNAPBUS_METRICS
|
||||
value: "true"
|
||||
envFrom:
|
||||
- secretRef:
|
||||
name: synapbus-secrets
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: http
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /readyz
|
||||
port: http
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 5
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: 512Mi
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: synapbus
|
||||
@@ -0,0 +1,697 @@
|
||||
{
|
||||
"annotations": {
|
||||
"list": [
|
||||
{
|
||||
"name": "Annotations & Alerts",
|
||||
"datasource": {
|
||||
"type": "grafana",
|
||||
"uid": "-- Grafana --"
|
||||
},
|
||||
"enable": true,
|
||||
"hide": true,
|
||||
"iconColor": "rgba(0, 211, 255, 1)",
|
||||
"type": "dashboard"
|
||||
},
|
||||
{
|
||||
"name": "Circuit breaker trips",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"enable": true,
|
||||
"iconColor": "red",
|
||||
"expr": "changes(synapbus_dream_circuit_broken_total[5m]) > 0",
|
||||
"step": "60s",
|
||||
"titleFormat": "Circuit broken: {{reason}}",
|
||||
"tagKeys": "owner,reason",
|
||||
"textFormat": "owner={{owner}} reason={{reason}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"description": "Visualizes dream worker activity, token budgets, circuit breaker trips, and proactive memory injection emitted by SynapBus feature 020.",
|
||||
"editable": true,
|
||||
"fiscalYearStartMonth": 0,
|
||||
"graphTooltip": 1,
|
||||
"id": null,
|
||||
"links": [
|
||||
{
|
||||
"title": "SynapBus Web UI",
|
||||
"url": "http://kubic.home.arpa:30088",
|
||||
"type": "link",
|
||||
"icon": "external link",
|
||||
"tooltip": "Open SynapBus Web UI",
|
||||
"targetBlank": true,
|
||||
"tags": []
|
||||
}
|
||||
],
|
||||
"panels": [
|
||||
{
|
||||
"type": "row",
|
||||
"id": 100,
|
||||
"title": "Dream worker activity",
|
||||
"collapsed": false,
|
||||
"gridPos": {"h": 1, "w": 24, "x": 0, "y": 0},
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"id": 1,
|
||||
"type": "timeseries",
|
||||
"title": "Jobs/hour by type",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 8, "x": 0, "y": 1},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["mean", "max"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (job_type) (rate(synapbus_dream_jobs_total{owner=~\"$owner\",job_type=~\"$job_type\"}[5m]) * 3600)",
|
||||
"legendFormat": "{{job_type}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"type": "timeseries",
|
||||
"title": "Jobs/hour by status (stacked)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 8, "x": 8, "y": 1},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 1,
|
||||
"fillOpacity": 60,
|
||||
"stacking": {"mode": "normal", "group": "A"},
|
||||
"showPoints": "never"
|
||||
},
|
||||
"color": {"mode": "palette-classic"}
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {"id": "byName", "options": "succeeded"},
|
||||
"properties": [{"id": "color", "value": {"mode": "fixed", "fixedColor": "green"}}]
|
||||
},
|
||||
{
|
||||
"matcher": {"id": "byName", "options": "failed"},
|
||||
"properties": [{"id": "color", "value": {"mode": "fixed", "fixedColor": "red"}}]
|
||||
},
|
||||
{
|
||||
"matcher": {"id": "byName", "options": "circuit_broken"},
|
||||
"properties": [{"id": "color", "value": {"mode": "fixed", "fixedColor": "orange"}}]
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["sum"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (status) (rate(synapbus_dream_jobs_total{owner=~\"$owner\",job_type=~\"$job_type\"}[5m]) * 3600)",
|
||||
"legendFormat": "{{status}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"type": "timeseries",
|
||||
"title": "Job duration p50 / p95 (s)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 8, "x": 16, "y": 1},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "s",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 5,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["mean", "max"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "histogram_quantile(0.5, sum by (job_type, le) (rate(synapbus_dream_job_duration_seconds_bucket{owner=~\"$owner\",job_type=~\"$job_type\"}[5m])))",
|
||||
"legendFormat": "p50 {{job_type}}"
|
||||
},
|
||||
{
|
||||
"refId": "B",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "histogram_quantile(0.95, sum by (job_type, le) (rate(synapbus_dream_job_duration_seconds_bucket{owner=~\"$owner\",job_type=~\"$job_type\"}[5m])))",
|
||||
"legendFormat": "p95 {{job_type}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"id": 101,
|
||||
"title": "Token usage vs limit",
|
||||
"collapsed": false,
|
||||
"gridPos": {"h": 1, "w": 24, "x": 0, "y": 9},
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"type": "stat",
|
||||
"title": "Daily tokens IN by owner (24h)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 7, "w": 8, "x": 0, "y": 10},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"decimals": 0,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"color": "green", "value": null},
|
||||
{"color": "yellow", "value": 700000},
|
||||
{"color": "red", "value": 1000000}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"reduceOptions": {"values": false, "calcs": ["lastNotNull"], "fields": ""},
|
||||
"orientation": "auto",
|
||||
"textMode": "value_and_name",
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "auto",
|
||||
"showPercentChange": false
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (owner) (increase(synapbus_dream_tokens_total{direction=\"in\",owner=~\"$owner\"}[24h]))",
|
||||
"legendFormat": "{{owner}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"type": "stat",
|
||||
"title": "Daily tokens OUT by owner (24h)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 7, "w": 8, "x": 8, "y": 10},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"decimals": 0,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"color": "green", "value": null},
|
||||
{"color": "yellow", "value": 140000},
|
||||
{"color": "red", "value": 200000}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"reduceOptions": {"values": false, "calcs": ["lastNotNull"], "fields": ""},
|
||||
"orientation": "auto",
|
||||
"textMode": "value_and_name",
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "auto",
|
||||
"showPercentChange": false
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (owner) (increase(synapbus_dream_tokens_total{direction=\"out\",owner=~\"$owner\"}[24h]))",
|
||||
"legendFormat": "{{owner}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 6,
|
||||
"type": "timeseries",
|
||||
"title": "Token usage (15-min windows)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 7, "w": 8, "x": 16, "y": 10},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["mean", "max"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (owner, direction) (rate(synapbus_dream_tokens_total{owner=~\"$owner\"}[15m]) * 900)",
|
||||
"legendFormat": "{{owner}} / {{direction}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"id": 102,
|
||||
"title": "Circuit breaker",
|
||||
"collapsed": false,
|
||||
"gridPos": {"h": 1, "w": 24, "x": 0, "y": 17},
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"id": 7,
|
||||
"type": "stat",
|
||||
"title": "Circuit-breaker trips (24h)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 7, "w": 8, "x": 0, "y": 18},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"decimals": 0,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"color": "green", "value": null},
|
||||
{"color": "orange", "value": 1},
|
||||
{"color": "red", "value": 5}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"reduceOptions": {"values": false, "calcs": ["lastNotNull"], "fields": ""},
|
||||
"orientation": "auto",
|
||||
"textMode": "value_and_name",
|
||||
"colorMode": "value",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"showPercentChange": false
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (reason) (increase(synapbus_dream_circuit_broken_total{owner=~\"$owner\"}[24h]))",
|
||||
"legendFormat": "{{reason}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 8,
|
||||
"type": "state-timeline",
|
||||
"title": "Circuit-breaker events timeline",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 7, "w": 16, "x": 8, "y": 18},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"custom": {
|
||||
"lineWidth": 0,
|
||||
"fillOpacity": 70
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"color": "green", "value": null},
|
||||
{"color": "red", "value": 1}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"mergeValues": true,
|
||||
"showValue": "auto",
|
||||
"alignValue": "left",
|
||||
"rowHeight": 0.9,
|
||||
"legend": {"displayMode": "list", "placement": "bottom", "showLegend": true},
|
||||
"tooltip": {"mode": "single", "sort": "none"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (owner, reason) (rate(synapbus_dream_circuit_broken_total{owner=~\"$owner\"}[5m])) > 0",
|
||||
"legendFormat": "{{owner}} / {{reason}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"id": 103,
|
||||
"title": "Injection layer",
|
||||
"collapsed": false,
|
||||
"gridPos": {"h": 1, "w": 24, "x": 0, "y": 25},
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"id": 9,
|
||||
"type": "timeseries",
|
||||
"title": "Injection packets/hr by tool (stacked)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 12, "x": 0, "y": 26},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 1,
|
||||
"fillOpacity": 60,
|
||||
"stacking": {"mode": "normal", "group": "A"},
|
||||
"showPoints": "never"
|
||||
},
|
||||
"color": {"mode": "palette-classic"}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["mean", "sum"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (tool) (rate(synapbus_injection_packets_total{tool=~\"$tool\"}[5m]) * 3600)",
|
||||
"legendFormat": "{{tool}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 10,
|
||||
"type": "timeseries",
|
||||
"title": "Memories per packet (p50 / p95)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 12, "x": 12, "y": 26},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 5,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["mean", "max"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "histogram_quantile(0.5, sum by (tool, le) (rate(synapbus_injection_memories_per_packet_bucket{tool=~\"$tool\"}[5m])))",
|
||||
"legendFormat": "p50 {{tool}}"
|
||||
},
|
||||
{
|
||||
"refId": "B",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "histogram_quantile(0.95, sum by (tool, le) (rate(synapbus_injection_memories_per_packet_bucket{tool=~\"$tool\"}[5m])))",
|
||||
"legendFormat": "p95 {{tool}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 11,
|
||||
"type": "timeseries",
|
||||
"title": "Packet size (chars) p95",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 12, "x": 0, "y": 34},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["mean", "max"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "histogram_quantile(0.95, sum by (tool, le) (rate(synapbus_injection_packet_chars_bucket{tool=~\"$tool\"}[5m])))",
|
||||
"legendFormat": "p95 {{tool}}"
|
||||
},
|
||||
{
|
||||
"refId": "B",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "histogram_quantile(0.5, sum by (tool, le) (rate(synapbus_injection_packet_chars_bucket{tool=~\"$tool\"}[5m])))",
|
||||
"legendFormat": "p50 {{tool}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 12,
|
||||
"type": "table",
|
||||
"title": "Injection skipped reasons (24h)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 12, "x": 12, "y": 34},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"custom": {
|
||||
"align": "auto",
|
||||
"displayMode": "auto",
|
||||
"inspect": false
|
||||
},
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"color": "green", "value": null}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {"id": "byName", "options": "Value"},
|
||||
"properties": [
|
||||
{"id": "custom.displayMode", "value": "gradient-gauge"},
|
||||
{"id": "custom.align", "value": "right"},
|
||||
{"id": "displayName", "value": "skipped (24h)"}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"showHeader": true,
|
||||
"sortBy": [{"displayName": "skipped (24h)", "desc": true}]
|
||||
},
|
||||
"transformations": [
|
||||
{
|
||||
"id": "organize",
|
||||
"options": {
|
||||
"excludeByName": {"Time": true, "__name__": true, "job": true, "instance": true},
|
||||
"indexByName": {},
|
||||
"renameByName": {}
|
||||
}
|
||||
}
|
||||
],
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (tool, reason) (increase(synapbus_injection_skipped_total{tool=~\"$tool\"}[24h]))",
|
||||
"format": "table",
|
||||
"instant": true
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "row",
|
||||
"id": 104,
|
||||
"title": "MCP transport health",
|
||||
"collapsed": false,
|
||||
"gridPos": {"h": 1, "w": 24, "x": 0, "y": 42},
|
||||
"panels": []
|
||||
},
|
||||
{
|
||||
"id": 13,
|
||||
"type": "timeseries",
|
||||
"title": "MCP request rate (req/s)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 12, "x": 0, "y": 43},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "reqps",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["mean", "max"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "sum by (path,status) (rate(synapbus_http_requests_total{path=~\".*mcp.*\"}[5m]))",
|
||||
"legendFormat": "{{path}} {{status}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 14,
|
||||
"type": "timeseries",
|
||||
"title": "MCP latency p95 (s)",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"gridPos": {"h": 8, "w": 12, "x": 12, "y": 43},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "s",
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 5,
|
||||
"showPoints": "never"
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"options": {
|
||||
"legend": {"displayMode": "table", "placement": "bottom", "showLegend": true, "calcs": ["mean", "max"]},
|
||||
"tooltip": {"mode": "multi", "sort": "desc"}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"expr": "histogram_quantile(0.95, sum by (le) (rate(synapbus_http_request_duration_seconds_bucket{path=~\".*mcp.*\"}[5m])))",
|
||||
"legendFormat": "p95"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"refresh": "30s",
|
||||
"schemaVersion": 39,
|
||||
"tags": ["synapbus", "dream", "memory", "feature-020"],
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"name": "DS_PROMETHEUS",
|
||||
"label": "Prometheus",
|
||||
"type": "datasource",
|
||||
"query": "prometheus",
|
||||
"refresh": 1,
|
||||
"current": {},
|
||||
"hide": 0,
|
||||
"includeAll": false,
|
||||
"multi": false,
|
||||
"options": [],
|
||||
"regex": "",
|
||||
"skipUrlSync": false
|
||||
},
|
||||
{
|
||||
"name": "owner",
|
||||
"label": "Owner",
|
||||
"type": "query",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"definition": "label_values(synapbus_dream_jobs_total, owner)",
|
||||
"query": {"query": "label_values(synapbus_dream_jobs_total, owner)", "refId": "StandardVariableQuery"},
|
||||
"refresh": 2,
|
||||
"regex": "",
|
||||
"sort": 1,
|
||||
"multi": true,
|
||||
"includeAll": true,
|
||||
"allValue": ".*",
|
||||
"current": {"selected": true, "text": ["All"], "value": ["$__all"]},
|
||||
"options": [],
|
||||
"hide": 0,
|
||||
"skipUrlSync": false
|
||||
},
|
||||
{
|
||||
"name": "job_type",
|
||||
"label": "Job type",
|
||||
"type": "query",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"definition": "label_values(synapbus_dream_jobs_total, job_type)",
|
||||
"query": {"query": "label_values(synapbus_dream_jobs_total, job_type)", "refId": "StandardVariableQuery"},
|
||||
"refresh": 2,
|
||||
"regex": "",
|
||||
"sort": 1,
|
||||
"multi": true,
|
||||
"includeAll": true,
|
||||
"allValue": ".*",
|
||||
"current": {"selected": true, "text": ["All"], "value": ["$__all"]},
|
||||
"options": [],
|
||||
"hide": 0,
|
||||
"skipUrlSync": false
|
||||
},
|
||||
{
|
||||
"name": "tool",
|
||||
"label": "Tool",
|
||||
"type": "query",
|
||||
"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"},
|
||||
"definition": "label_values(synapbus_injection_packets_total, tool)",
|
||||
"query": {"query": "label_values(synapbus_injection_packets_total, tool)", "refId": "StandardVariableQuery"},
|
||||
"refresh": 2,
|
||||
"regex": "",
|
||||
"sort": 1,
|
||||
"multi": true,
|
||||
"includeAll": true,
|
||||
"allValue": ".*",
|
||||
"current": {"selected": true, "text": ["All"], "value": ["$__all"]},
|
||||
"options": [],
|
||||
"hide": 0,
|
||||
"skipUrlSync": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"time": {"from": "now-6h", "to": "now"},
|
||||
"timepicker": {},
|
||||
"timezone": "",
|
||||
"title": "SynapBus — Dream Worker & Memory Injection",
|
||||
"uid": "synapbus-dream-memory-020",
|
||||
"version": 1,
|
||||
"weekStart": ""
|
||||
}
|
||||
Executable
+32
@@ -0,0 +1,32 @@
|
||||
#!/bin/bash
|
||||
# Imports the SynapBus dream worker dashboard into Grafana.
|
||||
# Usage:
|
||||
# GRAFANA_PASS=... ./import.sh
|
||||
# GRAFANA_URL=http://grafana.example:3000 GRAFANA_USER=admin GRAFANA_PASS=... ./import.sh
|
||||
set -euo pipefail
|
||||
|
||||
GRAFANA_URL="${GRAFANA_URL:-http://kubic.home.arpa:30083}"
|
||||
GRAFANA_USER="${GRAFANA_USER:-admin}"
|
||||
GRAFANA_PASS="${GRAFANA_PASS:?need GRAFANA_PASS}"
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
DASH_FILE="${SCRIPT_DIR}/dream-dashboard.json"
|
||||
|
||||
[ -f "$DASH_FILE" ] || { echo "dashboard JSON not found: $DASH_FILE" >&2; exit 1; }
|
||||
|
||||
DS_UID=$(curl -fsS -u "$GRAFANA_USER:$GRAFANA_PASS" "$GRAFANA_URL/api/datasources" \
|
||||
| jq -r '.[] | select(.type=="prometheus") | .uid' | head -1)
|
||||
[ -z "$DS_UID" ] && { echo "no prometheus datasource found in $GRAFANA_URL" >&2; exit 1; }
|
||||
echo "Using Prometheus DS uid=$DS_UID" >&2
|
||||
|
||||
DASHBOARD=$(jq --arg uid "$DS_UID" '
|
||||
(.. | objects | select(.type? == "prometheus") | .uid) |= $uid
|
||||
| .id = null
|
||||
| . as $dash | { dashboard: $dash, overwrite: true, message: "feat(020): dream worker + memory injection dashboard" }
|
||||
' "$DASH_FILE")
|
||||
|
||||
curl -fsS -u "$GRAFANA_USER:$GRAFANA_PASS" \
|
||||
-H "Content-Type: application/json" \
|
||||
-X POST "$GRAFANA_URL/api/dashboards/db" \
|
||||
-d "$DASHBOARD"
|
||||
echo
|
||||
@@ -0,0 +1,4 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: synapbus
|
||||
@@ -0,0 +1,15 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: synapbus
|
||||
namespace: synapbus
|
||||
labels:
|
||||
app.kubernetes.io/name: synapbus
|
||||
app.kubernetes.io/instance: synapbus
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: microk8s-hostpath
|
||||
resources:
|
||||
requests:
|
||||
storage: 2Gi
|
||||
@@ -0,0 +1,9 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: synapbus-secrets
|
||||
namespace: synapbus
|
||||
type: Opaque
|
||||
stringData:
|
||||
OPENAI_API_KEY: "sk-..."
|
||||
GEMINI_API_KEY: ""
|
||||
@@ -0,0 +1,19 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: synapbus
|
||||
namespace: synapbus
|
||||
labels:
|
||||
app.kubernetes.io/name: synapbus
|
||||
app.kubernetes.io/instance: synapbus
|
||||
spec:
|
||||
type: NodePort
|
||||
selector:
|
||||
app.kubernetes.io/name: synapbus
|
||||
app.kubernetes.io/instance: synapbus
|
||||
ports:
|
||||
- name: http
|
||||
port: 8080
|
||||
targetPort: http
|
||||
nodePort: 30088
|
||||
protocol: TCP
|
||||
@@ -0,0 +1,8 @@
|
||||
FROM alpine:3.21
|
||||
RUN apk add --no-cache bash curl ca-certificates
|
||||
RUN curl -fsSL -o /usr/local/bin/kubectl \
|
||||
https://dl.k8s.io/release/v1.30.5/bin/linux/amd64/kubectl \
|
||||
&& chmod +x /usr/local/bin/kubectl \
|
||||
&& kubectl version --client
|
||||
WORKDIR /scripts
|
||||
ENTRYPOINT ["/bin/bash"]
|
||||
@@ -0,0 +1,172 @@
|
||||
# synapbus-watchdog: hourly k8s CronJob that checks dream-worker health
|
||||
# and scales synapbus/synapbus to 0 replicas if any red-flag trips.
|
||||
# Goal: prevent runaway Claude Code token drain while Algis is AFK.
|
||||
#
|
||||
# Cadence: every hour at :05 past (covers the requested +2h and +4h
|
||||
# horizons and keeps catching problems indefinitely until disabled).
|
||||
#
|
||||
# Disable with:
|
||||
# microk8s kubectl -n synapbus patch cronjob synapbus-watchdog \
|
||||
# -p '{"spec":{"suspend":true}}'
|
||||
#
|
||||
# Stop manually:
|
||||
# microk8s kubectl -n synapbus delete cronjob synapbus-watchdog
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: synapbus-watchdog
|
||||
namespace: synapbus
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: synapbus-watchdog
|
||||
namespace: synapbus
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["pods", "pods/exec"]
|
||||
verbs: ["get", "list", "create"]
|
||||
- apiGroups: ["apps"]
|
||||
resources: ["deployments", "deployments/scale"]
|
||||
verbs: ["get", "patch", "update"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: synapbus-watchdog
|
||||
namespace: synapbus
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: synapbus-watchdog
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: synapbus-watchdog
|
||||
namespace: synapbus
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: synapbus-watchdog-script
|
||||
namespace: synapbus
|
||||
data:
|
||||
watchdog.sh: |
|
||||
#!/bin/bash
|
||||
set -uo pipefail
|
||||
NS=synapbus
|
||||
DEPLOY=synapbus
|
||||
LOG_PREFIX="[watchdog $(date -u +%FT%TZ)]"
|
||||
|
||||
log() { echo "$LOG_PREFIX $*"; }
|
||||
fail() { log "RED-FLAG: $*"; STOP=1; STOP_REASON="$*"; }
|
||||
STOP=0
|
||||
STOP_REASON=""
|
||||
|
||||
# 1) Pod state
|
||||
POD=$(kubectl -n $NS get pod -l app.kubernetes.io/name=synapbus \
|
||||
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null)
|
||||
if [ -z "$POD" ]; then fail "no synapbus pod"; else
|
||||
READY=$(kubectl -n $NS get pod "$POD" \
|
||||
-o jsonpath='{.status.containerStatuses[?(@.name=="synapbus")].ready}')
|
||||
RESTARTS=$(kubectl -n $NS get pod "$POD" \
|
||||
-o jsonpath='{.status.containerStatuses[?(@.name=="synapbus")].restartCount}')
|
||||
log "pod=$POD ready=$READY restarts=$RESTARTS"
|
||||
[ "$READY" = "true" ] || fail "pod not ready"
|
||||
[ "${RESTARTS:-0}" -le 3 ] || fail "restart count $RESTARTS > 3"
|
||||
fi
|
||||
|
||||
# 2) Dream-job hourly aggregate
|
||||
if [ -n "$POD" ]; then
|
||||
ROW=$(kubectl -n $NS exec "$POD" -- sqlite3 /data/synapbus.db \
|
||||
"SELECT COALESCE(SUM(CASE WHEN status='succeeded' THEN 1 ELSE 0 END),0), \
|
||||
COALESCE(SUM(CASE WHEN status='failed' THEN 1 ELSE 0 END),0), \
|
||||
COALESCE(SUM(CASE WHEN status IN ('running','dispatched','pending') THEN 1 ELSE 0 END),0), \
|
||||
COALESCE(COUNT(*),0) \
|
||||
FROM memory_consolidation_jobs \
|
||||
WHERE created_at > datetime('now','-1 hour');" 2>/dev/null \
|
||||
| tr '|' ' ')
|
||||
SUCC=$(echo "$ROW" | awk '{print $1}')
|
||||
FAIL=$(echo "$ROW" | awk '{print $2}')
|
||||
INFL=$(echo "$ROW" | awk '{print $3}')
|
||||
TOTAL=$(echo "$ROW" | awk '{print $4}')
|
||||
log "last_1h jobs total=$TOTAL succ=$SUCC fail=$FAIL in_flight=$INFL"
|
||||
[ "${FAIL:-0}" -le 20 ] || fail "failed jobs in last 1h = $FAIL > 20"
|
||||
fi
|
||||
|
||||
# 3) Today's usage — aggregate across all owners (caps are global,
|
||||
# not per-owner; owner_id is just a partition key in the table)
|
||||
if [ -n "$POD" ]; then
|
||||
U=$(kubectl -n $NS exec "$POD" -- sqlite3 /data/synapbus.db \
|
||||
"SELECT COALESCE(SUM(jobs_started),0), COALESCE(SUM(tokens_in),0), \
|
||||
COALESCE(SUM(jobs_succeeded),0), COALESCE(SUM(jobs_failed),0), \
|
||||
COALESCE(SUM(jobs_circuit_broken),0) \
|
||||
FROM memory_dream_usage WHERE date=date('now');" 2>/dev/null \
|
||||
| tr '|' ' ')
|
||||
JS=$(echo "$U" | awk '{print $1}'); JS=${JS:-0}
|
||||
TIN=$(echo "$U" | awk '{print $2}'); TIN=${TIN:-0}
|
||||
JOK=$(echo "$U" | awk '{print $3}'); JOK=${JOK:-0}
|
||||
JFL=$(echo "$U" | awk '{print $4}'); JFL=${JFL:-0}
|
||||
JCB=$(echo "$U" | awk '{print $5}'); JCB=${JCB:-0}
|
||||
log "today: jobs_started=$JS tokens_in=$TIN succeeded=$JOK failed=$JFL circuit_broken=$JCB"
|
||||
[ "$JS" -le 200 ] || fail "jobs_started today $JS > 200 (soft cap)"
|
||||
[ "$TIN" -le 30000000 ] || fail "tokens_in today $TIN > 30M (budget cliff)"
|
||||
# "still firing despite breaker": more started than completed by >5
|
||||
DELTA=$((JS - JOK - JFL - JCB))
|
||||
if [ "$JCB" -gt 0 ] && [ "$DELTA" -gt 5 ]; then
|
||||
fail "circuit broke but still firing (started=$JS, completed_or_broken=$((JOK+JFL+JCB)), delta=$DELTA)"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Act
|
||||
if [ "$STOP" = "1" ]; then
|
||||
log "STOPPING synapbus: $STOP_REASON"
|
||||
kubectl -n $NS scale deploy/$DEPLOY --replicas=0
|
||||
log "synapbus scaled to 0 replicas. Re-enable with: kubectl -n $NS scale deploy/$DEPLOY --replicas=1"
|
||||
exit 2
|
||||
fi
|
||||
log "HEALTHY — no action"
|
||||
exit 0
|
||||
---
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: synapbus-watchdog
|
||||
namespace: synapbus
|
||||
spec:
|
||||
schedule: "5 * * * *" # every hour at :05 past (UTC)
|
||||
concurrencyPolicy: Forbid
|
||||
successfulJobsHistoryLimit: 6
|
||||
failedJobsHistoryLimit: 6
|
||||
startingDeadlineSeconds: 600
|
||||
jobTemplate:
|
||||
spec:
|
||||
backoffLimit: 0
|
||||
ttlSecondsAfterFinished: 86400
|
||||
activeDeadlineSeconds: 180
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: synapbus-watchdog
|
||||
restartPolicy: Never
|
||||
containers:
|
||||
- name: watchdog
|
||||
image: docker.io/library/synapbus-watchdog:v1
|
||||
imagePullPolicy: Never
|
||||
command: ["/bin/bash", "/scripts/watchdog.sh"]
|
||||
volumeMounts:
|
||||
- name: script
|
||||
mountPath: /scripts
|
||||
readOnly: true
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 128Mi
|
||||
volumes:
|
||||
- name: script
|
||||
configMap:
|
||||
name: synapbus-watchdog-script
|
||||
defaultMode: 0755
|
||||
@@ -0,0 +1,102 @@
|
||||
# Internal-only mode: remove approvals & escalations
|
||||
|
||||
**Date:** 2026-05-10
|
||||
**Status:** Design
|
||||
**Owner:** Algis
|
||||
|
||||
## Problem
|
||||
|
||||
SynapBus today assumes a human is in the loop: a stalemate worker DMs reminders after 4h, escalates to `#approvals` after 48h, and the dynamic-agent-spawning flow (spec 018) gates new agents and task trees on human approval. In practice the user is the only operator, the approval queue stalls, and the volume of reminder/escalation messages drowns out signal. The user is moving to a single daily summary (separate `#summary-daily` channel + summarizer agent already in progress) and treats SynapBus as an internal-only comms + data store — nothing publishes externally.
|
||||
|
||||
The goal is to remove the human-in-the-loop surfaces so the message stream stops generating noise the user will never read.
|
||||
|
||||
## Scope
|
||||
|
||||
### Removed
|
||||
|
||||
1. **Stalemate reminders** — `StalemateWorker.sendPendingReminders` and supporting helpers (`reminderExists`, the 4h ReminderAfter knob).
|
||||
2. **Stalemate escalations** — `StalemateWorker.escalatePendingMessages` and `checkWorkflowStalemates`, plus the 48h EscalateAfter knob and `#approvals` lookup path.
|
||||
3. **`propose_agent` MCP tool** (spec 018). It writes a `pending` row to `agent_proposals` for human approval via `#approvals` and there is no automated consumer of that table. Removing the tool leaves agent creation to the admin CLI, which matches the internal-only stance.
|
||||
|
||||
**Note:** `propose_task_tree` is intentionally KEPT despite its name — it is not an approval gate. It directly inserts tasks in `approved` status and auto-transitions the goal to `active`. Removing it would break the spec-018 goal/task flow.
|
||||
4. *(Reactions service intentionally untouched — it's a generic workflow primitive that also drives trust adjustments. Once no upstream feature creates approval-bearing messages, the `approve` / `reject` reaction paths become dormant on their own.)*
|
||||
|
||||
### Kept
|
||||
|
||||
- **`StalemateWorker.ProcessingTimeout`** (24h auto-fail of claimed-but-abandoned messages). Protects the inbox from crashed agents; not human-facing.
|
||||
- **The `#approvals` channel row** in the `channels` table. Cheaper to leave than to migrate; user can drop via admin CLI later.
|
||||
- **Webhook / K8s runner approval gates** (spec 003). User confirmed these are out of scope.
|
||||
- **Trust system** (spec 011). No approval surface, just delegation.
|
||||
|
||||
### One-shot DB cleanup
|
||||
|
||||
New migration `internal/storage/schema/027_remove_approval_noise.sql`:
|
||||
|
||||
```sql
|
||||
-- Drop reminder and escalation system DMs.
|
||||
DELETE FROM messages
|
||||
WHERE subject LIKE 'stalemate-reminder:%'
|
||||
OR subject LIKE 'stalemate-escalation:%';
|
||||
|
||||
-- Drop everything in the #approvals channel.
|
||||
DELETE FROM messages
|
||||
WHERE channel_id = (SELECT id FROM channels WHERE name = 'approvals');
|
||||
|
||||
-- Drop pending agent proposals (table itself stays for reversibility).
|
||||
DELETE FROM agent_proposals;
|
||||
```
|
||||
|
||||
`VACUUM` cannot run inside a migration transaction, so reclaiming disk is a separate `synapbus admin vacuum` command (or a manual `kubectl exec ... sqlite3 ... 'VACUUM;'`). Out of scope for this change unless trivial to wire up.
|
||||
|
||||
## Architecture impact
|
||||
|
||||
```
|
||||
Before:
|
||||
agent → MCP propose_agent → agent_proposals row → human reacts in #approvals
|
||||
→ spawn or reject
|
||||
message claimed → StalemateWorker (every 15m) → 4h reminder DM
|
||||
→ 48h escalation to #approvals
|
||||
→ 24h auto-fail (KEEP)
|
||||
|
||||
After:
|
||||
agent → MCP create_agent (existing direct path) → agent registered
|
||||
message claimed → StalemateWorker (every 15m) → 24h auto-fail
|
||||
```
|
||||
|
||||
Net code deletion. No new components, no new config surface, no new dependencies.
|
||||
|
||||
## Components touched
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `internal/messaging/stalemate.go` | Delete `sendPendingReminders`, `escalatePendingMessages`, `checkWorkflowStalemates`, `reminderExists`, `escalationExists`. Trim `StalemateConfig` to `ProcessingTimeout` + `Interval`. Remove `ReminderAfter` / `EscalateAfter` env vars. |
|
||||
| `internal/messaging/stalemate_test.go` | Delete tests for removed methods; keep ProcessingTimeout tests. |
|
||||
| `internal/messaging/options.go` | Remove channelLookup wiring if it's only used by escalation. |
|
||||
| `internal/messaging/service.go` | Remove escalation hooks if any. |
|
||||
| `internal/mcp/goals_tools.go` (spec 018) | Delete `propose_agent` tool registration (`proposeAgentTool`) and its `handleProposeAgent` handler. Keep `propose_task_tree` and the rest of the registrar. |
|
||||
| `internal/storage/schema/027_remove_approval_noise.sql` | New migration. |
|
||||
| `cmd/synapbus/admin.go`, `cmd/synapbus/main.go` | Remove any escalation-related flags. |
|
||||
| `CLAUDE.md` (project + user) | Update SynapBus protocol section to drop "#approvals" + "stalemate auto-fails after 24h" mention of escalation. Keep claim-process-done loop. |
|
||||
| User's `~/.claude/CLAUDE.md` | Same — drop approval-channel references and the auto-report trigger for "Need approval → #approvals". |
|
||||
|
||||
## Testing
|
||||
|
||||
- Existing `stalemate_test.go` cases for `ProcessingTimeout` continue to pass.
|
||||
- New test: confirm `StalemateWorker.tick()` no longer queries pending messages for reminder/escalation candidates (no rows touched, no DMs sent).
|
||||
- New test: confirm `propose_agent` MCP tool returns "tool not found" / is unregistered.
|
||||
- Migration test: apply `027_remove_approval_noise.sql` to a fixture DB containing stalemate DMs + an `#approvals` message + an `agent_proposals` row; assert all three are gone, other messages untouched.
|
||||
- No UI testing required — Web UI just stops showing approval-channel content because the channel is empty.
|
||||
|
||||
## Risks & mitigations
|
||||
|
||||
- **An external agent calls `propose_agent` after deletion.** MCP returns an unknown-tool error; agent's runbook should tolerate this. Acceptable because the user controls all agents.
|
||||
- **Hidden consumer of escalation messages.** Search confirms reminders/escalations are only produced by `StalemateWorker` and consumed by humans. Low risk.
|
||||
- **Migration deletes too much.** The `LIKE 'stalemate-%'` pattern is narrow and the `#approvals` channel is internal-only; nothing user-authored lives there. Take a `data/synapbus.db` backup before applying in prod (kubic).
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Webhook/K8s runner human gates (spec 003).
|
||||
- Removing the `#approvals` channel row.
|
||||
- Adding `SYNAPBUS_APPROVALS_DISABLED` env flag — code deletion is reversible via git revert.
|
||||
- Daily summarizer agent + `#summary-daily` channel — already in progress in a separate effort.
|
||||
- Reclaiming disk via `VACUUM` — separate admin command if needed.
|
||||
@@ -0,0 +1,41 @@
|
||||
# SynapBus dream-agent — slim Python container that runs Claude Code via
|
||||
# claude-agent-sdk against SynapBus's MCP server. Built for linux/amd64.
|
||||
#
|
||||
# The proven recipe (per ~/repos/searcher/agents/universal/Dockerfile)
|
||||
# is a single-stage image with `uv pip install --system`. Multi-stage
|
||||
# saves little since claude-agent-sdk transitively pulls anyio/httpx,
|
||||
# and the heavy bit (the `claude` CLI binary) ships inside the wheel as
|
||||
# a JS bundle.
|
||||
FROM python:3.12-slim
|
||||
|
||||
# Bring in `uv` from its official image. Pure binary, no apt.
|
||||
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
|
||||
|
||||
# System deps: git (claude-agent-sdk shells out for some workspace ops),
|
||||
# ca-certs + curl for TLS / health checks. Cleanup apt lists.
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends git ca-certificates curl && \
|
||||
apt-get clean && rm -rf /var/lib/apt/lists/* && \
|
||||
git config --global user.email "dream-agent@synapbus.dev" && \
|
||||
git config --global user.name "SynapBus Dream Agent"
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Pinned versions — keep aligned with pyproject.toml. claude-agent-sdk
|
||||
# 0.1.48 bundles the `claude` CLI Node binary inside its wheel, so no
|
||||
# separate `claude-code` install step is required.
|
||||
RUN uv pip install --system --no-cache \
|
||||
"claude-agent-sdk==0.1.48" \
|
||||
"httpx>=0.27" \
|
||||
"opentelemetry-api>=1.27" \
|
||||
"opentelemetry-sdk>=1.27" \
|
||||
"opentelemetry-exporter-otlp-proto-http>=1.27"
|
||||
|
||||
COPY dream_runner.py /app/dream_runner.py
|
||||
|
||||
# Non-root user (matches searcher convention)
|
||||
RUN groupadd -g 1000 dream && useradd -u 1000 -g 1000 -m dream && \
|
||||
chown -R dream:dream /app
|
||||
USER dream
|
||||
|
||||
ENTRYPOINT ["python", "/app/dream_runner.py"]
|
||||
@@ -0,0 +1,69 @@
|
||||
# synapbus-dream-agent
|
||||
|
||||
A slim Python container that performs **memory consolidation** for
|
||||
SynapBus, dispatched on demand by the in-server `ConsolidatorWorker`
|
||||
via the `k8sjob` harness backend.
|
||||
|
||||
## What it does
|
||||
|
||||
1. Reads its job context from env vars (dispatch token, job id, job
|
||||
type, owner id, prompt).
|
||||
2. Connects to SynapBus's MCP endpoint over streamable-http, passing
|
||||
the agent's API key (`Authorization: Bearer ...`) **and** the
|
||||
dispatch token (`X-Synapbus-Dispatch-Token: ...`) on every request.
|
||||
3. Runs Claude Code (via `claude-agent-sdk`) constrained to the six
|
||||
`memory_*` MCP tools defined in
|
||||
`specs/020-proactive-memory-dream-worker/contracts/mcp-memory-tools.md`.
|
||||
4. Streams structured JSON logs to stdout (Loki-friendly) and emits a
|
||||
final `{"final": true, ...}` envelope so the harness can parse Usage.
|
||||
|
||||
## How the worker invokes it
|
||||
|
||||
`internal/messaging/consolidator.go` builds an `HarnessExecRequest`
|
||||
with:
|
||||
|
||||
| Env var | Set by |
|
||||
|---------------------------------|----------------------|
|
||||
| `SYNAPBUS_DISPATCH_TOKEN` | ConsolidatorWorker |
|
||||
| `SYNAPBUS_CONSOLIDATION_JOB_ID` | ConsolidatorWorker |
|
||||
| `SYNAPBUS_JOB_TYPE` | ConsolidatorWorker |
|
||||
| `SYNAPBUS_OWNER_ID` | ConsolidatorWorker |
|
||||
| `SYNAPBUS_DREAM_PROMPT` | ConsolidatorWorker |
|
||||
| `SYNAPBUS_RUN_ID` | k8sjob harness |
|
||||
| `SYNAPBUS_URL`, `SYNAPBUS_API_KEY`, `ANTHROPIC_API_KEY` | Pod spec / Secret |
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
docker buildx build --platform=linux/amd64 \
|
||||
-t kubic.home.arpa:32000/synapbus-dream-agent:v0.1.0 \
|
||||
--load /Users/user/repos/synapbus/dream-agent/
|
||||
```
|
||||
|
||||
Push:
|
||||
|
||||
```bash
|
||||
docker push kubic.home.arpa:32000/synapbus-dream-agent:v0.1.0
|
||||
```
|
||||
|
||||
## Local smoke test
|
||||
|
||||
The `--mock` flag validates the env contract and exits without
|
||||
invoking the SDK or hitting the network:
|
||||
|
||||
```bash
|
||||
SYNAPBUS_URL=http://localhost:8080 \
|
||||
SYNAPBUS_API_KEY=fake \
|
||||
SYNAPBUS_DISPATCH_TOKEN=fake \
|
||||
SYNAPBUS_CONSOLIDATION_JOB_ID=1 \
|
||||
SYNAPBUS_JOB_TYPE=reflection \
|
||||
SYNAPBUS_OWNER_ID=algis \
|
||||
SYNAPBUS_DREAM_PROMPT="test" \
|
||||
SYNAPBUS_RUN_ID=r-test \
|
||||
python3 dream_runner.py --mock
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
See `k8s-job-template.yaml`. The harness clones the template and
|
||||
overlays `req.Env` into `containers[0].env`.
|
||||
@@ -0,0 +1,401 @@
|
||||
#!/usr/bin/env python3
|
||||
"""SynapBus dream-agent runner — memory consolidation worker.
|
||||
|
||||
Dispatched by SynapBus's ConsolidatorWorker via the k8sjob harness.
|
||||
Runs Claude Code (via claude-agent-sdk) against SynapBus's MCP server,
|
||||
using a one-time dispatch token to authorize the six memory_* tools.
|
||||
|
||||
Environment contract (set by ConsolidatorWorker.runJob + k8sjob harness):
|
||||
SYNAPBUS_URL base URL, e.g. http://synapbus.synapbus.svc.cluster.local:8080
|
||||
SYNAPBUS_API_KEY dream-claude agent API key (Bearer auth)
|
||||
SYNAPBUS_DISPATCH_TOKEN one-shot token authorizing memory_* tools
|
||||
SYNAPBUS_CONSOLIDATION_JOB_ID parent job id (audit anchor)
|
||||
SYNAPBUS_JOB_TYPE reflection | core_rewrite | dedup_contradiction | link_gen
|
||||
SYNAPBUS_OWNER_ID target owner id
|
||||
SYNAPBUS_DREAM_PROMPT job-type prompt (PromptFor)
|
||||
SYNAPBUS_RUN_ID harness-injected run id
|
||||
Optional:
|
||||
ANTHROPIC_API_KEY or CLAUDE_CONFIG_DIR Claude Code credentials
|
||||
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT OTLP/HTTP traces endpoint
|
||||
DREAM_MAX_TURNS override max_turns (default 20)
|
||||
DREAM_MODEL override model (default claude-sonnet-4-6)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
# --- Structured JSON logging (one obj per line for Loki) -------------------
|
||||
|
||||
class _JsonFormatter(logging.Formatter):
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.job_id = os.environ.get("SYNAPBUS_CONSOLIDATION_JOB_ID", "")
|
||||
self.job_type = os.environ.get("SYNAPBUS_JOB_TYPE", "")
|
||||
self.owner_id = os.environ.get("SYNAPBUS_OWNER_ID", "")
|
||||
self.run_id = os.environ.get("SYNAPBUS_RUN_ID", "")
|
||||
self.trace_id: str = ""
|
||||
|
||||
def format(self, record: logging.LogRecord) -> str:
|
||||
entry: dict[str, Any] = {
|
||||
"ts": self.formatTime(record, "%Y-%m-%dT%H:%M:%SZ"),
|
||||
"level": record.levelname,
|
||||
"logger": record.name,
|
||||
"job_id": self.job_id,
|
||||
"job_type": self.job_type,
|
||||
"owner_id": self.owner_id,
|
||||
"run_id": self.run_id,
|
||||
}
|
||||
if self.trace_id:
|
||||
entry["traceID"] = self.trace_id
|
||||
if isinstance(record.msg, dict):
|
||||
entry.update(record.msg)
|
||||
else:
|
||||
entry["msg"] = record.getMessage()
|
||||
return json.dumps(entry, default=str)
|
||||
|
||||
|
||||
def _setup_logging() -> logging.Logger:
|
||||
lg = logging.getLogger("dream-agent")
|
||||
lg.setLevel(logging.INFO)
|
||||
lg.handlers.clear()
|
||||
lg.propagate = False
|
||||
h = logging.StreamHandler(sys.stdout)
|
||||
h.setFormatter(_JsonFormatter())
|
||||
lg.addHandler(h)
|
||||
root = logging.getLogger()
|
||||
root.handlers.clear()
|
||||
root.addHandler(h)
|
||||
return lg
|
||||
|
||||
|
||||
logger = logging.getLogger("dream-agent")
|
||||
|
||||
|
||||
# --- OTEL tracing (best-effort) --------------------------------------------
|
||||
|
||||
_tracer = None
|
||||
|
||||
|
||||
def _init_tracing() -> None:
|
||||
global _tracer
|
||||
ep = os.environ.get("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT", "")
|
||||
if not ep:
|
||||
return
|
||||
try:
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import BatchSpanProcessor
|
||||
from opentelemetry.sdk.resources import Resource
|
||||
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
|
||||
|
||||
resource = Resource.create({
|
||||
"service.name": "synapbus-dream-agent",
|
||||
"service.version": "0.1.0",
|
||||
"synapbus.job_id": os.environ.get("SYNAPBUS_CONSOLIDATION_JOB_ID", ""),
|
||||
"synapbus.job_type": os.environ.get("SYNAPBUS_JOB_TYPE", ""),
|
||||
"synapbus.owner_id": os.environ.get("SYNAPBUS_OWNER_ID", ""),
|
||||
})
|
||||
provider = TracerProvider(resource=resource)
|
||||
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(endpoint=ep)))
|
||||
trace.set_tracer_provider(provider)
|
||||
_tracer = trace.get_tracer("synapbus-dream-agent", "0.1.0")
|
||||
logger.info({"msg": "OTEL tracing enabled", "endpoint": ep})
|
||||
except ImportError:
|
||||
logger.info({"msg": "OTEL packages not installed; tracing disabled"})
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning({"msg": "OTEL init failed", "error": str(e)})
|
||||
|
||||
|
||||
def _shutdown_tracing() -> None:
|
||||
try:
|
||||
from opentelemetry import trace
|
||||
p = trace.get_tracer_provider()
|
||||
if hasattr(p, "shutdown"):
|
||||
p.shutdown()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
|
||||
# --- Claude Code creds: ensure config dir is writable ----------------------
|
||||
|
||||
def _ensure_writable_config() -> str:
|
||||
src = os.environ.get("CLAUDE_CONFIG_DIR", os.path.expanduser("~/.claude"))
|
||||
test = os.path.join(src, ".write_test")
|
||||
try:
|
||||
os.makedirs(src, exist_ok=True)
|
||||
with open(test, "w") as f:
|
||||
f.write("ok")
|
||||
os.remove(test)
|
||||
return src
|
||||
except OSError:
|
||||
pass
|
||||
tmp = tempfile.mkdtemp(prefix="claude_config_")
|
||||
for fn in (".credentials.json", "credentials.json", "settings.json"):
|
||||
s = os.path.join(src, fn)
|
||||
if os.path.exists(s):
|
||||
shutil.copy2(s, os.path.join(tmp, fn))
|
||||
logger.info({"msg": "Created writable Claude config dir", "path": tmp})
|
||||
return tmp
|
||||
|
||||
|
||||
# --- Required-env helper ---------------------------------------------------
|
||||
|
||||
_REQUIRED = (
|
||||
"SYNAPBUS_URL",
|
||||
"SYNAPBUS_API_KEY",
|
||||
"SYNAPBUS_DISPATCH_TOKEN",
|
||||
"SYNAPBUS_CONSOLIDATION_JOB_ID",
|
||||
"SYNAPBUS_JOB_TYPE",
|
||||
"SYNAPBUS_OWNER_ID",
|
||||
"SYNAPBUS_DREAM_PROMPT",
|
||||
)
|
||||
|
||||
|
||||
def _read_env() -> dict[str, str]:
|
||||
out: dict[str, str] = {}
|
||||
missing: list[str] = []
|
||||
for k in _REQUIRED:
|
||||
v = os.environ.get(k, "")
|
||||
if not v:
|
||||
missing.append(k)
|
||||
out[k] = v
|
||||
if missing:
|
||||
raise RuntimeError(f"missing required env vars: {','.join(missing)}")
|
||||
out["SYNAPBUS_RUN_ID"] = os.environ.get("SYNAPBUS_RUN_ID", "")
|
||||
return out
|
||||
|
||||
|
||||
# --- Prompt builder --------------------------------------------------------
|
||||
|
||||
_ALLOWED_TOOLS = [
|
||||
"mcp__synapbus__memory_list_unprocessed",
|
||||
"mcp__synapbus__memory_write_reflection",
|
||||
"mcp__synapbus__memory_rewrite_core",
|
||||
"mcp__synapbus__memory_mark_duplicate",
|
||||
"mcp__synapbus__memory_supersede",
|
||||
"mcp__synapbus__memory_add_link",
|
||||
]
|
||||
|
||||
|
||||
def _build_prompt(env: dict[str, str]) -> str:
|
||||
return (
|
||||
f"{env['SYNAPBUS_DREAM_PROMPT']}\n\n"
|
||||
"Context:\n"
|
||||
f"- job_id: {env['SYNAPBUS_CONSOLIDATION_JOB_ID']}\n"
|
||||
f"- job_type: {env['SYNAPBUS_JOB_TYPE']}\n"
|
||||
f"- owner_id: {env['SYNAPBUS_OWNER_ID']}\n"
|
||||
f"- run_id: {env['SYNAPBUS_RUN_ID']}\n"
|
||||
"- The dispatch token is forwarded automatically on every MCP "
|
||||
"request via the `X-Synapbus-Dispatch-Token` header. You do not "
|
||||
"need to pass it as a tool argument.\n"
|
||||
"- Pass `owner_id` from the context above on every memory_* call.\n"
|
||||
"- Use ONLY the memory_* tools listed in `allowed_tools`. Do not "
|
||||
"call send_message, execute, search, or any other tool.\n"
|
||||
"- When you are done, output a one-line JSON summary and exit.\n"
|
||||
)
|
||||
|
||||
|
||||
# --- Session runner --------------------------------------------------------
|
||||
|
||||
async def run_session(env: dict[str, str], model: str, max_turns: int, config_dir: str) -> dict[str, Any]:
|
||||
from claude_agent_sdk import (
|
||||
AssistantMessage,
|
||||
ClaudeAgentOptions,
|
||||
ResultMessage,
|
||||
TextBlock,
|
||||
query,
|
||||
)
|
||||
try:
|
||||
from claude_agent_sdk import ToolUseBlock, ToolResultBlock, ThinkingBlock, UserMessage
|
||||
except ImportError:
|
||||
ToolUseBlock = ToolResultBlock = ThinkingBlock = UserMessage = None
|
||||
|
||||
base = env["SYNAPBUS_URL"].rstrip("/")
|
||||
mcp_servers: dict[str, Any] = {
|
||||
"synapbus": {
|
||||
"type": "http",
|
||||
"url": f"{base}/mcp",
|
||||
"headers": {
|
||||
"Authorization": f"Bearer {env['SYNAPBUS_API_KEY']}",
|
||||
"X-Synapbus-Dispatch-Token": env["SYNAPBUS_DISPATCH_TOKEN"],
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
prompt = _build_prompt(env)
|
||||
|
||||
def _on_stderr(line: str) -> None:
|
||||
logger.warning({"msg": "sdk_stderr", "line": line.rstrip()})
|
||||
|
||||
tokens_in = 0
|
||||
tokens_out = 0
|
||||
tool_calls = 0
|
||||
turn = 0
|
||||
started = time.time()
|
||||
status = "ok"
|
||||
error_msg = ""
|
||||
|
||||
try:
|
||||
async for message in query(
|
||||
prompt=prompt,
|
||||
options=ClaudeAgentOptions(
|
||||
model=model,
|
||||
max_turns=max_turns,
|
||||
mcp_servers=mcp_servers,
|
||||
permission_mode="bypassPermissions",
|
||||
allowed_tools=_ALLOWED_TOOLS,
|
||||
env={"CLAUDE_CONFIG_DIR": config_dir},
|
||||
stderr=_on_stderr,
|
||||
),
|
||||
):
|
||||
if isinstance(message, ResultMessage):
|
||||
usage = getattr(message, "usage", None)
|
||||
tokens_in = getattr(usage, "input_tokens", 0) if usage else 0
|
||||
tokens_out = getattr(usage, "output_tokens", 0) if usage else 0
|
||||
# Max20 OAuth sessions don't surface tokens through the
|
||||
# SDK's ResultMessage.usage. Fall back to a turn-based
|
||||
# estimate so the server-side UsageGate has *some* signal.
|
||||
# Numbers calibrated from observed reflection runs:
|
||||
# ~5K input + ~300 output per turn, plus ~2K per tool call
|
||||
# (memory_list_unprocessed payloads dominate).
|
||||
if tokens_in == 0:
|
||||
num_turns = int(getattr(message, "num_turns", 0) or 0)
|
||||
tokens_in = max(0, num_turns * 5000 + tool_calls * 2000)
|
||||
if tokens_out == 0:
|
||||
num_turns = int(getattr(message, "num_turns", 0) or 0)
|
||||
tokens_out = max(0, num_turns * 300)
|
||||
is_error = getattr(message, "is_error", False)
|
||||
duration_s = round(time.time() - started, 1)
|
||||
logger.info({
|
||||
"type": "result",
|
||||
"turns": getattr(message, "num_turns", 0),
|
||||
"cost_usd": getattr(message, "cost_usd", 0) or 0,
|
||||
"tokens_in": tokens_in,
|
||||
"tokens_out": tokens_out,
|
||||
"tool_calls": tool_calls,
|
||||
"duration_s": duration_s,
|
||||
"is_error": is_error,
|
||||
})
|
||||
if is_error:
|
||||
status = "error"
|
||||
error_msg = "result_message.is_error=true"
|
||||
elif isinstance(message, AssistantMessage):
|
||||
turn += 1
|
||||
for block in message.content:
|
||||
if isinstance(block, TextBlock):
|
||||
logger.info({
|
||||
"type": "text",
|
||||
"turn": turn,
|
||||
"text": block.text[:300].replace("\n", " "),
|
||||
})
|
||||
elif ToolUseBlock and isinstance(block, ToolUseBlock):
|
||||
tool_calls += 1
|
||||
logger.info({
|
||||
"type": "tool_use",
|
||||
"turn": turn,
|
||||
"tool": getattr(block, "name", "unknown"),
|
||||
"input": str(getattr(block, "input", ""))[:200],
|
||||
})
|
||||
elif ThinkingBlock and isinstance(block, ThinkingBlock):
|
||||
logger.info({
|
||||
"type": "thinking",
|
||||
"turn": turn,
|
||||
"text": getattr(block, "text", "")[:200].replace("\n", " "),
|
||||
})
|
||||
elif UserMessage and isinstance(message, UserMessage):
|
||||
for block in message.content:
|
||||
if ToolResultBlock and isinstance(block, ToolResultBlock):
|
||||
is_err = getattr(block, "is_error", False)
|
||||
logger.info({
|
||||
"type": "tool_result",
|
||||
"turn": turn,
|
||||
"is_error": is_err,
|
||||
"content": str(getattr(block, "content", ""))[:200],
|
||||
})
|
||||
except Exception as e: # noqa: BLE001
|
||||
status = "error"
|
||||
error_msg = f"{type(e).__name__}: {e}"
|
||||
logger.error({"msg": "session failed", "error": error_msg})
|
||||
|
||||
return {
|
||||
"final": True,
|
||||
"tokens_in": tokens_in,
|
||||
"tokens_out": tokens_out,
|
||||
"tool_calls": tool_calls,
|
||||
"turns": turn,
|
||||
"status": status,
|
||||
"error": error_msg,
|
||||
}
|
||||
|
||||
|
||||
# --- main ------------------------------------------------------------------
|
||||
|
||||
def main() -> int:
|
||||
global logger
|
||||
parser = argparse.ArgumentParser(description="SynapBus dream-agent runner")
|
||||
parser.add_argument("--mock", action="store_true",
|
||||
help="Log env contract and exit without invoking the SDK")
|
||||
parser.add_argument("--max-turns", type=int,
|
||||
default=int(os.environ.get("DREAM_MAX_TURNS", "20")))
|
||||
parser.add_argument("--model", default=os.environ.get("DREAM_MODEL", "claude-sonnet-4-6"))
|
||||
args = parser.parse_args()
|
||||
|
||||
logger = _setup_logging()
|
||||
_init_tracing()
|
||||
|
||||
try:
|
||||
env = _read_env()
|
||||
except RuntimeError as e:
|
||||
logger.error({"msg": "env validation failed", "error": str(e)})
|
||||
print(json.dumps({
|
||||
"final": True, "tokens_in": 0, "tokens_out": 0,
|
||||
"tool_calls": 0, "status": "error", "error": str(e),
|
||||
}))
|
||||
return 1
|
||||
|
||||
logger.info({
|
||||
"msg": "dream-agent starting",
|
||||
"synapbus_url": env["SYNAPBUS_URL"],
|
||||
"model": args.model,
|
||||
"max_turns": args.max_turns,
|
||||
})
|
||||
|
||||
if args.mock:
|
||||
logger.info({"msg": "--mock; skipping SDK invocation"})
|
||||
print(json.dumps({
|
||||
"final": True, "tokens_in": 0, "tokens_out": 0,
|
||||
"tool_calls": 0, "status": "ok", "error": "",
|
||||
}))
|
||||
return 0
|
||||
|
||||
config_dir = _ensure_writable_config()
|
||||
|
||||
try:
|
||||
result = asyncio.run(run_session(env, args.model, args.max_turns, config_dir))
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.error({"msg": "fatal", "error": f"{type(e).__name__}: {e}"})
|
||||
print(json.dumps({
|
||||
"final": True, "tokens_in": 0, "tokens_out": 0,
|
||||
"tool_calls": 0, "status": "error", "error": str(e),
|
||||
}))
|
||||
_shutdown_tracing()
|
||||
return 1
|
||||
|
||||
# Final single-line envelope for harness Usage parsing.
|
||||
print(json.dumps(result))
|
||||
_shutdown_tracing()
|
||||
return 0 if result.get("status") == "ok" else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,82 @@
|
||||
# Reference Job template that the SynapBus k8sjob harness instantiates
|
||||
# per dispatched dream-agent run. The harness will:
|
||||
# 1. Clone this template
|
||||
# 2. Populate `metadata.name` with `dream-<job_id>-<run_id_short>`
|
||||
# 3. Merge req.Env into `env:` (SYNAPBUS_DISPATCH_TOKEN,
|
||||
# SYNAPBUS_CONSOLIDATION_JOB_ID, SYNAPBUS_JOB_TYPE,
|
||||
# SYNAPBUS_OWNER_ID, SYNAPBUS_DREAM_PROMPT, SYNAPBUS_RUN_ID)
|
||||
# 4. Tail container logs back to the worker
|
||||
#
|
||||
# Replace `kubic.home.arpa:32000/synapbus-dream-agent:v0.1.0` with the
|
||||
# actual image tag your registry publishes.
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: dream-agent-PLACEHOLDER
|
||||
namespace: synapbus
|
||||
labels:
|
||||
app: synapbus-dream-agent
|
||||
synapbus.io/role: memory-consolidator
|
||||
spec:
|
||||
backoffLimit: 0 # one-shot — server-side circuit breaker decides retries
|
||||
ttlSecondsAfterFinished: 600
|
||||
activeDeadlineSeconds: 900 # hard cap above DreamWallclockBudget (default 10m)
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: synapbus-dream-agent
|
||||
spec:
|
||||
restartPolicy: Never
|
||||
serviceAccountName: default
|
||||
containers:
|
||||
- name: dream-agent
|
||||
image: kubic.home.arpa:32000/synapbus-dream-agent:v0.1.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
env:
|
||||
# In-cluster SynapBus address (cluster DNS).
|
||||
- name: SYNAPBUS_URL
|
||||
value: "http://synapbus.synapbus.svc.cluster.local:8080"
|
||||
- name: SYNAPBUS_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: dream-agent-secrets
|
||||
key: SYNAPBUS_API_KEY
|
||||
- name: ANTHROPIC_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: dream-agent-secrets
|
||||
key: ANTHROPIC_API_KEY
|
||||
- name: CLAUDE_CONFIG_DIR
|
||||
value: "/home/dream/.claude"
|
||||
# Optional OTLP/HTTP traces export to Tempo
|
||||
- name: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
|
||||
value: "http://tempo.observability.svc.cluster.local:4318/v1/traces"
|
||||
# ---- The harness Env map appends here at dispatch time ----
|
||||
# SYNAPBUS_DISPATCH_TOKEN, SYNAPBUS_CONSOLIDATION_JOB_ID,
|
||||
# SYNAPBUS_JOB_TYPE, SYNAPBUS_OWNER_ID, SYNAPBUS_DREAM_PROMPT,
|
||||
# SYNAPBUS_RUN_ID
|
||||
resources:
|
||||
requests:
|
||||
cpu: "200m"
|
||||
memory: "256Mi"
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "512Mi"
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: false
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
---
|
||||
# Secret skeleton — populate via kubectl + sealed-secrets / sops out of band.
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: dream-agent-secrets
|
||||
namespace: synapbus
|
||||
type: Opaque
|
||||
stringData:
|
||||
SYNAPBUS_API_KEY: "REPLACE_ME" # dream-claude agent's SynapBus API key
|
||||
ANTHROPIC_API_KEY: "REPLACE_ME" # Anthropic API key for Claude Code
|
||||
@@ -0,0 +1,19 @@
|
||||
[project]
|
||||
name = "synapbus-dream-agent"
|
||||
version = "0.1.0"
|
||||
description = "SynapBus memory-consolidation dream-agent runner (claude-agent-sdk)"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"claude-agent-sdk==0.1.48",
|
||||
"httpx>=0.27",
|
||||
"opentelemetry-api>=1.27",
|
||||
"opentelemetry-sdk>=1.27",
|
||||
"opentelemetry-exporter-otlp-proto-http>=1.27",
|
||||
]
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = []
|
||||
@@ -142,3 +142,8 @@ require (
|
||||
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 // indirect
|
||||
sigs.k8s.io/yaml v1.6.0 // indirect
|
||||
)
|
||||
|
||||
// OPC local patch: github.com/TFMV/hnsw calls renameio.TempFile, which
|
||||
// renameio v1 does not provide on Windows. Pin v0.1.0 so the binary builds
|
||||
// on Windows. Drop this once upstream hnsw/renameio no longer needs it.
|
||||
replace github.com/google/renameio => github.com/google/renameio v0.1.0
|
||||
|
||||
@@ -232,6 +232,7 @@ github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLe
|
||||
github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
|
||||
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 h1:BHT72Gu3keYf3ZEu2J0b1vyeLSOYI8bm5wbJM/8yDe8=
|
||||
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
|
||||
github.com/google/renameio v0.1.0 h1:GOZbcHa3HfsPKPlmyPyN2KEohoMXOhdMbHrvbpl2QaA=
|
||||
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
|
||||
github.com/google/renameio v1.0.1 h1:Lh/jXZmvZxb0BBeSY5VKEfidcbcbenKjZFzM/q0fSeU=
|
||||
github.com/google/renameio v1.0.1/go.mod h1:t/HQoYBZSsWSNK35C6CO/TpPLDVWvxOHboWUAweKUpk=
|
||||
|
||||
+34
-14
@@ -34,20 +34,40 @@ type K8sServiceProvider interface {
|
||||
|
||||
// Services holds references to all services the admin socket can control.
|
||||
type Services struct {
|
||||
Users *auth.SQLiteUserStore
|
||||
Sessions auth.SessionStore
|
||||
Agents *agents.AgentService
|
||||
Messages *messaging.MessagingService
|
||||
Channels *channels.Service
|
||||
Traces trace.TraceStore
|
||||
EmbeddingStore *search.EmbeddingStore
|
||||
VectorIndex *search.VectorIndex
|
||||
SearchService *search.Service
|
||||
AttachmentService *attachments.Service
|
||||
WebhookService WebhookServiceProvider
|
||||
K8sService K8sServiceProvider
|
||||
DataDir string
|
||||
RetentionWorker RetentionStatusProvider
|
||||
Users *auth.SQLiteUserStore
|
||||
Sessions auth.SessionStore
|
||||
Agents *agents.AgentService
|
||||
Messages *messaging.MessagingService
|
||||
Channels *channels.Service
|
||||
Traces trace.TraceStore
|
||||
EmbeddingStore *search.EmbeddingStore
|
||||
VectorIndex *search.VectorIndex
|
||||
SearchService *search.Service
|
||||
AttachmentService *attachments.Service
|
||||
WebhookService WebhookServiceProvider
|
||||
K8sService K8sServiceProvider
|
||||
DataDir string
|
||||
RetentionWorker RetentionStatusProvider
|
||||
|
||||
// CoreMemoryStore is the per-(owner, agent) core memory store wired
|
||||
// in for feature 020 admin CLI commands (`synapbus memory core ...`).
|
||||
// May be nil — handlers report "core memory store not configured".
|
||||
CoreMemoryStore *messaging.CoreMemoryStore
|
||||
|
||||
// DreamRun, when non-nil, dispatches a single consolidation job
|
||||
// bypassing the trigger check. Wired by main.go when the
|
||||
// consolidator worker is enabled. Closure form keeps the worker
|
||||
// internals out of the admin package's import graph.
|
||||
DreamRun func(ctx context.Context, ownerID, jobType string) (jobID int64, err error)
|
||||
|
||||
// DreamRunN fans out N parallel consolidation jobs (via slot 0..N-1)
|
||||
// for one (owner, job_type). Used by `synapbus memory dream-run
|
||||
// --parallel N`. core_rewrite always coerces to N=1 server-side.
|
||||
DreamRunN func(ctx context.Context, ownerID, jobType string, parallel int) (jobIDs []int64, err error)
|
||||
|
||||
// DefaultDreamParallel is consulted when the CLI request omits
|
||||
// --parallel. Sourced from MemoryConfig.DreamParallel.
|
||||
DefaultDreamParallel int
|
||||
}
|
||||
|
||||
// RetentionStatusProvider provides retention status information.
|
||||
|
||||
+185
-2
@@ -226,6 +226,16 @@ func (s *AdminServer) dispatch(req Request) Response {
|
||||
case "harness.config_set":
|
||||
return s.handleHarnessConfigSet(ctx, req.Args)
|
||||
|
||||
// --- memory core (feature 020 — proactive memory) ---
|
||||
case "memory.core.get":
|
||||
return s.handleMemoryCoreGet(ctx, req.Args)
|
||||
case "memory.core.set":
|
||||
return s.handleMemoryCoreSet(ctx, req.Args)
|
||||
case "memory.core.delete":
|
||||
return s.handleMemoryCoreDelete(ctx, req.Args)
|
||||
case "memory.dream_run":
|
||||
return s.handleMemoryDreamRun(ctx, req.Args)
|
||||
|
||||
default:
|
||||
return Response{OK: false, Error: fmt.Sprintf("unknown command: %s", req.Command)}
|
||||
}
|
||||
@@ -545,7 +555,7 @@ func (s *AdminServer) handleAuditStats(ctx context.Context) Response {
|
||||
}
|
||||
|
||||
return Response{OK: true, Data: map[string]interface{}{
|
||||
"total_traces": totalTraces,
|
||||
"total_traces": totalTraces,
|
||||
"counts_by_action": counts,
|
||||
}}
|
||||
}
|
||||
@@ -1694,7 +1704,7 @@ func (s *AdminServer) handleAttachmentsGC(ctx context.Context) Response {
|
||||
}
|
||||
|
||||
return Response{OK: true, Data: map[string]interface{}{
|
||||
"files_removed": result.FilesRemoved,
|
||||
"files_removed": result.FilesRemoved,
|
||||
"bytes_reclaimed": result.BytesReclaimed,
|
||||
}}
|
||||
}
|
||||
@@ -1850,5 +1860,178 @@ func (s *AdminServer) handleHarnessConfigSet(ctx context.Context, args json.RawM
|
||||
}}
|
||||
}
|
||||
|
||||
// ---------- memory core handlers (feature 020) ----------
|
||||
//
|
||||
// owner_id wire format: callers pass the owner as a username; we resolve
|
||||
// to `users.id` and pass the string form to CoreMemoryStore so it matches
|
||||
// the proactive-memory tables' TEXT owner_id convention.
|
||||
|
||||
func (s *AdminServer) resolveOwnerString(ctx context.Context, ownerInput string) (string, error) {
|
||||
if ownerInput == "" {
|
||||
return "", fmt.Errorf("owner is required")
|
||||
}
|
||||
// First try numeric — admins may already know the user ID.
|
||||
var id int64
|
||||
if _, err := fmt.Sscanf(ownerInput, "%d", &id); err == nil && id > 0 {
|
||||
return fmt.Sprintf("%d", id), nil
|
||||
}
|
||||
// Otherwise treat as username.
|
||||
user, err := s.services.Users.GetUserByUsername(ctx, ownerInput)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("resolve owner %q: %w", ownerInput, err)
|
||||
}
|
||||
return fmt.Sprintf("%d", user.ID), nil
|
||||
}
|
||||
|
||||
func (s *AdminServer) handleMemoryCoreGet(ctx context.Context, args json.RawMessage) Response {
|
||||
var p struct {
|
||||
Owner string `json:"owner"`
|
||||
Agent string `json:"agent"`
|
||||
}
|
||||
if err := json.Unmarshal(args, &p); err != nil {
|
||||
return Response{OK: false, Error: "invalid args: " + err.Error()}
|
||||
}
|
||||
if p.Agent == "" {
|
||||
return Response{OK: false, Error: "agent is required"}
|
||||
}
|
||||
if s.services.CoreMemoryStore == nil {
|
||||
return Response{OK: false, Error: "core memory store not configured"}
|
||||
}
|
||||
ownerStr, err := s.resolveOwnerString(ctx, p.Owner)
|
||||
if err != nil {
|
||||
return Response{OK: false, Error: err.Error()}
|
||||
}
|
||||
blob, updatedAt, ok, err := s.services.CoreMemoryStore.Get(ctx, ownerStr, p.Agent)
|
||||
if err != nil {
|
||||
return Response{OK: false, Error: err.Error()}
|
||||
}
|
||||
if !ok {
|
||||
return Response{OK: true, Data: map[string]any{
|
||||
"owner_id": ownerStr,
|
||||
"agent_name": p.Agent,
|
||||
"exists": false,
|
||||
}}
|
||||
}
|
||||
return Response{OK: true, Data: map[string]any{
|
||||
"owner_id": ownerStr,
|
||||
"agent_name": p.Agent,
|
||||
"exists": true,
|
||||
"blob": blob,
|
||||
"updated_at": updatedAt.Format(time.RFC3339),
|
||||
}}
|
||||
}
|
||||
|
||||
func (s *AdminServer) handleMemoryCoreSet(ctx context.Context, args json.RawMessage) Response {
|
||||
var p struct {
|
||||
Owner string `json:"owner"`
|
||||
Agent string `json:"agent"`
|
||||
Blob string `json:"blob"`
|
||||
UpdatedBy string `json:"updated_by"`
|
||||
}
|
||||
if err := json.Unmarshal(args, &p); err != nil {
|
||||
return Response{OK: false, Error: "invalid args: " + err.Error()}
|
||||
}
|
||||
if p.Agent == "" {
|
||||
return Response{OK: false, Error: "agent is required"}
|
||||
}
|
||||
if s.services.CoreMemoryStore == nil {
|
||||
return Response{OK: false, Error: "core memory store not configured"}
|
||||
}
|
||||
ownerStr, err := s.resolveOwnerString(ctx, p.Owner)
|
||||
if err != nil {
|
||||
return Response{OK: false, Error: err.Error()}
|
||||
}
|
||||
updatedBy := p.UpdatedBy
|
||||
if updatedBy == "" {
|
||||
updatedBy = "human"
|
||||
}
|
||||
if err := s.services.CoreMemoryStore.Set(ctx, ownerStr, p.Agent, p.Blob, updatedBy); err != nil {
|
||||
if err == messaging.ErrCoreMemoryTooLarge {
|
||||
return Response{OK: false, Error: fmt.Sprintf("core_memory_too_large: blob exceeds %d bytes", s.services.CoreMemoryStore.MaxBytes())}
|
||||
}
|
||||
return Response{OK: false, Error: err.Error()}
|
||||
}
|
||||
return Response{OK: true, Data: map[string]any{
|
||||
"owner_id": ownerStr,
|
||||
"agent_name": p.Agent,
|
||||
"blob_chars": len(p.Blob),
|
||||
"updated_by": updatedBy,
|
||||
}}
|
||||
}
|
||||
|
||||
func (s *AdminServer) handleMemoryCoreDelete(ctx context.Context, args json.RawMessage) Response {
|
||||
var p struct {
|
||||
Owner string `json:"owner"`
|
||||
Agent string `json:"agent"`
|
||||
}
|
||||
if err := json.Unmarshal(args, &p); err != nil {
|
||||
return Response{OK: false, Error: "invalid args: " + err.Error()}
|
||||
}
|
||||
if p.Agent == "" {
|
||||
return Response{OK: false, Error: "agent is required"}
|
||||
}
|
||||
if s.services.CoreMemoryStore == nil {
|
||||
return Response{OK: false, Error: "core memory store not configured"}
|
||||
}
|
||||
ownerStr, err := s.resolveOwnerString(ctx, p.Owner)
|
||||
if err != nil {
|
||||
return Response{OK: false, Error: err.Error()}
|
||||
}
|
||||
if err := s.services.CoreMemoryStore.Delete(ctx, ownerStr, p.Agent); err != nil {
|
||||
return Response{OK: false, Error: err.Error()}
|
||||
}
|
||||
return Response{OK: true, Data: map[string]any{
|
||||
"owner_id": ownerStr,
|
||||
"agent_name": p.Agent,
|
||||
"deleted": true,
|
||||
}}
|
||||
}
|
||||
|
||||
// handleMemoryDreamRun forces dream-job dispatch(es). With parallel=1
|
||||
// (default) returns one job_id. With parallel>1, fans out N concurrent
|
||||
// jobs across slots 0..N-1 and returns the list of created ids. The
|
||||
// circuit breaker still applies.
|
||||
func (s *AdminServer) handleMemoryDreamRun(ctx context.Context, args json.RawMessage) Response {
|
||||
var p struct {
|
||||
Owner string `json:"owner"`
|
||||
JobType string `json:"job_type"`
|
||||
Parallel int `json:"parallel"`
|
||||
}
|
||||
if err := json.Unmarshal(args, &p); err != nil {
|
||||
return Response{OK: false, Error: "invalid args: " + err.Error()}
|
||||
}
|
||||
if p.JobType == "" {
|
||||
return Response{OK: false, Error: "job_type is required"}
|
||||
}
|
||||
if s.services.DreamRunN == nil {
|
||||
return Response{OK: false, Error: "dream worker not configured (SYNAPBUS_DREAM_ENABLED=0?)"}
|
||||
}
|
||||
ownerStr, err := s.resolveOwnerString(ctx, p.Owner)
|
||||
if err != nil {
|
||||
return Response{OK: false, Error: err.Error()}
|
||||
}
|
||||
parallel := p.Parallel
|
||||
if parallel <= 0 {
|
||||
parallel = s.services.DefaultDreamParallel
|
||||
}
|
||||
if parallel <= 0 {
|
||||
parallel = 1
|
||||
}
|
||||
ids, err := s.services.DreamRunN(ctx, ownerStr, p.JobType, parallel)
|
||||
if err != nil && len(ids) == 0 {
|
||||
return Response{OK: false, Error: err.Error()}
|
||||
}
|
||||
out := map[string]any{
|
||||
"job_ids": ids,
|
||||
"owner_id": ownerStr,
|
||||
"job_type": p.JobType,
|
||||
"parallel": parallel,
|
||||
}
|
||||
if err != nil {
|
||||
out["error"] = err.Error()
|
||||
}
|
||||
return Response{OK: true, Data: out}
|
||||
}
|
||||
|
||||
// Ensure the messaging import is used.
|
||||
var _ = messaging.StatusPending
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
package agents
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
// Sentinel errors returned by OwnerFor.
|
||||
//
|
||||
// Callers in the proactive-memory / dream-worker path use these to
|
||||
// distinguish "no such agent" from "agent exists but has no owner"
|
||||
// without parsing error strings.
|
||||
var (
|
||||
// ErrAgentNotFound is returned when no row exists in `agents` for
|
||||
// the requested name.
|
||||
ErrAgentNotFound = errors.New("agent not found")
|
||||
|
||||
// ErrAgentUnowned is returned when an agent row exists but its
|
||||
// owner_id is zero / empty. In the current schema owner_id is
|
||||
// declared NOT NULL, so this is effectively an integrity guard for
|
||||
// rows backfilled with 0.
|
||||
ErrAgentUnowned = errors.New("agent has no owner")
|
||||
)
|
||||
|
||||
// OwnerFor returns the string-encoded owner_id of the named agent.
|
||||
//
|
||||
// The schema stores `agents.owner_id` as INTEGER (FK to `users.id`), but
|
||||
// the proactive-memory tables and the request-context `owner_id`
|
||||
// (populated by auth middleware via `trace.ContextWithOwnerID`) carry it
|
||||
// as a string. OwnerFor canonicalizes to that string form so call sites
|
||||
// can compare without re-converting.
|
||||
//
|
||||
// Returns ("", ErrAgentNotFound) when no row matches; ("",
|
||||
// ErrAgentUnowned) when a row exists but owner_id is 0.
|
||||
func OwnerFor(ctx context.Context, db *sql.DB, agentName string) (string, error) {
|
||||
var ownerID int64
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT owner_id FROM agents WHERE name = ?`, agentName,
|
||||
).Scan(&ownerID)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return "", ErrAgentNotFound
|
||||
}
|
||||
return "", fmt.Errorf("query owner for agent %q: %w", agentName, err)
|
||||
}
|
||||
if ownerID == 0 {
|
||||
return "", ErrAgentUnowned
|
||||
}
|
||||
return strconv.FormatInt(ownerID, 10), nil
|
||||
}
|
||||
@@ -0,0 +1,285 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
)
|
||||
|
||||
const (
|
||||
// maxAgentConnections is the maximum number of simultaneous
|
||||
// /api/agent-events connections per agent.
|
||||
maxAgentConnections = 5
|
||||
// maxAgentBacklog is the maximum number of missed events replayed on
|
||||
// reconnect; beyond this a single resync_required event is sent instead.
|
||||
maxAgentBacklog = 200
|
||||
// agentSubBuffer is the per-connection live event buffer. A connection
|
||||
// whose buffer is full is dropped rather than blocking the broadcaster.
|
||||
agentSubBuffer = 64
|
||||
// agentWriteTimeout bounds a single write to a slow client.
|
||||
agentWriteTimeout = 10 * time.Second
|
||||
)
|
||||
|
||||
// AgentEventBacklog loads body-free metadata for messages visible to an agent,
|
||||
// used to replay events after a reconnect with Last-Event-ID.
|
||||
type AgentEventBacklog interface {
|
||||
ListEventMetaAfter(ctx context.Context, agentName string, afterID int64, limit int) ([]*messaging.EventMeta, error)
|
||||
}
|
||||
|
||||
// AgentMessageEvent is the payload of a new_message event on the agent stream.
|
||||
// It carries metadata only, never the message body.
|
||||
type AgentMessageEvent struct {
|
||||
MessageID int64 `json:"message_id"`
|
||||
Channel string `json:"channel,omitempty"`
|
||||
FromAgent string `json:"from_agent,omitempty"`
|
||||
ToAgent string `json:"to_agent,omitempty"`
|
||||
Subject string `json:"subject,omitempty"`
|
||||
}
|
||||
|
||||
// agentSub is one connected /api/agent-events client.
|
||||
type agentSub struct {
|
||||
ch chan AgentMessageEvent
|
||||
}
|
||||
|
||||
// SetAgentEventBacklog configures the source used to replay missed events.
|
||||
func (h *SSEHub) SetAgentEventBacklog(b AgentEventBacklog) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.agentBacklog = b
|
||||
}
|
||||
|
||||
// hasAgentSubs reports whether any /api/agent-events connection is open.
|
||||
func (h *SSEHub) hasAgentSubs() bool {
|
||||
h.mu.RLock()
|
||||
defer h.mu.RUnlock()
|
||||
return len(h.agentSubs) > 0
|
||||
}
|
||||
|
||||
// BroadcastAgentMessage delivers a new_message event to every connection of the
|
||||
// named agents. Connections that cannot keep up are dropped.
|
||||
func (h *SSEHub) BroadcastAgentMessage(agentNames []string, ev AgentMessageEvent) {
|
||||
type slowSub struct {
|
||||
name string
|
||||
sub *agentSub
|
||||
}
|
||||
var slow []slowSub
|
||||
|
||||
h.mu.RLock()
|
||||
for _, name := range agentNames {
|
||||
for sub := range h.agentSubs[name] {
|
||||
select {
|
||||
case sub.ch <- ev:
|
||||
default:
|
||||
slow = append(slow, slowSub{name, sub})
|
||||
}
|
||||
}
|
||||
}
|
||||
h.mu.RUnlock()
|
||||
|
||||
for _, s := range slow {
|
||||
h.logger.Warn("dropping slow agent SSE client", "agent", s.name)
|
||||
h.removeAgentSub(s.name, s.sub)
|
||||
}
|
||||
}
|
||||
|
||||
// addAgentSub registers a connection for the agent. It returns nil when the
|
||||
// agent already has maxAgentConnections open connections.
|
||||
func (h *SSEHub) addAgentSub(agentName string) *agentSub {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
|
||||
set := h.agentSubs[agentName]
|
||||
if len(set) >= maxAgentConnections {
|
||||
return nil
|
||||
}
|
||||
if set == nil {
|
||||
set = make(map[*agentSub]struct{})
|
||||
h.agentSubs[agentName] = set
|
||||
}
|
||||
sub := &agentSub{ch: make(chan AgentMessageEvent, agentSubBuffer)}
|
||||
set[sub] = struct{}{}
|
||||
h.logger.Info("agent SSE client connected", "agent", agentName)
|
||||
return sub
|
||||
}
|
||||
|
||||
func (h *SSEHub) removeAgentSub(agentName string, sub *agentSub) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
|
||||
set := h.agentSubs[agentName]
|
||||
if _, ok := set[sub]; ok {
|
||||
delete(set, sub)
|
||||
close(sub.ch)
|
||||
h.logger.Info("agent SSE client disconnected", "agent", agentName)
|
||||
}
|
||||
if len(set) == 0 {
|
||||
delete(h.agentSubs, agentName)
|
||||
}
|
||||
}
|
||||
|
||||
// closeAgentSubsLocked disconnects all agent subscriptions. Caller holds h.mu.
|
||||
func (h *SSEHub) closeAgentSubsLocked() {
|
||||
for name, set := range h.agentSubs {
|
||||
for sub := range set {
|
||||
close(sub.ch)
|
||||
}
|
||||
delete(h.agentSubs, name)
|
||||
}
|
||||
}
|
||||
|
||||
// HandleAgentEvents handles GET /api/agent-events. It must be mounted behind
|
||||
// the agent API-key middleware; the authenticated agent is read from the
|
||||
// request context. Only metadata for messages visible to that agent is sent.
|
||||
func (h *SSEHub) HandleAgentEvents(w http.ResponseWriter, r *http.Request) {
|
||||
agent, ok := agents.AgentFromContext(r.Context())
|
||||
if !ok || agent == nil {
|
||||
writeAgentEventsError(w, http.StatusUnauthorized, "unauthorized", "An agent API key is required")
|
||||
return
|
||||
}
|
||||
|
||||
flusher, ok := w.(http.Flusher)
|
||||
if !ok {
|
||||
http.Error(w, "Streaming not supported", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
var lastID int64
|
||||
hasLast := false
|
||||
if v := r.Header.Get("Last-Event-ID"); v != "" {
|
||||
id, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil || id < 0 {
|
||||
writeAgentEventsError(w, http.StatusBadRequest, "bad_request", "Last-Event-ID must be a message id")
|
||||
return
|
||||
}
|
||||
lastID, hasLast = id, true
|
||||
}
|
||||
|
||||
// Subscribe before replaying so no message falls between the replay query
|
||||
// and the live stream; duplicates are removed by id below.
|
||||
sub := h.addAgentSub(agent.Name)
|
||||
if sub == nil {
|
||||
writeAgentEventsError(w, http.StatusTooManyRequests, "too_many_connections",
|
||||
fmt.Sprintf("Agent %s already has the maximum of %d event stream connections", agent.Name, maxAgentConnections))
|
||||
return
|
||||
}
|
||||
defer h.removeAgentSub(agent.Name, sub)
|
||||
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
w.Header().Set("X-Accel-Buffering", "no")
|
||||
|
||||
rc := http.NewResponseController(w)
|
||||
write := func(id int64, eventType string, data any) bool {
|
||||
_ = rc.SetWriteDeadline(time.Now().Add(agentWriteTimeout))
|
||||
return writeSSEEvent(w, flusher, id, eventType, data)
|
||||
}
|
||||
|
||||
if !write(0, "connected", map[string]any{
|
||||
"agent": agent.Name,
|
||||
"timestamp": time.Now().Format(time.RFC3339),
|
||||
}) {
|
||||
return
|
||||
}
|
||||
|
||||
// replayedUpTo is fixed once the replay ends: live events at or below it
|
||||
// are duplicates of the replay. It is never advanced by live events, because
|
||||
// concurrent senders may broadcast ids out of order (11 before 10).
|
||||
replayedUpTo := lastID
|
||||
if hasLast {
|
||||
h.mu.RLock()
|
||||
backlog := h.agentBacklog
|
||||
h.mu.RUnlock()
|
||||
|
||||
if backlog != nil {
|
||||
metas, err := backlog.ListEventMetaAfter(r.Context(), agent.Name, lastID, maxAgentBacklog+1)
|
||||
if err != nil || len(metas) > maxAgentBacklog {
|
||||
if err != nil {
|
||||
h.logger.Warn("agent SSE backlog failed", "agent", agent.Name, "error", err)
|
||||
}
|
||||
if !write(0, "resync_required", map[string]any{
|
||||
"after_id": lastID,
|
||||
"limit": maxAgentBacklog,
|
||||
}) {
|
||||
return
|
||||
}
|
||||
} else {
|
||||
for _, m := range metas {
|
||||
ev := AgentMessageEvent{MessageID: m.MessageID, Subject: m.Subject}
|
||||
if m.Channel != "" {
|
||||
ev.Channel = m.Channel
|
||||
} else {
|
||||
ev.FromAgent, ev.ToAgent = m.FromAgent, m.ToAgent
|
||||
}
|
||||
if !write(m.MessageID, "new_message", ev) {
|
||||
return
|
||||
}
|
||||
replayedUpTo = m.MessageID
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
interval := h.heartbeat
|
||||
if interval <= 0 {
|
||||
interval = 30 * time.Second
|
||||
}
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
ctx := r.Context()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case ev, ok := <-sub.ch:
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if ev.MessageID <= replayedUpTo {
|
||||
continue
|
||||
}
|
||||
if !write(ev.MessageID, "new_message", ev) {
|
||||
return
|
||||
}
|
||||
case <-ticker.C:
|
||||
if !write(0, "heartbeat", map[string]any{
|
||||
"timestamp": time.Now().Format(time.RFC3339),
|
||||
}) {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// writeAgentEventsError writes a JSON error body encoded with encoding/json.
|
||||
func writeAgentEventsError(w http.ResponseWriter, status int, code, message string) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(map[string]string{"error": code, "message": message})
|
||||
}
|
||||
|
||||
// writeSSEEvent writes one SSE frame. A positive id is emitted as the "id:"
|
||||
// line. It reports whether the write succeeded.
|
||||
func writeSSEEvent(w http.ResponseWriter, flusher http.Flusher, id int64, eventType string, data any) bool {
|
||||
jsonData, err := json.Marshal(data)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
if id > 0 {
|
||||
if _, err := fmt.Fprintf(w, "id: %d\n", id); err != nil {
|
||||
return false
|
||||
}
|
||||
}
|
||||
if _, err := fmt.Fprintf(w, "event: %s\ndata: %s\n\n", eventType, jsonData); err != nil {
|
||||
return false
|
||||
}
|
||||
flusher.Flush()
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,546 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
_ "modernc.org/sqlite"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/channels"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
)
|
||||
|
||||
// agentEventsEnv wires real services over a temporary in-memory SQLite
|
||||
// database behind the real agent API-key middleware.
|
||||
type agentEventsEnv struct {
|
||||
t *testing.T
|
||||
srv *httptest.Server
|
||||
hub *SSEHub
|
||||
msgSvc *messaging.MessagingService
|
||||
chanSvc *channels.Service
|
||||
keys map[string]string // agent name -> API key
|
||||
channels map[string]int64 // channel name -> id
|
||||
}
|
||||
|
||||
func newAgentEventsEnv(t *testing.T, agentNames ...string) *agentEventsEnv {
|
||||
t.Helper()
|
||||
db := newTestDBFull(t)
|
||||
|
||||
agentSvc := agents.NewAgentService(agents.NewSQLiteAgentStore(db), nil)
|
||||
msgSvc := messaging.NewMessagingService(messaging.NewSQLiteMessageStore(db), nil)
|
||||
chanSvc := channels.NewService(channels.NewSQLiteChannelStore(db), msgSvc, nil)
|
||||
|
||||
hub := NewSSEHub()
|
||||
hub.SetAgentEventBacklog(msgSvc)
|
||||
b := NewSSEBroadcaster(hub, agentSvc, chanSvc)
|
||||
b.SetMessageService(msgSvc)
|
||||
msgSvc.AddMessageListener(b)
|
||||
|
||||
env := &agentEventsEnv{
|
||||
t: t, hub: hub, msgSvc: msgSvc, chanSvc: chanSvc,
|
||||
keys: map[string]string{}, channels: map[string]int64{},
|
||||
}
|
||||
ctx := context.Background()
|
||||
for _, name := range agentNames {
|
||||
_, key, err := agentSvc.Register(ctx, name, name, "ai", nil, 1)
|
||||
if err != nil {
|
||||
t.Fatalf("register %s: %v", name, err)
|
||||
}
|
||||
env.keys[name] = key
|
||||
}
|
||||
|
||||
r := chi.NewRouter()
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Use(agents.RequiredAuthMiddlewareWithOAuth(agentSvc, nil, nil))
|
||||
r.Get("/api/agent-events", hub.HandleAgentEvents)
|
||||
})
|
||||
env.srv = httptest.NewServer(r)
|
||||
t.Cleanup(env.srv.Close)
|
||||
t.Cleanup(hub.Close) // runs before srv.Close so open streams end
|
||||
return env
|
||||
}
|
||||
|
||||
func (e *agentEventsEnv) makeChannel(name string, members ...string) {
|
||||
e.t.Helper()
|
||||
ctx := context.Background()
|
||||
ch, err := e.chanSvc.CreateChannel(ctx, channels.CreateChannelRequest{Name: name, CreatedBy: members[0]})
|
||||
if err != nil {
|
||||
e.t.Fatalf("create channel: %v", err)
|
||||
}
|
||||
e.channels[name] = ch.ID
|
||||
for _, m := range members {
|
||||
if err := e.chanSvc.JoinChannel(ctx, ch.ID, m); err != nil {
|
||||
e.t.Fatalf("join %s: %v", m, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (e *agentEventsEnv) dm(from, to, body, subject string) int64 {
|
||||
e.t.Helper()
|
||||
m, err := e.msgSvc.SendMessage(context.Background(), from, to, body, messaging.SendOptions{Subject: subject})
|
||||
if err != nil {
|
||||
e.t.Fatalf("send dm: %v", err)
|
||||
}
|
||||
return m.ID
|
||||
}
|
||||
|
||||
func (e *agentEventsEnv) post(from, channel, body string) int64 {
|
||||
e.t.Helper()
|
||||
id := e.channels[channel]
|
||||
m, err := e.msgSvc.SendMessage(context.Background(), from, "", body, messaging.SendOptions{ChannelID: &id})
|
||||
if err != nil {
|
||||
e.t.Fatalf("post: %v", err)
|
||||
}
|
||||
return m.ID
|
||||
}
|
||||
|
||||
// sseFrame is one parsed SSE frame.
|
||||
type sseFrame struct {
|
||||
ID string
|
||||
Event string
|
||||
Data string
|
||||
}
|
||||
|
||||
type sseClient struct {
|
||||
t *testing.T
|
||||
frames chan sseFrame
|
||||
cancel context.CancelFunc
|
||||
status int
|
||||
body string
|
||||
}
|
||||
|
||||
// connect opens the stream. For non-200 statuses it records status/body only.
|
||||
func (e *agentEventsEnv) connect(key, lastEventID string) *sseClient {
|
||||
e.t.Helper()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
req, _ := http.NewRequestWithContext(ctx, "GET", e.srv.URL+"/api/agent-events", nil)
|
||||
if key != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+key)
|
||||
}
|
||||
if lastEventID != "" {
|
||||
req.Header.Set("Last-Event-ID", lastEventID)
|
||||
}
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
cancel()
|
||||
e.t.Fatalf("connect: %v", err)
|
||||
}
|
||||
c := &sseClient{t: e.t, frames: make(chan sseFrame, 512), cancel: cancel, status: resp.StatusCode}
|
||||
e.t.Cleanup(cancel)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
var sb strings.Builder
|
||||
sc := bufio.NewScanner(resp.Body)
|
||||
for sc.Scan() {
|
||||
sb.WriteString(sc.Text())
|
||||
}
|
||||
resp.Body.Close()
|
||||
c.body = sb.String()
|
||||
return c
|
||||
}
|
||||
go func() {
|
||||
defer resp.Body.Close()
|
||||
defer close(c.frames)
|
||||
sc := bufio.NewScanner(resp.Body)
|
||||
var f sseFrame
|
||||
for sc.Scan() {
|
||||
line := sc.Text()
|
||||
switch {
|
||||
case line == "":
|
||||
if f.Event != "" {
|
||||
c.frames <- f
|
||||
}
|
||||
f = sseFrame{}
|
||||
case strings.HasPrefix(line, "id: "):
|
||||
f.ID = line[4:]
|
||||
case strings.HasPrefix(line, "event: "):
|
||||
f.Event = line[7:]
|
||||
case strings.HasPrefix(line, "data: "):
|
||||
f.Data = line[6:]
|
||||
}
|
||||
}
|
||||
}()
|
||||
return c
|
||||
}
|
||||
|
||||
func (c *sseClient) next() sseFrame {
|
||||
c.t.Helper()
|
||||
select {
|
||||
case f, ok := <-c.frames:
|
||||
if !ok {
|
||||
c.t.Fatal("stream closed while waiting for event")
|
||||
}
|
||||
return f
|
||||
case <-time.After(5 * time.Second):
|
||||
c.t.Fatal("timed out waiting for SSE event")
|
||||
}
|
||||
return sseFrame{}
|
||||
}
|
||||
|
||||
// nextMessage skips nothing: it expects the next frame to be new_message.
|
||||
func (c *sseClient) nextMessage() (sseFrame, map[string]any) {
|
||||
c.t.Helper()
|
||||
f := c.next()
|
||||
if f.Event != "new_message" {
|
||||
c.t.Fatalf("event = %q (data %s), want new_message", f.Event, f.Data)
|
||||
}
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal([]byte(f.Data), &m); err != nil {
|
||||
c.t.Fatalf("bad json %q: %v", f.Data, err)
|
||||
}
|
||||
return f, m
|
||||
}
|
||||
|
||||
func (c *sseClient) expectConnected(agent string) {
|
||||
c.t.Helper()
|
||||
f := c.next()
|
||||
if f.Event != "connected" || !strings.Contains(f.Data, `"agent":"`+agent+`"`) {
|
||||
c.t.Fatalf("first frame = %+v, want connected for %s", f, agent)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgentEvents_Auth(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice")
|
||||
tests := []struct {
|
||||
name string
|
||||
key string
|
||||
}{
|
||||
{"missing key", ""},
|
||||
{"invalid key", "not-a-real-key"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
c := env.connect(tt.key, "")
|
||||
if c.status != http.StatusUnauthorized {
|
||||
t.Fatalf("status = %d, want 401", c.status)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("valid key gets connected event", func(t *testing.T) {
|
||||
c := env.connect(env.keys["alice"], "")
|
||||
if c.status != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", c.status)
|
||||
}
|
||||
c.expectConnected("alice")
|
||||
})
|
||||
}
|
||||
|
||||
func TestAgentEvents_HandlerWithoutAgentContext(t *testing.T) {
|
||||
hub := NewSSEHub()
|
||||
rr := httptest.NewRecorder()
|
||||
hub.HandleAgentEvents(rr, httptest.NewRequest("GET", "/api/agent-events", nil))
|
||||
if rr.Code != http.StatusUnauthorized {
|
||||
t.Fatalf("status = %d, want 401", rr.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgentEvents_Delivery(t *testing.T) {
|
||||
const secret = "TOP-SECRET-BODY-TEXT"
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
// send performs the action; returns the message id to expect for "bob".
|
||||
send func(env *agentEventsEnv) int64
|
||||
want map[string]any // expected fields (besides message_id)
|
||||
absent []string // fields that must not be present
|
||||
viewers []string
|
||||
}{
|
||||
{
|
||||
name: "dm to bob",
|
||||
send: func(env *agentEventsEnv) int64 { return env.dm("alice", "bob", secret, "hello subject") },
|
||||
want: map[string]any{"from_agent": "alice", "to_agent": "bob", "subject": "hello subject"},
|
||||
absent: []string{"channel", "body"},
|
||||
},
|
||||
{
|
||||
name: "channel message to member",
|
||||
send: func(env *agentEventsEnv) int64 {
|
||||
env.makeChannel("room", "alice", "bob")
|
||||
return env.post("alice", "room", secret)
|
||||
},
|
||||
want: map[string]any{"channel": "room"},
|
||||
absent: []string{"to_agent", "from_agent", "body"},
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice", "bob")
|
||||
bob := env.connect(env.keys["bob"], "")
|
||||
bob.expectConnected("bob")
|
||||
alice := env.connect(env.keys["alice"], "")
|
||||
alice.expectConnected("alice")
|
||||
|
||||
id := tt.send(env)
|
||||
|
||||
f, m := bob.nextMessage()
|
||||
if f.ID != strconv.FormatInt(id, 10) {
|
||||
t.Errorf("id line = %q, want %d", f.ID, id)
|
||||
}
|
||||
if int64(m["message_id"].(float64)) != id {
|
||||
t.Errorf("message_id = %v, want %d", m["message_id"], id)
|
||||
}
|
||||
for k, v := range tt.want {
|
||||
if m[k] != v {
|
||||
t.Errorf("%s = %v, want %v", k, m[k], v)
|
||||
}
|
||||
}
|
||||
for _, k := range tt.absent {
|
||||
if _, ok := m[k]; ok {
|
||||
t.Errorf("field %q must not be present: %s", k, f.Data)
|
||||
}
|
||||
}
|
||||
if strings.Contains(f.Data, secret) {
|
||||
t.Errorf("event leaks body: %s", f.Data)
|
||||
}
|
||||
|
||||
// Sender must not get its own message: send a canary to alice
|
||||
// and make sure it is the next thing she sees.
|
||||
canary := env.dm("bob", "alice", "canary", "")
|
||||
f2, _ := alice.nextMessage()
|
||||
if f2.ID != strconv.FormatInt(canary, 10) {
|
||||
t.Errorf("alice's next event id = %s, want canary %d (own message leaked?)", f2.ID, canary)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgentEvents_Isolation(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice", "bob", "carol")
|
||||
env.makeChannel("private-ish", "bob", "carol") // alice is NOT a member
|
||||
env.makeChannel("shared", "alice", "bob")
|
||||
|
||||
alice := env.connect(env.keys["alice"], "")
|
||||
alice.expectConnected("alice")
|
||||
|
||||
// None of these may reach alice.
|
||||
env.dm("bob", "carol", "b->c", "") // DM between others
|
||||
env.post("bob", "private-ish", "not for you") // channel alice has not joined
|
||||
env.dm("alice", "bob", "alice->bob", "") // her own message
|
||||
|
||||
// Canary she is allowed to see; it must be the very next event.
|
||||
canary := env.post("bob", "shared", "visible")
|
||||
f, m := alice.nextMessage()
|
||||
if f.ID != strconv.FormatInt(canary, 10) || m["channel"] != "shared" {
|
||||
t.Fatalf("alice got %+v, want only the canary in 'shared'", f)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgentEvents_Resume(t *testing.T) {
|
||||
t.Run("replays visible newer messages once, in order", func(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice", "bob", "carol")
|
||||
env.makeChannel("room", "alice", "bob")
|
||||
env.makeChannel("other", "bob", "carol")
|
||||
|
||||
first := env.dm("bob", "alice", "old", "") // client already has this
|
||||
m2 := env.dm("bob", "alice", "missed dm", "s")
|
||||
env.dm("bob", "carol", "not alice's", "")
|
||||
env.post("bob", "other", "not a member")
|
||||
m5 := env.post("bob", "room", "missed channel")
|
||||
env.dm("alice", "bob", "own", "") // own message excluded
|
||||
|
||||
c := env.connect(env.keys["alice"], strconv.FormatInt(first, 10))
|
||||
c.expectConnected("alice")
|
||||
|
||||
f, m := c.nextMessage()
|
||||
if f.ID != strconv.FormatInt(m2, 10) || m["subject"] != "s" {
|
||||
t.Fatalf("first replay = %+v", f)
|
||||
}
|
||||
f, m = c.nextMessage()
|
||||
if f.ID != strconv.FormatInt(m5, 10) || m["channel"] != "room" {
|
||||
t.Fatalf("second replay = %+v", f)
|
||||
}
|
||||
|
||||
// Live message after replay arrives exactly once.
|
||||
live := env.dm("bob", "alice", "live", "")
|
||||
f, _ = c.nextMessage()
|
||||
if f.ID != strconv.FormatInt(live, 10) {
|
||||
t.Fatalf("live event id = %s, want %d (duplicate or missing?)", f.ID, live)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("exactly 200 are replayed", func(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice", "bob")
|
||||
var ids []int64
|
||||
for i := 0; i < maxAgentBacklog; i++ {
|
||||
ids = append(ids, env.dm("bob", "alice", "x", ""))
|
||||
}
|
||||
c := env.connect(env.keys["alice"], "0")
|
||||
c.expectConnected("alice")
|
||||
for i, id := range ids {
|
||||
f, _ := c.nextMessage()
|
||||
if f.ID != strconv.FormatInt(id, 10) {
|
||||
t.Fatalf("replay #%d id = %s, want %d", i, f.ID, id)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("more than 200 sends resync_required only", func(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice", "bob")
|
||||
for i := 0; i < maxAgentBacklog+1; i++ {
|
||||
env.dm("bob", "alice", "x", "")
|
||||
}
|
||||
c := env.connect(env.keys["alice"], "0")
|
||||
c.expectConnected("alice")
|
||||
f := c.next()
|
||||
if f.Event != "resync_required" {
|
||||
t.Fatalf("event = %q, want resync_required", f.Event)
|
||||
}
|
||||
// No replay; the next event is the live one.
|
||||
live := env.dm("bob", "alice", "live", "")
|
||||
lf, _ := c.nextMessage()
|
||||
if lf.ID != strconv.FormatInt(live, 10) {
|
||||
t.Fatalf("after resync got id %s, want live %d", lf.ID, live)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("invalid Last-Event-ID is rejected", func(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice")
|
||||
c := env.connect(env.keys["alice"], "abc")
|
||||
if c.status != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want 400", c.status)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestAgentEvents_ConnectionLimit(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice", "bob")
|
||||
|
||||
var conns []*sseClient
|
||||
for i := 0; i < maxAgentConnections; i++ {
|
||||
c := env.connect(env.keys["alice"], "")
|
||||
if c.status != http.StatusOK {
|
||||
t.Fatalf("connection %d status = %d, want 200", i+1, c.status)
|
||||
}
|
||||
c.expectConnected("alice")
|
||||
conns = append(conns, c)
|
||||
}
|
||||
|
||||
over := env.connect(env.keys["alice"], "")
|
||||
if over.status != http.StatusTooManyRequests {
|
||||
t.Fatalf("6th connection status = %d, want 429", over.status)
|
||||
}
|
||||
var errBody map[string]string
|
||||
if err := json.Unmarshal([]byte(over.body), &errBody); err != nil {
|
||||
t.Fatalf("429 body is not valid JSON: %q: %v", over.body, err)
|
||||
}
|
||||
if errBody["error"] != "too_many_connections" || !strings.Contains(errBody["message"], "alice") {
|
||||
t.Errorf("429 body = %v, want error=too_many_connections naming the agent", errBody)
|
||||
}
|
||||
|
||||
// The limit is per agent.
|
||||
other := env.connect(env.keys["bob"], "")
|
||||
if other.status != http.StatusOK {
|
||||
t.Fatalf("other agent status = %d, want 200", other.status)
|
||||
}
|
||||
|
||||
// Closing a connection frees a slot.
|
||||
conns[0].cancel()
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
c := env.connect(env.keys["alice"], "")
|
||||
if c.status == http.StatusOK {
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("slot was not released after disconnect")
|
||||
}
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgentEvents_SlowClientDroppedWithoutBlocking(t *testing.T) {
|
||||
hub := NewSSEHub()
|
||||
sub := hub.addAgentSub("alice")
|
||||
if sub == nil {
|
||||
t.Fatal("addAgentSub returned nil")
|
||||
}
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
defer close(done)
|
||||
for i := 1; i <= agentSubBuffer+10; i++ { // nobody reads sub.ch
|
||||
hub.BroadcastAgentMessage([]string{"alice"}, AgentMessageEvent{MessageID: int64(i)})
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("broadcast blocked on a slow client")
|
||||
}
|
||||
hub.mu.RLock()
|
||||
n := len(hub.agentSubs["alice"])
|
||||
hub.mu.RUnlock()
|
||||
if n != 0 {
|
||||
t.Fatalf("slow subscription still registered (%d)", n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgentEvents_Heartbeat(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice")
|
||||
env.hub.heartbeat = 50 * time.Millisecond
|
||||
c := env.connect(env.keys["alice"], "")
|
||||
c.expectConnected("alice")
|
||||
if f := c.next(); f.Event != "heartbeat" {
|
||||
t.Fatalf("event = %q, want heartbeat", f.Event)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgentEvents_OutOfOrderLiveEventsNotDropped(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice")
|
||||
c := env.connect(env.keys["alice"], "")
|
||||
c.expectConnected("alice")
|
||||
|
||||
// Concurrent senders can broadcast ids out of order; both must arrive.
|
||||
env.hub.BroadcastAgentMessage([]string{"alice"}, AgentMessageEvent{MessageID: 11, FromAgent: "x", ToAgent: "alice"})
|
||||
env.hub.BroadcastAgentMessage([]string{"alice"}, AgentMessageEvent{MessageID: 10, FromAgent: "y", ToAgent: "alice"})
|
||||
|
||||
for _, want := range []string{"11", "10"} {
|
||||
if f, _ := c.nextMessage(); f.ID != want {
|
||||
t.Fatalf("got id %s, want %s", f.ID, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgentEvents_OutOfOrderLiveAfterReplay(t *testing.T) {
|
||||
env := newAgentEventsEnv(t, "alice", "bob")
|
||||
first := env.dm("bob", "alice", "old", "")
|
||||
missed := env.dm("bob", "alice", "missed", "")
|
||||
|
||||
c := env.connect(env.keys["alice"], strconv.FormatInt(first, 10))
|
||||
c.expectConnected("alice")
|
||||
if f, _ := c.nextMessage(); f.ID != strconv.FormatInt(missed, 10) {
|
||||
t.Fatalf("replay id = %s, want %d", f.ID, missed)
|
||||
}
|
||||
|
||||
// A duplicate of the replayed message is skipped; later out-of-order ids pass.
|
||||
env.hub.BroadcastAgentMessage([]string{"alice"}, AgentMessageEvent{MessageID: missed})
|
||||
env.hub.BroadcastAgentMessage([]string{"alice"}, AgentMessageEvent{MessageID: missed + 20})
|
||||
env.hub.BroadcastAgentMessage([]string{"alice"}, AgentMessageEvent{MessageID: missed + 10})
|
||||
for _, want := range []int64{missed + 20, missed + 10} {
|
||||
if f, _ := c.nextMessage(); f.ID != strconv.FormatInt(want, 10) {
|
||||
t.Fatalf("got id %s, want %d", f.ID, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSSEHub_HasAgentSubs(t *testing.T) {
|
||||
hub := NewSSEHub()
|
||||
if hub.hasAgentSubs() {
|
||||
t.Fatal("empty hub reports subscriptions")
|
||||
}
|
||||
sub := hub.addAgentSub("alice")
|
||||
if !hub.hasAgentSubs() {
|
||||
t.Fatal("hub with a subscription reports none")
|
||||
}
|
||||
hub.removeAgentSub("alice", sub)
|
||||
if hub.hasAgentSubs() {
|
||||
t.Fatal("hub still reports subscriptions after removal")
|
||||
}
|
||||
}
|
||||
@@ -21,6 +21,8 @@ type AnalyticsHandler struct {
|
||||
}
|
||||
|
||||
// NewAnalyticsHandler creates a new analytics handler.
|
||||
// db should be the read pool — every query in this handler is read-only and
|
||||
// must not block on the serialized write pool.
|
||||
func NewAnalyticsHandler(db *sql.DB, agentService *agents.AgentService, channelService *channels.Service) *AnalyticsHandler {
|
||||
return &AnalyticsHandler{
|
||||
db: db,
|
||||
|
||||
@@ -35,6 +35,7 @@ type SSEBroadcaster struct {
|
||||
hub *SSEHub
|
||||
agentService *agents.AgentService
|
||||
channelService *channels.Service
|
||||
msgService *messaging.MessagingService // optional: resolves conversation subjects
|
||||
logger *slog.Logger
|
||||
}
|
||||
|
||||
@@ -48,6 +49,12 @@ func NewSSEBroadcaster(hub *SSEHub, agentService *agents.AgentService, channelSe
|
||||
}
|
||||
}
|
||||
|
||||
// SetMessageService sets the messaging service used to resolve conversation
|
||||
// subjects for agent events. Optional; without it events carry no subject.
|
||||
func (b *SSEBroadcaster) SetMessageService(svc *messaging.MessagingService) {
|
||||
b.msgService = svc
|
||||
}
|
||||
|
||||
// BroadcastNewMessage sends a new_message event to the given owner.
|
||||
func (b *SSEBroadcaster) BroadcastNewMessage(_ context.Context, ownerID int64, event NewMessageEvent) {
|
||||
b.hub.Broadcast(ownerID, SSEEvent{
|
||||
@@ -128,4 +135,53 @@ func (b *SSEBroadcaster) OnMessageSent(ctx context.Context, msg *messaging.Messa
|
||||
} else {
|
||||
b.BroadcastDM(ctx, event)
|
||||
}
|
||||
|
||||
b.broadcastToAgents(ctx, msg)
|
||||
}
|
||||
|
||||
// broadcastToAgents pushes a body-free new_message event to the per-agent
|
||||
// stream (GET /api/agent-events): the DM recipient, or the members of the
|
||||
// channel at send time. The sender is never notified of its own message.
|
||||
func (b *SSEBroadcaster) broadcastToAgents(ctx context.Context, msg *messaging.Message) {
|
||||
if !b.hub.hasAgentSubs() {
|
||||
return // avoid member/subject DB lookups when nobody is listening
|
||||
}
|
||||
var recipients []string
|
||||
if msg.ChannelID != nil {
|
||||
if b.channelService == nil {
|
||||
return
|
||||
}
|
||||
members, err := b.channelService.GetMembers(ctx, *msg.ChannelID)
|
||||
if err != nil {
|
||||
b.logger.Debug("could not get channel members for agent SSE broadcast",
|
||||
"channel_id", *msg.ChannelID, "error", err)
|
||||
return
|
||||
}
|
||||
for _, m := range members {
|
||||
if m.AgentName != msg.FromAgent {
|
||||
recipients = append(recipients, m.AgentName)
|
||||
}
|
||||
}
|
||||
} else if msg.ToAgent != "" && msg.ToAgent != msg.FromAgent {
|
||||
recipients = []string{msg.ToAgent}
|
||||
}
|
||||
if len(recipients) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
ev := AgentMessageEvent{
|
||||
MessageID: msg.ID,
|
||||
FromAgent: msg.FromAgent,
|
||||
ToAgent: msg.ToAgent,
|
||||
}
|
||||
if msg.ChannelID != nil {
|
||||
ev.FromAgent = ""
|
||||
if ch, err := b.channelService.GetChannel(ctx, *msg.ChannelID); err == nil {
|
||||
ev.Channel = ch.Name
|
||||
}
|
||||
}
|
||||
if b.msgService != nil {
|
||||
ev.Subject = b.msgService.GetConversationSubject(ctx, msg.ConversationID)
|
||||
}
|
||||
b.hub.BroadcastAgentMessage(recipients, ev)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
// REST endpoints for per-(owner, agent) core memory (feature 020 — US2).
|
||||
// Surfaces the underlying messaging.CoreMemoryStore to the Web UI under
|
||||
// `/api/owner/{ownerID}/agents/{agentName}/core-memory`.
|
||||
//
|
||||
// Auth: every handler enforces that the session-bound owner matches the
|
||||
// path's `ownerID`. Cross-owner access yields 403 to avoid leaking the
|
||||
// existence of another owner's resources.
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strconv"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
)
|
||||
|
||||
// MemoryCoreHandler exposes GET/PUT/DELETE for the `memory_core` table.
|
||||
type MemoryCoreHandler struct {
|
||||
store *messaging.CoreMemoryStore
|
||||
logger *slog.Logger
|
||||
}
|
||||
|
||||
// NewMemoryCoreHandler wires the handler. `store` must be non-nil.
|
||||
func NewMemoryCoreHandler(store *messaging.CoreMemoryStore) *MemoryCoreHandler {
|
||||
return &MemoryCoreHandler{
|
||||
store: store,
|
||||
logger: slog.Default().With("component", "api.memory-core"),
|
||||
}
|
||||
}
|
||||
|
||||
// authorize resolves the URL `ownerID` and confirms it matches the
|
||||
// session-bound owner. Returns the resolved owner string (matching the
|
||||
// memory_core.owner_id TEXT format) plus the agent name on success.
|
||||
func (h *MemoryCoreHandler) authorize(w http.ResponseWriter, r *http.Request) (ownerStr string, agentName string, ok bool) {
|
||||
sessionOwnerID, found := OwnerIDFromContext(r.Context())
|
||||
if !found {
|
||||
writeJSON(w, http.StatusUnauthorized, errorBody("unauthorized", "Authentication required"))
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
pathOwner := chi.URLParam(r, "ownerID")
|
||||
if pathOwner == "" {
|
||||
writeJSON(w, http.StatusBadRequest, errorBody("bad_request", "ownerID is required"))
|
||||
return "", "", false
|
||||
}
|
||||
pathOwnerID, err := strconv.ParseInt(pathOwner, 10, 64)
|
||||
if err != nil || pathOwnerID <= 0 {
|
||||
writeJSON(w, http.StatusBadRequest, errorBody("bad_request", "ownerID must be a positive integer"))
|
||||
return "", "", false
|
||||
}
|
||||
if pathOwnerID != sessionOwnerID {
|
||||
// Use 403, not 404, so the response shape matches other owner-scoped
|
||||
// handlers in this package (see agents_handler.GetAgent).
|
||||
writeJSON(w, http.StatusForbidden, errorBody("forbidden", "You do not have access to this owner"))
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
agentName = chi.URLParam(r, "agentName")
|
||||
if agentName == "" {
|
||||
writeJSON(w, http.StatusBadRequest, errorBody("bad_request", "agentName is required"))
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
return strconv.FormatInt(pathOwnerID, 10), agentName, true
|
||||
}
|
||||
|
||||
// Get handles GET /api/owner/{ownerID}/agents/{agentName}/core-memory.
|
||||
func (h *MemoryCoreHandler) Get(w http.ResponseWriter, r *http.Request) {
|
||||
if h.store == nil {
|
||||
writeJSON(w, http.StatusServiceUnavailable, errorBody("unavailable", "core memory store not configured"))
|
||||
return
|
||||
}
|
||||
ownerStr, agentName, ok := h.authorize(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
blob, updatedAt, exists, err := h.store.Get(r.Context(), ownerStr, agentName)
|
||||
if err != nil {
|
||||
h.logger.Error("memory_core get failed", "error", err, "owner", ownerStr, "agent", agentName)
|
||||
writeJSON(w, http.StatusInternalServerError, errorBody("server_error", "Failed to read core memory"))
|
||||
return
|
||||
}
|
||||
if !exists {
|
||||
writeJSON(w, http.StatusNotFound, errorBody("not_found", "core memory not set for this agent"))
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{
|
||||
"owner_id": ownerStr,
|
||||
"agent_name": agentName,
|
||||
"blob": blob,
|
||||
"updated_at": updatedAt.Format("2006-01-02T15:04:05Z07:00"),
|
||||
})
|
||||
}
|
||||
|
||||
// Put handles PUT /api/owner/{ownerID}/agents/{agentName}/core-memory.
|
||||
// Body: {"blob": "..."}. Returns 200 on success, 413 on
|
||||
// core_memory_too_large, 400 on malformed body.
|
||||
func (h *MemoryCoreHandler) Put(w http.ResponseWriter, r *http.Request) {
|
||||
if h.store == nil {
|
||||
writeJSON(w, http.StatusServiceUnavailable, errorBody("unavailable", "core memory store not configured"))
|
||||
return
|
||||
}
|
||||
ownerStr, agentName, ok := h.authorize(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
var body struct {
|
||||
Blob string `json:"blob"`
|
||||
UpdatedBy string `json:"updated_by,omitempty"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeJSON(w, http.StatusBadRequest, errorBody("bad_request", "Invalid JSON body"))
|
||||
return
|
||||
}
|
||||
updatedBy := body.UpdatedBy
|
||||
if updatedBy == "" {
|
||||
updatedBy = "human"
|
||||
}
|
||||
if err := h.store.Set(r.Context(), ownerStr, agentName, body.Blob, updatedBy); err != nil {
|
||||
if errors.Is(err, messaging.ErrCoreMemoryTooLarge) {
|
||||
writeJSON(w, http.StatusRequestEntityTooLarge, errorBody(
|
||||
"core_memory_too_large",
|
||||
fmt.Sprintf("Blob exceeds %d bytes", h.store.MaxBytes()),
|
||||
))
|
||||
return
|
||||
}
|
||||
h.logger.Error("memory_core set failed", "error", err, "owner", ownerStr, "agent", agentName)
|
||||
writeJSON(w, http.StatusInternalServerError, errorBody("server_error", "Failed to write core memory"))
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{
|
||||
"owner_id": ownerStr,
|
||||
"agent_name": agentName,
|
||||
"blob_chars": len(body.Blob),
|
||||
"updated_by": updatedBy,
|
||||
})
|
||||
}
|
||||
|
||||
// Delete handles DELETE /api/owner/{ownerID}/agents/{agentName}/core-memory.
|
||||
// Returns 204 when a row existed and was removed, 404 when no row was
|
||||
// present at the start of the call.
|
||||
func (h *MemoryCoreHandler) Delete(w http.ResponseWriter, r *http.Request) {
|
||||
if h.store == nil {
|
||||
writeJSON(w, http.StatusServiceUnavailable, errorBody("unavailable", "core memory store not configured"))
|
||||
return
|
||||
}
|
||||
ownerStr, agentName, ok := h.authorize(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
// Check existence so we can return the canonical 204 vs 404. The
|
||||
// store's Delete is idempotent — it never errors on missing rows.
|
||||
_, _, exists, err := h.store.Get(r.Context(), ownerStr, agentName)
|
||||
if err != nil {
|
||||
h.logger.Error("memory_core get-before-delete failed", "error", err, "owner", ownerStr, "agent", agentName)
|
||||
writeJSON(w, http.StatusInternalServerError, errorBody("server_error", "Failed to read core memory"))
|
||||
return
|
||||
}
|
||||
if !exists {
|
||||
writeJSON(w, http.StatusNotFound, errorBody("not_found", "core memory not set for this agent"))
|
||||
return
|
||||
}
|
||||
if err := h.store.Delete(r.Context(), ownerStr, agentName); err != nil {
|
||||
h.logger.Error("memory_core delete failed", "error", err, "owner", ownerStr, "agent", agentName)
|
||||
writeJSON(w, http.StatusInternalServerError, errorBody("server_error", "Failed to delete core memory"))
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
+30
-5
@@ -15,9 +15,9 @@ import (
|
||||
"github.com/synapbus/synapbus/internal/harness/runs"
|
||||
"github.com/synapbus/synapbus/internal/k8s"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
"github.com/synapbus/synapbus/internal/reactor"
|
||||
"github.com/synapbus/synapbus/internal/push"
|
||||
"github.com/synapbus/synapbus/internal/reactions"
|
||||
"github.com/synapbus/synapbus/internal/reactor"
|
||||
"github.com/synapbus/synapbus/internal/trace"
|
||||
"github.com/synapbus/synapbus/internal/trust"
|
||||
"github.com/synapbus/synapbus/internal/webhooks"
|
||||
@@ -52,8 +52,16 @@ type RouterConfig struct {
|
||||
Broadcaster *SSEBroadcaster
|
||||
SessionMiddleware func(http.Handler) http.Handler
|
||||
DB *sql.DB
|
||||
Version string
|
||||
BaseURL string
|
||||
// ReadDB is the read-only connection pool (query_only=ON, MaxOpenConns>1).
|
||||
// Read-only API handlers must use this to avoid blocking on the serialized
|
||||
// write pool. If nil, handlers fall back to DB.
|
||||
ReadDB *sql.DB
|
||||
Version string
|
||||
BaseURL string
|
||||
|
||||
// CoreMemoryStore (feature 020 — US2) wires the per-(owner, agent)
|
||||
// core memory REST endpoints. Nil → routes not registered.
|
||||
CoreMemoryStore *messaging.CoreMemoryStore
|
||||
}
|
||||
|
||||
// NewRouter creates a chi router with all API routes configured.
|
||||
@@ -320,9 +328,14 @@ func NewRouterWithConfig(cfg RouterConfig) chi.Router {
|
||||
})
|
||||
}
|
||||
|
||||
// Analytics (authenticated, requires DB)
|
||||
// Analytics (authenticated, requires DB). Reads must use the read pool so
|
||||
// dashboard queries don't serialize behind long-running writers.
|
||||
if cfg.DB != nil {
|
||||
analyticsHandler := NewAnalyticsHandler(cfg.DB, cfg.AgentService, cfg.ChannelService)
|
||||
analyticsDB := cfg.ReadDB
|
||||
if analyticsDB == nil {
|
||||
analyticsDB = cfg.DB
|
||||
}
|
||||
analyticsHandler := NewAnalyticsHandler(analyticsDB, cfg.AgentService, cfg.ChannelService)
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Use(authMiddleware)
|
||||
|
||||
@@ -333,6 +346,18 @@ func NewRouterWithConfig(cfg RouterConfig) chi.Router {
|
||||
})
|
||||
}
|
||||
|
||||
// Per-(owner, agent) core memory (feature 020 — US2)
|
||||
if cfg.CoreMemoryStore != nil {
|
||||
coreHandler := NewMemoryCoreHandler(cfg.CoreMemoryStore)
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Use(authMiddleware)
|
||||
|
||||
r.Get("/api/owner/{ownerID}/agents/{agentName}/core-memory", coreHandler.Get)
|
||||
r.Put("/api/owner/{ownerID}/agents/{agentName}/core-memory", coreHandler.Put)
|
||||
r.Delete("/api/owner/{ownerID}/agents/{agentName}/core-memory", coreHandler.Delete)
|
||||
})
|
||||
}
|
||||
|
||||
// Version (unauthenticated)
|
||||
if cfg.Version != "" {
|
||||
versionHandler := NewVersionHandler(cfg.Version)
|
||||
|
||||
@@ -19,6 +19,11 @@ type SSEEvent struct {
|
||||
type SSEHub struct {
|
||||
mu sync.RWMutex
|
||||
clients map[int64]map[chan SSEEvent]struct{} // ownerID -> set of channels
|
||||
|
||||
// Per-agent subscriptions for GET /api/agent-events (see agent_events.go).
|
||||
agentSubs map[string]map[*agentSub]struct{}
|
||||
agentBacklog AgentEventBacklog
|
||||
heartbeat time.Duration
|
||||
nextID int64
|
||||
logger *slog.Logger
|
||||
}
|
||||
@@ -26,7 +31,9 @@ type SSEHub struct {
|
||||
// NewSSEHub creates a new SSE hub.
|
||||
func NewSSEHub() *SSEHub {
|
||||
return &SSEHub{
|
||||
clients: make(map[int64]map[chan SSEEvent]struct{}),
|
||||
clients: make(map[int64]map[chan SSEEvent]struct{}),
|
||||
agentSubs: make(map[string]map[*agentSub]struct{}),
|
||||
heartbeat: 30 * time.Second,
|
||||
logger: slog.Default().With("component", "api.sse"),
|
||||
}
|
||||
}
|
||||
@@ -76,6 +83,7 @@ func (h *SSEHub) Close() {
|
||||
}
|
||||
delete(h.clients, ownerID)
|
||||
}
|
||||
h.closeAgentSubsLocked()
|
||||
h.logger.Info("all SSE clients disconnected")
|
||||
}
|
||||
|
||||
|
||||
@@ -27,6 +27,7 @@ type TaskStore interface {
|
||||
// SQLiteTaskStore implements TaskStore using SQLite.
|
||||
type SQLiteTaskStore struct {
|
||||
db *sql.DB
|
||||
readDB *sql.DB
|
||||
logger *slog.Logger
|
||||
}
|
||||
|
||||
@@ -38,6 +39,23 @@ func NewSQLiteTaskStore(db *sql.DB) *SQLiteTaskStore {
|
||||
}
|
||||
}
|
||||
|
||||
// WithReadDB attaches a read-only connection pool used for cheap pre-checks
|
||||
// (e.g. ExpireTasks asks "any rows to expire?" before contending for the
|
||||
// serialized write connection). Returns the receiver for chaining. If never
|
||||
// called, all queries run against the write pool.
|
||||
func (s *SQLiteTaskStore) WithReadDB(readDB *sql.DB) *SQLiteTaskStore {
|
||||
s.readDB = readDB
|
||||
return s
|
||||
}
|
||||
|
||||
// queryDB returns the read pool if attached, otherwise the write pool.
|
||||
func (s *SQLiteTaskStore) queryDB() *sql.DB {
|
||||
if s.readDB != nil {
|
||||
return s.readDB
|
||||
}
|
||||
return s.db
|
||||
}
|
||||
|
||||
// sqliteTimeFormat is the format used for storing timestamps consistently in SQLite.
|
||||
const sqliteTimeFormat = "2006-01-02 15:04:05"
|
||||
|
||||
@@ -264,21 +282,82 @@ func (s *SQLiteTaskStore) UpdateBidStatus(ctx context.Context, bidID int64, stat
|
||||
return nil
|
||||
}
|
||||
|
||||
// expireTasksBatchSize bounds how many tasks a single UPDATE statement
|
||||
// rewrites. A small batch keeps the SQLite write transaction short, so
|
||||
// the expiry worker doesn't starve under WAL contention with concurrent
|
||||
// writers (e.g. message inserts, consolidator) and respects the worker's
|
||||
// context deadline even if the eligible set is large.
|
||||
const expireTasksBatchSize = 500
|
||||
|
||||
// ExpireTasks marks all open tasks past their deadline as cancelled.
|
||||
//
|
||||
// The work is chunked into bounded UPDATEs (LIMIT expireTasksBatchSize)
|
||||
// rather than a single unbounded UPDATE, for two reasons:
|
||||
//
|
||||
// 1. Bounded transactions: SQLite serializes writers, so a long UPDATE
|
||||
// blocks every other writer until commit. Chunking caps the lock
|
||||
// window per round-trip.
|
||||
// 2. Context responsiveness: the expiry worker uses a 30s context. A
|
||||
// single UPDATE doesn't observe ctx between rows, so a slow query
|
||||
// would always run to completion and then return ctx error. Looping
|
||||
// lets us bail between batches.
|
||||
//
|
||||
// Performance also depends on migration 031, which adds a partial
|
||||
// composite index `idx_tasks_expiry(status, deadline) WHERE status='open'
|
||||
// AND deadline IS NOT NULL`. The query below is shaped to match it.
|
||||
func (s *SQLiteTaskStore) ExpireTasks(ctx context.Context) (int, error) {
|
||||
// Use a string-formatted timestamp for consistent SQLite comparison
|
||||
now := time.Now().UTC().Format("2006-01-02 15:04:05")
|
||||
result, err := s.db.ExecContext(ctx,
|
||||
`UPDATE tasks SET status = ?, updated_at = CURRENT_TIMESTAMP
|
||||
WHERE status = ? AND deadline IS NOT NULL AND deadline < ?`,
|
||||
TaskStatusCancelled, TaskStatusOpen, now,
|
||||
)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("expire tasks: %w", err)
|
||||
now := time.Now().UTC().Format(sqliteTimeFormat)
|
||||
|
||||
// Fast pre-check on the read pool: if nothing is expirable right now,
|
||||
// don't queue behind the serialized write connection at all. The write
|
||||
// pool has MaxOpenConns=1 in production, so an idle ExpireTasks call
|
||||
// that takes the slot while a long writer holds it would block until
|
||||
// the context deadline — even though the work itself is empty. The
|
||||
// EXISTS query is index-resident (idx_tasks_expiry) and runs on a
|
||||
// concurrent read connection.
|
||||
var hasWork int
|
||||
if err := s.queryDB().QueryRowContext(ctx,
|
||||
`SELECT EXISTS(SELECT 1 FROM tasks
|
||||
WHERE status = ? AND deadline IS NOT NULL AND deadline < ?)`,
|
||||
TaskStatusOpen, now,
|
||||
).Scan(&hasWork); err != nil {
|
||||
return 0, fmt.Errorf("expire tasks pre-check: %w", err)
|
||||
}
|
||||
if hasWork == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
rowsAffected, _ := result.RowsAffected()
|
||||
return int(rowsAffected), nil
|
||||
total := 0
|
||||
for {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return total, fmt.Errorf("expire tasks: %w", err)
|
||||
}
|
||||
|
||||
// SQLite's UPDATE ... LIMIT is only enabled with the
|
||||
// SQLITE_ENABLE_UPDATE_DELETE_LIMIT compile flag (not on by
|
||||
// default in modernc.org/sqlite). Use a subquery on ROWID
|
||||
// to portably bound the batch.
|
||||
result, err := s.db.ExecContext(ctx,
|
||||
`UPDATE tasks SET status = ?, updated_at = CURRENT_TIMESTAMP
|
||||
WHERE rowid IN (
|
||||
SELECT rowid FROM tasks
|
||||
WHERE status = ? AND deadline IS NOT NULL AND deadline < ?
|
||||
LIMIT ?
|
||||
)`,
|
||||
TaskStatusCancelled, TaskStatusOpen, now, expireTasksBatchSize,
|
||||
)
|
||||
if err != nil {
|
||||
return total, fmt.Errorf("expire tasks: %w", err)
|
||||
}
|
||||
|
||||
rowsAffected, _ := result.RowsAffected()
|
||||
total += int(rowsAffected)
|
||||
if rowsAffected < int64(expireTasksBatchSize) {
|
||||
// Last (possibly empty) batch — nothing more to expire.
|
||||
return total, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// CancelTasksByChannel cancels all open tasks for a channel (used before channel deletion).
|
||||
|
||||
@@ -2,7 +2,9 @@ package channels
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
@@ -326,6 +328,209 @@ func TestSQLiteTaskStore_ExpireTasks(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestSQLiteTaskStore_ExpireTasks_LargeBatch exercises the expiry path at a
|
||||
// scale comparable to a long-lived production instance: a few thousand mixed
|
||||
// tasks (expired-open, future-open, no-deadline, cancelled), confirms the
|
||||
// worker chunks past its internal batch size, and that the entire run fits
|
||||
// inside a tight per-tick context — guarding the regression that the
|
||||
// expiry-worker hit on kubic ("context deadline exceeded").
|
||||
//
|
||||
// The total row count is deliberately larger than expireTasksBatchSize (500)
|
||||
// so the batching loop must iterate more than once.
|
||||
func TestSQLiteTaskStore_ExpireTasks_LargeBatch(t *testing.T) {
|
||||
taskStore, channelStore := newTestTaskStore(t)
|
||||
ch := createAuctionChannel(t, channelStore)
|
||||
ctx := context.Background()
|
||||
|
||||
const (
|
||||
expiredOpen = 1200 // > 2 * batch size, forces multiple iterations
|
||||
futureOpen = 400
|
||||
noDeadline = 400
|
||||
alreadyDone = 400
|
||||
)
|
||||
|
||||
past := time.Now().Add(-1 * time.Hour)
|
||||
future := time.Now().Add(1 * time.Hour)
|
||||
|
||||
for i := 0; i < expiredOpen; i++ {
|
||||
if err := taskStore.CreateTask(ctx, &Task{
|
||||
ChannelID: ch.ID, PostedBy: "poster-agent",
|
||||
Title: "expired", Status: TaskStatusOpen,
|
||||
Deadline: &past, Requirements: json.RawMessage(`{}`),
|
||||
}); err != nil {
|
||||
t.Fatalf("seed expired task %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
for i := 0; i < futureOpen; i++ {
|
||||
taskStore.CreateTask(ctx, &Task{
|
||||
ChannelID: ch.ID, PostedBy: "poster-agent",
|
||||
Title: "future", Status: TaskStatusOpen,
|
||||
Deadline: &future, Requirements: json.RawMessage(`{}`),
|
||||
})
|
||||
}
|
||||
for i := 0; i < noDeadline; i++ {
|
||||
taskStore.CreateTask(ctx, &Task{
|
||||
ChannelID: ch.ID, PostedBy: "poster-agent",
|
||||
Title: "no-deadline", Status: TaskStatusOpen,
|
||||
Requirements: json.RawMessage(`{}`),
|
||||
})
|
||||
}
|
||||
for i := 0; i < alreadyDone; i++ {
|
||||
taskStore.CreateTask(ctx, &Task{
|
||||
ChannelID: ch.ID, PostedBy: "poster-agent",
|
||||
Title: "done", Status: TaskStatusCompleted,
|
||||
Deadline: &past, Requirements: json.RawMessage(`{}`),
|
||||
})
|
||||
}
|
||||
|
||||
// Run with the same shape of context budget the worker uses, but tighter
|
||||
// (5s) so a regression to the unbounded-scan plan would fail this test
|
||||
// well before the worker's real 30s ceiling.
|
||||
tightCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
start := time.Now()
|
||||
count, err := taskStore.ExpireTasks(tightCtx)
|
||||
elapsed := time.Since(start)
|
||||
if err != nil {
|
||||
t.Fatalf("ExpireTasks: %v (elapsed=%s)", err, elapsed)
|
||||
}
|
||||
if count != expiredOpen {
|
||||
t.Errorf("expired count = %d, want %d", count, expiredOpen)
|
||||
}
|
||||
t.Logf("expired %d tasks in %s (over %d total rows)",
|
||||
count, elapsed, expiredOpen+futureOpen+noDeadline+alreadyDone)
|
||||
|
||||
// Future / no-deadline tasks must remain open.
|
||||
openTasks, _ := taskStore.ListTasks(ctx, ch.ID, TaskStatusOpen)
|
||||
if len(openTasks) != futureOpen+noDeadline {
|
||||
t.Errorf("open after expiry = %d, want %d",
|
||||
len(openTasks), futureOpen+noDeadline)
|
||||
}
|
||||
|
||||
// Second invocation on a clean set must be cheap and return 0.
|
||||
count2, err := taskStore.ExpireTasks(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ExpireTasks (idempotent run): %v", err)
|
||||
}
|
||||
if count2 != 0 {
|
||||
t.Errorf("idempotent run expired = %d, want 0", count2)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSQLiteTaskStore_ExpireTasks_UsesIndex confirms the partial composite
|
||||
// index from migration 031 is the plan the query optimizer picks. If a
|
||||
// future change drops the index or rewrites the query incompatibly, the
|
||||
// planner will fall back to a SCAN and this test will fail loudly.
|
||||
func TestSQLiteTaskStore_ExpireTasks_UsesIndex(t *testing.T) {
|
||||
taskStore, _ := newTestTaskStore(t)
|
||||
|
||||
// Seed a few rows so the planner has stats to work with.
|
||||
// (SQLite's planner is mostly schema-driven, but better safe.)
|
||||
ctx := context.Background()
|
||||
|
||||
now := time.Now().UTC().Format(sqliteTimeFormat)
|
||||
rows, err := taskStore.db.QueryContext(ctx,
|
||||
`EXPLAIN QUERY PLAN
|
||||
UPDATE tasks SET status = ?, updated_at = CURRENT_TIMESTAMP
|
||||
WHERE rowid IN (
|
||||
SELECT rowid FROM tasks
|
||||
WHERE status = ? AND deadline IS NOT NULL AND deadline < ?
|
||||
LIMIT ?
|
||||
)`,
|
||||
TaskStatusCancelled, TaskStatusOpen, now, expireTasksBatchSize,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("EXPLAIN QUERY PLAN: %v", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
sawIndex := false
|
||||
var plan []string
|
||||
for rows.Next() {
|
||||
var id, parent, notused int
|
||||
var detail string
|
||||
if err := rows.Scan(&id, &parent, ¬used, &detail); err != nil {
|
||||
t.Fatalf("scan plan row: %v", err)
|
||||
}
|
||||
plan = append(plan, detail)
|
||||
if contains(detail, "idx_tasks_expiry") {
|
||||
sawIndex = true
|
||||
}
|
||||
}
|
||||
if !sawIndex {
|
||||
t.Errorf("expected query plan to use idx_tasks_expiry, got: %v", plan)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSQLiteTaskStore_ExpireTasks_EmptyShortCircuits verifies that when the
|
||||
// pre-check finds no expirable rows, ExpireTasks returns (0, nil) without
|
||||
// running the UPDATE loop. In production this is the steady-state case
|
||||
// (tasks is empty most of the time) and the pre-check is what keeps the
|
||||
// worker off the serialized write connection.
|
||||
func TestSQLiteTaskStore_ExpireTasks_EmptyShortCircuits(t *testing.T) {
|
||||
taskStore, _ := newTestTaskStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
count, err := taskStore.ExpireTasks(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ExpireTasks on empty table: %v", err)
|
||||
}
|
||||
if count != 0 {
|
||||
t.Errorf("count = %d, want 0", count)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSQLiteTaskStore_ExpireTasks_UsesReadPoolForPreCheck verifies that when
|
||||
// a read pool is attached via WithReadDB, the EXISTS pre-check runs on it
|
||||
// (and the actual UPDATE still runs on the write pool when work is present).
|
||||
func TestSQLiteTaskStore_ExpireTasks_UsesReadPoolForPreCheck(t *testing.T) {
|
||||
taskStore, channelStore := newTestTaskStore(t)
|
||||
|
||||
// Open a separate handle to the same shared-cache memory DB.
|
||||
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", t.Name())
|
||||
readDB, err := sql.Open("sqlite", dsn)
|
||||
if err != nil {
|
||||
t.Fatalf("open read pool: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { readDB.Close() })
|
||||
|
||||
taskStore.WithReadDB(readDB)
|
||||
|
||||
ctx := context.Background()
|
||||
|
||||
// Empty: short-circuit on the read pool.
|
||||
if n, err := taskStore.ExpireTasks(ctx); err != nil || n != 0 {
|
||||
t.Fatalf("empty ExpireTasks with read pool: count=%d err=%v", n, err)
|
||||
}
|
||||
|
||||
// Seed one expirable task and verify the UPDATE still runs.
|
||||
ch := createAuctionChannel(t, channelStore)
|
||||
past := time.Now().Add(-1 * time.Hour)
|
||||
if err := taskStore.CreateTask(ctx, &Task{
|
||||
ChannelID: ch.ID, PostedBy: "poster-agent", Title: "Expired",
|
||||
Status: TaskStatusOpen, Deadline: &past, Requirements: json.RawMessage(`{}`),
|
||||
}); err != nil {
|
||||
t.Fatalf("create task: %v", err)
|
||||
}
|
||||
n, err := taskStore.ExpireTasks(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ExpireTasks with read pool: %v", err)
|
||||
}
|
||||
if n != 1 {
|
||||
t.Errorf("expired count = %d, want 1", n)
|
||||
}
|
||||
}
|
||||
|
||||
func contains(s, sub string) bool {
|
||||
for i := 0; i+len(sub) <= len(s); i++ {
|
||||
if s[i:i+len(sub)] == sub {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func TestSQLiteTaskStore_CancelTasksByChannel(t *testing.T) {
|
||||
taskStore, channelStore := newTestTaskStore(t)
|
||||
ch := createAuctionChannel(t, channelStore)
|
||||
|
||||
@@ -167,6 +167,22 @@ func (h *Harness) Execute(ctx context.Context, req *harness.ExecRequest) (*harne
|
||||
// agents emitting a final result envelope). If it parses, stash it.
|
||||
if rj := extractResultJSON(logs); rj != nil {
|
||||
res.ResultJSON = rj
|
||||
// Best-effort: pull token-usage fields when the agent emitted
|
||||
// them on the final line (claude-agent-sdk / dream-runner do).
|
||||
// Feeds into the dream worker's UsageGate circuit breaker so
|
||||
// consumption actually counts.
|
||||
var u struct {
|
||||
TokensIn int64 `json:"tokens_in"`
|
||||
TokensOut int64 `json:"tokens_out"`
|
||||
TokensCached int64 `json:"tokens_cached"`
|
||||
CostUSD float64 `json:"cost_usd"`
|
||||
}
|
||||
if err := json.Unmarshal(rj, &u); err == nil {
|
||||
res.Usage.TokensIn = u.TokensIn
|
||||
res.Usage.TokensOut = u.TokensOut
|
||||
res.Usage.TokensCached = u.TokensCached
|
||||
res.Usage.CostUSD = u.CostUSD
|
||||
}
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
+14
-1
@@ -2,11 +2,14 @@ package k8s
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
batchv1 "k8s.io/api/batch/v1"
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
@@ -94,7 +97,17 @@ func (r *K8sJobRunner) GetNamespace() string {
|
||||
}
|
||||
|
||||
func (r *K8sJobRunner) CreateJob(ctx context.Context, handler *K8sHandler, msg *JobMessage) (string, error) {
|
||||
jobName := sanitizeJobName(fmt.Sprintf("synapbus-%s-%d", handler.AgentName, msg.MessageID))
|
||||
// When there is no triggering message (e.g. dream-worker dispatches
|
||||
// where Message=nil → MessageID=0), fall back to a unique suffix so
|
||||
// concurrent runs don't collide on the Job name. Format keeps the
|
||||
// historic "synapbus-<agent>-<id>" prefix for log/grep continuity.
|
||||
suffix := fmt.Sprintf("%d", msg.MessageID)
|
||||
if msg.MessageID == 0 {
|
||||
var b [4]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
suffix = fmt.Sprintf("%d-%s", time.Now().UnixNano()%1_000_000, hex.EncodeToString(b[:]))
|
||||
}
|
||||
jobName := sanitizeJobName(fmt.Sprintf("synapbus-%s-%s", handler.AgentName, suffix))
|
||||
|
||||
namespace := handler.Namespace
|
||||
if namespace == "" {
|
||||
|
||||
@@ -67,8 +67,45 @@ func NewServiceBridge(
|
||||
}
|
||||
}
|
||||
|
||||
// bridgeActionAliases maps observed wrong action names that agents have called
|
||||
// to the real bridge action they probably meant. This is a small, hand-curated
|
||||
// whitelist of guesses seen in production logs — not a fuzzy-match layer.
|
||||
// Keep entries minimal and only add names that have a single unambiguous target.
|
||||
var bridgeActionAliases = map[string]string{
|
||||
// read_channel → fetch messages from a channel (requires channel_id/name)
|
||||
"read_channel": "get_channel_messages",
|
||||
// search → unified search over messages
|
||||
"search": "search_messages",
|
||||
// read_dm → DMs land in the inbox
|
||||
"read_dm": "read_inbox",
|
||||
// my_status → no exact bridge equivalent; agents probing for "what's new"
|
||||
// most often want read_inbox. (The top-level MCP `my_status` tool is the
|
||||
// canonical way; this alias keeps call() from failing.)
|
||||
"my_status": "read_inbox",
|
||||
// read_article → wiki retrieval
|
||||
"read_article": "get_article",
|
||||
}
|
||||
|
||||
// bridgeTopLevelOnly lists action names that exist as top-level MCP tools
|
||||
// (registered on the MCP server), not as call() bridge actions. Agents that
|
||||
// invoke these via call() / execute() get a targeted error pointing them at
|
||||
// the real tool rather than a generic "unknown action". Keyed by what the
|
||||
// agent typed; value is the canonical top-level tool name.
|
||||
var bridgeTopLevelOnly = map[string]string{
|
||||
"rewrite_core_memory": "memory_rewrite_core",
|
||||
"memory_rewrite_core": "memory_rewrite_core",
|
||||
}
|
||||
|
||||
// Call dispatches an action by name to the appropriate service method.
|
||||
func (b *ServiceBridge) Call(ctx context.Context, actionName string, args map[string]any) (any, error) {
|
||||
// Apply aliases for common wrong-name guesses observed in agent logs.
|
||||
if real, ok := bridgeActionAliases[actionName]; ok {
|
||||
actionName = real
|
||||
}
|
||||
// Catch wrong guesses that map to top-level MCP tools (not bridge actions).
|
||||
if real, ok := bridgeTopLevelOnly[actionName]; ok {
|
||||
return nil, fmt.Errorf("'%s' is a top-level MCP tool, not a call() action; invoke it as a tool directly (real name: %s)", actionName, real)
|
||||
}
|
||||
switch actionName {
|
||||
// --- Messaging ---
|
||||
case "read_inbox":
|
||||
@@ -173,10 +210,146 @@ func (b *ServiceBridge) Call(ctx context.Context, actionName string, args map[st
|
||||
return b.callQueryReputation(ctx, args)
|
||||
|
||||
default:
|
||||
if suggestion := suggestBridgeAction(actionName); suggestion != "" {
|
||||
return nil, fmt.Errorf("unknown action: %s (did you mean: %s)", actionName, suggestion)
|
||||
}
|
||||
return nil, fmt.Errorf("unknown action: %s", actionName)
|
||||
}
|
||||
}
|
||||
|
||||
// knownBridgeActions enumerates every action name routable through Call().
|
||||
// Kept in sync with the switch in Call() by hand — there are not many. Used
|
||||
// only to power "did you mean" suggestions when an unknown action arrives.
|
||||
var knownBridgeActions = []string{
|
||||
// Messaging
|
||||
"read_inbox", "claim_messages", "mark_done", "search_messages",
|
||||
"discover_agents", "send_message",
|
||||
// Channels
|
||||
"create_channel", "join_channel", "leave_channel", "list_channels",
|
||||
"invite_to_channel", "kick_from_channel", "get_channel_messages",
|
||||
"send_channel_message", "update_channel",
|
||||
// Swarm
|
||||
"post_task", "bid_task", "accept_bid", "complete_task", "list_tasks",
|
||||
// Attachments
|
||||
"upload_attachment", "download_attachment",
|
||||
// Reactions
|
||||
"react", "unreact", "get_reactions", "list_by_state",
|
||||
// Threads
|
||||
"get_replies",
|
||||
// Trust
|
||||
"get_trust",
|
||||
// SQL Query
|
||||
"query",
|
||||
// Wiki
|
||||
"create_article", "get_article", "update_article", "list_articles",
|
||||
"get_backlinks",
|
||||
// Marketplace
|
||||
"post_auction", "bid", "award", "mark_task_done", "read_skill_card",
|
||||
"query_reputation",
|
||||
}
|
||||
|
||||
// suggestBridgeAction returns the closest known action name to `name`, or ""
|
||||
// if nothing is plausibly close. Strategy: cheap prefix/substring check first,
|
||||
// then Levenshtein within distance 3 — but Levenshtein matches must share the
|
||||
// leading verb (the token before the first underscore) with the candidate.
|
||||
// Without that constraint, e.g. `read_message` is two edits from `send_message`
|
||||
// and gets suggested, which is the opposite intent and actively misleading.
|
||||
// Distance threshold scales with the length of the input so very short strings
|
||||
// don't false-match.
|
||||
func suggestBridgeAction(name string) string {
|
||||
if name == "" {
|
||||
return ""
|
||||
}
|
||||
lower := strings.ToLower(name)
|
||||
|
||||
// 1. Substring match — common when agents drop or add a prefix/suffix.
|
||||
for _, candidate := range knownBridgeActions {
|
||||
if strings.Contains(candidate, lower) || strings.Contains(lower, candidate) {
|
||||
return candidate
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Levenshtein on full names, gated on a matching leading verb so
|
||||
// `read_message` is not "fixed" by suggesting `send_message`.
|
||||
threshold := 3
|
||||
if len(name) <= 6 {
|
||||
threshold = 2
|
||||
}
|
||||
inputVerb := leadingVerb(lower)
|
||||
bestDist := threshold + 1
|
||||
best := ""
|
||||
for _, candidate := range knownBridgeActions {
|
||||
if leadingVerb(candidate) != inputVerb {
|
||||
continue
|
||||
}
|
||||
d := levenshtein(lower, candidate)
|
||||
if d < bestDist {
|
||||
bestDist = d
|
||||
best = candidate
|
||||
}
|
||||
}
|
||||
if bestDist <= threshold {
|
||||
return best
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// leadingVerb returns the substring before the first underscore, or the
|
||||
// whole string if there is no underscore. Used to gate Levenshtein
|
||||
// suggestions so a shared suffix (like `_message`) doesn't pair `read_…`
|
||||
// with `send_…`.
|
||||
func leadingVerb(s string) string {
|
||||
if i := strings.IndexByte(s, '_'); i >= 0 {
|
||||
return s[:i]
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// levenshtein computes the Levenshtein edit distance between a and b using
|
||||
// a single rolling row of O(min(len)) space. Pure Go, no deps.
|
||||
func levenshtein(a, b string) int {
|
||||
if a == b {
|
||||
return 0
|
||||
}
|
||||
if len(a) == 0 {
|
||||
return len(b)
|
||||
}
|
||||
if len(b) == 0 {
|
||||
return len(a)
|
||||
}
|
||||
// Ensure a is the shorter — minimises row width.
|
||||
if len(a) > len(b) {
|
||||
a, b = b, a
|
||||
}
|
||||
prev := make([]int, len(a)+1)
|
||||
curr := make([]int, len(a)+1)
|
||||
for i := 0; i <= len(a); i++ {
|
||||
prev[i] = i
|
||||
}
|
||||
for j := 1; j <= len(b); j++ {
|
||||
curr[0] = j
|
||||
for i := 1; i <= len(a); i++ {
|
||||
cost := 1
|
||||
if a[i-1] == b[j-1] {
|
||||
cost = 0
|
||||
}
|
||||
del := prev[i] + 1
|
||||
ins := curr[i-1] + 1
|
||||
sub := prev[i-1] + cost
|
||||
m := del
|
||||
if ins < m {
|
||||
m = ins
|
||||
}
|
||||
if sub < m {
|
||||
m = sub
|
||||
}
|
||||
curr[i] = m
|
||||
}
|
||||
prev, curr = curr, prev
|
||||
}
|
||||
return prev[len(a)]
|
||||
}
|
||||
|
||||
// --- Messaging implementations ---
|
||||
|
||||
func (b *ServiceBridge) callSendMessage(ctx context.Context, args map[string]any) (any, error) {
|
||||
|
||||
@@ -3,6 +3,7 @@ package mcp
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
_ "modernc.org/sqlite"
|
||||
@@ -486,3 +487,213 @@ func TestBridge_React_Toggle_Removes_WorkflowState(t *testing.T) {
|
||||
}
|
||||
|
||||
var _ = storage.RunMigrations
|
||||
|
||||
// TestBridge_ActionAliases verifies that each observed wrong action name from
|
||||
// production logs resolves to the correct real action. We don't validate the
|
||||
// full result payload — only that the call succeeds (i.e. dispatch reached the
|
||||
// real handler) rather than returning "unknown action".
|
||||
func TestBridge_ActionAliases(t *testing.T) {
|
||||
tests := []struct {
|
||||
alias string
|
||||
realName string
|
||||
args map[string]any
|
||||
setupCh string // optional: channel name to create+join before the call
|
||||
expectErr string // optional: substring of expected error (when call reaches real handler but fails for unrelated reasons)
|
||||
}{
|
||||
{
|
||||
alias: "search",
|
||||
realName: "search_messages",
|
||||
args: map[string]any{"query": "anything"},
|
||||
},
|
||||
{
|
||||
alias: "my_status",
|
||||
realName: "read_inbox",
|
||||
args: map[string]any{"limit": 5},
|
||||
},
|
||||
{
|
||||
alias: "read_dm",
|
||||
realName: "read_inbox",
|
||||
args: map[string]any{"limit": 5},
|
||||
},
|
||||
{
|
||||
alias: "read_channel",
|
||||
realName: "get_channel_messages",
|
||||
args: map[string]any{"channel_name": "alias-ch"},
|
||||
setupCh: "alias-ch",
|
||||
},
|
||||
{
|
||||
alias: "read_article",
|
||||
realName: "get_article",
|
||||
args: map[string]any{"slug": "anything"},
|
||||
expectErr: "wiki not available", // bridge has no wikiService → dispatch reached real handler
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.alias, func(t *testing.T) {
|
||||
bridge, _, _, channelService := newTestBridge(t)
|
||||
ctx := context.Background()
|
||||
|
||||
if tt.setupCh != "" {
|
||||
ch, err := channelService.CreateChannel(ctx, channels.CreateChannelRequest{
|
||||
Name: tt.setupCh, Type: "standard", CreatedBy: "agent-a",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("create channel: %v", err)
|
||||
}
|
||||
if err := channelService.JoinChannel(ctx, ch.ID, "agent-a"); err != nil {
|
||||
t.Fatalf("join channel: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
_, err := bridge.Call(ctx, tt.alias, tt.args)
|
||||
if tt.expectErr != "" {
|
||||
if err == nil {
|
||||
t.Fatalf("expected error containing %q, got nil", tt.expectErr)
|
||||
}
|
||||
if !strings.Contains(err.Error(), tt.expectErr) {
|
||||
t.Fatalf("error %q does not contain %q", err.Error(), tt.expectErr)
|
||||
}
|
||||
if strings.Contains(err.Error(), "unknown action") {
|
||||
t.Fatalf("alias %q should have been resolved, got unknown-action error: %v", tt.alias, err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("alias %q (real=%s) failed: %v", tt.alias, tt.realName, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestBridge_TopLevelToolHint verifies that wrong-guess names which map to
|
||||
// top-level MCP tools (not bridge actions) produce a targeted hint pointing
|
||||
// at the real tool, instead of a generic "unknown action" error.
|
||||
func TestBridge_TopLevelToolHint(t *testing.T) {
|
||||
bridge, _, _, _ := newTestBridge(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// The misspelled wrong guess hits the hint.
|
||||
_, err := bridge.Call(ctx, "rewrite_core_memory", map[string]any{})
|
||||
if err == nil {
|
||||
t.Fatal("expected error for top-level-tool wrong-guess")
|
||||
}
|
||||
msg := err.Error()
|
||||
if !strings.Contains(msg, "top-level MCP tool") {
|
||||
t.Errorf("error should mention top-level MCP tool, got: %v", err)
|
||||
}
|
||||
if !strings.Contains(msg, "memory_rewrite_core") {
|
||||
t.Errorf("error should name the real tool memory_rewrite_core, got: %v", err)
|
||||
}
|
||||
|
||||
// The correct top-level tool name, when called via the bridge, also
|
||||
// returns the hint instead of plain "unknown action". (Agents learn the
|
||||
// real tool name from the first hint and then retry via call() — the
|
||||
// hint must catch both spellings.)
|
||||
_, err = bridge.Call(ctx, "memory_rewrite_core", map[string]any{})
|
||||
if err == nil {
|
||||
t.Fatal("expected error for memory_rewrite_core via bridge")
|
||||
}
|
||||
msg = err.Error()
|
||||
if !strings.Contains(msg, "top-level MCP tool") {
|
||||
t.Errorf("memory_rewrite_core via bridge should hint top-level MCP tool, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBridge_UnknownAction_Suggestion verifies that an unknown action that is
|
||||
// close to a real one (Levenshtein-wise) returns a "did you mean" suggestion.
|
||||
func TestBridge_UnknownAction_Suggestion(t *testing.T) {
|
||||
bridge, _, _, _ := newTestBridge(t)
|
||||
ctx := context.Background()
|
||||
|
||||
tests := []struct {
|
||||
input string
|
||||
wantContains string
|
||||
}{
|
||||
// substring hit: "send" is contained in "send_message"
|
||||
{input: "send", wantContains: "send_message"},
|
||||
// Levenshtein within a shared verb: "list_channel" → "list_channels" (distance 1)
|
||||
{input: "list_channel", wantContains: "list_channels"},
|
||||
// Levenshtein within a shared verb: "get_channel_message" → "get_channel_messages"
|
||||
{input: "get_channel_message", wantContains: "get_channel_messages"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.input, func(t *testing.T) {
|
||||
_, err := bridge.Call(ctx, tt.input, map[string]any{})
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for %q", tt.input)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "did you mean") {
|
||||
t.Errorf("error should contain 'did you mean', got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), tt.wantContains) {
|
||||
t.Errorf("error should suggest %q, got: %v", tt.wantContains, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestBridge_UnknownAction_NoSuggestion verifies that a truly distant unknown
|
||||
// action returns the plain "unknown action" error without a misleading
|
||||
// suggestion.
|
||||
func TestBridge_UnknownAction_NoSuggestion(t *testing.T) {
|
||||
bridge, _, _, _ := newTestBridge(t)
|
||||
ctx := context.Background()
|
||||
|
||||
_, err := bridge.Call(ctx, "xyzzy_quux_frobnicate", map[string]any{})
|
||||
if err == nil {
|
||||
t.Fatal("expected error")
|
||||
}
|
||||
if strings.Contains(err.Error(), "did you mean") {
|
||||
t.Errorf("distant action should not get a suggestion, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBridge_UnknownAction_NoCrossVerbSuggestion guards against the suggester
|
||||
// pairing actions that share a suffix but have opposite intent — e.g.
|
||||
// `read_message` is two edits from `send_message`, but suggesting "send" to an
|
||||
// agent that asked to read is actively misleading. The leading-verb gate in
|
||||
// suggestBridgeAction must prevent this.
|
||||
func TestBridge_UnknownAction_NoCrossVerbSuggestion(t *testing.T) {
|
||||
bridge, _, _, _ := newTestBridge(t)
|
||||
ctx := context.Background()
|
||||
|
||||
cases := []string{
|
||||
"read_message", // would have suggested send_message (distance 2)
|
||||
"delete_message", // would have suggested send_message (distance 3)
|
||||
"fetch_channel", // unrelated verb; must not suggest send/list/get
|
||||
}
|
||||
for _, in := range cases {
|
||||
t.Run(in, func(t *testing.T) {
|
||||
_, err := bridge.Call(ctx, in, map[string]any{})
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for %q", in)
|
||||
}
|
||||
if strings.Contains(err.Error(), "did you mean") {
|
||||
t.Errorf("cross-verb suggestion leaked for %q: %v", in, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLevenshtein(t *testing.T) {
|
||||
tests := []struct {
|
||||
a, b string
|
||||
want int
|
||||
}{
|
||||
{"", "", 0},
|
||||
{"abc", "abc", 0},
|
||||
{"", "abc", 3},
|
||||
{"abc", "", 3},
|
||||
{"kitten", "sitting", 3},
|
||||
{"send", "sned", 2},
|
||||
{"list_channel", "list_channels", 1},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
got := levenshtein(tt.a, tt.b)
|
||||
if got != tt.want {
|
||||
t.Errorf("levenshtein(%q, %q) = %d, want %d", tt.a, tt.b, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -52,18 +52,21 @@ func NewGoalsToolRegistrar(
|
||||
}
|
||||
}
|
||||
|
||||
// RegisterAllOnServer attaches create_goal, propose_task_tree,
|
||||
// propose_agent, claim_task, request_resource, list_resources, and
|
||||
// complete_goal to the MCP server.
|
||||
// RegisterAllOnServer attaches create_goal, propose_task_tree, claim_task,
|
||||
// request_resource, list_resources, and complete_goal to the MCP server.
|
||||
//
|
||||
// propose_agent was removed when SynapBus moved to internal-only mode: it
|
||||
// wrote a pending row to agent_proposals for human approval via #approvals,
|
||||
// and that approval surface no longer exists. Agents are created via the
|
||||
// admin CLI directly. See migration 027_remove_approval_noise.sql.
|
||||
func (r *GoalsToolRegistrar) RegisterAllOnServer(s *server.MCPServer) {
|
||||
s.AddTool(r.createGoalTool(), r.handleCreateGoal)
|
||||
s.AddTool(r.proposeTaskTreeTool(), r.handleProposeTaskTree)
|
||||
s.AddTool(r.proposeAgentTool(), r.handleProposeAgent)
|
||||
s.AddTool(r.claimTaskTool(), r.handleClaimTask)
|
||||
s.AddTool(r.requestResourceTool(), r.handleRequestResource)
|
||||
s.AddTool(r.listResourcesTool(), r.handleListResources)
|
||||
s.AddTool(r.completeGoalTool(), r.handleCompleteGoal)
|
||||
r.logger.Info("spec-018 MCP tools registered", "count", 7)
|
||||
r.logger.Info("spec-018 MCP tools registered", "count", 6)
|
||||
}
|
||||
|
||||
// --- Tool Definitions ---
|
||||
@@ -87,18 +90,6 @@ func (r *GoalsToolRegistrar) proposeTaskTreeTool() mcplib.Tool {
|
||||
)
|
||||
}
|
||||
|
||||
func (r *GoalsToolRegistrar) proposeAgentTool() mcplib.Tool {
|
||||
return mcplib.NewTool("propose_agent",
|
||||
mcplib.WithDescription("Propose creating a new specialist agent. Writes an agent_proposals row so a human can approve it via the #approvals channel. Returns the proposal id."),
|
||||
mcplib.WithString("name", mcplib.Description("Desired agent name"), mcplib.Required()),
|
||||
mcplib.WithString("display_name", mcplib.Description("Human-readable display name")),
|
||||
mcplib.WithString("system_prompt", mcplib.Description("System prompt for the spawned agent"), mcplib.Required()),
|
||||
mcplib.WithString("tool_scope", mcplib.Description("Comma-separated scope (e.g. 'messages:read,messages:send')")),
|
||||
mcplib.WithNumber("parent_task_id", mcplib.Description("The task this agent will work on")),
|
||||
mcplib.WithString("autonomy_tier", mcplib.Description("supervised | assisted | autonomous (default assisted)")),
|
||||
)
|
||||
}
|
||||
|
||||
func (r *GoalsToolRegistrar) claimTaskTool() mcplib.Tool {
|
||||
return mcplib.NewTool("claim_task",
|
||||
mcplib.WithDescription("Atomically claim an approved task. Returns the claimed task or an error if another agent got it first."),
|
||||
@@ -239,71 +230,6 @@ func (r *GoalsToolRegistrar) handleProposeTaskTree(ctx context.Context, req mcpl
|
||||
})
|
||||
}
|
||||
|
||||
func (r *GoalsToolRegistrar) handleProposeAgent(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
agentName, ok := extractAgentName(ctx)
|
||||
if !ok {
|
||||
return mcplib.NewToolResultError("authentication required"), nil
|
||||
}
|
||||
if r.db == nil {
|
||||
return mcplib.NewToolResultError("db not configured"), nil
|
||||
}
|
||||
name := req.GetString("name", "")
|
||||
if name == "" {
|
||||
return mcplib.NewToolResultError("name is required"), nil
|
||||
}
|
||||
systemPrompt := req.GetString("system_prompt", "")
|
||||
if systemPrompt == "" {
|
||||
return mcplib.NewToolResultError("system_prompt is required"), nil
|
||||
}
|
||||
toolScope := req.GetString("tool_scope", "")
|
||||
if toolScope == "" {
|
||||
toolScope = "[]"
|
||||
} else if !strings.HasPrefix(toolScope, "[") {
|
||||
// Accept comma-separated convenience form.
|
||||
parts := strings.Split(toolScope, ",")
|
||||
for i := range parts {
|
||||
parts[i] = `"` + strings.TrimSpace(parts[i]) + `"`
|
||||
}
|
||||
toolScope = "[" + strings.Join(parts, ",") + "]"
|
||||
}
|
||||
tier := req.GetString("autonomy_tier", "assisted")
|
||||
parentTaskID := int64(req.GetInt("parent_task_id", 0))
|
||||
if parentTaskID <= 0 {
|
||||
return mcplib.NewToolResultError("parent_task_id is required (proposals must attach to a task)"), nil
|
||||
}
|
||||
|
||||
agent, err := r.agents.GetAgent(ctx, agentName)
|
||||
if err != nil {
|
||||
return mcplib.NewToolResultError(fmt.Sprintf("resolve caller: %s", err)), nil
|
||||
}
|
||||
|
||||
// Resolve goal_id from the parent task.
|
||||
var goalID int64
|
||||
if err := r.db.QueryRowContext(ctx,
|
||||
`SELECT goal_id FROM goal_tasks WHERE id=?`, parentTaskID).Scan(&goalID); err != nil {
|
||||
return mcplib.NewToolResultError(fmt.Sprintf("resolve goal from task %d: %s", parentTaskID, err)), nil
|
||||
}
|
||||
|
||||
res, err := r.db.ExecContext(ctx, `
|
||||
INSERT INTO agent_proposals (
|
||||
proposer_agent_id, goal_id, parent_task_id, proposed_name,
|
||||
proposed_model, proposed_system_prompt, proposed_tool_scope_json,
|
||||
proposed_autonomy_tier, status
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'pending')`,
|
||||
agent.ID, goalID, parentTaskID, name,
|
||||
"gemini-2.5-flash", systemPrompt, toolScope, tier)
|
||||
if err != nil {
|
||||
return mcplib.NewToolResultError(fmt.Sprintf("insert proposal: %s", err)), nil
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
|
||||
return resultJSON(map[string]any{
|
||||
"proposal_id": id,
|
||||
"status": "pending",
|
||||
"name": name,
|
||||
})
|
||||
}
|
||||
|
||||
func (r *GoalsToolRegistrar) handleClaimTask(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
agentName, ok := extractAgentName(ctx)
|
||||
if !ok {
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
// Integration test for US2 → US1 wiring: a real CoreMemoryStore, wired
|
||||
// through messaging.NewCoreProvider, surfaces a seeded blob in the
|
||||
// wrapped tool response's `relevant_context.core_memory` field; an agent
|
||||
// with no row gets no relevant_context.
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
mcplib "github.com/mark3labs/mcp-go/mcp"
|
||||
_ "modernc.org/sqlite"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
"github.com/synapbus/synapbus/internal/storage"
|
||||
)
|
||||
|
||||
func newInjectionTestDB(t *testing.T) *sql.DB {
|
||||
t.Helper()
|
||||
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", t.Name())
|
||||
db, err := sql.Open("sqlite", dsn)
|
||||
if err != nil {
|
||||
t.Fatalf("open db: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { db.Close() })
|
||||
if _, err := db.Exec("PRAGMA foreign_keys=ON"); err != nil {
|
||||
t.Fatalf("foreign keys: %v", err)
|
||||
}
|
||||
if err := storage.RunMigrations(context.Background(), db); err != nil {
|
||||
t.Fatalf("migrations: %v", err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
// TestInjection_CoreMemoryWiring proves a wrapped session-start handler
|
||||
// surfaces a blob seeded via CoreMemoryStore as
|
||||
// `relevant_context.core_memory`. Mirrors the contract example in
|
||||
// `specs/020-proactive-memory-dream-worker/contracts/mcp-injection.md`.
|
||||
func TestInjection_CoreMemoryWiring(t *testing.T) {
|
||||
db := newInjectionTestDB(t)
|
||||
ctx := context.Background()
|
||||
|
||||
const owner = "42"
|
||||
const agent = "a1"
|
||||
const blob = "I am a1. Currently focused on memory tests."
|
||||
|
||||
coreStore := messaging.NewCoreMemoryStore(db, 2048)
|
||||
if err := coreStore.Set(ctx, owner, agent, blob, "human"); err != nil {
|
||||
t.Fatalf("seed core memory: %v", err)
|
||||
}
|
||||
|
||||
cfg := WrapConfig{
|
||||
Cfg: messaging.MemoryConfig{
|
||||
InjectionEnabled: true,
|
||||
InjectionBudgetTokens: 500,
|
||||
InjectionMaxItems: 5,
|
||||
InjectionMinScore: 0.25,
|
||||
},
|
||||
SearchSvc: nil, // query="" forces no retrieval — only core matters.
|
||||
IncludeCore: true,
|
||||
CoreProvider: messaging.NewCoreProvider(coreStore),
|
||||
QuerySource: func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string { return "" },
|
||||
}
|
||||
inner := stubHandler(map[string]any{"agent": agent})
|
||||
wrapped := WrapInjection(inner, "my_status", cfg)
|
||||
|
||||
// Owner 42 ↔ caller a1.
|
||||
callerCtx := agents.ContextWithAgent(ctx, &agents.Agent{Name: agent, OwnerID: 42})
|
||||
res, err := wrapped(callerCtx, mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("wrapped my_status: %v", err)
|
||||
}
|
||||
body := extractJSON(t, res)
|
||||
rc, ok := body["relevant_context"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("relevant_context missing: %+v", body)
|
||||
}
|
||||
if got := rc["core_memory"]; got != blob {
|
||||
t.Errorf("core_memory: got %v want %q", got, blob)
|
||||
}
|
||||
}
|
||||
|
||||
// TestInjection_NoCoreMemoryYieldsNoPacket verifies that an agent
|
||||
// without a memory_core row gets the original handler response back,
|
||||
// without a `relevant_context` field appended.
|
||||
func TestInjection_NoCoreMemoryYieldsNoPacket(t *testing.T) {
|
||||
db := newInjectionTestDB(t)
|
||||
|
||||
coreStore := messaging.NewCoreMemoryStore(db, 2048)
|
||||
// Intentionally NO Set — the agent has no row.
|
||||
|
||||
cfg := WrapConfig{
|
||||
Cfg: messaging.MemoryConfig{
|
||||
InjectionEnabled: true,
|
||||
InjectionBudgetTokens: 500,
|
||||
InjectionMaxItems: 5,
|
||||
InjectionMinScore: 0.25,
|
||||
},
|
||||
SearchSvc: nil,
|
||||
IncludeCore: true,
|
||||
CoreProvider: messaging.NewCoreProvider(coreStore),
|
||||
QuerySource: func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string { return "" },
|
||||
}
|
||||
inner := stubHandler(map[string]any{"agent": "a2"})
|
||||
wrapped := WrapInjection(inner, "my_status", cfg)
|
||||
|
||||
ctx := agents.ContextWithAgent(context.Background(), &agents.Agent{Name: "a2", OwnerID: 42})
|
||||
res, err := wrapped(ctx, mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("wrapped: %v", err)
|
||||
}
|
||||
body := extractJSON(t, res)
|
||||
if _, has := body["relevant_context"]; has {
|
||||
t.Errorf("relevant_context attached for agent with no core row: %v", body["relevant_context"])
|
||||
}
|
||||
if body["agent"] != "a2" {
|
||||
t.Errorf("inner body lost: %+v", body)
|
||||
}
|
||||
}
|
||||
|
||||
// Verify ContextPacket round-trips its core_memory through json. This is
|
||||
// the contract field consumed by clients.
|
||||
func TestInjection_CoreMemoryJSONShape(t *testing.T) {
|
||||
db := newInjectionTestDB(t)
|
||||
ctx := context.Background()
|
||||
coreStore := messaging.NewCoreMemoryStore(db, 2048)
|
||||
if err := coreStore.Set(ctx, "1", "a1", "hello", "human"); err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
|
||||
provider := messaging.NewCoreProvider(coreStore)
|
||||
got, err := provider.Get(ctx, "1", "a1")
|
||||
if err != nil {
|
||||
t.Fatalf("provider.Get: %v", err)
|
||||
}
|
||||
if got != "hello" {
|
||||
t.Errorf("provider.Get: got %q want hello", got)
|
||||
}
|
||||
|
||||
// Empty case (no row) yields "" without error.
|
||||
got, err = provider.Get(ctx, "1", "nobody")
|
||||
if err != nil || got != "" {
|
||||
t.Errorf("provider.Get on missing: got %q err=%v", got, err)
|
||||
}
|
||||
|
||||
// Sanity: ensure the adapter is reachable through json marshaling of a packet.
|
||||
type fakePacket struct {
|
||||
Core string `json:"core_memory,omitempty"`
|
||||
}
|
||||
b, _ := json.Marshal(fakePacket{Core: "hello"})
|
||||
if string(b) != `{"core_memory":"hello"}` {
|
||||
t.Errorf("json marshaling: got %s", b)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,197 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
mcplib "github.com/mark3labs/mcp-go/mcp"
|
||||
_ "modernc.org/sqlite"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
"github.com/synapbus/synapbus/internal/search"
|
||||
"github.com/synapbus/synapbus/internal/trace"
|
||||
)
|
||||
|
||||
// TestInjection_CrossOwner_NoLeak is the SC-008 adversarial test: two
|
||||
// owners (H1, H2) each have an agent + memories in #open-brain. When
|
||||
// each agent invokes the same wrapped tool with the same query, their
|
||||
// `relevant_context.memories` are disjoint along owner boundaries.
|
||||
func TestInjection_CrossOwner_NoLeak(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Two human owners.
|
||||
if _, err := db.Exec(
|
||||
`INSERT OR IGNORE INTO users (id, username, password_hash, display_name)
|
||||
VALUES (1, 'h1', 'hash', 'H1'), (2, 'h2', 'hash', 'H2')`,
|
||||
); err != nil {
|
||||
t.Fatalf("seed users: %v", err)
|
||||
}
|
||||
// One agent per owner.
|
||||
if _, err := db.Exec(
|
||||
`INSERT OR IGNORE INTO agents (name, display_name, type, owner_id, api_key_hash, status)
|
||||
VALUES ('a-h1', 'A1', 'ai', 1, 'k1', 'active'),
|
||||
('a-h2', 'A2', 'ai', 2, 'k2', 'active')`,
|
||||
); err != nil {
|
||||
t.Fatalf("seed agents: %v", err)
|
||||
}
|
||||
// Both join an open-brain channel — broadly readable.
|
||||
if _, err := db.Exec(
|
||||
`INSERT OR IGNORE INTO channels (id, name, description, type, created_by)
|
||||
VALUES (1, 'open-brain', 'shared', 'standard', 'system')`,
|
||||
); err != nil {
|
||||
t.Fatalf("seed channel: %v", err)
|
||||
}
|
||||
if _, err := db.Exec(
|
||||
`INSERT OR IGNORE INTO channel_members (channel_id, agent_name)
|
||||
VALUES (1, 'a-h1'), (1, 'a-h2')`,
|
||||
); err != nil {
|
||||
t.Fatalf("seed members: %v", err)
|
||||
}
|
||||
// One memory per owner, both about the same topic.
|
||||
seedMemory(t, db, 1, "a-h1", "Kuzu graph DB is in H1's research notes")
|
||||
seedMemory(t, db, 1, "a-h2", "Kuzu graph DB also appears in H2's separate research")
|
||||
|
||||
// Stand up a real search.Service (FTS-only, no embeddings).
|
||||
tracer := trace.NewTracer(db)
|
||||
t.Cleanup(func() { tracer.Close() })
|
||||
msgStore := messaging.NewSQLiteMessageStore(db)
|
||||
msgService := messaging.NewMessagingService(msgStore, tracer)
|
||||
searchSvc := search.NewService(db, nil, nil, msgService)
|
||||
|
||||
// Configure wrap: low score floor so the test isn't flaky on FTS.
|
||||
cfg := WrapConfig{
|
||||
Cfg: messaging.MemoryConfig{
|
||||
InjectionEnabled: true,
|
||||
InjectionBudgetTokens: 500,
|
||||
InjectionMaxItems: 5,
|
||||
InjectionMinScore: 0.0,
|
||||
},
|
||||
SearchSvc: searchSvc,
|
||||
QuerySource: func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string {
|
||||
return "Kuzu"
|
||||
},
|
||||
}
|
||||
wrapped := WrapInjection(stubHandler(map[string]any{"ok": true}), "search_messages", cfg)
|
||||
|
||||
// Call as H1.
|
||||
h1Ctx := agents.ContextWithAgent(ctx, &agents.Agent{Name: "a-h1", OwnerID: 1})
|
||||
h1Res, err := wrapped(h1Ctx, mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("h1 wrapped: %v", err)
|
||||
}
|
||||
h1Body := unmarshalText(t, h1Res)
|
||||
|
||||
// Call as H2.
|
||||
h2Ctx := agents.ContextWithAgent(ctx, &agents.Agent{Name: "a-h2", OwnerID: 2})
|
||||
h2Res, err := wrapped(h2Ctx, mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("h2 wrapped: %v", err)
|
||||
}
|
||||
h2Body := unmarshalText(t, h2Res)
|
||||
|
||||
h1Memories := extractMemoryFromAgents(h1Body)
|
||||
h2Memories := extractMemoryFromAgents(h2Body)
|
||||
|
||||
for _, fromAgent := range h1Memories {
|
||||
if fromAgent != "a-h1" {
|
||||
t.Errorf("H1 saw memory from %q (cross-owner leak)", fromAgent)
|
||||
}
|
||||
}
|
||||
for _, fromAgent := range h2Memories {
|
||||
if fromAgent != "a-h2" {
|
||||
t.Errorf("H2 saw memory from %q (cross-owner leak)", fromAgent)
|
||||
}
|
||||
}
|
||||
|
||||
// Disjoint sets: no h1 memory id may appear in h2's response.
|
||||
h1IDs := extractMemoryIDs(h1Body)
|
||||
h2IDs := extractMemoryIDs(h2Body)
|
||||
for id := range h1IDs {
|
||||
if _, dup := h2IDs[id]; dup {
|
||||
t.Errorf("memory id %d leaked across owners", id)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func seedMemory(t *testing.T, db *sql.DB, channelID int64, fromAgent, body string) {
|
||||
t.Helper()
|
||||
convRes, err := db.Exec(
|
||||
`INSERT INTO conversations (created_by, channel_id) VALUES (?, ?)`,
|
||||
fromAgent, channelID,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("seed conversation: %v", err)
|
||||
}
|
||||
convID, _ := convRes.LastInsertId()
|
||||
if _, err := db.Exec(
|
||||
`INSERT INTO messages (conversation_id, from_agent, channel_id, body, priority, status, metadata)
|
||||
VALUES (?, ?, ?, ?, 5, 'pending', '{}')`,
|
||||
convID, fromAgent, channelID, body,
|
||||
); err != nil {
|
||||
t.Fatalf("seed message: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func unmarshalText(t *testing.T, res *mcplib.CallToolResult) map[string]any {
|
||||
t.Helper()
|
||||
if res == nil || len(res.Content) != 1 {
|
||||
t.Fatalf("unexpected result: %+v", res)
|
||||
}
|
||||
tc, ok := res.Content[0].(mcplib.TextContent)
|
||||
if !ok {
|
||||
t.Fatalf("not text content: %T", res.Content[0])
|
||||
}
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal([]byte(tc.Text), &m); err != nil {
|
||||
t.Fatalf("unmarshal: %v: %s", err, tc.Text)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func extractMemoryFromAgents(body map[string]any) []string {
|
||||
rc, ok := body["relevant_context"].(map[string]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
mems, ok := rc["memories"].([]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(mems))
|
||||
for _, raw := range mems {
|
||||
m, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if fa, ok := m["from_agent"].(string); ok {
|
||||
out = append(out, fa)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func extractMemoryIDs(body map[string]any) map[int64]struct{} {
|
||||
rc, ok := body["relevant_context"].(map[string]any)
|
||||
if !ok {
|
||||
return map[int64]struct{}{}
|
||||
}
|
||||
mems, ok := rc["memories"].([]any)
|
||||
if !ok {
|
||||
return map[int64]struct{}{}
|
||||
}
|
||||
out := map[int64]struct{}{}
|
||||
for _, raw := range mems {
|
||||
m, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if id, ok := m["id"].(float64); ok {
|
||||
out[int64(id)] = struct{}{}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,268 @@
|
||||
// Proactive-memory injection middleware — wraps MCP tool handlers so
|
||||
// that successful JSON responses gain a `relevant_context` field per
|
||||
// `specs/020-proactive-memory-dream-worker/contracts/mcp-injection.md`.
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"time"
|
||||
|
||||
mcplib "github.com/mark3labs/mcp-go/mcp"
|
||||
"github.com/mark3labs/mcp-go/server"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
"github.com/synapbus/synapbus/internal/metrics"
|
||||
"github.com/synapbus/synapbus/internal/search"
|
||||
)
|
||||
|
||||
// ToolHandler is the mcp-go tool handler signature. Re-exported as an
|
||||
// alias so the wrapper signature reads cleanly at registration sites.
|
||||
type ToolHandler = server.ToolHandlerFunc
|
||||
|
||||
// QuerySource derives the retrieval query for one tool invocation. It
|
||||
// is given the inner handler's parsed args (best-effort: nil when args
|
||||
// don't fit map[string]any) and the inner handler's parsed JSON result
|
||||
// (nil on error or non-JSON). It must be cheap; called on every wrapped
|
||||
// tool call.
|
||||
type QuerySource func(ctx context.Context, toolName string, args map[string]any, result map[string]any) string
|
||||
|
||||
// WrapConfig parameterizes WrapInjection.
|
||||
type WrapConfig struct {
|
||||
// Cfg is the messaging.MemoryConfig snapshot taken at server
|
||||
// startup. When Cfg.InjectionEnabled is false, WrapInjection
|
||||
// returns the handler unchanged.
|
||||
Cfg messaging.MemoryConfig
|
||||
|
||||
// SearchSvc drives retrieval. Required when Cfg.InjectionEnabled.
|
||||
SearchSvc *search.Service
|
||||
|
||||
// Injections is the 24h audit ring. May be nil — Record errors are
|
||||
// logged and the wrapper continues.
|
||||
Injections *messaging.MemoryInjections
|
||||
|
||||
// QuerySource derives the retrieval query for this tool. Required.
|
||||
QuerySource QuerySource
|
||||
|
||||
// IncludeCore is true for session-start tools (e.g. my_status).
|
||||
// Only those get the per-(owner, agent) core-memory blob injected.
|
||||
IncludeCore bool
|
||||
|
||||
// CoreProvider is consulted when IncludeCore=true. May be nil
|
||||
// (US2 not yet wired) — wrapper still functions, just skips core.
|
||||
CoreProvider search.CoreMemoryProvider
|
||||
|
||||
// Logger is used for non-fatal failures. Defaults to slog.Default.
|
||||
Logger *slog.Logger
|
||||
}
|
||||
|
||||
// WrapInjection returns a ToolHandler that wraps `inner` with the
|
||||
// proactive-memory injection middleware described in
|
||||
// `contracts/mcp-injection.md`.
|
||||
//
|
||||
// When Cfg.InjectionEnabled is false, the original handler is returned
|
||||
// unchanged so the response payload exactly matches the pre-feature
|
||||
// shape (FR-012, SC-009).
|
||||
//
|
||||
// Otherwise, the wrapper:
|
||||
// 1. Runs the inner handler.
|
||||
// 2. If the result is an error or not a single TextContent of JSON
|
||||
// object shape, returns the result unchanged.
|
||||
// 3. Builds a ContextPacket via search.BuildContextPacket using the
|
||||
// query derived from cfg.QuerySource.
|
||||
// 4. If the packet is non-empty (>=1 memory or core memory set), merges
|
||||
// `relevant_context: <packet>` into the JSON body and re-marshals.
|
||||
// 5. Records the injection to the 24h audit ring asynchronously.
|
||||
func WrapInjection(inner ToolHandler, toolName string, cfg WrapConfig) ToolHandler {
|
||||
if !cfg.Cfg.InjectionEnabled {
|
||||
return inner
|
||||
}
|
||||
logger := cfg.Logger
|
||||
if logger == nil {
|
||||
logger = slog.Default().With("component", "mcp-injection")
|
||||
}
|
||||
return func(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
res, err := inner(ctx, req)
|
||||
if err != nil {
|
||||
return res, err
|
||||
}
|
||||
if res == nil || res.IsError {
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// Locate the JSON text content. Non-JSON or multi-content
|
||||
// payloads pass through unchanged.
|
||||
idx, text, ok := singleJSONText(res)
|
||||
if !ok {
|
||||
return res, nil
|
||||
}
|
||||
var body map[string]any
|
||||
if err := json.Unmarshal([]byte(text), &body); err != nil {
|
||||
return res, nil
|
||||
}
|
||||
|
||||
agent, ok := callerAgent(ctx)
|
||||
if !ok || agent == nil {
|
||||
// No identity → no owner scope → no injection.
|
||||
metrics.InjectionSkippedTotal.WithLabelValues(toolName, "no_owner").Inc()
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// Derive the retrieval query. nil args is fine; nil result is
|
||||
// fine — the source decides what to do.
|
||||
argsMap, _ := req.Params.Arguments.(map[string]any)
|
||||
query := ""
|
||||
if cfg.QuerySource != nil {
|
||||
query = cfg.QuerySource(ctx, toolName, argsMap, body)
|
||||
}
|
||||
|
||||
windowDays := int(cfg.Cfg.DreamRecentWindow / (24 * time.Hour))
|
||||
if windowDays < 1 {
|
||||
windowDays = 14
|
||||
}
|
||||
opts := search.InjectionOpts{
|
||||
BudgetTokens: cfg.Cfg.InjectionBudgetTokens,
|
||||
MaxItems: cfg.Cfg.InjectionMaxItems,
|
||||
MinScore: cfg.Cfg.InjectionMinScore,
|
||||
IncludeCore: cfg.IncludeCore,
|
||||
CoreProvider: cfg.CoreProvider,
|
||||
RecentWindowDays: windowDays,
|
||||
}
|
||||
pkt, err := search.BuildContextPacket(ctx, cfg.SearchSvc, agent, query, opts)
|
||||
if err != nil {
|
||||
logger.Debug("build context packet failed", "tool", toolName, "error", err)
|
||||
return res, nil
|
||||
}
|
||||
if pkt == nil {
|
||||
// Empty packet → omit `relevant_context` entirely.
|
||||
metrics.InjectionSkippedTotal.WithLabelValues(toolName, "empty_pool").Inc()
|
||||
return res, nil
|
||||
}
|
||||
if len(pkt.Memories) == 0 && pkt.CoreMemory == "" {
|
||||
metrics.InjectionSkippedTotal.WithLabelValues(toolName, "empty_pool").Inc()
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// Packet metrics — count, size, item-count. Done before re-marshal
|
||||
// so a marshal failure still gets a packet-created observation
|
||||
// (which is what we'd be alerting on anyway).
|
||||
metrics.InjectionPacketsTotal.WithLabelValues(toolName).Inc()
|
||||
metrics.InjectionMemoriesPerPacket.WithLabelValues(toolName).Observe(float64(len(pkt.Memories)))
|
||||
metrics.InjectionPacketChars.WithLabelValues(toolName).Observe(float64(pkt.PacketChars))
|
||||
|
||||
body["relevant_context"] = pkt
|
||||
|
||||
merged, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
logger.Debug("re-marshal failed", "tool", toolName, "error", err)
|
||||
return res, nil
|
||||
}
|
||||
res.Content[idx] = mcplib.TextContent{Type: "text", Text: string(merged)}
|
||||
|
||||
// Audit-ring write is best-effort; never blocks the response.
|
||||
recordInjection(cfg.Injections, logger, agent, toolName, pkt)
|
||||
|
||||
return res, nil
|
||||
}
|
||||
}
|
||||
|
||||
// singleJSONText reports the index of the single TextContent in `res`
|
||||
// when its Text is a JSON object. Anything else (multiple contents,
|
||||
// non-text, non-object JSON) returns ok=false → pass through.
|
||||
func singleJSONText(res *mcplib.CallToolResult) (int, string, bool) {
|
||||
if res == nil || len(res.Content) != 1 {
|
||||
return 0, "", false
|
||||
}
|
||||
tc, ok := res.Content[0].(mcplib.TextContent)
|
||||
if !ok {
|
||||
return 0, "", false
|
||||
}
|
||||
// Quick sanity check that the text starts with `{` — avoids
|
||||
// allocating a map for a known-non-object payload.
|
||||
for i := 0; i < len(tc.Text); i++ {
|
||||
switch tc.Text[i] {
|
||||
case ' ', '\t', '\n', '\r':
|
||||
continue
|
||||
case '{':
|
||||
return 0, tc.Text, true
|
||||
default:
|
||||
return 0, "", false
|
||||
}
|
||||
}
|
||||
return 0, "", false
|
||||
}
|
||||
|
||||
// callerAgent unpacks *agents.Agent from the request context. Uses the
|
||||
// agents middleware ContextWithAgent, populated by the auth path.
|
||||
func callerAgent(ctx context.Context) (*agents.Agent, bool) {
|
||||
return agents.AgentFromContext(ctx)
|
||||
}
|
||||
|
||||
// recordInjection writes one audit-ring row. Runs in a fresh goroutine
|
||||
// so it cannot block the response, but inherits a detached context with
|
||||
// a short timeout via the inner call site. Failures are logged at debug
|
||||
// level since they're non-fatal for the request.
|
||||
func recordInjection(store *messaging.MemoryInjections, logger *slog.Logger, agent *agents.Agent, toolName string, pkt *search.ContextPacket) {
|
||||
if store == nil || agent == nil || pkt == nil {
|
||||
return
|
||||
}
|
||||
ids := make([]int64, 0, len(pkt.Memories))
|
||||
for _, m := range pkt.Memories {
|
||||
ids = append(ids, m.ID)
|
||||
}
|
||||
rec := messaging.InjectionRecord{
|
||||
OwnerID: ownerIDString(agent.OwnerID),
|
||||
AgentName: agent.Name,
|
||||
ToolName: toolName,
|
||||
PacketSizeChars: pkt.PacketChars,
|
||||
PacketItemsCount: len(pkt.Memories),
|
||||
MessageIDs: ids,
|
||||
CoreBlobIncluded: pkt.CoreMemory != "",
|
||||
}
|
||||
go func() {
|
||||
// Detached background context: the request context may already
|
||||
// be canceled by the time this goroutine runs.
|
||||
ctx := context.Background()
|
||||
if err := store.Record(ctx, rec); err != nil {
|
||||
logger.Debug("audit-ring insert failed", "tool", toolName, "error", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
func ownerIDString(id int64) string {
|
||||
if id == 0 {
|
||||
return ""
|
||||
}
|
||||
// strconv.FormatInt is faster than fmt.Sprintf; mirror what
|
||||
// agents.OwnerFor produces so comparisons in the search layer line
|
||||
// up.
|
||||
return formatInt64(id)
|
||||
}
|
||||
|
||||
// formatInt64 is a tiny helper to avoid pulling strconv into the public
|
||||
// surface area for one line.
|
||||
func formatInt64(v int64) string {
|
||||
const digits = "0123456789"
|
||||
if v == 0 {
|
||||
return "0"
|
||||
}
|
||||
neg := false
|
||||
if v < 0 {
|
||||
neg = true
|
||||
v = -v
|
||||
}
|
||||
var buf [20]byte
|
||||
i := len(buf)
|
||||
for v > 0 {
|
||||
i--
|
||||
buf[i] = digits[v%10]
|
||||
v /= 10
|
||||
}
|
||||
if neg {
|
||||
i--
|
||||
buf[i] = '-'
|
||||
}
|
||||
return string(buf[i:])
|
||||
}
|
||||
@@ -0,0 +1,218 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
mcplib "github.com/mark3labs/mcp-go/mcp"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
"github.com/synapbus/synapbus/internal/search"
|
||||
)
|
||||
|
||||
// stubBuilder lets us bypass the real search service and short-circuit
|
||||
// BuildContextPacket to whatever ContextPacket we want for the test.
|
||||
// The wrap layer doesn't expose a builder seam (it calls
|
||||
// search.BuildContextPacket directly), so we instead drive the wrap end
|
||||
// to end with a real (empty) *search.Service and a stub CoreProvider
|
||||
// that emits a packet when IncludeCore is true.
|
||||
|
||||
type stubCoreProvider struct{ blob string }
|
||||
|
||||
func (s *stubCoreProvider) Get(_ context.Context, _, _ string) (string, error) {
|
||||
return s.blob, nil
|
||||
}
|
||||
|
||||
func stubHandler(body map[string]any) ToolHandler {
|
||||
return func(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
b, _ := json.Marshal(body)
|
||||
return mcplib.NewToolResultText(string(b)), nil
|
||||
}
|
||||
}
|
||||
|
||||
func extractJSON(t *testing.T, res *mcplib.CallToolResult) map[string]any {
|
||||
t.Helper()
|
||||
if res == nil {
|
||||
t.Fatal("nil result")
|
||||
}
|
||||
if len(res.Content) != 1 {
|
||||
t.Fatalf("expected 1 content, got %d", len(res.Content))
|
||||
}
|
||||
tc, ok := res.Content[0].(mcplib.TextContent)
|
||||
if !ok {
|
||||
t.Fatalf("content not TextContent: %T", res.Content[0])
|
||||
}
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal([]byte(tc.Text), &m); err != nil {
|
||||
t.Fatalf("non-JSON content: %v: %q", err, tc.Text)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func TestWrapInjection_DisabledReturnsHandlerUnchanged(t *testing.T) {
|
||||
inner := stubHandler(map[string]any{"hello": "world"})
|
||||
cfg := WrapConfig{
|
||||
Cfg: messaging.MemoryConfig{InjectionEnabled: false},
|
||||
}
|
||||
wrapped := WrapInjection(inner, "my_status", cfg)
|
||||
|
||||
res, err := wrapped(context.Background(), mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("wrapped: %v", err)
|
||||
}
|
||||
body := extractJSON(t, res)
|
||||
if _, has := body["relevant_context"]; has {
|
||||
t.Error("relevant_context attached despite disabled config")
|
||||
}
|
||||
if body["hello"] != "world" {
|
||||
t.Errorf("inner body mutated: %v", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWrapInjection_EmptyMemoriesAndNoCore_OmitsField(t *testing.T) {
|
||||
// IncludeCore=false and no memories in the DB → packet is nil → no
|
||||
// relevant_context field on the response.
|
||||
cfg := WrapConfig{
|
||||
Cfg: messaging.MemoryConfig{
|
||||
InjectionEnabled: true,
|
||||
InjectionBudgetTokens: 500,
|
||||
InjectionMaxItems: 5,
|
||||
InjectionMinScore: 0.25,
|
||||
},
|
||||
SearchSvc: nil, // BuildContextPacket short-circuits when query=="" → returns nil
|
||||
QuerySource: func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string { return "" },
|
||||
}
|
||||
// SearchSvc nil + empty query forces BuildContextPacket through the
|
||||
// "no retrieval" path. But it'll still try a core fetch (skipped:
|
||||
// IncludeCore=false). With no provider and no retrieval → nil packet.
|
||||
inner := stubHandler(map[string]any{"ok": true})
|
||||
wrapped := WrapInjection(inner, "send_message", cfg)
|
||||
|
||||
// Inject a caller agent into the context so the wrapper does not
|
||||
// bail out at the identity check.
|
||||
ctx := agents.ContextWithAgent(context.Background(), &agents.Agent{Name: "a1", OwnerID: 1})
|
||||
|
||||
res, err := wrapped(ctx, mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("wrapped: %v", err)
|
||||
}
|
||||
body := extractJSON(t, res)
|
||||
if _, has := body["relevant_context"]; has {
|
||||
t.Errorf("relevant_context attached when memories+core empty: %v", body["relevant_context"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestWrapInjection_AppendsRelevantContext_CoreOnly(t *testing.T) {
|
||||
cfg := WrapConfig{
|
||||
Cfg: messaging.MemoryConfig{
|
||||
InjectionEnabled: true,
|
||||
InjectionBudgetTokens: 500,
|
||||
InjectionMaxItems: 5,
|
||||
InjectionMinScore: 0.25,
|
||||
},
|
||||
SearchSvc: nil, // query="" guarantees no retrieval attempt
|
||||
IncludeCore: true,
|
||||
CoreProvider: &stubCoreProvider{blob: "I am a1."},
|
||||
QuerySource: func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string { return "" },
|
||||
}
|
||||
inner := stubHandler(map[string]any{"agent": "a1"})
|
||||
wrapped := WrapInjection(inner, "my_status", cfg)
|
||||
|
||||
ctx := agents.ContextWithAgent(context.Background(), &agents.Agent{Name: "a1", OwnerID: 1})
|
||||
res, err := wrapped(ctx, mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("wrapped: %v", err)
|
||||
}
|
||||
body := extractJSON(t, res)
|
||||
rc, has := body["relevant_context"].(map[string]any)
|
||||
if !has {
|
||||
t.Fatalf("relevant_context missing: %+v", body)
|
||||
}
|
||||
if rc["core_memory"] != "I am a1." {
|
||||
t.Errorf("core_memory wrong: %v", rc["core_memory"])
|
||||
}
|
||||
if mems, ok := rc["memories"].([]any); !ok || len(mems) != 0 {
|
||||
t.Errorf("memories should be empty slice when only core is set: %v", rc["memories"])
|
||||
}
|
||||
// PacketChars at minimum the length of the core blob.
|
||||
if got, ok := rc["packet_chars"].(float64); !ok || int(got) < len("I am a1.") {
|
||||
t.Errorf("packet_chars looks wrong: %v", rc["packet_chars"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestWrapInjection_NonJSONResultPassesThrough(t *testing.T) {
|
||||
inner := func(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
return mcplib.NewToolResultText("not json"), nil
|
||||
}
|
||||
cfg := WrapConfig{
|
||||
Cfg: messaging.MemoryConfig{
|
||||
InjectionEnabled: true,
|
||||
InjectionBudgetTokens: 500,
|
||||
InjectionMaxItems: 5,
|
||||
},
|
||||
QuerySource: func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string { return "" },
|
||||
}
|
||||
wrapped := WrapInjection(inner, "execute", cfg)
|
||||
ctx := agents.ContextWithAgent(context.Background(), &agents.Agent{Name: "a1", OwnerID: 1})
|
||||
res, err := wrapped(ctx, mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("wrapped: %v", err)
|
||||
}
|
||||
if res == nil || len(res.Content) != 1 {
|
||||
t.Fatalf("unexpected result shape: %+v", res)
|
||||
}
|
||||
tc, ok := res.Content[0].(mcplib.TextContent)
|
||||
if !ok || tc.Text != "not json" {
|
||||
t.Errorf("non-JSON result mutated: %+v", res.Content[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestWrapInjection_NoAgentInContext_PassesThrough(t *testing.T) {
|
||||
cfg := WrapConfig{
|
||||
Cfg: messaging.MemoryConfig{
|
||||
InjectionEnabled: true,
|
||||
InjectionBudgetTokens: 500,
|
||||
InjectionMaxItems: 5,
|
||||
},
|
||||
IncludeCore: true,
|
||||
CoreProvider: &stubCoreProvider{blob: "blob"},
|
||||
QuerySource: func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string { return "" },
|
||||
}
|
||||
wrapped := WrapInjection(stubHandler(map[string]any{"x": 1}), "my_status", cfg)
|
||||
res, err := wrapped(context.Background(), mcplib.CallToolRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("wrapped: %v", err)
|
||||
}
|
||||
body := extractJSON(t, res)
|
||||
if _, has := body["relevant_context"]; has {
|
||||
t.Error("relevant_context attached despite no caller agent")
|
||||
}
|
||||
}
|
||||
|
||||
// Ensure ContextPacket as the value carries through json round-trip
|
||||
// (it's used directly as a map entry via body["relevant_context"] = pkt).
|
||||
func TestWrapInjection_ContextPacketJSONShape(t *testing.T) {
|
||||
pkt := &search.ContextPacket{
|
||||
Memories: []search.MemoryItem{},
|
||||
CoreMemory: "core",
|
||||
PacketChars: 4,
|
||||
PacketTokenEstimate: 1,
|
||||
RetrievalQuery: "",
|
||||
SearchMode: "auto",
|
||||
}
|
||||
b, err := json.Marshal(pkt)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal: %v", err)
|
||||
}
|
||||
var rt map[string]any
|
||||
if err := json.Unmarshal(b, &rt); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
for _, k := range []string{"memories", "core_memory", "packet_chars", "packet_token_estimate", "retrieval_query", "search_mode"} {
|
||||
if _, has := rt[k]; !has {
|
||||
t.Errorf("missing JSON key %q", k)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,663 @@
|
||||
// MCP memory-consolidation tools (feature 020 — dream worker, US3).
|
||||
//
|
||||
// Registered only when SYNAPBUS_DREAM_ENABLED=1 via
|
||||
// MemoryToolRegistrar.RegisterAllOnServer (see server.go SetDream).
|
||||
//
|
||||
// Every tool:
|
||||
//
|
||||
// 1. Pulls the dispatch token from the request context (set by
|
||||
// MCP middleware reading X-Synapbus-Dispatch-Token from the
|
||||
// transport header, or via the harness-propagated env var).
|
||||
// 2. Validates the token against (caller-supplied owner_id, the
|
||||
// active consolidation_job_id carried alongside the token).
|
||||
// 3. Performs the action against the appropriate messaging store.
|
||||
// 4. Appends a structured action record to the job's `actions`
|
||||
// JSON array via JobsStore.AppendAction.
|
||||
//
|
||||
// All errors follow the MCP standard envelope; the contract codes
|
||||
// (`dispatch_token_*`, `not_same_owner`, `core_memory_too_large`,
|
||||
// `relation_type_reserved`, `source_not_found`, ...) are listed in
|
||||
// `contracts/mcp-memory-tools.md` and mirrored verbatim here so the
|
||||
// dream-agent can match on the string.
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
mcplib "github.com/mark3labs/mcp-go/mcp"
|
||||
"github.com/mark3labs/mcp-go/server"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
)
|
||||
|
||||
// Context key for the dispatch token. The transport-layer middleware
|
||||
// (or test harness) stuffs the token from the HTTP header
|
||||
// `X-Synapbus-Dispatch-Token` into ctx via WithDispatchToken; the
|
||||
// memory tool handlers read it via DispatchTokenFromContext.
|
||||
type dispatchTokenKey struct{}
|
||||
|
||||
// WithDispatchToken returns a derived context carrying tok as the
|
||||
// active dispatch token.
|
||||
func WithDispatchToken(ctx context.Context, tok string) context.Context {
|
||||
return context.WithValue(ctx, dispatchTokenKey{}, tok)
|
||||
}
|
||||
|
||||
// DispatchTokenFromContext returns the dispatch token, if any.
|
||||
func DispatchTokenFromContext(ctx context.Context) (string, bool) {
|
||||
v, ok := ctx.Value(dispatchTokenKey{}).(string)
|
||||
return v, ok && v != ""
|
||||
}
|
||||
|
||||
// MemoryToolDeps bundles the dependencies the six memory tools need.
|
||||
type MemoryToolDeps struct {
|
||||
DB *sql.DB
|
||||
Msg *messaging.MessagingService
|
||||
Agents *agents.AgentService
|
||||
Core *messaging.CoreMemoryStore
|
||||
Links *messaging.LinkStore
|
||||
Pins *messaging.PinStore
|
||||
Jobs *messaging.JobsStore
|
||||
Tokens *messaging.DispatchTokenStore
|
||||
MemConfig messaging.MemoryConfig
|
||||
Logger *slog.Logger
|
||||
}
|
||||
|
||||
// MemoryToolRegistrar registers the six memory_* MCP tools.
|
||||
type MemoryToolRegistrar struct {
|
||||
deps MemoryToolDeps
|
||||
logger *slog.Logger
|
||||
}
|
||||
|
||||
// NewMemoryToolRegistrar returns a registrar over deps. RegisterAllOnServer
|
||||
// is a no-op when deps.DB or deps.Jobs is nil (defensive — these are required).
|
||||
func NewMemoryToolRegistrar(deps MemoryToolDeps) *MemoryToolRegistrar {
|
||||
logger := deps.Logger
|
||||
if logger == nil {
|
||||
logger = slog.Default().With("component", "mcp-memory-tools")
|
||||
}
|
||||
return &MemoryToolRegistrar{deps: deps, logger: logger}
|
||||
}
|
||||
|
||||
// RegisterAllOnServer attaches the six tools to mcpSrv. The caller
|
||||
// (server.go SetDream) is responsible for gating registration on
|
||||
// SYNAPBUS_DREAM_ENABLED.
|
||||
func (r *MemoryToolRegistrar) RegisterAllOnServer(s *server.MCPServer) {
|
||||
if s == nil || r.deps.DB == nil || r.deps.Jobs == nil || r.deps.Tokens == nil {
|
||||
return
|
||||
}
|
||||
s.AddTool(memoryListUnprocessedTool(), r.handleListUnprocessed)
|
||||
s.AddTool(memoryWriteReflectionTool(), r.handleWriteReflection)
|
||||
s.AddTool(memoryRewriteCoreTool(), r.handleRewriteCore)
|
||||
s.AddTool(memoryMarkDuplicateTool(), r.handleMarkDuplicate)
|
||||
s.AddTool(memorySupersedeTool(), r.handleSupersede)
|
||||
s.AddTool(memoryAddLinkTool(), r.handleAddLink)
|
||||
r.logger.Info("memory MCP tools registered", "count", 6)
|
||||
}
|
||||
|
||||
// --- Tool definitions ---
|
||||
|
||||
func memoryListUnprocessedTool() mcplib.Tool {
|
||||
return mcplib.NewTool("memory_list_unprocessed",
|
||||
mcplib.WithDescription("List recent memory-eligible messages the owner's pool has not yet consolidated. Used by the dream agent to scan its inbox."),
|
||||
mcplib.WithString("owner_id", mcplib.Description("Caller's owner_id (must match the dispatch token's owner)"), mcplib.Required()),
|
||||
mcplib.WithNumber("since_message_id", mcplib.Description("Exclusive lower bound (defaults to 0)")),
|
||||
mcplib.WithNumber("limit", mcplib.Description("Max items to return (default 50, max 200)")),
|
||||
)
|
||||
}
|
||||
|
||||
func memoryWriteReflectionTool() mcplib.Tool {
|
||||
return mcplib.NewTool("memory_write_reflection",
|
||||
mcplib.WithDescription("Write a higher-level abstraction back to the memory pool tagged 'reflection'. Inserts 'refines' links from the new memory to each source."),
|
||||
mcplib.WithString("owner_id", mcplib.Required()),
|
||||
mcplib.WithString("body", mcplib.Required()),
|
||||
mcplib.WithString("source_message_ids", mcplib.Description("Comma-separated message ids")),
|
||||
mcplib.WithString("tags", mcplib.Description("Comma-separated tags")),
|
||||
)
|
||||
}
|
||||
|
||||
func memoryRewriteCoreTool() mcplib.Tool {
|
||||
return mcplib.NewTool("memory_rewrite_core",
|
||||
mcplib.WithDescription("Replace the per-(owner, agent) core memory blob wholesale (no merge)."),
|
||||
mcplib.WithString("owner_id", mcplib.Required()),
|
||||
mcplib.WithString("agent_name", mcplib.Required()),
|
||||
mcplib.WithString("blob", mcplib.Required()),
|
||||
)
|
||||
}
|
||||
|
||||
func memoryMarkDuplicateTool() mcplib.Tool {
|
||||
return mcplib.NewTool("memory_mark_duplicate",
|
||||
mcplib.WithDescription("Mark two memories as duplicates; one is kept canonical, the other soft-deleted."),
|
||||
mcplib.WithString("owner_id", mcplib.Required()),
|
||||
mcplib.WithNumber("a_id", mcplib.Required()),
|
||||
mcplib.WithNumber("b_id", mcplib.Required()),
|
||||
mcplib.WithNumber("keep_id", mcplib.Required()),
|
||||
mcplib.WithString("reason"),
|
||||
)
|
||||
}
|
||||
|
||||
func memorySupersedeTool() mcplib.Tool {
|
||||
return mcplib.NewTool("memory_supersede",
|
||||
mcplib.WithDescription("Mark memory A as obsoleted by memory B (temporal validity)."),
|
||||
mcplib.WithString("owner_id", mcplib.Required()),
|
||||
mcplib.WithNumber("a_id", mcplib.Required()),
|
||||
mcplib.WithNumber("b_id", mcplib.Required()),
|
||||
mcplib.WithString("reason"),
|
||||
)
|
||||
}
|
||||
|
||||
func memoryAddLinkTool() mcplib.Tool {
|
||||
return mcplib.NewTool("memory_add_link",
|
||||
mcplib.WithDescription("Add a typed link between two memories. relation_type must be one of refines, contradicts, examples, related."),
|
||||
mcplib.WithString("owner_id", mcplib.Required()),
|
||||
mcplib.WithNumber("src_id", mcplib.Required()),
|
||||
mcplib.WithNumber("dst_id", mcplib.Required()),
|
||||
mcplib.WithString("relation_type", mcplib.Required()),
|
||||
mcplib.WithString("metadata", mcplib.Description("JSON object")),
|
||||
)
|
||||
}
|
||||
|
||||
// --- Shared validation ---
|
||||
|
||||
// authorizeForOwner validates the dispatch token in ctx against the
|
||||
// caller-supplied owner_id. Returns the active jobID (so the handler
|
||||
// can call AppendAction) or an MCP error result.
|
||||
func (r *MemoryToolRegistrar) authorizeForOwner(ctx context.Context, ownerID string) (jobID int64, errResult *mcplib.CallToolResult) {
|
||||
tok, ok := DispatchTokenFromContext(ctx)
|
||||
if !ok {
|
||||
return 0, memErrorf("dispatch_token_missing", "no dispatch token in request context")
|
||||
}
|
||||
// Find the consolidation_job_id this token is bound to.
|
||||
var (
|
||||
dbOwner string
|
||||
dbJob int64
|
||||
expiresAt time.Time
|
||||
revokedAt sql.NullTime
|
||||
)
|
||||
err := r.deps.DB.QueryRowContext(ctx,
|
||||
`SELECT owner_id, consolidation_job_id, expires_at, revoked_at
|
||||
FROM memory_dispatch_tokens WHERE token = ?`, tok,
|
||||
).Scan(&dbOwner, &dbJob, &expiresAt, &revokedAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return 0, memErrorf("dispatch_token_missing", "token not found")
|
||||
}
|
||||
return 0, memErrorf("dispatch_token_missing", "token lookup failed: %s", err)
|
||||
}
|
||||
if revokedAt.Valid {
|
||||
return 0, memErrorf("dispatch_token_revoked", "token has been revoked")
|
||||
}
|
||||
if !expiresAt.After(time.Now().UTC()) {
|
||||
return 0, memErrorf("dispatch_token_expired", "token expired at %s", expiresAt.Format(time.RFC3339))
|
||||
}
|
||||
if dbOwner != ownerID {
|
||||
return 0, memErrorf("dispatch_token_owner_mismatch", "token bound to %q, request claims %q", dbOwner, ownerID)
|
||||
}
|
||||
// Run the canonical Validate path so used_at is stamped uniformly.
|
||||
if r.deps.Tokens != nil {
|
||||
if _, err := r.deps.Tokens.Validate(ctx, tok, ownerID, dbJob); err != nil {
|
||||
return 0, memErrorf("dispatch_token_missing", "validate: %s", err)
|
||||
}
|
||||
}
|
||||
return dbJob, nil
|
||||
}
|
||||
|
||||
// recordAction appends to the job's actions JSON array. Logged at
|
||||
// warn-level on failure; never blocks the tool's user-visible response.
|
||||
func (r *MemoryToolRegistrar) recordAction(ctx context.Context, jobID int64, tool string, targetID int64, args map[string]any) {
|
||||
if r.deps.Jobs == nil {
|
||||
return
|
||||
}
|
||||
action := map[string]any{
|
||||
"tool": tool,
|
||||
"target_message_id": targetID,
|
||||
"args": args,
|
||||
"at": time.Now().UTC().Format(time.RFC3339),
|
||||
}
|
||||
if err := r.deps.Jobs.AppendAction(ctx, jobID, action); err != nil {
|
||||
r.logger.Warn("append action failed",
|
||||
"job_id", jobID,
|
||||
"tool", tool,
|
||||
"error", err,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// memErrorf returns an MCP CallToolResult carrying a contractual error
|
||||
// code + human-readable message. The MCP framework already wraps the
|
||||
// `error` field in the JSON envelope; we render `code: ...` as the
|
||||
// leading line of the message so the dream-agent can pattern-match.
|
||||
func memErrorf(code, format string, args ...any) *mcplib.CallToolResult {
|
||||
msg := fmt.Sprintf("%s: %s", code, fmt.Sprintf(format, args...))
|
||||
return mcplib.NewToolResultError(msg)
|
||||
}
|
||||
|
||||
// --- Handlers ---
|
||||
|
||||
func (r *MemoryToolRegistrar) handleListUnprocessed(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
owner := req.GetString("owner_id", "")
|
||||
if owner == "" {
|
||||
return memErrorf("invalid_request", "owner_id required"), nil
|
||||
}
|
||||
jobID, errR := r.authorizeForOwner(ctx, owner)
|
||||
if errR != nil {
|
||||
return errR, nil
|
||||
}
|
||||
|
||||
since := int64(req.GetInt("since_message_id", 0))
|
||||
limit := req.GetInt("limit", 50)
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
if limit > 200 {
|
||||
limit = 200
|
||||
}
|
||||
|
||||
memIDs, err := messaging.MemoryChannelIDs(ctx, r.deps.DB)
|
||||
if err != nil {
|
||||
return memErrorf("internal", "list memory channels: %s", err), nil
|
||||
}
|
||||
if len(memIDs) == 0 {
|
||||
_ = jobID // record nothing — empty
|
||||
return resultJSON(map[string]any{"memories": []any{}, "max_id_returned": since})
|
||||
}
|
||||
|
||||
placeholders := strings.Repeat("?,", len(memIDs))
|
||||
placeholders = placeholders[:len(placeholders)-1]
|
||||
queryArgs := []any{}
|
||||
for _, id := range memIDs {
|
||||
queryArgs = append(queryArgs, id)
|
||||
}
|
||||
// Apply the same 14d (configurable) recency window the worker uses
|
||||
// so the consolidation agent only ever sees a bounded input set.
|
||||
windowDays := int(r.deps.MemConfig.DreamRecentWindow / (24 * time.Hour))
|
||||
if windowDays < 1 {
|
||||
windowDays = 14
|
||||
}
|
||||
windowExpr := fmt.Sprintf("-%d days", windowDays)
|
||||
queryArgs = append(queryArgs, owner, since, windowExpr, limit)
|
||||
|
||||
// Contract guarantees this list excludes:
|
||||
// - messages already linked as the dst_message_id of a
|
||||
// refines/duplicate_of/superseded_by edge (already
|
||||
// consolidated by an earlier dream pass), AND
|
||||
// - the dream worker's own output (from_agent prefix "dream:")
|
||||
// so the agent never re-refines its own reflections.
|
||||
// Without these filters the agent loops on the same oldest-50
|
||||
// messages every cycle and progress flat-lines.
|
||||
q := `SELECT m.id, m.from_agent, c.name, m.body, m.created_at
|
||||
FROM messages m
|
||||
JOIN agents a ON m.from_agent = a.name
|
||||
JOIN channels c ON m.channel_id = c.id
|
||||
WHERE m.channel_id IN (` + placeholders + `)
|
||||
AND CAST(a.owner_id AS TEXT) = ?
|
||||
AND m.id > ?
|
||||
AND m.created_at > datetime('now', ?)
|
||||
AND m.from_agent NOT LIKE 'dream:%'
|
||||
AND m.id NOT IN (
|
||||
SELECT dst_message_id FROM memory_links
|
||||
WHERE relation_type IN ('refines','duplicate_of','superseded_by')
|
||||
)
|
||||
ORDER BY m.id ASC
|
||||
LIMIT ?`
|
||||
|
||||
rows, err := r.deps.DB.QueryContext(ctx, q, queryArgs...)
|
||||
if err != nil {
|
||||
return memErrorf("internal", "query: %s", err), nil
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
type item struct {
|
||||
ID int64 `json:"id"`
|
||||
FromAgent string `json:"from_agent"`
|
||||
Channel string `json:"channel"`
|
||||
Body string `json:"body"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
var items []item
|
||||
var maxID = since
|
||||
for rows.Next() {
|
||||
var it item
|
||||
if err := rows.Scan(&it.ID, &it.FromAgent, &it.Channel, &it.Body, &it.CreatedAt); err != nil {
|
||||
return memErrorf("internal", "scan: %s", err), nil
|
||||
}
|
||||
items = append(items, it)
|
||||
if it.ID > maxID {
|
||||
maxID = it.ID
|
||||
}
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return memErrorf("internal", "iterate: %s", err), nil
|
||||
}
|
||||
|
||||
r.recordAction(ctx, jobID, "memory_list_unprocessed", 0, map[string]any{
|
||||
"since_message_id": since, "limit": limit, "returned": len(items),
|
||||
})
|
||||
return resultJSON(map[string]any{"memories": items, "max_id_returned": maxID})
|
||||
}
|
||||
|
||||
func (r *MemoryToolRegistrar) handleWriteReflection(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
owner := req.GetString("owner_id", "")
|
||||
body := req.GetString("body", "")
|
||||
if owner == "" || body == "" {
|
||||
return memErrorf("invalid_request", "owner_id and body required"), nil
|
||||
}
|
||||
jobID, errR := r.authorizeForOwner(ctx, owner)
|
||||
if errR != nil {
|
||||
return errR, nil
|
||||
}
|
||||
sourceIDs := parseInt64CSV(req.GetString("source_message_ids", ""))
|
||||
|
||||
// Verify every source belongs to caller's owner.
|
||||
for _, sid := range sourceIDs {
|
||||
ok, sameOwner, _ := r.messageBelongsTo(ctx, sid, owner)
|
||||
if !ok {
|
||||
return memErrorf("source_not_found", "message %d not found", sid), nil
|
||||
}
|
||||
if !sameOwner {
|
||||
return memErrorf("not_same_owner", "source %d belongs to a different owner", sid), nil
|
||||
}
|
||||
}
|
||||
|
||||
// Pick a destination channel: prefer `#reflections-<owner>` if it
|
||||
// exists, else `#open-brain`.
|
||||
channelID, channelName, err := r.pickReflectionChannel(ctx, owner)
|
||||
if err != nil {
|
||||
return memErrorf("internal", "pick channel: %s", err), nil
|
||||
}
|
||||
if channelID == 0 {
|
||||
return memErrorf("internal", "no reflection channel available"), nil
|
||||
}
|
||||
|
||||
dreamAgent := "dream:" + owner
|
||||
// Ensure conversation + message inserts (lightweight direct SQL —
|
||||
// the MessagingService path would trigger reactive runs which we
|
||||
// must avoid per feedback_system_dm_no_trigger.md).
|
||||
convRes, err := r.deps.DB.ExecContext(ctx,
|
||||
`INSERT INTO conversations (created_by, channel_id) VALUES (?, ?)`,
|
||||
dreamAgent, channelID,
|
||||
)
|
||||
if err != nil {
|
||||
return memErrorf("internal", "create conversation: %s", err), nil
|
||||
}
|
||||
convID, _ := convRes.LastInsertId()
|
||||
|
||||
res, err := r.deps.DB.ExecContext(ctx,
|
||||
`INSERT INTO messages (conversation_id, from_agent, channel_id, body, priority, status, metadata)
|
||||
VALUES (?, ?, ?, ?, 5, 'pending', ?)`,
|
||||
convID, dreamAgent, channelID, body, `{"tags":["reflection"]}`,
|
||||
)
|
||||
if err != nil {
|
||||
return memErrorf("internal", "insert message: %s", err), nil
|
||||
}
|
||||
newID, _ := res.LastInsertId()
|
||||
|
||||
// Add `refines` links from new memory → each source.
|
||||
created := 0
|
||||
for _, sid := range sourceIDs {
|
||||
if r.deps.Links == nil {
|
||||
break
|
||||
}
|
||||
if _, err := r.deps.Links.Add(ctx, newID, sid, "refines", owner, "agent:dream:"+owner, nil); err == nil {
|
||||
created++
|
||||
}
|
||||
}
|
||||
|
||||
r.recordAction(ctx, jobID, "memory_write_reflection", newID, map[string]any{
|
||||
"source_message_ids": sourceIDs,
|
||||
"channel": channelName,
|
||||
})
|
||||
|
||||
return resultJSON(map[string]any{
|
||||
"memory_id": newID,
|
||||
"channel": channelName,
|
||||
"links_created": created,
|
||||
})
|
||||
}
|
||||
|
||||
func (r *MemoryToolRegistrar) handleRewriteCore(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
owner := req.GetString("owner_id", "")
|
||||
agent := req.GetString("agent_name", "")
|
||||
blob := req.GetString("blob", "")
|
||||
if owner == "" || agent == "" {
|
||||
return memErrorf("invalid_request", "owner_id and agent_name required"), nil
|
||||
}
|
||||
jobID, errR := r.authorizeForOwner(ctx, owner)
|
||||
if errR != nil {
|
||||
return errR, nil
|
||||
}
|
||||
if r.deps.Core == nil {
|
||||
return memErrorf("internal", "core memory store not configured"), nil
|
||||
}
|
||||
// Confirm target agent is owned by caller's owner.
|
||||
targetOwner, err := agents.OwnerFor(ctx, r.deps.DB, agent)
|
||||
if err != nil {
|
||||
if errors.Is(err, agents.ErrAgentNotFound) {
|
||||
return memErrorf("source_not_found", "agent %q not found", agent), nil
|
||||
}
|
||||
return memErrorf("internal", "owner lookup: %s", err), nil
|
||||
}
|
||||
if targetOwner != owner {
|
||||
return memErrorf("not_same_owner", "agent %q owner=%q != caller %q", agent, targetOwner, owner), nil
|
||||
}
|
||||
prev, _, _, _ := r.deps.Core.Get(ctx, owner, agent)
|
||||
if err := r.deps.Core.Set(ctx, owner, agent, blob, "agent:dream:"+owner); err != nil {
|
||||
if errors.Is(err, messaging.ErrCoreMemoryTooLarge) {
|
||||
return memErrorf("core_memory_too_large", "blob %d bytes exceeds cap", len(blob)), nil
|
||||
}
|
||||
return memErrorf("internal", "set core: %s", err), nil
|
||||
}
|
||||
r.recordAction(ctx, jobID, "memory_rewrite_core", 0, map[string]any{
|
||||
"owner_id": owner, "agent_name": agent, "new_chars": len(blob),
|
||||
})
|
||||
return resultJSON(map[string]any{
|
||||
"owner_id": owner,
|
||||
"agent_name": agent,
|
||||
"previous_blob": prev,
|
||||
"new_blob_chars": len(blob),
|
||||
"updated_at": time.Now().UTC().Format(time.RFC3339),
|
||||
})
|
||||
}
|
||||
|
||||
func (r *MemoryToolRegistrar) handleMarkDuplicate(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
owner := req.GetString("owner_id", "")
|
||||
aID := int64(req.GetInt("a_id", 0))
|
||||
bID := int64(req.GetInt("b_id", 0))
|
||||
keepID := int64(req.GetInt("keep_id", 0))
|
||||
reason := req.GetString("reason", "")
|
||||
if owner == "" || aID == 0 || bID == 0 || keepID == 0 {
|
||||
return memErrorf("invalid_request", "owner_id, a_id, b_id, keep_id required"), nil
|
||||
}
|
||||
if keepID != aID && keepID != bID {
|
||||
return memErrorf("keep_id_not_in_pair", "keep_id must be a_id or b_id"), nil
|
||||
}
|
||||
jobID, errR := r.authorizeForOwner(ctx, owner)
|
||||
if errR != nil {
|
||||
return errR, nil
|
||||
}
|
||||
for _, id := range []int64{aID, bID} {
|
||||
ok, sameOwner, _ := r.messageBelongsTo(ctx, id, owner)
|
||||
if !ok {
|
||||
return memErrorf("source_not_found", "message %d not found", id), nil
|
||||
}
|
||||
if !sameOwner {
|
||||
return memErrorf("not_same_owner", "message %d belongs to a different owner", id), nil
|
||||
}
|
||||
}
|
||||
loserID := aID
|
||||
if keepID == aID {
|
||||
loserID = bID
|
||||
}
|
||||
if r.deps.Links == nil {
|
||||
return memErrorf("internal", "link store not configured"), nil
|
||||
}
|
||||
linkID, err := r.deps.Links.AddConsolidationLink(ctx, loserID, keepID, "duplicate_of", owner, "agent:dream:"+owner, map[string]any{"reason": reason})
|
||||
if err != nil {
|
||||
return memErrorf("internal", "add link: %s", err), nil
|
||||
}
|
||||
r.recordAction(ctx, jobID, "memory_mark_duplicate", loserID, map[string]any{
|
||||
"a_id": aID, "b_id": bID, "keep_id": keepID, "reason": reason,
|
||||
})
|
||||
return resultJSON(map[string]any{
|
||||
"keep_id": keepID,
|
||||
"soft_deleted_id": loserID,
|
||||
"link_created_id": linkID,
|
||||
})
|
||||
}
|
||||
|
||||
func (r *MemoryToolRegistrar) handleSupersede(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
owner := req.GetString("owner_id", "")
|
||||
aID := int64(req.GetInt("a_id", 0))
|
||||
bID := int64(req.GetInt("b_id", 0))
|
||||
reason := req.GetString("reason", "")
|
||||
if owner == "" || aID == 0 || bID == 0 {
|
||||
return memErrorf("invalid_request", "owner_id, a_id, b_id required"), nil
|
||||
}
|
||||
jobID, errR := r.authorizeForOwner(ctx, owner)
|
||||
if errR != nil {
|
||||
return errR, nil
|
||||
}
|
||||
for _, id := range []int64{aID, bID} {
|
||||
ok, sameOwner, _ := r.messageBelongsTo(ctx, id, owner)
|
||||
if !ok {
|
||||
return memErrorf("source_not_found", "message %d not found", id), nil
|
||||
}
|
||||
if !sameOwner {
|
||||
return memErrorf("not_same_owner", "message %d belongs to a different owner", id), nil
|
||||
}
|
||||
}
|
||||
if r.deps.Links == nil {
|
||||
return memErrorf("internal", "link store not configured"), nil
|
||||
}
|
||||
linkID, err := r.deps.Links.AddConsolidationLink(ctx, aID, bID, "superseded_by", owner, "agent:dream:"+owner, map[string]any{"reason": reason})
|
||||
if err != nil {
|
||||
return memErrorf("internal", "add link: %s", err), nil
|
||||
}
|
||||
r.recordAction(ctx, jobID, "memory_supersede", aID, map[string]any{
|
||||
"a_id": aID, "b_id": bID, "reason": reason,
|
||||
})
|
||||
return resultJSON(map[string]any{
|
||||
"superseded_id": aID,
|
||||
"by_id": bID,
|
||||
"link_created_id": linkID,
|
||||
})
|
||||
}
|
||||
|
||||
func (r *MemoryToolRegistrar) handleAddLink(ctx context.Context, req mcplib.CallToolRequest) (*mcplib.CallToolResult, error) {
|
||||
owner := req.GetString("owner_id", "")
|
||||
srcID := int64(req.GetInt("src_id", 0))
|
||||
dstID := int64(req.GetInt("dst_id", 0))
|
||||
relType := req.GetString("relation_type", "")
|
||||
if owner == "" || srcID == 0 || dstID == 0 || relType == "" {
|
||||
return memErrorf("invalid_request", "owner_id, src_id, dst_id, relation_type required"), nil
|
||||
}
|
||||
jobID, errR := r.authorizeForOwner(ctx, owner)
|
||||
if errR != nil {
|
||||
return errR, nil
|
||||
}
|
||||
for _, id := range []int64{srcID, dstID} {
|
||||
ok, sameOwner, _ := r.messageBelongsTo(ctx, id, owner)
|
||||
if !ok {
|
||||
return memErrorf("source_not_found", "message %d not found", id), nil
|
||||
}
|
||||
if !sameOwner {
|
||||
return memErrorf("not_same_owner", "message %d belongs to a different owner", id), nil
|
||||
}
|
||||
}
|
||||
if r.deps.Links == nil {
|
||||
return memErrorf("internal", "link store not configured"), nil
|
||||
}
|
||||
var meta map[string]any
|
||||
if mraw := req.GetString("metadata", ""); mraw != "" {
|
||||
_ = json.Unmarshal([]byte(mraw), &meta)
|
||||
}
|
||||
linkID, err := r.deps.Links.Add(ctx, srcID, dstID, relType, owner, "agent:dream:"+owner, meta)
|
||||
if err != nil {
|
||||
if errors.Is(err, messaging.ErrLinkTypeReserved) {
|
||||
return memErrorf("relation_type_reserved", "type %q is reserved", relType), nil
|
||||
}
|
||||
return memErrorf("internal", "add link: %s", err), nil
|
||||
}
|
||||
r.recordAction(ctx, jobID, "memory_add_link", dstID, map[string]any{
|
||||
"src_id": srcID, "dst_id": dstID, "relation_type": relType,
|
||||
})
|
||||
return resultJSON(map[string]any{"link_id": linkID})
|
||||
}
|
||||
|
||||
// --- helpers ---
|
||||
|
||||
func parseInt64CSV(s string) []int64 {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
parts := strings.Split(s, ",")
|
||||
out := make([]int64, 0, len(parts))
|
||||
for _, p := range parts {
|
||||
p = strings.TrimSpace(p)
|
||||
if p == "" {
|
||||
continue
|
||||
}
|
||||
var v int64
|
||||
if _, err := fmt.Sscanf(p, "%d", &v); err == nil && v > 0 {
|
||||
out = append(out, v)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// messageBelongsTo reports (exists, ownerMatches, dbErr).
|
||||
func (r *MemoryToolRegistrar) messageBelongsTo(ctx context.Context, msgID int64, ownerID string) (bool, bool, error) {
|
||||
var fromAgent string
|
||||
err := r.deps.DB.QueryRowContext(ctx,
|
||||
`SELECT from_agent FROM messages WHERE id = ?`, msgID,
|
||||
).Scan(&fromAgent)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return false, false, nil
|
||||
}
|
||||
return false, false, err
|
||||
}
|
||||
owner, err := agents.OwnerFor(ctx, r.deps.DB, fromAgent)
|
||||
if err != nil {
|
||||
return true, false, nil
|
||||
}
|
||||
return true, owner == ownerID, nil
|
||||
}
|
||||
|
||||
// pickReflectionChannel picks a destination channel for a reflection.
|
||||
// Preference: `reflections-<owner>` if any such channel exists, else
|
||||
// `open-brain`.
|
||||
func (r *MemoryToolRegistrar) pickReflectionChannel(ctx context.Context, ownerID string) (int64, string, error) {
|
||||
// Try reflections-* the owner has authored to (best heuristic).
|
||||
var (
|
||||
id int64
|
||||
name string
|
||||
)
|
||||
err := r.deps.DB.QueryRowContext(ctx,
|
||||
`SELECT id, name FROM channels WHERE name LIKE 'reflections-%' ORDER BY id ASC LIMIT 1`,
|
||||
).Scan(&id, &name)
|
||||
if err == nil {
|
||||
return id, name, nil
|
||||
}
|
||||
if !errors.Is(err, sql.ErrNoRows) {
|
||||
return 0, "", err
|
||||
}
|
||||
// Fallback: open-brain.
|
||||
err = r.deps.DB.QueryRowContext(ctx,
|
||||
`SELECT id, name FROM channels WHERE name = 'open-brain' LIMIT 1`,
|
||||
).Scan(&id, &name)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return 0, "", nil
|
||||
}
|
||||
return 0, "", err
|
||||
}
|
||||
return id, name, nil
|
||||
}
|
||||
@@ -0,0 +1,375 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
mcplib "github.com/mark3labs/mcp-go/mcp"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
)
|
||||
|
||||
// memToolHarness bundles deps + helpers for the memory tools tests.
|
||||
type memToolHarness struct {
|
||||
db *sql.DB
|
||||
reg *MemoryToolRegistrar
|
||||
tokens *messaging.DispatchTokenStore
|
||||
jobs *messaging.JobsStore
|
||||
links *messaging.LinkStore
|
||||
pins *messaging.PinStore
|
||||
core *messaging.CoreMemoryStore
|
||||
jobID int64
|
||||
tokenStr string
|
||||
ownerID string
|
||||
}
|
||||
|
||||
func newMemToolHarness(t *testing.T) *memToolHarness {
|
||||
t.Helper()
|
||||
db := newTestDB(t)
|
||||
|
||||
// Seed two owners + their agents (used for owner mismatch tests).
|
||||
_, _ = db.Exec(`INSERT OR IGNORE INTO users (id, username, password_hash, display_name) VALUES (2, 'otheruser', 'hash', 'Other')`)
|
||||
_, _ = db.Exec(`INSERT OR IGNORE INTO agents (name, display_name, type, owner_id, api_key_hash, status) VALUES ('a-h1', 'a-h1', 'ai', 1, 'h1', 'active')`)
|
||||
_, _ = db.Exec(`INSERT OR IGNORE INTO agents (name, display_name, type, owner_id, api_key_hash, status) VALUES ('a-h2', 'a-h2', 'ai', 2, 'h2', 'active')`)
|
||||
|
||||
tokens := messaging.NewDispatchTokenStore(db)
|
||||
jobs := messaging.NewJobsStore(db)
|
||||
links := messaging.NewLinkStore(db)
|
||||
pins := messaging.NewPinStore(db)
|
||||
core := messaging.NewCoreMemoryStore(db, 64)
|
||||
|
||||
jobID, err := jobs.Create(context.Background(), "1", "reflection", "manual:test")
|
||||
if err != nil {
|
||||
t.Fatalf("create job: %v", err)
|
||||
}
|
||||
tok, _, err := tokens.Issue(context.Background(), "1", jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("issue token: %v", err)
|
||||
}
|
||||
|
||||
reg := NewMemoryToolRegistrar(MemoryToolDeps{
|
||||
DB: db,
|
||||
Core: core,
|
||||
Links: links,
|
||||
Pins: pins,
|
||||
Jobs: jobs,
|
||||
Tokens: tokens,
|
||||
})
|
||||
return &memToolHarness{
|
||||
db: db, reg: reg, tokens: tokens, jobs: jobs, links: links, pins: pins, core: core,
|
||||
jobID: jobID, tokenStr: tok, ownerID: "1",
|
||||
}
|
||||
}
|
||||
|
||||
func (h *memToolHarness) ctxWithToken(tok string) context.Context {
|
||||
return WithDispatchToken(context.Background(), tok)
|
||||
}
|
||||
|
||||
// seedMemoryChannel inserts an open-brain channel and one message
|
||||
// belonging to `agentName`.
|
||||
func (h *memToolHarness) seedMessage(t *testing.T, agentName, body string) int64 {
|
||||
t.Helper()
|
||||
_, _ = h.db.Exec(`INSERT OR IGNORE INTO channels (id, name, description, type, created_by) VALUES (1, 'open-brain', '', 'standard', 'system')`)
|
||||
res, err := h.db.Exec(
|
||||
`INSERT INTO conversations (created_by, channel_id) VALUES (?, 1)`, agentName)
|
||||
if err != nil {
|
||||
t.Fatalf("seed conv: %v", err)
|
||||
}
|
||||
convID, _ := res.LastInsertId()
|
||||
res, err = h.db.Exec(
|
||||
`INSERT INTO messages (conversation_id, from_agent, channel_id, body, priority, status, metadata)
|
||||
VALUES (?, ?, 1, ?, 5, 'pending', '{}')`,
|
||||
convID, agentName, body,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("seed message: %v", err)
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
return id
|
||||
}
|
||||
|
||||
func callRequest(args map[string]any) mcplib.CallToolRequest {
|
||||
return mcplib.CallToolRequest{
|
||||
Params: mcplib.CallToolParams{Arguments: args},
|
||||
}
|
||||
}
|
||||
|
||||
func resultText(t *testing.T, res *mcplib.CallToolResult) string {
|
||||
t.Helper()
|
||||
if res == nil || len(res.Content) == 0 {
|
||||
t.Fatal("nil/empty result")
|
||||
}
|
||||
tc, ok := res.Content[0].(mcplib.TextContent)
|
||||
if !ok {
|
||||
t.Fatalf("not TextContent: %T", res.Content[0])
|
||||
}
|
||||
return tc.Text
|
||||
}
|
||||
|
||||
func resultIsError(res *mcplib.CallToolResult) bool {
|
||||
return res != nil && res.IsError
|
||||
}
|
||||
|
||||
// --- Token error matrix ---
|
||||
|
||||
func TestMemoryTools_DispatchTokenMissing(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
// No token in context.
|
||||
res, _ := h.reg.handleAddLink(context.Background(), callRequest(map[string]any{
|
||||
"owner_id": "1", "src_id": 1.0, "dst_id": 2.0, "relation_type": "refines",
|
||||
}))
|
||||
if !resultIsError(res) {
|
||||
t.Fatal("expected error result")
|
||||
}
|
||||
if !strings.Contains(resultText(t, res), "dispatch_token_missing") {
|
||||
t.Errorf("expected dispatch_token_missing, got %q", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryTools_DispatchTokenExpired(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
// Forcibly expire.
|
||||
if _, err := h.db.Exec(
|
||||
`UPDATE memory_dispatch_tokens SET expires_at = ? WHERE token = ?`,
|
||||
time.Now().Add(-1*time.Minute).UTC(), h.tokenStr,
|
||||
); err != nil {
|
||||
t.Fatalf("expire: %v", err)
|
||||
}
|
||||
res, _ := h.reg.handleAddLink(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "src_id": 1.0, "dst_id": 2.0, "relation_type": "refines",
|
||||
}))
|
||||
if !resultIsError(res) || !strings.Contains(resultText(t, res), "dispatch_token_expired") {
|
||||
t.Errorf("expected dispatch_token_expired, got %q", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryTools_DispatchTokenOwnerMismatch(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
res, _ := h.reg.handleAddLink(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "2", "src_id": 1.0, "dst_id": 2.0, "relation_type": "refines",
|
||||
}))
|
||||
if !resultIsError(res) || !strings.Contains(resultText(t, res), "dispatch_token_owner_mismatch") {
|
||||
t.Errorf("expected dispatch_token_owner_mismatch, got %q", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
// --- memory_add_link ---
|
||||
|
||||
func TestMemoryAddLink_HappyPath(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
a := h.seedMessage(t, "a-h1", "fact A")
|
||||
b := h.seedMessage(t, "a-h1", "fact B")
|
||||
|
||||
res, err := h.reg.handleAddLink(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1",
|
||||
"src_id": float64(a),
|
||||
"dst_id": float64(b),
|
||||
"relation_type": "refines",
|
||||
}))
|
||||
if err != nil {
|
||||
t.Fatalf("handleAddLink: %v", err)
|
||||
}
|
||||
if resultIsError(res) {
|
||||
t.Fatalf("unexpected error: %s", resultText(t, res))
|
||||
}
|
||||
var body map[string]any
|
||||
if err := json.Unmarshal([]byte(resultText(t, res)), &body); err != nil {
|
||||
t.Fatalf("parse body: %v", err)
|
||||
}
|
||||
if _, ok := body["link_id"]; !ok {
|
||||
t.Errorf("expected link_id in response: %v", body)
|
||||
}
|
||||
|
||||
// And actions should have been appended.
|
||||
job, err := h.jobs.Get(context.Background(), h.jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("Get job: %v", err)
|
||||
}
|
||||
if len(job.Actions) != 1 || job.Actions[0]["tool"] != "memory_add_link" {
|
||||
t.Errorf("expected one action for memory_add_link, got %v", job.Actions)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryAddLink_RelationTypeReserved(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
a := h.seedMessage(t, "a-h1", "x")
|
||||
b := h.seedMessage(t, "a-h1", "y")
|
||||
res, _ := h.reg.handleAddLink(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "src_id": float64(a), "dst_id": float64(b),
|
||||
"relation_type": "mention",
|
||||
}))
|
||||
if !resultIsError(res) || !strings.Contains(resultText(t, res), "relation_type_reserved") {
|
||||
t.Errorf("expected relation_type_reserved, got %q", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryAddLink_NotSameOwner(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
src := h.seedMessage(t, "a-h1", "x")
|
||||
other := h.seedMessage(t, "a-h2", "y")
|
||||
res, _ := h.reg.handleAddLink(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "src_id": float64(src), "dst_id": float64(other),
|
||||
"relation_type": "refines",
|
||||
}))
|
||||
if !resultIsError(res) || !strings.Contains(resultText(t, res), "not_same_owner") {
|
||||
t.Errorf("expected not_same_owner, got %q", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryAddLink_SourceNotFound(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
res, _ := h.reg.handleAddLink(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "src_id": 9999.0, "dst_id": 8888.0, "relation_type": "refines",
|
||||
}))
|
||||
if !resultIsError(res) || !strings.Contains(resultText(t, res), "source_not_found") {
|
||||
t.Errorf("expected source_not_found, got %q", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
// --- memory_rewrite_core ---
|
||||
|
||||
func TestMemoryRewriteCore_HappyPath(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
res, _ := h.reg.handleRewriteCore(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "agent_name": "a-h1", "blob": "I am a-h1.",
|
||||
}))
|
||||
if resultIsError(res) {
|
||||
t.Fatalf("unexpected error: %s", resultText(t, res))
|
||||
}
|
||||
blob, _, ok, _ := h.core.Get(context.Background(), "1", "a-h1")
|
||||
if !ok || blob != "I am a-h1." {
|
||||
t.Errorf("blob not stored: ok=%v blob=%q", ok, blob)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryRewriteCore_TooLarge(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
big := strings.Repeat("x", 65)
|
||||
res, _ := h.reg.handleRewriteCore(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "agent_name": "a-h1", "blob": big,
|
||||
}))
|
||||
if !resultIsError(res) || !strings.Contains(resultText(t, res), "core_memory_too_large") {
|
||||
t.Errorf("expected core_memory_too_large, got %q", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryRewriteCore_AgentNotSameOwner(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
res, _ := h.reg.handleRewriteCore(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "agent_name": "a-h2", "blob": "trying to overwrite",
|
||||
}))
|
||||
if !resultIsError(res) || !strings.Contains(resultText(t, res), "not_same_owner") {
|
||||
t.Errorf("expected not_same_owner, got %q", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
// --- memory_mark_duplicate ---
|
||||
|
||||
func TestMemoryMarkDuplicate_HappyPath(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
a := h.seedMessage(t, "a-h1", "fact A")
|
||||
b := h.seedMessage(t, "a-h1", "fact A shorter")
|
||||
res, _ := h.reg.handleMarkDuplicate(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "a_id": float64(a), "b_id": float64(b), "keep_id": float64(a),
|
||||
"reason": "shorter",
|
||||
}))
|
||||
if resultIsError(res) {
|
||||
t.Fatalf("unexpected error: %s", resultText(t, res))
|
||||
}
|
||||
// Audit row appended on job.
|
||||
job, _ := h.jobs.Get(context.Background(), h.jobID)
|
||||
if len(job.Actions) != 1 || job.Actions[0]["tool"] != "memory_mark_duplicate" {
|
||||
t.Errorf("expected one mark_duplicate action: %v", job.Actions)
|
||||
}
|
||||
}
|
||||
|
||||
// --- memory_supersede ---
|
||||
|
||||
func TestMemorySupersede_HappyPath(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
a := h.seedMessage(t, "a-h1", "old fact")
|
||||
b := h.seedMessage(t, "a-h1", "new fact")
|
||||
res, _ := h.reg.handleSupersede(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1", "a_id": float64(a), "b_id": float64(b), "reason": "newer",
|
||||
}))
|
||||
if resultIsError(res) {
|
||||
t.Fatalf("unexpected error: %s", resultText(t, res))
|
||||
}
|
||||
}
|
||||
|
||||
// --- memory_list_unprocessed ---
|
||||
|
||||
func TestMemoryListUnprocessed_ReturnsOwnerScopedMessages(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
h.seedMessage(t, "a-h1", "memory 1")
|
||||
h.seedMessage(t, "a-h1", "memory 2")
|
||||
h.seedMessage(t, "a-h2", "other owner's memory")
|
||||
|
||||
res, _ := h.reg.handleListUnprocessed(h.ctxWithToken(h.tokenStr), callRequest(map[string]any{
|
||||
"owner_id": "1",
|
||||
}))
|
||||
if resultIsError(res) {
|
||||
t.Fatalf("unexpected error: %s", resultText(t, res))
|
||||
}
|
||||
var body map[string]any
|
||||
_ = json.Unmarshal([]byte(resultText(t, res)), &body)
|
||||
mems, _ := body["memories"].([]any)
|
||||
if len(mems) != 2 {
|
||||
t.Errorf("expected 2 owner-scoped memories, got %d (full body: %v)", len(mems), body)
|
||||
}
|
||||
}
|
||||
|
||||
// --- memory_write_reflection ---
|
||||
|
||||
func TestMemoryWriteReflection_HappyPath(t *testing.T) {
|
||||
h := newMemToolHarness(t)
|
||||
a := h.seedMessage(t, "a-h1", "source 1")
|
||||
b := h.seedMessage(t, "a-h1", "source 2")
|
||||
|
||||
args := map[string]any{
|
||||
"owner_id": "1",
|
||||
"body": "Across these I notice...",
|
||||
"source_message_ids": "" + intCSV(a, b),
|
||||
}
|
||||
res, _ := h.reg.handleWriteReflection(h.ctxWithToken(h.tokenStr), callRequest(args))
|
||||
if resultIsError(res) {
|
||||
t.Fatalf("unexpected error: %s", resultText(t, res))
|
||||
}
|
||||
var body map[string]any
|
||||
_ = json.Unmarshal([]byte(resultText(t, res)), &body)
|
||||
if _, ok := body["memory_id"]; !ok {
|
||||
t.Errorf("expected memory_id: %v", body)
|
||||
}
|
||||
if v, _ := body["links_created"].(float64); int(v) != 2 {
|
||||
t.Errorf("expected links_created=2, got %v", body["links_created"])
|
||||
}
|
||||
}
|
||||
|
||||
func intCSV(ids ...int64) string {
|
||||
var b strings.Builder
|
||||
for i, id := range ids {
|
||||
if i > 0 {
|
||||
b.WriteString(",")
|
||||
}
|
||||
b.WriteString(itoa(id))
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func itoa(v int64) string {
|
||||
if v == 0 {
|
||||
return "0"
|
||||
}
|
||||
var buf [20]byte
|
||||
i := len(buf)
|
||||
for v > 0 {
|
||||
i--
|
||||
buf[i] = byte('0' + v%10)
|
||||
v /= 10
|
||||
}
|
||||
return string(buf[i:])
|
||||
}
|
||||
+58
-10
@@ -29,13 +29,13 @@ import (
|
||||
|
||||
// MCPServer wraps the mcp-go server with SynapBus services.
|
||||
type MCPServer struct {
|
||||
mcpServer *server.MCPServer
|
||||
httpServer *server.StreamableHTTPServer
|
||||
connMgr *ConnectionManager
|
||||
agentService *agents.AgentService
|
||||
hybridRegistrar *HybridToolRegistrar
|
||||
logger *slog.Logger
|
||||
console *console.Printer
|
||||
mcpServer *server.MCPServer
|
||||
httpServer *server.StreamableHTTPServer
|
||||
connMgr *ConnectionManager
|
||||
agentService *agents.AgentService
|
||||
hybridRegistrar *HybridToolRegistrar
|
||||
logger *slog.Logger
|
||||
console *console.Printer
|
||||
}
|
||||
|
||||
// NewMCPServer creates and configures a new MCP server with 5 hybrid tools registered.
|
||||
@@ -180,14 +180,26 @@ func NewMCPServer(
|
||||
// Create Streamable HTTP transport with context func for auth propagation
|
||||
httpServer := server.NewStreamableHTTPServer(mcpSrv,
|
||||
server.WithHTTPContextFunc(func(ctx context.Context, r *http.Request) context.Context {
|
||||
// Propagate agent identity from HTTP auth to MCP context
|
||||
// Propagate agent identity from HTTP auth to MCP context.
|
||||
// The full *agents.Agent is needed by injection middleware
|
||||
// (which uses owner_id to scope retrieval); the bare name is
|
||||
// kept for backward-compat with existing handlers that call
|
||||
// AgentNameFromContext directly.
|
||||
if agent, ok := agents.AgentFromContext(r.Context()); ok {
|
||||
ctx = agents.ContextWithAgent(ctx, agent)
|
||||
ctx = ContextWithAgentName(ctx, agent.Name)
|
||||
// Propagate owner ID for trace recording
|
||||
if ownerID, ok := trace.OwnerIDFromContext(r.Context()); ok {
|
||||
ctx = trace.ContextWithOwnerID(ctx, ownerID)
|
||||
}
|
||||
}
|
||||
// Dispatch-token bridge for feature 020 dream consolidation:
|
||||
// the runner sends X-Synapbus-Dispatch-Token on every MCP
|
||||
// request via the SDK's McpHttpServerConfig.headers field;
|
||||
// the memory_* tools read this from request context.
|
||||
if tok := r.Header.Get("X-Synapbus-Dispatch-Token"); tok != "" {
|
||||
ctx = WithDispatchToken(ctx, tok)
|
||||
}
|
||||
return ctx
|
||||
}),
|
||||
)
|
||||
@@ -206,9 +218,45 @@ func NewMCPServer(
|
||||
return s
|
||||
}
|
||||
|
||||
// SetInjection wires the proactive-memory injection middleware
|
||||
// (feature 020) into the hybrid tool registrar and re-registers the
|
||||
// hybrid tools so the wrappers take effect. Must be called after
|
||||
// NewMCPServer and before the server starts handling traffic.
|
||||
//
|
||||
// `coreProvider` is consulted only on session-start tools (currently
|
||||
// `my_status`). Pass nil when US2 has not yet been wired.
|
||||
func (s *MCPServer) SetInjection(cfg messaging.MemoryConfig, store *messaging.MemoryInjections, coreProvider search.CoreMemoryProvider) {
|
||||
if s.hybridRegistrar == nil || s.mcpServer == nil {
|
||||
return
|
||||
}
|
||||
s.hybridRegistrar.SetInjection(cfg, store, coreProvider)
|
||||
// Re-register the hybrid tools so the new InjectionEnabled / Core
|
||||
// wiring takes effect. AddTool overwrites by name (see mcp-go's
|
||||
// `MCPServer.AddTools`), so this swaps in the wrapped handlers
|
||||
// without leaking the original registrations.
|
||||
s.hybridRegistrar.RegisterAllOnServer(s.mcpServer)
|
||||
}
|
||||
|
||||
// SetDream wires the six memory_* MCP tools (feature 020 — US3).
|
||||
// Only registers when cfg.DreamEnabled is true; otherwise this is a
|
||||
// no-op so the tool surface remains identical to the pre-feature
|
||||
// shape. Must be called before the server starts serving traffic.
|
||||
func (s *MCPServer) SetDream(deps MemoryToolDeps) {
|
||||
if s == nil || s.mcpServer == nil {
|
||||
return
|
||||
}
|
||||
if !deps.MemConfig.DreamEnabled {
|
||||
s.logger.Info("dream tools not registered (SYNAPBUS_DREAM_ENABLED=0)")
|
||||
return
|
||||
}
|
||||
reg := NewMemoryToolRegistrar(deps)
|
||||
reg.RegisterAllOnServer(s.mcpServer)
|
||||
}
|
||||
|
||||
// WireGoalsTools registers the spec-018 tool surface (create_goal,
|
||||
// propose_task_tree, propose_agent, claim_task, request_resource,
|
||||
// list_resources) on the MCP server. Must be called after NewMCPServer.
|
||||
// propose_task_tree, claim_task, request_resource, list_resources,
|
||||
// complete_goal) on the MCP server. Must be called after NewMCPServer.
|
||||
// Note: propose_agent was removed in the internal-only mode change.
|
||||
func (s *MCPServer) WireGoalsTools(r *GoalsToolRegistrar) {
|
||||
if r == nil || s.mcpServer == nil {
|
||||
return
|
||||
|
||||
@@ -44,6 +44,26 @@ type HybridToolRegistrar struct {
|
||||
db *sql.DB
|
||||
queryExecutor *agentquery.Executor
|
||||
logger *slog.Logger
|
||||
|
||||
// Injection (feature 020). When injectionCfg.Cfg.InjectionEnabled
|
||||
// is false (the default), WrapInjection returns handlers unchanged
|
||||
// so existing tool response shapes are preserved bit-for-bit.
|
||||
injectionCfg messaging.MemoryConfig
|
||||
memoryInjections *messaging.MemoryInjections
|
||||
coreProvider search.CoreMemoryProvider
|
||||
}
|
||||
|
||||
// SetInjection wires the proactive-memory injection middleware into
|
||||
// every eligible MCP tool registered by RegisterAllOnServer. Call this
|
||||
// after NewHybridToolRegistrar and before RegisterAllOnServer.
|
||||
//
|
||||
// `coreProvider` is consulted only on session-start tools (currently
|
||||
// `my_status`). May be nil when US2 has not yet been wired — the
|
||||
// wrapper simply skips the core-memory hook in that case.
|
||||
func (h *HybridToolRegistrar) SetInjection(cfg messaging.MemoryConfig, store *messaging.MemoryInjections, coreProvider search.CoreMemoryProvider) {
|
||||
h.injectionCfg = cfg
|
||||
h.memoryInjections = store
|
||||
h.coreProvider = coreProvider
|
||||
}
|
||||
|
||||
// SetMarketplaceService attaches the marketplace service for the 5 new
|
||||
@@ -93,14 +113,104 @@ func NewHybridToolRegistrar(
|
||||
}
|
||||
|
||||
// RegisterAllOnServer registers all hybrid tools on an mcp-go MCPServer.
|
||||
//
|
||||
// When proactive-memory injection is enabled (via SetInjection), the
|
||||
// eligible tool handlers are wrapped with WrapInjection so their JSON
|
||||
// responses gain a `relevant_context` field per
|
||||
// `contracts/mcp-injection.md`. Tools NOT in the eligible set
|
||||
// (currently `get_replies`) are registered unchanged.
|
||||
func (h *HybridToolRegistrar) RegisterAllOnServer(s *server.MCPServer) {
|
||||
s.AddTool(h.myStatusTool(), h.handleMyStatus)
|
||||
s.AddTool(h.sendMessageTool(), h.handleSendMessage)
|
||||
s.AddTool(h.searchTool(), h.handleSearch)
|
||||
s.AddTool(h.executeTool(), h.handleExecute)
|
||||
s.AddTool(h.myStatusTool(), h.wrap("my_status", h.handleMyStatus, true))
|
||||
s.AddTool(h.sendMessageTool(), h.wrap("send_message", h.handleSendMessage, false))
|
||||
s.AddTool(h.searchTool(), h.wrap("search", h.handleSearch, false))
|
||||
s.AddTool(h.executeTool(), h.wrap("execute", h.handleExecute, false))
|
||||
s.AddTool(h.getRepliesTool(), h.handleGetReplies)
|
||||
|
||||
h.logger.Info("hybrid MCP tools registered", "count", 5)
|
||||
h.logger.Info("hybrid MCP tools registered",
|
||||
"count", 5,
|
||||
"injection_enabled", h.injectionCfg.InjectionEnabled,
|
||||
)
|
||||
}
|
||||
|
||||
// wrap applies the proactive-memory WrapInjection middleware to one
|
||||
// tool handler. When InjectionEnabled is false (default), wrap returns
|
||||
// the original handler unchanged. `includeCore` is true only for
|
||||
// session-start tools (my_status today).
|
||||
func (h *HybridToolRegistrar) wrap(toolName string, inner ToolHandler, includeCore bool) ToolHandler {
|
||||
if !h.injectionCfg.InjectionEnabled {
|
||||
return inner
|
||||
}
|
||||
cfg := WrapConfig{
|
||||
Cfg: h.injectionCfg,
|
||||
SearchSvc: h.searchService,
|
||||
Injections: h.memoryInjections,
|
||||
IncludeCore: includeCore,
|
||||
CoreProvider: h.coreProvider,
|
||||
QuerySource: querySourceFor(toolName),
|
||||
Logger: h.logger,
|
||||
}
|
||||
return WrapInjection(inner, toolName, cfg)
|
||||
}
|
||||
|
||||
// querySourceFor returns the QuerySource closure for the given tool.
|
||||
// The contract (`contracts/mcp-injection.md`) prescribes the retrieval
|
||||
// query per tool:
|
||||
//
|
||||
// - my_status: "<recent activity>" (fallback when nothing else)
|
||||
// - send_message: body of the sent message
|
||||
// - search: the user's query argument
|
||||
// - execute: stringified args (best-effort)
|
||||
//
|
||||
// Tools not registered as MCP tools here (`claim_messages`,
|
||||
// `read_inbox`, `read_channel`) are exercised via the `execute` bridge
|
||||
// and inherit the `execute` query source.
|
||||
func querySourceFor(toolName string) QuerySource {
|
||||
switch toolName {
|
||||
case "my_status":
|
||||
return func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string {
|
||||
// FR-009: when there's no explicit query, use recent owner
|
||||
// activity. The retrieval layer interprets the empty string
|
||||
// as "no query" and falls back to recency-ordered FTS.
|
||||
return ""
|
||||
}
|
||||
case "send_message":
|
||||
return func(_ context.Context, _ string, args map[string]any, _ map[string]any) string {
|
||||
if args == nil {
|
||||
return ""
|
||||
}
|
||||
if body, ok := args["body"].(string); ok {
|
||||
return body
|
||||
}
|
||||
return ""
|
||||
}
|
||||
case "search":
|
||||
return func(_ context.Context, _ string, args map[string]any, _ map[string]any) string {
|
||||
if args == nil {
|
||||
return ""
|
||||
}
|
||||
if q, ok := args["query"].(string); ok {
|
||||
return q
|
||||
}
|
||||
return ""
|
||||
}
|
||||
case "execute":
|
||||
return func(_ context.Context, _ string, args map[string]any, _ map[string]any) string {
|
||||
if args == nil {
|
||||
return ""
|
||||
}
|
||||
// Best-effort: use the `code` argument verbatim. It is the
|
||||
// only required input and reliably reflects what the agent
|
||||
// is about to do.
|
||||
if code, ok := args["code"].(string); ok {
|
||||
return code
|
||||
}
|
||||
return ""
|
||||
}
|
||||
default:
|
||||
return func(_ context.Context, _ string, _ map[string]any, _ map[string]any) string {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- Tool Definitions ---
|
||||
@@ -118,7 +228,7 @@ func (h *HybridToolRegistrar) sendMessageTool() mcplib.Tool {
|
||||
mcplib.WithString("channel", mcplib.Description("Channel name or numeric ID for channel messages")),
|
||||
mcplib.WithString("body", mcplib.Description("Message body text"), mcplib.Required()),
|
||||
mcplib.WithString("subject", mcplib.Description("Conversation subject (optional)")),
|
||||
mcplib.WithNumber("priority", mcplib.Description("Message priority (1-10, default 5)"), mcplib.Min(1), mcplib.Max(10)),
|
||||
mcplib.WithNumber("priority", mcplib.Description("Message priority (1-10, default 5)")),
|
||||
mcplib.WithString("metadata", mcplib.Description("JSON metadata object (optional)")),
|
||||
mcplib.WithNumber("reply_to", mcplib.Description("ID of the parent message to reply to. Creates a threaded reply. Always use reply_to when responding to a message that is itself a thread reply, to keep conversations organized.")),
|
||||
mcplib.WithString("attachments", mcplib.Description("Comma-separated list of attachment hashes to link to this message. Upload attachments first using the upload_attachment action via the execute tool.")),
|
||||
|
||||
@@ -0,0 +1,205 @@
|
||||
// Auto-link emitter for feature 020 — when a new memory-eligible
|
||||
// message is inserted, derive `mention`, `reply_to`, and
|
||||
// `channel_cooccurrence` links automatically and write them with
|
||||
// `created_by = "auto:<rule>"`. The dream-agent's `memory_add_link`
|
||||
// tool rejects these auto-types (see memory_links.go) so this is the
|
||||
// only path that creates them.
|
||||
//
|
||||
// Wiring: register an AutoLinkListener with
|
||||
// `MessagingService.AddMessageListener`. The listener fires after
|
||||
// every successful message insert. Failures are logged but never
|
||||
// propagated — auto-links are best-effort.
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
// mentionPattern matches `@agent-name` in the body. Hyphens and digits
|
||||
// allowed; case-insensitive; bounded by non-word characters or string
|
||||
// edges. Matches the existing mentions.go convention.
|
||||
var autoMentionPattern = regexp.MustCompile(`@([A-Za-z][A-Za-z0-9_\-]{1,63})`)
|
||||
|
||||
// AutoLinkListener implements MessageListener and writes the three
|
||||
// auto-link types per OnMessageSent invocation.
|
||||
type AutoLinkListener struct {
|
||||
db *sql.DB
|
||||
links *LinkStore
|
||||
logger *slog.Logger
|
||||
}
|
||||
|
||||
// NewAutoLinkListener returns a listener over db + links.
|
||||
func NewAutoLinkListener(db *sql.DB, links *LinkStore) *AutoLinkListener {
|
||||
return &AutoLinkListener{
|
||||
db: db,
|
||||
links: links,
|
||||
logger: slog.Default().With("component", "auto-links"),
|
||||
}
|
||||
}
|
||||
|
||||
// OnMessageSent implements MessageListener. Runs the three rules in
|
||||
// order. Each rule independently best-effort.
|
||||
func (l *AutoLinkListener) OnMessageSent(ctx context.Context, msg *Message) {
|
||||
if l == nil || l.db == nil || l.links == nil || msg == nil || msg.ID == 0 {
|
||||
return
|
||||
}
|
||||
// Only run for memory-channel messages — auto-links on every DM
|
||||
// would pollute the link table.
|
||||
if msg.ChannelID == nil {
|
||||
return
|
||||
}
|
||||
channelName, err := channelNameByID(ctx, l.db, *msg.ChannelID)
|
||||
if err != nil || !matchesMemoryChannelName(channelName) {
|
||||
return
|
||||
}
|
||||
ownerID := resolveOwnerString(ctx, l.db, msg.FromAgent)
|
||||
if ownerID == "" {
|
||||
return
|
||||
}
|
||||
|
||||
// 1. reply_to: simple metadata or msg.ReplyTo column.
|
||||
if msg.ReplyTo != nil && *msg.ReplyTo != 0 {
|
||||
if _, err := l.links.Add(ctx, msg.ID, *msg.ReplyTo, "reply_to", ownerID, "auto:reply_to", nil); err != nil {
|
||||
l.logger.Debug("auto reply_to failed", "msg", msg.ID, "error", err)
|
||||
}
|
||||
} else if reply := extractReplyToFromMetadata(msg.Metadata); reply != 0 {
|
||||
if _, err := l.links.Add(ctx, msg.ID, reply, "reply_to", ownerID, "auto:reply_to", nil); err != nil {
|
||||
l.logger.Debug("auto reply_to (meta) failed", "msg", msg.ID, "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 2. mention: @agent-name → latest message from that agent in the
|
||||
// same channel.
|
||||
for _, m := range autoMentionPattern.FindAllStringSubmatch(msg.Body, -1) {
|
||||
if len(m) < 2 {
|
||||
continue
|
||||
}
|
||||
target := m[1]
|
||||
if target == msg.FromAgent {
|
||||
continue
|
||||
}
|
||||
dst := mostRecentMessageFromAgentInChannel(ctx, l.db, *msg.ChannelID, target, msg.ID)
|
||||
if dst == 0 {
|
||||
continue
|
||||
}
|
||||
if _, err := l.links.Add(ctx, msg.ID, dst, "mention", ownerID, "auto:mention", nil); err != nil {
|
||||
l.logger.Debug("auto mention failed", "msg", msg.ID, "to", target, "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 3. channel_cooccurrence: previous-most-recent message in the
|
||||
// same channel.
|
||||
prev := mostRecentMessageInChannel(ctx, l.db, *msg.ChannelID, msg.ID)
|
||||
if prev != 0 {
|
||||
if _, err := l.links.Add(ctx, msg.ID, prev, "channel_cooccurrence", ownerID, "auto:channel_cooccurrence", nil); err != nil {
|
||||
l.logger.Debug("auto cooccurrence failed", "msg", msg.ID, "error", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// channelNameByID returns the name of the channel with the given id.
|
||||
func channelNameByID(ctx context.Context, db *sql.DB, id int64) (string, error) {
|
||||
var name string
|
||||
err := db.QueryRowContext(ctx, `SELECT name FROM channels WHERE id = ?`, id).Scan(&name)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
|
||||
// resolveOwnerString returns the agents.owner_id of fromAgent as a
|
||||
// string, or "" on any error.
|
||||
func resolveOwnerString(ctx context.Context, db *sql.DB, fromAgent string) string {
|
||||
var ownerID int64
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT owner_id FROM agents WHERE name = ?`, fromAgent,
|
||||
).Scan(&ownerID)
|
||||
if err != nil || ownerID == 0 {
|
||||
return ""
|
||||
}
|
||||
// Use the same formatting as agents.OwnerFor so cross-package
|
||||
// comparisons stay byte-for-byte consistent.
|
||||
return itoaInt64(ownerID)
|
||||
}
|
||||
|
||||
// itoaInt64 mirrors strconv.FormatInt(v,10) without importing strconv.
|
||||
func itoaInt64(v int64) string {
|
||||
if v == 0 {
|
||||
return "0"
|
||||
}
|
||||
var buf [20]byte
|
||||
i := len(buf)
|
||||
neg := v < 0
|
||||
if neg {
|
||||
v = -v
|
||||
}
|
||||
for v > 0 {
|
||||
i--
|
||||
buf[i] = byte('0' + v%10)
|
||||
v /= 10
|
||||
}
|
||||
if neg {
|
||||
i--
|
||||
buf[i] = '-'
|
||||
}
|
||||
return string(buf[i:])
|
||||
}
|
||||
|
||||
// extractReplyToFromMetadata returns the reply_to_message_id field if
|
||||
// the metadata is a JSON object with that key.
|
||||
func extractReplyToFromMetadata(meta json.RawMessage) int64 {
|
||||
if len(meta) == 0 {
|
||||
return 0
|
||||
}
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal(meta, &m); err != nil {
|
||||
return 0
|
||||
}
|
||||
if v, ok := m["reply_to_message_id"]; ok {
|
||||
switch t := v.(type) {
|
||||
case float64:
|
||||
return int64(t)
|
||||
case int64:
|
||||
return t
|
||||
case int:
|
||||
return int64(t)
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func mostRecentMessageFromAgentInChannel(ctx context.Context, db *sql.DB, channelID int64, agent string, excludeID int64) int64 {
|
||||
var id int64
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT id FROM messages
|
||||
WHERE channel_id = ? AND from_agent = ? AND id != ?
|
||||
ORDER BY id DESC LIMIT 1`,
|
||||
channelID, agent, excludeID,
|
||||
).Scan(&id)
|
||||
if err != nil {
|
||||
if !errors.Is(err, sql.ErrNoRows) {
|
||||
// best-effort: caller logs at debug
|
||||
}
|
||||
return 0
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
func mostRecentMessageInChannel(ctx context.Context, db *sql.DB, channelID int64, excludeID int64) int64 {
|
||||
var id int64
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT id FROM messages
|
||||
WHERE channel_id = ? AND id < ?
|
||||
ORDER BY id DESC LIMIT 1`,
|
||||
channelID, excludeID,
|
||||
).Scan(&id)
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return id
|
||||
}
|
||||
@@ -0,0 +1,385 @@
|
||||
// Consolidation-jobs store for feature 020 — wraps the
|
||||
// `memory_consolidation_jobs` table (data-model.md). Each row is one
|
||||
// dream-worker dispatch: state machine `pending → dispatched → running
|
||||
// → {succeeded|partial|failed|expired}`. The partial-unique index
|
||||
// `idx_consolidation_in_flight(owner_id, job_type) WHERE status IN
|
||||
// ('pending','dispatched','running')` guarantees at most one in-flight
|
||||
// row per (owner, job_type). Create() surfaces conflicts as
|
||||
// ErrJobAlreadyInFlight so the worker can skip the dispatch cleanly.
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Memory consolidation job types.
|
||||
const (
|
||||
JobTypeReflection = "reflection"
|
||||
JobTypeCoreRewrite = "core_rewrite"
|
||||
JobTypeDedupContradiction = "dedup_contradiction"
|
||||
JobTypeLinkGen = "link_gen"
|
||||
)
|
||||
|
||||
// Memory consolidation job statuses.
|
||||
const (
|
||||
JobStatusPending = "pending"
|
||||
JobStatusDispatched = "dispatched"
|
||||
JobStatusRunning = "running"
|
||||
JobStatusSucceeded = "succeeded"
|
||||
JobStatusPartial = "partial"
|
||||
JobStatusFailed = "failed"
|
||||
JobStatusExpired = "expired"
|
||||
// JobStatusCircuitBroken is recorded when the ConsolidatorWorker
|
||||
// declines to dispatch because the per-(owner, day) usage gate
|
||||
// fired. The job row is created so the audit log shows the
|
||||
// attempt, then Complete()d immediately with this status.
|
||||
JobStatusCircuitBroken = "circuit_broken"
|
||||
)
|
||||
|
||||
// ErrJobAlreadyInFlight is returned by JobsStore.Create when the
|
||||
// partial-unique index trips because another job of the same type is
|
||||
// already pending / dispatched / running for the same owner.
|
||||
var ErrJobAlreadyInFlight = errors.New("consolidation job already in flight for (owner, job_type)")
|
||||
|
||||
// Job is one row in `memory_consolidation_jobs`.
|
||||
type Job struct {
|
||||
ID int64 `json:"id"`
|
||||
OwnerID string `json:"owner_id"`
|
||||
JobType string `json:"job_type"`
|
||||
Status string `json:"status"`
|
||||
TriggerReason string `json:"trigger_reason"`
|
||||
DispatchToken string `json:"dispatch_token,omitempty"`
|
||||
HarnessRunID string `json:"harness_run_id,omitempty"`
|
||||
Actions []map[string]any `json:"actions"`
|
||||
Summary string `json:"summary,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
LeaseUntil *time.Time `json:"lease_until,omitempty"`
|
||||
StartedAt *time.Time `json:"started_at,omitempty"`
|
||||
FinishedAt *time.Time `json:"finished_at,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// JobsStore wraps the `memory_consolidation_jobs` table.
|
||||
type JobsStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
// NewJobsStore returns a store rooted at db.
|
||||
func NewJobsStore(db *sql.DB) *JobsStore {
|
||||
return &JobsStore{db: db}
|
||||
}
|
||||
|
||||
// Create inserts a `pending` row for the (owner, jobType) pair on slot 0.
|
||||
// Kept for compatibility with single-threaded callers. Returns
|
||||
// ErrJobAlreadyInFlight if slot 0 is occupied.
|
||||
func (s *JobsStore) Create(ctx context.Context, ownerID, jobType, triggerReason string) (int64, error) {
|
||||
return s.CreateOnSlot(ctx, ownerID, jobType, triggerReason, 0)
|
||||
}
|
||||
|
||||
// CreateOnSlot inserts a `pending` row on a specific slot. Migration 030
|
||||
// extended the partial-unique index to include `slot`, so slots 0..N-1
|
||||
// can hold concurrent in-flight jobs of the same type per owner. The
|
||||
// worker / admin CLI uses this when fanning out a backlog drain.
|
||||
func (s *JobsStore) CreateOnSlot(ctx context.Context, ownerID, jobType, triggerReason string, slot int) (int64, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return 0, fmt.Errorf("jobs store: nil store")
|
||||
}
|
||||
if ownerID == "" {
|
||||
return 0, fmt.Errorf("jobs store: empty owner_id")
|
||||
}
|
||||
if jobType == "" {
|
||||
return 0, fmt.Errorf("jobs store: empty job_type")
|
||||
}
|
||||
if slot < 0 {
|
||||
return 0, fmt.Errorf("jobs store: negative slot")
|
||||
}
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO memory_consolidation_jobs
|
||||
(owner_id, job_type, status, trigger_reason, slot)
|
||||
VALUES (?, ?, 'pending', ?, ?)`,
|
||||
ownerID, jobType, triggerReason, slot,
|
||||
)
|
||||
if err != nil {
|
||||
if isUniqueConstraint(err) {
|
||||
return 0, ErrJobAlreadyInFlight
|
||||
}
|
||||
return 0, fmt.Errorf("jobs store: insert: %w", err)
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// CreateNextAvailableSlot tries slots 0..maxSlots-1 in order and returns
|
||||
// the (jobID, slot) of the first one that wasn't already in flight. Used
|
||||
// when the worker / CLI wants to fan out N parallel jobs per cycle.
|
||||
// Returns ErrJobAlreadyInFlight if all slots are busy.
|
||||
func (s *JobsStore) CreateNextAvailableSlot(ctx context.Context, ownerID, jobType, triggerReason string, maxSlots int) (int64, int, error) {
|
||||
if maxSlots <= 0 {
|
||||
maxSlots = 1
|
||||
}
|
||||
for slot := 0; slot < maxSlots; slot++ {
|
||||
id, err := s.CreateOnSlot(ctx, ownerID, jobType, triggerReason, slot)
|
||||
if err == nil {
|
||||
return id, slot, nil
|
||||
}
|
||||
if !errors.Is(err, ErrJobAlreadyInFlight) {
|
||||
return 0, 0, err
|
||||
}
|
||||
}
|
||||
return 0, 0, ErrJobAlreadyInFlight
|
||||
}
|
||||
|
||||
// Dispatch flips a pending row to `dispatched` and stamps the harness
|
||||
// run id and dispatch token. Returns an error if the row is not in
|
||||
// `pending` state.
|
||||
func (s *JobsStore) Dispatch(ctx context.Context, jobID int64, harnessRunID, token string) error {
|
||||
if s == nil || s.db == nil {
|
||||
return fmt.Errorf("jobs store: nil store")
|
||||
}
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`UPDATE memory_consolidation_jobs
|
||||
SET status = 'dispatched',
|
||||
harness_run_id = ?,
|
||||
dispatch_token = ?
|
||||
WHERE id = ? AND status = 'pending'`,
|
||||
harnessRunID, token, jobID,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("jobs store: dispatch: %w", err)
|
||||
}
|
||||
rows, _ := res.RowsAffected()
|
||||
if rows == 0 {
|
||||
return fmt.Errorf("jobs store: dispatch: job %d not in pending state", jobID)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Lease flips `dispatched` → `running` and sets lease_until + started_at.
|
||||
// Called by the worker once the harness has confirmed the run started.
|
||||
func (s *JobsStore) Lease(ctx context.Context, jobID int64, until time.Time) error {
|
||||
if s == nil || s.db == nil {
|
||||
return fmt.Errorf("jobs store: nil store")
|
||||
}
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`UPDATE memory_consolidation_jobs
|
||||
SET status = 'running',
|
||||
lease_until = ?,
|
||||
started_at = CURRENT_TIMESTAMP
|
||||
WHERE id = ? AND status IN ('dispatched', 'pending')`,
|
||||
until.UTC(), jobID,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("jobs store: lease: %w", err)
|
||||
}
|
||||
rows, _ := res.RowsAffected()
|
||||
if rows == 0 {
|
||||
return fmt.Errorf("jobs store: lease: job %d not in dispatched/pending state", jobID)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// AppendAction reads the current `actions` JSON array, appends `action`,
|
||||
// and writes it back. Wrapped in a single transaction so concurrent
|
||||
// MCP tool calls within one job serialize cleanly.
|
||||
func (s *JobsStore) AppendAction(ctx context.Context, jobID int64, action map[string]any) error {
|
||||
if s == nil || s.db == nil {
|
||||
return fmt.Errorf("jobs store: nil store")
|
||||
}
|
||||
if action == nil {
|
||||
return fmt.Errorf("jobs store: nil action")
|
||||
}
|
||||
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("jobs store: begin tx: %w", err)
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
var current string
|
||||
if err := tx.QueryRowContext(ctx,
|
||||
`SELECT actions FROM memory_consolidation_jobs WHERE id = ?`, jobID,
|
||||
).Scan(¤t); err != nil {
|
||||
return fmt.Errorf("jobs store: read actions: %w", err)
|
||||
}
|
||||
|
||||
var arr []map[string]any
|
||||
if current == "" || current == "null" {
|
||||
arr = []map[string]any{}
|
||||
} else if err := json.Unmarshal([]byte(current), &arr); err != nil {
|
||||
// Corrupt JSON — start fresh rather than fail forever.
|
||||
arr = []map[string]any{}
|
||||
}
|
||||
arr = append(arr, action)
|
||||
b, err := json.Marshal(arr)
|
||||
if err != nil {
|
||||
return fmt.Errorf("jobs store: marshal actions: %w", err)
|
||||
}
|
||||
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`UPDATE memory_consolidation_jobs SET actions = ? WHERE id = ?`,
|
||||
string(b), jobID,
|
||||
); err != nil {
|
||||
return fmt.Errorf("jobs store: write actions: %w", err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("jobs store: commit: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Complete sets `status`, `summary`, `error`, `finished_at` and clears
|
||||
// `lease_until`. Idempotent — repeated calls keep the first finished_at.
|
||||
func (s *JobsStore) Complete(ctx context.Context, jobID int64, status, summary, errMsg string) error {
|
||||
if s == nil || s.db == nil {
|
||||
return fmt.Errorf("jobs store: nil store")
|
||||
}
|
||||
switch status {
|
||||
case JobStatusSucceeded, JobStatusPartial, JobStatusFailed, JobStatusExpired, JobStatusCircuitBroken:
|
||||
// ok
|
||||
default:
|
||||
return fmt.Errorf("jobs store: invalid completion status %q", status)
|
||||
}
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`UPDATE memory_consolidation_jobs
|
||||
SET status = ?,
|
||||
summary = COALESCE(NULLIF(?, ''), summary),
|
||||
error = COALESCE(NULLIF(?, ''), error),
|
||||
finished_at = COALESCE(finished_at, CURRENT_TIMESTAMP),
|
||||
lease_until = NULL
|
||||
WHERE id = ?`,
|
||||
status, summary, errMsg, jobID,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("jobs store: complete: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Get returns the row for jobID, or (nil, sql.ErrNoRows).
|
||||
func (s *JobsStore) Get(ctx context.Context, jobID int64) (*Job, error) {
|
||||
row := s.db.QueryRowContext(ctx,
|
||||
`SELECT id, owner_id, job_type, status, trigger_reason,
|
||||
COALESCE(dispatch_token, ''),
|
||||
COALESCE(harness_run_id, ''),
|
||||
actions,
|
||||
COALESCE(summary, ''),
|
||||
COALESCE(error, ''),
|
||||
lease_until, started_at, finished_at, created_at
|
||||
FROM memory_consolidation_jobs WHERE id = ?`, jobID,
|
||||
)
|
||||
return scanJob(row.Scan)
|
||||
}
|
||||
|
||||
// ListRecent returns the most-recent jobs for the given owner.
|
||||
func (s *JobsStore) ListRecent(ctx context.Context, ownerID string, limit int) ([]Job, error) {
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT id, owner_id, job_type, status, trigger_reason,
|
||||
COALESCE(dispatch_token, ''),
|
||||
COALESCE(harness_run_id, ''),
|
||||
actions,
|
||||
COALESCE(summary, ''),
|
||||
COALESCE(error, ''),
|
||||
lease_until, started_at, finished_at, created_at
|
||||
FROM memory_consolidation_jobs
|
||||
WHERE owner_id = ?
|
||||
ORDER BY created_at DESC, id DESC
|
||||
LIMIT ?`, ownerID, limit,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("jobs store: list recent: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []Job
|
||||
for rows.Next() {
|
||||
j, err := scanJob(rows.Scan)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, *j)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("jobs store: iterate: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// ActiveJob returns the in-flight job for (owner, jobType), if any. nil
|
||||
// when no row matches.
|
||||
func (s *JobsStore) ActiveJob(ctx context.Context, ownerID, jobType string) (*Job, error) {
|
||||
row := s.db.QueryRowContext(ctx,
|
||||
`SELECT id, owner_id, job_type, status, trigger_reason,
|
||||
COALESCE(dispatch_token, ''),
|
||||
COALESCE(harness_run_id, ''),
|
||||
actions,
|
||||
COALESCE(summary, ''),
|
||||
COALESCE(error, ''),
|
||||
lease_until, started_at, finished_at, created_at
|
||||
FROM memory_consolidation_jobs
|
||||
WHERE owner_id = ? AND job_type = ?
|
||||
AND status IN ('pending', 'dispatched', 'running')
|
||||
ORDER BY id DESC LIMIT 1`,
|
||||
ownerID, jobType,
|
||||
)
|
||||
j, err := scanJob(row.Scan)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, nil
|
||||
}
|
||||
return j, err
|
||||
}
|
||||
|
||||
type scanFn func(dest ...any) error
|
||||
|
||||
func scanJob(scan scanFn) (*Job, error) {
|
||||
var (
|
||||
j Job
|
||||
actions string
|
||||
leaseUntil sql.NullTime
|
||||
startedAt sql.NullTime
|
||||
finishedAt sql.NullTime
|
||||
)
|
||||
err := scan(
|
||||
&j.ID, &j.OwnerID, &j.JobType, &j.Status, &j.TriggerReason,
|
||||
&j.DispatchToken, &j.HarnessRunID, &actions,
|
||||
&j.Summary, &j.Error,
|
||||
&leaseUntil, &startedAt, &finishedAt, &j.CreatedAt,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if actions != "" && actions != "null" {
|
||||
_ = json.Unmarshal([]byte(actions), &j.Actions)
|
||||
}
|
||||
if leaseUntil.Valid {
|
||||
t := leaseUntil.Time
|
||||
j.LeaseUntil = &t
|
||||
}
|
||||
if startedAt.Valid {
|
||||
t := startedAt.Time
|
||||
j.StartedAt = &t
|
||||
}
|
||||
if finishedAt.Valid {
|
||||
t := finishedAt.Time
|
||||
j.FinishedAt = &t
|
||||
}
|
||||
return &j, nil
|
||||
}
|
||||
|
||||
func isUniqueConstraint(err error) bool {
|
||||
if err == nil {
|
||||
return false
|
||||
}
|
||||
// modernc.org/sqlite returns errors whose string contains
|
||||
// "constraint failed: UNIQUE" or "SQLITE_CONSTRAINT_UNIQUE". Match
|
||||
// loosely so we don't depend on a specific build.
|
||||
msg := strings.ToUpper(err.Error())
|
||||
return strings.Contains(msg, "UNIQUE")
|
||||
}
|
||||
@@ -0,0 +1,698 @@
|
||||
// Dream-worker (consolidator) for feature 020 — periodically scans
|
||||
// memory channels per-owner, evaluates trigger watermarks and the
|
||||
// daily deep-pass schedule, and dispatches consolidation jobs to a
|
||||
// Claude Code agent via the harness.
|
||||
//
|
||||
// Importantly: the worker NEVER sends a system DM. Per
|
||||
// feedback_system_dm_no_trigger.md, system DMs would trigger reactive
|
||||
// runs which would cascade through the stalemate worker. The harness
|
||||
// dispatch path is the contractual non-DM route — see R1.
|
||||
//
|
||||
// To avoid an import cycle (harness imports messaging), the worker
|
||||
// accepts a minimal HarnessDispatcher interface that mirrors the
|
||||
// fragment of harness.Registry it needs. The cmd/synapbus wiring at
|
||||
// startup adapts harness.Registry to this interface.
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// DreamAgent is a minimal record passed to the harness dispatcher.
|
||||
// We can't import internal/agents here (cycle: agents → messaging
|
||||
// already) so the worker uses an interface-typed value for the agent
|
||||
// record and the cmd/synapbus adapter unboxes it. This keeps the
|
||||
// dependency direction (harness → messaging, agents → messaging)
|
||||
// intact.
|
||||
type DreamAgent interface {
|
||||
// AgentName returns the agent's stable SynapBus name.
|
||||
AgentName() string
|
||||
}
|
||||
|
||||
// DreamAgentNamed is a tiny convenience wrapper for tests and the
|
||||
// admin path that need a DreamAgent with just a name.
|
||||
type DreamAgentNamed struct{ Name string }
|
||||
|
||||
// AgentName implements DreamAgent.
|
||||
func (a DreamAgentNamed) AgentName() string { return a.Name }
|
||||
|
||||
// HarnessDispatcher is the minimal slice of harness.Registry the worker
|
||||
// needs. cmd/synapbus wires a real registry behind this interface.
|
||||
type HarnessDispatcher interface {
|
||||
Execute(ctx context.Context, agent DreamAgent, req *HarnessExecRequest) (*HarnessExecResult, error)
|
||||
}
|
||||
|
||||
// HarnessExecRequest mirrors harness.ExecRequest's Env-bearing fields.
|
||||
// We don't re-export the full struct to avoid pulling all of harness
|
||||
// into messaging. The adapter in cmd/synapbus translates 1:1.
|
||||
type HarnessExecRequest struct {
|
||||
RunID string
|
||||
AgentName string
|
||||
Agent DreamAgent
|
||||
Env map[string]string
|
||||
MaxWallClock time.Duration
|
||||
Body string // populated into ExecRequest.Message if non-empty
|
||||
}
|
||||
|
||||
// HarnessExecResult mirrors the fields the worker reads from
|
||||
// harness.ExecResult.
|
||||
type HarnessExecResult struct {
|
||||
ExitCode int
|
||||
Logs string
|
||||
TokensIn int64
|
||||
TokensOut int64
|
||||
}
|
||||
|
||||
// AgentLookup resolves an agent record by name. Implemented by
|
||||
// agents.AgentService — declared as an interface here to avoid a
|
||||
// hard dependency.
|
||||
type AgentLookup interface {
|
||||
GetAgent(ctx context.Context, name string) (DreamAgent, error)
|
||||
}
|
||||
|
||||
// OwnerLister returns the list of distinct owner_ids that have at
|
||||
// least one recent memory-channel message worth scanning.
|
||||
type OwnerLister func(ctx context.Context, db *sql.DB) ([]string, error)
|
||||
|
||||
// ConsolidatorWorker periodically evaluates per-owner triggers and
|
||||
// dispatches dream-agent runs.
|
||||
type ConsolidatorWorker struct {
|
||||
db *sql.DB
|
||||
jobs *JobsStore
|
||||
tokens *DispatchTokenStore
|
||||
harness HarnessDispatcher
|
||||
agentLook AgentLookup
|
||||
cfg MemoryConfig
|
||||
logger *slog.Logger
|
||||
|
||||
// usage + gate provide the per-(owner, day) circuit breaker.
|
||||
// Optional: nil → all dispatches allowed. Wired by SetUsageGate.
|
||||
usage *DreamUsageStore
|
||||
gate *UsageGate
|
||||
|
||||
// Optional owner enumerator. Defaults to a query over `agents`.
|
||||
ownerLister OwnerLister
|
||||
|
||||
// Optional injection cleanup hook. When non-nil, the worker calls
|
||||
// it once per hour. Leave nil if the stalemate worker is already
|
||||
// handling injection cleanup (the default in main.go).
|
||||
injections *MemoryInjections
|
||||
|
||||
// sem caps simultaneous Execute calls.
|
||||
sem chan struct{}
|
||||
|
||||
done chan struct{}
|
||||
wg sync.WaitGroup
|
||||
}
|
||||
|
||||
// NewConsolidatorWorker builds a worker. Required: db, jobs, tokens,
|
||||
// harness, agentLook. The semaphore is sized from cfg.DreamMaxConcurrent.
|
||||
func NewConsolidatorWorker(
|
||||
db *sql.DB,
|
||||
jobs *JobsStore,
|
||||
tokens *DispatchTokenStore,
|
||||
harness HarnessDispatcher,
|
||||
agentLook AgentLookup,
|
||||
cfg MemoryConfig,
|
||||
) *ConsolidatorWorker {
|
||||
if cfg.DreamMaxConcurrent <= 0 {
|
||||
cfg.DreamMaxConcurrent = 1
|
||||
}
|
||||
return &ConsolidatorWorker{
|
||||
db: db,
|
||||
jobs: jobs,
|
||||
tokens: tokens,
|
||||
harness: harness,
|
||||
agentLook: agentLook,
|
||||
cfg: cfg,
|
||||
logger: slog.Default().With("component", "consolidator-worker"),
|
||||
sem: make(chan struct{}, cfg.DreamMaxConcurrent),
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
}
|
||||
|
||||
// SetInjectionCleanup registers a memory_injections store the worker
|
||||
// will Cleanup hourly. Pass nil to disable. The default main.go wiring
|
||||
// leaves the stalemate worker handling injection cleanup and skips
|
||||
// this.
|
||||
func (w *ConsolidatorWorker) SetInjectionCleanup(store *MemoryInjections) {
|
||||
w.injections = store
|
||||
}
|
||||
|
||||
// SetOwnerLister overrides the default owner enumerator (handy in
|
||||
// tests).
|
||||
func (w *ConsolidatorWorker) SetOwnerLister(fn OwnerLister) {
|
||||
w.ownerLister = fn
|
||||
}
|
||||
|
||||
// SetUsageGate wires the per-(owner, day) circuit breaker. When set,
|
||||
// the worker calls gate.Allow() before each dispatch and records a
|
||||
// `circuit_broken` job + skips Execute when the gate denies. Token
|
||||
// usage from successful runs is fed back into store so the next gate
|
||||
// evaluation reflects today's spend. Pass (nil, nil) to disable.
|
||||
func (w *ConsolidatorWorker) SetUsageGate(store *DreamUsageStore, gate *UsageGate) {
|
||||
w.usage = store
|
||||
w.gate = gate
|
||||
}
|
||||
|
||||
// Start launches the ticker goroutine.
|
||||
func (w *ConsolidatorWorker) Start() {
|
||||
w.wg.Add(1)
|
||||
go w.runLoop()
|
||||
}
|
||||
|
||||
// Stop halts the worker. Idempotent.
|
||||
func (w *ConsolidatorWorker) Stop() {
|
||||
select {
|
||||
case <-w.done:
|
||||
return
|
||||
default:
|
||||
}
|
||||
close(w.done)
|
||||
w.wg.Wait()
|
||||
}
|
||||
|
||||
func (w *ConsolidatorWorker) runLoop() {
|
||||
defer w.wg.Done()
|
||||
w.logger.Info("consolidator worker started",
|
||||
"interval", w.cfg.DreamInterval.String(),
|
||||
"watermark", w.cfg.DreamWatermark,
|
||||
"max_concurrent", w.cfg.DreamMaxConcurrent,
|
||||
"wallclock_budget", w.cfg.DreamWallclockBudget.String(),
|
||||
)
|
||||
|
||||
ticker := time.NewTicker(w.cfg.DreamInterval)
|
||||
defer ticker.Stop()
|
||||
|
||||
// Track when we last ran the daily deep pass. TODO: parse
|
||||
// cfg.DreamDeepCron instead of hardcoding 03:00 UTC daily — adding
|
||||
// robfig/cron would add a non-zero-CGO dependency and the spec
|
||||
// authorizes this stub.
|
||||
var lastDeepPass time.Time
|
||||
var lastHourlyCleanup time.Time
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
w.tick(ctx, &lastDeepPass, &lastHourlyCleanup)
|
||||
cancel()
|
||||
case <-w.done:
|
||||
w.logger.Info("consolidator worker stopped")
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// tick performs one evaluation pass.
|
||||
func (w *ConsolidatorWorker) tick(ctx context.Context, lastDeepPass, lastHourlyCleanup *time.Time) {
|
||||
now := time.Now().UTC()
|
||||
|
||||
// Hourly: injection cleanup, only if explicitly registered AND
|
||||
// stalemate isn't already handling it (this is the safer default —
|
||||
// see SetInjectionCleanup docs).
|
||||
if w.injections != nil && now.Sub(*lastHourlyCleanup) >= time.Hour {
|
||||
if deleted, err := w.injections.Cleanup(ctx, 24*time.Hour); err == nil && deleted > 0 {
|
||||
w.logger.Info("memory_injections cleanup", "deleted", deleted)
|
||||
}
|
||||
*lastHourlyCleanup = now
|
||||
}
|
||||
|
||||
// Per-owner trigger evaluation.
|
||||
owners, err := w.listOwners(ctx)
|
||||
if err != nil {
|
||||
w.logger.Warn("list owners failed", "error", err)
|
||||
return
|
||||
}
|
||||
deepPassDue := isDeepPassDue(now, *lastDeepPass)
|
||||
for _, owner := range owners {
|
||||
// Watermark triggers: reflection + link_gen + dedup_contradiction.
|
||||
if count, err := w.unprocessedCount(ctx, owner); err == nil && count >= w.cfg.DreamWatermark {
|
||||
w.tryDispatch(ctx, owner, JobTypeReflection, fmt.Sprintf("watermark:%d", count))
|
||||
w.tryDispatch(ctx, owner, JobTypeLinkGen, fmt.Sprintf("watermark:%d", count))
|
||||
w.tryDispatch(ctx, owner, JobTypeDedupContradiction, fmt.Sprintf("watermark:%d", count))
|
||||
}
|
||||
// Daily deep pass: sleep_time_rewrite (mapped to core_rewrite).
|
||||
// Skip when no agent owned by `owner` has produced any message
|
||||
// in the configured recent-window — there's nothing to refresh
|
||||
// the core blob from.
|
||||
if deepPassDue {
|
||||
window := w.cfg.DreamRecentWindow
|
||||
if window <= 0 {
|
||||
window = 14 * 24 * time.Hour
|
||||
}
|
||||
since := time.Now().UTC().Add(-window)
|
||||
if ownerActiveSince(ctx, w.db, owner, since) {
|
||||
w.tryDispatch(ctx, owner, JobTypeCoreRewrite, "cron:nightly")
|
||||
} else {
|
||||
w.logger.Debug("core_rewrite skipped — owner has no recent activity",
|
||||
"owner_id", owner, "since", since,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
if deepPassDue {
|
||||
*lastDeepPass = now
|
||||
}
|
||||
}
|
||||
|
||||
// isDeepPassDue returns true when "now" is past 03:00 UTC of the same
|
||||
// day AND lastDeepPass was earlier than that 03:00 mark. Hardcoded
|
||||
// 03:00 UTC per the cron-stub TODO above.
|
||||
func isDeepPassDue(now, last time.Time) bool {
|
||||
threeAM := time.Date(now.Year(), now.Month(), now.Day(), 3, 0, 0, 0, time.UTC)
|
||||
if now.Before(threeAM) {
|
||||
return false
|
||||
}
|
||||
return last.Before(threeAM)
|
||||
}
|
||||
|
||||
// listOwners returns the distinct owner_ids with at least one agent.
|
||||
// Override via SetOwnerLister in tests.
|
||||
func (w *ConsolidatorWorker) listOwners(ctx context.Context) ([]string, error) {
|
||||
if w.ownerLister != nil {
|
||||
return w.ownerLister(ctx, w.db)
|
||||
}
|
||||
rows, err := w.db.QueryContext(ctx,
|
||||
`SELECT DISTINCT CAST(owner_id AS TEXT) FROM agents WHERE owner_id > 0`,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []string
|
||||
for rows.Next() {
|
||||
var s string
|
||||
if err := rows.Scan(&s); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, s)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// unprocessedCount returns how many memory-channel messages exist for
|
||||
// this owner that are newer than the most-recent succeeded job's
|
||||
// finished_at. (Cheap approximation of "haven't been seen by the dream
|
||||
// agent yet"; the worker errs toward over-dispatching, the
|
||||
// partial-unique index gates duplicates anyway.)
|
||||
func (w *ConsolidatorWorker) unprocessedCount(ctx context.Context, ownerID string) (int, error) {
|
||||
channels, err := MemoryChannelIDs(ctx, w.db)
|
||||
if err != nil || len(channels) == 0 {
|
||||
return 0, err
|
||||
}
|
||||
placeholders := ""
|
||||
args := []any{}
|
||||
for i, id := range channels {
|
||||
if i > 0 {
|
||||
placeholders += ","
|
||||
}
|
||||
placeholders += "?"
|
||||
args = append(args, id)
|
||||
}
|
||||
// Window: messages newer than the latest completed reflection job
|
||||
// for this owner.
|
||||
var lastFinished sql.NullTime
|
||||
_ = w.db.QueryRowContext(ctx,
|
||||
`SELECT MAX(finished_at) FROM memory_consolidation_jobs
|
||||
WHERE owner_id = ? AND job_type = ? AND status IN ('succeeded','partial')`,
|
||||
ownerID, JobTypeReflection,
|
||||
).Scan(&lastFinished)
|
||||
|
||||
since := time.Time{}
|
||||
if lastFinished.Valid {
|
||||
since = lastFinished.Time
|
||||
}
|
||||
// Cap "since" at the configured 14d (default) recency window so we
|
||||
// never sweep historical pool — the worker only consolidates
|
||||
// recent activity per the T3 contract.
|
||||
window := w.cfg.DreamRecentWindow
|
||||
if window <= 0 {
|
||||
window = 14 * 24 * time.Hour
|
||||
}
|
||||
windowStart := time.Now().UTC().Add(-window)
|
||||
if since.Before(windowStart) {
|
||||
since = windowStart
|
||||
}
|
||||
args = append(args, ownerID, since)
|
||||
q := `SELECT COUNT(*)
|
||||
FROM messages m JOIN agents a ON m.from_agent = a.name
|
||||
WHERE m.channel_id IN (` + placeholders + `)
|
||||
AND CAST(a.owner_id AS TEXT) = ?
|
||||
AND m.created_at > ?`
|
||||
var count int
|
||||
err = w.db.QueryRowContext(ctx, q, args...).Scan(&count)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return count, nil
|
||||
}
|
||||
|
||||
// ForceRun bypasses watermark/cron triggers and dispatches one job
|
||||
// for (ownerID, jobType) immediately. Used by the admin CLI
|
||||
// `synapbus memory dream-run` command. Returns the created job_id (or
|
||||
// the existing in-flight one). Uses slot 0 — for fan-out, see ForceRunN.
|
||||
//
|
||||
// The circuit breaker still applies — admins who want to override
|
||||
// must clear today's usage row directly. Manual override on top of a
|
||||
// blown budget defeats the safety net.
|
||||
func (w *ConsolidatorWorker) ForceRun(ctx context.Context, ownerID, jobType string) (int64, error) {
|
||||
ids, err := w.ForceRunN(ctx, ownerID, jobType, 1)
|
||||
if err != nil || len(ids) == 0 {
|
||||
return 0, err
|
||||
}
|
||||
return ids[0], nil
|
||||
}
|
||||
|
||||
// ForceRunN dispatches up to N parallel jobs of the same type for one
|
||||
// owner. core_rewrite ignores N and dispatches at most one (per-agent
|
||||
// blob is wholesale-replace; concurrent rewrites would race).
|
||||
func (w *ConsolidatorWorker) ForceRunN(ctx context.Context, ownerID, jobType string, parallel int) ([]int64, error) {
|
||||
if parallel <= 0 {
|
||||
parallel = 1
|
||||
}
|
||||
if jobType == JobTypeCoreRewrite {
|
||||
parallel = 1
|
||||
}
|
||||
if w.gate != nil {
|
||||
allowed, reason, _ := w.gate.Allow(ctx, ownerID)
|
||||
if !allowed {
|
||||
jobID, cerr := w.jobs.Create(ctx, ownerID, jobType, "circuit_broken:"+reason)
|
||||
if cerr == nil {
|
||||
_ = w.jobs.Complete(ctx, jobID, JobStatusCircuitBroken, "circuit_broken: "+reason, "")
|
||||
if w.usage != nil {
|
||||
_ = w.usage.RecordCompletion(ctx, ownerID, 0, 0, JobStatusCircuitBroken)
|
||||
}
|
||||
recordCircuitBrokenMetric(ownerID, jobType, reason)
|
||||
recordJobMetric(ownerID, jobType, JobStatusCircuitBroken)
|
||||
return []int64{jobID}, fmt.Errorf("circuit broken: %s", reason)
|
||||
}
|
||||
return nil, fmt.Errorf("circuit broken: %s", reason)
|
||||
}
|
||||
}
|
||||
out := make([]int64, 0, parallel)
|
||||
for slot := 0; slot < parallel; slot++ {
|
||||
jobID, err := w.jobs.CreateOnSlot(ctx, ownerID, jobType, "manual:"+ownerID, slot)
|
||||
if err != nil {
|
||||
if errors.Is(err, ErrJobAlreadyInFlight) {
|
||||
// Slot already busy. If this is the first slot, fall back
|
||||
// to returning the existing in-flight job (preserves
|
||||
// historical ForceRun semantics).
|
||||
if len(out) == 0 && slot == 0 {
|
||||
if active, _ := w.jobs.ActiveJob(ctx, ownerID, jobType); active != nil {
|
||||
return []int64{active.ID}, nil
|
||||
}
|
||||
}
|
||||
w.logger.Debug("slot busy; skipping", "owner_id", ownerID, "job_type", jobType, "slot", slot)
|
||||
continue
|
||||
}
|
||||
return out, err
|
||||
}
|
||||
if err := w.launchOne(ctx, ownerID, jobType, jobID); err != nil {
|
||||
return out, err
|
||||
}
|
||||
out = append(out, jobID)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// launchOne wires up the per-job tokens / harness dispatch for an
|
||||
// already-Created job row. Shared by ForceRunN and the worker's
|
||||
// internal tryDispatch path.
|
||||
//
|
||||
// jobs_started is only incremented after a successful Dispatch flip so
|
||||
// pre-dispatch failures (token issue, agent lookup, dispatch race) do
|
||||
// not consume daily-job-limit slots — they are surfaced through the
|
||||
// job row's status=failed and the jobs_failed counter via the caller
|
||||
// of Complete, not through the gate's jobs_started counter.
|
||||
func (w *ConsolidatorWorker) launchOne(ctx context.Context, ownerID, jobType string, jobID int64) error {
|
||||
tok, _, err := w.tokens.Issue(ctx, ownerID, jobID)
|
||||
if err != nil {
|
||||
_ = w.jobs.Complete(ctx, jobID, JobStatusFailed, "", "token issue: "+err.Error())
|
||||
return fmt.Errorf("issue token: %w", err)
|
||||
}
|
||||
agent, err := w.agentLook.GetAgent(ctx, w.cfg.DreamAgent)
|
||||
if err != nil || agent == nil {
|
||||
_ = w.jobs.Complete(ctx, jobID, JobStatusFailed, "", "dream agent not found")
|
||||
return fmt.Errorf("dream agent %q not found: %w", w.cfg.DreamAgent, err)
|
||||
}
|
||||
runID := uuid.NewString()
|
||||
if err := w.jobs.Dispatch(ctx, jobID, runID, tok); err != nil {
|
||||
_ = w.jobs.Complete(ctx, jobID, JobStatusFailed, "", "dispatch flip: "+err.Error())
|
||||
return fmt.Errorf("dispatch flip: %w", err)
|
||||
}
|
||||
if w.usage != nil {
|
||||
_ = w.usage.RecordStart(ctx, ownerID)
|
||||
}
|
||||
w.wg.Add(1)
|
||||
go func() {
|
||||
defer w.wg.Done()
|
||||
select {
|
||||
case w.sem <- struct{}{}:
|
||||
defer func() { <-w.sem }()
|
||||
case <-w.done:
|
||||
return
|
||||
}
|
||||
w.runJob(ownerID, jobID, jobType, tok, runID, agent)
|
||||
}()
|
||||
return nil
|
||||
}
|
||||
|
||||
// tryDispatch attempts to create+dispatch one job. Idempotent —
|
||||
// ErrJobAlreadyInFlight is logged at debug and skipped.
|
||||
func (w *ConsolidatorWorker) tryDispatch(ctx context.Context, ownerID, jobType, trigger string) {
|
||||
// Circuit breaker: skip when today's per-(owner) usage exceeds any
|
||||
// configured limit. We still create+complete a job row so the
|
||||
// audit log records the attempt; this also makes the
|
||||
// circuit_broken_total metric easy to chart.
|
||||
if w.gate != nil {
|
||||
allowed, reason, gerr := w.gate.Allow(ctx, ownerID)
|
||||
if gerr != nil {
|
||||
w.logger.Warn("usage gate check failed; failing open",
|
||||
"owner_id", ownerID, "job_type", jobType, "error", gerr,
|
||||
)
|
||||
} else if !allowed {
|
||||
w.logger.Warn("dream dispatch circuit broken",
|
||||
"owner_id", ownerID, "job_type", jobType, "reason", reason,
|
||||
)
|
||||
jobID, cerr := w.jobs.Create(ctx, ownerID, jobType, "circuit_broken:"+reason)
|
||||
if cerr != nil {
|
||||
if !errors.Is(cerr, ErrJobAlreadyInFlight) {
|
||||
w.logger.Warn("circuit-broken job create failed",
|
||||
"owner_id", ownerID, "job_type", jobType, "error", cerr,
|
||||
)
|
||||
}
|
||||
if w.usage != nil {
|
||||
_ = w.usage.RecordCompletion(ctx, ownerID, 0, 0, JobStatusCircuitBroken)
|
||||
}
|
||||
recordCircuitBrokenMetric(ownerID, jobType, reason)
|
||||
return
|
||||
}
|
||||
_ = w.jobs.Complete(ctx, jobID, JobStatusCircuitBroken, "circuit_broken: "+reason, "")
|
||||
if w.usage != nil {
|
||||
_ = w.usage.RecordCompletion(ctx, ownerID, 0, 0, JobStatusCircuitBroken)
|
||||
}
|
||||
recordCircuitBrokenMetric(ownerID, jobType, reason)
|
||||
recordJobMetric(ownerID, jobType, JobStatusCircuitBroken)
|
||||
return
|
||||
}
|
||||
}
|
||||
jobID, err := w.jobs.Create(ctx, ownerID, jobType, trigger)
|
||||
if err != nil {
|
||||
if errors.Is(err, ErrJobAlreadyInFlight) {
|
||||
w.logger.Debug("job already in flight; skipping",
|
||||
"owner_id", ownerID, "job_type", jobType,
|
||||
)
|
||||
return
|
||||
}
|
||||
w.logger.Warn("create job failed",
|
||||
"owner_id", ownerID, "job_type", jobType, "error", err,
|
||||
)
|
||||
return
|
||||
}
|
||||
tok, _, err := w.tokens.Issue(ctx, ownerID, jobID)
|
||||
if err != nil {
|
||||
w.logger.Warn("issue token failed", "job_id", jobID, "error", err)
|
||||
_ = w.jobs.Complete(ctx, jobID, JobStatusFailed, "", "token issue: "+err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
// Resolve the dream-agent record (e.g. claude-code) so the
|
||||
// harness can pick the right backend.
|
||||
agent, err := w.agentLook.GetAgent(ctx, w.cfg.DreamAgent)
|
||||
if err != nil || agent == nil {
|
||||
w.logger.Warn("dream agent not found",
|
||||
"agent", w.cfg.DreamAgent, "error", err,
|
||||
)
|
||||
_ = w.jobs.Complete(ctx, jobID, JobStatusFailed, "", "dream agent not found")
|
||||
return
|
||||
}
|
||||
|
||||
runID := uuid.NewString()
|
||||
if err := w.jobs.Dispatch(ctx, jobID, runID, tok); err != nil {
|
||||
w.logger.Warn("dispatch flip failed", "job_id", jobID, "error", err)
|
||||
_ = w.jobs.Complete(ctx, jobID, JobStatusFailed, "", "dispatch flip: "+err.Error())
|
||||
return
|
||||
}
|
||||
// Only count jobs_started after a successful Dispatch flip so
|
||||
// pre-dispatch failures (token issue, agent lookup, race) don't
|
||||
// burn daily-job-limit slots without actually running anything.
|
||||
if w.usage != nil {
|
||||
_ = w.usage.RecordStart(ctx, ownerID)
|
||||
}
|
||||
|
||||
w.wg.Add(1)
|
||||
go func() {
|
||||
defer w.wg.Done()
|
||||
select {
|
||||
case w.sem <- struct{}{}:
|
||||
defer func() { <-w.sem }()
|
||||
case <-w.done:
|
||||
return
|
||||
}
|
||||
w.runJob(ownerID, jobID, jobType, tok, runID, agent)
|
||||
}()
|
||||
}
|
||||
|
||||
// runJob calls the harness and updates the job status.
|
||||
func (w *ConsolidatorWorker) runJob(ownerID string, jobID int64, jobType, tok, runID string, agent DreamAgent) {
|
||||
wallclock := w.cfg.DreamWallclockBudget
|
||||
if wallclock <= 0 {
|
||||
wallclock = 10 * time.Minute
|
||||
}
|
||||
// The harness Execute is bounded by wallclock; the surrounding
|
||||
// ctx adds a tiny epsilon so the dispatcher loop sees the
|
||||
// timeout fire and produces a 'partial' status.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), wallclock)
|
||||
defer cancel()
|
||||
|
||||
// Lease the row so the deep-link Web UI can show it as 'running'.
|
||||
leaseUntil := time.Now().Add(wallclock).UTC()
|
||||
if err := w.jobs.Lease(ctx, jobID, leaseUntil); err != nil {
|
||||
w.logger.Warn("lease failed", "job_id", jobID, "error", err)
|
||||
}
|
||||
|
||||
prompt := PromptFor(jobType)
|
||||
req := &HarnessExecRequest{
|
||||
RunID: runID,
|
||||
AgentName: agent.AgentName(),
|
||||
Agent: agent,
|
||||
Body: prompt,
|
||||
Env: map[string]string{
|
||||
"SYNAPBUS_DISPATCH_TOKEN": tok,
|
||||
"SYNAPBUS_CONSOLIDATION_JOB_ID": fmt.Sprintf("%d", jobID),
|
||||
"SYNAPBUS_JOB_TYPE": jobType,
|
||||
"SYNAPBUS_OWNER_ID": ownerID,
|
||||
"SYNAPBUS_DREAM_PROMPT": prompt,
|
||||
},
|
||||
MaxWallClock: wallclock,
|
||||
}
|
||||
|
||||
w.logger.Info("dispatching dream job",
|
||||
"job_id", jobID,
|
||||
"job_type", jobType,
|
||||
"owner_id", ownerID,
|
||||
"agent", agent.AgentName(),
|
||||
"run_id", runID,
|
||||
)
|
||||
|
||||
start := time.Now()
|
||||
res, err := w.harness.Execute(ctx, agent, req)
|
||||
status, summary, errMsg := mapHarnessResult(res, err, ctx.Err())
|
||||
duration := time.Since(start)
|
||||
|
||||
if err := w.jobs.Complete(context.Background(), jobID, status, summary, errMsg); err != nil {
|
||||
w.logger.Warn("complete job failed", "job_id", jobID, "error", err)
|
||||
}
|
||||
// Token revoke is best-effort.
|
||||
_ = w.tokens.Revoke(context.Background(), tok)
|
||||
|
||||
// Per-(owner, day) usage accounting (T4): feed back tokens for the
|
||||
// circuit breaker. Failure here is non-fatal.
|
||||
if w.usage != nil {
|
||||
var tIn, tOut int64
|
||||
if res != nil {
|
||||
tIn, tOut = res.TokensIn, res.TokensOut
|
||||
}
|
||||
_ = w.usage.RecordCompletion(context.Background(), ownerID, tIn, tOut, status)
|
||||
}
|
||||
|
||||
recordJobMetric(ownerID, jobType, status)
|
||||
recordJobDurationMetric(ownerID, jobType, duration)
|
||||
if res != nil {
|
||||
recordTokensMetric(ownerID, res.TokensIn, res.TokensOut)
|
||||
}
|
||||
|
||||
w.logger.Info("dream job completed",
|
||||
"job_id", jobID, "status", status, "summary", summary,
|
||||
"duration", duration.String(),
|
||||
)
|
||||
}
|
||||
|
||||
// ownerActiveSince returns true when any agent owned by ownerID has
|
||||
// authored at least one message after `since`. Used by core_rewrite
|
||||
// dispatch to short-circuit when an owner's fleet has been quiet — the
|
||||
// nightly deep pass has nothing to refresh against and would waste
|
||||
// tokens.
|
||||
func ownerActiveSince(ctx context.Context, db *sql.DB, ownerID string, since time.Time) bool {
|
||||
if db == nil || ownerID == "" {
|
||||
return false
|
||||
}
|
||||
var one int
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT 1
|
||||
FROM messages m JOIN agents a ON m.from_agent = a.name
|
||||
WHERE CAST(a.owner_id AS TEXT) = ?
|
||||
AND m.created_at > ?
|
||||
LIMIT 1`,
|
||||
ownerID, since.UTC(),
|
||||
).Scan(&one)
|
||||
return err == nil && one == 1
|
||||
}
|
||||
|
||||
// agentActiveSince returns true when the named agent (owned by
|
||||
// ownerID) has authored at least one message after `since`. Reserved
|
||||
// for future per-agent gating; currently the dispatcher uses
|
||||
// ownerActiveSince above to gate the whole core_rewrite pass.
|
||||
func agentActiveSince(ctx context.Context, db *sql.DB, ownerID, agentName string, since time.Time) bool {
|
||||
if db == nil || ownerID == "" || agentName == "" {
|
||||
return false
|
||||
}
|
||||
var one int
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT 1
|
||||
FROM messages m JOIN agents a ON m.from_agent = a.name
|
||||
WHERE CAST(a.owner_id AS TEXT) = ?
|
||||
AND a.name = ?
|
||||
AND m.created_at > ?
|
||||
LIMIT 1`,
|
||||
ownerID, agentName, since.UTC(),
|
||||
).Scan(&one)
|
||||
return err == nil && one == 1
|
||||
}
|
||||
|
||||
// mapHarnessResult translates harness output to a job status. Context
|
||||
// timeouts → 'partial' (the agent ran but was killed by the budget).
|
||||
func mapHarnessResult(res *HarnessExecResult, execErr, ctxErr error) (status, summary, errMsg string) {
|
||||
if ctxErr != nil && errors.Is(ctxErr, context.DeadlineExceeded) {
|
||||
return JobStatusPartial, "wallclock budget exhausted", ctxErr.Error()
|
||||
}
|
||||
if execErr != nil {
|
||||
return JobStatusFailed, "", execErr.Error()
|
||||
}
|
||||
if res == nil {
|
||||
return JobStatusFailed, "", "nil result"
|
||||
}
|
||||
if res.ExitCode == 0 {
|
||||
return JobStatusSucceeded, "completed cleanly", ""
|
||||
}
|
||||
return JobStatusPartial, fmt.Sprintf("exit code %d", res.ExitCode), ""
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
// Dream-agent prompts (feature 020 — US3). One short prompt per
|
||||
// consolidation job type. Passed to the dispatched Claude Code agent
|
||||
// as the task body via harness.ExecRequest. Each prompt: explains the
|
||||
// job goal in two or three sentences, lists the memory_* MCP tools
|
||||
// the agent is allowed to call, and references that the dispatch
|
||||
// token is already in SYNAPBUS_DISPATCH_TOKEN in the agent's env.
|
||||
package messaging
|
||||
|
||||
const (
|
||||
promptReflection = `You are running a memory-reflection pass on the open-brain pool.
|
||||
Your job: read recent unprocessed memories via memory_list_unprocessed,
|
||||
synthesize one or two short higher-level reflections that connect threads
|
||||
across them, and write each back via memory_write_reflection with the
|
||||
source ids listed.
|
||||
The dispatch token to authorize every memory_* tool call is already in
|
||||
your environment as SYNAPBUS_DISPATCH_TOKEN; pass owner_id from
|
||||
SYNAPBUS_OWNER_ID. Use only memory_* tools — do not send messages,
|
||||
do not call send_message, do not call execute. Keep each reflection
|
||||
under 600 chars. When done, exit 0.`
|
||||
|
||||
promptCoreRewrite = `You are running a sleep-time-rewrite pass on per-(owner, agent) core memory.
|
||||
Your job: for each owned agent, decide whether its core memory blob
|
||||
needs an update based on recent activity, and if so call
|
||||
memory_rewrite_core with the new blob (max 2048 bytes). The blob must
|
||||
be a tight, second-person identity-and-focus statement (e.g. "You are
|
||||
research-mcpproxy. Currently focused on benchmarking against ...").
|
||||
The dispatch token is in SYNAPBUS_DISPATCH_TOKEN. Use only memory_*
|
||||
tools. When done, exit 0.`
|
||||
|
||||
promptDedupContradiction = `You are running a deduplication / contradiction pass on the memory pool.
|
||||
Your job: read recent memories via memory_list_unprocessed, find pairs
|
||||
that say the same fact (call memory_mark_duplicate with keep_id being
|
||||
the canonical / longer / more recent of the two) or that contradict an
|
||||
older fact (call memory_supersede with a_id = the older fact, b_id =
|
||||
the newer one). Always provide a short reason. The dispatch token is
|
||||
in SYNAPBUS_DISPATCH_TOKEN. Use only memory_* tools. When done, exit 0.`
|
||||
|
||||
promptLinkGen = `You are running a link-generation pass on the memory pool.
|
||||
Your job: read recent memories via memory_list_unprocessed and add
|
||||
semantic links between related ones via memory_add_link with
|
||||
relation_type in {refines, contradicts, examples, related}. Do NOT use
|
||||
mention/reply_to/channel_cooccurrence (reserved for the messaging
|
||||
layer) or duplicate_of/superseded_by (use the dedicated tools). The
|
||||
dispatch token is in SYNAPBUS_DISPATCH_TOKEN. Use only memory_* tools.
|
||||
When done, exit 0.`
|
||||
)
|
||||
|
||||
// PromptFor returns the short task prompt for the given job type, or
|
||||
// the empty string for unknown types.
|
||||
func PromptFor(jobType string) string {
|
||||
switch jobType {
|
||||
case JobTypeReflection:
|
||||
return promptReflection
|
||||
case JobTypeCoreRewrite:
|
||||
return promptCoreRewrite
|
||||
case JobTypeDedupContradiction:
|
||||
return promptDedupContradiction
|
||||
case JobTypeLinkGen:
|
||||
return promptLinkGen
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,294 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// stubHarness implements messaging.HarnessDispatcher for the worker.
|
||||
type stubHarness struct {
|
||||
calls int32
|
||||
execDur time.Duration
|
||||
exitCode int
|
||||
execErr error
|
||||
}
|
||||
|
||||
func (s *stubHarness) Execute(ctx context.Context, agent DreamAgent, req *HarnessExecRequest) (*HarnessExecResult, error) {
|
||||
atomic.AddInt32(&s.calls, 1)
|
||||
if s.execDur > 0 {
|
||||
select {
|
||||
case <-time.After(s.execDur):
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
}
|
||||
if s.execErr != nil {
|
||||
return nil, s.execErr
|
||||
}
|
||||
return &HarnessExecResult{ExitCode: s.exitCode}, nil
|
||||
}
|
||||
|
||||
// stubAgentLookup returns a static agent.
|
||||
type stubAgentLookup struct {
|
||||
agent DreamAgent
|
||||
}
|
||||
|
||||
func (s *stubAgentLookup) GetAgent(ctx context.Context, name string) (DreamAgent, error) {
|
||||
if s.agent == nil {
|
||||
return nil, errors.New("not found")
|
||||
}
|
||||
return s.agent, nil
|
||||
}
|
||||
|
||||
func newWorkerForTest(t *testing.T, h HarnessDispatcher, agent DreamAgent, cfg MemoryConfig) (*ConsolidatorWorker, *sql.DB) {
|
||||
t.Helper()
|
||||
db := newTestDB(t)
|
||||
jobs := NewJobsStore(db)
|
||||
tokens := NewDispatchTokenStore(db)
|
||||
lookup := &stubAgentLookup{agent: agent}
|
||||
w := NewConsolidatorWorker(db, jobs, tokens, h, lookup, cfg)
|
||||
w.SetOwnerLister(func(ctx context.Context, db *sql.DB) ([]string, error) {
|
||||
return []string{"1"}, nil
|
||||
})
|
||||
return w, db
|
||||
}
|
||||
|
||||
// TestConsolidator_WatermarkBelowThresholdNoDispatch confirms tickets do
|
||||
// not fire when fewer than N unprocessed memories exist.
|
||||
func TestConsolidator_WatermarkBelowThresholdNoDispatch(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skip in short")
|
||||
}
|
||||
h := &stubHarness{}
|
||||
agent := DreamAgentNamed{Name: "claude-code"}
|
||||
cfg := MemoryConfig{
|
||||
DreamEnabled: true,
|
||||
DreamWatermark: 100,
|
||||
DreamMaxConcurrent: 1,
|
||||
DreamWallclockBudget: 100 * time.Millisecond,
|
||||
DreamInterval: 50 * time.Millisecond,
|
||||
DreamAgent: "claude-code",
|
||||
}
|
||||
w, _ := newWorkerForTest(t, h, agent, cfg)
|
||||
var last time.Time
|
||||
var lastCleanup time.Time
|
||||
w.tick(context.Background(), &last, &lastCleanup)
|
||||
|
||||
if got := atomic.LoadInt32(&h.calls); got != 0 {
|
||||
t.Errorf("harness.Execute called %d times despite no triggers", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestConsolidator_AtMostOneInFlightPerOwnerJobType verifies the
|
||||
// partial-unique index prevents a second pending job from being created
|
||||
// before the first completes.
|
||||
func TestConsolidator_AtMostOneInFlightPerOwnerJobType(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
jobs := NewJobsStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
id1, err := jobs.Create(ctx, "1", "reflection", "manual:test")
|
||||
if err != nil {
|
||||
t.Fatalf("first Create: %v", err)
|
||||
}
|
||||
if _, err := jobs.Create(ctx, "1", "reflection", "manual:test"); !errors.Is(err, ErrJobAlreadyInFlight) {
|
||||
t.Errorf("second Create: want ErrJobAlreadyInFlight, got %v", err)
|
||||
}
|
||||
|
||||
// Once the first completes, the next Create should succeed.
|
||||
if err := jobs.Complete(ctx, id1, JobStatusSucceeded, "", ""); err != nil {
|
||||
t.Fatalf("Complete: %v", err)
|
||||
}
|
||||
if _, err := jobs.Create(ctx, "1", "reflection", "manual:test"); err != nil {
|
||||
t.Errorf("third Create after Complete: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestConsolidator_NoSystemDMSent verifies the worker NEVER calls
|
||||
// MessagingService.SendMessage. We achieve this by passing a nil
|
||||
// messaging service and confirming no panic / no implicit call path.
|
||||
func TestConsolidator_NoSystemDMSent(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skip in short")
|
||||
}
|
||||
// Seed the memory channel + enough messages to trip the watermark.
|
||||
h := &stubHarness{}
|
||||
agent := DreamAgentNamed{Name: "claude-code"}
|
||||
cfg := MemoryConfig{
|
||||
DreamEnabled: true,
|
||||
DreamWatermark: 1,
|
||||
DreamMaxConcurrent: 1,
|
||||
DreamWallclockBudget: 200 * time.Millisecond,
|
||||
DreamAgent: "claude-code",
|
||||
}
|
||||
w, db := newWorkerForTest(t, h, agent, cfg)
|
||||
seedMemoryWithChannel(t, db, "a1", 1, "fact 1")
|
||||
|
||||
var last, lastCleanup time.Time
|
||||
w.tick(context.Background(), &last, &lastCleanup)
|
||||
// Give the dispatch goroutine a moment.
|
||||
time.Sleep(300 * time.Millisecond)
|
||||
w.Stop()
|
||||
|
||||
if got := atomic.LoadInt32(&h.calls); got == 0 {
|
||||
t.Logf("note: harness was not invoked (watermark may not have fired). Test still passes; the assertion is about *not* sending a DM, which is structural.")
|
||||
}
|
||||
}
|
||||
|
||||
// TestConsolidator_WallclockTerminatesRunaway verifies a runaway harness
|
||||
// call is killed by the budget and the job moves to `partial`.
|
||||
func TestConsolidator_WallclockTerminatesRunaway(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("skip in short")
|
||||
}
|
||||
h := &stubHarness{execDur: 2 * time.Second}
|
||||
agent := DreamAgentNamed{Name: "claude-code"}
|
||||
cfg := MemoryConfig{
|
||||
DreamEnabled: true,
|
||||
DreamWatermark: 1,
|
||||
DreamMaxConcurrent: 1,
|
||||
DreamWallclockBudget: 100 * time.Millisecond,
|
||||
DreamAgent: "claude-code",
|
||||
}
|
||||
w, db := newWorkerForTest(t, h, agent, cfg)
|
||||
seedMemoryWithChannel(t, db, "a1", 1, "fact 1")
|
||||
|
||||
// Manually invoke tryDispatch + runJob synchronously for a deterministic test.
|
||||
// Create a job, issue token, run runJob directly.
|
||||
jobID, err := w.jobs.Create(context.Background(), "1", "reflection", "test")
|
||||
if err != nil {
|
||||
t.Fatalf("Create: %v", err)
|
||||
}
|
||||
tok, _, err := w.tokens.Issue(context.Background(), "1", jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("Issue: %v", err)
|
||||
}
|
||||
_ = w.jobs.Dispatch(context.Background(), jobID, "test-run", tok)
|
||||
|
||||
w.runJob("1", jobID, JobTypeReflection, tok, "test-run", agent)
|
||||
|
||||
job, err := w.jobs.Get(context.Background(), jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("Get: %v", err)
|
||||
}
|
||||
if job.Status != JobStatusPartial {
|
||||
t.Errorf("expected status partial after wallclock kill, got %q (err=%q)", job.Status, job.Error)
|
||||
}
|
||||
}
|
||||
|
||||
// TestConsolidator_PreDispatchFailureDoesNotBurnJobsStarted verifies
|
||||
// that when launchOne fails before the Dispatch flip (e.g. agent
|
||||
// lookup fails), the per-(owner, day) jobs_started counter is NOT
|
||||
// incremented. Otherwise, repeated pre-dispatch failures inflate the
|
||||
// counter without any actual runs and eventually trip the
|
||||
// jobs_exceeded circuit breaker, wedging the worker for the rest of
|
||||
// the UTC day.
|
||||
func TestConsolidator_PreDispatchFailureDoesNotBurnJobsStarted(t *testing.T) {
|
||||
h := &stubHarness{}
|
||||
// Pass nil agent → stubAgentLookup.GetAgent returns "not found".
|
||||
// launchOne will Complete the job as failed before reaching Dispatch.
|
||||
cfg := MemoryConfig{
|
||||
DreamEnabled: true,
|
||||
DreamWatermark: 1,
|
||||
DreamMaxConcurrent: 1,
|
||||
DreamParallel: 1,
|
||||
DreamWallclockBudget: 100 * time.Millisecond,
|
||||
DreamAgent: "claude-code",
|
||||
DreamDailyJobLimit: 2,
|
||||
}
|
||||
w, db := newWorkerForTest(t, h, nil, cfg)
|
||||
usage := NewDreamUsageStore(db)
|
||||
gate := NewUsageGate(cfg, usage)
|
||||
w.SetUsageGate(usage, gate)
|
||||
|
||||
ctx := context.Background()
|
||||
owner := "1"
|
||||
|
||||
// Three attempts to ForceRun. Each one fails inside launchOne at the
|
||||
// agent-lookup step. None of them actually dispatch, so none should
|
||||
// count against DreamDailyJobLimit.
|
||||
for i := 0; i < 3; i++ {
|
||||
_, _ = w.ForceRun(ctx, owner, JobTypeReflection)
|
||||
}
|
||||
|
||||
u, err := usage.Today(ctx, owner)
|
||||
if err != nil {
|
||||
t.Fatalf("Today: %v", err)
|
||||
}
|
||||
if u.JobsStarted != 0 {
|
||||
t.Errorf("jobs_started must stay 0 when no dispatch flip succeeded; got %d", u.JobsStarted)
|
||||
}
|
||||
|
||||
allowed, reason, _ := gate.Allow(ctx, owner)
|
||||
if !allowed {
|
||||
t.Errorf("gate should still allow after pre-dispatch failures; got denied (%s)", reason)
|
||||
}
|
||||
}
|
||||
|
||||
// TestConsolidator_DispatchSuccessIncrementsJobsStarted is the positive
|
||||
// counterpart: when launchOne succeeds end-to-end (token issue + agent
|
||||
// lookup + dispatch flip), jobs_started IS incremented so the daily
|
||||
// limit is enforced correctly.
|
||||
func TestConsolidator_DispatchSuccessIncrementsJobsStarted(t *testing.T) {
|
||||
h := &stubHarness{}
|
||||
agent := DreamAgentNamed{Name: "claude-code"}
|
||||
cfg := MemoryConfig{
|
||||
DreamEnabled: true,
|
||||
DreamWatermark: 1,
|
||||
DreamMaxConcurrent: 1,
|
||||
DreamParallel: 1,
|
||||
DreamWallclockBudget: 500 * time.Millisecond,
|
||||
DreamAgent: "claude-code",
|
||||
DreamDailyJobLimit: 10,
|
||||
}
|
||||
w, db := newWorkerForTest(t, h, agent, cfg)
|
||||
usage := NewDreamUsageStore(db)
|
||||
gate := NewUsageGate(cfg, usage)
|
||||
w.SetUsageGate(usage, gate)
|
||||
|
||||
ctx := context.Background()
|
||||
owner := "1"
|
||||
|
||||
ids, err := w.ForceRun(ctx, owner, JobTypeReflection)
|
||||
if err != nil {
|
||||
t.Fatalf("ForceRun: %v", err)
|
||||
}
|
||||
if ids == 0 {
|
||||
t.Fatalf("ForceRun returned zero job id")
|
||||
}
|
||||
// Wait for the async runJob goroutine to settle so the counter
|
||||
// snapshot is stable. The stub harness returns immediately.
|
||||
w.Stop()
|
||||
|
||||
u, _ := usage.Today(ctx, owner)
|
||||
if u.JobsStarted != 1 {
|
||||
t.Errorf("jobs_started: want 1 after a successful dispatch, got %d", u.JobsStarted)
|
||||
}
|
||||
}
|
||||
|
||||
// seedMemoryWithChannel creates the open-brain channel + an agent
|
||||
// owned by owner_id=1 + one message.
|
||||
func seedMemoryWithChannel(t *testing.T, db *sql.DB, agentName string, channelID int64, body string) int64 {
|
||||
t.Helper()
|
||||
_, _ = db.Exec(`INSERT OR IGNORE INTO users (id, username, password_hash, display_name) VALUES (1, 'testowner', 'hash', 'Test Owner')`)
|
||||
_, _ = db.Exec(`INSERT OR IGNORE INTO agents (name, display_name, type, owner_id, api_key_hash, status) VALUES (?, ?, 'ai', 1, ?, 'active')`, agentName, agentName, agentName+"-hash")
|
||||
_, _ = db.Exec(`INSERT OR IGNORE INTO channels (id, name, description, type, created_by) VALUES (?, 'open-brain', '', 'standard', 'system')`, channelID)
|
||||
res, err := db.Exec(`INSERT INTO conversations (created_by, channel_id) VALUES (?, ?)`, agentName, channelID)
|
||||
if err != nil {
|
||||
t.Fatalf("seed conv: %v", err)
|
||||
}
|
||||
convID, _ := res.LastInsertId()
|
||||
res, err = db.Exec(`INSERT INTO messages (conversation_id, from_agent, channel_id, body, priority, status, metadata)
|
||||
VALUES (?, ?, ?, ?, 5, 'pending', '{}')`,
|
||||
convID, agentName, channelID, body,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("seed message: %v", err)
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
return id
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"database/sql"
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DispatchTokenTTL is the lifetime of a freshly-issued dispatch token.
|
||||
// Covers the dream-worker wallclock budget (10m default) plus dispatch
|
||||
// slack.
|
||||
const DispatchTokenTTL = 15 * time.Minute
|
||||
|
||||
// DispatchTokenStore manages single-use, owner-bound, job-bound tokens
|
||||
// passed to a dream-worker agent through harness.Execute env vars. The
|
||||
// agent presents the token on every memory_* MCP call; Validate is the
|
||||
// single point where authorization for a consolidation action is
|
||||
// resolved.
|
||||
type DispatchTokenStore struct {
|
||||
db *sql.DB
|
||||
// now is overridable in tests.
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
// NewDispatchTokenStore wraps a *sql.DB.
|
||||
func NewDispatchTokenStore(db *sql.DB) *DispatchTokenStore {
|
||||
return &DispatchTokenStore{db: db, now: time.Now}
|
||||
}
|
||||
|
||||
// Issue mints a new 32-byte random token bound to (ownerID, jobID) and
|
||||
// inserts it into `memory_dispatch_tokens` with expires_at = now() + TTL.
|
||||
func (s *DispatchTokenStore) Issue(ctx context.Context, ownerID string, jobID int64) (string, time.Time, error) {
|
||||
if ownerID == "" {
|
||||
return "", time.Time{}, fmt.Errorf("dispatch token: ownerID required")
|
||||
}
|
||||
if jobID == 0 {
|
||||
return "", time.Time{}, fmt.Errorf("dispatch token: jobID required")
|
||||
}
|
||||
|
||||
raw := make([]byte, 32)
|
||||
if _, err := rand.Read(raw); err != nil {
|
||||
return "", time.Time{}, fmt.Errorf("dispatch token: read random: %w", err)
|
||||
}
|
||||
token := base64.RawURLEncoding.EncodeToString(raw)
|
||||
|
||||
now := s.now().UTC()
|
||||
expiresAt := now.Add(DispatchTokenTTL)
|
||||
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO memory_dispatch_tokens (token, owner_id, consolidation_job_id, issued_at, expires_at)
|
||||
VALUES (?, ?, ?, ?, ?)`,
|
||||
token, ownerID, jobID, now, expiresAt,
|
||||
)
|
||||
if err != nil {
|
||||
return "", time.Time{}, fmt.Errorf("dispatch token: insert: %w", err)
|
||||
}
|
||||
return token, expiresAt, nil
|
||||
}
|
||||
|
||||
// Validate returns true when the token row exists, is not revoked, has
|
||||
// not expired, and matches the provided ownerID and jobID. On the first
|
||||
// successful validate it stamps `used_at = now()` (informational; subsequent
|
||||
// validates within the same job are still allowed per R7).
|
||||
func (s *DispatchTokenStore) Validate(ctx context.Context, token, ownerID string, jobID int64) (bool, error) {
|
||||
if token == "" {
|
||||
return false, nil
|
||||
}
|
||||
var (
|
||||
dbOwner string
|
||||
dbJob int64
|
||||
expiresAt time.Time
|
||||
usedAt sql.NullTime
|
||||
revokedAt sql.NullTime
|
||||
)
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT owner_id, consolidation_job_id, expires_at, used_at, revoked_at
|
||||
FROM memory_dispatch_tokens
|
||||
WHERE token = ?`, token,
|
||||
).Scan(&dbOwner, &dbJob, &expiresAt, &usedAt, &revokedAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return false, nil
|
||||
}
|
||||
return false, fmt.Errorf("dispatch token: query: %w", err)
|
||||
}
|
||||
if revokedAt.Valid {
|
||||
return false, nil
|
||||
}
|
||||
if !expiresAt.After(s.now().UTC()) {
|
||||
return false, nil
|
||||
}
|
||||
if dbOwner != ownerID {
|
||||
return false, nil
|
||||
}
|
||||
if dbJob != jobID {
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// Stamp used_at on first successful validate (idempotent: COALESCE
|
||||
// keeps the original timestamp on later calls).
|
||||
if !usedAt.Valid {
|
||||
if _, err := s.db.ExecContext(ctx,
|
||||
`UPDATE memory_dispatch_tokens SET used_at = ? WHERE token = ? AND used_at IS NULL`,
|
||||
s.now().UTC(), token,
|
||||
); err != nil {
|
||||
return false, fmt.Errorf("dispatch token: stamp used_at: %w", err)
|
||||
}
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// Revoke marks the given token as revoked. Idempotent; revoking a
|
||||
// non-existent token is not an error (so the dream worker can revoke on
|
||||
// best-effort cleanup without races).
|
||||
func (s *DispatchTokenStore) Revoke(ctx context.Context, token string) error {
|
||||
if token == "" {
|
||||
return nil
|
||||
}
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`UPDATE memory_dispatch_tokens SET revoked_at = ?
|
||||
WHERE token = ? AND revoked_at IS NULL`,
|
||||
s.now().UTC(), token,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("dispatch token: revoke: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
_ "modernc.org/sqlite"
|
||||
)
|
||||
|
||||
// insertJob seeds a memory_consolidation_jobs row and returns its id.
|
||||
// Dispatch tokens FK-reference this row.
|
||||
func insertJob(t *testing.T, db *sql.DB, ownerID, jobType string) int64 {
|
||||
t.Helper()
|
||||
res, err := db.Exec(
|
||||
`INSERT INTO memory_consolidation_jobs (owner_id, job_type, trigger_reason)
|
||||
VALUES (?, ?, ?)`,
|
||||
ownerID, jobType, "manual:test",
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("insert job: %v", err)
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
return id
|
||||
}
|
||||
|
||||
func TestDispatchTokenStore_IssueAndValidate(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDispatchTokenStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
jobID := insertJob(t, db, "1", "reflection")
|
||||
|
||||
token, expiresAt, err := store.Issue(ctx, "1", jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("Issue: %v", err)
|
||||
}
|
||||
if token == "" {
|
||||
t.Fatal("Issue returned empty token")
|
||||
}
|
||||
if !expiresAt.After(time.Now()) {
|
||||
t.Errorf("expiresAt = %v, want > now()", expiresAt)
|
||||
}
|
||||
|
||||
ok, err := store.Validate(ctx, token, "1", jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("Validate: %v", err)
|
||||
}
|
||||
if !ok {
|
||||
t.Fatal("Validate returned false for a freshly-issued token")
|
||||
}
|
||||
|
||||
// Second Validate must also succeed (token is bound to one job;
|
||||
// multiple tool calls within the job are expected per R7).
|
||||
ok, err = store.Validate(ctx, token, "1", jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("second Validate: %v", err)
|
||||
}
|
||||
if !ok {
|
||||
t.Error("second Validate returned false (token should remain valid within its job)")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDispatchTokenStore_RejectCases(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDispatchTokenStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
jobID := insertJob(t, db, "1", "reflection")
|
||||
otherJobID := insertJob(t, db, "1", "core_rewrite")
|
||||
|
||||
mintFresh := func(t *testing.T) string {
|
||||
t.Helper()
|
||||
tok, _, err := store.Issue(ctx, "1", jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("Issue: %v", err)
|
||||
}
|
||||
return tok
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
setup func(t *testing.T) (token, ownerID string, job int64)
|
||||
wantValid bool
|
||||
}{
|
||||
{
|
||||
name: "happy path",
|
||||
setup: func(t *testing.T) (string, string, int64) {
|
||||
return mintFresh(t), "1", jobID
|
||||
},
|
||||
wantValid: true,
|
||||
},
|
||||
{
|
||||
name: "owner mismatch",
|
||||
setup: func(t *testing.T) (string, string, int64) {
|
||||
return mintFresh(t), "2", jobID
|
||||
},
|
||||
wantValid: false,
|
||||
},
|
||||
{
|
||||
name: "wrong job",
|
||||
setup: func(t *testing.T) (string, string, int64) {
|
||||
return mintFresh(t), "1", otherJobID
|
||||
},
|
||||
wantValid: false,
|
||||
},
|
||||
{
|
||||
name: "expired",
|
||||
setup: func(t *testing.T) (string, string, int64) {
|
||||
tok := mintFresh(t)
|
||||
// Forcibly expire the token by direct UPDATE.
|
||||
if _, err := db.Exec(
|
||||
`UPDATE memory_dispatch_tokens SET expires_at = ? WHERE token = ?`,
|
||||
time.Now().Add(-1*time.Minute).UTC(), tok,
|
||||
); err != nil {
|
||||
t.Fatalf("expire token: %v", err)
|
||||
}
|
||||
return tok, "1", jobID
|
||||
},
|
||||
wantValid: false,
|
||||
},
|
||||
{
|
||||
name: "revoked",
|
||||
setup: func(t *testing.T) (string, string, int64) {
|
||||
tok := mintFresh(t)
|
||||
if err := store.Revoke(ctx, tok); err != nil {
|
||||
t.Fatalf("Revoke: %v", err)
|
||||
}
|
||||
return tok, "1", jobID
|
||||
},
|
||||
wantValid: false,
|
||||
},
|
||||
{
|
||||
name: "unknown token",
|
||||
setup: func(t *testing.T) (string, string, int64) {
|
||||
return "definitely-not-a-real-token", "1", jobID
|
||||
},
|
||||
wantValid: false,
|
||||
},
|
||||
{
|
||||
name: "empty token",
|
||||
setup: func(t *testing.T) (string, string, int64) {
|
||||
return "", "1", jobID
|
||||
},
|
||||
wantValid: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
tc := tc
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
tok, owner, job := tc.setup(t)
|
||||
got, err := store.Validate(ctx, tok, owner, job)
|
||||
if err != nil {
|
||||
t.Fatalf("Validate: %v", err)
|
||||
}
|
||||
if got != tc.wantValid {
|
||||
t.Errorf("Validate = %v, want %v", got, tc.wantValid)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestDispatchTokenStore_RevokeIdempotent(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDispatchTokenStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
jobID := insertJob(t, db, "1", "reflection")
|
||||
tok, _, err := store.Issue(ctx, "1", jobID)
|
||||
if err != nil {
|
||||
t.Fatalf("Issue: %v", err)
|
||||
}
|
||||
|
||||
if err := store.Revoke(ctx, tok); err != nil {
|
||||
t.Fatalf("first Revoke: %v", err)
|
||||
}
|
||||
if err := store.Revoke(ctx, tok); err != nil {
|
||||
t.Fatalf("second Revoke: %v", err)
|
||||
}
|
||||
if err := store.Revoke(ctx, "unknown"); err != nil {
|
||||
t.Fatalf("Revoke unknown: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDispatchTokenStore_IssueRequiresArgs(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDispatchTokenStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
if _, _, err := store.Issue(ctx, "", 1); err == nil {
|
||||
t.Error("Issue with empty owner should error")
|
||||
}
|
||||
if _, _, err := store.Issue(ctx, "1", 0); err == nil {
|
||||
t.Error("Issue with zero job id should error")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/metrics"
|
||||
)
|
||||
|
||||
// recordJobMetric increments the synapbus_dream_jobs_total counter for
|
||||
// (owner, job_type, status). Pulled into a helper so callers don't have
|
||||
// to import the metrics package directly and so test builds can stub it
|
||||
// later if needed.
|
||||
func recordJobMetric(ownerID, jobType, status string) {
|
||||
metrics.DreamJobsTotal.WithLabelValues(ownerID, jobType, status).Inc()
|
||||
}
|
||||
|
||||
// recordTokensMetric adds to the synapbus_dream_tokens_total counter
|
||||
// for both in and out directions. Zero deltas are no-ops.
|
||||
func recordTokensMetric(ownerID string, tokensIn, tokensOut int64) {
|
||||
if tokensIn > 0 {
|
||||
metrics.DreamTokensTotal.WithLabelValues(ownerID, "in").Add(float64(tokensIn))
|
||||
}
|
||||
if tokensOut > 0 {
|
||||
metrics.DreamTokensTotal.WithLabelValues(ownerID, "out").Add(float64(tokensOut))
|
||||
}
|
||||
}
|
||||
|
||||
// recordJobDurationMetric observes one dream-job wallclock duration in
|
||||
// the synapbus_dream_job_duration_seconds histogram.
|
||||
func recordJobDurationMetric(ownerID, jobType string, d time.Duration) {
|
||||
metrics.DreamJobDuration.WithLabelValues(ownerID, jobType).Observe(d.Seconds())
|
||||
}
|
||||
|
||||
// recordCircuitBrokenMetric bumps synapbus_dream_circuit_broken_total
|
||||
// for (owner, reason). Used when the UsageGate denies a dispatch.
|
||||
func recordCircuitBrokenMetric(ownerID, jobType, reason string) {
|
||||
// jobType is intentionally unused as a label here — the gate
|
||||
// decision is per-owner per-day, not per-job-type. Keeping the
|
||||
// parameter in the signature so call sites stay symmetric with
|
||||
// recordJobMetric.
|
||||
_ = jobType
|
||||
metrics.DreamCircuitBrokenTotal.WithLabelValues(ownerID, reason).Inc()
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
// Dream-worker daily usage tracker + circuit breaker (feature 020
|
||||
// follow-up). Wraps the `memory_dream_usage` table from migration 029.
|
||||
//
|
||||
// Each row aggregates one (UTC date, owner_id) bucket. The
|
||||
// ConsolidatorWorker:
|
||||
//
|
||||
// 1. Calls UsageGate.Allow() before Create+Issue+Execute. If today's
|
||||
// counters exceed any configured threshold the gate returns
|
||||
// allowed=false with a reason code; the worker then records a
|
||||
// `circuit_broken` job row and skips dispatch.
|
||||
// 2. On successful Execute completion, calls RecordCompletion with
|
||||
// the harness.ExecResult.Usage tokens so tomorrow's gate decisions
|
||||
// incorporate today's spend.
|
||||
//
|
||||
// The circuit "resets" naturally — Today() pivots on UTC date so the
|
||||
// first call after midnight UTC reads a fresh row with zero counters.
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DreamDailyUsage is one row of `memory_dream_usage`.
|
||||
type DreamDailyUsage struct {
|
||||
Date string `json:"date"`
|
||||
OwnerID string `json:"owner_id"`
|
||||
TokensIn int64 `json:"tokens_in"`
|
||||
TokensOut int64 `json:"tokens_out"`
|
||||
JobsStarted int `json:"jobs_started"`
|
||||
JobsSucceeded int `json:"jobs_succeeded"`
|
||||
JobsFailed int `json:"jobs_failed"`
|
||||
JobsCircuitBroken int `json:"jobs_circuit_broken"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
// DreamUsageStore wraps the `memory_dream_usage` table.
|
||||
type DreamUsageStore struct {
|
||||
db *sql.DB
|
||||
// now is overridable in tests so callers can pin the UTC date used
|
||||
// when bucketing.
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
// NewDreamUsageStore returns a store rooted at db.
|
||||
func NewDreamUsageStore(db *sql.DB) *DreamUsageStore {
|
||||
return &DreamUsageStore{db: db, now: time.Now}
|
||||
}
|
||||
|
||||
// utcDate returns the YYYY-MM-DD key for the store's clock.
|
||||
func (s *DreamUsageStore) utcDate() string {
|
||||
return s.now().UTC().Format("2006-01-02")
|
||||
}
|
||||
|
||||
// upsert is the common path for all increments — UPSERT (date, owner_id).
|
||||
// Counters are added with COALESCE so deltas accumulate cleanly.
|
||||
func (s *DreamUsageStore) upsert(ctx context.Context, ownerID string,
|
||||
dIn, dOut int64, dStart, dSuc, dFail, dCB int,
|
||||
) error {
|
||||
if s == nil || s.db == nil {
|
||||
return nil
|
||||
}
|
||||
if ownerID == "" {
|
||||
return fmt.Errorf("dream usage: empty owner_id")
|
||||
}
|
||||
date := s.utcDate()
|
||||
_, err := s.db.ExecContext(ctx, `
|
||||
INSERT INTO memory_dream_usage
|
||||
(date, owner_id, tokens_in, tokens_out,
|
||||
jobs_started, jobs_succeeded, jobs_failed, jobs_circuit_broken, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)
|
||||
ON CONFLICT(date, owner_id) DO UPDATE SET
|
||||
tokens_in = tokens_in + excluded.tokens_in,
|
||||
tokens_out = tokens_out + excluded.tokens_out,
|
||||
jobs_started = jobs_started + excluded.jobs_started,
|
||||
jobs_succeeded = jobs_succeeded + excluded.jobs_succeeded,
|
||||
jobs_failed = jobs_failed + excluded.jobs_failed,
|
||||
jobs_circuit_broken = jobs_circuit_broken + excluded.jobs_circuit_broken,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
`, date, ownerID, dIn, dOut, dStart, dSuc, dFail, dCB)
|
||||
if err != nil {
|
||||
return fmt.Errorf("dream usage: upsert: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RecordStart increments today's jobs_started for ownerID.
|
||||
func (s *DreamUsageStore) RecordStart(ctx context.Context, ownerID string) error {
|
||||
return s.upsert(ctx, ownerID, 0, 0, 1, 0, 0, 0)
|
||||
}
|
||||
|
||||
// RecordCompletion increments tokens and the appropriate per-status
|
||||
// counter. status ∈ {succeeded, failed, circuit_broken}. Unknown
|
||||
// statuses are counted as failed so the gate errs on the safe side.
|
||||
func (s *DreamUsageStore) RecordCompletion(ctx context.Context, ownerID string, tokensIn, tokensOut int64, status string) error {
|
||||
var dSuc, dFail, dCB int
|
||||
switch status {
|
||||
case JobStatusSucceeded:
|
||||
dSuc = 1
|
||||
case JobStatusCircuitBroken:
|
||||
dCB = 1
|
||||
case JobStatusPartial:
|
||||
// Partial counts as succeeded for the circuit-breaker — it ran.
|
||||
dSuc = 1
|
||||
default:
|
||||
dFail = 1
|
||||
}
|
||||
return s.upsert(ctx, ownerID, tokensIn, tokensOut, 0, dSuc, dFail, dCB)
|
||||
}
|
||||
|
||||
// Today returns today's counters for ownerID. Missing row → zero-value.
|
||||
func (s *DreamUsageStore) Today(ctx context.Context, ownerID string) (DreamDailyUsage, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return DreamDailyUsage{}, nil
|
||||
}
|
||||
if ownerID == "" {
|
||||
return DreamDailyUsage{}, fmt.Errorf("dream usage: empty owner_id")
|
||||
}
|
||||
date := s.utcDate()
|
||||
u := DreamDailyUsage{Date: date, OwnerID: ownerID}
|
||||
err := s.db.QueryRowContext(ctx, `
|
||||
SELECT tokens_in, tokens_out, jobs_started, jobs_succeeded,
|
||||
jobs_failed, jobs_circuit_broken, updated_at
|
||||
FROM memory_dream_usage
|
||||
WHERE date = ? AND owner_id = ?
|
||||
`, date, ownerID).Scan(
|
||||
&u.TokensIn, &u.TokensOut, &u.JobsStarted, &u.JobsSucceeded,
|
||||
&u.JobsFailed, &u.JobsCircuitBroken, &u.UpdatedAt,
|
||||
)
|
||||
if err == sql.ErrNoRows {
|
||||
return u, nil
|
||||
}
|
||||
if err != nil {
|
||||
return u, fmt.Errorf("dream usage: today: %w", err)
|
||||
}
|
||||
return u, nil
|
||||
}
|
||||
|
||||
// Cleanup deletes usage rows older than olderThanDays. Default callers
|
||||
// can pass 30 to keep a month of history for diagnostics.
|
||||
func (s *DreamUsageStore) Cleanup(ctx context.Context, olderThanDays int) (int64, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return 0, nil
|
||||
}
|
||||
if olderThanDays <= 0 {
|
||||
return 0, nil
|
||||
}
|
||||
cutoff := s.now().UTC().AddDate(0, 0, -olderThanDays).Format("2006-01-02")
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`DELETE FROM memory_dream_usage WHERE date < ?`, cutoff,
|
||||
)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("dream usage: cleanup: %w", err)
|
||||
}
|
||||
n, _ := res.RowsAffected()
|
||||
return n, nil
|
||||
}
|
||||
|
||||
// UsageGate is the circuit breaker the ConsolidatorWorker consults
|
||||
// before dispatching a job. It reads today's counters via DreamUsageStore
|
||||
// and compares them against the per-day limits in MemoryConfig.
|
||||
type UsageGate struct {
|
||||
cfg MemoryConfig
|
||||
store *DreamUsageStore
|
||||
}
|
||||
|
||||
// NewUsageGate ties a MemoryConfig to a DreamUsageStore.
|
||||
func NewUsageGate(cfg MemoryConfig, store *DreamUsageStore) *UsageGate {
|
||||
return &UsageGate{cfg: cfg, store: store}
|
||||
}
|
||||
|
||||
// Allow returns (allowed, reasonCode, err). When allowed is false the
|
||||
// caller should record a `circuit_broken` job and skip dispatch.
|
||||
// reasonCode is one of {tokens_in_exceeded, tokens_out_exceeded,
|
||||
// jobs_exceeded, ""}. err is non-nil only for database failures —
|
||||
// callers may treat err != nil as "fail open" to avoid wedging the
|
||||
// worker on a transient store glitch.
|
||||
func (g *UsageGate) Allow(ctx context.Context, ownerID string) (bool, string, error) {
|
||||
if g == nil || g.store == nil {
|
||||
return true, "", nil
|
||||
}
|
||||
u, err := g.store.Today(ctx, ownerID)
|
||||
if err != nil {
|
||||
return true, "", err
|
||||
}
|
||||
if g.cfg.DreamDailyTokenLimitIn > 0 && u.TokensIn >= g.cfg.DreamDailyTokenLimitIn {
|
||||
return false, "tokens_in_exceeded", nil
|
||||
}
|
||||
if g.cfg.DreamDailyTokenLimitOut > 0 && u.TokensOut >= g.cfg.DreamDailyTokenLimitOut {
|
||||
return false, "tokens_out_exceeded", nil
|
||||
}
|
||||
if g.cfg.DreamDailyJobLimit > 0 && u.JobsStarted >= g.cfg.DreamDailyJobLimit {
|
||||
return false, "jobs_exceeded", nil
|
||||
}
|
||||
return true, "", nil
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestDreamUsageStore_RecordAndQuery(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDreamUsageStore(db)
|
||||
|
||||
ctx := context.Background()
|
||||
owner := "42"
|
||||
|
||||
// Start two jobs, complete one succeeded with tokens, one failed.
|
||||
if err := store.RecordStart(ctx, owner); err != nil {
|
||||
t.Fatalf("RecordStart: %v", err)
|
||||
}
|
||||
if err := store.RecordStart(ctx, owner); err != nil {
|
||||
t.Fatalf("RecordStart: %v", err)
|
||||
}
|
||||
if err := store.RecordCompletion(ctx, owner, 1500, 700, JobStatusSucceeded); err != nil {
|
||||
t.Fatalf("RecordCompletion succeeded: %v", err)
|
||||
}
|
||||
if err := store.RecordCompletion(ctx, owner, 0, 0, JobStatusFailed); err != nil {
|
||||
t.Fatalf("RecordCompletion failed: %v", err)
|
||||
}
|
||||
|
||||
u, err := store.Today(ctx, owner)
|
||||
if err != nil {
|
||||
t.Fatalf("Today: %v", err)
|
||||
}
|
||||
if u.JobsStarted != 2 || u.JobsSucceeded != 1 || u.JobsFailed != 1 {
|
||||
t.Errorf("counters mismatch: %+v", u)
|
||||
}
|
||||
if u.TokensIn != 1500 || u.TokensOut != 700 {
|
||||
t.Errorf("tokens mismatch: in=%d out=%d", u.TokensIn, u.TokensOut)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDreamUsageStore_CircuitBrokenAccounting(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDreamUsageStore(db)
|
||||
ctx := context.Background()
|
||||
owner := "99"
|
||||
|
||||
if err := store.RecordCompletion(ctx, owner, 0, 0, JobStatusCircuitBroken); err != nil {
|
||||
t.Fatalf("RecordCompletion: %v", err)
|
||||
}
|
||||
u, _ := store.Today(ctx, owner)
|
||||
if u.JobsCircuitBroken != 1 {
|
||||
t.Errorf("expected jobs_circuit_broken=1, got %d", u.JobsCircuitBroken)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUsageGate_AllowDeny(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDreamUsageStore(db)
|
||||
ctx := context.Background()
|
||||
owner := "7"
|
||||
|
||||
cfg := DefaultMemoryConfig()
|
||||
cfg.DreamDailyTokenLimitIn = 1000
|
||||
cfg.DreamDailyTokenLimitOut = 500
|
||||
cfg.DreamDailyJobLimit = 3
|
||||
gate := NewUsageGate(cfg, store)
|
||||
|
||||
// Initially allowed.
|
||||
allowed, reason, err := gate.Allow(ctx, owner)
|
||||
if err != nil || !allowed || reason != "" {
|
||||
t.Fatalf("expected initial allow, got allowed=%v reason=%q err=%v", allowed, reason, err)
|
||||
}
|
||||
|
||||
// Push tokens_in just below the limit — still allowed.
|
||||
if err := store.RecordCompletion(ctx, owner, 999, 0, JobStatusSucceeded); err != nil {
|
||||
t.Fatalf("RecordCompletion: %v", err)
|
||||
}
|
||||
allowed, reason, _ = gate.Allow(ctx, owner)
|
||||
if !allowed {
|
||||
t.Errorf("expected allow at 999/1000 in, got %q", reason)
|
||||
}
|
||||
|
||||
// Cross the input threshold — denied.
|
||||
if err := store.RecordCompletion(ctx, owner, 2, 0, JobStatusSucceeded); err != nil {
|
||||
t.Fatalf("RecordCompletion: %v", err)
|
||||
}
|
||||
allowed, reason, _ = gate.Allow(ctx, owner)
|
||||
if allowed || reason != "tokens_in_exceeded" {
|
||||
t.Errorf("expected tokens_in_exceeded, got allowed=%v reason=%q", allowed, reason)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUsageGate_JobsExceeded(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDreamUsageStore(db)
|
||||
ctx := context.Background()
|
||||
owner := "5"
|
||||
|
||||
cfg := DefaultMemoryConfig()
|
||||
cfg.DreamDailyTokenLimitIn = 0 // disable token gates
|
||||
cfg.DreamDailyTokenLimitOut = 0
|
||||
cfg.DreamDailyJobLimit = 2
|
||||
gate := NewUsageGate(cfg, store)
|
||||
|
||||
for i := 0; i < 2; i++ {
|
||||
_ = store.RecordStart(ctx, owner)
|
||||
}
|
||||
allowed, reason, _ := gate.Allow(ctx, owner)
|
||||
if allowed || reason != "jobs_exceeded" {
|
||||
t.Errorf("expected jobs_exceeded, got allowed=%v reason=%q", allowed, reason)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUsageGate_NilStoreAllows(t *testing.T) {
|
||||
gate := NewUsageGate(DefaultMemoryConfig(), nil)
|
||||
allowed, reason, err := gate.Allow(context.Background(), "1")
|
||||
if !allowed || reason != "" || err != nil {
|
||||
t.Errorf("nil store: expected allow, got %v %q %v", allowed, reason, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDreamUsageStore_Cleanup(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewDreamUsageStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
// Insert an "old" row by overriding the clock.
|
||||
store.now = func() time.Time { return time.Now().AddDate(0, 0, -40).UTC() }
|
||||
if err := store.RecordStart(ctx, "1"); err != nil {
|
||||
t.Fatalf("RecordStart: %v", err)
|
||||
}
|
||||
// Restore now.
|
||||
store.now = time.Now
|
||||
if err := store.RecordStart(ctx, "1"); err != nil {
|
||||
t.Fatalf("RecordStart: %v", err)
|
||||
}
|
||||
|
||||
n, err := store.Cleanup(ctx, 30)
|
||||
if err != nil {
|
||||
t.Fatalf("Cleanup: %v", err)
|
||||
}
|
||||
if n != 1 {
|
||||
t.Errorf("expected 1 row deleted, got %d", n)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// EventMeta is the metadata-only view of a message used by event feeds
|
||||
// (e.g. the agent SSE stream). It deliberately carries no message body.
|
||||
type EventMeta struct {
|
||||
MessageID int64
|
||||
FromAgent string
|
||||
ToAgent string // set for DMs
|
||||
Channel string // set for channel messages
|
||||
Subject string
|
||||
}
|
||||
|
||||
// ListEventMetaAfter returns metadata for messages with id > afterID that are
|
||||
// visible to agentName, ordered by id ascending, at most limit rows.
|
||||
//
|
||||
// Visible means: DMs addressed to the agent, plus channel messages in channels
|
||||
// the agent is a member of (same scope as SearchMessages). Messages sent by the
|
||||
// agent itself are excluded.
|
||||
//
|
||||
// Membership is evaluated at query time (current membership), so messages
|
||||
// posted in a channel before the agent joined it are also returned when their
|
||||
// id > afterID. Channel members can read that history anyway.
|
||||
//
|
||||
// Note: channel messages are stored with to_agent NULL (see InsertMessage), so
|
||||
// the channel branch matches both NULL and ''.
|
||||
func (s *SQLiteMessageStore) ListEventMetaAfter(ctx context.Context, agentName string, afterID int64, limit int) ([]*EventMeta, error) {
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT m.id, m.from_agent, COALESCE(m.to_agent, ''), COALESCE(ch.name, ''), COALESCE(cv.subject, '')
|
||||
FROM messages m
|
||||
LEFT JOIN channels ch ON ch.id = m.channel_id
|
||||
LEFT JOIN conversations cv ON cv.id = m.conversation_id
|
||||
WHERE m.id > ? AND m.from_agent <> ?
|
||||
AND (m.to_agent = ? OR (m.channel_id IS NOT NULL AND (m.to_agent IS NULL OR m.to_agent = '') AND EXISTS (SELECT 1 FROM channel_members cm WHERE cm.channel_id = m.channel_id AND cm.agent_name = ?)))
|
||||
ORDER BY m.id ASC
|
||||
LIMIT ?`,
|
||||
afterID, agentName, agentName, agentName, limit,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("query event meta: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []*EventMeta
|
||||
for rows.Next() {
|
||||
var e EventMeta
|
||||
if err := rows.Scan(&e.MessageID, &e.FromAgent, &e.ToAgent, &e.Channel, &e.Subject); err != nil {
|
||||
return nil, fmt.Errorf("scan event meta: %w", err)
|
||||
}
|
||||
out = append(out, &e)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// ListEventMetaAfter returns body-free metadata for messages visible to
|
||||
// agentName with id > afterID, ascending by id, at most limit rows.
|
||||
func (s *MessagingService) ListEventMetaAfter(ctx context.Context, agentName string, afterID int64, limit int) ([]*EventMeta, error) {
|
||||
return s.store.ListEventMetaAfter(ctx, agentName, afterID, limit)
|
||||
}
|
||||
|
||||
// GetConversationSubject returns the subject of a conversation, or "" if unknown.
|
||||
func (s *MessagingService) GetConversationSubject(ctx context.Context, conversationID int64) string {
|
||||
conv, err := s.store.GetConversation(ctx, conversationID)
|
||||
if err != nil || conv == nil {
|
||||
return ""
|
||||
}
|
||||
return conv.Subject
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSQLiteMessageStore_ListEventMetaAfter(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewSQLiteMessageStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
for _, a := range []string{"a", "b", "c"} {
|
||||
seedAgent(t, db, a)
|
||||
}
|
||||
for _, q := range []string{
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, created_at, updated_at)
|
||||
VALUES (1, 'room', '', '', 'standard', 0, 0, 'a', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`,
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, created_at, updated_at)
|
||||
VALUES (2, 'hidden', '', '', 'standard', 0, 0, 'c', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`,
|
||||
`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (1, 'a', 'member', CURRENT_TIMESTAMP)`,
|
||||
`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (1, 'b', 'member', CURRENT_TIMESTAMP)`,
|
||||
`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (2, 'c', 'member', CURRENT_TIMESTAMP)`,
|
||||
} {
|
||||
if _, err := db.ExecContext(ctx, q); err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
room, hidden := int64(1), int64(2)
|
||||
// Channel messages are stored with to_agent NULL (InsertMessage).
|
||||
seed := []struct {
|
||||
from, to string
|
||||
ch *int64
|
||||
}{
|
||||
{"b", "a", nil}, // 1: DM to a
|
||||
{"b", "c", nil}, // 2: DM to c
|
||||
{"b", "", &room}, // 3: room, from b
|
||||
{"c", "", &hidden}, // 4: hidden, from c
|
||||
{"a", "b", nil}, // 5: DM from a to b
|
||||
{"a", "", &room}, // 6: room, from a
|
||||
{"b", "a", nil}, // 7: DM to a
|
||||
}
|
||||
for _, s := range seed {
|
||||
conv := &Conversation{Subject: "subj-" + s.from, CreatedBy: s.from}
|
||||
if err := store.InsertConversation(ctx, conv); err != nil {
|
||||
t.Fatalf("InsertConversation: %v", err)
|
||||
}
|
||||
m := &Message{ConversationID: conv.ID, FromAgent: s.from, ToAgent: s.to, ChannelID: s.ch,
|
||||
Body: "secret", Priority: 5, Status: StatusPending}
|
||||
if err := store.InsertMessage(ctx, m); err != nil {
|
||||
t.Fatalf("InsertMessage: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
agent string
|
||||
afterID int64
|
||||
limit int
|
||||
want []int64
|
||||
}{
|
||||
{"a sees own DMs and joined channel, not others or own", "a", 0, 100, []int64{1, 3, 7}},
|
||||
{"c sees DM and own channel only", "c", 0, 100, []int64{2}}, // 4 is c's own message
|
||||
{"b sees DM and room message from a", "b", 0, 100, []int64{5, 6}},
|
||||
{"unknown agent sees nothing", "zed", 0, 100, nil},
|
||||
{"afterID is exclusive", "a", 1, 100, []int64{3, 7}},
|
||||
{"afterID past end", "a", 7, 100, nil},
|
||||
{"limit applies, ascending", "a", 0, 2, []int64{1, 3}},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := store.ListEventMetaAfter(ctx, tt.agent, tt.afterID, tt.limit)
|
||||
if err != nil {
|
||||
t.Fatalf("ListEventMetaAfter: %v", err)
|
||||
}
|
||||
var ids []int64
|
||||
for _, e := range got {
|
||||
ids = append(ids, e.MessageID)
|
||||
}
|
||||
if !reflect.DeepEqual(ids, tt.want) {
|
||||
t.Fatalf("ids = %v, want %v", ids, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("metadata fields", func(t *testing.T) {
|
||||
got, _ := store.ListEventMetaAfter(ctx, "a", 0, 100)
|
||||
if got[0].FromAgent != "b" || got[0].ToAgent != "a" || got[0].Channel != "" || got[0].Subject != "subj-b" {
|
||||
t.Errorf("DM meta = %+v", got[0])
|
||||
}
|
||||
if got[1].Channel != "room" || got[1].ToAgent != "" {
|
||||
t.Errorf("channel meta = %+v", got[1])
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,125 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// MemoryChannelNames lists exact channel names that are always treated
|
||||
// as memory channels regardless of metadata.
|
||||
var MemoryChannelNames = []string{"open-brain"}
|
||||
|
||||
// MemoryChannelPrefixes lists channel-name prefixes that are always
|
||||
// treated as memory channels. Matching uses HasPrefix (e.g.
|
||||
// "reflections-personal-brand" matches "reflections-").
|
||||
var MemoryChannelPrefixes = []string{"reflections-"}
|
||||
|
||||
// MemoryChannel is the lightweight shape this package needs to decide
|
||||
// memory-pool participation. We use a local type rather than
|
||||
// `channels.Channel` to avoid an import cycle: package `channels`
|
||||
// already imports `messaging`. Callers that hold a `*channels.Channel`
|
||||
// can construct a `MemoryChannel{Name: ch.Name}` and pass it in.
|
||||
type MemoryChannel struct {
|
||||
ID int64
|
||||
Name string
|
||||
Metadata string // raw JSON; empty when unknown
|
||||
}
|
||||
|
||||
// IsMemoryChannel reports whether the given channel participates in
|
||||
// the proactive-memory pool.
|
||||
//
|
||||
// A channel is a memory channel when ANY of:
|
||||
//
|
||||
// 1. Its name appears in MemoryChannelNames (currently `open-brain`).
|
||||
// 2. Its name starts with a MemoryChannelPrefixes entry (currently
|
||||
// `reflections-*`).
|
||||
// 3. Its raw metadata JSON contains `"is_memory": true`.
|
||||
//
|
||||
// The metadata path is documented in 020-data-model.md but the
|
||||
// `channels` table does not yet expose a metadata column; the third
|
||||
// rule is wired up so it just-works once that lands and is harmless
|
||||
// in the meantime (empty Metadata always returns false).
|
||||
func IsMemoryChannel(ch *MemoryChannel) bool {
|
||||
if ch == nil {
|
||||
return false
|
||||
}
|
||||
if matchesMemoryChannelName(ch.Name) {
|
||||
return true
|
||||
}
|
||||
return isMemoryChannelMetadata(ch.Metadata)
|
||||
}
|
||||
|
||||
func matchesMemoryChannelName(name string) bool {
|
||||
for _, n := range MemoryChannelNames {
|
||||
if name == n {
|
||||
return true
|
||||
}
|
||||
}
|
||||
for _, p := range MemoryChannelPrefixes {
|
||||
if strings.HasPrefix(name, p) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isMemoryChannelMetadata(metadata string) bool {
|
||||
m := strings.TrimSpace(metadata)
|
||||
if m == "" || m == "{}" {
|
||||
return false
|
||||
}
|
||||
var parsed map[string]any
|
||||
if err := json.Unmarshal([]byte(m), &parsed); err != nil {
|
||||
return false
|
||||
}
|
||||
v, ok := parsed["is_memory"]
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
b, ok := v.(bool)
|
||||
return ok && b
|
||||
}
|
||||
|
||||
// MemoryChannelIDs returns the IDs of every channel that participates
|
||||
// in the memory pool. Driven by name patterns today; will pick up the
|
||||
// metadata flag automatically once `channels.metadata` ships (this
|
||||
// function would then SELECT metadata as well and apply
|
||||
// IsMemoryChannel per row).
|
||||
func MemoryChannelIDs(ctx context.Context, db *sql.DB) ([]int64, error) {
|
||||
var clauses []string
|
||||
var args []any
|
||||
for _, n := range MemoryChannelNames {
|
||||
clauses = append(clauses, "name = ?")
|
||||
args = append(args, n)
|
||||
}
|
||||
for _, p := range MemoryChannelPrefixes {
|
||||
clauses = append(clauses, "name LIKE ?")
|
||||
args = append(args, p+"%")
|
||||
}
|
||||
if len(clauses) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
query := "SELECT id FROM channels WHERE " + strings.Join(clauses, " OR ")
|
||||
rows, err := db.QueryContext(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("query memory channels: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var ids []int64
|
||||
for rows.Next() {
|
||||
var id int64
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
return nil, fmt.Errorf("scan memory channel id: %w", err)
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("iterate memory channel ids: %w", err)
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
@@ -0,0 +1,219 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
// MemoryConfig holds feature-flag and tunable knobs for proactive memory
|
||||
// injection and the dream worker. All fields are populated from
|
||||
// environment variables by ParseMemoryConfig; defaults match the values
|
||||
// listed in the 020-proactive-memory-dream-worker spec.
|
||||
type MemoryConfig struct {
|
||||
// InjectionEnabled toggles the `relevant_context` packet on MCP tool
|
||||
// responses. When false, the wrapper is a no-op and the field is
|
||||
// omitted from responses entirely.
|
||||
InjectionEnabled bool
|
||||
|
||||
// InjectionBudgetTokens is the soft cap for the merged packet
|
||||
// (memories + core_memory), counted as `(chars+3)/4`.
|
||||
InjectionBudgetTokens int
|
||||
|
||||
// InjectionMaxItems caps the number of memory items in a single packet.
|
||||
InjectionMaxItems int
|
||||
|
||||
// InjectionMinScore is the relevance floor; items below this are
|
||||
// dropped unless explicitly pinned.
|
||||
InjectionMinScore float64
|
||||
|
||||
// CoreMemoryMaxBytes is the upper bound on a per-(owner, agent) core
|
||||
// memory blob. Enforced in the Set path.
|
||||
CoreMemoryMaxBytes int
|
||||
|
||||
// DreamEnabled gates the entire ConsolidatorWorker lifecycle and the
|
||||
// six memory_* MCP tools.
|
||||
DreamEnabled bool
|
||||
|
||||
// DreamInterval is the ticker period for the consolidator worker.
|
||||
DreamInterval time.Duration
|
||||
|
||||
// DreamDeepCron is the cron expression for the nightly deep pass
|
||||
// (e.g. core_rewrite). Standard 5-field cron; not parsed here.
|
||||
DreamDeepCron string
|
||||
|
||||
// DreamMaxConcurrent caps the number of owners with in-flight jobs.
|
||||
DreamMaxConcurrent int
|
||||
|
||||
// DreamParallel is the number of concurrent dream jobs the worker
|
||||
// (and `synapbus memory dream-run`) will fan out per (owner, job_type)
|
||||
// when triggered. 1 = historical single-job behaviour. Higher values
|
||||
// let one watermark/manual trigger spawn N parallel reflection /
|
||||
// link_gen / dedup_contradiction agents to drain a backlog. core_rewrite
|
||||
// is always single-slot regardless of this knob.
|
||||
DreamParallel int
|
||||
|
||||
// DreamWallclockBudget caps how long a single consolidation job is
|
||||
// allowed to run before the worker terminates it with status=partial.
|
||||
DreamWallclockBudget time.Duration
|
||||
|
||||
// DreamWatermark is the number of new memory messages required
|
||||
// before a reflection job triggers automatically for an owner.
|
||||
DreamWatermark int
|
||||
|
||||
// DreamAgent is the name of the agent invoked as the dream worker
|
||||
// via harness.Harness.Execute.
|
||||
DreamAgent string
|
||||
|
||||
// DreamRecentWindow bounds the lookback for the dream worker's
|
||||
// per-owner input set (unprocessed-count queries, recency
|
||||
// injection fallback, and memory_list_unprocessed). Messages older
|
||||
// than `now - DreamRecentWindow` are invisible to the worker — the
|
||||
// pool is effectively a rolling window of recent activity. Default
|
||||
// 14 days. Env: SYNAPBUS_DREAM_RECENT_WINDOW. Accepts Go
|
||||
// duration syntax plus the "Nd" days extension.
|
||||
DreamRecentWindow time.Duration
|
||||
|
||||
// DreamDailyTokenLimitIn is the per-(owner, day) input-token
|
||||
// circuit-breaker threshold. When the running sum of TokensIn
|
||||
// across today's dream jobs exceeds this, the worker skips further
|
||||
// dispatches for that owner until the date rolls over. Default 1M.
|
||||
// Env: SYNAPBUS_DREAM_DAILY_TOKEN_LIMIT_IN.
|
||||
DreamDailyTokenLimitIn int64
|
||||
|
||||
// DreamDailyTokenLimitOut is the per-(owner, day) output-token
|
||||
// circuit-breaker threshold. Default 200k.
|
||||
// Env: SYNAPBUS_DREAM_DAILY_TOKEN_LIMIT_OUT.
|
||||
DreamDailyTokenLimitOut int64
|
||||
|
||||
// DreamDailyJobLimit is the per-(owner, day) ceiling on jobs
|
||||
// started. Default 100. Env: SYNAPBUS_DREAM_DAILY_JOB_LIMIT.
|
||||
DreamDailyJobLimit int
|
||||
}
|
||||
|
||||
// DefaultMemoryConfig returns the defaults exactly as listed in the
|
||||
// 020-proactive-memory-dream-worker spec.
|
||||
func DefaultMemoryConfig() MemoryConfig {
|
||||
return MemoryConfig{
|
||||
InjectionEnabled: false,
|
||||
InjectionBudgetTokens: 500,
|
||||
InjectionMaxItems: 5,
|
||||
InjectionMinScore: 0.25,
|
||||
CoreMemoryMaxBytes: 2048,
|
||||
DreamEnabled: false,
|
||||
DreamInterval: 1 * time.Hour,
|
||||
DreamDeepCron: "0 3 * * *",
|
||||
DreamMaxConcurrent: 4,
|
||||
DreamParallel: 1,
|
||||
DreamWallclockBudget: 10 * time.Minute,
|
||||
DreamWatermark: 20,
|
||||
DreamAgent: "claude-code",
|
||||
// 14 days of recent activity.
|
||||
DreamRecentWindow: 336 * time.Hour,
|
||||
DreamDailyTokenLimitIn: 1_000_000,
|
||||
DreamDailyTokenLimitOut: 200_000,
|
||||
DreamDailyJobLimit: 100,
|
||||
}
|
||||
}
|
||||
|
||||
// ParseMemoryConfig reads MemoryConfig fields from environment
|
||||
// variables, falling back to the spec defaults for any unset or
|
||||
// unparseable value.
|
||||
func ParseMemoryConfig() MemoryConfig {
|
||||
cfg := DefaultMemoryConfig()
|
||||
|
||||
if v := os.Getenv("SYNAPBUS_INJECTION_ENABLED"); v != "" {
|
||||
if b, ok := parseBoolFlag(v); ok {
|
||||
cfg.InjectionEnabled = b
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_INJECTION_BUDGET_TOKENS"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
cfg.InjectionBudgetTokens = n
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_INJECTION_MAX_ITEMS"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
cfg.InjectionMaxItems = n
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_INJECTION_MIN_SCORE"); v != "" {
|
||||
if f, err := strconv.ParseFloat(v, 64); err == nil && f >= 0 {
|
||||
cfg.InjectionMinScore = f
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_CORE_MEMORY_MAX_BYTES"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
cfg.CoreMemoryMaxBytes = n
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_ENABLED"); v != "" {
|
||||
if b, ok := parseBoolFlag(v); ok {
|
||||
cfg.DreamEnabled = b
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_INTERVAL"); v != "" {
|
||||
if d, err := parseDurationWithDays(v); err == nil && d > 0 {
|
||||
cfg.DreamInterval = d
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_DEEP_CRON"); v != "" {
|
||||
cfg.DreamDeepCron = v
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_MAX_CONCURRENT"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
cfg.DreamMaxConcurrent = n
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_PARALLEL"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
cfg.DreamParallel = n
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_WALLCLOCK_BUDGET"); v != "" {
|
||||
if d, err := parseDurationWithDays(v); err == nil && d > 0 {
|
||||
cfg.DreamWallclockBudget = d
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_WATERMARK"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
cfg.DreamWatermark = n
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_AGENT"); v != "" {
|
||||
cfg.DreamAgent = v
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_RECENT_WINDOW"); v != "" {
|
||||
if d, err := parseDurationWithDays(v); err == nil && d > 0 {
|
||||
cfg.DreamRecentWindow = d
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_DAILY_TOKEN_LIMIT_IN"); v != "" {
|
||||
if n, err := strconv.ParseInt(v, 10, 64); err == nil && n > 0 {
|
||||
cfg.DreamDailyTokenLimitIn = n
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_DAILY_TOKEN_LIMIT_OUT"); v != "" {
|
||||
if n, err := strconv.ParseInt(v, 10, 64); err == nil && n > 0 {
|
||||
cfg.DreamDailyTokenLimitOut = n
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_DREAM_DAILY_JOB_LIMIT"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
cfg.DreamDailyJobLimit = n
|
||||
}
|
||||
}
|
||||
|
||||
return cfg
|
||||
}
|
||||
|
||||
// parseBoolFlag accepts the common 0/1/true/false/yes/no spellings.
|
||||
func parseBoolFlag(v string) (bool, bool) {
|
||||
switch v {
|
||||
case "1", "true", "TRUE", "True", "yes", "YES", "y", "on", "ON":
|
||||
return true, true
|
||||
case "0", "false", "FALSE", "False", "no", "NO", "n", "off", "OFF":
|
||||
return false, true
|
||||
}
|
||||
return false, false
|
||||
}
|
||||
@@ -0,0 +1,195 @@
|
||||
// Per-(owner, agent) core memory store. Backs User Story 2 of feature
|
||||
// 020-proactive-memory-dream-worker — small, owner-scoped, replace-wholesale
|
||||
// blobs surfaced in `relevant_context.core_memory` on session-start tools
|
||||
// (e.g. `my_status`).
|
||||
//
|
||||
// Schema lives in `internal/storage/schema/028_memory_consolidation.sql`
|
||||
// (table `memory_core`). owner_id is stored as TEXT (string form of
|
||||
// `users.id`) to match the proactive-memory tables and the request-context
|
||||
// owner_id propagated by auth middleware.
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ErrCoreMemoryTooLarge is returned by CoreMemoryStore.Set when the blob
|
||||
// exceeds the configured max bytes (default SYNAPBUS_CORE_MEMORY_MAX_BYTES = 2048).
|
||||
// The MCP `memory_rewrite_core` tool surfaces this as the contractual
|
||||
// `core_memory_too_large` error code (see contracts/mcp-memory-tools.md).
|
||||
var ErrCoreMemoryTooLarge = errors.New("core memory blob exceeds max bytes")
|
||||
|
||||
// CoreMemoryRecord is one row of the `memory_core` table.
|
||||
type CoreMemoryRecord struct {
|
||||
OwnerID string
|
||||
AgentName string
|
||||
Blob string
|
||||
UpdatedAt time.Time
|
||||
UpdatedBy string
|
||||
}
|
||||
|
||||
// CoreMemoryStore wraps the `memory_core` table. All operations are
|
||||
// owner-scoped — owner_id is part of the primary key — so callers cannot
|
||||
// cross-read another owner's blobs.
|
||||
type CoreMemoryStore struct {
|
||||
db *sql.DB
|
||||
maxBytes int
|
||||
}
|
||||
|
||||
// NewCoreMemoryStore returns a store rooted at db enforcing the given
|
||||
// max-bytes cap on Set. When maxBytes <= 0, defaults to 2048 (the spec
|
||||
// default for SYNAPBUS_CORE_MEMORY_MAX_BYTES).
|
||||
func NewCoreMemoryStore(db *sql.DB, maxBytes int) *CoreMemoryStore {
|
||||
if maxBytes <= 0 {
|
||||
maxBytes = 2048
|
||||
}
|
||||
return &CoreMemoryStore{db: db, maxBytes: maxBytes}
|
||||
}
|
||||
|
||||
// MaxBytes returns the configured upper bound for Set blobs.
|
||||
func (s *CoreMemoryStore) MaxBytes() int { return s.maxBytes }
|
||||
|
||||
// Get returns the core memory blob for (ownerID, agentName). When no row
|
||||
// exists, returns ok=false with no error. Unexpected DB errors surface as
|
||||
// err.
|
||||
func (s *CoreMemoryStore) Get(ctx context.Context, ownerID, agentName string) (blob string, updatedAt time.Time, ok bool, err error) {
|
||||
row := s.db.QueryRowContext(ctx,
|
||||
`SELECT blob, updated_at FROM memory_core WHERE owner_id = ? AND agent_name = ?`,
|
||||
ownerID, agentName,
|
||||
)
|
||||
if err := row.Scan(&blob, &updatedAt); err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return "", time.Time{}, false, nil
|
||||
}
|
||||
return "", time.Time{}, false, fmt.Errorf("core memory get: %w", err)
|
||||
}
|
||||
return blob, updatedAt, true, nil
|
||||
}
|
||||
|
||||
// Set wholesale-replaces the blob for (ownerID, agentName). Enforces the
|
||||
// configured size cap and returns ErrCoreMemoryTooLarge when violated.
|
||||
// `updatedBy` is recorded for audit (typically the caller agent name, or
|
||||
// "human" for admin/web edits).
|
||||
func (s *CoreMemoryStore) Set(ctx context.Context, ownerID, agentName, blob, updatedBy string) error {
|
||||
if len(blob) > s.maxBytes {
|
||||
return ErrCoreMemoryTooLarge
|
||||
}
|
||||
if ownerID == "" {
|
||||
return fmt.Errorf("core memory set: empty owner_id")
|
||||
}
|
||||
if agentName == "" {
|
||||
return fmt.Errorf("core memory set: empty agent_name")
|
||||
}
|
||||
if updatedBy == "" {
|
||||
return fmt.Errorf("core memory set: empty updated_by")
|
||||
}
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO memory_core (owner_id, agent_name, blob, updated_at, updated_by)
|
||||
VALUES (?, ?, ?, CURRENT_TIMESTAMP, ?)
|
||||
ON CONFLICT(owner_id, agent_name) DO UPDATE SET
|
||||
blob = excluded.blob,
|
||||
updated_at = CURRENT_TIMESTAMP,
|
||||
updated_by = excluded.updated_by`,
|
||||
ownerID, agentName, blob, updatedBy,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("core memory set: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Delete removes the (ownerID, agentName) row. Returns nil even if no
|
||||
// row matched — callers should treat "deleted" and "did not exist" the
|
||||
// same way (the REST endpoint distinguishes via a separate Get).
|
||||
func (s *CoreMemoryStore) Delete(ctx context.Context, ownerID, agentName string) error {
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`DELETE FROM memory_core WHERE owner_id = ? AND agent_name = ?`,
|
||||
ownerID, agentName,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("core memory delete: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// List returns all core memory rows for the given owner. Used by the
|
||||
// future audit UI (deferred US4) and by admin tooling.
|
||||
func (s *CoreMemoryStore) List(ctx context.Context, ownerID string) ([]CoreMemoryRecord, error) {
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT owner_id, agent_name, blob, updated_at, updated_by
|
||||
FROM memory_core
|
||||
WHERE owner_id = ?
|
||||
ORDER BY agent_name`,
|
||||
ownerID,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("core memory list: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []CoreMemoryRecord
|
||||
for rows.Next() {
|
||||
var r CoreMemoryRecord
|
||||
if err := rows.Scan(&r.OwnerID, &r.AgentName, &r.Blob, &r.UpdatedAt, &r.UpdatedBy); err != nil {
|
||||
return nil, fmt.Errorf("core memory list scan: %w", err)
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("core memory list rows: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// GetForInjection is the adapter implementing
|
||||
// `search.CoreMemoryProvider.Get`. Returns "" (no error) when no row
|
||||
// exists — the empty-string convention lets the injection wrapper treat a
|
||||
// missing core memory the same as "field omitted", per
|
||||
// `contracts/mcp-injection.md`.
|
||||
//
|
||||
// The matching interface contract is in
|
||||
// `internal/search/injection.go`'s `CoreMemoryProvider`:
|
||||
//
|
||||
// Get(ctx context.Context, ownerID, agentName string) (string, error)
|
||||
//
|
||||
// We expose this as a method on the store (not a separate type) so
|
||||
// callers can pass `coreStore.GetForInjection` as a method value — but the
|
||||
// store itself also satisfies the interface via its `Get` method below.
|
||||
func (s *CoreMemoryStore) GetForInjection(ctx context.Context, ownerID, agentName string) (string, error) {
|
||||
blob, _, ok, err := s.Get(ctx, ownerID, agentName)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if !ok {
|
||||
return "", nil
|
||||
}
|
||||
return blob, nil
|
||||
}
|
||||
|
||||
// coreProviderAdapter wraps a *CoreMemoryStore so it satisfies
|
||||
// `search.CoreMemoryProvider` (which requires a `Get(ctx, ownerID,
|
||||
// agentName) (string, error)` signature — distinct from the store's
|
||||
// 4-return Get). Use NewCoreProvider to construct.
|
||||
type coreProviderAdapter struct {
|
||||
store *CoreMemoryStore
|
||||
}
|
||||
|
||||
// NewCoreProvider returns an object satisfying
|
||||
// `search.CoreMemoryProvider` so callers can wire the store into
|
||||
// WrapConfig.CoreProvider without leaking the store's richer Get
|
||||
// signature.
|
||||
func NewCoreProvider(store *CoreMemoryStore) *coreProviderAdapter {
|
||||
return &coreProviderAdapter{store: store}
|
||||
}
|
||||
|
||||
// Get implements `search.CoreMemoryProvider.Get`. Returns "" when no row
|
||||
// exists for the (owner, agent) pair.
|
||||
func (a *coreProviderAdapter) Get(ctx context.Context, ownerID, agentName string) (string, error) {
|
||||
if a == nil || a.store == nil {
|
||||
return "", nil
|
||||
}
|
||||
return a.store.GetForInjection(ctx, ownerID, agentName)
|
||||
}
|
||||
@@ -0,0 +1,179 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestCoreMemoryStore_GetSetRoundTrip(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewCoreMemoryStore(db, 2048)
|
||||
ctx := context.Background()
|
||||
|
||||
const owner = "1"
|
||||
const agent = "research-mcpproxy"
|
||||
const blob = "You are research-mcpproxy. Focus on benchmarking."
|
||||
|
||||
if err := store.Set(ctx, owner, agent, blob, "human"); err != nil {
|
||||
t.Fatalf("Set: %v", err)
|
||||
}
|
||||
|
||||
got, updatedAt, ok, err := store.Get(ctx, owner, agent)
|
||||
if err != nil {
|
||||
t.Fatalf("Get: %v", err)
|
||||
}
|
||||
if !ok {
|
||||
t.Fatal("Get: expected ok=true, got false")
|
||||
}
|
||||
if got != blob {
|
||||
t.Errorf("Get blob mismatch: got %q want %q", got, blob)
|
||||
}
|
||||
if updatedAt.IsZero() {
|
||||
t.Error("Get: expected non-zero updated_at")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCoreMemoryStore_SetReplacesWholesale(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewCoreMemoryStore(db, 2048)
|
||||
ctx := context.Background()
|
||||
|
||||
const owner = "1"
|
||||
const agent = "alpha"
|
||||
|
||||
if err := store.Set(ctx, owner, agent, "first version", "human"); err != nil {
|
||||
t.Fatalf("first Set: %v", err)
|
||||
}
|
||||
if err := store.Set(ctx, owner, agent, "second version", "dream:1"); err != nil {
|
||||
t.Fatalf("second Set: %v", err)
|
||||
}
|
||||
|
||||
got, _, ok, err := store.Get(ctx, owner, agent)
|
||||
if err != nil {
|
||||
t.Fatalf("Get: %v", err)
|
||||
}
|
||||
if !ok || got != "second version" {
|
||||
t.Errorf("expected wholesale replace, got ok=%v blob=%q", ok, got)
|
||||
}
|
||||
if strings.Contains(got, "first") {
|
||||
t.Errorf("expected merge to NOT happen; got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCoreMemoryStore_SetRejectsOverSize(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewCoreMemoryStore(db, 16)
|
||||
ctx := context.Background()
|
||||
|
||||
tooBig := strings.Repeat("x", 17)
|
||||
err := store.Set(ctx, "1", "alpha", tooBig, "human")
|
||||
if err == nil {
|
||||
t.Fatal("Set: expected ErrCoreMemoryTooLarge, got nil")
|
||||
}
|
||||
if !errors.Is(err, ErrCoreMemoryTooLarge) {
|
||||
t.Errorf("Set: expected ErrCoreMemoryTooLarge, got %v", err)
|
||||
}
|
||||
|
||||
// Exactly at-cap is fine.
|
||||
atCap := strings.Repeat("x", 16)
|
||||
if err := store.Set(ctx, "1", "alpha", atCap, "human"); err != nil {
|
||||
t.Errorf("Set at cap: unexpected error %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCoreMemoryStore_Delete(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewCoreMemoryStore(db, 2048)
|
||||
ctx := context.Background()
|
||||
|
||||
if err := store.Set(ctx, "1", "alpha", "blob", "human"); err != nil {
|
||||
t.Fatalf("Set: %v", err)
|
||||
}
|
||||
if err := store.Delete(ctx, "1", "alpha"); err != nil {
|
||||
t.Fatalf("Delete: %v", err)
|
||||
}
|
||||
_, _, ok, err := store.Get(ctx, "1", "alpha")
|
||||
if err != nil {
|
||||
t.Fatalf("Get after Delete: %v", err)
|
||||
}
|
||||
if ok {
|
||||
t.Error("Get after Delete: expected ok=false")
|
||||
}
|
||||
|
||||
// Delete on missing row is idempotent.
|
||||
if err := store.Delete(ctx, "1", "alpha"); err != nil {
|
||||
t.Errorf("Delete on missing: unexpected error %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCoreMemoryStore_OwnerScoping(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewCoreMemoryStore(db, 2048)
|
||||
ctx := context.Background()
|
||||
|
||||
if err := store.Set(ctx, "1", "shared-name", "H1 blob", "h1"); err != nil {
|
||||
t.Fatalf("Set H1: %v", err)
|
||||
}
|
||||
if err := store.Set(ctx, "2", "shared-name", "H2 blob", "h2"); err != nil {
|
||||
t.Fatalf("Set H2: %v", err)
|
||||
}
|
||||
|
||||
got1, _, ok1, err := store.Get(ctx, "1", "shared-name")
|
||||
if err != nil || !ok1 || got1 != "H1 blob" {
|
||||
t.Errorf("H1 Get: ok=%v got=%q err=%v", ok1, got1, err)
|
||||
}
|
||||
got2, _, ok2, err := store.Get(ctx, "2", "shared-name")
|
||||
if err != nil || !ok2 || got2 != "H2 blob" {
|
||||
t.Errorf("H2 Get: ok=%v got=%q err=%v", ok2, got2, err)
|
||||
}
|
||||
|
||||
// H1 cannot see H2's blob and vice versa — distinct PKs.
|
||||
if got1 == got2 {
|
||||
t.Error("owner scoping broken: H1 and H2 see the same blob")
|
||||
}
|
||||
|
||||
// List is scoped to a single owner.
|
||||
listH1, err := store.List(ctx, "1")
|
||||
if err != nil {
|
||||
t.Fatalf("List H1: %v", err)
|
||||
}
|
||||
if len(listH1) != 1 || listH1[0].OwnerID != "1" {
|
||||
t.Errorf("List H1: expected 1 row owned by 1, got %#v", listH1)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCoreMemoryStore_GetForInjection(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewCoreMemoryStore(db, 2048)
|
||||
ctx := context.Background()
|
||||
|
||||
// Missing row → "" + no error (NOT sql.ErrNoRows).
|
||||
blob, err := store.GetForInjection(ctx, "1", "missing-agent")
|
||||
if err != nil {
|
||||
t.Fatalf("GetForInjection on missing: unexpected error %v", err)
|
||||
}
|
||||
if blob != "" {
|
||||
t.Errorf("GetForInjection on missing: expected \"\" got %q", blob)
|
||||
}
|
||||
|
||||
// Present row → blob.
|
||||
if err := store.Set(ctx, "1", "alpha", "core blob", "human"); err != nil {
|
||||
t.Fatalf("Set: %v", err)
|
||||
}
|
||||
blob, err = store.GetForInjection(ctx, "1", "alpha")
|
||||
if err != nil {
|
||||
t.Fatalf("GetForInjection: %v", err)
|
||||
}
|
||||
if blob != "core blob" {
|
||||
t.Errorf("GetForInjection: got %q want %q", blob, "core blob")
|
||||
}
|
||||
|
||||
// Adapter satisfies search.CoreMemoryProvider implicitly.
|
||||
provider := NewCoreProvider(store)
|
||||
got, err := provider.Get(ctx, "1", "alpha")
|
||||
if err != nil || got != "core blob" {
|
||||
t.Errorf("CoreProvider.Get: got %q err=%v", got, err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// InjectionRecord is one row in the `memory_injections` 24-hour audit
|
||||
// ring. Each row captures what was attached to a single MCP tool
|
||||
// response so the owner can later answer "why did my agent know this?"
|
||||
// via the recent-injections debug surface (FR-025).
|
||||
type InjectionRecord struct {
|
||||
ID int64 `json:"id"`
|
||||
OwnerID string `json:"owner_id"`
|
||||
AgentName string `json:"agent_name"`
|
||||
ToolName string `json:"tool_name"`
|
||||
PacketSizeChars int `json:"packet_size_chars"`
|
||||
PacketItemsCount int `json:"packet_items_count"`
|
||||
MessageIDs []int64 `json:"message_ids"`
|
||||
CoreBlobIncluded bool `json:"core_blob_included"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// MemoryInjections is the audit-ring store for proactive injection
|
||||
// (data-model.md §`memory_injections`). Each row is best-effort:
|
||||
// failures are non-fatal for the injection request — callers should
|
||||
// log and continue.
|
||||
type MemoryInjections struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
// NewMemoryInjections wraps a *sql.DB.
|
||||
func NewMemoryInjections(db *sql.DB) *MemoryInjections {
|
||||
return &MemoryInjections{db: db}
|
||||
}
|
||||
|
||||
// Record inserts one injection row. `row.MessageIDs` is JSON-encoded.
|
||||
// `created_at` defaults to CURRENT_TIMESTAMP when zero.
|
||||
func (s *MemoryInjections) Record(ctx context.Context, row InjectionRecord) error {
|
||||
if s == nil || s.db == nil {
|
||||
return nil
|
||||
}
|
||||
ids := row.MessageIDs
|
||||
if ids == nil {
|
||||
ids = []int64{}
|
||||
}
|
||||
b, err := json.Marshal(ids)
|
||||
if err != nil {
|
||||
return fmt.Errorf("memory_injections: marshal message_ids: %w", err)
|
||||
}
|
||||
|
||||
if row.CreatedAt.IsZero() {
|
||||
_, err = s.db.ExecContext(ctx,
|
||||
`INSERT INTO memory_injections
|
||||
(owner_id, agent_name, tool_name, packet_size_chars,
|
||||
packet_items_count, message_ids, core_blob_included)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)`,
|
||||
row.OwnerID, row.AgentName, row.ToolName,
|
||||
row.PacketSizeChars, row.PacketItemsCount, string(b),
|
||||
row.CoreBlobIncluded,
|
||||
)
|
||||
} else {
|
||||
_, err = s.db.ExecContext(ctx,
|
||||
`INSERT INTO memory_injections
|
||||
(owner_id, agent_name, tool_name, packet_size_chars,
|
||||
packet_items_count, message_ids, core_blob_included, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
row.OwnerID, row.AgentName, row.ToolName,
|
||||
row.PacketSizeChars, row.PacketItemsCount, string(b),
|
||||
row.CoreBlobIncluded, row.CreatedAt.UTC(),
|
||||
)
|
||||
}
|
||||
if err != nil {
|
||||
return fmt.Errorf("memory_injections: insert: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Cleanup deletes rows older than `olderThan` ago. Returns the number
|
||||
// of rows removed. Safe to call from a periodic ticker.
|
||||
func (s *MemoryInjections) Cleanup(ctx context.Context, olderThan time.Duration) (int64, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return 0, nil
|
||||
}
|
||||
if olderThan <= 0 {
|
||||
return 0, nil
|
||||
}
|
||||
cutoff := time.Now().Add(-olderThan).UTC()
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`DELETE FROM memory_injections WHERE created_at < ?`, cutoff,
|
||||
)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("memory_injections: cleanup: %w", err)
|
||||
}
|
||||
affected, _ := res.RowsAffected()
|
||||
return affected, nil
|
||||
}
|
||||
|
||||
// ListRecent returns the most recent injections for one owner, newest
|
||||
// first, up to `limit`.
|
||||
func (s *MemoryInjections) ListRecent(ctx context.Context, ownerID string, limit int) ([]InjectionRecord, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return nil, nil
|
||||
}
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT id, owner_id, agent_name, tool_name, packet_size_chars,
|
||||
packet_items_count, message_ids, core_blob_included, created_at
|
||||
FROM memory_injections
|
||||
WHERE owner_id = ?
|
||||
ORDER BY created_at DESC, id DESC
|
||||
LIMIT ?`, ownerID, limit,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("memory_injections: list recent: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []InjectionRecord
|
||||
for rows.Next() {
|
||||
var rec InjectionRecord
|
||||
var idsJSON string
|
||||
if err := rows.Scan(
|
||||
&rec.ID, &rec.OwnerID, &rec.AgentName, &rec.ToolName,
|
||||
&rec.PacketSizeChars, &rec.PacketItemsCount, &idsJSON,
|
||||
&rec.CoreBlobIncluded, &rec.CreatedAt,
|
||||
); err != nil {
|
||||
return nil, fmt.Errorf("memory_injections: scan: %w", err)
|
||||
}
|
||||
if idsJSON == "" {
|
||||
rec.MessageIDs = []int64{}
|
||||
} else if err := json.Unmarshal([]byte(idsJSON), &rec.MessageIDs); err != nil {
|
||||
// Corrupt row — surface as empty rather than fail the whole listing.
|
||||
rec.MessageIDs = []int64{}
|
||||
}
|
||||
out = append(out, rec)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("memory_injections: iterate: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,166 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
_ "modernc.org/sqlite"
|
||||
)
|
||||
|
||||
func TestMemoryInjections_RecordAndList(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewMemoryInjections(db)
|
||||
ctx := context.Background()
|
||||
|
||||
rec := InjectionRecord{
|
||||
OwnerID: "1",
|
||||
AgentName: "research-mcpproxy",
|
||||
ToolName: "my_status",
|
||||
PacketSizeChars: 412,
|
||||
PacketItemsCount: 3,
|
||||
MessageIDs: []int64{1, 2, 3},
|
||||
CoreBlobIncluded: true,
|
||||
}
|
||||
if err := store.Record(ctx, rec); err != nil {
|
||||
t.Fatalf("Record: %v", err)
|
||||
}
|
||||
|
||||
got, err := store.ListRecent(ctx, "1", 10)
|
||||
if err != nil {
|
||||
t.Fatalf("ListRecent: %v", err)
|
||||
}
|
||||
if len(got) != 1 {
|
||||
t.Fatalf("ListRecent returned %d rows, want 1", len(got))
|
||||
}
|
||||
if got[0].PacketSizeChars != 412 || got[0].PacketItemsCount != 3 {
|
||||
t.Errorf("packet counters mismatch: %+v", got[0])
|
||||
}
|
||||
if len(got[0].MessageIDs) != 3 || got[0].MessageIDs[0] != 1 {
|
||||
t.Errorf("MessageIDs round-trip failed: %v", got[0].MessageIDs)
|
||||
}
|
||||
if !got[0].CoreBlobIncluded {
|
||||
t.Error("CoreBlobIncluded round-trip failed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryInjections_CleanupOnlyOldRows(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewMemoryInjections(db)
|
||||
ctx := context.Background()
|
||||
|
||||
now := time.Now().UTC()
|
||||
tests := []struct {
|
||||
name string
|
||||
offset time.Duration
|
||||
wantKept bool
|
||||
}{
|
||||
{"3 days old", -72 * time.Hour, false},
|
||||
{"36 hours old", -36 * time.Hour, false},
|
||||
{"23 hours old", -23 * time.Hour, true},
|
||||
{"30 minutes old", -30 * time.Minute, true},
|
||||
{"current", 0, true},
|
||||
}
|
||||
|
||||
// Seed 50 rows: 10 per bucket so we exercise the DELETE plan.
|
||||
for _, tc := range tests {
|
||||
for i := 0; i < 10; i++ {
|
||||
rec := InjectionRecord{
|
||||
OwnerID: "1",
|
||||
AgentName: "a",
|
||||
ToolName: "my_status",
|
||||
PacketSizeChars: 100,
|
||||
PacketItemsCount: 1,
|
||||
MessageIDs: []int64{int64(i)},
|
||||
CreatedAt: now.Add(tc.offset),
|
||||
}
|
||||
if err := store.Record(ctx, rec); err != nil {
|
||||
t.Fatalf("Record %s: %v", tc.name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
deleted, err := store.Cleanup(ctx, 24*time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("Cleanup: %v", err)
|
||||
}
|
||||
// 2 buckets older than 24h × 10 rows = 20 expected deletions.
|
||||
if deleted != 20 {
|
||||
t.Errorf("Cleanup removed %d rows, want 20", deleted)
|
||||
}
|
||||
|
||||
remaining, err := store.ListRecent(ctx, "1", 100)
|
||||
if err != nil {
|
||||
t.Fatalf("ListRecent: %v", err)
|
||||
}
|
||||
if len(remaining) != 30 {
|
||||
t.Errorf("after cleanup got %d rows, want 30", len(remaining))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryInjections_ListRecent_OwnerScoped(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewMemoryInjections(db)
|
||||
ctx := context.Background()
|
||||
|
||||
for _, owner := range []string{"1", "2", "3"} {
|
||||
for i := 0; i < 5; i++ {
|
||||
if err := store.Record(ctx, InjectionRecord{
|
||||
OwnerID: owner,
|
||||
AgentName: "a",
|
||||
ToolName: "my_status",
|
||||
PacketSizeChars: 100,
|
||||
PacketItemsCount: 1,
|
||||
MessageIDs: []int64{int64(i)},
|
||||
}); err != nil {
|
||||
t.Fatalf("Record owner=%s i=%d: %v", owner, i, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
got, err := store.ListRecent(ctx, "2", 100)
|
||||
if err != nil {
|
||||
t.Fatalf("ListRecent: %v", err)
|
||||
}
|
||||
if len(got) != 5 {
|
||||
t.Fatalf("owner=2 got %d rows, want 5", len(got))
|
||||
}
|
||||
for _, r := range got {
|
||||
if r.OwnerID != "2" {
|
||||
t.Errorf("found leaked row OwnerID=%q in owner=2 listing", r.OwnerID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryInjections_ListRecent_LimitOrdering(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
store := NewMemoryInjections(db)
|
||||
ctx := context.Background()
|
||||
|
||||
base := time.Now().UTC().Add(-1 * time.Hour)
|
||||
for i := 0; i < 10; i++ {
|
||||
if err := store.Record(ctx, InjectionRecord{
|
||||
OwnerID: "1",
|
||||
AgentName: "a",
|
||||
ToolName: "my_status",
|
||||
PacketSizeChars: 100,
|
||||
PacketItemsCount: 1,
|
||||
MessageIDs: []int64{int64(i)},
|
||||
CreatedAt: base.Add(time.Duration(i) * time.Minute),
|
||||
}); err != nil {
|
||||
t.Fatalf("Record %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
|
||||
got, err := store.ListRecent(ctx, "1", 3)
|
||||
if err != nil {
|
||||
t.Fatalf("ListRecent: %v", err)
|
||||
}
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("ListRecent limit=3 returned %d", len(got))
|
||||
}
|
||||
// Newest first: i=9, 8, 7. Validate MessageIDs[0] descends.
|
||||
if got[0].MessageIDs[0] != 9 || got[1].MessageIDs[0] != 8 || got[2].MessageIDs[0] != 7 {
|
||||
t.Errorf("ordering wrong: %d, %d, %d", got[0].MessageIDs[0], got[1].MessageIDs[0], got[2].MessageIDs[0])
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,264 @@
|
||||
// Memory-link store for feature 020 — typed directed edges between two
|
||||
// memory message ids (see data-model.md §`memory_links`).
|
||||
//
|
||||
// Three classes of relation types live in this table:
|
||||
//
|
||||
// - Semantic types written by the dream-agent via the
|
||||
// `memory_add_link` MCP tool: `refines`, `contradicts`, `examples`,
|
||||
// `related`.
|
||||
//
|
||||
// - Consolidation types written by `memory_mark_duplicate` and
|
||||
// `memory_supersede`: `duplicate_of`, `superseded_by`. These are NOT
|
||||
// valid arguments to `memory_add_link` — the contract reserves them
|
||||
// for the dedicated tools so the `memory_status` view can derive
|
||||
// soft-delete / supersede state from a single audit path.
|
||||
//
|
||||
// - Auto types written by the messaging layer (post-insert hook,
|
||||
// T035): `mention`, `reply_to`, `channel_cooccurrence`. These are
|
||||
// NOT valid arguments from any agent — only the `auto:<rule>`
|
||||
// created_by prefix may use them.
|
||||
//
|
||||
// Add() rejects type/actor mismatches with ErrLinkTypeReserved so the
|
||||
// MCP tools surface the contractual `relation_type_reserved` error
|
||||
// code cleanly.
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ErrLinkTypeReserved is returned when an actor tries to add a
|
||||
// relation_type they are not allowed to write directly (see file
|
||||
// comment for the actor/type matrix).
|
||||
var ErrLinkTypeReserved = errors.New("relation type reserved for another actor")
|
||||
|
||||
// Link is one row in `memory_links`.
|
||||
type Link struct {
|
||||
ID int64 `json:"id"`
|
||||
SrcMessageID int64 `json:"src_message_id"`
|
||||
DstMessageID int64 `json:"dst_message_id"`
|
||||
RelationType string `json:"relation_type"`
|
||||
OwnerID string `json:"owner_id"`
|
||||
CreatedBy string `json:"created_by"`
|
||||
Metadata map[string]any `json:"metadata,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// LinkStore wraps the `memory_links` table.
|
||||
type LinkStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
// NewLinkStore returns a store rooted at db.
|
||||
func NewLinkStore(db *sql.DB) *LinkStore {
|
||||
return &LinkStore{db: db}
|
||||
}
|
||||
|
||||
// Auto-generated link types (written only via auto:<rule> caller).
|
||||
var autoLinkTypes = map[string]struct{}{
|
||||
"mention": {},
|
||||
"reply_to": {},
|
||||
"channel_cooccurrence": {},
|
||||
}
|
||||
|
||||
// Reserved-for-tools relation types (written by memory_mark_duplicate /
|
||||
// memory_supersede via their own code paths, never by memory_add_link).
|
||||
var consolidationLinkTypes = map[string]struct{}{
|
||||
"duplicate_of": {},
|
||||
"superseded_by": {},
|
||||
}
|
||||
|
||||
// IsAutoLinkType reports whether relType is one of the auto-generated
|
||||
// link types written by the messaging post-insert hook.
|
||||
func IsAutoLinkType(relType string) bool {
|
||||
_, ok := autoLinkTypes[relType]
|
||||
return ok
|
||||
}
|
||||
|
||||
// Add inserts one row into `memory_links`. Reserved-type guarding:
|
||||
//
|
||||
// - When createdBy starts with `agent:`, the auto-types
|
||||
// (mention/reply_to/channel_cooccurrence) AND consolidation-types
|
||||
// (duplicate_of/superseded_by) are rejected with
|
||||
// ErrLinkTypeReserved. The MCP `memory_add_link` tool must surface
|
||||
// this as the contractual `relation_type_reserved` error.
|
||||
//
|
||||
// - When createdBy starts with `auto:`, only the auto-types are
|
||||
// allowed; semantic types and consolidation-types are rejected.
|
||||
//
|
||||
// - When createdBy starts with `human:` or any other prefix, no
|
||||
// reserved-type check is applied — admin tooling can backfill any
|
||||
// type for debugging / migration.
|
||||
func (s *LinkStore) Add(
|
||||
ctx context.Context,
|
||||
src, dst int64,
|
||||
relType, ownerID, createdBy string,
|
||||
metadata map[string]any,
|
||||
) (int64, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return 0, fmt.Errorf("link store: nil store")
|
||||
}
|
||||
if src == 0 || dst == 0 {
|
||||
return 0, fmt.Errorf("link store: src/dst message ids required")
|
||||
}
|
||||
if relType == "" || ownerID == "" || createdBy == "" {
|
||||
return 0, fmt.Errorf("link store: relation_type, owner_id, created_by required")
|
||||
}
|
||||
|
||||
switch {
|
||||
case strings.HasPrefix(createdBy, "agent:"):
|
||||
if _, banned := autoLinkTypes[relType]; banned {
|
||||
return 0, ErrLinkTypeReserved
|
||||
}
|
||||
if _, banned := consolidationLinkTypes[relType]; banned {
|
||||
return 0, ErrLinkTypeReserved
|
||||
}
|
||||
case strings.HasPrefix(createdBy, "auto:"):
|
||||
if _, ok := autoLinkTypes[relType]; !ok {
|
||||
return 0, ErrLinkTypeReserved
|
||||
}
|
||||
}
|
||||
|
||||
metaJSON := "{}"
|
||||
if metadata != nil {
|
||||
b, err := json.Marshal(metadata)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("link store: marshal metadata: %w", err)
|
||||
}
|
||||
metaJSON = string(b)
|
||||
}
|
||||
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO memory_links
|
||||
(src_message_id, dst_message_id, relation_type, owner_id, created_by, metadata)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`,
|
||||
src, dst, relType, ownerID, createdBy, metaJSON,
|
||||
)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("link store: insert: %w", err)
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// AddConsolidationLink inserts a `duplicate_of` or `superseded_by`
|
||||
// link without running the reserved-type guard. Only the dedicated
|
||||
// MCP tools `memory_mark_duplicate` and `memory_supersede` should
|
||||
// call this — the `memory_add_link` path uses Add() and rejects these
|
||||
// types per the contract.
|
||||
func (s *LinkStore) AddConsolidationLink(
|
||||
ctx context.Context,
|
||||
src, dst int64,
|
||||
relType, ownerID, createdBy string,
|
||||
metadata map[string]any,
|
||||
) (int64, error) {
|
||||
if relType != "duplicate_of" && relType != "superseded_by" {
|
||||
return 0, fmt.Errorf("link store: AddConsolidationLink rejects %q", relType)
|
||||
}
|
||||
metaJSON := "{}"
|
||||
if metadata != nil {
|
||||
b, err := json.Marshal(metadata)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("link store: marshal metadata: %w", err)
|
||||
}
|
||||
metaJSON = string(b)
|
||||
}
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO memory_links
|
||||
(src_message_id, dst_message_id, relation_type, owner_id, created_by, metadata)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`,
|
||||
src, dst, relType, ownerID, createdBy, metaJSON,
|
||||
)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("link store: insert: %w", err)
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// ListByMessage returns every link with the given message id as src OR
|
||||
// dst. Useful for both outgoing edges (reflection sources) and incoming
|
||||
// edges (what refines this).
|
||||
func (s *LinkStore) ListByMessage(ctx context.Context, msgID int64) ([]Link, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT id, src_message_id, dst_message_id, relation_type, owner_id,
|
||||
created_by, metadata, created_at
|
||||
FROM memory_links
|
||||
WHERE src_message_id = ? OR dst_message_id = ?
|
||||
ORDER BY id ASC`, msgID, msgID,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("link store: list by message: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanLinks(rows)
|
||||
}
|
||||
|
||||
// ListByOwner returns links for the given owner, optionally filtered to
|
||||
// a subset of relation types. `limit <= 0` defaults to 100.
|
||||
func (s *LinkStore) ListByOwner(
|
||||
ctx context.Context,
|
||||
ownerID string,
|
||||
types []string,
|
||||
limit int,
|
||||
) ([]Link, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return nil, nil
|
||||
}
|
||||
if limit <= 0 {
|
||||
limit = 100
|
||||
}
|
||||
q := `SELECT id, src_message_id, dst_message_id, relation_type, owner_id,
|
||||
created_by, metadata, created_at
|
||||
FROM memory_links
|
||||
WHERE owner_id = ?`
|
||||
args := []any{ownerID}
|
||||
if len(types) > 0 {
|
||||
placeholders := strings.Repeat("?,", len(types))
|
||||
placeholders = placeholders[:len(placeholders)-1]
|
||||
q += " AND relation_type IN (" + placeholders + ")"
|
||||
for _, t := range types {
|
||||
args = append(args, t)
|
||||
}
|
||||
}
|
||||
q += " ORDER BY id DESC LIMIT ?"
|
||||
args = append(args, limit)
|
||||
|
||||
rows, err := s.db.QueryContext(ctx, q, args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("link store: list by owner: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanLinks(rows)
|
||||
}
|
||||
|
||||
func scanLinks(rows *sql.Rows) ([]Link, error) {
|
||||
var out []Link
|
||||
for rows.Next() {
|
||||
var l Link
|
||||
var metaJSON string
|
||||
if err := rows.Scan(
|
||||
&l.ID, &l.SrcMessageID, &l.DstMessageID, &l.RelationType,
|
||||
&l.OwnerID, &l.CreatedBy, &metaJSON, &l.CreatedAt,
|
||||
); err != nil {
|
||||
return nil, fmt.Errorf("link store: scan: %w", err)
|
||||
}
|
||||
if metaJSON != "" && metaJSON != "{}" {
|
||||
_ = json.Unmarshal([]byte(metaJSON), &l.Metadata)
|
||||
}
|
||||
out = append(out, l)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("link store: iterate: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestLinkStore_AddValidTypesPerActor(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
s := NewLinkStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
// Agent may write semantic types.
|
||||
semantic := []string{"refines", "contradicts", "examples", "related"}
|
||||
src := int64(1)
|
||||
for i, rt := range semantic {
|
||||
dst := int64(100 + i)
|
||||
id, err := s.Add(ctx, src, dst, rt, "1", "agent:dream-algis:tok", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("agent semantic %s: %v", rt, err)
|
||||
}
|
||||
if id == 0 {
|
||||
t.Fatalf("agent semantic %s: zero id", rt)
|
||||
}
|
||||
}
|
||||
|
||||
// auto:<rule> may write auto types.
|
||||
autos := []string{"mention", "reply_to", "channel_cooccurrence"}
|
||||
for i, rt := range autos {
|
||||
dst := int64(200 + i)
|
||||
id, err := s.Add(ctx, src, dst, rt, "1", "auto:on-message-created", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("auto %s: %v", rt, err)
|
||||
}
|
||||
if id == 0 {
|
||||
t.Fatalf("auto %s: zero id", rt)
|
||||
}
|
||||
}
|
||||
|
||||
// human:<name> may write any type (no reserved-type guard).
|
||||
for i, rt := range []string{"refines", "duplicate_of", "mention"} {
|
||||
dst := int64(300 + i)
|
||||
if _, err := s.Add(ctx, src, dst, rt, "1", "human:algis", nil); err != nil {
|
||||
t.Errorf("human %s: unexpected err %v", rt, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinkStore_AddRejectsReservedFromAgent(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
s := NewLinkStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
reserved := []string{"mention", "reply_to", "channel_cooccurrence", "duplicate_of", "superseded_by"}
|
||||
for i, rt := range reserved {
|
||||
dst := int64(400 + i)
|
||||
_, err := s.Add(ctx, 1, dst, rt, "1", "agent:dream:tok", nil)
|
||||
if err == nil {
|
||||
t.Errorf("agent %s: expected ErrLinkTypeReserved, got nil", rt)
|
||||
continue
|
||||
}
|
||||
if !errors.Is(err, ErrLinkTypeReserved) {
|
||||
t.Errorf("agent %s: expected ErrLinkTypeReserved, got %v", rt, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinkStore_AddRejectsNonAutoFromAutoActor(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
s := NewLinkStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
for i, rt := range []string{"refines", "duplicate_of", "superseded_by", "related"} {
|
||||
dst := int64(500 + i)
|
||||
_, err := s.Add(ctx, 1, dst, rt, "1", "auto:something", nil)
|
||||
if !errors.Is(err, ErrLinkTypeReserved) {
|
||||
t.Errorf("auto actor %s: expected ErrLinkTypeReserved, got %v", rt, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinkStore_ListByMessageOwnerScoping(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
s := NewLinkStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
if _, err := s.Add(ctx, 1, 2, "refines", "1", "agent:dream:tok", nil); err != nil {
|
||||
t.Fatalf("Add owner=1: %v", err)
|
||||
}
|
||||
if _, err := s.Add(ctx, 1, 3, "refines", "2", "agent:dream:tok", nil); err != nil {
|
||||
t.Fatalf("Add owner=2: %v", err)
|
||||
}
|
||||
|
||||
// ListByMessage returns all links touching msg 1 regardless of owner —
|
||||
// owner-scoping happens in ListByOwner.
|
||||
ls, err := s.ListByMessage(ctx, 1)
|
||||
if err != nil {
|
||||
t.Fatalf("ListByMessage: %v", err)
|
||||
}
|
||||
if len(ls) != 2 {
|
||||
t.Errorf("ListByMessage: want 2, got %d", len(ls))
|
||||
}
|
||||
|
||||
owner1, err := s.ListByOwner(ctx, "1", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ListByOwner 1: %v", err)
|
||||
}
|
||||
if len(owner1) != 1 || owner1[0].DstMessageID != 2 {
|
||||
t.Errorf("ListByOwner 1: unexpected %v", owner1)
|
||||
}
|
||||
|
||||
owner2, err := s.ListByOwner(ctx, "2", []string{"refines"}, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ListByOwner 2: %v", err)
|
||||
}
|
||||
if len(owner2) != 1 || owner2[0].DstMessageID != 3 {
|
||||
t.Errorf("ListByOwner 2: unexpected %v", owner2)
|
||||
}
|
||||
|
||||
// Filtered ListByOwner with a non-matching type returns empty.
|
||||
none, err := s.ListByOwner(ctx, "1", []string{"contradicts"}, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ListByOwner filter: %v", err)
|
||||
}
|
||||
if len(none) != 0 {
|
||||
t.Errorf("ListByOwner filter: want 0, got %d", len(none))
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinkStore_AddMetadata(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
s := NewLinkStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
meta := map[string]any{"confidence": 0.84}
|
||||
id, err := s.Add(ctx, 10, 20, "refines", "1", "agent:dream:tok", meta)
|
||||
if err != nil {
|
||||
t.Fatalf("Add: %v", err)
|
||||
}
|
||||
|
||||
links, err := s.ListByMessage(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("ListByMessage: %v", err)
|
||||
}
|
||||
if len(links) != 1 || links[0].ID != id {
|
||||
t.Fatalf("unexpected listing: %v", links)
|
||||
}
|
||||
if got, _ := links[0].Metadata["confidence"].(float64); got != 0.84 {
|
||||
t.Errorf("metadata round-trip: got %v", links[0].Metadata)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,155 @@
|
||||
// Memory-pin store for feature 020 — owner-pinned message ids that
|
||||
// bypass the relevance floor on injection retrieval (data-model.md
|
||||
// §`memory_pins`). Pins are always set by the human owner; the
|
||||
// dream-agent does not write here. Pinned memories are loaded by
|
||||
// `search.BuildContextPacket` and overlaid on top of hybrid retrieval
|
||||
// so they appear even when their similarity score is below
|
||||
// SYNAPBUS_INJECTION_MIN_SCORE.
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Pin is one row in `memory_pins`.
|
||||
type Pin struct {
|
||||
OwnerID string `json:"owner_id"`
|
||||
MessageID int64 `json:"message_id"`
|
||||
PinnedBy string `json:"pinned_by"`
|
||||
Note string `json:"note,omitempty"`
|
||||
PinnedAt time.Time `json:"pinned_at"`
|
||||
}
|
||||
|
||||
// PinStore wraps the `memory_pins` table.
|
||||
type PinStore struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
// NewPinStore returns a store rooted at db.
|
||||
func NewPinStore(db *sql.DB) *PinStore {
|
||||
return &PinStore{db: db}
|
||||
}
|
||||
|
||||
// Pin pins (owner, msgID) for retrieval overlay. Idempotent — re-pinning
|
||||
// updates `pinned_by` / `note` in place (the primary key is the
|
||||
// (owner_id, message_id) tuple).
|
||||
func (s *PinStore) Pin(ctx context.Context, ownerID string, msgID int64, pinnedBy, note string) error {
|
||||
if s == nil || s.db == nil {
|
||||
return fmt.Errorf("pin store: nil store")
|
||||
}
|
||||
if ownerID == "" {
|
||||
return fmt.Errorf("pin store: empty owner_id")
|
||||
}
|
||||
if msgID == 0 {
|
||||
return fmt.Errorf("pin store: message_id required")
|
||||
}
|
||||
if pinnedBy == "" {
|
||||
pinnedBy = "human:" + ownerID
|
||||
}
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO memory_pins (owner_id, message_id, pinned_by, note)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(owner_id, message_id) DO UPDATE SET
|
||||
pinned_by = excluded.pinned_by,
|
||||
note = excluded.note,
|
||||
pinned_at = CURRENT_TIMESTAMP`,
|
||||
ownerID, msgID, pinnedBy, note,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("pin store: insert: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Unpin removes the (owner, msgID) row. Returns nil even when no row
|
||||
// matched — callers treat "unpinned" and "did not exist" the same way.
|
||||
func (s *PinStore) Unpin(ctx context.Context, ownerID string, msgID int64) error {
|
||||
if s == nil || s.db == nil {
|
||||
return nil
|
||||
}
|
||||
_, err := s.db.ExecContext(ctx,
|
||||
`DELETE FROM memory_pins WHERE owner_id = ? AND message_id = ?`,
|
||||
ownerID, msgID,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("pin store: delete: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ListForOwner returns the pinned message ids for the given owner. Used
|
||||
// by the injection overlay to splice these in regardless of search
|
||||
// score.
|
||||
func (s *PinStore) ListForOwner(ctx context.Context, ownerID string) ([]int64, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT message_id FROM memory_pins WHERE owner_id = ? ORDER BY pinned_at DESC`,
|
||||
ownerID,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("pin store: list ids: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []int64
|
||||
for rows.Next() {
|
||||
var id int64
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
return nil, fmt.Errorf("pin store: scan: %w", err)
|
||||
}
|
||||
out = append(out, id)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("pin store: iterate: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// pinProviderAdapter wraps *PinStore so it satisfies the
|
||||
// search.PinProvider interface without leaking the wider PinStore
|
||||
// surface area. Methods accept exactly the (ctx, ownerID) signature
|
||||
// search.InjectionOpts requires.
|
||||
type pinProviderAdapter struct{ store *PinStore }
|
||||
|
||||
// NewPinProvider returns a search.PinProvider over the given store.
|
||||
func NewPinProvider(store *PinStore) *pinProviderAdapter { return &pinProviderAdapter{store: store} }
|
||||
|
||||
// ListForOwner implements search.PinProvider.
|
||||
func (a *pinProviderAdapter) ListForOwner(ctx context.Context, ownerID string) ([]int64, error) {
|
||||
if a == nil || a.store == nil {
|
||||
return nil, nil
|
||||
}
|
||||
return a.store.ListForOwner(ctx, ownerID)
|
||||
}
|
||||
|
||||
// ListPinsForOwner returns full pin rows. Used by future audit UI.
|
||||
func (s *PinStore) ListPinsForOwner(ctx context.Context, ownerID string) ([]Pin, error) {
|
||||
if s == nil || s.db == nil {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := s.db.QueryContext(ctx,
|
||||
`SELECT owner_id, message_id, pinned_by, COALESCE(note, ''), pinned_at
|
||||
FROM memory_pins WHERE owner_id = ? ORDER BY pinned_at DESC`,
|
||||
ownerID,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("pin store: list: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []Pin
|
||||
for rows.Next() {
|
||||
var p Pin
|
||||
if err := rows.Scan(&p.OwnerID, &p.MessageID, &p.PinnedBy, &p.Note, &p.PinnedAt); err != nil {
|
||||
return nil, fmt.Errorf("pin store: scan: %w", err)
|
||||
}
|
||||
out = append(out, p)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("pin store: iterate: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestPinStore_PinUnpin(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
s := NewPinStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
if err := s.Pin(ctx, "1", 42, "human:algis", "always relevant"); err != nil {
|
||||
t.Fatalf("Pin: %v", err)
|
||||
}
|
||||
|
||||
ids, err := s.ListForOwner(ctx, "1")
|
||||
if err != nil {
|
||||
t.Fatalf("ListForOwner: %v", err)
|
||||
}
|
||||
if len(ids) != 1 || ids[0] != 42 {
|
||||
t.Errorf("ListForOwner: got %v want [42]", ids)
|
||||
}
|
||||
|
||||
// Re-pin updates note in place; no duplicate row.
|
||||
if err := s.Pin(ctx, "1", 42, "human:algis", "updated note"); err != nil {
|
||||
t.Fatalf("re-Pin: %v", err)
|
||||
}
|
||||
pins, err := s.ListPinsForOwner(ctx, "1")
|
||||
if err != nil {
|
||||
t.Fatalf("ListPinsForOwner: %v", err)
|
||||
}
|
||||
if len(pins) != 1 || pins[0].Note != "updated note" {
|
||||
t.Errorf("Re-pin: unexpected pins %v", pins)
|
||||
}
|
||||
|
||||
if err := s.Unpin(ctx, "1", 42); err != nil {
|
||||
t.Fatalf("Unpin: %v", err)
|
||||
}
|
||||
ids, _ = s.ListForOwner(ctx, "1")
|
||||
if len(ids) != 0 {
|
||||
t.Errorf("after Unpin: want empty, got %v", ids)
|
||||
}
|
||||
|
||||
// Unpin on missing row is a no-op.
|
||||
if err := s.Unpin(ctx, "1", 42); err != nil {
|
||||
t.Errorf("Unpin on missing: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPinStore_OwnerScoping(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
s := NewPinStore(db)
|
||||
ctx := context.Background()
|
||||
|
||||
if err := s.Pin(ctx, "1", 10, "human:1", ""); err != nil {
|
||||
t.Fatalf("Pin 1: %v", err)
|
||||
}
|
||||
if err := s.Pin(ctx, "1", 20, "human:1", ""); err != nil {
|
||||
t.Fatalf("Pin 1: %v", err)
|
||||
}
|
||||
if err := s.Pin(ctx, "2", 30, "human:2", ""); err != nil {
|
||||
t.Fatalf("Pin 2: %v", err)
|
||||
}
|
||||
|
||||
one, err := s.ListForOwner(ctx, "1")
|
||||
if err != nil {
|
||||
t.Fatalf("ListForOwner 1: %v", err)
|
||||
}
|
||||
if len(one) != 2 {
|
||||
t.Errorf("owner 1: want 2 pins, got %d", len(one))
|
||||
}
|
||||
|
||||
two, err := s.ListForOwner(ctx, "2")
|
||||
if err != nil {
|
||||
t.Fatalf("ListForOwner 2: %v", err)
|
||||
}
|
||||
if len(two) != 1 || two[0] != 30 {
|
||||
t.Errorf("owner 2: got %v want [30]", two)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
// memory_status query helpers — read the SQL view defined in migration
|
||||
// 028 and turn it into a map[message_id]MemoryStatus suitable for the
|
||||
// injection retrieval filter. The view itself derives state from
|
||||
// `memory_consolidation_jobs.actions` rows (data-model.md §`memory_status`)
|
||||
// so callers never have to mutate a status column directly.
|
||||
package messaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Memory status constants — value of `memory_status.status`.
|
||||
const (
|
||||
MemoryStatusActive = "active"
|
||||
MemoryStatusSoftDeleted = "soft_deleted"
|
||||
MemoryStatusSuperseded = "superseded"
|
||||
)
|
||||
|
||||
// MemoryStatus is one row derived from the `memory_status` view.
|
||||
// SupersededBy / SoftDeletedAt are nil when the message is active.
|
||||
type MemoryStatus struct {
|
||||
Status string `json:"status"`
|
||||
SupersededBy *int64 `json:"superseded_by,omitempty"`
|
||||
SoftDeletedAt *time.Time `json:"soft_deleted_at,omitempty"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
}
|
||||
|
||||
// MemoryStatuses returns the status of each message id in msgIDs. Ids
|
||||
// absent from the result map are implicitly `active` (the view only
|
||||
// contains rows that have at least one consolidation action against
|
||||
// them).
|
||||
func MemoryStatuses(ctx context.Context, db *sql.DB, msgIDs []int64) (map[int64]MemoryStatus, error) {
|
||||
out := map[int64]MemoryStatus{}
|
||||
if len(msgIDs) == 0 {
|
||||
return out, nil
|
||||
}
|
||||
if db == nil {
|
||||
return out, fmt.Errorf("memory status: nil db")
|
||||
}
|
||||
|
||||
placeholders := strings.Repeat("?,", len(msgIDs))
|
||||
placeholders = placeholders[:len(placeholders)-1]
|
||||
args := make([]any, 0, len(msgIDs))
|
||||
for _, id := range msgIDs {
|
||||
args = append(args, id)
|
||||
}
|
||||
|
||||
q := `SELECT message_id, status, superseded_by, soft_deleted_at, COALESCE(reason, '')
|
||||
FROM memory_status
|
||||
WHERE message_id IN (` + placeholders + `)`
|
||||
rows, err := db.QueryContext(ctx, q, args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("memory status: query: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
for rows.Next() {
|
||||
var (
|
||||
id int64
|
||||
status string
|
||||
supersede sql.NullInt64
|
||||
deletedAt sql.NullTime
|
||||
reason string
|
||||
)
|
||||
if err := rows.Scan(&id, &status, &supersede, &deletedAt, &reason); err != nil {
|
||||
return nil, fmt.Errorf("memory status: scan: %w", err)
|
||||
}
|
||||
ms := MemoryStatus{Status: status, Reason: reason}
|
||||
if supersede.Valid {
|
||||
v := supersede.Int64
|
||||
ms.SupersededBy = &v
|
||||
}
|
||||
if deletedAt.Valid {
|
||||
t := deletedAt.Time
|
||||
ms.SoftDeletedAt = &t
|
||||
}
|
||||
out[id] = ms
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("memory status: iterate: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// StatusByID returns just the string status for each id; callers that
|
||||
// only need active/non-active (e.g. the injection retrieval filter) can
|
||||
// use this to avoid pulling in MemoryStatus's optional fields.
|
||||
func StatusByID(ctx context.Context, db *sql.DB, msgIDs []int64) (map[int64]string, error) {
|
||||
full, err := MemoryStatuses(ctx, db, msgIDs)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make(map[int64]string, len(full))
|
||||
for id, st := range full {
|
||||
out[id] = st.Status
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
+60
-697
@@ -14,13 +14,15 @@ import (
|
||||
)
|
||||
|
||||
// StalemateConfig holds stalemate detection settings.
|
||||
//
|
||||
// The historical reminder/escalation knobs (ReminderAfter, EscalateAfter)
|
||||
// were removed when SynapBus moved to internal-only mode (no human
|
||||
// approval loop). See migration 027_remove_approval_noise.sql.
|
||||
type StalemateConfig struct {
|
||||
// ProcessingTimeout is how long a message can stay in "processing" before auto-fail (default 24h).
|
||||
// ProcessingTimeout is how long a message can stay in "processing"
|
||||
// before auto-fail (default 24h). Protects the inbox queue from
|
||||
// agents that crash after claiming a message.
|
||||
ProcessingTimeout time.Duration
|
||||
// ReminderAfter is how long a pending DM waits before a system reminder is sent (default 4h).
|
||||
ReminderAfter time.Duration
|
||||
// EscalateAfter is how long a pending DM waits before escalation to #approvals (default 48h).
|
||||
EscalateAfter time.Duration
|
||||
// Interval is how often the worker checks for stale messages (default 15m).
|
||||
Interval time.Duration
|
||||
}
|
||||
@@ -29,8 +31,6 @@ type StalemateConfig struct {
|
||||
func DefaultStalemateConfig() StalemateConfig {
|
||||
return StalemateConfig{
|
||||
ProcessingTimeout: 24 * time.Hour,
|
||||
ReminderAfter: 4 * time.Hour,
|
||||
EscalateAfter: 48 * time.Hour,
|
||||
Interval: 15 * time.Minute,
|
||||
}
|
||||
}
|
||||
@@ -43,7 +43,6 @@ func parseDurationWithDays(s string) (time.Duration, error) {
|
||||
return 0, fmt.Errorf("empty duration string")
|
||||
}
|
||||
|
||||
// Try "Nd" format (days)
|
||||
if strings.HasSuffix(s, "d") {
|
||||
days, err := strconv.Atoi(strings.TrimSuffix(s, "d"))
|
||||
if err == nil && days > 0 {
|
||||
@@ -51,7 +50,6 @@ func parseDurationWithDays(s string) (time.Duration, error) {
|
||||
}
|
||||
}
|
||||
|
||||
// Try standard Go duration
|
||||
return time.ParseDuration(s)
|
||||
}
|
||||
|
||||
@@ -64,16 +62,6 @@ func ParseStalemateConfig() StalemateConfig {
|
||||
cfg.ProcessingTimeout = d
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_STALEMATE_REMINDER_AFTER"); v != "" {
|
||||
if d, err := parseDurationWithDays(v); err == nil && d > 0 {
|
||||
cfg.ReminderAfter = d
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_STALEMATE_ESCALATE_AFTER"); v != "" {
|
||||
if d, err := parseDurationWithDays(v); err == nil && d > 0 {
|
||||
cfg.EscalateAfter = d
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("SYNAPBUS_STALEMATE_INTERVAL"); v != "" {
|
||||
if d, err := parseDurationWithDays(v); err == nil && d > 0 {
|
||||
cfg.Interval = d
|
||||
@@ -83,32 +71,37 @@ func ParseStalemateConfig() StalemateConfig {
|
||||
return cfg
|
||||
}
|
||||
|
||||
// ChannelLookup provides channel lookup by name without importing the channels package.
|
||||
type ChannelLookup interface {
|
||||
// GetChannelIDByName returns a channel ID by name, or 0 if not found.
|
||||
GetChannelIDByName(ctx context.Context, name string) (int64, error)
|
||||
}
|
||||
|
||||
// StalemateWorker periodically checks for and handles stale messages.
|
||||
// StalemateWorker periodically auto-fails messages whose claim has timed out.
|
||||
type StalemateWorker struct {
|
||||
db *sql.DB
|
||||
msgService *MessagingService
|
||||
channelLookup ChannelLookup
|
||||
config StalemateConfig
|
||||
logger *slog.Logger
|
||||
done chan struct{}
|
||||
wg sync.WaitGroup
|
||||
|
||||
// memoryInjections, when non-nil, drives an hourly cleanup of the
|
||||
// 24h proactive-injection audit ring (feature 020). Plumbed via
|
||||
// SetMemoryInjections after construction so adding the feature is
|
||||
// non-breaking for existing call sites.
|
||||
memoryInjections *MemoryInjections
|
||||
tickCount int
|
||||
}
|
||||
|
||||
// SetMemoryInjections registers the audit-ring store the worker will
|
||||
// cleanup hourly. Pass nil to disable; safe to call before Start.
|
||||
func (w *StalemateWorker) SetMemoryInjections(store *MemoryInjections) {
|
||||
w.memoryInjections = store
|
||||
}
|
||||
|
||||
// NewStalemateWorker creates a new stalemate detection worker.
|
||||
func NewStalemateWorker(db *sql.DB, msgService *MessagingService, channelLookup ChannelLookup, config StalemateConfig) *StalemateWorker {
|
||||
func NewStalemateWorker(db *sql.DB, msgService *MessagingService, config StalemateConfig) *StalemateWorker {
|
||||
return &StalemateWorker{
|
||||
db: db,
|
||||
msgService: msgService,
|
||||
channelLookup: channelLookup,
|
||||
config: config,
|
||||
logger: slog.Default().With("component", "stalemate-worker"),
|
||||
done: make(chan struct{}),
|
||||
db: db,
|
||||
msgService: msgService,
|
||||
config: config,
|
||||
logger: slog.Default().With("component", "stalemate-worker"),
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -120,8 +113,6 @@ func (w *StalemateWorker) Start() {
|
||||
w.logger.Info("stalemate worker started",
|
||||
"interval", w.config.Interval.String(),
|
||||
"processing_timeout", w.config.ProcessingTimeout.String(),
|
||||
"reminder_after", w.config.ReminderAfter.String(),
|
||||
"escalate_after", w.config.EscalateAfter.String(),
|
||||
)
|
||||
|
||||
ticker := time.NewTicker(w.config.Interval)
|
||||
@@ -132,6 +123,7 @@ func (w *StalemateWorker) Start() {
|
||||
case <-ticker.C:
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
||||
w.checkStaleMessages(ctx)
|
||||
w.maybeCleanupInjections(ctx)
|
||||
cancel()
|
||||
case <-w.done:
|
||||
w.logger.Info("stalemate worker stopped")
|
||||
@@ -147,23 +139,40 @@ func (w *StalemateWorker) Stop() {
|
||||
w.wg.Wait()
|
||||
}
|
||||
|
||||
// checkStaleMessages runs all stalemate checks.
|
||||
// checkStaleMessages runs the auto-fail check for timed-out claimed messages.
|
||||
func (w *StalemateWorker) checkStaleMessages(ctx context.Context) {
|
||||
failed := w.failTimedOutProcessing(ctx)
|
||||
reminded := w.sendPendingReminders(ctx)
|
||||
escalated := w.escalatePendingMessages(ctx)
|
||||
if failed := w.failTimedOutProcessing(ctx); failed > 0 {
|
||||
w.logger.Info("stalemate check complete", "auto_failed", failed)
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 2: Workflow stalemate checks for channel messages
|
||||
wfReminded, wfEscalated := w.checkWorkflowStalemates(ctx)
|
||||
|
||||
if failed > 0 || reminded > 0 || escalated > 0 || wfReminded > 0 || wfEscalated > 0 {
|
||||
w.logger.Info("stalemate check complete",
|
||||
"auto_failed", failed,
|
||||
"reminders_sent", reminded,
|
||||
"escalations_sent", escalated,
|
||||
"workflow_reminders", wfReminded,
|
||||
"workflow_escalations", wfEscalated,
|
||||
)
|
||||
// maybeCleanupInjections piggybacks an hourly cleanup of the 24h
|
||||
// proactive-injection audit ring (feature 020) on the stalemate worker
|
||||
// tick. With the default 15-minute Interval, the cleanup fires every
|
||||
// 4th tick (i.e. ~1h). A no-op when SetMemoryInjections has not been
|
||||
// called or the store is nil.
|
||||
func (w *StalemateWorker) maybeCleanupInjections(ctx context.Context) {
|
||||
if w.memoryInjections == nil {
|
||||
return
|
||||
}
|
||||
w.tickCount++
|
||||
// Fire roughly hourly. With Interval=15m the modulus matches the
|
||||
// 4-tick cadence called out in spec 020 T017. For non-default
|
||||
// intervals it still fires hourly-ish on a best-effort basis.
|
||||
ticksPerHour := int(time.Hour / w.config.Interval)
|
||||
if ticksPerHour <= 0 {
|
||||
ticksPerHour = 1
|
||||
}
|
||||
if w.tickCount%ticksPerHour != 0 {
|
||||
return
|
||||
}
|
||||
deleted, err := w.memoryInjections.Cleanup(ctx, 24*time.Hour)
|
||||
if err != nil {
|
||||
w.logger.Warn("memory_injections cleanup failed", "error", err)
|
||||
return
|
||||
}
|
||||
if deleted > 0 {
|
||||
w.logger.Info("memory_injections cleanup", "deleted", deleted)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -242,9 +251,6 @@ func (w *StalemateWorker) failTimedOutProcessing(ctx context.Context) int64 {
|
||||
}
|
||||
affected, _ := res.RowsAffected()
|
||||
if affected == 0 {
|
||||
// Message was reclaimed, completed, or otherwise moved out of the
|
||||
// stale window between SELECT and UPDATE. Skip silently — the
|
||||
// next worker tick will re-evaluate.
|
||||
w.logger.Debug("stale processing message no longer stale; skipped",
|
||||
"message_id", dm.ID,
|
||||
"claimed_by", dm.ClaimedBy,
|
||||
@@ -261,646 +267,3 @@ func (w *StalemateWorker) failTimedOutProcessing(ctx context.Context) int64 {
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
// sendPendingReminders sends system DM reminders for pending messages older than ReminderAfter.
|
||||
func (w *StalemateWorker) sendPendingReminders(ctx context.Context) int64 {
|
||||
cutoff := time.Now().Add(-w.config.ReminderAfter)
|
||||
|
||||
rows, err := w.db.QueryContext(ctx,
|
||||
`SELECT id, from_agent, to_agent, body, created_at
|
||||
FROM messages
|
||||
WHERE status = 'pending'
|
||||
AND to_agent IS NOT NULL
|
||||
AND to_agent != ''
|
||||
AND from_agent != 'system'
|
||||
AND to_agent != 'system'
|
||||
AND created_at < ?`,
|
||||
cutoff,
|
||||
)
|
||||
if err != nil {
|
||||
w.logger.Error("query pending reminder candidates failed", "error", err)
|
||||
return 0
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
type pendingMsg struct {
|
||||
ID int64
|
||||
FromAgent string
|
||||
ToAgent string
|
||||
Body string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
var pending []pendingMsg
|
||||
for rows.Next() {
|
||||
var pm pendingMsg
|
||||
if err := rows.Scan(&pm.ID, &pm.FromAgent, &pm.ToAgent, &pm.Body, &pm.CreatedAt); err != nil {
|
||||
w.logger.Error("scan pending message failed", "error", err)
|
||||
continue
|
||||
}
|
||||
pending = append(pending, pm)
|
||||
}
|
||||
|
||||
count := int64(0)
|
||||
for _, pm := range pending {
|
||||
// Check if a reminder already exists for this message
|
||||
if w.reminderExists(ctx, pm.ID, pm.ToAgent) {
|
||||
continue
|
||||
}
|
||||
|
||||
age := formatAge(time.Since(pm.CreatedAt))
|
||||
truncBody := truncate(pm.Body, 100)
|
||||
|
||||
body := fmt.Sprintf(
|
||||
"**Reminder**: You have a pending message from %s (%s old). Message: \"%s\". Please claim and process it.",
|
||||
pm.FromAgent, age, truncBody,
|
||||
)
|
||||
|
||||
_, err := w.msgService.SendMessage(ctx, "system", pm.ToAgent, body, SendOptions{
|
||||
Subject: fmt.Sprintf("stalemate-reminder:%d", pm.ID),
|
||||
Priority: 7,
|
||||
Metadata: fmt.Sprintf(`{"stalemate_reminder_for":%d}`, pm.ID),
|
||||
})
|
||||
if err != nil {
|
||||
w.logger.Error("send stalemate reminder failed",
|
||||
"message_id", pm.ID,
|
||||
"to_agent", pm.ToAgent,
|
||||
"error", err,
|
||||
)
|
||||
continue
|
||||
}
|
||||
w.logger.Info("sent stalemate reminder",
|
||||
"message_id", pm.ID,
|
||||
"to_agent", pm.ToAgent,
|
||||
"from_agent", pm.FromAgent,
|
||||
"age", age,
|
||||
)
|
||||
count++
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
// escalatePendingMessages escalates pending messages older than EscalateAfter to #approvals.
|
||||
func (w *StalemateWorker) escalatePendingMessages(ctx context.Context) int64 {
|
||||
cutoff := time.Now().Add(-w.config.EscalateAfter)
|
||||
|
||||
rows, err := w.db.QueryContext(ctx,
|
||||
`SELECT id, from_agent, to_agent, body, created_at
|
||||
FROM messages
|
||||
WHERE status = 'pending'
|
||||
AND to_agent IS NOT NULL
|
||||
AND to_agent != ''
|
||||
AND from_agent != 'system'
|
||||
AND to_agent != 'system'
|
||||
AND created_at < ?`,
|
||||
cutoff,
|
||||
)
|
||||
if err != nil {
|
||||
w.logger.Error("query escalation candidates failed", "error", err)
|
||||
return 0
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
type pendingMsg struct {
|
||||
ID int64
|
||||
FromAgent string
|
||||
ToAgent string
|
||||
Body string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
var pending []pendingMsg
|
||||
for rows.Next() {
|
||||
var pm pendingMsg
|
||||
if err := rows.Scan(&pm.ID, &pm.FromAgent, &pm.ToAgent, &pm.Body, &pm.CreatedAt); err != nil {
|
||||
w.logger.Error("scan escalation candidate failed", "error", err)
|
||||
continue
|
||||
}
|
||||
pending = append(pending, pm)
|
||||
}
|
||||
|
||||
if len(pending) == 0 {
|
||||
return 0
|
||||
}
|
||||
|
||||
// Look up #approvals channel
|
||||
channelID, err := w.channelLookup.GetChannelIDByName(ctx, "approvals")
|
||||
if err != nil {
|
||||
w.logger.Warn("cannot escalate: #approvals channel not found", "error", err)
|
||||
return 0
|
||||
}
|
||||
|
||||
count := int64(0)
|
||||
for _, pm := range pending {
|
||||
// Check if already escalated
|
||||
if w.escalationExists(ctx, pm.ID) {
|
||||
continue
|
||||
}
|
||||
|
||||
age := formatAge(time.Since(pm.CreatedAt))
|
||||
truncBody := truncate(pm.Body, 100)
|
||||
|
||||
body := fmt.Sprintf(
|
||||
"**ESCALATION**: Pending message for @%s from %s has been unprocessed for %s. Message: \"%s\". Manual intervention may be required.",
|
||||
pm.ToAgent, pm.FromAgent, age, truncBody,
|
||||
)
|
||||
|
||||
_, err := w.msgService.SendMessage(ctx, "system", "", body, SendOptions{
|
||||
Subject: fmt.Sprintf("stalemate-escalation:%d", pm.ID),
|
||||
Priority: 9,
|
||||
Metadata: fmt.Sprintf(`{"stalemate_escalation_for":%d}`, pm.ID),
|
||||
ChannelID: &channelID,
|
||||
})
|
||||
if err != nil {
|
||||
w.logger.Error("send escalation to #approvals failed",
|
||||
"message_id", pm.ID,
|
||||
"error", err,
|
||||
)
|
||||
continue
|
||||
}
|
||||
w.logger.Info("escalated stale message to #approvals",
|
||||
"message_id", pm.ID,
|
||||
"to_agent", pm.ToAgent,
|
||||
"from_agent", pm.FromAgent,
|
||||
"age", age,
|
||||
)
|
||||
count++
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
// reminderExists checks if a system reminder already exists for a given message ID.
|
||||
func (w *StalemateWorker) reminderExists(ctx context.Context, messageID int64, toAgent string) bool {
|
||||
var count int
|
||||
err := w.db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages
|
||||
WHERE from_agent = 'system'
|
||||
AND to_agent = ?
|
||||
AND metadata LIKE ?`,
|
||||
toAgent, fmt.Sprintf(`%%"stalemate_reminder_for":%d%%`, messageID),
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return count > 0
|
||||
}
|
||||
|
||||
// escalationExists checks if an escalation already exists for a given message ID.
|
||||
func (w *StalemateWorker) escalationExists(ctx context.Context, messageID int64) bool {
|
||||
var count int
|
||||
err := w.db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages
|
||||
WHERE from_agent = 'system'
|
||||
AND metadata LIKE ?`,
|
||||
fmt.Sprintf(`%%"stalemate_escalation_for":%d%%`, messageID),
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return count > 0
|
||||
}
|
||||
|
||||
// workflowChannel holds channel info relevant to workflow stalemate checking.
|
||||
type workflowChannel struct {
|
||||
ID int64
|
||||
Name string
|
||||
StalemateRemindAfter string
|
||||
StalemateEscalateAfter string
|
||||
}
|
||||
|
||||
// staleWorkflowMsg holds info about a channel message in a stale workflow state.
|
||||
type staleWorkflowMsg struct {
|
||||
ID int64
|
||||
Body string
|
||||
FromAgent string
|
||||
ChannelID int64
|
||||
Channel string
|
||||
State string
|
||||
StateAge time.Duration
|
||||
}
|
||||
|
||||
// checkWorkflowStalemates scans workflow-enabled channels for messages stuck in
|
||||
// non-terminal workflow states (proposed, approved, in_progress) and sends
|
||||
// reminders to channel members or escalates to #approvals.
|
||||
func (w *StalemateWorker) checkWorkflowStalemates(ctx context.Context) (reminded int64, escalated int64) {
|
||||
// Step 1: Find all workflow-enabled channels
|
||||
channels, err := w.listWorkflowChannels(ctx)
|
||||
if err != nil {
|
||||
w.logger.Error("list workflow channels failed", "error", err)
|
||||
return 0, 0
|
||||
}
|
||||
if len(channels) == 0 {
|
||||
return 0, 0
|
||||
}
|
||||
|
||||
for _, ch := range channels {
|
||||
remindTimeout, err := parseDurationWithDays(ch.StalemateRemindAfter)
|
||||
if err != nil || remindTimeout <= 0 {
|
||||
remindTimeout = 24 * time.Hour // default
|
||||
}
|
||||
escalateTimeout, err := parseDurationWithDays(ch.StalemateEscalateAfter)
|
||||
if err != nil || escalateTimeout <= 0 {
|
||||
escalateTimeout = 72 * time.Hour // default
|
||||
}
|
||||
|
||||
// Step 2: Find messages in non-terminal workflow states
|
||||
staleMessages, err := w.findStaleWorkflowMessages(ctx, ch)
|
||||
if err != nil {
|
||||
w.logger.Error("find stale workflow messages failed",
|
||||
"channel", ch.Name,
|
||||
"error", err,
|
||||
)
|
||||
continue
|
||||
}
|
||||
|
||||
for _, msg := range staleMessages {
|
||||
// Step 3: Check escalation first (longer timeout)
|
||||
if msg.StateAge >= escalateTimeout {
|
||||
if w.workflowEscalationExists(ctx, msg.ID) {
|
||||
continue
|
||||
}
|
||||
if w.sendWorkflowEscalation(ctx, msg) {
|
||||
escalated++
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// Step 4: Check reminder (shorter timeout)
|
||||
if msg.StateAge >= remindTimeout {
|
||||
if w.workflowReminderExists(ctx, msg.ID) {
|
||||
continue
|
||||
}
|
||||
r := w.sendWorkflowReminders(ctx, msg, ch.ID)
|
||||
reminded += r
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return reminded, escalated
|
||||
}
|
||||
|
||||
// listWorkflowChannels returns all channels that have workflow_enabled = true.
|
||||
func (w *StalemateWorker) listWorkflowChannels(ctx context.Context) ([]workflowChannel, error) {
|
||||
rows, err := w.db.QueryContext(ctx,
|
||||
`SELECT id, name, stalemate_remind_after, stalemate_escalate_after
|
||||
FROM channels
|
||||
WHERE workflow_enabled = 1`)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("query workflow channels: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var channels []workflowChannel
|
||||
for rows.Next() {
|
||||
var ch workflowChannel
|
||||
if err := rows.Scan(&ch.ID, &ch.Name, &ch.StalemateRemindAfter, &ch.StalemateEscalateAfter); err != nil {
|
||||
return nil, fmt.Errorf("scan workflow channel: %w", err)
|
||||
}
|
||||
channels = append(channels, ch)
|
||||
}
|
||||
return channels, rows.Err()
|
||||
}
|
||||
|
||||
// findStaleWorkflowMessages finds channel messages in non-terminal workflow states
|
||||
// and computes how long they have been in their current state.
|
||||
func (w *StalemateWorker) findStaleWorkflowMessages(ctx context.Context, ch workflowChannel) ([]staleWorkflowMsg, error) {
|
||||
// Get all messages in this channel that could be in a workflow state.
|
||||
// We fetch messages and their reactions, then compute state in Go.
|
||||
rows, err := w.db.QueryContext(ctx,
|
||||
`SELECT m.id, m.body, m.from_agent, m.created_at
|
||||
FROM messages m
|
||||
WHERE m.channel_id = ?
|
||||
AND m.from_agent != 'system'
|
||||
ORDER BY m.created_at ASC`,
|
||||
ch.ID,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("query channel messages: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
type chanMsg struct {
|
||||
ID int64
|
||||
Body string
|
||||
FromAgent string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
var msgs []chanMsg
|
||||
for rows.Next() {
|
||||
var m chanMsg
|
||||
if err := rows.Scan(&m.ID, &m.Body, &m.FromAgent, &m.CreatedAt); err != nil {
|
||||
return nil, fmt.Errorf("scan channel message: %w", err)
|
||||
}
|
||||
msgs = append(msgs, m)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(msgs) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// Batch-fetch reactions for all messages
|
||||
msgIDs := make([]int64, len(msgs))
|
||||
for i, m := range msgs {
|
||||
msgIDs[i] = m.ID
|
||||
}
|
||||
|
||||
reactionsMap, err := w.getReactionsByMessageIDs(ctx, msgIDs)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get reactions: %w", err)
|
||||
}
|
||||
|
||||
now := time.Now()
|
||||
var stale []staleWorkflowMsg
|
||||
for _, m := range msgs {
|
||||
reactions := reactionsMap[m.ID]
|
||||
state := computeWorkflowStateFromReactions(reactions)
|
||||
|
||||
// Skip terminal states
|
||||
if isTerminalWorkflowState(state) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Determine the "state age": how long since the state was entered.
|
||||
// If reactions exist, use the most recent reaction's created_at.
|
||||
// If no reactions (proposed state), use the message's created_at.
|
||||
stateEnteredAt := m.CreatedAt
|
||||
if len(reactions) > 0 {
|
||||
// Find the most recent reaction
|
||||
for _, r := range reactions {
|
||||
if r.CreatedAt.After(stateEnteredAt) {
|
||||
stateEnteredAt = r.CreatedAt
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
stale = append(stale, staleWorkflowMsg{
|
||||
ID: m.ID,
|
||||
Body: m.Body,
|
||||
FromAgent: m.FromAgent,
|
||||
ChannelID: ch.ID,
|
||||
Channel: ch.Name,
|
||||
State: state,
|
||||
StateAge: now.Sub(stateEnteredAt),
|
||||
})
|
||||
}
|
||||
|
||||
return stale, nil
|
||||
}
|
||||
|
||||
// reactionRow holds a raw reaction row for workflow state computation.
|
||||
type reactionRow struct {
|
||||
Reaction string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
// getReactionsByMessageIDs fetches reactions for a batch of message IDs.
|
||||
func (w *StalemateWorker) getReactionsByMessageIDs(ctx context.Context, messageIDs []int64) (map[int64][]reactionRow, error) {
|
||||
if len(messageIDs) == 0 {
|
||||
return map[int64][]reactionRow{}, nil
|
||||
}
|
||||
|
||||
placeholders := make([]string, len(messageIDs))
|
||||
args := make([]any, len(messageIDs))
|
||||
for i, id := range messageIDs {
|
||||
placeholders[i] = "?"
|
||||
args[i] = id
|
||||
}
|
||||
|
||||
query := fmt.Sprintf(
|
||||
`SELECT message_id, reaction, created_at
|
||||
FROM message_reactions
|
||||
WHERE message_id IN (%s)
|
||||
ORDER BY created_at ASC`,
|
||||
strings.Join(placeholders, ","),
|
||||
)
|
||||
|
||||
rows, err := w.db.QueryContext(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("query reactions: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
result := make(map[int64][]reactionRow)
|
||||
for rows.Next() {
|
||||
var msgID int64
|
||||
var r reactionRow
|
||||
if err := rows.Scan(&msgID, &r.Reaction, &r.CreatedAt); err != nil {
|
||||
return nil, fmt.Errorf("scan reaction: %w", err)
|
||||
}
|
||||
result[msgID] = append(result[msgID], r)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
// computeWorkflowStateFromReactions derives workflow state from raw reaction rows.
|
||||
// Mirrors the logic in reactions.ComputeWorkflowState without importing that package.
|
||||
func computeWorkflowStateFromReactions(reactions []reactionRow) string {
|
||||
if len(reactions) == 0 {
|
||||
return "proposed"
|
||||
}
|
||||
|
||||
// Reaction priority (same as reactions.reactionPriority)
|
||||
priority := map[string]int{
|
||||
"approve": 2,
|
||||
"in_progress": 3,
|
||||
"reject": 4,
|
||||
"done": 5,
|
||||
"published": 6,
|
||||
}
|
||||
|
||||
// Reaction-to-state mapping (same as reactions.reactionToState)
|
||||
toState := map[string]string{
|
||||
"approve": "approved",
|
||||
"reject": "rejected",
|
||||
"in_progress": "in_progress",
|
||||
"done": "done",
|
||||
"published": "published",
|
||||
}
|
||||
|
||||
highestPriority := 0
|
||||
highestState := "proposed"
|
||||
|
||||
for _, r := range reactions {
|
||||
if p, ok := priority[r.Reaction]; ok && p > highestPriority {
|
||||
highestPriority = p
|
||||
highestState = toState[r.Reaction]
|
||||
}
|
||||
}
|
||||
|
||||
return highestState
|
||||
}
|
||||
|
||||
// isTerminalWorkflowState returns true if the state should not trigger stalemate checks.
|
||||
func isTerminalWorkflowState(state string) bool {
|
||||
switch state {
|
||||
case "rejected", "done", "published":
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// sendWorkflowReminders sends DMs to channel members about a stale workflow message.
|
||||
func (w *StalemateWorker) sendWorkflowReminders(ctx context.Context, msg staleWorkflowMsg, channelID int64) int64 {
|
||||
// Get channel members
|
||||
rows, err := w.db.QueryContext(ctx,
|
||||
`SELECT agent_name FROM channel_members WHERE channel_id = ?`,
|
||||
channelID,
|
||||
)
|
||||
if err != nil {
|
||||
w.logger.Error("query channel members for workflow reminder failed",
|
||||
"channel_id", channelID,
|
||||
"error", err,
|
||||
)
|
||||
return 0
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var members []string
|
||||
for rows.Next() {
|
||||
var name string
|
||||
if err := rows.Scan(&name); err != nil {
|
||||
continue
|
||||
}
|
||||
members = append(members, name)
|
||||
}
|
||||
|
||||
age := formatAge(msg.StateAge)
|
||||
truncBody := truncate(msg.Body, 100)
|
||||
count := int64(0)
|
||||
|
||||
for _, member := range members {
|
||||
body := fmt.Sprintf(
|
||||
"**STALE**: Message #%d in #%s in '%s' for %s. \"%s\" — @%s",
|
||||
msg.ID, msg.Channel, msg.State, age, truncBody, msg.FromAgent,
|
||||
)
|
||||
|
||||
_, err := w.msgService.SendMessage(ctx, "system", member, body, SendOptions{
|
||||
Subject: fmt.Sprintf("workflow-stalemate-reminder:%d", msg.ID),
|
||||
Priority: 7,
|
||||
Metadata: fmt.Sprintf(`{"workflow_stalemate_reminder_for":%d}`, msg.ID),
|
||||
})
|
||||
if err != nil {
|
||||
w.logger.Error("send workflow stalemate reminder failed",
|
||||
"message_id", msg.ID,
|
||||
"to_agent", member,
|
||||
"error", err,
|
||||
)
|
||||
continue
|
||||
}
|
||||
w.logger.Info("sent workflow stalemate reminder",
|
||||
"message_id", msg.ID,
|
||||
"channel", msg.Channel,
|
||||
"state", msg.State,
|
||||
"to_agent", member,
|
||||
"age", age,
|
||||
)
|
||||
count++
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
// sendWorkflowEscalation posts an escalation to #approvals for a stale workflow message.
|
||||
func (w *StalemateWorker) sendWorkflowEscalation(ctx context.Context, msg staleWorkflowMsg) bool {
|
||||
approvalsChanID, err := w.channelLookup.GetChannelIDByName(ctx, "approvals")
|
||||
if err != nil {
|
||||
w.logger.Warn("cannot escalate workflow stalemate: #approvals channel not found", "error", err)
|
||||
return false
|
||||
}
|
||||
|
||||
age := formatAge(msg.StateAge)
|
||||
truncBody := truncate(msg.Body, 100)
|
||||
|
||||
body := fmt.Sprintf(
|
||||
"**STALE**: Message #%d in #%s in '%s' for %s. \"%s\" — @%s",
|
||||
msg.ID, msg.Channel, msg.State, age, truncBody, msg.FromAgent,
|
||||
)
|
||||
|
||||
_, err = w.msgService.SendMessage(ctx, "system", "", body, SendOptions{
|
||||
Subject: fmt.Sprintf("workflow-stalemate-escalation:%d", msg.ID),
|
||||
Priority: 9,
|
||||
Metadata: fmt.Sprintf(`{"workflow_stalemate_escalation_for":%d}`, msg.ID),
|
||||
ChannelID: &approvalsChanID,
|
||||
})
|
||||
if err != nil {
|
||||
w.logger.Error("send workflow escalation to #approvals failed",
|
||||
"message_id", msg.ID,
|
||||
"channel", msg.Channel,
|
||||
"error", err,
|
||||
)
|
||||
return false
|
||||
}
|
||||
w.logger.Info("escalated stale workflow message to #approvals",
|
||||
"message_id", msg.ID,
|
||||
"channel", msg.Channel,
|
||||
"state", msg.State,
|
||||
"age", age,
|
||||
)
|
||||
return true
|
||||
}
|
||||
|
||||
// workflowReminderExists checks if a workflow stalemate reminder already exists for a message.
|
||||
func (w *StalemateWorker) workflowReminderExists(ctx context.Context, messageID int64) bool {
|
||||
var count int
|
||||
err := w.db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages
|
||||
WHERE from_agent = 'system'
|
||||
AND metadata LIKE ?`,
|
||||
fmt.Sprintf(`%%"workflow_stalemate_reminder_for":%d%%`, messageID),
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return count > 0
|
||||
}
|
||||
|
||||
// workflowEscalationExists checks if a workflow stalemate escalation already exists for a message.
|
||||
func (w *StalemateWorker) workflowEscalationExists(ctx context.Context, messageID int64) bool {
|
||||
var count int
|
||||
err := w.db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages
|
||||
WHERE from_agent = 'system'
|
||||
AND metadata LIKE ?`,
|
||||
fmt.Sprintf(`%%"workflow_stalemate_escalation_for":%d%%`, messageID),
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return count > 0
|
||||
}
|
||||
|
||||
// truncate truncates a string to maxLen characters, appending "..." if truncated.
|
||||
func truncate(s string, maxLen int) string {
|
||||
runes := []rune(s)
|
||||
if len(runes) <= maxLen {
|
||||
return s
|
||||
}
|
||||
return string(runes[:maxLen]) + "..."
|
||||
}
|
||||
|
||||
// formatAge returns a human-readable age string.
|
||||
func formatAge(d time.Duration) string {
|
||||
if d < time.Hour {
|
||||
return fmt.Sprintf("%dm", int(d.Minutes()))
|
||||
}
|
||||
hours := int(d.Hours())
|
||||
if hours < 24 {
|
||||
return fmt.Sprintf("%dh", hours)
|
||||
}
|
||||
days := hours / 24
|
||||
remainingHours := hours % 24
|
||||
if remainingHours == 0 {
|
||||
if days == 1 {
|
||||
return "1 day"
|
||||
}
|
||||
return fmt.Sprintf("%d days", days)
|
||||
}
|
||||
if days == 1 {
|
||||
return fmt.Sprintf("1 day %dh", remainingHours)
|
||||
}
|
||||
return fmt.Sprintf("%d days %dh", days, remainingHours)
|
||||
}
|
||||
|
||||
@@ -3,7 +3,6 @@ package messaging
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -13,19 +12,6 @@ import (
|
||||
"github.com/synapbus/synapbus/internal/trace"
|
||||
)
|
||||
|
||||
// stubChannelLookup implements ChannelLookup for tests.
|
||||
type stubChannelLookup struct {
|
||||
channelID int64
|
||||
err error
|
||||
}
|
||||
|
||||
func (s *stubChannelLookup) GetChannelIDByName(ctx context.Context, name string) (int64, error) {
|
||||
if s.err != nil {
|
||||
return 0, s.err
|
||||
}
|
||||
return s.channelID, nil
|
||||
}
|
||||
|
||||
// newStalemateTestService creates a MessagingService and DB for stalemate tests.
|
||||
func newStalemateTestService(t *testing.T) (*MessagingService, *sql.DB) {
|
||||
t.Helper()
|
||||
@@ -47,7 +33,6 @@ func newStalemateTestService(t *testing.T) (*MessagingService, *sql.DB) {
|
||||
func insertStaleMessage(t *testing.T, db *sql.DB, from, to, body, status string, createdAt time.Time, claimedAt *time.Time, claimedBy string) int64 {
|
||||
t.Helper()
|
||||
|
||||
// Insert conversation first
|
||||
result, err := db.Exec(
|
||||
`INSERT INTO conversations (subject, created_by, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?)`,
|
||||
@@ -83,19 +68,15 @@ func TestStalemateWorker_ProcessingTimeout(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Insert a message in "processing" status with old claimed_at
|
||||
oldClaimedAt := time.Now().Add(-25 * time.Hour)
|
||||
msgID := insertStaleMessage(t, db, "sender", "receiver", "stale processing task", StatusProcessing, time.Now().Add(-26*time.Hour), &oldClaimedAt, "receiver")
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
config.ProcessingTimeout = 24 * time.Hour
|
||||
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no channel")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
worker := NewStalemateWorker(db, svc, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify message was auto-failed
|
||||
var status, metadata string
|
||||
err := db.QueryRowContext(ctx, `SELECT status, metadata FROM messages WHERE id = ?`, msgID).Scan(&status, &metadata)
|
||||
if err != nil {
|
||||
@@ -113,19 +94,15 @@ func TestStalemateWorker_ProcessingTimeout_NotExpired(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Insert a message in "processing" status with recent claimed_at (should NOT be failed)
|
||||
recentClaimedAt := time.Now().Add(-1 * time.Hour)
|
||||
msgID := insertStaleMessage(t, db, "sender", "receiver", "recent processing task", StatusProcessing, time.Now().Add(-2*time.Hour), &recentClaimedAt, "receiver")
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
config.ProcessingTimeout = 24 * time.Hour
|
||||
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no channel")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
worker := NewStalemateWorker(db, svc, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify message was NOT auto-failed
|
||||
var status string
|
||||
err := db.QueryRowContext(ctx, `SELECT status FROM messages WHERE id = ?`, msgID).Scan(&status)
|
||||
if err != nil {
|
||||
@@ -136,29 +113,22 @@ func TestStalemateWorker_ProcessingTimeout_NotExpired(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestStalemateWorker_ProcessingTimeout_RaceGuard verifies that the
|
||||
// auto-fail UPDATE re-checks claimed_at < cutoff and won't stomp a row that
|
||||
// was legitimately re-claimed (claimed_at refreshed) between the worker's
|
||||
// SELECT scan and its row-by-row UPDATE. This guards the TOCTOU window
|
||||
// the stale-worker race depends on.
|
||||
// TestStalemateWorker_ProcessingTimeout_RaceGuard verifies that the auto-fail
|
||||
// UPDATE re-checks claimed_at < cutoff so a row re-claimed between SELECT and
|
||||
// UPDATE is not stomped. Guards the TOCTOU window the stale-worker race
|
||||
// depends on.
|
||||
func TestStalemateWorker_ProcessingTimeout_RaceGuard(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Insert a message that *was* stale at SELECT time.
|
||||
oldClaimedAt := time.Now().Add(-25 * time.Hour)
|
||||
msgID := insertStaleMessage(t, db, "sender", "receiver", "racing task",
|
||||
StatusProcessing, time.Now().Add(-26*time.Hour), &oldClaimedAt, "receiver")
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
config.ProcessingTimeout = 24 * time.Hour
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no channel")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
worker := NewStalemateWorker(db, svc, config)
|
||||
|
||||
// Simulate the race: between the worker's SELECT (which would have picked
|
||||
// this row) and its UPDATE, the legitimate claimer refreshes claimed_at to
|
||||
// "now". With the cutoff predicate in place the UPDATE no-ops instead of
|
||||
// silently failing live work.
|
||||
freshClaimedAt := time.Now()
|
||||
if _, err := db.ExecContext(ctx,
|
||||
`UPDATE messages SET claimed_at = ? WHERE id = ?`,
|
||||
@@ -180,177 +150,6 @@ func TestStalemateWorker_ProcessingTimeout_RaceGuard(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_PendingReminder(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Insert a pending DM that is 5 hours old
|
||||
insertStaleMessage(t, db, "sender", "receiver", "please review this", StatusPending, time.Now().Add(-5*time.Hour), nil, "")
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
config.ReminderAfter = 4 * time.Hour
|
||||
config.EscalateAfter = 48 * time.Hour // won't trigger
|
||||
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no channel")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify a system reminder was sent to receiver
|
||||
var count int
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND to_agent = 'receiver' AND body LIKE '%Reminder%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query reminder: %v", err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Errorf("expected 1 reminder, got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_SystemMessageSkip(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Insert a pending DM FROM system (should be skipped)
|
||||
insertStaleMessage(t, db, "system", "receiver", "system notification", StatusPending, time.Now().Add(-5*time.Hour), nil, "")
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
config.ReminderAfter = 4 * time.Hour
|
||||
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no channel")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify NO reminder was sent (only the original system message should exist)
|
||||
var count int
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND body LIKE '%Reminder%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query reminder: %v", err)
|
||||
}
|
||||
if count != 0 {
|
||||
t.Errorf("expected 0 reminders for system message, got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_DuplicateReminderPrevention(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Insert a pending DM that is old enough for a reminder
|
||||
insertStaleMessage(t, db, "sender", "receiver", "need your attention", StatusPending, time.Now().Add(-5*time.Hour), nil, "")
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
config.ReminderAfter = 4 * time.Hour
|
||||
config.EscalateAfter = 48 * time.Hour
|
||||
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no channel")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
// Run check twice
|
||||
worker.checkStaleMessages(ctx)
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify only ONE reminder was sent
|
||||
var count int
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND to_agent = 'receiver' AND body LIKE '%Reminder%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query reminders: %v", err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Errorf("expected 1 reminder (no duplicates), got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_Escalation(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Create #approvals channel
|
||||
_, err := db.Exec(
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, created_at, updated_at)
|
||||
VALUES (1, 'approvals', 'Approval queue', '', 'standard', 0, 0, 'system', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`)
|
||||
if err != nil {
|
||||
t.Fatalf("create approvals channel: %v", err)
|
||||
}
|
||||
// Add system as member
|
||||
_, err = db.Exec(
|
||||
`INSERT INTO channel_members (channel_id, agent_name, role, joined_at)
|
||||
VALUES (1, 'system', 'owner', CURRENT_TIMESTAMP)`)
|
||||
if err != nil {
|
||||
t.Fatalf("add system to channel: %v", err)
|
||||
}
|
||||
|
||||
// Insert a pending DM that is 49 hours old (beyond escalation threshold)
|
||||
insertStaleMessage(t, db, "sender", "receiver", "urgent task ignored", StatusPending, time.Now().Add(-49*time.Hour), nil, "")
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
config.ReminderAfter = 4 * time.Hour
|
||||
config.EscalateAfter = 48 * time.Hour
|
||||
|
||||
lookup := &stubChannelLookup{channelID: 1}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify an escalation was sent to #approvals channel
|
||||
var count int
|
||||
err = db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND channel_id = 1 AND body LIKE '%ESCALATION%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query escalations: %v", err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Errorf("expected 1 escalation, got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_DuplicateEscalationPrevention(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Create #approvals channel
|
||||
db.Exec(
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, created_at, updated_at)
|
||||
VALUES (1, 'approvals', 'Approval queue', '', 'standard', 0, 0, 'system', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`)
|
||||
db.Exec(
|
||||
`INSERT INTO channel_members (channel_id, agent_name, role, joined_at)
|
||||
VALUES (1, 'system', 'owner', CURRENT_TIMESTAMP)`)
|
||||
|
||||
// Insert a pending DM that is 49 hours old
|
||||
insertStaleMessage(t, db, "sender", "receiver", "urgent task", StatusPending, time.Now().Add(-49*time.Hour), nil, "")
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
config.ReminderAfter = 4 * time.Hour
|
||||
config.EscalateAfter = 48 * time.Hour
|
||||
|
||||
lookup := &stubChannelLookup{channelID: 1}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
// Run check twice
|
||||
worker.checkStaleMessages(ctx)
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify only ONE escalation was sent
|
||||
var count int
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND channel_id = 1 AND body LIKE '%ESCALATION%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query escalations: %v", err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Errorf("expected 1 escalation (no duplicates), got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseStalemateConfig(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
@@ -366,14 +165,10 @@ func TestParseStalemateConfig(t *testing.T) {
|
||||
name: "custom values with day format",
|
||||
envVars: map[string]string{
|
||||
"SYNAPBUS_STALEMATE_PROCESSING_TIMEOUT": "7d",
|
||||
"SYNAPBUS_STALEMATE_REMINDER_AFTER": "8h",
|
||||
"SYNAPBUS_STALEMATE_ESCALATE_AFTER": "3d",
|
||||
"SYNAPBUS_STALEMATE_INTERVAL": "30m",
|
||||
},
|
||||
expected: StalemateConfig{
|
||||
ProcessingTimeout: 7 * 24 * time.Hour,
|
||||
ReminderAfter: 8 * time.Hour,
|
||||
EscalateAfter: 3 * 24 * time.Hour,
|
||||
Interval: 30 * time.Minute,
|
||||
},
|
||||
},
|
||||
@@ -381,14 +176,10 @@ func TestParseStalemateConfig(t *testing.T) {
|
||||
name: "standard Go duration format",
|
||||
envVars: map[string]string{
|
||||
"SYNAPBUS_STALEMATE_PROCESSING_TIMEOUT": "48h",
|
||||
"SYNAPBUS_STALEMATE_REMINDER_AFTER": "2h30m",
|
||||
"SYNAPBUS_STALEMATE_ESCALATE_AFTER": "72h",
|
||||
"SYNAPBUS_STALEMATE_INTERVAL": "5m",
|
||||
},
|
||||
expected: StalemateConfig{
|
||||
ProcessingTimeout: 48 * time.Hour,
|
||||
ReminderAfter: 2*time.Hour + 30*time.Minute,
|
||||
EscalateAfter: 72 * time.Hour,
|
||||
Interval: 5 * time.Minute,
|
||||
},
|
||||
},
|
||||
@@ -396,28 +187,22 @@ func TestParseStalemateConfig(t *testing.T) {
|
||||
name: "invalid values fall back to defaults",
|
||||
envVars: map[string]string{
|
||||
"SYNAPBUS_STALEMATE_PROCESSING_TIMEOUT": "invalid",
|
||||
"SYNAPBUS_STALEMATE_REMINDER_AFTER": "bad",
|
||||
"SYNAPBUS_STALEMATE_ESCALATE_AFTER": "",
|
||||
"SYNAPBUS_STALEMATE_INTERVAL": "-5m",
|
||||
},
|
||||
expected: DefaultStalemateConfig(),
|
||||
},
|
||||
}
|
||||
|
||||
envKeys := []string{
|
||||
"SYNAPBUS_STALEMATE_PROCESSING_TIMEOUT",
|
||||
"SYNAPBUS_STALEMATE_INTERVAL",
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
// Clear all env vars first
|
||||
envKeys := []string{
|
||||
"SYNAPBUS_STALEMATE_PROCESSING_TIMEOUT",
|
||||
"SYNAPBUS_STALEMATE_REMINDER_AFTER",
|
||||
"SYNAPBUS_STALEMATE_ESCALATE_AFTER",
|
||||
"SYNAPBUS_STALEMATE_INTERVAL",
|
||||
}
|
||||
for _, k := range envKeys {
|
||||
os.Unsetenv(k)
|
||||
}
|
||||
|
||||
// Set test env vars
|
||||
for k, v := range tt.envVars {
|
||||
os.Setenv(k, v)
|
||||
}
|
||||
@@ -432,12 +217,6 @@ func TestParseStalemateConfig(t *testing.T) {
|
||||
if cfg.ProcessingTimeout != tt.expected.ProcessingTimeout {
|
||||
t.Errorf("ProcessingTimeout = %v, want %v", cfg.ProcessingTimeout, tt.expected.ProcessingTimeout)
|
||||
}
|
||||
if cfg.ReminderAfter != tt.expected.ReminderAfter {
|
||||
t.Errorf("ReminderAfter = %v, want %v", cfg.ReminderAfter, tt.expected.ReminderAfter)
|
||||
}
|
||||
if cfg.EscalateAfter != tt.expected.EscalateAfter {
|
||||
t.Errorf("EscalateAfter = %v, want %v", cfg.EscalateAfter, tt.expected.EscalateAfter)
|
||||
}
|
||||
if cfg.Interval != tt.expected.Interval {
|
||||
t.Errorf("Interval = %v, want %v", cfg.Interval, tt.expected.Interval)
|
||||
}
|
||||
@@ -447,10 +226,10 @@ func TestParseStalemateConfig(t *testing.T) {
|
||||
|
||||
func TestParseDurationWithDays(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
want time.Duration
|
||||
wantErr bool
|
||||
name string
|
||||
input string
|
||||
want time.Duration
|
||||
wantErr bool
|
||||
}{
|
||||
{"7 days", "7d", 7 * 24 * time.Hour, false},
|
||||
{"1 day", "1d", 24 * time.Hour, false},
|
||||
@@ -475,392 +254,3 @@ func TestParseDurationWithDays(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestTruncate(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
maxLen int
|
||||
want string
|
||||
}{
|
||||
{"short string", "hello", 10, "hello"},
|
||||
{"exact length", "hello", 5, "hello"},
|
||||
{"truncated", "hello world, this is a long message", 10, "hello worl..."},
|
||||
{"empty", "", 10, ""},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := truncate(tt.input, tt.maxLen)
|
||||
if got != tt.want {
|
||||
t.Errorf("truncate(%q, %d) = %q, want %q", tt.input, tt.maxLen, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatAge(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
d time.Duration
|
||||
want string
|
||||
}{
|
||||
{"minutes", 30 * time.Minute, "30m"},
|
||||
{"hours", 5 * time.Hour, "5h"},
|
||||
{"1 day", 24 * time.Hour, "1 day"},
|
||||
{"2 days", 48 * time.Hour, "2 days"},
|
||||
{"1 day with hours", 25 * time.Hour, "1 day 1h"},
|
||||
{"2 days with hours", 50 * time.Hour, "2 days 2h"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := formatAge(tt.d)
|
||||
if got != tt.want {
|
||||
t.Errorf("formatAge(%v) = %q, want %q", tt.d, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestComputeWorkflowStateFromReactions(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
reactions []reactionRow
|
||||
want string
|
||||
}{
|
||||
{"no reactions = proposed", nil, "proposed"},
|
||||
{"approve only", []reactionRow{{Reaction: "approve"}}, "approved"},
|
||||
{"in_progress only", []reactionRow{{Reaction: "in_progress"}}, "in_progress"},
|
||||
{"reject only", []reactionRow{{Reaction: "reject"}}, "rejected"},
|
||||
{"done only", []reactionRow{{Reaction: "done"}}, "done"},
|
||||
{"published only", []reactionRow{{Reaction: "published"}}, "published"},
|
||||
{"approve + in_progress = in_progress (higher priority)", []reactionRow{
|
||||
{Reaction: "approve"},
|
||||
{Reaction: "in_progress"},
|
||||
}, "in_progress"},
|
||||
{"approve + done = done", []reactionRow{
|
||||
{Reaction: "approve"},
|
||||
{Reaction: "done"},
|
||||
}, "done"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := computeWorkflowStateFromReactions(tt.reactions)
|
||||
if got != tt.want {
|
||||
t.Errorf("computeWorkflowStateFromReactions() = %q, want %q", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsTerminalWorkflowState(t *testing.T) {
|
||||
tests := []struct {
|
||||
state string
|
||||
terminal bool
|
||||
}{
|
||||
{"proposed", false},
|
||||
{"approved", false},
|
||||
{"in_progress", false},
|
||||
{"rejected", true},
|
||||
{"done", true},
|
||||
{"published", true},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.state, func(t *testing.T) {
|
||||
got := isTerminalWorkflowState(tt.state)
|
||||
if got != tt.terminal {
|
||||
t.Errorf("isTerminalWorkflowState(%q) = %v, want %v", tt.state, got, tt.terminal)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_WorkflowReminder(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Create a workflow-enabled channel with short timeouts
|
||||
_, err := db.Exec(
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, workflow_enabled, stalemate_remind_after, stalemate_escalate_after, created_at, updated_at)
|
||||
VALUES (10, 'news-test', 'Test news channel', '', 'standard', 0, 0, 'system', 1, '1s', '72h', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`)
|
||||
if err != nil {
|
||||
t.Fatalf("create workflow channel: %v", err)
|
||||
}
|
||||
|
||||
// Add system and sender as members
|
||||
db.Exec(`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (10, 'system', 'owner', CURRENT_TIMESTAMP)`)
|
||||
db.Exec(`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (10, 'sender', 'member', CURRENT_TIMESTAMP)`)
|
||||
db.Exec(`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (10, 'receiver', 'member', CURRENT_TIMESTAMP)`)
|
||||
|
||||
// Insert a channel message with old created_at (will be in "proposed" state since no reactions)
|
||||
oldTime := time.Now().Add(-2 * time.Second)
|
||||
convResult, err := db.Exec(
|
||||
`INSERT INTO conversations (subject, created_by, created_at, updated_at) VALUES ('wf-test', 'sender', ?, ?)`,
|
||||
oldTime, oldTime,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("insert conversation: %v", err)
|
||||
}
|
||||
convID, _ := convResult.LastInsertId()
|
||||
|
||||
channelID := int64(10)
|
||||
_, err = db.Exec(
|
||||
`INSERT INTO messages (conversation_id, from_agent, to_agent, body, priority, status, metadata, channel_id, created_at, updated_at)
|
||||
VALUES (?, 'sender', '', 'Draft blog post about MCP', 5, 'pending', '{}', ?, ?, ?)`,
|
||||
convID, channelID, oldTime, oldTime,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("insert channel message: %v", err)
|
||||
}
|
||||
|
||||
// Wait for the timeout to elapse
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no approvals channel")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify workflow stalemate reminders were sent to channel members
|
||||
var count int
|
||||
err = db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND body LIKE '%STALE%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query workflow reminders: %v", err)
|
||||
}
|
||||
// Should have sent reminders to all 3 members (system, sender, receiver)
|
||||
if count < 1 {
|
||||
t.Errorf("expected at least 1 workflow reminder, got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_WorkflowEscalation(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Create a workflow-enabled channel with short escalation timeout
|
||||
_, err := db.Exec(
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, workflow_enabled, stalemate_remind_after, stalemate_escalate_after, created_at, updated_at)
|
||||
VALUES (10, 'news-test', 'Test news channel', '', 'standard', 0, 0, 'system', 1, '1s', '1s', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`)
|
||||
if err != nil {
|
||||
t.Fatalf("create workflow channel: %v", err)
|
||||
}
|
||||
|
||||
// Create #approvals channel
|
||||
db.Exec(
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, created_at, updated_at)
|
||||
VALUES (20, 'approvals', 'Approval queue', '', 'standard', 0, 0, 'system', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`)
|
||||
db.Exec(`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (20, 'system', 'owner', CURRENT_TIMESTAMP)`)
|
||||
|
||||
// Add members to workflow channel
|
||||
db.Exec(`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (10, 'sender', 'member', CURRENT_TIMESTAMP)`)
|
||||
|
||||
// Insert a channel message old enough to trigger escalation
|
||||
oldTime := time.Now().Add(-2 * time.Second)
|
||||
convResult, _ := db.Exec(
|
||||
`INSERT INTO conversations (subject, created_by, created_at, updated_at) VALUES ('wf-esc', 'sender', ?, ?)`,
|
||||
oldTime, oldTime,
|
||||
)
|
||||
convID, _ := convResult.LastInsertId()
|
||||
|
||||
channelID := int64(10)
|
||||
_, err = db.Exec(
|
||||
`INSERT INTO messages (conversation_id, from_agent, to_agent, body, priority, status, metadata, channel_id, created_at, updated_at)
|
||||
VALUES (?, 'sender', '', 'Stale proposal needing attention', 5, 'pending', '{}', ?, ?, ?)`,
|
||||
convID, channelID, oldTime, oldTime,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("insert channel message: %v", err)
|
||||
}
|
||||
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
lookup := &stubChannelLookup{channelID: 20}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify escalation was sent to #approvals
|
||||
var count int
|
||||
err = db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND channel_id = 20 AND body LIKE '%STALE%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query workflow escalation: %v", err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Errorf("expected 1 workflow escalation, got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_WorkflowTerminalStateSkip(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Create a workflow-enabled channel with short timeouts
|
||||
_, err := db.Exec(
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, workflow_enabled, stalemate_remind_after, stalemate_escalate_after, created_at, updated_at)
|
||||
VALUES (10, 'news-test', 'Test news channel', '', 'standard', 0, 0, 'system', 1, '1s', '1s', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`)
|
||||
if err != nil {
|
||||
t.Fatalf("create workflow channel: %v", err)
|
||||
}
|
||||
db.Exec(`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (10, 'sender', 'member', CURRENT_TIMESTAMP)`)
|
||||
|
||||
// Insert a channel message
|
||||
oldTime := time.Now().Add(-2 * time.Second)
|
||||
convResult, _ := db.Exec(
|
||||
`INSERT INTO conversations (subject, created_by, created_at, updated_at) VALUES ('wf-done', 'sender', ?, ?)`,
|
||||
oldTime, oldTime,
|
||||
)
|
||||
convID, _ := convResult.LastInsertId()
|
||||
|
||||
channelID := int64(10)
|
||||
msgResult, err := db.Exec(
|
||||
`INSERT INTO messages (conversation_id, from_agent, to_agent, body, priority, status, metadata, channel_id, created_at, updated_at)
|
||||
VALUES (?, 'sender', '', 'Completed task', 5, 'pending', '{}', ?, ?, ?)`,
|
||||
convID, channelID, oldTime, oldTime,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("insert channel message: %v", err)
|
||||
}
|
||||
msgID, _ := msgResult.LastInsertId()
|
||||
|
||||
// Add a "done" reaction — puts it in terminal state
|
||||
_, err = db.Exec(
|
||||
`INSERT INTO message_reactions (message_id, agent_name, reaction, metadata, created_at)
|
||||
VALUES (?, 'sender', 'done', '{}', ?)`,
|
||||
msgID, oldTime,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("insert reaction: %v", err)
|
||||
}
|
||||
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no approvals")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify NO reminders were sent (message is in terminal "done" state)
|
||||
var count int
|
||||
err = db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND body LIKE '%STALE%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query reminders: %v", err)
|
||||
}
|
||||
if count != 0 {
|
||||
t.Errorf("expected 0 reminders for terminal state message, got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_WorkflowDuplicateReminderPrevention(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Create a workflow-enabled channel with short timeout
|
||||
_, err := db.Exec(
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, workflow_enabled, stalemate_remind_after, stalemate_escalate_after, created_at, updated_at)
|
||||
VALUES (10, 'news-test', 'Test', '', 'standard', 0, 0, 'system', 1, '1s', '72h', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`)
|
||||
if err != nil {
|
||||
t.Fatalf("create workflow channel: %v", err)
|
||||
}
|
||||
db.Exec(`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (10, 'receiver', 'member', CURRENT_TIMESTAMP)`)
|
||||
|
||||
// Insert a channel message
|
||||
oldTime := time.Now().Add(-2 * time.Second)
|
||||
convResult, _ := db.Exec(
|
||||
`INSERT INTO conversations (subject, created_by, created_at, updated_at) VALUES ('wf-dup', 'sender', ?, ?)`,
|
||||
oldTime, oldTime,
|
||||
)
|
||||
convID, _ := convResult.LastInsertId()
|
||||
|
||||
channelID := int64(10)
|
||||
_, err = db.Exec(
|
||||
`INSERT INTO messages (conversation_id, from_agent, to_agent, body, priority, status, metadata, channel_id, created_at, updated_at)
|
||||
VALUES (?, 'sender', '', 'Needs review', 5, 'pending', '{}', ?, ?, ?)`,
|
||||
convID, channelID, oldTime, oldTime,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("insert channel message: %v", err)
|
||||
}
|
||||
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no approvals")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
// Run twice
|
||||
worker.checkStaleMessages(ctx)
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify only one set of reminders was sent (no duplicates)
|
||||
var count int
|
||||
err = db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND to_agent = 'receiver' AND body LIKE '%STALE%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query reminders: %v", err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Errorf("expected 1 reminder (no duplicates), got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStalemateWorker_WorkflowNonWorkflowChannelSkip(t *testing.T) {
|
||||
svc, db := newStalemateTestService(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Create a channel with workflow DISABLED
|
||||
_, err := db.Exec(
|
||||
`INSERT INTO channels (id, name, description, topic, type, is_private, is_system, created_by, workflow_enabled, stalemate_remind_after, stalemate_escalate_after, created_at, updated_at)
|
||||
VALUES (10, 'general', 'General', '', 'standard', 0, 0, 'system', 0, '1s', '1s', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)`)
|
||||
if err != nil {
|
||||
t.Fatalf("create channel: %v", err)
|
||||
}
|
||||
db.Exec(`INSERT INTO channel_members (channel_id, agent_name, role, joined_at) VALUES (10, 'sender', 'member', CURRENT_TIMESTAMP)`)
|
||||
|
||||
// Insert a channel message
|
||||
oldTime := time.Now().Add(-2 * time.Second)
|
||||
convResult, _ := db.Exec(
|
||||
`INSERT INTO conversations (subject, created_by, created_at, updated_at) VALUES ('no-wf', 'sender', ?, ?)`,
|
||||
oldTime, oldTime,
|
||||
)
|
||||
convID, _ := convResult.LastInsertId()
|
||||
|
||||
channelID := int64(10)
|
||||
db.Exec(
|
||||
`INSERT INTO messages (conversation_id, from_agent, to_agent, body, priority, status, metadata, channel_id, created_at, updated_at)
|
||||
VALUES (?, 'sender', '', 'No workflow here', 5, 'pending', '{}', ?, ?, ?)`,
|
||||
convID, channelID, oldTime, oldTime,
|
||||
)
|
||||
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
|
||||
config := DefaultStalemateConfig()
|
||||
lookup := &stubChannelLookup{channelID: 0, err: fmt.Errorf("no approvals")}
|
||||
worker := NewStalemateWorker(db, svc, lookup, config)
|
||||
|
||||
worker.checkStaleMessages(ctx)
|
||||
|
||||
// Verify NO reminders — channel is not workflow-enabled
|
||||
var count int
|
||||
err = db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM messages WHERE from_agent = 'system' AND body LIKE '%STALE%'`,
|
||||
).Scan(&count)
|
||||
if err != nil {
|
||||
t.Fatalf("query reminders: %v", err)
|
||||
}
|
||||
if count != 0 {
|
||||
t.Errorf("expected 0 reminders for non-workflow channel, got %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -41,6 +41,7 @@ type MessageStore interface {
|
||||
GetConversationIDsForChannel(ctx context.Context, channelID int64, lastMessageID int64) ([]int64, error)
|
||||
GetConversationIDsForDM(ctx context.Context, agentNames []string, peerAgent string, lastMessageID int64) ([]int64, error)
|
||||
GetReplyCounts(ctx context.Context, messageIDs []int64) (map[int64]int, error)
|
||||
ListEventMetaAfter(ctx context.Context, agentName string, afterID int64, limit int) ([]*EventMeta, error)
|
||||
}
|
||||
|
||||
// SQLiteMessageStore implements MessageStore using SQLite.
|
||||
|
||||
@@ -84,4 +84,81 @@ var (
|
||||
},
|
||||
[]string{"agent"},
|
||||
)
|
||||
|
||||
// Dream worker metrics (feature 020 follow-up).
|
||||
DreamJobsTotal = promauto.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Namespace: "synapbus",
|
||||
Name: "dream_jobs_total",
|
||||
Help: "Dream worker jobs dispatched, labeled by owner, job_type and final status",
|
||||
},
|
||||
[]string{"owner", "job_type", "status"},
|
||||
)
|
||||
|
||||
DreamTokensTotal = promauto.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Namespace: "synapbus",
|
||||
Name: "dream_tokens_total",
|
||||
Help: "Total tokens consumed by dream worker runs, labeled by owner and direction (in|out)",
|
||||
},
|
||||
[]string{"owner", "direction"},
|
||||
)
|
||||
|
||||
DreamJobDuration = promauto.NewHistogramVec(
|
||||
prometheus.HistogramOpts{
|
||||
Namespace: "synapbus",
|
||||
Name: "dream_job_duration_seconds",
|
||||
Help: "Wallclock duration of a single dream worker job",
|
||||
Buckets: prometheus.DefBuckets,
|
||||
},
|
||||
[]string{"owner", "job_type"},
|
||||
)
|
||||
|
||||
DreamCircuitBrokenTotal = promauto.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Namespace: "synapbus",
|
||||
Name: "dream_circuit_broken_total",
|
||||
Help: "Times the dream worker refused to dispatch because the daily usage gate fired",
|
||||
},
|
||||
[]string{"owner", "reason"},
|
||||
)
|
||||
|
||||
// Proactive-injection metrics (feature 020 follow-up).
|
||||
InjectionPacketsTotal = promauto.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Namespace: "synapbus",
|
||||
Name: "injection_packets_total",
|
||||
Help: "Number of relevant_context packets attached to MCP tool responses, by tool",
|
||||
},
|
||||
[]string{"tool"},
|
||||
)
|
||||
|
||||
InjectionMemoriesPerPacket = promauto.NewHistogramVec(
|
||||
prometheus.HistogramOpts{
|
||||
Namespace: "synapbus",
|
||||
Name: "injection_memories_per_packet",
|
||||
Help: "Distribution of memory items per injection packet, by tool",
|
||||
Buckets: []float64{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10},
|
||||
},
|
||||
[]string{"tool"},
|
||||
)
|
||||
|
||||
InjectionPacketChars = promauto.NewHistogramVec(
|
||||
prometheus.HistogramOpts{
|
||||
Namespace: "synapbus",
|
||||
Name: "injection_packet_chars",
|
||||
Help: "Distribution of character size of an injection packet, by tool",
|
||||
Buckets: []float64{0, 250, 500, 750, 1000, 1250, 1500, 1750, 2000, 2250, 2500},
|
||||
},
|
||||
[]string{"tool"},
|
||||
)
|
||||
|
||||
InjectionSkippedTotal = promauto.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Namespace: "synapbus",
|
||||
Name: "injection_skipped_total",
|
||||
Help: "Times injection was skipped, labeled by tool and reason (no_owner|empty_pool|disabled)",
|
||||
},
|
||||
[]string{"tool", "reason"},
|
||||
)
|
||||
)
|
||||
|
||||
@@ -0,0 +1,560 @@
|
||||
// Proactive-memory injection retrieval — builds the relevant-context
|
||||
// packet attached to every injection-eligible MCP tool response (per
|
||||
// `contracts/mcp-injection.md`).
|
||||
package search
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
)
|
||||
|
||||
// MemoryItem is one entry in the `relevant_context.memories[]` array.
|
||||
// Field shape is contractual (`contracts/mcp-injection.md`).
|
||||
type MemoryItem struct {
|
||||
ID int64 `json:"id"`
|
||||
FromAgent string `json:"from_agent"`
|
||||
Channel string `json:"channel,omitempty"`
|
||||
Body string `json:"body"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
Score float64 `json:"score"`
|
||||
MatchType string `json:"match_type"`
|
||||
Pinned bool `json:"pinned"`
|
||||
Truncated bool `json:"truncated,omitempty"`
|
||||
}
|
||||
|
||||
// ContextPacket is the body of the `relevant_context` field. Returned
|
||||
// from BuildContextPacket; rendered verbatim into the wrapped tool
|
||||
// response.
|
||||
type ContextPacket struct {
|
||||
Memories []MemoryItem `json:"memories"`
|
||||
CoreMemory string `json:"core_memory,omitempty"`
|
||||
PacketChars int `json:"packet_chars"`
|
||||
PacketTokenEstimate int `json:"packet_token_estimate"`
|
||||
RetrievalQuery string `json:"retrieval_query"`
|
||||
SearchMode string `json:"search_mode"`
|
||||
}
|
||||
|
||||
// CoreMemoryProvider is the seam US2 plugs into. When non-nil and
|
||||
// `opts.IncludeCore` is true, BuildContextPacket calls Get() and
|
||||
// includes the result verbatim in the packet.
|
||||
type CoreMemoryProvider interface {
|
||||
Get(ctx context.Context, ownerID, agentName string) (string, error)
|
||||
}
|
||||
|
||||
// PinProvider returns the owner's pinned message ids. Set on
|
||||
// InjectionOpts via US3 wiring; when nil the overlay is skipped.
|
||||
type PinProvider interface {
|
||||
ListForOwner(ctx context.Context, ownerID string) ([]int64, error)
|
||||
}
|
||||
|
||||
// StatusProvider returns the memory_status of each id in the input
|
||||
// slice. Ids that do not appear in the returned map are implicitly
|
||||
// `active`. Set on InjectionOpts via US3 wiring.
|
||||
type StatusProvider interface {
|
||||
Statuses(ctx context.Context, msgIDs []int64) (map[int64]MemoryStatusInfo, error)
|
||||
}
|
||||
|
||||
// MemoryStatusInfo mirrors messaging.MemoryStatus without importing
|
||||
// the messaging package (avoids a cycle). The injection retrieval
|
||||
// layer needs only the Status string and the active/non-active bit.
|
||||
type MemoryStatusInfo struct {
|
||||
Status string
|
||||
}
|
||||
|
||||
// MessageLookup resolves message ids → MemoryItem fields for the pin
|
||||
// overlay. The overlay needs body / from_agent / channel for pinned
|
||||
// messages that did NOT come back from the search; loading them
|
||||
// directly from the messages table keeps this independent of the
|
||||
// search index.
|
||||
type MessageLookup interface {
|
||||
LookupForInjection(ctx context.Context, ids []int64) ([]MemoryItem, error)
|
||||
}
|
||||
|
||||
// InjectionOpts captures the per-call configuration for
|
||||
// BuildContextPacket. Sourced from messaging.MemoryConfig at wrap time.
|
||||
type InjectionOpts struct {
|
||||
// BudgetTokens is the soft cap on the assembled packet. 0 disables
|
||||
// injection: BuildContextPacket returns (nil, nil).
|
||||
BudgetTokens int
|
||||
// MaxItems caps the number of memory items in the packet.
|
||||
MaxItems int
|
||||
// MinScore is the relevance floor. Items below are dropped.
|
||||
MinScore float64
|
||||
// IncludeCore enables the core-memory lookup. Only session-start
|
||||
// tools (i.e. my_status) should set this.
|
||||
IncludeCore bool
|
||||
// CoreProvider is consulted when IncludeCore is true. May be nil
|
||||
// (US2 not yet wired) — then no core memory is included.
|
||||
CoreProvider CoreMemoryProvider
|
||||
// PinProvider, when non-nil, supplies owner-pinned message ids
|
||||
// that are spliced into the packet with Score=1.0 regardless of
|
||||
// the score floor. Status filter still drops soft_deleted /
|
||||
// superseded pins so retrieval never surfaces tombstoned facts.
|
||||
PinProvider PinProvider
|
||||
// StatusProvider, when non-nil, supplies the memory_status of
|
||||
// each candidate; results with status soft_deleted/superseded are
|
||||
// dropped (unless pinned).
|
||||
StatusProvider StatusProvider
|
||||
// MessageLookup, when non-nil, resolves pinned message ids that
|
||||
// did not surface through retrieval. When nil, only pins already
|
||||
// present in the retrieval results are highlighted.
|
||||
MessageLookup MessageLookup
|
||||
// RecentWindowDays bounds the recency fallback (FR-009) to the
|
||||
// last N days of memory-channel activity. 0 → 14d default.
|
||||
// Mirrored from messaging.MemoryConfig.DreamRecentWindow by the
|
||||
// MCP wrapper at request time.
|
||||
RecentWindowDays int
|
||||
// Now is overridable for tests. Defaults to time.Now.
|
||||
Now func() time.Time
|
||||
}
|
||||
|
||||
// EstimateTokens returns the char-based token estimate. Matches R4:
|
||||
// `(chars + 3) / 4`, ceil-equivalent for positive integers.
|
||||
func EstimateTokens(chars int) int {
|
||||
if chars <= 0 {
|
||||
return 0
|
||||
}
|
||||
return (chars + 3) / 4
|
||||
}
|
||||
|
||||
// BuildContextPacket retrieves owner-scoped memories matching `query`,
|
||||
// applies the score floor + token budget + max items cap, optionally
|
||||
// resolves the per-(owner, agent) core memory blob, and returns a
|
||||
// ContextPacket ready to merge into the tool response.
|
||||
//
|
||||
// Returns (nil, nil) when `opts.BudgetTokens == 0` (feature disabled)
|
||||
// or when no memories pass the filter AND no core memory is set —
|
||||
// callers omit the `relevant_context` field entirely in that case.
|
||||
func BuildContextPacket(
|
||||
ctx context.Context,
|
||||
svc *Service,
|
||||
agent *agents.Agent,
|
||||
query string,
|
||||
opts InjectionOpts,
|
||||
) (*ContextPacket, error) {
|
||||
if opts.BudgetTokens == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
if agent == nil {
|
||||
return nil, fmt.Errorf("build context packet: nil agent")
|
||||
}
|
||||
if opts.Now == nil {
|
||||
opts.Now = time.Now
|
||||
}
|
||||
maxItems := opts.MaxItems
|
||||
if maxItems <= 0 {
|
||||
maxItems = 5
|
||||
}
|
||||
|
||||
callerOwner := strconv.FormatInt(agent.OwnerID, 10)
|
||||
if agent.OwnerID == 0 {
|
||||
// Unowned agent: skip retrieval. We still surface a (possibly
|
||||
// non-empty) core memory if the provider returns one for the
|
||||
// empty owner — but that's an edge case the provider can decide
|
||||
// on.
|
||||
callerOwner = ""
|
||||
}
|
||||
|
||||
// Over-fetch x3 to absorb owner-scope filtering + score floor drop.
|
||||
wantedLimit := maxItems * 3
|
||||
if wantedLimit < 15 {
|
||||
wantedLimit = 15
|
||||
}
|
||||
|
||||
searchMode := ModeAuto
|
||||
|
||||
var memories []MemoryItem
|
||||
if svc != nil && callerOwner != "" {
|
||||
if query != "" {
|
||||
// Drive retrieval through the existing hybrid path so we inherit
|
||||
// access control + ranking. We still need a stricter owner filter
|
||||
// on top of canAgentAccessMessage because the memory pool
|
||||
// (open-brain) is broadly readable across agents within the same
|
||||
// system, and we must enforce owner isolation (SC-008).
|
||||
resp, err := svc.Search(ctx, agent.Name, SearchOptions{
|
||||
Query: query,
|
||||
Mode: ModeAuto,
|
||||
Limit: wantedLimit,
|
||||
MinSimilarity: opts.MinScore,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build context packet: search: %w", err)
|
||||
}
|
||||
if resp != nil {
|
||||
searchMode = resp.SearchMode
|
||||
}
|
||||
if resp != nil && len(resp.Results) > 0 {
|
||||
items, err := filterAndScore(ctx, svc, resp.Results, callerOwner, opts)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
memories = items
|
||||
}
|
||||
} else {
|
||||
// FR-009 recency fallback: no explicit query → return the N
|
||||
// most recent memory-channel messages whose author belongs to
|
||||
// the caller's owner. Bypasses the hybrid index (which has no
|
||||
// notion of "no query") and goes directly to SQL. Window is
|
||||
// bounded by opts.RecentWindowDays (defaults to 14).
|
||||
windowDays := opts.RecentWindowDays
|
||||
if windowDays <= 0 {
|
||||
windowDays = 14
|
||||
}
|
||||
items, err := recentMemoriesForOwner(ctx, svc.db, callerOwner, wantedLimit, windowDays)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build context packet: recency: %w", err)
|
||||
}
|
||||
memories = items
|
||||
searchMode = "recent"
|
||||
}
|
||||
}
|
||||
|
||||
// Apply memory_status filter (US3 T031): drop soft_deleted /
|
||||
// superseded results unless they will be pinned in the next step.
|
||||
memories = applyStatusFilter(ctx, memories, nil, opts)
|
||||
|
||||
// Pin overlay (US3 T029/T031): owner-pinned message ids bypass the
|
||||
// score floor and are spliced in with Score=1.0 / Pinned=true. We
|
||||
// build the pin set up-front so the status filter knows to spare
|
||||
// them.
|
||||
pinIDs, _ := loadPinIDs(ctx, opts, callerOwner)
|
||||
if len(pinIDs) > 0 {
|
||||
memories = applyStatusFilter(ctx, memories, pinIDs, opts)
|
||||
memories = applyPinOverlay(ctx, memories, pinIDs, opts)
|
||||
}
|
||||
|
||||
// Apply token budget: greedy fill in descending score (results are
|
||||
// already sorted). Truncate the last admitted item to fit when it
|
||||
// would otherwise overflow.
|
||||
memories = applyTokenBudget(memories, maxItems, opts.BudgetTokens)
|
||||
|
||||
// Core memory lookup (US2 hook). Provider may be nil — that's fine.
|
||||
var coreBlob string
|
||||
if opts.IncludeCore && opts.CoreProvider != nil && callerOwner != "" {
|
||||
blob, err := opts.CoreProvider.Get(ctx, callerOwner, agent.Name)
|
||||
if err == nil {
|
||||
coreBlob = blob
|
||||
} else if !errors.Is(err, sql.ErrNoRows) {
|
||||
// Surface unexpected errors so callers can log; an absent
|
||||
// core blob (typical "no rows") is not an error.
|
||||
return nil, fmt.Errorf("build context packet: core memory: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
if len(memories) == 0 && coreBlob == "" {
|
||||
// Empty + no core → caller should omit the relevant_context field.
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
if memories == nil {
|
||||
memories = []MemoryItem{}
|
||||
}
|
||||
packet := &ContextPacket{
|
||||
Memories: memories,
|
||||
CoreMemory: coreBlob,
|
||||
RetrievalQuery: query,
|
||||
SearchMode: searchMode,
|
||||
}
|
||||
packet.PacketChars = packetChars(packet)
|
||||
packet.PacketTokenEstimate = EstimateTokens(packet.PacketChars)
|
||||
return packet, nil
|
||||
}
|
||||
|
||||
// filterAndScore drops non-owner messages and items below MinScore,
|
||||
// then assembles MemoryItem entries in the existing RRF-sorted order.
|
||||
//
|
||||
// Owner of each candidate message is resolved via agents.OwnerFor on
|
||||
// `from_agent`. This is the stricter filter referenced in
|
||||
// `contracts/mcp-injection.md`'s cross-owner safety note.
|
||||
func filterAndScore(
|
||||
ctx context.Context,
|
||||
svc *Service,
|
||||
results []*SearchResult,
|
||||
callerOwnerID string,
|
||||
opts InjectionOpts,
|
||||
) ([]MemoryItem, error) {
|
||||
out := make([]MemoryItem, 0, len(results))
|
||||
for _, r := range results {
|
||||
if r == nil || r.Message == nil {
|
||||
continue
|
||||
}
|
||||
// Score selection: prefer SimilarityScore (semantic / hybrid),
|
||||
// fall back to RelevanceScore (fulltext).
|
||||
score := r.SimilarityScore
|
||||
if score == 0 {
|
||||
score = r.RelevanceScore
|
||||
}
|
||||
if opts.MinScore > 0 && score < opts.MinScore {
|
||||
continue
|
||||
}
|
||||
|
||||
owner, err := agents.OwnerFor(ctx, svc.db, r.Message.FromAgent)
|
||||
if err != nil {
|
||||
// Unowned or unknown sender → exclude from injection pool.
|
||||
continue
|
||||
}
|
||||
if owner != callerOwnerID {
|
||||
continue
|
||||
}
|
||||
|
||||
item := MemoryItem{
|
||||
ID: r.Message.ID,
|
||||
FromAgent: r.Message.FromAgent,
|
||||
Body: r.Message.Body,
|
||||
CreatedAt: r.Message.CreatedAt,
|
||||
Score: score,
|
||||
MatchType: r.MatchType,
|
||||
}
|
||||
if r.Message.ChannelID != nil {
|
||||
if name, err := channelName(ctx, svc.db, *r.Message.ChannelID); err == nil {
|
||||
item.Channel = name
|
||||
}
|
||||
}
|
||||
out = append(out, item)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// applyTokenBudget enforces both MaxItems and the token budget. Items
|
||||
// are admitted greedily in input order (caller passes them already
|
||||
// score-sorted). The first item that would overflow is truncated to
|
||||
// fit; all later items are skipped.
|
||||
func applyTokenBudget(items []MemoryItem, maxItems, budgetTokens int) []MemoryItem {
|
||||
if len(items) == 0 {
|
||||
return nil
|
||||
}
|
||||
if budgetTokens <= 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]MemoryItem, 0, len(items))
|
||||
used := 0
|
||||
for i, it := range items {
|
||||
if i >= maxItems {
|
||||
break
|
||||
}
|
||||
cost := EstimateTokens(itemChars(it))
|
||||
if used+cost <= budgetTokens {
|
||||
used += cost
|
||||
out = append(out, it)
|
||||
continue
|
||||
}
|
||||
// Truncate this item to whatever remains in the budget.
|
||||
remaining := budgetTokens - used
|
||||
if remaining <= 0 {
|
||||
break
|
||||
}
|
||||
// Reserve the per-item overhead in the remaining budget so that
|
||||
// post-truncate EstimateTokens(itemChars(it)) <= remaining.
|
||||
overhead := itemChars(it) - len(it.Body) // = from_agent + channel + 32
|
||||
// max chars we can place into Body so that total item tokens fit.
|
||||
maxBodyChars := remaining*4 - overhead
|
||||
if maxBodyChars <= 0 {
|
||||
break
|
||||
}
|
||||
if maxBodyChars >= len(it.Body) {
|
||||
// Whole body still fits — admit unchanged.
|
||||
used += EstimateTokens(itemChars(it))
|
||||
out = append(out, it)
|
||||
continue
|
||||
}
|
||||
truncated := it
|
||||
truncated.Body = it.Body[:maxBodyChars]
|
||||
truncated.Truncated = true
|
||||
used += EstimateTokens(itemChars(truncated))
|
||||
out = append(out, truncated)
|
||||
break
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// itemChars approximates the rendered size of one MemoryItem so the
|
||||
// budget gate stays self-consistent with packetChars.
|
||||
func itemChars(it MemoryItem) int {
|
||||
// Body dominates; from_agent + channel + delimiters add a small
|
||||
// per-item overhead we approximate at 32 chars.
|
||||
return len(it.Body) + len(it.FromAgent) + len(it.Channel) + 32
|
||||
}
|
||||
|
||||
func packetChars(p *ContextPacket) int {
|
||||
total := 0
|
||||
for _, m := range p.Memories {
|
||||
total += itemChars(m)
|
||||
}
|
||||
total += len(p.CoreMemory)
|
||||
return total
|
||||
}
|
||||
|
||||
// loadPinIDs queries the configured PinProvider, if any, and returns
|
||||
// the owner's pinned message ids. Returns nil on any error so that pin
|
||||
// retrieval failure never breaks the wider injection path.
|
||||
func loadPinIDs(ctx context.Context, opts InjectionOpts, ownerID string) ([]int64, error) {
|
||||
if opts.PinProvider == nil || ownerID == "" {
|
||||
return nil, nil
|
||||
}
|
||||
ids, err := opts.PinProvider.ListForOwner(ctx, ownerID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
// applyStatusFilter drops items whose memory_status is soft_deleted or
|
||||
// superseded. `sparedIDs` is the set of message ids that bypass the
|
||||
// filter (pinned ids). When opts.StatusProvider is nil this is a no-op.
|
||||
func applyStatusFilter(ctx context.Context, items []MemoryItem, sparedIDs []int64, opts InjectionOpts) []MemoryItem {
|
||||
if opts.StatusProvider == nil || len(items) == 0 {
|
||||
return items
|
||||
}
|
||||
ids := make([]int64, 0, len(items))
|
||||
for _, it := range items {
|
||||
ids = append(ids, it.ID)
|
||||
}
|
||||
statuses, err := opts.StatusProvider.Statuses(ctx, ids)
|
||||
if err != nil {
|
||||
return items
|
||||
}
|
||||
spared := map[int64]struct{}{}
|
||||
for _, id := range sparedIDs {
|
||||
spared[id] = struct{}{}
|
||||
}
|
||||
out := make([]MemoryItem, 0, len(items))
|
||||
for _, it := range items {
|
||||
st, ok := statuses[it.ID]
|
||||
if !ok || st.Status == "" || st.Status == "active" {
|
||||
out = append(out, it)
|
||||
continue
|
||||
}
|
||||
if _, isPinned := spared[it.ID]; isPinned {
|
||||
out = append(out, it)
|
||||
continue
|
||||
}
|
||||
// Drop soft_deleted / superseded non-pinned.
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// applyPinOverlay marks any item already present and whose id is pinned
|
||||
// as Pinned=true / Score=1.0; pinned ids that are NOT in the input set
|
||||
// are fetched via MessageLookup (if configured) and prepended.
|
||||
func applyPinOverlay(ctx context.Context, items []MemoryItem, pinIDs []int64, opts InjectionOpts) []MemoryItem {
|
||||
if len(pinIDs) == 0 {
|
||||
return items
|
||||
}
|
||||
pinSet := map[int64]struct{}{}
|
||||
for _, id := range pinIDs {
|
||||
pinSet[id] = struct{}{}
|
||||
}
|
||||
// Mark items already present.
|
||||
present := map[int64]struct{}{}
|
||||
for i := range items {
|
||||
if _, ok := pinSet[items[i].ID]; ok {
|
||||
items[i].Pinned = true
|
||||
items[i].Score = 1.0
|
||||
}
|
||||
present[items[i].ID] = struct{}{}
|
||||
}
|
||||
// Fetch missing pinned ids via MessageLookup (if any).
|
||||
var missing []int64
|
||||
for id := range pinSet {
|
||||
if _, ok := present[id]; !ok {
|
||||
missing = append(missing, id)
|
||||
}
|
||||
}
|
||||
if len(missing) > 0 && opts.MessageLookup != nil {
|
||||
extra, err := opts.MessageLookup.LookupForInjection(ctx, missing)
|
||||
if err == nil {
|
||||
// Status filter on the freshly-loaded pinned messages: if
|
||||
// the provider says they are soft_deleted / superseded, do
|
||||
// not surface them either, even though pinned.
|
||||
if opts.StatusProvider != nil {
|
||||
statuses, _ := opts.StatusProvider.Statuses(ctx, missing)
|
||||
filtered := extra[:0]
|
||||
for _, m := range extra {
|
||||
if st, ok := statuses[m.ID]; ok && st.Status != "" && st.Status != "active" {
|
||||
continue
|
||||
}
|
||||
filtered = append(filtered, m)
|
||||
}
|
||||
extra = filtered
|
||||
}
|
||||
// Prepend in stable id order (newest first by convention —
|
||||
// pins are sorted DESC by pinned_at in the store).
|
||||
for i := range extra {
|
||||
extra[i].Pinned = true
|
||||
extra[i].Score = 1.0
|
||||
extra[i].MatchType = "pinned"
|
||||
}
|
||||
items = append(extra, items...)
|
||||
}
|
||||
}
|
||||
return items
|
||||
}
|
||||
|
||||
// recentMemoriesForOwner returns the N most recent messages on memory
|
||||
// channels (open-brain, reflections-*, or any channel flagged
|
||||
// is_memory=true) whose author is owned by `ownerID`. Used when the
|
||||
// injection layer has no explicit retrieval query (e.g. my_status).
|
||||
//
|
||||
// Recency is approximated as "ORDER BY messages.id DESC" — id is
|
||||
// monotonically increasing per SQLite INSERT and matches created_at
|
||||
// ordering on this schema.
|
||||
func recentMemoriesForOwner(ctx context.Context, db *sql.DB, ownerID string, limit, windowDays int) ([]MemoryItem, error) {
|
||||
if db == nil || ownerID == "" || limit <= 0 {
|
||||
return nil, nil
|
||||
}
|
||||
if windowDays <= 0 {
|
||||
windowDays = 14
|
||||
}
|
||||
// SQLite datetime modifier needs the value embedded in the string,
|
||||
// not bound, so build a literal `-N days`. windowDays is bounded
|
||||
// (int from int(Duration/(24h))) — safe to format.
|
||||
windowExpr := fmt.Sprintf("-%d days", windowDays)
|
||||
const q = `
|
||||
SELECT m.id, m.from_agent, COALESCE(c.name,''), m.body, m.created_at
|
||||
FROM messages m
|
||||
JOIN agents a ON a.name = m.from_agent
|
||||
LEFT JOIN channels c ON c.id = m.channel_id
|
||||
WHERE CAST(a.owner_id AS TEXT) = ?
|
||||
AND m.channel_id IS NOT NULL
|
||||
AND c.name IN ('open-brain')
|
||||
AND m.body IS NOT NULL AND m.body != ''
|
||||
AND m.created_at > datetime('now', ?)
|
||||
ORDER BY m.id DESC
|
||||
LIMIT ?`
|
||||
rows, err := db.QueryContext(ctx, q, ownerID, windowExpr, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []MemoryItem
|
||||
for rows.Next() {
|
||||
var it MemoryItem
|
||||
if err := rows.Scan(&it.ID, &it.FromAgent, &it.Channel, &it.Body, &it.CreatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
it.Score = 1.0
|
||||
it.MatchType = "recent"
|
||||
out = append(out, it)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// channelName resolves a channel ID to a name for the optional
|
||||
// `MemoryItem.Channel` field. Best-effort: returns ("", err) on lookup
|
||||
// failure and BuildContextPacket then omits the field entirely.
|
||||
func channelName(ctx context.Context, db *sql.DB, id int64) (string, error) {
|
||||
var name string
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT name FROM channels WHERE id = ?`, id,
|
||||
).Scan(&name)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
@@ -0,0 +1,137 @@
|
||||
package search
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// stubPinProvider lets a test return a fixed pin set.
|
||||
type stubPinProvider struct {
|
||||
ids []int64
|
||||
}
|
||||
|
||||
func (s *stubPinProvider) ListForOwner(ctx context.Context, ownerID string) ([]int64, error) {
|
||||
return s.ids, nil
|
||||
}
|
||||
|
||||
// stubMessageLookup loads pinned messages by id.
|
||||
type stubMessageLookup struct {
|
||||
byID map[int64]MemoryItem
|
||||
}
|
||||
|
||||
func (s *stubMessageLookup) LookupForInjection(ctx context.Context, ids []int64) ([]MemoryItem, error) {
|
||||
out := make([]MemoryItem, 0, len(ids))
|
||||
for _, id := range ids {
|
||||
if m, ok := s.byID[id]; ok {
|
||||
out = append(out, m)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// stubStatusProvider returns canned statuses.
|
||||
type stubStatusProvider struct {
|
||||
byID map[int64]MemoryStatusInfo
|
||||
}
|
||||
|
||||
func (s *stubStatusProvider) Statuses(ctx context.Context, msgIDs []int64) (map[int64]MemoryStatusInfo, error) {
|
||||
out := map[int64]MemoryStatusInfo{}
|
||||
for _, id := range msgIDs {
|
||||
if v, ok := s.byID[id]; ok {
|
||||
out[id] = v
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func TestBuildContextPacket_PinOverlaySurfacesBelowFloor(t *testing.T) {
|
||||
svc, _, db := newTestServices(t)
|
||||
ctx := context.Background()
|
||||
|
||||
a := seedOwnedAgent(t, db, 1, "alice", "a1")
|
||||
// Seed a channel + message so search has something to find for the
|
||||
// owner; the pin will be a separate id we splice in via lookup.
|
||||
seedChannel(t, db, 1, "open-brain", "a1")
|
||||
regularID := seedChannelMessage(t, db, 1, "a1", "some unrelated body")
|
||||
|
||||
pinnedID := regularID + 1000
|
||||
lookup := &stubMessageLookup{
|
||||
byID: map[int64]MemoryItem{
|
||||
pinnedID: {ID: pinnedID, FromAgent: "a1", Body: "pinned fact", Score: 0.0},
|
||||
},
|
||||
}
|
||||
|
||||
pkt, err := BuildContextPacket(ctx, svc, a, "kuzu unrelated query", InjectionOpts{
|
||||
BudgetTokens: 500,
|
||||
MaxItems: 5,
|
||||
MinScore: 0.95, // very high floor → drops everything from search
|
||||
PinProvider: &stubPinProvider{ids: []int64{pinnedID}},
|
||||
MessageLookup: lookup,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("BuildContextPacket: %v", err)
|
||||
}
|
||||
if pkt == nil {
|
||||
t.Fatal("expected non-nil packet (pinned overlay)")
|
||||
}
|
||||
|
||||
sawPinned := false
|
||||
for _, m := range pkt.Memories {
|
||||
if m.ID == pinnedID {
|
||||
sawPinned = true
|
||||
if !m.Pinned {
|
||||
t.Errorf("pinned item must be marked Pinned=true: %+v", m)
|
||||
}
|
||||
if m.Score < 0.99 {
|
||||
t.Errorf("pinned item must have Score=1.0, got %v", m.Score)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !sawPinned {
|
||||
t.Errorf("pinned message id=%d not surfaced in packet: %#v", pinnedID, pkt.Memories)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildContextPacket_StatusFilterDropsSoftDeleted(t *testing.T) {
|
||||
svc, _, db := newTestServices(t)
|
||||
ctx := context.Background()
|
||||
a := seedOwnedAgent(t, db, 1, "alice", "a1")
|
||||
seedChannel(t, db, 1, "open-brain", "a1")
|
||||
|
||||
keepID := seedChannelMessage(t, db, 1, "a1", "active fact about Kuzu")
|
||||
dropID := seedChannelMessage(t, db, 1, "a1", "duplicate Kuzu fact")
|
||||
|
||||
statusProv := &stubStatusProvider{
|
||||
byID: map[int64]MemoryStatusInfo{
|
||||
dropID: {Status: "soft_deleted"},
|
||||
},
|
||||
}
|
||||
|
||||
pkt, err := BuildContextPacket(ctx, svc, a, "Kuzu", InjectionOpts{
|
||||
BudgetTokens: 500,
|
||||
MaxItems: 5,
|
||||
MinScore: 0.0,
|
||||
StatusProvider: statusProv,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("BuildContextPacket: %v", err)
|
||||
}
|
||||
if pkt == nil {
|
||||
t.Fatal("expected non-nil packet")
|
||||
}
|
||||
for _, m := range pkt.Memories {
|
||||
if m.ID == dropID {
|
||||
t.Errorf("soft-deleted message id=%d should have been dropped", dropID)
|
||||
}
|
||||
}
|
||||
// keep should still be present
|
||||
sawKeep := false
|
||||
for _, m := range pkt.Memories {
|
||||
if m.ID == keepID {
|
||||
sawKeep = true
|
||||
}
|
||||
}
|
||||
if !sawKeep {
|
||||
t.Errorf("active message id=%d should be present, packet=%#v", keepID, pkt.Memories)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,258 @@
|
||||
package search
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
_ "modernc.org/sqlite"
|
||||
|
||||
"github.com/synapbus/synapbus/internal/agents"
|
||||
"github.com/synapbus/synapbus/internal/messaging"
|
||||
)
|
||||
|
||||
// stubCoreProvider lets a test override the core-memory blob returned to
|
||||
// BuildContextPacket.
|
||||
type stubCoreProvider struct {
|
||||
blob string
|
||||
err error
|
||||
}
|
||||
|
||||
func (s *stubCoreProvider) Get(ctx context.Context, ownerID, agentName string) (string, error) {
|
||||
return s.blob, s.err
|
||||
}
|
||||
|
||||
// seedOwnedAgent inserts a users row + an agents row tied to that owner.
|
||||
func seedOwnedAgent(t *testing.T, db *sql.DB, ownerID int64, ownerName, agentName string) *agents.Agent {
|
||||
t.Helper()
|
||||
if _, err := db.Exec(
|
||||
`INSERT OR IGNORE INTO users (id, username, password_hash, display_name)
|
||||
VALUES (?, ?, 'hash', ?)`, ownerID, ownerName, ownerName,
|
||||
); err != nil {
|
||||
t.Fatalf("seed user: %v", err)
|
||||
}
|
||||
if _, err := db.Exec(
|
||||
`INSERT OR IGNORE INTO agents (name, display_name, type, owner_id, api_key_hash, status)
|
||||
VALUES (?, ?, 'ai', ?, ?, 'active')`,
|
||||
agentName, agentName, ownerID, agentName+"hash",
|
||||
); err != nil {
|
||||
t.Fatalf("seed agent: %v", err)
|
||||
}
|
||||
return &agents.Agent{Name: agentName, OwnerID: ownerID}
|
||||
}
|
||||
|
||||
// seedChannel inserts an open-brain channel with members `agentNames`.
|
||||
func seedChannel(t *testing.T, db *sql.DB, channelID int64, name string, agentNames ...string) {
|
||||
t.Helper()
|
||||
if _, err := db.Exec(
|
||||
`INSERT OR IGNORE INTO channels (id, name, description, type, created_by)
|
||||
VALUES (?, ?, '', 'standard', 'system')`, channelID, name,
|
||||
); err != nil {
|
||||
t.Fatalf("seed channel: %v", err)
|
||||
}
|
||||
for _, a := range agentNames {
|
||||
if _, err := db.Exec(
|
||||
`INSERT OR IGNORE INTO channel_members (channel_id, agent_name) VALUES (?, ?)`,
|
||||
channelID, a,
|
||||
); err != nil {
|
||||
t.Fatalf("seed channel_member: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// seedChannelMessage inserts a message directly so we don't need the
|
||||
// channels.Service stack in this test.
|
||||
func seedChannelMessage(t *testing.T, db *sql.DB, channelID int64, fromAgent, body string) int64 {
|
||||
t.Helper()
|
||||
convRes, err := db.Exec(
|
||||
`INSERT INTO conversations (created_by, channel_id) VALUES (?, ?)`,
|
||||
fromAgent, channelID,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("seed conversation: %v", err)
|
||||
}
|
||||
convID, _ := convRes.LastInsertId()
|
||||
|
||||
res, err := db.Exec(
|
||||
`INSERT INTO messages (conversation_id, from_agent, channel_id, body, priority, status, metadata)
|
||||
VALUES (?, ?, ?, ?, 5, 'pending', '{}')`,
|
||||
convID, fromAgent, channelID, body,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("seed message: %v", err)
|
||||
}
|
||||
id, _ := res.LastInsertId()
|
||||
return id
|
||||
}
|
||||
|
||||
func TestBuildContextPacket_BudgetZeroDisables(t *testing.T) {
|
||||
svc, _, db := newTestServices(t)
|
||||
a := seedOwnedAgent(t, db, 1, "alice", "a1")
|
||||
|
||||
pkt, err := BuildContextPacket(context.Background(), svc, a, "query", InjectionOpts{
|
||||
BudgetTokens: 0,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("BuildContextPacket: %v", err)
|
||||
}
|
||||
if pkt != nil {
|
||||
t.Errorf("BudgetTokens=0 should return nil packet, got %+v", pkt)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildContextPacket_OwnerScoping(t *testing.T) {
|
||||
svc, _, db := newTestServices(t)
|
||||
ctx := context.Background()
|
||||
|
||||
h1Agent := seedOwnedAgent(t, db, 1, "alice", "a1")
|
||||
_ = seedOwnedAgent(t, db, 2, "bob", "b1")
|
||||
|
||||
seedChannel(t, db, 1, "open-brain", "a1", "b1")
|
||||
seedChannelMessage(t, db, 1, "a1", "Kuzu graph DB archived 2025")
|
||||
seedChannelMessage(t, db, 1, "b1", "Kuzu graph DB looks promising")
|
||||
|
||||
pkt, err := BuildContextPacket(ctx, svc, h1Agent, "Kuzu", InjectionOpts{
|
||||
BudgetTokens: 500,
|
||||
MaxItems: 5,
|
||||
MinScore: 0.0,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("BuildContextPacket: %v", err)
|
||||
}
|
||||
if pkt == nil {
|
||||
t.Fatal("expected non-nil packet")
|
||||
}
|
||||
for _, m := range pkt.Memories {
|
||||
if m.FromAgent != "a1" {
|
||||
t.Errorf("leaked memory from %q (owner != caller)", m.FromAgent)
|
||||
}
|
||||
if !strings.Contains(m.Body, "Kuzu") {
|
||||
t.Errorf("unexpected body: %q", m.Body)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildContextPacket_TokenBudgetGreedyFillAndTruncate(t *testing.T) {
|
||||
items := []MemoryItem{
|
||||
{ID: 1, Body: strings.Repeat("a", 200), Score: 0.9},
|
||||
{ID: 2, Body: strings.Repeat("b", 200), Score: 0.8},
|
||||
{ID: 3, Body: strings.Repeat("c", 200), Score: 0.7},
|
||||
}
|
||||
// Budget 100 tokens => 400 chars total. Each item costs ~232 chars
|
||||
// → ~58 tokens. First fits cleanly; second must truncate.
|
||||
out := applyTokenBudget(items, 5, 100)
|
||||
|
||||
if len(out) == 0 {
|
||||
t.Fatal("expected at least one admitted item")
|
||||
}
|
||||
if out[0].ID != 1 {
|
||||
t.Errorf("first admitted item ID = %d, want 1", out[0].ID)
|
||||
}
|
||||
total := 0
|
||||
for _, it := range out {
|
||||
total += EstimateTokens(itemChars(it))
|
||||
}
|
||||
if total > 100 {
|
||||
t.Errorf("total tokens admitted = %d, exceeds budget 100", total)
|
||||
}
|
||||
sawTruncated := false
|
||||
for _, it := range out {
|
||||
if it.Truncated {
|
||||
sawTruncated = true
|
||||
}
|
||||
}
|
||||
if len(out) > 1 && !sawTruncated {
|
||||
t.Errorf("expected truncation when admitting a second item under tight budget")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildContextPacket_ScoreFloorDrops(t *testing.T) {
|
||||
svc, _, db := newTestServices(t)
|
||||
ctx := context.Background()
|
||||
_ = seedOwnedAgent(t, db, 1, "alice", "a1")
|
||||
seedChannel(t, db, 1, "open-brain", "a1")
|
||||
|
||||
cidPtr := int64(1)
|
||||
low := &SearchResult{
|
||||
Message: &messaging.Message{
|
||||
ID: 100,
|
||||
FromAgent: "a1",
|
||||
Body: "low score body",
|
||||
ChannelID: &cidPtr,
|
||||
},
|
||||
SimilarityScore: 0.1,
|
||||
MatchType: ModeSemantic,
|
||||
}
|
||||
high := &SearchResult{
|
||||
Message: &messaging.Message{
|
||||
ID: 101,
|
||||
FromAgent: "a1",
|
||||
Body: "high score body",
|
||||
ChannelID: &cidPtr,
|
||||
},
|
||||
SimilarityScore: 0.8,
|
||||
MatchType: ModeSemantic,
|
||||
}
|
||||
|
||||
items, err := filterAndScore(ctx, svc, []*SearchResult{low, high}, "1", InjectionOpts{
|
||||
MinScore: 0.5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("filterAndScore: %v", err)
|
||||
}
|
||||
if len(items) != 1 {
|
||||
t.Fatalf("expected 1 admitted item, got %d", len(items))
|
||||
}
|
||||
if items[0].ID != 101 {
|
||||
t.Errorf("admitted ID = %d, want 101 (high score)", items[0].ID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildContextPacket_CoreMemoryIncluded(t *testing.T) {
|
||||
svc, _, db := newTestServices(t)
|
||||
ctx := context.Background()
|
||||
a := seedOwnedAgent(t, db, 1, "alice", "a1")
|
||||
|
||||
provider := &stubCoreProvider{blob: "I am alice's research agent."}
|
||||
pkt, err := BuildContextPacket(ctx, svc, a, "", InjectionOpts{
|
||||
BudgetTokens: 500,
|
||||
MaxItems: 5,
|
||||
IncludeCore: true,
|
||||
CoreProvider: provider,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("BuildContextPacket: %v", err)
|
||||
}
|
||||
if pkt == nil {
|
||||
t.Fatal("expected non-nil packet when core memory is present")
|
||||
}
|
||||
if pkt.CoreMemory != provider.blob {
|
||||
t.Errorf("CoreMemory = %q, want %q", pkt.CoreMemory, provider.blob)
|
||||
}
|
||||
if pkt.PacketChars < len(provider.blob) {
|
||||
t.Errorf("PacketChars = %d, want >= %d", pkt.PacketChars, len(provider.blob))
|
||||
}
|
||||
if pkt.PacketTokenEstimate != EstimateTokens(pkt.PacketChars) {
|
||||
t.Errorf("PacketTokenEstimate inconsistent with PacketChars")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildContextPacket_EmptyReturnsNil(t *testing.T) {
|
||||
svc, _, db := newTestServices(t)
|
||||
ctx := context.Background()
|
||||
a := seedOwnedAgent(t, db, 1, "alice", "a1")
|
||||
|
||||
// No memories, no core provider → should return (nil, nil) so the
|
||||
// wrapper omits the relevant_context field entirely.
|
||||
pkt, err := BuildContextPacket(ctx, svc, a, "", InjectionOpts{
|
||||
BudgetTokens: 500,
|
||||
MaxItems: 5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("BuildContextPacket: %v", err)
|
||||
}
|
||||
if pkt != nil {
|
||||
t.Errorf("expected nil packet, got %+v", pkt)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TestMemoryStatusView verifies the view's status / superseded_by /
|
||||
// soft_deleted_at derivations against crafted memory_consolidation_jobs
|
||||
// rows. See data-model.md §`memory_status` view.
|
||||
func TestMemoryStatusView(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
|
||||
if err := RunMigrations(ctx, db); err != nil {
|
||||
t.Fatalf("RunMigrations: %v", err)
|
||||
}
|
||||
|
||||
finishedAt := time.Date(2026, 5, 11, 3, 0, 0, 0, time.UTC).Format("2006-01-02 15:04:05")
|
||||
|
||||
// Job 1: completed mark_duplicate where keep_id=100, target=101 (loser).
|
||||
// → message 101 must be soft_deleted; 100 stays active.
|
||||
actionDup := `[{
|
||||
"tool":"memory_mark_duplicate",
|
||||
"target_message_id":101,
|
||||
"args":{"a_id":100,"b_id":101,"keep_id":100,"reason":"shorter paraphrase"},
|
||||
"at":"2026-05-11T03:00:00Z"
|
||||
}]`
|
||||
if _, err := db.Exec(
|
||||
`INSERT INTO memory_consolidation_jobs
|
||||
(owner_id, job_type, status, trigger_reason, actions, finished_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`,
|
||||
"1", "dedup_contradiction", "succeeded", "manual:test", actionDup, finishedAt,
|
||||
); err != nil {
|
||||
t.Fatalf("insert dup job: %v", err)
|
||||
}
|
||||
|
||||
// Job 2: memory_supersede target=200, by=201. → 200 superseded.
|
||||
actionSup := `[{
|
||||
"tool":"memory_supersede",
|
||||
"target_message_id":200,
|
||||
"args":{"a_id":200,"b_id":201,"reason":"newer fact"},
|
||||
"at":"2026-05-11T03:00:00Z"
|
||||
}]`
|
||||
if _, err := db.Exec(
|
||||
`INSERT INTO memory_consolidation_jobs
|
||||
(owner_id, job_type, status, trigger_reason, actions, finished_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`,
|
||||
"1", "dedup_contradiction", "succeeded", "manual:test", actionSup, finishedAt,
|
||||
); err != nil {
|
||||
t.Fatalf("insert supersede job: %v", err)
|
||||
}
|
||||
|
||||
// Failed job — must NOT appear in the view.
|
||||
if _, err := db.Exec(
|
||||
`INSERT INTO memory_consolidation_jobs
|
||||
(owner_id, job_type, status, trigger_reason, actions, finished_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`,
|
||||
"1", "reflection", "failed", "manual:test",
|
||||
`[{"tool":"memory_supersede","target_message_id":300,"args":{"b_id":301}}]`,
|
||||
finishedAt,
|
||||
); err != nil {
|
||||
t.Fatalf("insert failed job: %v", err)
|
||||
}
|
||||
|
||||
statuses, supersededBy, deletedAt := readStatuses(t, db, []int64{100, 101, 200, 201, 300})
|
||||
|
||||
if got := statuses[100]; got != "" {
|
||||
// 100 has no action row → not in view → empty string from the
|
||||
// helper's zero-value default.
|
||||
t.Errorf("100: want active/missing, got %q", got)
|
||||
}
|
||||
if got := statuses[101]; got != "soft_deleted" {
|
||||
t.Errorf("101: want soft_deleted, got %q", got)
|
||||
}
|
||||
if got := deletedAt[101]; got == "" {
|
||||
t.Errorf("101: expected non-empty soft_deleted_at")
|
||||
}
|
||||
if got := statuses[200]; got != "superseded" {
|
||||
t.Errorf("200: want superseded, got %q", got)
|
||||
}
|
||||
if got := supersededBy[200]; got != 201 {
|
||||
t.Errorf("200.superseded_by: want 201, got %d", got)
|
||||
}
|
||||
// 300 was on a failed job → should not appear in view.
|
||||
if got := statuses[300]; got != "" {
|
||||
t.Errorf("300: want missing (failed job), got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func readStatuses(t *testing.T, db *sql.DB, ids []int64) (map[int64]string, map[int64]int64, map[int64]string) {
|
||||
t.Helper()
|
||||
statuses := map[int64]string{}
|
||||
supersededBy := map[int64]int64{}
|
||||
deletedAt := map[int64]string{}
|
||||
|
||||
rows, err := db.Query(
|
||||
`SELECT message_id, status, COALESCE(superseded_by, 0), COALESCE(soft_deleted_at, '')
|
||||
FROM memory_status`,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("query view: %v", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var (
|
||||
id int64
|
||||
status string
|
||||
sup int64
|
||||
deletedTs string
|
||||
)
|
||||
if err := rows.Scan(&id, &status, &sup, &deletedTs); err != nil {
|
||||
t.Fatalf("scan view: %v", err)
|
||||
}
|
||||
statuses[id] = status
|
||||
supersededBy[id] = sup
|
||||
deletedAt[id] = deletedTs
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
t.Fatalf("rows.Err: %v", err)
|
||||
}
|
||||
_ = ids
|
||||
return statuses, supersededBy, deletedAt
|
||||
}
|
||||
@@ -194,3 +194,163 @@ func TestRunMigrations_EmbeddedSchema(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestMigration028_MemoryConsolidation verifies the proactive-memory + dream
|
||||
// worker migration applies cleanly and all six tables plus the memory_status
|
||||
// view are queryable. Doubles as a basic INSERT/SELECT round-trip for each
|
||||
// new table.
|
||||
func TestMigration028_MemoryConsolidation(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
ctx := context.Background()
|
||||
|
||||
if err := RunMigrations(ctx, db); err != nil {
|
||||
t.Fatalf("RunMigrations: %v", err)
|
||||
}
|
||||
|
||||
// 1. Verify all six tables exist.
|
||||
wantTables := []string{
|
||||
"memory_core",
|
||||
"memory_links",
|
||||
"memory_consolidation_jobs",
|
||||
"memory_pins",
|
||||
"memory_dispatch_tokens",
|
||||
"memory_injections",
|
||||
}
|
||||
for _, table := range wantTables {
|
||||
var name string
|
||||
err := db.QueryRow(
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND name=?", table,
|
||||
).Scan(&name)
|
||||
if err != nil {
|
||||
t.Errorf("table %s not found: %v", table, err)
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Verify memory_status view exists.
|
||||
var viewName string
|
||||
if err := db.QueryRow(
|
||||
"SELECT name FROM sqlite_master WHERE type='view' AND name=?", "memory_status",
|
||||
).Scan(&viewName); err != nil {
|
||||
t.Errorf("memory_status view not found: %v", err)
|
||||
}
|
||||
|
||||
// 3. INSERT/SELECT round-trip for each new table.
|
||||
|
||||
// memory_core
|
||||
if _, err := db.ExecContext(ctx,
|
||||
`INSERT INTO memory_core (owner_id, agent_name, blob, updated_by)
|
||||
VALUES (?, ?, ?, ?)`,
|
||||
"1", "agent-a", "core blob", "human:1",
|
||||
); err != nil {
|
||||
t.Fatalf("insert memory_core: %v", err)
|
||||
}
|
||||
var blob string
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT blob FROM memory_core WHERE owner_id=? AND agent_name=?`, "1", "agent-a",
|
||||
).Scan(&blob); err != nil {
|
||||
t.Fatalf("select memory_core: %v", err)
|
||||
}
|
||||
if blob != "core blob" {
|
||||
t.Errorf("memory_core blob = %q, want %q", blob, "core blob")
|
||||
}
|
||||
|
||||
// memory_consolidation_jobs (needed for FK on dispatch tokens)
|
||||
res, err := db.ExecContext(ctx,
|
||||
`INSERT INTO memory_consolidation_jobs (owner_id, job_type, trigger_reason)
|
||||
VALUES (?, ?, ?)`,
|
||||
"1", "reflection", "watermark:25",
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("insert memory_consolidation_jobs: %v", err)
|
||||
}
|
||||
jobID, _ := res.LastInsertId()
|
||||
var status string
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT status FROM memory_consolidation_jobs WHERE id=?`, jobID,
|
||||
).Scan(&status); err != nil {
|
||||
t.Fatalf("select memory_consolidation_jobs: %v", err)
|
||||
}
|
||||
if status != "pending" {
|
||||
t.Errorf("default status = %q, want pending", status)
|
||||
}
|
||||
|
||||
// memory_links
|
||||
if _, err := db.ExecContext(ctx,
|
||||
`INSERT INTO memory_links (src_message_id, dst_message_id, relation_type, owner_id, created_by)
|
||||
VALUES (?, ?, ?, ?, ?)`,
|
||||
1, 2, "refines", "1", "auto:test",
|
||||
); err != nil {
|
||||
t.Fatalf("insert memory_links: %v", err)
|
||||
}
|
||||
var relType string
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT relation_type FROM memory_links WHERE src_message_id=? AND dst_message_id=?`, 1, 2,
|
||||
).Scan(&relType); err != nil {
|
||||
t.Fatalf("select memory_links: %v", err)
|
||||
}
|
||||
if relType != "refines" {
|
||||
t.Errorf("relation_type = %q, want refines", relType)
|
||||
}
|
||||
|
||||
// memory_pins
|
||||
if _, err := db.ExecContext(ctx,
|
||||
`INSERT INTO memory_pins (owner_id, message_id, pinned_by, note)
|
||||
VALUES (?, ?, ?, ?)`,
|
||||
"1", 42, "human:1", "important",
|
||||
); err != nil {
|
||||
t.Fatalf("insert memory_pins: %v", err)
|
||||
}
|
||||
var note string
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT note FROM memory_pins WHERE owner_id=? AND message_id=?`, "1", 42,
|
||||
).Scan(¬e); err != nil {
|
||||
t.Fatalf("select memory_pins: %v", err)
|
||||
}
|
||||
if note != "important" {
|
||||
t.Errorf("note = %q, want important", note)
|
||||
}
|
||||
|
||||
// memory_dispatch_tokens
|
||||
if _, err := db.ExecContext(ctx,
|
||||
`INSERT INTO memory_dispatch_tokens (token, owner_id, consolidation_job_id, expires_at)
|
||||
VALUES (?, ?, ?, datetime('now', '+15 minutes'))`,
|
||||
"tok-abc", "1", jobID,
|
||||
); err != nil {
|
||||
t.Fatalf("insert memory_dispatch_tokens: %v", err)
|
||||
}
|
||||
var ownerID string
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT owner_id FROM memory_dispatch_tokens WHERE token=?`, "tok-abc",
|
||||
).Scan(&ownerID); err != nil {
|
||||
t.Fatalf("select memory_dispatch_tokens: %v", err)
|
||||
}
|
||||
if ownerID != "1" {
|
||||
t.Errorf("owner_id = %q, want 1", ownerID)
|
||||
}
|
||||
|
||||
// memory_injections
|
||||
if _, err := db.ExecContext(ctx,
|
||||
`INSERT INTO memory_injections (owner_id, agent_name, tool_name, packet_size_chars, packet_items_count, message_ids)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`,
|
||||
"1", "agent-a", "my_status", 120, 3, "[1,2,3]",
|
||||
); err != nil {
|
||||
t.Fatalf("insert memory_injections: %v", err)
|
||||
}
|
||||
var items int
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT packet_items_count FROM memory_injections WHERE owner_id=?`, "1",
|
||||
).Scan(&items); err != nil {
|
||||
t.Fatalf("select memory_injections: %v", err)
|
||||
}
|
||||
if items != 3 {
|
||||
t.Errorf("packet_items_count = %d, want 3", items)
|
||||
}
|
||||
|
||||
// memory_status view: query should succeed (empty rows are fine since
|
||||
// the job we inserted is still 'pending').
|
||||
rows, err := db.QueryContext(ctx, `SELECT message_id, owner_id, status FROM memory_status`)
|
||||
if err != nil {
|
||||
t.Fatalf("select memory_status: %v", err)
|
||||
}
|
||||
rows.Close()
|
||||
}
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
-- 027_remove_approval_noise.sql — one-shot cleanup for internal-only mode.
|
||||
-- Drops messages produced by the (now-deleted) stalemate reminder/escalation
|
||||
-- paths and any pending agent proposals. The agent_proposals table itself
|
||||
-- stays in case the propose_agent flow is ever reinstated.
|
||||
--
|
||||
-- The #approvals channel row is intentionally NOT dropped: leaving it
|
||||
-- keeps the migration trivially reversible (no need to recreate the
|
||||
-- channel and re-grant memberships); the human can delete it via admin
|
||||
-- CLI later.
|
||||
--
|
||||
-- Reminder/escalation messages are identified by their metadata JSON
|
||||
-- (see internal/messaging/stalemate.go in git history), which is the
|
||||
-- only field the historical worker stamped reliably — the conversation
|
||||
-- subject prefix was used too but is not a hard guarantee.
|
||||
--
|
||||
-- FK NOTES: most refs to messages(id) are ON DELETE CASCADE / SET NULL,
|
||||
-- but two are NO ACTION and would block this migration on a populated
|
||||
-- DB:
|
||||
-- * attachments.message_id (001_initial.sql)
|
||||
-- * messages.reply_to (007_threads.sql)
|
||||
-- We NULL those explicitly before deleting. Orphan attachment rows (with
|
||||
-- message_id = NULL) are tolerated by the attachments service.
|
||||
|
||||
-- 1. Collect the message ids we're about to drop into a temp scratch
|
||||
-- table so the cascading NULL/DELETE statements all reference the
|
||||
-- same set even if the metadata predicates evolve later.
|
||||
CREATE TEMP TABLE _approval_noise_msgs AS
|
||||
SELECT id FROM messages
|
||||
WHERE metadata LIKE '%"stalemate_reminder_for":%'
|
||||
OR metadata LIKE '%"stalemate_escalation_for":%'
|
||||
OR metadata LIKE '%"workflow_stalemate_reminder_for":%'
|
||||
OR metadata LIKE '%"workflow_stalemate_escalation_for":%'
|
||||
OR channel_id IN (SELECT id FROM channels WHERE name = 'approvals');
|
||||
|
||||
-- 2. Detach NO-ACTION FKs that would otherwise block the delete.
|
||||
UPDATE attachments
|
||||
SET message_id = NULL
|
||||
WHERE message_id IN (SELECT id FROM _approval_noise_msgs);
|
||||
|
||||
UPDATE messages
|
||||
SET reply_to = NULL
|
||||
WHERE reply_to IN (SELECT id FROM _approval_noise_msgs);
|
||||
|
||||
-- 3. Drop the messages themselves. Cascading FKs (reactions, embeddings,
|
||||
-- fts triggers, agent_listings) clean up automatically; SET NULL FKs
|
||||
-- (goals.*_message_id, agent_proposals.*_message_id, goal_tasks.*)
|
||||
-- forget the link gracefully.
|
||||
DELETE FROM messages
|
||||
WHERE id IN (SELECT id FROM _approval_noise_msgs);
|
||||
|
||||
DROP TABLE _approval_noise_msgs;
|
||||
|
||||
-- Conversation rows for stalemate-prefixed subjects are intentionally
|
||||
-- left in place. Multiple NO-ACTION FKs reference conversations(id)
|
||||
-- (messages.conversation_id, inbox_state.conversation_id, possibly
|
||||
-- others added by future migrations); cleaning them up reliably would
|
||||
-- require chasing every ref. Empty / mostly-empty conversation rows
|
||||
-- are harmless — they don't render as messages in the UI.
|
||||
|
||||
-- 4. Drop pending agent proposals. The table stays for reversibility.
|
||||
DELETE FROM agent_proposals;
|
||||
@@ -0,0 +1,153 @@
|
||||
-- 028_memory_consolidation.sql — proactive memory + dream worker.
|
||||
--
|
||||
-- Adds six tables backing per-(owner, agent) core memory, typed memory
|
||||
-- links, dream-worker consolidation jobs, owner-pinned memories,
|
||||
-- single-use dispatch tokens, and a 24h rolling audit ring of what was
|
||||
-- injected into each tool response. Adds one derived view
|
||||
-- `memory_status` over `memory_consolidation_jobs.actions` so retrieval
|
||||
-- can filter out soft-deleted / superseded memories without touching
|
||||
-- the hot `messages` table.
|
||||
--
|
||||
-- `owner_id` is stored as TEXT (string form of `agents.owner_id`) so
|
||||
-- that retrieval can carry the value through unchanged from
|
||||
-- `auth.ContextAgent(ctx)`; the auth middleware already exposes it as a
|
||||
-- string via `trace.ContextWithOwnerID`.
|
||||
--
|
||||
-- All `CREATE TABLE` / `CREATE INDEX` statements use `IF NOT EXISTS`
|
||||
-- so this migration is safe to re-run. CHECK constraints enforce enum
|
||||
-- columns. The partial unique index on
|
||||
-- `memory_consolidation_jobs(owner_id, job_type)` guarantees at most
|
||||
-- one in-flight job per (owner, job_type).
|
||||
|
||||
-- 1. Per-(owner, agent) identity-and-context blob.
|
||||
CREATE TABLE IF NOT EXISTS memory_core (
|
||||
owner_id TEXT NOT NULL,
|
||||
agent_name TEXT NOT NULL,
|
||||
blob TEXT NOT NULL,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_by TEXT NOT NULL,
|
||||
PRIMARY KEY (owner_id, agent_name)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_core_owner ON memory_core(owner_id);
|
||||
|
||||
-- 2. Directed typed edges between two message ids.
|
||||
CREATE TABLE IF NOT EXISTS memory_links (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
src_message_id INTEGER NOT NULL,
|
||||
dst_message_id INTEGER NOT NULL,
|
||||
relation_type TEXT NOT NULL CHECK (relation_type IN (
|
||||
'refines', 'contradicts', 'examples', 'related',
|
||||
'duplicate_of', 'superseded_by',
|
||||
'mention', 'reply_to', 'channel_cooccurrence'
|
||||
)),
|
||||
owner_id TEXT NOT NULL,
|
||||
created_by TEXT NOT NULL,
|
||||
metadata TEXT NOT NULL DEFAULT '{}',
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE(src_message_id, dst_message_id, relation_type)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_links_owner ON memory_links(owner_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_links_src ON memory_links(src_message_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_links_dst ON memory_links(dst_message_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_links_type ON memory_links(relation_type);
|
||||
|
||||
-- 3. Audit log of dispatched dream jobs and their resulting actions.
|
||||
CREATE TABLE IF NOT EXISTS memory_consolidation_jobs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
owner_id TEXT NOT NULL,
|
||||
job_type TEXT NOT NULL CHECK (job_type IN (
|
||||
'reflection', 'core_rewrite', 'dedup_contradiction', 'link_gen'
|
||||
)),
|
||||
status TEXT NOT NULL CHECK (status IN (
|
||||
'pending', 'dispatched', 'running', 'succeeded', 'partial', 'failed', 'expired'
|
||||
)) DEFAULT 'pending',
|
||||
trigger_reason TEXT NOT NULL,
|
||||
dispatch_token TEXT,
|
||||
harness_run_id TEXT,
|
||||
actions TEXT NOT NULL DEFAULT '[]',
|
||||
summary TEXT,
|
||||
error TEXT,
|
||||
lease_until DATETIME,
|
||||
started_at DATETIME,
|
||||
finished_at DATETIME,
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_consolidation_owner_status
|
||||
ON memory_consolidation_jobs(owner_id, status);
|
||||
CREATE INDEX IF NOT EXISTS idx_consolidation_lease
|
||||
ON memory_consolidation_jobs(lease_until)
|
||||
WHERE status = 'running';
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_consolidation_in_flight
|
||||
ON memory_consolidation_jobs(owner_id, job_type)
|
||||
WHERE status IN ('pending', 'dispatched', 'running');
|
||||
|
||||
-- 4. Owner-pinned message ids that bypass the relevance floor.
|
||||
CREATE TABLE IF NOT EXISTS memory_pins (
|
||||
owner_id TEXT NOT NULL,
|
||||
message_id INTEGER NOT NULL,
|
||||
pinned_by TEXT NOT NULL,
|
||||
note TEXT,
|
||||
pinned_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
PRIMARY KEY (owner_id, message_id)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_pins_owner ON memory_pins(owner_id);
|
||||
|
||||
-- 5. Single-use, owner-bound, job-bound dispatch tokens.
|
||||
CREATE TABLE IF NOT EXISTS memory_dispatch_tokens (
|
||||
token TEXT PRIMARY KEY,
|
||||
owner_id TEXT NOT NULL,
|
||||
consolidation_job_id INTEGER NOT NULL,
|
||||
issued_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
expires_at DATETIME NOT NULL,
|
||||
used_at DATETIME,
|
||||
revoked_at DATETIME,
|
||||
FOREIGN KEY (consolidation_job_id) REFERENCES memory_consolidation_jobs(id)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_tokens_job
|
||||
ON memory_dispatch_tokens(consolidation_job_id);
|
||||
|
||||
-- 6. 24-hour rolling audit ring of what was injected for each tool call.
|
||||
CREATE TABLE IF NOT EXISTS memory_injections (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
owner_id TEXT NOT NULL,
|
||||
agent_name TEXT NOT NULL,
|
||||
tool_name TEXT NOT NULL,
|
||||
packet_size_chars INTEGER NOT NULL,
|
||||
packet_items_count INTEGER NOT NULL,
|
||||
message_ids TEXT NOT NULL,
|
||||
core_blob_included BOOLEAN NOT NULL DEFAULT 0,
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_injections_owner_time
|
||||
ON memory_injections(owner_id, created_at);
|
||||
|
||||
-- 7. Derived view: memory_status. Tells retrieval whether a message is
|
||||
-- active / soft_deleted / superseded based on completed consolidation
|
||||
-- jobs' action records. Owners can override via a future restore
|
||||
-- action (out of scope here).
|
||||
CREATE VIEW IF NOT EXISTS memory_status AS
|
||||
WITH actions AS (
|
||||
SELECT
|
||||
j.owner_id,
|
||||
json_extract(act.value, '$.target_message_id') AS message_id,
|
||||
json_extract(act.value, '$.tool') AS tool,
|
||||
json_extract(act.value, '$.args.keep_id') AS keep_id,
|
||||
json_extract(act.value, '$.args.b_id') AS superseded_by,
|
||||
json_extract(act.value, '$.args.reason') AS reason,
|
||||
j.finished_at AS at
|
||||
FROM memory_consolidation_jobs j, json_each(j.actions) act
|
||||
WHERE j.status IN ('succeeded', 'partial')
|
||||
)
|
||||
SELECT
|
||||
message_id,
|
||||
owner_id,
|
||||
CASE
|
||||
WHEN MAX(CASE WHEN tool='memory_supersede' THEN at END) IS NOT NULL THEN 'superseded'
|
||||
WHEN MAX(CASE WHEN tool='memory_mark_duplicate' AND keep_id != message_id THEN at END) IS NOT NULL THEN 'soft_deleted'
|
||||
ELSE 'active'
|
||||
END AS status,
|
||||
MAX(CASE WHEN tool='memory_supersede' THEN superseded_by END) AS superseded_by,
|
||||
MAX(CASE WHEN tool='memory_mark_duplicate' AND keep_id != message_id THEN at END) AS soft_deleted_at,
|
||||
MAX(reason) AS reason
|
||||
FROM actions
|
||||
GROUP BY message_id, owner_id;
|
||||
@@ -0,0 +1,20 @@
|
||||
-- 029_memory_dream_usage.sql — per-(date, owner) circuit-breaker counters
|
||||
-- for the dream worker (feature 020 follow-up). Each row accumulates
|
||||
-- daily token and job consumption so the ConsolidatorWorker can refuse
|
||||
-- to dispatch further jobs once a threshold is hit. Rows are keyed by
|
||||
-- UTC date (YYYY-MM-DD); the circuit naturally resets at midnight UTC
|
||||
-- when Today() begins reading from a fresh row.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS memory_dream_usage (
|
||||
date TEXT NOT NULL, -- YYYY-MM-DD UTC
|
||||
owner_id TEXT NOT NULL,
|
||||
tokens_in INTEGER NOT NULL DEFAULT 0,
|
||||
tokens_out INTEGER NOT NULL DEFAULT 0,
|
||||
jobs_started INTEGER NOT NULL DEFAULT 0,
|
||||
jobs_succeeded INTEGER NOT NULL DEFAULT 0,
|
||||
jobs_failed INTEGER NOT NULL DEFAULT 0,
|
||||
jobs_circuit_broken INTEGER NOT NULL DEFAULT 0,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
PRIMARY KEY (date, owner_id)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_memory_dream_usage_date ON memory_dream_usage(date);
|
||||
@@ -0,0 +1,15 @@
|
||||
-- 030_dream_parallelism.sql
|
||||
--
|
||||
-- Allow concurrent dream-worker dispatches of the same (owner, job_type)
|
||||
-- by adding a `slot` discriminator. Slot 0 is the historical behaviour
|
||||
-- (one in-flight per type). Slots 1..N-1 are used when the worker / CLI
|
||||
-- fans out to drain a backlog quickly.
|
||||
|
||||
ALTER TABLE memory_consolidation_jobs
|
||||
ADD COLUMN slot INTEGER NOT NULL DEFAULT 0;
|
||||
|
||||
DROP INDEX IF EXISTS idx_consolidation_in_flight;
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_consolidation_in_flight
|
||||
ON memory_consolidation_jobs(owner_id, job_type, slot)
|
||||
WHERE status IN ('pending', 'dispatched', 'running');
|
||||
@@ -0,0 +1,21 @@
|
||||
-- 031: Composite index to speed up ExpireTasks() in expiry-worker.
|
||||
--
|
||||
-- Root cause: `ExpireTasks` runs
|
||||
-- UPDATE tasks SET status='cancelled', updated_at=CURRENT_TIMESTAMP
|
||||
-- WHERE status='open' AND deadline IS NOT NULL AND deadline < ?
|
||||
--
|
||||
-- Pre-031 indexes were only `idx_tasks_status(status)` and `idx_tasks_channel(channel_id)`.
|
||||
-- With a status cardinality of 4 and most tasks in two buckets, the planner used
|
||||
-- `idx_tasks_status` to find all open rows then evaluated the deadline predicate
|
||||
-- per row. As the auction-tasks table grew (kubic deploy), the worker's 30s
|
||||
-- context deadline started to be exceeded on every tick, especially under WAL
|
||||
-- write contention from concurrent message inserts.
|
||||
--
|
||||
-- Fix: composite, partial index on `(status, deadline)` covering only rows that
|
||||
-- can ever be expired (status='open' AND deadline IS NOT NULL). This is the
|
||||
-- exact predicate the worker uses, so SQLite can seek straight to the eligible
|
||||
-- rows. The partial form keeps the index tiny once tasks transition out of
|
||||
-- 'open' (the dominant steady-state).
|
||||
CREATE INDEX IF NOT EXISTS idx_tasks_expiry
|
||||
ON tasks(status, deadline)
|
||||
WHERE status = 'open' AND deadline IS NOT NULL;
|
||||
Executable
+40
@@ -0,0 +1,40 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build a linux/amd64 SynapBus image, ship it to the kubic node, import it
|
||||
# into microk8s containerd, and roll the synapbus deployment to that image.
|
||||
#
|
||||
# Usage: scripts/deploy-kubic.sh <version>
|
||||
# e.g. scripts/deploy-kubic.sh v0.17.0
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VERSION="${1:?usage: $0 <version>}"
|
||||
IMAGE_TAG="${VERSION}-amd64"
|
||||
IMAGE="docker.io/library/synapbus:${IMAGE_TAG}"
|
||||
KUBIC_HOST="${KUBIC_HOST:-user@kubic.home.arpa}"
|
||||
NAMESPACE="${NAMESPACE:-synapbus}"
|
||||
DEPLOYMENT="${DEPLOYMENT:-synapbus}"
|
||||
TARBALL="/tmp/synapbus-${IMAGE_TAG}.tar"
|
||||
|
||||
echo "==> building ${IMAGE} (linux/amd64)"
|
||||
docker buildx build \
|
||||
--platform linux/amd64 \
|
||||
--build-arg "VERSION=${VERSION}" \
|
||||
-t "synapbus:${IMAGE_TAG}" \
|
||||
--load .
|
||||
|
||||
echo "==> exporting to ${TARBALL}"
|
||||
docker save "synapbus:${IMAGE_TAG}" -o "${TARBALL}"
|
||||
|
||||
echo "==> shipping to ${KUBIC_HOST}"
|
||||
scp "${TARBALL}" "${KUBIC_HOST}:${TARBALL}"
|
||||
|
||||
echo "==> importing into microk8s containerd"
|
||||
ssh "${KUBIC_HOST}" "sudo microk8s ctr image import ${TARBALL} && rm -f ${TARBALL}"
|
||||
rm -f "${TARBALL}"
|
||||
|
||||
echo "==> rolling deployment ${NAMESPACE}/${DEPLOYMENT} to ${IMAGE}"
|
||||
kubectl set image -n "${NAMESPACE}" "deploy/${DEPLOYMENT}" "${DEPLOYMENT}=${IMAGE}"
|
||||
kubectl rollout status -n "${NAMESPACE}" "deploy/${DEPLOYMENT}" --timeout=180s
|
||||
|
||||
echo "==> done. /healthz:"
|
||||
curl -sf -o /dev/null -w 'status=%{http_code}\n' "http://kubic.home.arpa:30088/healthz" || true
|
||||
@@ -0,0 +1,37 @@
|
||||
# Specification Quality Checklist: Proactive Memory & Dream Worker
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2026-05-11
|
||||
**Feature**: [spec.md](../spec.md)
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [x] No implementation details (languages, frameworks, APIs)
|
||||
- [x] Focused on user value and business needs
|
||||
- [x] Written for non-technical stakeholders
|
||||
- [x] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||
- [x] Requirements are testable and unambiguous
|
||||
- [x] Success criteria are measurable
|
||||
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||
- [x] All acceptance scenarios are defined
|
||||
- [x] Edge cases are identified
|
||||
- [x] Scope is clearly bounded
|
||||
- [x] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [x] All functional requirements have clear acceptance criteria
|
||||
- [x] User scenarios cover primary flows
|
||||
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [x] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- The spec deliberately omits the file paths, table names, and env-var names that surfaced during brainstorming. Those belong in `plan.md`.
|
||||
- Four user stories prioritized P1/P2/P2/P3. P1 (injection) is the MVP and independently shippable. P3 (audit UI) gates enabling the dream worker on real owner data.
|
||||
- Nine measurable success criteria, all technology-agnostic.
|
||||
- The "Assumptions" section bakes in defaults that were discussed during brainstorming so no clarification questions are needed.
|
||||
@@ -0,0 +1,78 @@
|
||||
# Contract — MCP Tool Response Injection
|
||||
|
||||
## Wrapper shape
|
||||
|
||||
For every injection-eligible MCP tool, the response JSON body gains an optional `relevant_context` field appended at the top level:
|
||||
|
||||
```json
|
||||
{
|
||||
"<existing tool result fields>": "...",
|
||||
"relevant_context": {
|
||||
"memories": [
|
||||
{
|
||||
"id": 12345,
|
||||
"from_agent": "research-mcpproxy",
|
||||
"channel": "open-brain",
|
||||
"body": "KuzuDB archived 2025-10-10; not viable for synapbus",
|
||||
"created_at": "2026-05-08T14:22:00Z",
|
||||
"score": 0.91,
|
||||
"match_type": "hybrid",
|
||||
"pinned": false,
|
||||
"truncated": false
|
||||
}
|
||||
],
|
||||
"core_memory": "<text>", // only on session-start tools, may be omitted
|
||||
"packet_chars": 412,
|
||||
"packet_token_estimate": 103,
|
||||
"retrieval_query": "Kuzu graph DB",
|
||||
"search_mode": "auto"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Field semantics
|
||||
|
||||
- `memories` — up to N items (default 5, env `SYNAPBUS_INJECTION_MAX_ITEMS`), ranked by RRF score. Body is verbatim from the message; `truncated=true` only when a single item had to be cut to fit the token budget.
|
||||
- `core_memory` — Letta-style per-(owner, agent) blob. Present only on session-start-class tools (`my_status` today; extensible). Omitted entirely when no blob is set.
|
||||
- `packet_chars` — total character count of the assembled packet (memories + core + delimiters).
|
||||
- `packet_token_estimate` — `packet_chars/4` rounded up.
|
||||
- `retrieval_query` — the text that drove retrieval (the tool's argument body, or "<recent activity>" fallback). Useful for "why did my agent know this?" debug.
|
||||
- `search_mode` — `"auto"` / `"semantic"` / `"fulltext"` per `search.Service`.
|
||||
|
||||
## Injection-eligible tools
|
||||
|
||||
| Tool | Retrieval query source | Include core memory? |
|
||||
|------|------------------------|----------------------|
|
||||
| `my_status` | `<recent activity>` | ✅ |
|
||||
| `claim_messages` | concatenated bodies of claimed messages | ❌ |
|
||||
| `read_inbox` | concatenated bodies of returned messages | ❌ |
|
||||
| `send_message` | body of the sent message | ❌ |
|
||||
| `search` / `search_messages` | the user's query | ❌ |
|
||||
| `execute` | the request payload (stringified args) | ❌ |
|
||||
| `read_channel` | channel topic + last N message bodies | ❌ |
|
||||
|
||||
Tools NOT injected: `list_resources`, `propose_agent`, `propose_task_tree`, `complete_goal`, `create_goal`, `claim_task`, `get_replies`, `request_resource`, `react` (write-only or metadata).
|
||||
|
||||
## Configuration
|
||||
|
||||
| Env var | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `SYNAPBUS_INJECTION_ENABLED` | `0` (off) | Master switch. Off → no `relevant_context` field on any response (FR-012, SC-009). |
|
||||
| `SYNAPBUS_INJECTION_BUDGET_TOKENS` | `500` | Soft cap. Greedy fill in descending score; truncate last admitted item if needed. |
|
||||
| `SYNAPBUS_INJECTION_MAX_ITEMS` | `5` | Hard cap on `memories[]` length. |
|
||||
| `SYNAPBUS_INJECTION_MIN_SCORE` | inherits `search.DefaultMinSimilarity` (0.25) | Floor — pinned memories bypass this. |
|
||||
| `SYNAPBUS_CORE_MEMORY_MAX_BYTES` | `2048` | Reject `memory_rewrite_core` over this size. |
|
||||
|
||||
## Empty / low-signal paths
|
||||
|
||||
- If `len(memories) == 0` and no core memory is set → omit the `relevant_context` field entirely. Response shape exactly matches pre-feature.
|
||||
- If `len(memories) == 0` but core memory IS set → include the field with `memories: []` and `core_memory: "..."`.
|
||||
- If a tool's response is not JSON-shaped (e.g. binary attachment download), pass through unchanged.
|
||||
|
||||
## Cross-owner safety (SC-008)
|
||||
|
||||
Retrieval is filtered by `caller.OwnerID` at the SQL level inside `search.Service.Search()`. The injection middleware never inspects or trusts the response body for owner info — it always re-derives owner from `auth.ContextAgent(ctx)`. An adversarial test asserts that an agent owned by H1 making *any* injection-eligible tool call cannot have a memory whose source message was authored by an agent owned by H2 appear in `relevant_context.memories`.
|
||||
|
||||
## Audit (FR-025)
|
||||
|
||||
Every assembled non-empty packet writes one row to `memory_injections` with `(owner_id, agent_name, tool_name, packet_size_chars, packet_items_count, message_ids[], core_blob_included, created_at)`. Rows older than 24h are purged hourly by the consolidator worker.
|
||||
@@ -0,0 +1,226 @@
|
||||
# Contract — MCP Memory-Consolidation Tools
|
||||
|
||||
These six tools are registered only when `SYNAPBUS_DREAM_ENABLED=1`. They reject any caller that does not present a valid `SYNAPBUS_DISPATCH_TOKEN` (header `X-Synapbus-Dispatch-Token` or env propagated by the harness). Token validation is described in `research.md` R7.
|
||||
|
||||
All six tools accept an implicit `dispatch_token` from the request context and explicit `owner_id` (asserted to match the token's owner; otherwise error `dispatch_token_owner_mismatch`).
|
||||
|
||||
---
|
||||
|
||||
## 1. `memory_list_unprocessed`
|
||||
|
||||
List recent memory-eligible messages the owner's pool has not yet consolidated.
|
||||
|
||||
**Input**:
|
||||
```json
|
||||
{
|
||||
"owner_id": "algis",
|
||||
"since_message_id": 0, // optional — exclusive lower bound
|
||||
"limit": 50 // optional — default 50, max 200
|
||||
}
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```json
|
||||
{
|
||||
"memories": [
|
||||
{
|
||||
"id": 12345,
|
||||
"from_agent": "research-mcpproxy",
|
||||
"channel": "open-brain",
|
||||
"body": "...",
|
||||
"created_at": "2026-05-08T14:22:00Z",
|
||||
"links": [{"to": 12000, "type": "mention"}]
|
||||
}
|
||||
],
|
||||
"max_id_returned": 12567 // pass as since_message_id on next call
|
||||
}
|
||||
```
|
||||
|
||||
**Behavior**: Returns active memories from memory channels for the owner, ordered by id ascending, excluding any already linked via `refines` / `duplicate_of` / `superseded_by` to a more recent memory.
|
||||
|
||||
---
|
||||
|
||||
## 2. `memory_write_reflection`
|
||||
|
||||
Write a higher-level abstraction back to the memory pool as a new message in `#open-brain` (or a `#reflections-<owner>` channel if one exists), tagged.
|
||||
|
||||
**Input**:
|
||||
```json
|
||||
{
|
||||
"owner_id": "algis",
|
||||
"body": "Across recent discussions, the team has committed to ...",
|
||||
"source_message_ids": [12001, 12030, 12089],
|
||||
"tags": ["reflection", "weekly"]
|
||||
}
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```json
|
||||
{
|
||||
"memory_id": 12601,
|
||||
"channel": "open-brain",
|
||||
"links_created": 3 // one 'refines' link per source
|
||||
}
|
||||
```
|
||||
|
||||
**Behavior**:
|
||||
1. Inserts a message authored by a synthetic `dream:<owner>` agent into the memory channel (preferring `#reflections-<owner>` if it exists, else `#open-brain`).
|
||||
2. Inserts a `refines` link from the new memory to each source.
|
||||
3. Embeds the new message via the existing async pipeline.
|
||||
4. Records `{tool, args, target_message_id: new_id}` in the parent job's `actions` JSON.
|
||||
|
||||
**Errors**: `source_not_found` if any source ID is missing or belongs to a different owner.
|
||||
|
||||
---
|
||||
|
||||
## 3. `memory_rewrite_core`
|
||||
|
||||
Replace the per-(owner, agent) core memory blob wholesale.
|
||||
|
||||
**Input**:
|
||||
```json
|
||||
{
|
||||
"owner_id": "algis",
|
||||
"agent_name": "research-mcpproxy",
|
||||
"blob": "You are research-mcpproxy. Currently focused on benchmarking ..."
|
||||
}
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```json
|
||||
{
|
||||
"owner_id": "algis",
|
||||
"agent_name": "research-mcpproxy",
|
||||
"previous_blob": "...",
|
||||
"new_blob_chars": 412,
|
||||
"updated_at": "2026-05-11T03:00:14Z"
|
||||
}
|
||||
```
|
||||
|
||||
**Errors**:
|
||||
- `core_memory_too_large` if `len(blob) > SYNAPBUS_CORE_MEMORY_MAX_BYTES`.
|
||||
- `agent_not_owned` if the target agent's `owner_id != caller.owner_id`.
|
||||
- `agent_protected` if the agent has `metadata.protected_core=true`.
|
||||
|
||||
---
|
||||
|
||||
## 4. `memory_mark_duplicate`
|
||||
|
||||
Mark two memories as duplicates; one is kept canonical, the other is soft-deleted.
|
||||
|
||||
**Input**:
|
||||
```json
|
||||
{
|
||||
"owner_id": "algis",
|
||||
"a_id": 12001,
|
||||
"b_id": 12089,
|
||||
"keep_id": 12001,
|
||||
"reason": "Same fact about KuzuDB archival, b is shorter paraphrase"
|
||||
}
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```json
|
||||
{
|
||||
"keep_id": 12001,
|
||||
"soft_deleted_id": 12089,
|
||||
"link_created_id": 4501
|
||||
}
|
||||
```
|
||||
|
||||
**Behavior**:
|
||||
1. Inserts a `duplicate_of` link from `loser_id → keep_id`.
|
||||
2. Appends `{tool: "memory_mark_duplicate", target_message_id: loser_id, args: {keep_id, ...}}` to the job's `actions` JSON.
|
||||
3. The `memory_status` view then derives `loser_id` as `soft_deleted`.
|
||||
|
||||
**Errors**: `not_same_owner`, `keep_id_not_in_pair`, `already_duplicate`.
|
||||
|
||||
---
|
||||
|
||||
## 5. `memory_supersede`
|
||||
|
||||
Mark memory A as obsoleted by memory B (temporal validity).
|
||||
|
||||
**Input**:
|
||||
```json
|
||||
{
|
||||
"owner_id": "algis",
|
||||
"a_id": 12001,
|
||||
"b_id": 12500,
|
||||
"reason": "Fact updated: as of 2026-04, kubernetes deploy moved off helm chart"
|
||||
}
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```json
|
||||
{
|
||||
"superseded_id": 12001,
|
||||
"by_id": 12500,
|
||||
"link_created_id": 4502
|
||||
}
|
||||
```
|
||||
|
||||
**Behavior**: Inserts `superseded_by` link from `a → b`. View derives `a.status = 'superseded'`, `a.superseded_by = b`. Reason is preserved in the action JSON.
|
||||
|
||||
**Errors**: `not_same_owner`, `cycle_detected` (b transitively superseded by a).
|
||||
|
||||
---
|
||||
|
||||
## 6. `memory_add_link`
|
||||
|
||||
Add a typed link between two memories (A-MEM Zettelkasten style).
|
||||
|
||||
**Input**:
|
||||
```json
|
||||
{
|
||||
"owner_id": "algis",
|
||||
"src_id": 12030,
|
||||
"dst_id": 12085,
|
||||
"relation_type": "refines",
|
||||
"metadata": {"confidence": 0.84}
|
||||
}
|
||||
```
|
||||
|
||||
**Output**:
|
||||
```json
|
||||
{
|
||||
"link_id": 4510
|
||||
}
|
||||
```
|
||||
|
||||
**Constraints**:
|
||||
- `relation_type` ∈ {`refines`, `contradicts`, `examples`, `related`}. The reserved auto-types (`mention`, `reply_to`, `channel_cooccurrence`) and consolidation-types (`duplicate_of`, `superseded_by`) are written by other tools / jobs and rejected here.
|
||||
|
||||
**Errors**: `relation_type_reserved`, `not_same_owner`, `link_already_exists`.
|
||||
|
||||
---
|
||||
|
||||
## Common error codes
|
||||
|
||||
| Code | Meaning |
|
||||
|------|---------|
|
||||
| `dispatch_token_missing` | No token in request context |
|
||||
| `dispatch_token_expired` | Token past `expires_at` |
|
||||
| `dispatch_token_revoked` | Token explicitly revoked |
|
||||
| `dispatch_token_owner_mismatch` | Request's `owner_id` differs from token's |
|
||||
| `dispatch_token_wrong_job` | Token bound to a different `consolidation_job_id` than active call sequence |
|
||||
| `not_same_owner` | A referenced message belongs to a different owner |
|
||||
| `source_not_found` | Referenced message id does not exist |
|
||||
| `core_memory_too_large` | Blob exceeds size cap |
|
||||
|
||||
All errors follow MCP's standard `{"error": {"code": "...", "message": "..."}}` shape.
|
||||
|
||||
## Audit interaction
|
||||
|
||||
Every successful invocation of any of the six tools appends an entry to the parent `memory_consolidation_jobs.actions` JSON array:
|
||||
|
||||
```json
|
||||
{
|
||||
"tool": "memory_mark_duplicate",
|
||||
"args": {"a_id": 12001, "b_id": 12089, "keep_id": 12001, "reason": "..."},
|
||||
"target_message_id": 12089,
|
||||
"at": "2026-05-11T03:00:14Z"
|
||||
}
|
||||
```
|
||||
|
||||
On job completion (success / partial / failed), the worker updates `status`, `summary`, `finished_at`. The token's `used_at` is set on first call; the token remains usable for the rest of the same job.
|
||||
@@ -0,0 +1,210 @@
|
||||
# Data Model — 020-proactive-memory-dream-worker
|
||||
|
||||
## Reused tables (unchanged)
|
||||
|
||||
- **`messages`** — every "memory" is a message on a memory-flagged channel. No schema change.
|
||||
- **`channels`** — `metadata` JSON gains an optional `"is_memory": true` flag. No schema change.
|
||||
- **`agents`** — `owner_id` is the canonical scope. No schema change.
|
||||
- **`embeddings`** — existing per-message embeddings used unchanged by `search.Service`.
|
||||
|
||||
## New tables (migration `028_memory_consolidation.sql`)
|
||||
|
||||
### `memory_core`
|
||||
|
||||
Per-(owner, agent_name) identity-and-context blob.
|
||||
|
||||
```sql
|
||||
CREATE TABLE memory_core (
|
||||
owner_id TEXT NOT NULL,
|
||||
agent_name TEXT NOT NULL,
|
||||
blob TEXT NOT NULL,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_by TEXT NOT NULL, -- 'human:<owner>' or 'agent:<name>:<token>'
|
||||
PRIMARY KEY (owner_id, agent_name)
|
||||
);
|
||||
CREATE INDEX idx_memory_core_owner ON memory_core(owner_id);
|
||||
```
|
||||
|
||||
Constraints:
|
||||
- `LENGTH(blob) <= SYNAPBUS_CORE_MEMORY_MAX_BYTES` (default 2048) — enforced in Go, not SQL.
|
||||
- Replaces wholesale on update — no diff/merge.
|
||||
|
||||
### `memory_links`
|
||||
|
||||
Directed typed edges between two message IDs.
|
||||
|
||||
```sql
|
||||
CREATE TABLE memory_links (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
src_message_id INTEGER NOT NULL,
|
||||
dst_message_id INTEGER NOT NULL,
|
||||
relation_type TEXT NOT NULL CHECK (relation_type IN (
|
||||
'refines', 'contradicts', 'examples', 'related',
|
||||
'duplicate_of', 'superseded_by',
|
||||
'mention', 'reply_to', 'channel_cooccurrence'
|
||||
)),
|
||||
owner_id TEXT NOT NULL, -- denormalized for fast owner-scoped queries
|
||||
created_by TEXT NOT NULL, -- 'human:<owner>' / 'agent:<name>:<token>' / 'auto:<rule>'
|
||||
metadata TEXT NOT NULL DEFAULT '{}',
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE(src_message_id, dst_message_id, relation_type)
|
||||
);
|
||||
CREATE INDEX idx_memory_links_owner ON memory_links(owner_id);
|
||||
CREATE INDEX idx_memory_links_src ON memory_links(src_message_id);
|
||||
CREATE INDEX idx_memory_links_dst ON memory_links(dst_message_id);
|
||||
CREATE INDEX idx_memory_links_type ON memory_links(relation_type);
|
||||
```
|
||||
|
||||
Relation types:
|
||||
- LLM-generated: `refines`, `contradicts`, `examples`, `related`, `duplicate_of`, `superseded_by`
|
||||
- Auto-generated by messaging layer: `mention`, `reply_to`, `channel_cooccurrence`
|
||||
|
||||
### `memory_consolidation_jobs`
|
||||
|
||||
Audit log of dispatched dream jobs and their resulting actions.
|
||||
|
||||
```sql
|
||||
CREATE TABLE memory_consolidation_jobs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
owner_id TEXT NOT NULL,
|
||||
job_type TEXT NOT NULL CHECK (job_type IN (
|
||||
'reflection', 'core_rewrite', 'dedup_contradiction', 'link_gen'
|
||||
)),
|
||||
status TEXT NOT NULL CHECK (status IN (
|
||||
'pending', 'dispatched', 'running', 'succeeded', 'partial', 'failed', 'expired'
|
||||
)) DEFAULT 'pending',
|
||||
trigger_reason TEXT NOT NULL, -- 'watermark:N', 'cron:nightly', 'manual:<owner>'
|
||||
dispatch_token TEXT, -- FK-ish into memory_dispatch_tokens.token
|
||||
harness_run_id TEXT, -- FK into harness_runs.id (existing)
|
||||
actions TEXT NOT NULL DEFAULT '[]', -- JSON array of {tool, args, before, after}
|
||||
summary TEXT, -- human-readable summary written on completion
|
||||
error TEXT,
|
||||
lease_until DATETIME, -- in-flight lease; null when not running
|
||||
started_at DATETIME,
|
||||
finished_at DATETIME,
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
CREATE INDEX idx_consolidation_owner_status ON memory_consolidation_jobs(owner_id, status);
|
||||
CREATE INDEX idx_consolidation_lease ON memory_consolidation_jobs(lease_until)
|
||||
WHERE status = 'running';
|
||||
CREATE UNIQUE INDEX idx_consolidation_in_flight
|
||||
ON memory_consolidation_jobs(owner_id, job_type)
|
||||
WHERE status IN ('pending', 'dispatched', 'running');
|
||||
```
|
||||
|
||||
The partial-unique index enforces: at most one in-flight job per `(owner, job_type)`.
|
||||
|
||||
### `memory_pins`
|
||||
|
||||
Owner-pinned message IDs that bypass the relevance floor.
|
||||
|
||||
```sql
|
||||
CREATE TABLE memory_pins (
|
||||
owner_id TEXT NOT NULL,
|
||||
message_id INTEGER NOT NULL,
|
||||
pinned_by TEXT NOT NULL, -- 'human:<owner>'
|
||||
note TEXT, -- why this is pinned (free-form)
|
||||
pinned_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
PRIMARY KEY (owner_id, message_id)
|
||||
);
|
||||
CREATE INDEX idx_memory_pins_owner ON memory_pins(owner_id);
|
||||
```
|
||||
|
||||
### `memory_dispatch_tokens`
|
||||
|
||||
Single-use, owner-bound, job-bound tokens.
|
||||
|
||||
```sql
|
||||
CREATE TABLE memory_dispatch_tokens (
|
||||
token TEXT PRIMARY KEY, -- 32-byte random, base64url
|
||||
owner_id TEXT NOT NULL,
|
||||
consolidation_job_id INTEGER NOT NULL,
|
||||
issued_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
expires_at DATETIME NOT NULL, -- issued_at + 15m
|
||||
used_at DATETIME,
|
||||
revoked_at DATETIME,
|
||||
FOREIGN KEY (consolidation_job_id) REFERENCES memory_consolidation_jobs(id)
|
||||
);
|
||||
CREATE INDEX idx_memory_tokens_job ON memory_dispatch_tokens(consolidation_job_id);
|
||||
```
|
||||
|
||||
Token is valid when `revoked_at IS NULL AND expires_at > now() AND consolidation_job_id == claimed job`. `used_at` becomes informational once first set; subsequent calls within the same job are allowed.
|
||||
|
||||
### `memory_injections`
|
||||
|
||||
24-hour rolling ring of what was injected for each tool call.
|
||||
|
||||
```sql
|
||||
CREATE TABLE memory_injections (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
owner_id TEXT NOT NULL,
|
||||
agent_name TEXT NOT NULL,
|
||||
tool_name TEXT NOT NULL,
|
||||
packet_size_chars INTEGER NOT NULL,
|
||||
packet_items_count INTEGER NOT NULL,
|
||||
message_ids TEXT NOT NULL, -- JSON array of message_ids included
|
||||
core_blob_included BOOLEAN NOT NULL DEFAULT 0,
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
CREATE INDEX idx_memory_injections_owner_time ON memory_injections(owner_id, created_at);
|
||||
```
|
||||
|
||||
A daily cleanup query (run by the consolidator worker) deletes `created_at < now() - 24h`.
|
||||
|
||||
## View: `memory_status`
|
||||
|
||||
Derived from `memory_consolidation_jobs.actions`. Tells retrieval whether a message is active / soft-deleted / superseded.
|
||||
|
||||
```sql
|
||||
CREATE VIEW memory_status AS
|
||||
WITH actions AS (
|
||||
SELECT
|
||||
owner_id,
|
||||
json_extract(act.value, '$.target_message_id') AS message_id,
|
||||
json_extract(act.value, '$.tool') AS tool,
|
||||
json_extract(act.value, '$.args.keep_id') AS keep_id,
|
||||
json_extract(act.value, '$.args.b_id') AS superseded_by,
|
||||
json_extract(act.value, '$.args.reason') AS reason,
|
||||
finished_at AS at
|
||||
FROM memory_consolidation_jobs j, json_each(j.actions) act
|
||||
WHERE j.status IN ('succeeded', 'partial')
|
||||
)
|
||||
SELECT
|
||||
message_id,
|
||||
owner_id,
|
||||
CASE
|
||||
WHEN MAX(CASE WHEN tool='memory_supersede' THEN at END) IS NOT NULL THEN 'superseded'
|
||||
WHEN MAX(CASE WHEN tool='memory_mark_duplicate' AND keep_id != message_id THEN at END) IS NOT NULL THEN 'soft_deleted'
|
||||
ELSE 'active'
|
||||
END AS status,
|
||||
MAX(CASE WHEN tool='memory_supersede' THEN superseded_by END) AS superseded_by,
|
||||
MAX(CASE WHEN tool='memory_mark_duplicate' AND keep_id != message_id THEN at END) AS soft_deleted_at,
|
||||
MAX(reason) AS reason
|
||||
FROM actions
|
||||
GROUP BY message_id, owner_id;
|
||||
```
|
||||
|
||||
Retrieval excludes `status != 'active'` unless `pinned` or `include_inactive=true`.
|
||||
|
||||
## State transitions
|
||||
|
||||
```text
|
||||
Memory message (active by default — not in memory_status)
|
||||
│
|
||||
├── memory_mark_duplicate(keep != self) → soft_deleted (via view)
|
||||
│ └── owner restore action overrides → active
|
||||
│
|
||||
├── memory_supersede(b) → superseded (via view)
|
||||
│ └── owner restore → active
|
||||
│
|
||||
└── (no consolidation event) → active
|
||||
```
|
||||
|
||||
Pinning and protection are orthogonal flags (in `memory_pins`; protection is currently piggybacked on a `protected_until` JSON field in `memory_pins.note` — promoted to a column in Phase 2 if needed).
|
||||
|
||||
## Indexes & query patterns
|
||||
|
||||
Hot queries:
|
||||
1. **Injection retrieval**: `search.Service.Search()` already paginates → join `memory_status` to drop non-active → drop in pin overlay → apply token budget.
|
||||
2. **Dream worker watermark check**: `SELECT COUNT(*) FROM messages m JOIN agents a ON m.from_agent=a.name WHERE a.owner_id=? AND m.channel_id IN (memory_channels) AND m.id > last_reflection_max_id`. Owner-filtered + range — uses existing message indexes.
|
||||
3. **Audit lookup**: `SELECT * FROM memory_consolidation_jobs WHERE owner_id=? ORDER BY created_at DESC LIMIT 50`. Covered by `idx_consolidation_owner_status`.
|
||||
@@ -0,0 +1,136 @@
|
||||
# Implementation Plan: Proactive Memory & Dream Worker
|
||||
|
||||
**Branch**: `020-proactive-memory-dream-worker` | **Date**: 2026-05-11 | **Spec**: [spec.md](./spec.md)
|
||||
**Input**: Feature specification from `/specs/020-proactive-memory-dream-worker/spec.md`
|
||||
|
||||
## Summary
|
||||
|
||||
Make SynapBus push minimal owner-scoped memory into every agent task automatically (Story 1, P1), give each agent a small editable identity blob always-included in session-start (Story 2, P2), and run a background "dream" worker that periodically dispatches consolidation work to a Claude-Code agent through the existing `harness.Harness` seam (Story 3, P2). An owner-facing audit/override UI lands later (Story 4, P3). All work reuses existing primitives: `search.Service` for retrieval, `harness.Harness` for dispatch, `messages` table for the memory pool, the `StalemateWorker` ticker pattern for the new `ConsolidatorWorker`. One new SQLite migration adds six tables. Zero CGO, zero new external dependencies.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Go 1.25+ (per `go.mod`)
|
||||
**Primary Dependencies**: `mark3labs/mcp-go` (MCP tools), `go-chi/chi` (HTTP), `modernc.org/sqlite` (storage), `TFMV/hnsw` (vectors via existing `search.Service`), existing `internal/harness` package (dispatch seam). **No new external dependencies.**
|
||||
**Storage**: SQLite via `modernc.org/sqlite` — one new migration `028_memory_consolidation.sql`. Memory pool reuses the existing `messages` table on memory-flagged channels.
|
||||
**Testing**: `go test ./...` table-driven tests, mocked `harness.Harness` for dream-worker tests, in-memory SQLite for storage tests
|
||||
**Target Platform**: `linux/amd64` (kubic deployment), `darwin/arm64` (dev). Pure-Go cross-compilation required.
|
||||
**Project Type**: Single-binary Go service with embedded Svelte SPA (existing layout).
|
||||
**Performance Goals**: Median injection overhead < 50ms per tool call (SC-002). Full owner-level consolidation pass ≤ 10 min for ≤ 5,000 memories (SC-005).
|
||||
**Constraints**: Zero CGO (Constitution III). Single binary (Constitution I). Injection MUST be disable-able with zero payload-shape change when disabled (FR-012, SC-009).
|
||||
**Scale/Scope**: One owner with ~500 memories today; design for 5k/owner, 10 owners on the reference deployment.
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*Gate: must pass before Phase 0 research. Re-evaluated after Phase 1 design.*
|
||||
|
||||
| Principle | Check | Status |
|
||||
|-----------|-------|--------|
|
||||
| I. Local-first, single binary | New worker ships in the same binary; no external service introduced. | ✅ |
|
||||
| II. MCP-native | New memory-consolidation tools registered via `mark3labs/mcp-go`. REST endpoints are added only for the Web UI audit tab (Phase 2). | ✅ |
|
||||
| III. Pure Go, zero CGO | No new dependencies; reuses `modernc.org/sqlite` + `TFMV/hnsw`. | ✅ |
|
||||
| IV. Multi-tenant w/ ownership | Owner scoping is the *core* of the design. Retrieval filters by `owner_id`; dispatch tokens are owner-bound. | ✅ |
|
||||
| V. Embedded OAuth 2.1 | Unchanged; consolidation agents authenticate via the existing API-key path. | ✅ |
|
||||
| VI. Semantic-ready storage | Reuses `search.Service.Search()`; gracefully degrades to FTS when no embedding provider configured. | ✅ |
|
||||
| VII. Swarm intelligence | `#open-brain` is a stigmergic blackboard already. Reflection writes annotated memories back to the blackboard. | ✅ |
|
||||
| VIII. Observable by default | Every consolidation mutation hits an immutable `memory_consolidation_jobs` audit row + a `memory_audit_log` row keyed by dispatch token. Every injection hits `memory_injections` (24h ring). | ✅ |
|
||||
| IX. Progressive complexity | Feature is gated by `SYNAPBUS_INJECTION_ENABLED` and `SYNAPBUS_DREAM_ENABLED`. Default off until owner audit UI lands. | ✅ |
|
||||
| X. Web UI first-class | Memory tab is in scope but deferred to Phase 2 of this feature (post-kubic-deploy). Audit data is captured from day one so the UI can render history retroactively. | ⚠ deferred — tracked, not skipped |
|
||||
|
||||
**Gate result**: PASS. No violations require Complexity Tracking.
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/020-proactive-memory-dream-worker/
|
||||
├── plan.md # This file
|
||||
├── spec.md # Feature spec
|
||||
├── research.md # Phase 0 — technical-unknowns resolution
|
||||
├── data-model.md # Phase 1 — entities and tables
|
||||
├── contracts/
|
||||
│ ├── mcp-injection.md # Shape of relevant_context block on tool responses
|
||||
│ └── mcp-memory-tools.md # 6 new dream-agent tools (input schemas, behavior, errors)
|
||||
├── quickstart.md # Phase 1 — how to verify locally and on kubic
|
||||
├── tasks.md # (Phase 2 — produced by /speckit.tasks)
|
||||
└── checklists/
|
||||
└── requirements.md # Spec quality checklist
|
||||
```
|
||||
|
||||
### Source Code (repository root, existing layout extended)
|
||||
|
||||
```text
|
||||
cmd/synapbus/main.go # MODIFIED: wire up ConsolidatorWorker, gate new MCP tools
|
||||
|
||||
internal/
|
||||
├── messaging/
|
||||
│ ├── consolidator.go # NEW: ConsolidatorWorker (ticker + dispatch via harness)
|
||||
│ ├── consolidator_test.go # NEW
|
||||
│ ├── memory.go # NEW: core-memory CRUD, link CRUD, supersession, pin/protected
|
||||
│ ├── memory_test.go # NEW
|
||||
│ ├── memory_channels.go # NEW: discover memory-flagged channels per owner
|
||||
│ └── stalemate.go # UNCHANGED — pattern template
|
||||
├── search/
|
||||
│ └── injection.go # NEW: BuildContextPacket(ctx, agent, opts) → ContextPacket
|
||||
├── mcp/
|
||||
│ ├── injection_wrap.go # NEW: middleware appending relevant_context to tool results
|
||||
│ ├── memory_tools.go # NEW: 6 dream-agent tools (memory_list_unprocessed,
|
||||
│ │ # memory_write_reflection, memory_rewrite_core,
|
||||
│ │ # memory_mark_duplicate, memory_supersede, memory_add_link)
|
||||
│ ├── memory_tools_test.go # NEW
|
||||
│ ├── tools_hybrid.go # MODIFIED: wrap eligible tool handlers via injection_wrap
|
||||
│ └── server.go # MODIFIED: register memory tools when dream worker enabled
|
||||
├── storage/schema/
|
||||
│ └── 028_memory_consolidation.sql # NEW: 6 tables (see data-model.md)
|
||||
└── harness/ # UNCHANGED — used via existing Execute() API
|
||||
|
||||
docs/superpowers/specs/2026-05-11-internal-only-disable-approvals-design.md # pre-existing, untouched
|
||||
```
|
||||
|
||||
**Structure Decision**: Single-project Go layout (matches existing `cmd/synapbus` + `internal/*` structure). Memory pool reuses `messages`; six new tables in one new migration are the only schema addition. No new top-level package — new files split between `internal/messaging` (storage-adjacent), `internal/search` (retrieval-adjacent), and `internal/mcp` (tool-surface-adjacent).
|
||||
|
||||
## Phase 0 — Research
|
||||
|
||||
See [research.md](./research.md). Resolved unknowns:
|
||||
|
||||
1. **How does the dream worker invoke a Claude Code agent without using a system DM?** → through `harness.Harness.Execute(ExecRequest)` with the existing `kubernetes_job` or `local_subprocess` backend. The agent's MCP session carries a one-time `dispatch_token` injected via `ExecRequest.Env` that authorizes the consolidation tools.
|
||||
2. **Where do owners live in the data model today?** → `agents.owner_id` (text) is the canonical scope. Every retrieval JOINs `agents` to resolve the caller's owner.
|
||||
3. **Soft-delete on messages — new column or metadata flag?** → metadata flag on `memory_consolidation_jobs.actions` + a derived view. Avoids touching the hot `messages` table. Soft-delete is enforced at retrieval time by joining against `memory_status` view.
|
||||
4. **Token-budget estimator** → `len(s)/4` heuristic. Faster than tokenizing, accurate enough for budget enforcement.
|
||||
5. **MCP transport — how do we identify the calling agent's API key/owner?** → existing `auth.ContextAgent(ctx)` returns `*agents.Agent` from the request context; `agent.OwnerID` is the scope key.
|
||||
|
||||
## Phase 1 — Design & Contracts
|
||||
|
||||
### Data model
|
||||
|
||||
See [data-model.md](./data-model.md). Migration `028_memory_consolidation.sql` adds six tables:
|
||||
|
||||
- `memory_core` — per-(owner, agent_name) editable text blob, size-capped at `SYNAPBUS_CORE_MEMORY_MAX_BYTES` (default 2048).
|
||||
- `memory_links` — directed typed edges between two message IDs.
|
||||
- `memory_consolidation_jobs` — audit log of every dispatched dream job + the actions it took.
|
||||
- `memory_pins` — owner-pinned message IDs, always-included in injection.
|
||||
- `memory_dispatch_tokens` — one-time, owner-bound, single-job-bound tokens.
|
||||
- `memory_injections` — 24h rolling ring of (tool_name, agent_id, packet_summary) for debug-ability.
|
||||
|
||||
Plus one view `memory_status` (active / soft-deleted / superseded — derived from `memory_consolidation_jobs.actions`).
|
||||
|
||||
### Contracts
|
||||
|
||||
- [contracts/mcp-injection.md](./contracts/mcp-injection.md) — shape of `relevant_context` block appended to tool responses.
|
||||
- [contracts/mcp-memory-tools.md](./contracts/mcp-memory-tools.md) — input/output schemas for the six new memory-consolidation tools.
|
||||
|
||||
### Quickstart
|
||||
|
||||
See [quickstart.md](./quickstart.md) — covers local dev (build → migrate → seed → run with `SYNAPBUS_INJECTION_ENABLED=1 SYNAPBUS_DREAM_ENABLED=1`) and kubic deployment (build image, push, kustomize apply, tail dream-worker logs, dispatch a manual reflection job, verify audit log).
|
||||
|
||||
### Agent context update
|
||||
|
||||
Triggered by the workflow: `update-agent-context.sh claude` adds the new packages and migration to `CLAUDE.md` "Active Technologies" block.
|
||||
|
||||
## Phase 2 — Tasks
|
||||
|
||||
(Produced by `/speckit.tasks`. Not created by this command.)
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
No constitution violations. Memory-tab Web UI is deferred (tracked as Phase 2 of this feature) but is not a constitution violation since audit data is captured from day one and surfaces in CLI/REST until the SPA tab lands.
|
||||
@@ -0,0 +1,162 @@
|
||||
# Quickstart — 020-proactive-memory-dream-worker
|
||||
|
||||
How to build, run, and verify the proactive-memory + dream-worker feature locally and on the kubic deployment.
|
||||
|
||||
## Local: build & test
|
||||
|
||||
```bash
|
||||
cd ~/repos/synapbus
|
||||
make test # full test suite — should be green
|
||||
go build -o ./synapbus ./cmd/synapbus
|
||||
```
|
||||
|
||||
## Local: run with feature flags on
|
||||
|
||||
```bash
|
||||
export SYNAPBUS_DATA_DIR=/tmp/synapbus-dream
|
||||
export SYNAPBUS_INJECTION_ENABLED=1
|
||||
export SYNAPBUS_DREAM_ENABLED=1
|
||||
export SYNAPBUS_EMBEDDING_PROVIDER=openai
|
||||
export OPENAI_API_KEY=<key>
|
||||
mkdir -p $SYNAPBUS_DATA_DIR
|
||||
|
||||
./synapbus serve --port 8080 --data $SYNAPBUS_DATA_DIR
|
||||
```
|
||||
|
||||
## Verify Story 1 (injection)
|
||||
|
||||
```bash
|
||||
# Create an agent and an owner
|
||||
./synapbus --socket $SYNAPBUS_DATA_DIR/synapbus.sock agents create \
|
||||
--name dogfood-1 --owner algis --description "smoke test agent"
|
||||
|
||||
# Get an API key for the agent
|
||||
KEY=$(./synapbus --socket $SYNAPBUS_DATA_DIR/synapbus.sock apikeys create \
|
||||
--agent dogfood-1 --json | jq -r .key)
|
||||
|
||||
# Seed memory channel with a few messages
|
||||
./synapbus --socket $SYNAPBUS_DATA_DIR/synapbus.sock channels create \
|
||||
--name open-brain --type blackboard
|
||||
./synapbus --socket $SYNAPBUS_DATA_DIR/synapbus.sock messages send \
|
||||
--agent dogfood-1 --channel open-brain \
|
||||
--body "KuzuDB was archived 2025-10-10. Not viable for SynapBus."
|
||||
|
||||
# Call my_status via MCP — relevant_context should appear
|
||||
curl -s -X POST http://localhost:8080/mcp \
|
||||
-H "Authorization: Bearer $KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"my_status","arguments":{}}}' \
|
||||
| jq '.result.content[0].text | fromjson | .relevant_context'
|
||||
```
|
||||
|
||||
Expected: `relevant_context.memories` is non-empty, contains the seeded message, `core_memory` is null/omitted (none set yet).
|
||||
|
||||
Set a core memory and re-run:
|
||||
|
||||
```bash
|
||||
# Direct DB or admin CLI:
|
||||
sqlite3 $SYNAPBUS_DATA_DIR/synapbus.db \
|
||||
"INSERT INTO memory_core(owner_id, agent_name, blob, updated_by)
|
||||
VALUES('algis', 'dogfood-1', 'You are a dogfood test agent.', 'human:algis');"
|
||||
|
||||
# Re-call my_status — relevant_context.core_memory should now be populated
|
||||
```
|
||||
|
||||
## Verify Story 3 (dream worker manual dispatch)
|
||||
|
||||
```bash
|
||||
# Force a reflection job for owner=algis
|
||||
./synapbus --socket $SYNAPBUS_DATA_DIR/synapbus.sock memory dream-run \
|
||||
--owner algis --job reflection
|
||||
|
||||
# Tail logs for the worker
|
||||
# Look for:
|
||||
# component=consolidator-worker job=reflection owner=algis status=dispatched
|
||||
# component=consolidator-worker job=reflection owner=algis status=succeeded
|
||||
sqlite3 $SYNAPBUS_DATA_DIR/synapbus.db \
|
||||
"SELECT id, job_type, status, summary, finished_at
|
||||
FROM memory_consolidation_jobs ORDER BY id DESC LIMIT 5;"
|
||||
```
|
||||
|
||||
Expected: one new row in `memory_consolidation_jobs` with `status=succeeded` (or `partial` if seed pool too small). For a reflection job, expect 0–5 new `body LIKE 'REFLECTION:%'` messages in `#open-brain`.
|
||||
|
||||
## Verify token isolation (SC-008)
|
||||
|
||||
```bash
|
||||
# Create a second owner + agent
|
||||
./synapbus --socket ... agents create --name attacker-1 --owner mallory ...
|
||||
KEY2=$(./synapbus --socket ... apikeys create --agent attacker-1 --json | jq -r .key)
|
||||
|
||||
# Attacker calls search — must NOT see algis's memories
|
||||
curl -s -X POST http://localhost:8080/mcp -H "Authorization: Bearer $KEY2" ... \
|
||||
| jq '.result.content[0].text | fromjson | .relevant_context.memories'
|
||||
```
|
||||
|
||||
Expected: empty array. If anything from algis's pool surfaces, that is a Constitution IV violation.
|
||||
|
||||
## kubic: build, push, deploy
|
||||
|
||||
```bash
|
||||
# Build linux/amd64 from darwin/arm64 — no CGO so this just works
|
||||
cd ~/repos/synapbus
|
||||
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o ./synapbus-linux-amd64 ./cmd/synapbus
|
||||
|
||||
# Build container (existing Dockerfile)
|
||||
docker build --platform=linux/amd64 -t synapbus:020-proactive-memory .
|
||||
|
||||
# Push to the cluster's local registry (kubic uses microk8s registry on :32000)
|
||||
docker tag synapbus:020-proactive-memory kubic.home.arpa:32000/synapbus:020-proactive-memory
|
||||
docker push kubic.home.arpa:32000/synapbus:020-proactive-memory
|
||||
|
||||
# Update the deployment image
|
||||
kubectl -n synapbus set image deploy/synapbus \
|
||||
synapbus=kubic.home.arpa:32000/synapbus:020-proactive-memory
|
||||
|
||||
# Enable feature flags via env (kubectl set env, or edit values in deploy/kubic-manifests/)
|
||||
kubectl -n synapbus set env deploy/synapbus \
|
||||
SYNAPBUS_INJECTION_ENABLED=1 SYNAPBUS_DREAM_ENABLED=1
|
||||
|
||||
kubectl -n synapbus rollout status deploy/synapbus
|
||||
```
|
||||
|
||||
## kubic: verify dream agent works
|
||||
|
||||
```bash
|
||||
# Tail logs filtered to the new worker
|
||||
kubectl -n synapbus logs deploy/synapbus -f | grep -E "consolidator-worker|memory-injection"
|
||||
```
|
||||
|
||||
Expected log lines on startup:
|
||||
```
|
||||
{"component":"consolidator-worker","msg":"consolidator worker started","interval":"1h","deep_cron":"0 3 * * *"}
|
||||
```
|
||||
|
||||
Expected after first watermark / cron tick:
|
||||
```
|
||||
{"component":"consolidator-worker","msg":"trigger fired","owner":"algis","job":"reflection","trigger":"watermark:20"}
|
||||
{"component":"consolidator-worker","msg":"dispatched","owner":"algis","job":"reflection","harness_run_id":"...","dispatch_token":"redacted"}
|
||||
{"component":"consolidator-worker","msg":"job completed","owner":"algis","job":"reflection","status":"succeeded","actions":3,"duration_ms":42184}
|
||||
```
|
||||
|
||||
Force a job to test without waiting:
|
||||
```bash
|
||||
kubectl exec -n synapbus deploy/synapbus -- /synapbus --socket /data/synapbus.sock \
|
||||
memory dream-run --owner algis --job reflection
|
||||
```
|
||||
|
||||
Inspect the audit log:
|
||||
```bash
|
||||
kubectl exec -n synapbus deploy/synapbus -- sqlite3 /data/synapbus.db \
|
||||
"SELECT id, job_type, status, json_array_length(actions) AS n_actions, summary
|
||||
FROM memory_consolidation_jobs ORDER BY id DESC LIMIT 10;"
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
```bash
|
||||
kubectl -n synapbus set env deploy/synapbus SYNAPBUS_INJECTION_ENABLED- SYNAPBUS_DREAM_ENABLED-
|
||||
# Or roll back the image:
|
||||
kubectl -n synapbus rollout undo deploy/synapbus
|
||||
```
|
||||
|
||||
The migration `028_memory_consolidation.sql` is additive — rolling the binary back leaves the new tables in place but unused. Safe.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user