352f9f6c61594673f1230d3bdf0ff0dfeecce0e0
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Algis DumbrisandClaude Opus 4.7
|
bfb2551b45 |
feat(020): configurable dream parallelism + 3 bug fixes from kubic drain
Drain-on-demand: SYNAPBUS_DREAM_PARALLEL (default 1) and `synapbus memory dream-run --parallel N` fan out N concurrent dream-agent k8s Jobs per (owner, job_type) in one shot. Set high (e.g. 8) to drain backlog quickly, then back to 1 for normal hourly operation. Schema: - migration 030_dream_parallelism: adds slot INTEGER NOT NULL DEFAULT 0 to memory_consolidation_jobs. Drops + recreates the partial unique in-flight index as (owner, job_type, slot) so slots 0..N-1 each hold one in-flight job independently. Stores: - JobsStore.CreateOnSlot + CreateNextAvailableSlot. - ConsolidatorWorker.ForceRunN dispatches N parallel jobs through the existing launchOne path (extracted from ForceRun). - core_rewrite coerces to N=1 regardless of the knob — per-(owner, agent) blob is wholesale-replace and concurrent rewrites would race. Three bug fixes discovered while bringing the parallel path up on kubic: 1. k8s Job names collided on rapid relaunch because runner.go used "synapbus-<agent>-<msg_id>", and dream dispatches have msg_id=0. Now appends a unique (timestamp%1e6, 4-byte random) suffix when msg_id is zero; historical "synapbus-<agent>-<id>" prefix preserved. 2. memory_list_unprocessed didn't actually exclude already-refined messages — the contract said it should, the implementation returned the same oldest-50 every cycle. The dream agent kept re-refining the same set: 221 refines links touched only 55 unique dst messages, so progress flat-lined. Added the NOT IN (refines/duplicate_of/superseded_by) filter and a from_agent NOT LIKE 'dream:%' clause so the agent never refines its own reflections. 3. The k8sjob harness was constructed with nil Waiter in main.go, so every dream dispatch failed instantly with "k8sjob: no Waiter configured". Now builds a ClientsetWaiter from the in-cluster clientset. Plus admin/server.go gets DreamRunN closure + DefaultDreamParallel (sourced from MemoryConfig.DreamParallel). admin/socket.go handleMemoryDreamRun accepts `parallel` arg and returns job_ids[]. CLI admin command grows --parallel N flag. Live evidence from kubic (image v0.21.0-amd64): 1 CLI call with --parallel 8 produced 8 job rows on slots 0..7, spawned 8 distinct k8s Jobs with unique suffixes, retired ~86 unprocessed messages in <1 min (vs ~10/cycle for the buggy serial version pre-fix-2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.7
|
2044b199b8 |
feat(020): US3 — dream worker + 6 MCP consolidation tools
Background ConsolidatorWorker dispatches consolidation work to a Claude Code agent through harness.Harness.Execute (NOT via system DMs — per feedback_system_dm_no_trigger.md) with a one-time 15m dispatch token. The dispatched agent uses six new MCP tools, all token-gated and recording every action to memory_consolidation_jobs.actions JSON. Stores: - memory_links.go (+ test): typed edges with actor-prefix reserved-type guard; AddConsolidationLink bypass for memory_mark_duplicate / memory_supersede (their contractual writers). - memory_pins.go (+ test): owner pin overlay, bypasses score floor. - memory_status.go: queries the memory_status view. - consolidation_jobs.go: Create / Dispatch / Lease / AppendAction / Complete with ErrJobAlreadyInFlight via partial unique index. - auto_links.go: MessageListener generating mention / reply_to / channel_cooccurrence links automatically on send. Worker: - consolidator.go (+ test): ticker pattern modeled on StalemateWorker. Watermark trigger for link_gen / dedup_contradiction; daily 03:00 UTC for sleep-time core rewrite. Wallclock budget via harness Budget. Global semaphore gates concurrent owners. Mocked-harness test asserts no system DM is ever sent. - consolidator_prompts.go: four job-type prompts passed via env to the dispatched agent. MCP tools (internal/mcp/memory_tools.go + test): - memory_list_unprocessed, memory_write_reflection, memory_rewrite_core, memory_mark_duplicate, memory_supersede, memory_add_link. - Full error-code matrix tested per contracts/mcp-memory-tools.md. - Registered only when SYNAPBUS_DREAM_ENABLED=1. Injection extensions: - search/injection.go: pin overlay applied after retrieval; status filter drops soft_deleted / superseded unless pinned. New PinProvider, StatusProvider, MessageLookup hooks on InjectionOpts. Wiring: - cmd/synapbus/main.go: stores constructed, AutoLinkListener attached to MessagingService, mcpSrv.SetDream wired, ConsolidatorWorker start/stop, admin DreamRun closure. - cmd/synapbus/admin.go: synapbus memory dream-run --owner --job socket-RPC command (forces a single job bypassing trigger). Cycle workarounds (documented in code): - messaging.DreamAgent / HarnessDispatcher are local interfaces (the agents and harness packages import messaging, not the reverse). main.go wraps the real types via adapter structs. Stubbed: - Cron expression parsing (DreamDeepCron). Hardcoded daily 03:00 UTC. Adding robfig/cron deferred to keep no-new-deps. Pre-existing reactor test failures (5) are unchanged; confirmed pre-020 via stash check. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.7
|
a52d68ed88 |
feat(020): US2 — per-agent core memory blob
Letta-style identity blob, one per (owner, agent), always included in
session-start (my_status) responses. Replaces wholesale on rewrite; size
capped at SYNAPBUS_CORE_MEMORY_MAX_BYTES (default 2048); owner-scoped.
Components:
- internal/messaging/memory_core.go (+ test): CoreMemoryStore with
Get/Set/Delete/List, ErrCoreMemoryTooLarge, NewCoreProvider adapter
for search.CoreMemoryProvider.
- internal/mcp/server.go SetInjection: wires the core provider into
the my_status handler wrap.
- internal/mcp/injection_core_test.go: seed → wrapped my_status →
relevant_context.core_memory matches; missing row → no field.
- internal/api/memory_core.go + router: GET/PUT/DELETE
/api/owner/{ownerID}/agents/{agentName}/core-memory, session-auth,
413 on oversize.
- internal/admin/socket.go: memory.core.{get,set,delete} dispatch
handlers with username→user.id resolution.
- cmd/synapbus/admin.go: `synapbus memory core {get,set,delete}` cobra
subtree.
- cmd/synapbus/main.go: wires ParseMemoryConfig, CoreMemoryStore,
MemoryInjections; calls mcpSrv.SetInjection on startup.
Pin overlay still TODO (US3-T029).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
04e4e3ca37 |
feat(harness): GEMINI.md support + cold-topic-explainer example
Adds everything needed to run a real multi-Gemini-model reactive agent
loop end-to-end on SynapBus.
internal/harness/subprocess/config.go:
* AgentConfig.GeminiMD — content of workdir/GEMINI.md
* MaterialiseAgentConfig writes GEMINI.md AND workdir/.gemini/settings.json
when gemini_md is set. The settings file carries the same mcp_servers
list as .mcp.json (so a Gemini child running from the workdir gets
the exact MCP surface the operator configured, not the user's
~/.gemini/settings.json).
* 2 new config_test cases: GEMINI.md + .gemini/settings.json round
trip, GEMINI.md with empty mcp_servers still writes the settings
file (explicitly clearing any inherited home config).
internal/harness/registry.go — BUG FIX:
Resolve() now honours agent.HarnessName (explicit selection) BEFORE
the inference chain, matching the reactor's own agentBackendKind
policy. Previously, when multiple backends were registered,
Resolve would pick "webhook" for every non-K8s agent — even when the
agent's HarnessName was "subprocess" — because the original fallback
chain put webhook first. This is why the first cold-topic-explainer
run failed with "webhook: agent has no webhook config". Discovered
during e2e testing.
internal/admin/socket.go + cmd/synapbus/admin.go:
New `messages.send` admin command (socket + CLI). Sends a DM as any
agent through the messaging service, bypassing the REST/MCP auth
layers. Local-only via the admin Unix socket, so the threat model is
"whoever can reach the socket is already admin".
CLI:
synapbus messages send --from X --to Y --body "..." [--priority N]
synapbus messages send --from X --to Y --body-file path
echo "..." | synapbus messages send --from X --to Y
Used by the harness shell wrappers (so Gemini subprocess agents can
DM each other) and by run_task.sh (to kick off a chain as a human
user without implementing the REST session flow).
examples/cold-topic-explainer/ (NEW):
Runnable 3-agent Gemini demo that exercises the subprocess harness,
reactive triggers, recursive update, and all the preconditions (depth,
budget, cooldown) end-to-end on a separate isolated synapbus instance.
Layout:
README.md — usage + troubleshooting + cost notes
start.sh — builds synapbus, launches on port 18088 with
./data, creates user + agents + harness configs,
marks agents reactive via sqlite3
run_task.sh — sends initial DM algis → decomposer-pro, polls
reactive_runs + messages for the FINAL: reply,
prints the result or dumps reactive_runs on
timeout for debugging
stop.sh — SIGTERM + 5s grace + SIGKILL fallback
wrapper.sh — shared subprocess local_command: reads
message.json + GEMINI.md, calls gemini headless
with --approval-mode yolo, strips the
"MCP issues detected" noise prefix, routes the
cleaned response to the next agent via
`synapbus messages send` over the admin socket
configs/
decomposer-pro.json — gemini-3.1-pro-preview
(gemini-2.5-pro is currently capacity-
exhausted on Google's side)
writer-flash.json — gemini-2.5-flash
critic-lite.json — gemini-2.5-flash-lite
.gitignore — data/, bin/, synapbus.log, .synapbus.pid
The wrapper does NOT rely on gemini's MCP tool-calling (which was
unreliable in testing). Gemini is used as a pure text generator; the
shell decides routing based on AGENT_ROLE:
- decomposer → NEXT_AGENT (writer)
- writer → NEXT_AGENT (critic)
- critic → OWNER_AGENT if response starts with FINAL:,
REVISE_AGENT otherwise
E2E VERIFICATION (real run, real Gemini, not a mock):
Topic: "how does SynapBus unify message delivery, reactive agent
triggers, and harness runs on a single SQLite database?"
Result (from data/synapbus.db after one successful run):
harness_runs:
#1 decomposer-pro subprocess success 106s
#2 writer-flash subprocess success 155s
#3 critic-lite subprocess success 10s
reactive_runs: 3 rows, all succeeded, trigger_from chain:
algis → decomposer-pro → writer-flash → critic-lite
messages:
#1 algis → decomposer-pro (topic)
#2 decomposer-pro → writer-flash (Q1/Q2/Q3 breakdown)
#3 writer-flash → critic-lite (3-paragraph draft)
#4 critic-lite → algis (FINAL: + polished 3-paragraph explainer)
Critic converged in one pass (all scores ≥ 8), so the writer↔critic
refinement loop didn't need to recurse — but the plumbing for it
(REVISE: branch in wrapper.sh, depth limit in reactor) is wired and
ready. Flipping the critic's acceptance bar exercises the recursion.
Full `go test ./...` remained green through all changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
b8a70bfe66 |
feat(harness): Option C — subprocess agent config (CLAUDE.md, MCP, skills)
Makes the subprocess backend fully self-contained: each agent carries
its instructions, MCP servers, skills, and subagents in its
harness_config_json column, viewable in the Web UI, editable via CLI.
internal/harness/subprocess/config.go (NEW):
AgentConfig struct with optional fields:
- claude_md → workdir/CLAUDE.md
- agents_md → workdir/AGENTS.md
- mcp_servers → workdir/.mcp.json (Claude Code format)
- skills → workdir/.claude/skills/<name>/SKILL.md
- subagents → workdir/.claude/agents/<name>.md
- env → layered into child env (after k8s_env_json,
before caller overrides)
ParseAgentConfig tolerates empty / returns error on invalid JSON.
MaterialiseAgentConfig writes all artifacts into the workdir with
path-traversal sanitisation on skill/subagent names.
subprocess.Harness.Execute now calls Parse + Materialise before exec,
so an agent's declarative config is on disk by the time the child
CLI's cwd lookup fires. buildEnv takes the parsed config and overlays
cfg.Env on top of k8s_env_json.
Tests:
config_test.go — 6 cases: empty, invalid JSON, full round-trip,
materialise writes all artefacts, empty is a no-op, skill names
are sanitised against "../escape" / "/etc/passwd", mcp entries
without a name are dropped.
subprocess_test.go — 2 new e2e cases: agent with CLAUDE.md + mcp
servers + skills + env sees all of them from inside the child via
cat/echo; invalid harness_config_json surfaces as Execute error.
internal/agents/store.go:
AgentStore gains UpdateHarnessConfig(ctx, name, harnessName,
localCommand, harnessConfigJSON). Empty strings leave a field
unchanged; literal "-" clears (sets to NULL). Returns sql.ErrNoRows
on missing agent. AgentService exposes Store() so admin handlers
can reach it without adding a full service method for a
config-set-style operation.
store_test.go: 6-subcase test covers set-all, partial update, clear,
unknown agent, and no-field no-op.
internal/admin/socket.go:
Two new admin commands:
harness.config_get {agent_name} → {harness_name, local_command,
harness_config_json, harness_config (parsed), parse_error?}
harness.config_set {agent_name, harness_name?, local_command?,
harness_config_json?} → updated fields
config_set validates JSON shape before calling the store; null /
"-" literals clear the column.
cmd/synapbus/admin.go:
New top-level `harness config` command group:
synapbus harness config get --agent <name> [--raw]
synapbus harness config set --agent <name>
[--harness-name subprocess]
[--local-command '["claude","--print"]']
[--file config.json] # or pipe from stdin
[--clear]
synapbus harness config edit --agent <name>
# fetches current config, opens $VISUAL/$EDITOR/vi,
# validates JSON on save, writes back via config_set
web/src/routes/agents/[name]/+page.svelte:
New read-only "Harness" panel on the agent detail page:
- Resolved backend badge (explicit or inferred from k8s_image /
local_command / harness_config_json.url)
- Grid summary: CLAUDE.md size, AGENTS.md size, MCP server count,
skills count
- Collapsible details for CLAUDE.md, AGENTS.md, each MCP server
(name / type / url|command / header count), skill filenames,
subagent filenames, env vars
- Footer hint showing the CLI edit command
No edit controls — editing is CLI-only by design (safer, fits an
ops-heavy workflow).
Verified: full project test suite (40+ packages including integration
tests) plus `vite build` of the Svelte app all green; `go vet ./...`
clean; the existing TestSubprocess_Execute_MaterialisesHarnessConfig
e2e test proves the round-trip from harness_config_json → workdir →
child process works end-to-end.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
e51adc376e |
feat: message reactions and workflow states (010-reactions-workflows)
Add typed reactions (approve/reject/in_progress/done/published) with
toggle semantics. Workflow state derived from highest-priority reaction.
New reactions package with model, SQLite store, and service layer.
REST API: POST/GET/DELETE /api/messages/{id}/reactions for toggle/query,
PUT /api/channels/{name}/settings for workflow config, GET by-state
endpoint for listing messages by workflow state.
MCP: react/unreact/get_reactions/list_by_state actions via bridge.
Web UI: WorkflowBadge (colored state pills) and ReactionPills (toggle
pills with agent names) components integrated into channel view.
Channel settings: auto_approve, stalemate_remind_after,
stalemate_escalate_after columns. CLI: channels update command.
Migration 013_reactions.sql adds message_reactions table and channel
workflow columns. 29+ new test cases across model and store.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
050b3cbde3 |
feat: A2A Agent Card discovery endpoint at /.well-known/agent-card.json
Add public A2A Agent Card endpoint that exposes registered agents as skills for cross-platform agent discovery. Includes admin CLI for updating agent capabilities, which populate skill tags and descriptions in the card. - internal/a2a: new package with AgentCard generator, HTTP handler, and tests - agents store/service: add ListAllActiveAgents (excludes human accounts) - admin socket: add agent.update_capabilities command - admin CLI: add `agent update-capabilities --name --capabilities` subcommand - main.go: register /.well-known/agent-card.json route (public, no auth) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
9dd27c4b9b |
feat: admin CLI & Docker fixes — alpine base, channels create/join, absolute socket path
Switch Docker runtime from scratch to alpine:3.19 so kubectl exec works for admin CLI operations. Add `synapbus channels create` and `synapbus channels join` CLI commands with corresponding admin socket handlers. Change default socket path to /data/synapbus.sock (absolute). Also add Helm envFrom support and NodePort configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
d5ca5b915f |
feat: add webhook, k8s, and attachments gc CLI subcommands
Add admin CLI subcommands for managing webhooks, K8s job handlers,
and running attachment garbage collection via the Unix admin socket.
Server-side:
- Add WebhookServiceProvider and K8sServiceProvider interfaces to admin pkg
- Add 7 new command handlers: webhook.{register,list,delete},
k8s.{register,list,delete}, attachments.gc
- Wire webhook and k8s services into admin.Services struct in main.go
CLI-side:
- Add `synapbus webhook {register,list,delete}` commands
- Add `synapbus k8s {register,list,delete}` commands
- Add `synapbus attachments gc` command
- All commands follow existing patterns (adminRequest, printTable, printJSON)
Tests:
- Add cmd/synapbus/admin_test.go with 9 tests covering command
registration, required flag validation, and existing command preservation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
67783566d7 |
feat: embeddings management, message retention & agent inbox improvements
Release / Build darwin/amd64 (push) Canceled after 0s
Release / Build linux/amd64 (push) Canceled after 0s
Release / Build darwin/arm64 (push) Canceled after 0s
Release / Build linux/arm64 (push) Canceled after 0s
Release / Generate Homebrew Formula (push) Canceled after 0s
Release / GitHub Release (push) Canceled after 0s
Release / Docker Image (push) Canceled after 0s
- Add `my_status` MCP tool: single call returns agent identity, pending DMs, channel mentions, system notifications, channel summaries, and stats with truncation for large inboxes - Add embeddings CLI: `synapbus embeddings status|reindex|clear` for managing vectors when switching embedding providers - Add automatic message retention worker with configurable period (--message-retention, default 12m), warning notifications 1 month before deletion, cascade cleanup, and incremental vacuum - Add manual purge: `synapbus messages purge --older-than --agent --channel` and `synapbus db vacuum` for on-demand cleanup - Add `synapbus retention status` CLI for admin visibility - Create system agent at startup for sending retention warnings - Filter system agent from discover_agents results Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
cad0138337 |
chore: migrate to github.com/synapbus org and add logo assets
Move module path from github.com/smart-mcp-proxy/synapbus to github.com/synapbus/synapbus across all Go imports (47 files). Add constellation logo options generated via FLUX 1.1 Pro. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
b88d52276a |
feat: add admin CLI, API keys, threads, and Slack-like UI redesign
Major feature additions across backend and frontend: Backend: - Unix domain socket admin server with JSON-RPC protocol - CLI subcommands: user/agent management, audit, backup, messages, channels - Managed API keys (sb_ prefix) with permissions, channel limits, expiry - Thread/reply support in messaging core (reply_to column) - Context-based trace owner_id propagation for proper audit filtering - Two-step auth middleware supporting both agent keys and managed API keys Frontend: - Complete Slack-like dark theme redesign with custom CSS properties - Sidebar with Channels, Direct Messages, and Admin sections - Thread panel (slide-in) for viewing message replies - API key management page with create form, key display, and ready-to-use MCP/Claude Code config snippets with copy-to-clipboard - All pages restyled: login, dashboard, agents, conversations, settings E2E Tests: - Fixed test runner binary path resolution - Added pyproject.toml for test dependencies Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |