main
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Algis DumbrisandClaude Opus 4.6
|
b5b5d6d26c |
fix(reactor): don't truncate body for subprocess + docker dispatch
dispatchHarness was capping event.Body at 4096 bytes before handing it to the subprocess/docker harness. Those backends write the body to a message.json file in the per-run workdir (bind-mounted into the container) and have no shell/env-var size limits, so silent truncation was hostile. The doc-gardener inspector routinely produces 10-20 KiB findings JSON (drift report with per-flag evidence). Truncation cut off the trailing artifact.findings entries + artifact.recommendation, making the report look incomplete to the critic — which then spuriously REVISE'd, blowing the 600s deadline. The K8s job path still truncates in createJob() because Kubernetes imposes a 1 MiB env-var cap and most shells misbehave past a few KiB. That's a separate code path, untouched. Also: critic prompt rewrite (examples/doc-gardener/configs/critic.json). The old critic spec told the critic to "spot-check evidence by re-running the inspector's commands". That's structurally wrong: the critic runs in a fresh container with no install state, so re-running mcpproxy --help always fails and produces a false REVISE. New prompt says: audit by structural consistency only, never run shell commands to re-verify, default to FINAL, never REVISE more than once, and FINAL the failure summary back to the owner when the inspector reports status: failed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
560d9d4125 |
feat(harness): docker isolation backend + canonical synapbus-agent image
New `internal/harness/docker` package: per-run ephemeral container
backend that runs each agent in `docker run --rm`, bind-mounts the
materialized workdir at /workspace, and captures stdout/stderr/exit
code/result.json the same way the subprocess backend does.
Inspired by scion's pkg/runtime/docker.go: shell out to the docker
CLI (zero new Go deps, zero CGO), per-task ephemeral containers with
no warm pool, host-side scratch dir bind-mounted in.
Default security posture (overridable per agent):
--rm
--cap-drop=ALL
--security-opt=no-new-privileges
--read-only with tmpfs /tmp
--pids-limit=512
--user=<host uid:gid>
--network=bridge (configurable; --network=none for air-gap)
--memory / --cpus from agent config
--add-host host.docker.internal:host-gateway on Linux
The backend reuses subprocess.AgentConfig for gemini_md/claude_md/
mcp_servers/skills materialization so existing example configs work
unchanged. Per-agent docker tunables go under a new `docker` block
in harness_config_json: image, memory, cpus, network, extra_mounts,
cap_add, read_only_root, user, entrypoint, command, extra_args.
MCP host rewrite: `.gemini/settings.json` URLs of the form
http://127.0.0.1:<port>/mcp are rewritten to
http://host.docker.internal:<port>/mcp at materialization time so the
in-container Gemini CLI can reach the SynapBus MCP server on the host
without code changes in the example wrappers.
Wired into the reactor and Registry resolver:
- Registry.Resolve picks "docker" when harness_config_json contains a
`"docker"` block, taking precedence over local_command so explicit
isolation never silently downgrades.
- reactor.agentBackendKind() returns backendDocker for the same case.
- evaluateTrigger's harness-backend gate accepts backendDocker
alongside subprocess + webhook.
- main.go registers docker.Harness with the harness registry, passing
the SynapBus listen port so the URL rewrite uses the correct host
port.
Smoke tests in docker_test.go (skipped when no docker daemon):
- TestExecute_Hello: env injection + bind-mount writeback + message.json
+ result.json + stdout capture using alpine:3.20
- TestExecute_NoImage: rejects agents missing docker.image
- TestExecute_TimeoutCancel: wall-clock budget kills the container
New canonical agent image at image-build/synapbus-agent/:
- Debian bookworm-slim base
- Node 22 + @google/gemini-cli + @anthropic-ai/claude-code
- jq, sqlite3, curl, git, python3, tini (PID 1 for signal forwarding)
- Non-root agent user uid/gid 1000
- ENTRYPOINT tini, CMD /workspace/wrapper.sh
No SynapBus binary inside the image — agents reach the host MCP server
over the network at host.docker.internal:<port>.
Pre-existing reactor test failures (TestReactorNoK8sImage,
TestReactorDepthExceeded, TestReactorBudgetExhausted,
TestReactorCooldownSkipped, TestReactorSequentialExecution) verified
to exist on
|
||
|
Algis DumbrisandClaude Opus 4.6
|
f319290ef9 |
feat(goal-coordinator): native MCP tool surface via Gemini session
The coordinator now reaches SynapBus's MCP endpoint directly from inside the Gemini session. wrapper.sh's coordinator branch is a pure pass-through — no more JSON-plan parsing. When the coordinator runs, Gemini connects to /mcp with the coordinator's own Bearer API key and calls `create_goal`, `propose_task_tree`, and `send_message` as native tools. Goal rows, task trees, and DMs all land in the DB in one in-session flow. - start.sh mints a fresh API key for goal-coordinator via `agent revoke-key` and substitutes it into configs/coordinator.json (plus the port) at apply_config time. - coordinator.json declares the synapbus MCP server in mcp_servers; the subprocess harness already writes .gemini/settings.json from that array, so gemini picks it up automatically. - GEMINI.md rewritten to instruct the model to call MCP tools instead of emitting a JSON action blob. Stdout is explicitly discarded; every reply goes through send_message. - wrapper.sh coordinator branch is ~15 lines: invoke gemini, log, exit. Inspector + critic keep the legacy JSON-plan pattern since they're workers with fixed contracts. - SYNAPBUS_KEEP_WORKDIR=1 preserves per-run workdirs for debugging MCP traces, gemini output, and materialized configs. - Reactor checkPendingWork now fires after subprocess run completion (previously only K8s poller hit this path). The synthetic coalesced trigger uses a `__coalesced__` sentinel instead of `system` so it bypasses the FromAgent=="system" dispatch guard. Verified e2e (with rate-limit-induced retries): - TRIVIAL: "what is 2+2?" → coordinator send_message(algis, "4") - INFEASIBLE: "Transfer \$50…" → coordinator send_message(algis, "CANNOT: …") - SINGLE-STEP: 3-node task tree materialized in goal_tasks, TASK JSON forwarded to generic-inspector → critic-auditor chain. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
f9d8f1a1c9 |
feat(018): /goals page, budget cascade, quarantine, secrets loop
- New /api/goals + /api/goals/{id} endpoints serving list + task tree
+ cost rollup + billing breakdown + spawned agents + timeline.
- New Svelte /goals and /goals/[id] pages with sidebar link.
- goals.Service.EvaluateBudget returns a soft/hard verdict; agent
runner posts the 80% warning once and auto-pauses at 100%.
- Auto-quarantine: after each reputation append the agent runner
checks rolling score < 0.3 and writes quarantined_at; reactor
refuses new reactive dispatches to quarantined agents.
- Reactor exposes SetSecretProvider; main.go wires secrets.Store
so reactive subprocess runs inherit user/agent-scoped env vars.
- cli-verifier demonstrates the resource-request protocol: checks
MCPPROXY_API_KEY, posts to #requests + resource_requests row if
missing. New `synapbus secrets set/list` CLI (direct-DB) closes
the loop.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
fee73e33a0 |
feat(ux): run detail page + reaction pills + captured prompt/response
Makes the Web UI reflect what agents are actually doing: reactions
on DMs that trigger a subprocess run, a per-run detail page that
shows the exact prompt the model received and the raw response, and
cross-linked reactive_runs ↔ harness_runs data for a single composite
API call.
Migration 020 (internal/storage/schema/020_harness_run_detail.sql):
ALTER TABLE harness_runs ADD COLUMN reactive_run_id INTEGER;
ALTER TABLE harness_runs ADD COLUMN prompt TEXT;
ALTER TABLE harness_runs ADD COLUMN response TEXT;
CREATE INDEX idx_harness_runs_reactive ON harness_runs(reactive_run_id);
internal/harness:
* ExecRequest.ReactiveRunID — reactor pins the reactive_runs row id
so the observer can JOIN the two tables.
* ExecResult.Prompt / Response — the subprocess harness reads
prompt.txt / response.txt that wrappers write into the workdir,
and runs.Store persists them (capped at 32 KiB each).
* runs.Run struct now has JSON tags — previously the API returned
PascalCase field names that didn't match the Web UI's snake_case
TypeScript types.
* New runs.Store.GetByReactiveRunID for the composite API endpoint.
* Test schema updated to include the new columns.
internal/reactor:
* New ReactionNotifier interface + SetReactionNotifier.
* dispatchHarness now reacts `in_progress` on the triggering DM
before spawning the goroutine.
* runHarness reacts `done` on success, `reject` on failure. The
existing reactionPriority ordering means the terminal reaction
wins for badge display — no need to remove in_progress first.
* dispatchHarness sets ExecRequest.ReactiveRunID.
cmd/synapbus/main.go:
* reactorReactionAdapter: adapts reactions.Service.Toggle to the
reactor's one-shot AddReaction signature.
* HarnessRunsStore wired into the API router config.
internal/api/runs_handler.go — GetRun composite endpoint:
The GET /api/runs/{id} response now returns everything the Web UI
needs to render the run detail page in one call:
{
"run": <reactive_runs row>,
"harness_run": <linked harness_runs row with prompt/response>,
"agent": <current agent snapshot with harness_config_json>,
"trigger_message": <DM that started the run>,
"outgoing_message": <first DM the agent produced after startedAt>
}
The outgoing-message lookup wraps both sides of the created_at
comparison in datetime() so SQLite parses the stored 'YYYY-MM-DD
HH:MM:SS' and the Go-emitted RFC3339 into the same canonical form
before comparing — a raw string compare was silently returning no
rows.
internal/api/router.go: HarnessRunsStore field in RouterConfig, wired
through to NewRunsHandler.
examples/cold-topic-explainer/wrapper.sh:
Writes prompt.txt and response.txt alongside gemini.stdout.raw so
the subprocess harness can capture "what the model saw" and "what
the model said" post-hoc.
web/src/lib/components/MessageList.svelte:
New ReactionPills render below each message body when the message
carries a `reactions` array (already populated by
EnrichMessages/ReactionEnricher on the server side). Makes the
👀 in_progress / ✔ done / ❌ reject lifecycle visible in every DM
view and conversation.
web/src/routes/runs/[id]/+page.svelte (NEW):
New run detail page at /runs/:id with sections:
1. Header strip — agent, status pill, backend badge, trigger
info, duration, tokens in/out, cost, exit code, trace id.
2. Triggering message — body + sender.
3. What the model saw — GEMINI.md / CLAUDE.md from agent snapshot
+ the captured rendered prompt (byte count on each summary
bar, collapsible details).
4. What the model said — captured response, falling back to
logs_excerpt or error_log when unavailable.
5. Outgoing message — body + recipient + status.
6. Metadata — reactive_run.id, harness_run.run_id, backend,
session_id, tokens_cached, k8s_job, agent trigger config.
Styled against the existing dark tailwind system — no design
overhaul, fits the current aesthetic (editorial sectioning,
monospace for code-like content, accent-blue for links,
accent-purple for system-instructions, accent-green for model
output, accent-red for errors).
web/src/routes/runs/+page.svelte: the inline expand panel now has
a "View full details →" link next to the Retry button.
E2E VERIFIED on a live subprocess run:
* Topic: "why does the subprocess harness materialise GEMINI.md
alongside .gemini/settings.json in the per-run workdir?"
* 3 subprocess runs + 3 reactive_runs + 3 harness_runs, all linked.
* message_reactions: 6 rows — in_progress + done for each hop.
* GET /api/runs/1 returns a composite with
harness_run.prompt=883 bytes, harness_run.response=550 bytes,
reactive_run_id=1, trigger_message populated, outgoing_message
populated (decomposer-pro → writer-flash), agent.gemini_md=747
bytes. All keys are snake_case as the Svelte types expect.
Full go test ./... green. `vite build` green. Demo instance still
running on port 18088 for browser verification.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
8ddccd2452 |
feat(reactor): route non-K8s reactive runs through harness registry
Phase 7: the reactor now branches on agent backend kind.
Reactor changes (internal/reactor/reactor.go):
* Adds `registry *harness.Registry` field + `SetHarnessRegistry`.
* `agentBackendKind()` picks k8s | subprocess | webhook | none from
the agent's `HarnessName`, `K8sImage`, `LocalCommand`, and
`HarnessConfigJSON` fields. Explicit `HarnessName` wins.
* `evaluateTrigger` applies the same preconditions (depth, daily
budget, cooldown, already-running, pending_work coalescing) to
every backend — a subprocess agent mentioned in a channel now
goes through the exact same rate limits a K8s agent does.
* K8s agents keep the existing `createJob` fast-return path with
the async poller for restart safety. Non-K8s agents use a new
`dispatchHarness` that inserts the reactive_runs row, spawns a
detached goroutine, blocks on `Registry.Execute`, and writes the
terminal status / error_log / metrics / failure DM on return.
* Import `harness`, `messaging`, `google/uuid` for building the
ExecRequest.
main.go wiring:
* Build one `harness.Registry` with all three real backends:
`k8sjob.New(k8sRunner, …)`, `subprocess.New(Config{BaseDir:
dataDir/harness/subprocess}, …)`, `webhook.New(Config{}, …)`.
* Attach a `runs.Store` as the registry Observer so every dispatch
writes a harness_runs row — no per-caller code required.
* Hand the registry to the reactor via `SetHarnessRegistry`.
* Log the registered backend names at startup.
Tests (internal/reactor/reactor_test.go):
* New `insertSubprocessAgent`, `newHarnessReactor`, `waitForRun`,
and `fakeNotifier` helpers.
* Seven new tests that register a stub harness under "subprocess"
and verify: success from @mention, failure recorded + DM sent,
depth-exceeded skipped, budget-exhausted skipped, cooldown
skipped, already-running queued, no-backend fails cleanly. Each
checks the harness stub is NOT called when a precondition skips.
* Existing `TestReactorNoK8sImage` keeps working — the old
k8s-specific error message is replaced with the backend-agnostic
"no backend configured" phrasing.
* `setupTestDB` now pins `SetMaxOpenConns(1)`: modernc.org/sqlite
in-memory DBs give each pool connection a fresh empty database,
which races the new dispatchHarness goroutine and main-test
goroutine. Pinning is the standard workaround.
The overall behaviour: `@local-agent` in a channel message now starts
the configured subprocess/webhook under the same depth/budget/cooldown
rate limits as a K8s agent, tracked in reactive_runs and harness_runs,
instrumented with an OTel span, with trace context propagated into the
child via env vars. Failure DMs go to the human owner, as with K8s.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
574897df4d |
feat: harness-agnostic wrappers + OpenTelemetry integration
Introduces internal/harness — a minimal Harness interface inspired by
GoogleCloudPlatform/scion — plus four backends (k8sjob, subprocess,
webhook, stub) and an OTel-traced Registry that spans every dispatch
and injects W3C trace context into child processes via env vars.
Phases landed together on this branch:
1. internal/harness scaffold: Harness/Capabilities/ExecRequest/
ExecResult/Budget/Usage types, Registry with Resolve/Execute,
in-memory stub backend.
2. internal/harness/k8sjob: wraps existing k8s.JobRunner behind the
Harness interface with a Waiter abstraction (real clientset +
test fake). BuildHandler exports the per-agent config logic.
3. internal/harness/subprocess: os/exec-based backend (Mac+Linux),
per-run workdir, result.json handoff, bounded log capture,
Budget-driven wall-clock timeout.
4. internal/harness/webhook: synchronous HTTP POST with HMAC
signing via internal/webhooks.ComputeHMACSignature, per-agent
URL/secret/timeout read from harness_config_json.
5. internal/observability: OTel tracer init via OTLP HTTP (opt-in
via SYNAPBUS_OTEL_ENABLED), W3C propagator always installed;
Registry.Execute starts a harness.execute span per dispatch and
calls InjectTraceContext into req.Env so children inherit it.
6. internal/harness/runs: SQLite-backed Observer that persists a
harness_runs row per dispatch with status, usage, cost, duration,
trace_id, session_id, and a bounded logs excerpt.
Schema: new migration 019_harness.sql adds agents.harness_name /
local_command / harness_config_json columns and the backend-agnostic
harness_runs table with indices on (agent, created_at), (status),
(trace_id), (run_id). internal/reactor/reactor_test.go inline schema
updated to match.
Deployment: deploy/kubic/otel-collector.yaml stands up an otel-collector
Deployment + ConfigMap + ClusterIP Service in the synapbus namespace on
kubic, receiving OTLP gRPC (4317) and HTTP (4318) and exporting debug
output until a Tempo/Jaeger backend lands.
Docs: docs/harness-otel-research.html compares scion and paperclip
side-by-side and maps the current synapbus executor surface; its
companion docs/harness-otel-design.md carries the phase plan, span
taxonomy, and migration schema verbatim.
The reactor currently still calls k8s.JobRunner directly — rewiring it
through the Registry is a follow-up, intentionally out of scope for
this branch to keep the refactor reversible. The new packages are
independently tested (~78 new tests across 7 packages) and the full
project test suite passes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Algis DumbrisandClaude Opus 4.6
|
7848911a5f |
fix: ignore system DMs in reactor — prevent stalemate notification cascade
Release / Build darwin/amd64 (push) Canceled after 0s
Release / Build linux/amd64 (push) Canceled after 0s
Release / Build darwin/arm64 (push) Canceled after 0s
Release / Build linux/arm64 (push) Canceled after 0s
Release / Generate Homebrew Formula (push) Canceled after 0s
Release / GitHub Release (push) Canceled after 0s
Release / Docker Image (push) Canceled after 0s
Release / Publish to MCP Registry (push) Canceled after 0s
The StaleWorker sends DMs from 'system' to all channel members when workflow messages are stuck in 'proposed' state. These DMs were triggering reactive agent runs, which couldn't action the stale messages, burning daily budget on wasted K8s Jobs. Now: reactor silently ignores all messages from 'system' sender. System notifications are for human review, not agent action. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
64c68c22be |
fix(014): prevent stuck runs by creating K8s Job before DB insert
The reactor was inserting the run record first, then creating the K8s Job, then updating the record with the job name. If the update failed (SQLITE_BUSY), the run would be stuck in 'running' with no job name, making it invisible to the poller. Now: create K8s Job first, then insert the run record with job name already set in a single atomic write. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
cf6066229f |
feat(014): add Prometheus metrics, Grafana dashboard, volume mounts, resource tuning
- Reactor Prometheus metrics: triggers_total, run_duration_seconds, agent_running, budget_used_today - Integrated promauto metrics into hand-rolled WritePrometheus endpoint - K8s runner: ImagePullPolicy=IfNotPresent, volume mounts, CLI args support - Reactor: 2Gi/500m default resources (agent SDK needs it), 1h timeout - Grafana dashboard "SynapBus Reactive Agents" with 8 panels: triggers by status, agent state, budget gauge, run duration, agent turns from Loki, reactor events log, agent container logs - SQLite: busy_timeout=15s, synchronous=NORMAL, MaxOpenConns=4 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Algis DumbrisandClaude Opus 4.6
|
6afe1853ad |
feat(014): implement reactive agent triggering engine
- Migration 015: extends agents with trigger config, adds reactive_runs table - Reactor engine: decision chain (mode, depth, budget, cooldown, sequential) - Reactor store: SQLite persistence for runs with RFC3339 timestamps - Reactor poller: K8s Job status polling (15s interval) - Failure notifier: system DM to owner on job failure - REST API: /api/runs, /api/runs/:id, /api/runs/:id/retry, /api/agents/reactive - Agent model: trigger_mode, cooldown, budget, depth, k8s_image, pending_work - K8s runner: GetClientset() for poller - All 28 test packages pass (8 new reactor tests) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |