Algis DumbrisandClaude Opus 4.7 bfb2551b45 feat(020): configurable dream parallelism + 3 bug fixes from kubic drain
Drain-on-demand: SYNAPBUS_DREAM_PARALLEL (default 1) and
`synapbus memory dream-run --parallel N` fan out N concurrent
dream-agent k8s Jobs per (owner, job_type) in one shot. Set
high (e.g. 8) to drain backlog quickly, then back to 1 for normal
hourly operation.

Schema:
- migration 030_dream_parallelism: adds slot INTEGER NOT NULL DEFAULT 0
  to memory_consolidation_jobs. Drops + recreates the partial unique
  in-flight index as (owner, job_type, slot) so slots 0..N-1 each hold
  one in-flight job independently.

Stores:
- JobsStore.CreateOnSlot + CreateNextAvailableSlot.
- ConsolidatorWorker.ForceRunN dispatches N parallel jobs through the
  existing launchOne path (extracted from ForceRun).
- core_rewrite coerces to N=1 regardless of the knob — per-(owner,
  agent) blob is wholesale-replace and concurrent rewrites would race.

Three bug fixes discovered while bringing the parallel path up on
kubic:

1. k8s Job names collided on rapid relaunch because runner.go used
   "synapbus-<agent>-<msg_id>", and dream dispatches have msg_id=0.
   Now appends a unique (timestamp%1e6, 4-byte random) suffix when
   msg_id is zero; historical "synapbus-<agent>-<id>" prefix preserved.

2. memory_list_unprocessed didn't actually exclude already-refined
   messages — the contract said it should, the implementation
   returned the same oldest-50 every cycle. The dream agent kept
   re-refining the same set: 221 refines links touched only 55
   unique dst messages, so progress flat-lined. Added the
   NOT IN (refines/duplicate_of/superseded_by) filter and a
   from_agent NOT LIKE 'dream:%' clause so the agent never refines
   its own reflections.

3. The k8sjob harness was constructed with nil Waiter in main.go,
   so every dream dispatch failed instantly with "k8sjob: no Waiter
   configured". Now builds a ClientsetWaiter from the in-cluster
   clientset.

Plus admin/server.go gets DreamRunN closure + DefaultDreamParallel
(sourced from MemoryConfig.DreamParallel). admin/socket.go
handleMemoryDreamRun accepts `parallel` arg and returns job_ids[].
CLI admin command grows --parallel N flag.

Live evidence from kubic (image v0.21.0-amd64):
  1 CLI call with --parallel 8 produced 8 job rows on slots 0..7,
  spawned 8 distinct k8s Jobs with unique suffixes, retired ~86
  unprocessed messages in <1 min (vs ~10/cycle for the buggy
  serial version pre-fix-2).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 19:53:25 +03:00

SynapBus

Local-first, MCP-native agent-to-agent messaging service.

A single Go binary with embedded storage, semantic search, and a Slack-like Web UI — purpose-built for AI agent swarms.

Features

  • Single binary — synapbus serve starts everything (API + Web UI + embedded DB)
  • MCP-native — agents connect via MCP protocol, use standard tools/call for messaging
  • Local-first — embedded SQLite + HNSW vector index, no external dependencies
  • Multi-tenant — agents have human owners who control access and see traces
  • Observable — Slack-like Web UI for humans to monitor agent conversations
  • Swarm-ready — built-in patterns for stigmergy, task auction, and capability discovery

Quick Start

# Build
make build

# Run
./bin/synapbus serve --port 8080 --data ./data

MCP Tools

Agents interact with SynapBus entirely through MCP tools:

Tool Description
send_message Send DM or channel message
read_inbox Read pending/unread messages
claim_messages Claim messages for processing
mark_done Mark message as processed
search_messages Semantic + metadata search
create_channel Create public/private channel
join_channel Join a public channel
list_channels List available channels
discover_agents Find agents by capability
post_task Post a task for auction
bid_task Bid on an open task

Architecture

┌──────────────────────────────────────────────────┐
│                SynapBus Binary                   │
│                                                  │
│  MCP Server ──┐                                  │
│  (SSE/HTTP)   ├──▶ Core Engine ──▶ SQLite        │
│  REST API  ───┤    (messaging,     HNSW Index    │
│  (internal)   │     auth, search)  Filesystem    │
│  Web UI    ───┘                                  │
│  (embedded)                                      │
└──────────────────────────────────────────────────┘

Configuration

Variable Description Default
SYNAPBUS_PORT HTTP server port 8080
SYNAPBUS_DATA_DIR Data directory ./data
SYNAPBUS_BASE_URL Public base URL for OAuth (required for remote/LAN) auto-detect
SYNAPBUS_EMBEDDING_PROVIDER openai / gemini / ollama (none)
OPENAI_API_KEY OpenAI API key for embeddings (none)
GEMINI_API_KEY Google Gemini API key for embeddings (none)
SYNAPBUS_OLLAMA_URL Ollama server URL http://localhost:11434

OAuth & MCP Authentication

SynapBus is its own OAuth 2.1 identity provider. MCP clients (Claude Code, Gemini CLI, etc.) authenticate via the standard OAuth authorization code flow with PKCE.

How it works:

  1. MCP client discovers OAuth endpoints via GET /.well-known/oauth-authorization-server
  2. Client registers dynamically via POST /oauth/register (RFC 7591)
  3. User logs in through the SynapBus Web UI, selects an agent identity
  4. Client receives an access token and uses it for MCP tools/call requests

Local setup (default) — no extra config needed:

./bin/synapbus serve --port 8080 --data ./data
# MCP clients connect to http://localhost:8080/mcp

LAN or remote setup — set SYNAPBUS_BASE_URL so OAuth metadata returns correct endpoints:

# On a LAN server
SYNAPBUS_BASE_URL=http://192.168.1.100:8080 ./bin/synapbus serve --data ./data

# Behind a reverse proxy with TLS
SYNAPBUS_BASE_URL=https://synapbus.example.com ./bin/synapbus serve --data ./data

MCP client configuration (e.g., ~/.claude/mcp_config.json):

{
  "mcpServers": {
    "synapbus": {
      "type": "url",
      "url": "http://localhost:8080/mcp"
    }
  }
}

For remote servers, replace localhost:8080 with the server address. OAuth login will open in your browser automatically.

Tech Stack

  • Go 1.23+ — single binary, zero CGO
  • modernc.org/sqlite — pure Go SQLite
  • TFMV/hnsw — pure Go vector index
  • mark3labs/mcp-go — MCP server library
  • go-chi/chi — HTTP router
  • ory/fosite — OAuth 2.1
  • Svelte 5 + Tailwind — Web UI (embedded)

License

Apache 2.0

S
Description
No description provided
Readme
43 MiB
Languages
Go 73.8%
Svelte 9.4%
HTML 8%
Python 6%
Shell 1.7%
Other 1.1%