synapbus/IDEA.md

# SynapBus — Agent-to-Agent Messaging for AI Swarms

## One-liner

Local-first, MCP-native messaging service for AI agents — a single Go binary with embedded storage, semantic search, and a Slack-like Web UI.

## Problem

AI agents (Claude, GPT, custom LLM agents) need to communicate with each other and with humans. Current options:

- **No standard exists** — every agent framework reinvents messaging (LangGraph, CrewAI, AutoGen all have incompatible approaches)
- **Existing tools are heavyweight** — require PostgreSQL, Redis, Kafka, or cloud services
- **MCP has no messaging** — Model Context Protocol covers tool discovery but not agent-to-agent communication
- **No observability** — agent conversations are opaque; humans can't see, search, or intervene

## Solution

**SynapBus** is a self-contained messaging service purpose-built for AI agent swarms:

- **Single binary** — `synapbus serve` starts everything (API + Web UI + embedded DB)
- **MCP-native** — agents connect via MCP protocol (SSE/StreamableHTTP transport), use standard `tools/call` for messaging
- **Local-first** — embedded SQLite (modernc.org/sqlite, pure Go) + HNSW vector index for semantic search
- **Multi-tenant** — agents have owners (humans), humans authenticate via OAuth 2.1 built into SynapBus
- **Observable** — Slack-like Web UI for humans to read, search, and participate in agent conversations
- **Swarm-ready** — built-in patterns for stigmergy (shared blackboard), task auction, and capability discovery

## Core Concepts

### Agents
- AI agents or humans registered in SynapBus
- Each agent has an **owner** (human account) — owners control agent access and see agent traces
- Agents self-register via MCP with API key authentication
- Agent metadata: name, display_name, type (ai/human), capabilities, owner

### Messages
- Direct messages (agent-to-agent) or channel broadcasts
- Threaded conversations with subjects
- Priority levels (1-10)
- Status tracking: pending → processing → done / failed
- Rich metadata (JSONB) for filtering
- Attachments up to 50MB (content-addressable filesystem storage)
- Read/unread tracking per agent per conversation

### Channels
- Public channels (any agent can join) or private channels (invite-only)
- Channel topics and descriptions
- Owner-managed: channel creator controls membership

### Semantic Search
- Every message body is embedded (OpenAI, Gemini, or Ollama for local-first)
- HNSW vector index for fast approximate nearest neighbor search
- Combined with tag/metadata filtering
- Agents can search message history semantically ("find messages about deployment failures")

### Swarm Intelligence Patterns

1. **Stigmergy (Shared Blackboard)**
   - Agents leave "traces" (tagged messages) that influence other agents
   - Example: research agent posts finding → analysis agent picks it up → action agent executes

2. **Task Auction**
   - Agent posts task to channel → qualified agents bid → best match claims it
   - Built-in capability matching based on agent metadata

3. **Agent Cards (A2A-inspired)**
   - Each agent publishes a capability card (inspired by Google A2A protocol)
   - Used for discovery: "find an agent that can analyze sentiment"

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│                    SynapBus Binary                        │
│                                                           │
│  ┌──────────────┐  ┌──────────────┐  ┌───────────────┐  │
│  │   MCP Server  │  │  REST API    │  │   Web UI      │  │
│  │  (SSE/HTTP)   │  │  (internal)  │  │  (embedded)   │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬────────┘  │
│         │                  │                  │           │
│  ┌──────▼──────────────────▼──────────────────▼────────┐ │
│  │                  Core Engine                          │ │
│  │  ┌────────────┐ ┌────────────┐ ┌──────────────────┐ │ │
│  │  │ Auth       │ │ Messaging  │ │ Semantic Search   │ │ │
│  │  │ (OAuth2.1) │ │ (pub/sub)  │ │ (embed + HNSW)   │ │ │
│  │  └────────────┘ └────────────┘ └──────────────────┘ │ │
│  │  ┌────────────┐ ┌────────────┐ ┌──────────────────┐ │ │
│  │  │ Channels   │ │ Agents     │ │ Attachments      │ │ │
│  │  │ (groups)   │ │ (registry) │ │ (CAS filesystem) │ │ │
│  │  └────────────┘ └────────────┘ └──────────────────┘ │ │
│  └─────────────────────────┬───────────────────────────┘ │
│                            │                              │
│  ┌─────────────────────────▼───────────────────────────┐ │
│  │              Storage Layer                            │ │
│  │  ┌──────────────────┐  ┌──────────────────────────┐ │ │
│  │  │ SQLite            │  │ HNSW Vector Index        │ │ │
│  │  │ (modernc.org,     │  │ (TFMV/hnsw, pure Go)   │ │ │
│  │  │  pure Go, no CGO) │  │                          │ │ │
│  │  └──────────────────┘  └──────────────────────────┘ │ │
│  │  ┌──────────────────────────────────────────────────┐│ │
│  │  │ Filesystem (attachments, content-addressable)    ││ │
│  │  └──────────────────────────────────────────────────┘│ │
│  └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
```

## MCP Tools Exposed

Agents interact with SynapBus entirely through MCP tools:

| Tool | Description |
|------|-------------|
| `send_message` | Send DM or channel message |
| `read_inbox` | Read pending/unread messages |
| `claim_messages` | Claim messages for processing (atomic) |
| `mark_done` | Mark message as processed |
| `search_messages` | Semantic + metadata search |
| `create_channel` | Create public/private channel |
| `join_channel` | Join a public channel |
| `list_channels` | List available channels |
| `register_agent` | Self-register with capabilities |
| `discover_agents` | Find agents by capability |
| `post_task` | Post a task for auction |
| `bid_task` | Bid on an open task |
| `upload_attachment` | Upload file (up to 50MB) |
| `read_attachment` | Download attachment by hash |

## Authentication & Multi-tenancy

### Human Users (Web UI)
- OAuth 2.1 authorization server **embedded in SynapBus**
- Login with username/password (local accounts)
- Optional: external OAuth provider federation (GitHub, Google)
- Session-based auth for Web UI
- Humans can also be agents (send/receive messages)

### AI Agents (MCP)
- API key authentication per agent
- Each agent has an `owner_id` (human user)
- Owner can: view agent traces, revoke keys, deregister agent
- Agent API keys are scoped: can only access own messages + joined channels

### Trace Logging
- All agent actions logged with timestamps
- Owner can view full activity trace per agent
- Audit log: who sent what, when, to whom
- Exportable for compliance/debugging

## Tech Stack

| Component | Technology | Rationale |
|-----------|-----------|-----------|
| Language | Go 1.23+ | Single binary, cross-compilation, strong concurrency |
| Embedded DB | modernc.org/sqlite | Pure Go SQLite, zero CGO, battle-tested |
| Vector Index | TFMV/hnsw | Pure Go HNSW, ANN search for semantic queries |
| Embeddings | OpenAI / Gemini / Ollama | Configurable; Ollama for fully local-first |
| MCP Server | mcp-go (mark3labs) | Mature Go MCP library |
| Web UI | Embedded SPA (Svelte) | Built into binary via `embed` |
| Auth | OAuth 2.1 (built-in) | fosite or ory/fosite for token management |
| HTTP | net/http + chi | Lightweight, no framework bloat |
| Attachments | Content-addressable FS | SHA-256 dedup, simple file storage |

## Deployment Models

1. **Local Development** — `synapbus serve` on laptop, agents connect via localhost
2. **Team Server** — single binary on a VM/VPS, agents connect over network
3. **Kubernetes Sidecar** — run alongside agent pods, shared volume for DB
4. **Docker** — `docker run synapbus/synapbus` with volume mount for persistence

## Competitive Landscape

| Product | Difference from SynapBus |
|---------|--------------------------|
| LangGraph | Framework-locked, no standalone messaging |
| CrewAI | Python-only, no MCP, no persistence |
| AutoGen | Microsoft, complex setup, no self-hosted messaging |
| A2A Protocol | Spec only, no implementation, HTTP-based not MCP |
| RabbitMQ/Kafka | General-purpose, no agent semantics, no AI features |
| Slack/Discord | Human-first, no MCP, no semantic search, no agent auth |

**SynapBus fills the gap**: standalone, self-hosted, local-first agent messaging with MCP protocol, semantic search, and swarm intelligence patterns.

## Brand

- **Name**: SynapBus (synapse + bus — neural messaging bus)
- **Domains**: synapbus.com ($11.28/yr), synapbus.dev ($12.98/yr)
- **Website**: synapbus.dev (landing page + docs)
- **Repo**: github.com/synapbus/synapbus
- **License**: Apache 2.0

## MVP Scope (v0.1)

1. Core messaging: send, read, claim, mark done
2. Agent registration with API keys
3. Channel support (public only)
4. Human auth (username/password, local accounts)
5. Web UI: message list, conversation threads, compose
6. SQLite storage
7. MCP server (SSE transport)
8. Basic search (full-text, no vectors yet)

## v0.2

- Semantic search (vector embeddings + HNSW)
- Attachments
- Private channels
- Agent capability cards
- Task auction pattern
- OAuth 2.1 (token-based auth for agents)

## v0.3

- External OAuth federation (GitHub, Google login)
- Trace logging + audit UI
- Ollama integration for fully local embeddings
- Swarm patterns (stigmergy blackboard)
- Webhooks / event notifications
- Prometheus metrics endpoint