plan(019): plugin system plan + research + data-model + contracts + quickstart

Phase 0 research resolves all 12 open decisions (interface shape, registration,
host API, dynamic toggle, migrations, UI panel integration, config format,
testing, boundary enforcement, squash, failure notification, integration test).

Phase 1 artifacts: data-model.md (Plugin, Registry, Migration, Host, Status,
Backup), contracts/plugin.md (Plugin + HasX interfaces), contracts/host.md
(Host struct + plugintest constructor), contracts/rest.md (/api/plugins/*),
quickstart.md (end-to-end "hello" plugin in 8 steps).

All ten constitution gates pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Algis Dumbris
2026-04-19 06:56:36 +03:00
co-authored by Claude Opus 4.7
parent b021768a9a
commit 69cd13dce1
8 changed files with 948 additions and 0 deletions
+2
View File
@@ -112,6 +112,8 @@ make lint # Run linters
- SQLite via modernc.org/sqlite — new migration 015_reactive_triggers.sql (014-reactive-agent-triggers)
- Go 1.25+ (per `go.mod`), no CGO, cross-compiled for `linux/amd64` + `darwin/arm64` + `mark3labs/mcp-go` (MCP tools), `go-chi/chi` (HTTP), `spf13/cobra` (CLI), `modernc.org/sqlite` (storage), `golang.org/x/crypto/nacl/secretbox` (secret encryption — pure Go, already in ecosystem), existing `SherClockHolmes/webpush-go`, `TFMV/hnsw`, `ory/fosite` (018-dynamic-agent-spawning)
- SQLite via `modernc.org/sqlite` — five new migrations (`021_goals_tasks.sql`, `022_agent_proposals.sql`, `023_agent_trust_model.sql`, `024_secrets.sql`, `025_harness_runs_task_id.sql`); existing content-addressable attachment store reused for encrypted secret blobs (018-dynamic-agent-spawning)
- Go 1.25+ (per go.mod) + `mark3labs/mcp-go` (MCP), `go-chi/chi` (HTTP), `spf13/cobra` (CLI), `modernc.org/sqlite` (storage), `jmoiron/sqlx` (query helpers), `cloudflare/tableflip` (graceful restart — NEW), `gopkg.in/yaml.v3` (config), `xeipuuv/gojsonschema` (config-schema validation) (019-plugin-system)
- SQLite via `modernc.org/sqlite` (pure Go, zero CGO). New core table `plugin_migrations`. Plugin tables namespaced `plugin_<name>_*`. (019-plugin-system)
## Recent Changes
- 002-mcp-auth-ux-polish: Added Go 1.23+ + ory/fosite (OAuth 2.1), mark3labs/mcp-go (MCP server), go-chi/chi (HTTP), Svelte 5 + Tailwind (Web UI)
+78
View File
@@ -0,0 +1,78 @@
# Contract — `Host` struct
```go
package plugin
import (
"context"
"database/sql"
"encoding/json"
"log/slog"
"github.com/jmoiron/sqlx"
"github.com/prometheus/client_golang/prometheus"
"go.opentelemetry.io/otel/trace"
"github.com/synapbus/synapbus/internal/attachments"
"github.com/synapbus/synapbus/internal/channels"
"github.com/synapbus/synapbus/internal/eventbus"
"github.com/synapbus/synapbus/internal/messaging"
"github.com/synapbus/synapbus/internal/search"
"github.com/synapbus/synapbus/internal/secrets"
"github.com/synapbus/synapbus/internal/users"
)
// Host is the bundle of core services handed to a plugin at Init.
// Plugins MUST treat it as read-only and MUST NOT cache sub-handles
// across process lifetimes.
type Host struct {
Logger *slog.Logger // pre-tagged plugin=<name>
DB *sqlx.DB // shared core handle
Messaging messaging.API
Channels channels.API
Attachments attachments.Store
Search search.Index
Secrets secrets.Scoped // scoped to calling plugin
Events eventbus.Bus
Config json.RawMessage // plugins[<name>] YAML sub-tree
DataDir string // <--data>/plugins/<name>/
Tracer trace.Tracer
Metrics prometheus.Registerer
DefaultOwner *users.User // for failure notifications
baseURL string // unexported; plugins read via BaseURL()
}
// BaseURL returns the public base URL of this instance, for building
// absolute links in panel HTML.
func (h Host) BaseURL() string { return h.baseURL }
// ExecTx runs a function inside a database transaction against the shared
// handle. Plugins should use this for multi-statement writes.
func (h Host) ExecTx(ctx context.Context, fn func(*sqlx.Tx) error) error { … }
```
## Security / scope invariants
- `Host.Secrets` returns only secrets that were written with the caller's plugin name. Calling `.Get("X")` for a secret owned by another plugin returns `secrets.ErrNotFound`, same as missing.
- `Host.DB` gives full SQL access — but tables read/written are expected to be `plugin_<name>_*`. A static linter (part of this feature) flags unqualified reads from core tables.
- `Host.DataDir` is guaranteed to exist, to be a directory, and to be writable only by this plugin (permissions 0700).
- `Host.Metrics` is a sub-registerer; labels automatically carry `plugin="<name>"`.
## Test constructor (from plugintest)
```go
package plugintest
// NopHost returns an in-memory Host backed by an in-memory SQLite database
// and a temporary directory. Suitable for unit tests.
func NopHost(t *testing.T) *plugin.Host { … }
// Run performs a full lifecycle dry-run of a plugin against a NopHost:
// 1. Migrations are applied.
// 2. Init is called.
// 3. If HasLifecycle, Start is called.
// 4. All declared capabilities are asserted to be registered.
// 5. Shutdown + close.
// t.Fatal on any error.
func Run(t *testing.T, p plugin.Plugin) { … }
```
+150
View File
@@ -0,0 +1,150 @@
# Contract — `internal/plugin` interfaces
## Base interface
```go
package plugin
import (
"context"
)
// Plugin is the minimum contract. Every compiled-in plugin implements this.
type Plugin interface {
// Name returns the plugin's globally unique identifier.
// Must match ^[a-z][a-z0-9_]{1,31}$.
Name() string
// Version returns the plugin's semver version.
Version() string
// Init is called once during startup after migrations have applied.
// The plugin registers its capabilities here. Must be fast (<100 ms)
// and idempotent. Returning an error marks the plugin failed; the
// core continues to start.
Init(ctx context.Context, host Host) error
}
```
## Optional capability interfaces
Plugins implement any subset. Host uses type assertion at Init time.
```go
// HasMigrations: plugin owns a numbered chain of SQL migrations.
type HasMigrations interface {
Migrations() []Migration
}
type Migration struct {
Version int // 1..N, monotonic
Name string // "001_initial"
SQL string // single file, runs in one transaction
}
// HasMCPTools: plugin adds first-class MCP tools (rare; most plugins use HasActions).
type HasMCPTools interface {
MCPTools() []MCPTool
}
type MCPTool struct {
Name string // "linkedin_post"
Description string
InputSchema json.RawMessage
Handler func(ctx context.Context, args map[string]any) (any, error)
}
// HasActions: plugin adds bridged actions callable via the core execute() tool.
type HasActions interface {
Actions() []ActionRegistration
}
type ActionRegistration struct {
Name string // "create_article"
Description string
InputSchema json.RawMessage
RequiredScope Scope // read | write | admin
Handler func(ctx context.Context, args map[string]any) (any, error)
}
type Scope string
const (
ScopeRead Scope = "read"
ScopeWrite Scope = "write"
ScopeAdmin Scope = "admin"
)
// HasHTTPRoutes: plugin mounts REST routes under /api/plugins/<name>/*.
type HasHTTPRoutes interface {
RegisterRoutes(r chi.Router)
}
// HasWebPanels: plugin contributes one or more UI panels, served under /ui/plugins/<name>/*.
type HasWebPanels interface {
WebPanels() []PanelManifest
PanelHandler() http.Handler // serves /ui/plugins/<name>/*
}
type PanelManifest struct {
ID string // "wiki"
Title string // "Wiki"
Icon string // lucide-icon name
Route string // "/ui/plugins/wiki"
Scope string // "owner" | "member"
}
// HasCLICommands: plugin adds subcommands under `synapbus plugin <name>`.
type HasCLICommands interface {
CLICommands() []*cobra.Command
}
// HasChannelType: plugin defines a channel behavior (blackboard/auction-like).
type HasChannelType interface {
ChannelTypes() []ChannelTypeDef
}
type ChannelTypeDef struct {
Name string
OnMessage func(ctx context.Context, channelID int64, msgID int64) error
OnReaction func(ctx context.Context, channelID int64, msgID int64, reaction string) error
}
// HasEventHook: plugin subscribes to internal events.
type HasEventHook interface {
OnEvent(ctx context.Context, e Event) error
}
type Event struct {
Topic string // "message.created", "reaction.added", "plugin.status.changed"
Payload any // typed by topic
Meta map[string]string
}
// HasLifecycle: plugin runs background work and needs explicit start/shutdown.
type HasLifecycle interface {
Start(ctx context.Context) error
Shutdown(ctx context.Context) error
}
// HasConfigSchema: plugin publishes a JSON Schema for its config so the Web UI can generate a form.
type HasConfigSchema interface {
ConfigSchema() json.RawMessage
}
```
## Stability hint (optional)
```go
// HasStability: plugin declares stability level. Default: "stable".
type HasStability interface {
Stability() string // "stable" | "beta" | "experimental"
}
```
## Invariants
- `Plugin.Name()` MUST be constant over the plugin's lifetime (memoized is fine).
- `Init` MUST NOT block for more than 100 ms; long-running work goes in `HasLifecycle.Start`.
- `Init` MUST be safe to call exactly once; calling twice is a host bug, not a plugin bug.
- Plugins MUST NOT hold a package-level reference to the `Host`; only the `Init` ctx-scoped copy.
- `Shutdown` MUST be idempotent; the host may call it more than once on chained signals.
+55
View File
@@ -0,0 +1,55 @@
# Contract — REST endpoints
## Core endpoints (added by this feature)
### `GET /api/plugins/status`
Returns the registry state for all plugins.
**Auth**: session cookie or `Authorization: Bearer <api-key>` (any authenticated user).
**Response 200**:
```json
{
"plugins": [
{
"name": "wiki",
"version": "0.1.0",
"stability": "stable",
"enabled": true,
"status": "started",
"started_at": "2026-04-19T10:30:12Z",
"error": null,
"capabilities": ["migrations", "actions", "http_routes", "web_panel", "config_schema"],
"tools_registered": ["create_article","get_article","list_articles","update_article","get_backlinks"],
"migration_versions": [1]
}
]
}
```
Error shapes: standard `{"error":"…"}` with appropriate HTTP status.
### `POST /api/plugins/{name}/enable` (admin only)
Sets `plugins.<name>.enabled: true` in `synapbus.yaml` and sends self-SIGHUP. Returns 202 Accepted with `{"restart": true}`; operator observes the graceful restart.
### `POST /api/plugins/{name}/disable` (admin only)
Sets `plugins.<name>.enabled: false` in `synapbus.yaml` and sends self-SIGHUP.
### `GET /api/plugins/{name}/schema` (any auth)
Returns the plugin's `HasConfigSchema().ConfigSchema()` JSON Schema, or 404 if the plugin does not implement it.
## Plugin-owned endpoints
Plugins mount routes via `HasHTTPRoutes.RegisterRoutes(r chi.Router)`. The registry mounts the returned router under `/api/plugins/<name>/`. Plugins MUST NOT register routes outside this namespace.
## UI panel endpoints
Plugins with `HasWebPanels` MUST serve assets under `/ui/plugins/<name>/`. The registry mounts `plugin.PanelHandler()` under this prefix. The plugin's `index.html` is served at `/ui/plugins/<name>/`.
## Core shell changes
The existing `/ui/*` shell (Svelte) fetches `/api/plugins/status` on load and renders plugin panels in the nav based on the `capabilities` including `"web_panel"` and the `panels` field derived from `HasWebPanels().WebPanels()`. Clicking a panel entry in the nav opens the route `/ui/plugins/<name>` in an iframe in the content pane.
+181
View File
@@ -0,0 +1,181 @@
# Phase 1 Data Model — Plugin System
## Entities
### Plugin (in-memory, code-defined)
Not persisted directly; it is a Go value constructed by a factory.
| Field | Type | Notes |
|---|---|---|
| Name | string | Unique, lowercased, snake-case. Panic on duplicate at registry build time. |
| Version | string | Semver ("0.1.0"). Informational only in this feature. |
| Stability | string | "stable" / "beta" / "experimental". Defaults to "stable". Non-stable emits a log warning but is not gated. |
### Plugin Registry (in-memory, one per process)
Assembled at boot from `defaultPlugins()` filtered by config.
| Field | Type | Notes |
|---|---|---|
| plugins | `map[string]*entry` | Name → entry, panic on duplicate. |
| order | `[]string` | Registration order, used for migrate/init/start sequence. |
Per-entry `struct` fields:
| Field | Type | Notes |
|---|---|---|
| plugin | `Plugin` | The plugin value. |
| config | `json.RawMessage` | From YAML sub-tree. |
| enabled | `bool` | From YAML `plugins.<name>.enabled`. Default false unless the plugin has a default-enabled hint. |
| status | `Status` | `registered` → `migrated` → `initialized` → `started` \| `failed` \| `disabled` |
| error | `error` | Populated when status == `failed`. |
| startedAt | `time.Time` | Populated on transition to `started`. |
### Plugin Migration Record (SQLite, core table)
Table `plugin_migrations`, created by core migration `000_initial.sql`.
| Column | Type | Notes |
|---|---|---|
| plugin | TEXT | Plugin name. PK (plugin, version). |
| version | INTEGER | Plugin-local migration number, monotonically increasing within plugin. |
| name | TEXT | Human-readable slug ("001_initial"). |
| applied_at | DATETIME | Default `CURRENT_TIMESTAMP`. |
| checksum | TEXT | SHA-256 of the migration SQL; compared on re-apply to detect drift. |
### Plugin-owned tables (example: wiki)
The wiki plugin's migration `001_initial.sql` (executed against the same SQLite handle as core):
```sql
CREATE TABLE IF NOT EXISTS plugin_wiki_articles (
id INTEGER PRIMARY KEY AUTOINCREMENT,
slug TEXT NOT NULL UNIQUE,
title TEXT NOT NULL,
body TEXT NOT NULL,
revision INTEGER NOT NULL DEFAULT 1,
word_count INTEGER NOT NULL DEFAULT 0,
created_by TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_by TEXT,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS plugin_wiki_backlinks (
from_slug TEXT NOT NULL,
to_slug TEXT NOT NULL,
PRIMARY KEY (from_slug, to_slug)
);
CREATE INDEX IF NOT EXISTS idx_plugin_wiki_articles_slug
ON plugin_wiki_articles(slug);
```
Tables are prefixed `plugin_wiki_*` for audit / lint clarity.
### Host (in-memory, one per plugin per process)
Constructed by the registry when calling `Plugin.Init(ctx, host)`. Not persisted.
| Field | Type | Purpose |
|---|---|---|
| Logger | `*slog.Logger` | Pre-tagged `plugin=<name>`. |
| DB | `*sqlx.DB` | Shared core handle. |
| Messaging | `messaging.API` | Send DMs / channel messages. |
| Channels | `channels.API` | CRUD, post, reactions. |
| Attachments | `attachments.Store` | CAS blob API. |
| Search | `search.Index` | Read-only semantic + FTS. |
| Secrets | `secrets.Scoped` | Scoped to this plugin only. |
| Events | `eventbus.Bus` | Subscribe / publish. |
| Config | `json.RawMessage` | This plugin's YAML sub-tree. |
| DataDir | `string` | `<--data>/plugins/<name>/`, pre-created. |
| Tracer | `trace.Tracer` | OTel tracer scoped to plugin. |
| Metrics | `prometheus.Registerer` | Sub-registry scoped to plugin. |
| DefaultOwner | `*users.User` | For failure notifications. Read-only snapshot. |
### Plugin Status (runtime JSON, served by `/api/plugins/status`)
```json
{
"plugins": [
{
"name": "wiki",
"version": "0.1.0",
"stability": "stable",
"enabled": true,
"status": "started",
"started_at": "2026-04-19T10:30:12Z",
"error": null,
"capabilities": ["migrations", "mcp_tools", "actions", "http_routes", "web_panel", "config_schema"],
"tools_registered": ["create_article", "get_article", "list_articles", "update_article", "get_backlinks"],
"migration_versions": [1]
},
{
"name": "demo_broken",
"version": "0.0.1",
"stability": "experimental",
"enabled": true,
"status": "failed",
"started_at": null,
"error": "Init: missing required config field 'token'",
"capabilities": ["config_schema"],
"tools_registered": [],
"migration_versions": []
}
]
}
```
### Backup Archive (filesystem artifact)
Produced by `scripts/backup-kubic.sh`. Structure:
```
synapbus-backup-YYYY-MM-DDTHH-MM-SSZ.tar.gz
├── manifest.json # name + SHA-256 + size of each entry
├── synapbus.db # SQLite main file
├── synapbus.db-wal # SQLite WAL (if present)
├── attachments/ # CAS directory
│ └── <hash>/<rest> # existing structure preserved
├── secrets.key # master key (mode 0600)
└── hnsw.idx # vector index snapshot
```
`manifest.json` example:
```json
{
"created_at": "2026-04-19T11:00:00Z",
"synapbus_version": "0.12.3+pre-plugin-refactor",
"source_host": "hub.synapbus.dev",
"entries": [
{"path": "synapbus.db", "sha256": "…", "bytes": 48_218_112},
{"path": "secrets.key", "sha256": "…", "bytes": 32},
…
]
}
```
## State transitions
```
Plugin lifecycle:
registered → migrated → initialized → started (happy path)
│ │ │
└── failed ──┴── failed ──┴── failed (on any error)
Disabled plugins stay in:
registered → disabled
```
Transitions fire `EventBus` events on `plugin.status.changed` with `{name, from, to, error?}`.
## Validation rules (derived from FRs)
- Plugin name MUST match `^[a-z][a-z0-9_]{1,31}$`. (Prevents path traversal, YAML quirks, table name explosions.)
- Duplicate plugin name → panic at `defaultPlugins()` processing (FR-004).
- Duplicate tool name across plugins → panic during `Init` phase (FR-022).
- Plugin tables not prefixed `plugin_<name>_` → migration refused at apply time, plugin marked failed (FR-010).
- `HasConfigSchema` schema validation failure → plugin marked failed with the validator diagnostic (edge case: invalid config references).
- Cross-plugin secret access attempt → `secrets.Scoped` returns `ErrNotFound` deterministically; FR-007 + SC-006.
+119
View File
@@ -0,0 +1,119 @@
# Implementation Plan: Plugin System for SynapBus Core
**Branch**: `019-plugin-system` | **Date**: 2026-04-19 | **Spec**: [spec.md](./spec.md)
**Input**: Feature specification from `/specs/019-plugin-system/spec.md`
## Summary
Refactor SynapBus into a tiny core (messaging, channels, reactions, auth, storage, MCP transport, search, attachments, Web UI shell) plus a set of compile-in plugins. Deliver in three phases: Phase 0 backs up the live kubic instance and squashes 26 migrations into a single `000_initial.sql`; Phase 1 introduces `internal/plugin/` (interface, host, registry, lifecycle) plus the `plugintest` helper and wires SIGHUP-triggered graceful restart; Phase 2 extracts the wiki feature as the canonical pilot plugin. The remaining nine plugin candidates are mechanical follow-ups not in this spec.
## Technical Context
**Language/Version**: Go 1.25+ (per go.mod)
**Primary Dependencies**: `mark3labs/mcp-go` (MCP), `go-chi/chi` (HTTP), `spf13/cobra` (CLI), `modernc.org/sqlite` (storage), `jmoiron/sqlx` (query helpers), `cloudflare/tableflip` (graceful restart — NEW), `gopkg.in/yaml.v3` (config), `xeipuuv/gojsonschema` (config-schema validation)
**Storage**: SQLite via `modernc.org/sqlite` (pure Go, zero CGO). New core table `plugin_migrations`. Plugin tables namespaced `plugin_<name>_*`.
**Testing**: `go test`, table-driven; new `internal/plugin/plugintest/` package shipped alongside `internal/plugin/`; integration test boots the binary with a fixture config and exercises wiki via curl + MCP.
**Target Platform**: `linux/amd64`, `darwin/arm64`, `darwin/amd64` (per constitution III)
**Project Type**: Single-binary Go server with embedded Svelte Web UI
**Performance Goals**: Graceful restart < 2 s visible downtime on dev laptop; plugin `Init` < 100 ms each; `plugintest.Run` < 1 s per plugin.
**Constraints**: Zero CGO; plugins may not import from `internal/` outside `internal/plugin/*`; core may not import from `internal/plugins/*`; enforced via a compile-time lint pass.
**Scale/Scope**: ~12 eventual plugins; in this feature 1 (wiki) is extracted. Current codebase is ~44 KLOC Go + 28 KLOC tests across 37 `internal/` packages.
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
| Principle | Status | Notes |
|---|---|---|
| I. Local-First, Single Binary | ✅ Pass | Plugins compile-in; no RPC, no .so, no Wasm. `cloudflare/tableflip` is pure-Go, no daemon dependency. |
| II. MCP-Native | ✅ Pass | Plugins contribute MCP tools through `HasMCPTools` / `HasActions`; core MCP server remains the only agent interface. |
| III. Pure Go, Zero CGO | ✅ Pass | All new deps (`tableflip`, `yaml.v3`, `gojsonschema`) are pure Go. |
| IV. Multi-Tenant with Ownership | ✅ Pass | Plugin failure notification targets the default-agent owner. Host exposes owner-scoped messaging. |
| V. Embedded OAuth 2.1 | ✅ Pass | Auth stays in core; plugins receive an action-scope check via `Registration.RequiredScope`. |
| VI. Semantic-Ready Storage | ✅ Pass | Search index remains in core and is exposed via `Host.Search`. Plugins use it read-only. |
| VII. Swarm Intelligence Patterns | ✅ Pass | Channel-type pattern (`HasChannelType`) makes auction/blackboard a plug-point rather than a hardcode. |
| VIII. Observable by Default | ✅ Pass | Per-plugin tracer and metrics registerer on Host; `/api/plugins/status` exposes state. Failure paths log + DM the owner. |
| IX. Progressive Complexity | ✅ Pass | Disabling every plugin leaves a working tier-1 system (messaging + agent registration). |
| X. Web UI as First-Class | ✅ Pass | Plugin panels served from `/ui/plugins/<name>`; shell continues to own navigation and auth. |
All ten gates pass. No complexity-tracking entries required.
## Project Structure
### Documentation (this feature)
```
specs/019-plugin-system/
├── plan.md # this file
├── spec.md # feature spec
├── research.md # Phase 0 research synthesis
├── data-model.md # Phase 1 entity model
├── quickstart.md # "build your first plugin" walkthrough
├── contracts/
│ ├── plugin.md # Plugin + HasX interface contracts
│ ├── host.md # Host API contract
│ └── rest.md # /api/plugins/* contract
├── checklists/
│ └── requirements.md # spec quality checklist
└── tasks.md # Phase 2 output (generated by /speckit.tasks)
```
### Source Code (repository root)
```
internal/
├── plugin/ # NEW — plugin framework (~300 LOC)
│ ├── plugin.go # Plugin, HasX interfaces
│ ├── host.go # Host struct and scoped accessors
│ ├── registry.go # registry, enable/disable, panic-on-dup
│ ├── lifecycle.go # three-phase boot, shutdown in reverse
│ ├── status.go # Plugin status tracking + /api/plugins/status handler
│ ├── config.go # YAML load, per-plugin config extraction
│ ├── migrator.go # per-plugin migration runner using plugin_migrations table
│ ├── restart.go # SIGHUP → tableflip.Upgrade re-exec
│ └── plugintest/
│ ├── nop_host.go # NopHost with in-memory SQLite
│ ├── run.go # Run(t, plugin) full-lifecycle smoke
│ └── assertions.go # HasTool/HasRoute/HasMigration helpers
├── plugins/ # NEW — all plugins live here
│ ├── standard/
│ │ └── import.go # var Standard = []plugin.Plugin{ ... } consumed by main.go
│ └── wiki/ # FIRST EXTRACTED PLUGIN
│ ├── plugin.go # WikiPlugin implementing Plugin + HasActions + HasHTTPRoutes + HasWebPanels + HasMigrations + HasMCPTools + HasConfigSchema
│ ├── actions.go # create_article, get_article, list_articles, update_article, get_backlinks
│ ├── routes.go # /api/plugins/wiki/* (if any needed for the UI panel)
│ ├── panel.go # /ui/plugins/wiki/* serving embedded HTML+JS
│ ├── store.go # SQL access against plugin_wiki_articles (moved from internal/wiki)
│ ├── schema/
│ │ └── 001_initial.sql # creates plugin_wiki_articles, plugin_wiki_backlinks
│ ├── ui/ # embedded assets (Svelte or plain HTML — choose simplest)
│ │ └── index.html
│ └── plugin_test.go # plugintest.Run + action-level unit tests
│
cmd/synapbus/
├── main.go # calls plugin.NewRegistry(defaultPlugins()) and bootstraps
└── plugins.go # NEW — defaultPlugins() returns []plugin.Plugin
│
schema/ # core migrations
├── 000_initial.sql # NEW — squashed from all 26 prior migrations
└── (legacy files archived into ../legacy-migrations/ at squash time)
│
scripts/
├── backup-kubic.sh # NEW — SSH into kubic, dump DB + attachments + secrets key + hnsw index, produce manifest
├── verify-backup.sh # NEW — restore archive into scratch dir, schema diff
└── generate-squash.sh # NEW — sqlite .schema + light edits → 000_initial.sql
│
docs/plugins/ # NEW
├── authoring.md # step-by-step "write your first plugin"
├── lifecycle.md # migrate → init → start, shutdown semantics
└── capabilities.md # each HasX interface with example snippets
│
test/integration/
└── plugin_system_test.go # NEW — boots binary, curls MCP + REST, asserts enable/disable
```
**Structure Decision**: Standard Go `internal/` layout. The plugin framework lives at `internal/plugin/`; all plugins live at `internal/plugins/<name>/`. A single import barrel `internal/plugins/standard/import.go` is consumed by `cmd/synapbus/plugins.go` → `defaultPlugins()`. No `init()` registration is used. The compile-time boundary (core files must not import `internal/plugins/*`) is enforced by a `go vet` analyzer added in Phase 1.
## Complexity Tracking
No constitution violations. No entries required.
+178
View File
@@ -0,0 +1,178 @@
# Quickstart — Write your first SynapBus plugin
Goal: add a "hello" plugin that exposes an MCP action, a REST route, a Web UI panel, and a tiny migration — in under 20 minutes.
## 1. Create the package
```
internal/plugins/hello/
├── plugin.go
├── schema/
│ └── 001_initial.sql
├── ui/
│ └── index.html
└── plugin_test.go
```
## 2. Write the plugin
`internal/plugins/hello/plugin.go`:
```go
package hello
import (
"context"
_ "embed"
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/synapbus/synapbus/internal/plugin"
)
//go:embed schema/001_initial.sql
var migration001 string
//go:embed ui/index.html
var panelHTML []byte
type HelloPlugin struct{ host plugin.Host }
func New() *HelloPlugin { return &HelloPlugin{} }
func (p *HelloPlugin) Name() string { return "hello" }
func (p *HelloPlugin) Version() string { return "0.1.0" }
func (p *HelloPlugin) Init(ctx context.Context, host plugin.Host) error {
p.host = host
return nil
}
func (p *HelloPlugin) Migrations() []plugin.Migration {
return []plugin.Migration{
{Version: 1, Name: "001_initial", SQL: migration001},
}
}
func (p *HelloPlugin) Actions() []plugin.ActionRegistration {
return []plugin.ActionRegistration{{
Name: "say_hello",
Description: "Return a greeting for the given name.",
InputSchema: json.RawMessage(`{"type":"object","properties":{"name":{"type":"string"}}}`),
RequiredScope: plugin.ScopeRead,
Handler: p.sayHello,
}}
}
func (p *HelloPlugin) sayHello(ctx context.Context, args map[string]any) (any, error) {
name, _ := args["name"].(string)
if name == "" { name = "world" }
return map[string]string{"greeting": "hello, " + name}, nil
}
func (p *HelloPlugin) RegisterRoutes(r chi.Router) {
r.Get("/ping", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"ok":true}`))
w.Header().Set("Content-Type", "application/json")
})
}
func (p *HelloPlugin) WebPanels() []plugin.PanelManifest {
return []plugin.PanelManifest{{ID: "hello", Title: "Hello", Icon: "hand", Route: "/ui/plugins/hello", Scope: "member"}}
}
func (p *HelloPlugin) PanelHandler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/html")
_, _ = w.Write(panelHTML)
})
}
```
## 3. Write the migration
`internal/plugins/hello/schema/001_initial.sql`:
```sql
CREATE TABLE IF NOT EXISTS plugin_hello_greetings (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```
## 4. Write the UI panel
`internal/plugins/hello/ui/index.html`:
```html
<!doctype html>
<html><head><title>Hello</title></head>
<body><h1>Hello, plugin world</h1></body></html>
```
## 5. Register it
`cmd/synapbus/plugins.go`:
```go
package main
import (
"github.com/synapbus/synapbus/internal/plugin"
"github.com/synapbus/synapbus/internal/plugins/hello"
"github.com/synapbus/synapbus/internal/plugins/wiki"
)
func defaultPlugins() []plugin.Plugin {
return []plugin.Plugin{
wiki.New(),
hello.New(), // ← one line
}
}
```
## 6. Write the smoke test
`internal/plugins/hello/plugin_test.go`:
```go
package hello_test
import (
"testing"
"github.com/synapbus/synapbus/internal/plugin/plugintest"
"github.com/synapbus/synapbus/internal/plugins/hello"
)
func TestHelloPlugin(t *testing.T) {
plugintest.Run(t, hello.New())
}
```
## 7. Enable in config
`synapbus.yaml`:
```yaml
plugins:
wiki: { enabled: true }
hello: { enabled: true }
```
## 8. Run
```
make build
./synapbus serve --data ./data
```
- `curl http://localhost:8080/api/plugins/status | jq` — you see `hello` status=started.
- `curl http://localhost:8080/api/plugins/hello/ping` — returns `{"ok":true}`.
- Open `http://localhost:8080/ui/plugins/hello` in a browser — renders the HTML.
- Call `say_hello` via MCP: `mcp execute "call('say_hello', {name: 'you'})"` → `{"greeting":"hello, you"}`.
- Flip to `enabled: false`, send SIGHUP — everything above goes away cleanly.
Total new code: ~60 lines of Go. No core code modified.
+185
View File
@@ -0,0 +1,185 @@
# Phase 0 Research — Plugin System for SynapBus
**Status**: All unknowns resolved. Decisions below derived from prior brainstorming session + three parallel research agents (HashiCorp, Caddy v2, broader Go plugin landscape — OpenTelemetry Collector, Prometheus SD, Coraza, Cosmos-SDK, etc.).
---
## Decision 1 — Plugin interface shape
**Decision**: Tiny `Plugin` interface (Name, Version, Init) + optional `HasX` capability sub-interfaces detected by type assertion at registration.
**Rationale**:
- Terraform framework's growth from monolithic provider interface → optional sub-interfaces is a cautionary tale in reverse: the decomposition is inevitable.
- OpenTelemetry Collector `component.Component` is this shape and is the gold standard for compile-in Go plugins.
- Plugins pay only for the capabilities they need; a 15-line plugin is a valid plugin.
**Alternatives considered**:
- Monolithic `Plugin` with all methods — rejected: Cosmos-SDK's `AppModule` is living proof this ossifies.
- `Host.RegisterXxx()` imperative registration during `Init` — viable but harder to reason about ordering; capability interfaces make the contract declarative.
---
## Decision 2 — Registration mechanism
**Decision**: Explicit list in `defaultPlugins()` (cmd/synapbus/plugins.go), not `init()` side effects.
**Rationale**:
- OpenTelemetry Collector moved away from `init()` registration because test builds, OSS vs. enterprise distributions, and CI need to supply alternate plugin sets; `init()` makes this painful.
- One line per plugin is trivial ceremony.
- `go test` order-independence is easier to guarantee.
**Alternatives considered**:
- `init()` + blank imports (Caddy style) — rejected for the above reasons; the ergonomic win doesn't outweigh the control loss.
- Hybrid (init-registers + config allow-list) — rejected as unnecessarily two-layered.
---
## Decision 3 — Host API shape
**Decision**: Single `Host` struct passed to `Plugin.Init(ctx, host) error`. Fields are typed accessors for core services. Not globals, not a service locator with string keys.
**Rationale**:
- Vault's `BackendConfig` pattern + Kong's PDK struct both land here.
- Trivially mockable in tests — one `mockHost{}` covers 100% of plugin-side testing.
- Adding a new service means adding a field; no breaking signature change to `Init`.
**Alternatives considered**:
- `Host` as an interface — rejected: interfaces force one-at-a-time mocking and obscure which fields exist. A struct with exported typed fields is more Go-idiomatic for a closed set.
- Global singletons — rejected outright; violates DI and hurts testability.
---
## Decision 4 — Dynamic enable/disable approach
**Decision**: Compile-in all plugins. Enable/disable via YAML config. Runtime toggle triggers SIGHUP-based graceful restart using `cloudflare/tableflip`.
**Rationale**:
- Go's `plugin` package is effectively broken (Linux-only, no unload, deps must match exactly).
- HashiCorp go-plugin (subprocess + gRPC) would require a wire protocol for every extension point, including Web UI panels, which is impractical.
- Wasm (wazero) would constrain plugin authors significantly and adds a whole toolchain.
- Graceful restart with tableflip = ~200–800 ms visible downtime on dev; MCP clients reconnect automatically. This is 99% indistinguishable from real hot-load for the user.
**Alternatives considered**:
- Real hot-reload via Wasm — rejected for toolchain burden.
- Full restart (kill -TERM + re-spawn by systemd) — rejected: > 1 s downtime, in-flight HTTP/SSE requests lost.
---
## Decision 5 — SQL migration ownership
**Decision**: Each plugin owns its own numbered migration chain under `internal/plugins/<name>/schema/`. Core records applied migrations in a new `plugin_migrations (plugin, version)` table. Plugin tables namespaced `plugin_<name>_*`.
**Rationale**:
- Isolation: a plugin's schema lives with its code.
- Re-enable-after-disable works: tables and data survive.
- Namespace prefix prevents collisions and makes a renegade `SELECT * FROM plugin_wiki_articles` visible in reviews.
**Alternatives considered**:
- Single monolithic numbered chain — rejected: couples plugin code to a global numbering, makes alt distributions impossible.
- Per-plugin separate SQLite file — rejected: loses the simplicity advantage of one database handle.
---
## Decision 6 — Web UI panel integration
**Decision**: Each plugin serves its own HTML/JS from `/ui/plugins/<name>/*` backed by its own `embed.FS`. Core shell renders panels as iframes in the existing content pane, driven by the plugin-supplied `PanelManifest`.
**Rationale**:
- Zero build-time coupling between plugin releases and the core Svelte shell — upgrading the wiki doesn't rebuild the shell.
- Security isolation via iframe origin is a free bonus.
- Simple to implement — `http.FS(embed.FS)` with a Chi subrouter.
**Alternatives considered**:
- Dynamic ES module imports of Svelte components from the shell — rejected: requires a manifest merge at build time and couples versions.
- Server-rendered panels into the shell via fetch — rejected: still couples layout/style.
---
## Decision 7 — Config format
**Decision**: YAML (`synapbus.yaml`) at a known path (next to binary or `$SYNAPBUS_DATA_DIR/synapbus.yaml`). Each plugin's sub-tree is passed into its `Host.Config` as `json.RawMessage`; the plugin unmarshals into its typed struct. Optional `HasConfigSchema` lets the Web UI generate a form.
**Rationale**:
- YAML is the prevailing agent-tooling config format (LangChain, AutoGen, Helm, OTel all use YAML).
- `json.RawMessage` is used internally because that's what `yaml.v3` hands back after first-pass decode and Go's JSON-struct-tags are widely known.
- `HasConfigSchema` being optional means "zero-config" plugins still work.
**Alternatives considered**:
- HCL — rejected: adds HashiCorp dep, less familiar to Python-first audience.
- TOML — rejected: less familiar than YAML in agent ecosystem.
- Environment variables only — rejected: typed per-plugin blobs don't fit env-var flat keyspace.
---
## Decision 8 — Testing approach
**Decision**: Ship `internal/plugin/plugintest` package with `NopHost()` (in-memory SQLite + tmpdir) and `Run(t, plugin)` smoke helper.
**Rationale**:
- Copied from OTel Collector `componenttest.NewNopHost`.
- Every plugin gets a ≤ 20-line smoke test for free.
- Mocking the Host struct is the primary testing surface anyway; this formalizes it.
**Alternatives considered**:
- No helper (each plugin invents its own scaffolding) — rejected: produces inconsistent test quality and discourages testing.
---
## Decision 9 — Boundary enforcement
**Decision**: A custom `go/analysis` analyzer runs in CI that rejects imports from `internal/plugins/*` by files outside that subtree (and rejects imports into `internal/plugins/*` from core `internal/*` packages that aren't `internal/plugin*`).
**Rationale**:
- Compile-time enforcement of the architectural invariant.
- Without it, the `internal/` flat layout makes it trivially easy to accidentally cross the line.
**Alternatives considered**:
- `go-arch-lint` — rejected: another dep; a ~50-line custom analyzer is simpler.
- Honor system — rejected: precisely what we're escaping.
---
## Decision 10 — Squash of existing 26 migrations
**Decision**: Generate `000_initial.sql` from the live `.schema` output plus curated seed data for reference rows, after the kubic backup has been verified restorable. Archive legacy migration files under `schema/legacy/` for historical reference; they are not applied.
**Rationale**:
- No external users exist; backward-compat across the 26-step chain is unneeded.
- A single file is ~50× faster to boot and is the new baseline.
- Legacy chain is preserved for human forensic use.
**Alternatives considered**:
- Keep all 26 migrations forever — rejected: user explicitly authorized squash.
- Truncate legacy and delete — rejected: losing schema history is a small-but-real audit cost.
---
## Decision 11 — Failure-notification channel
**Decision**: On plugin `Init` error or `Start` panic: (a) mark the plugin failed in the registry, (b) log the error with `slog`, (c) send a direct message to the owner of the default system agent naming the plugin and the error.
**Rationale**:
- DM integration uses existing messaging; no new ops channel to manage.
- Surfaces plugin problems in the operator's normal workflow (the same inbox they read every day).
- Owner-of-default-agent is always defined and the simplest routing decision.
**Alternatives considered**:
- Dedicated `#plugin-alerts` channel — rejected: over-designed for a single-operator deployment.
- Webhook to external destination — rejected: violates local-first principle and adds config burden.
---
## Decision 12 — Integration test strategy
**Decision**: One `test/integration/plugin_system_test.go` boots the binary with a fixture YAML, waits for readiness, exercises the wiki plugin via `curl` + MCP JSON-RPC, then flips the config, sends SIGHUP, waits for restart, and re-checks. Uses `testcontainers`-free approach — just spawns `./synapbus serve` as a subprocess.
**Rationale**:
- No need for Docker in unit CI.
- End-to-end guarantees the three-phase boot works for real, not just in mocks.
**Alternatives considered**:
- Containerized e2e — rejected: slow, adds infra; this binary is single-file.
---
All Phase 0 unknowns resolved. Proceeding to Phase 1.