Two silent failures making the watchdog's circuit-breaker checks
toothless:
1) The Alpine runtime image had no sqlite3 binary, so every
`kubectl exec -- sqlite3 …` call inside the watchdog returned empty.
$JS/$TIN/$JOK/$JFL/$JCB all defaulted to 0 → every "soft cap" /
"tokens > 30M" / "circuit broke but still firing" check trivially
passed regardless of real state. apk add sqlite (≈700KB).
2) The "today usage" query filtered to owner_id='2' only, but dream
jobs run for any owner (we just saw a clean owner_id=1 dispatch).
Replace with SUM across all rows for date=date('now'); the caps
are intentionally global, not per-owner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SynapBus on kubic
Plain Kubernetes manifests for the kubic single-node MicroK8s cluster
(kubic.home.arpa). No Helm — the image is built locally, imported directly
into MicroK8s containerd, and rolled with kubectl set image.
Files
| File | Purpose |
|---|---|
namespace.yaml |
synapbus namespace |
pvc.yaml |
2 Gi PVC on microk8s-hostpath for /data (DB + WAL + attachments + HNSW index) |
secret.example.yaml |
Template for synapbus-secrets (OpenAI/Gemini keys, mounted via envFrom) |
deployment.yaml |
Single replica, docker.io/library/synapbus:vX.Y.Z-amd64, imagePullPolicy: IfNotPresent (image is pre-loaded into containerd) |
service.yaml |
NodePort 30088 on port 8080 |
otel-collector.yaml |
OpenTelemetry collector for traces/metrics |
Initial install
kubectl apply -f deploy/kubic/namespace.yaml
kubectl apply -f deploy/kubic/pvc.yaml
# Edit secret.example.yaml first — never commit real keys.
kubectl apply -f deploy/kubic/secret.example.yaml
kubectl apply -f deploy/kubic/service.yaml
kubectl apply -f deploy/kubic/deployment.yaml
Releasing a new version
scripts/deploy-kubic.sh v0.17.0
The script:
docker buildx build --platform linux/amd64with the version baked in.docker saveto a tarball.scptokubic.home.arpa.ssh kubic 'sudo microk8s ctr image import …'(loads the image into the in-cluster containerd registry — the image is not pushed to a remote registry).kubectl set image deploy/synapbus synapbus=docker.io/library/synapbus:vX.Y.Z-amd64.kubectl rollout status …and a/healthzsmoke test.
The docker.io/library/ prefix is required because that's how containerd
resolves image references that don't specify a registry — synapbus:v…
written into the deployment is normalised to docker.io/library/synapbus:v…
on the node.
Why no Helm?
The original chart under deploy/helm/ (since deleted) was used for the very
first install (Mar 2026) and then went into a failed state when someone
ran kubectl set image for a hotfix; subsequent helm upgrade attempts hit
server-side-apply ownership conflicts. Rather than reconcile, we now own the
manifests directly. The deploy flow is simple enough that templating buys
nothing.
Backups
Before any version that touches schema, snapshot /data:
kubectl exec -n synapbus deploy/synapbus -- \
tar -C /data -cf - synapbus.db synapbus.db-shm synapbus.db-wal vapid_keys.json \
| tar -xf - -C "$HOME/synapbus-backups/$(date -u +%Y%m%dT%H%M%SZ)/"
Then sqlite3 synapbus.db 'PRAGMA wal_checkpoint(TRUNCATE); PRAGMA integrity_check;'
to fold the WAL into the main file and verify integrity before archiving.