ADR-0041: Containerised daemon — single-binary build, distribution image, Kubernetes operation

• Status: Accepted
• Date: 2026-06-14
• Deciders: @karasu
• Supersedes: —
• Superseded by: —
• Depends on: ADR-0040 — the no-embedded-UI binary requires the UI to be independently hostable (ui.kaged.dev + the daemon registry).
• Relates to: ADR-0004 (Bun runtime; bun build --compile already blessed), ADR-0009 (bwrap), ADR-0008 (plugin tiers), the @kaged/natives distribution gap (STATUS infra table).

Context

Two goals drive this, and they are not the same goal:

1. IP protection during closed-source testing. Core source is not released until v2 (operator's call). Shipping a binary instead of source prevents casual theft by testers. The public plugin contracts (@kaged/plugin-types, the DSL, @kaged/sso-fixtures, the wire contract) are already public and are explicitly out of the protected set.
2. Zero-dependency containerised distribution + in-cluster operation. A tester should run the daemon with nothing installed but Docker. The operator should be able to run a daemon in a pod, in the same cluster as a project's running environment, develop the dev version in-pod, and push to staging/prod within that cluster. This foreshadows a Kubernetes plugin that extends daemon hooks/agent capabilities (cluster deploy/rollout/exec) — that plugin is out of scope here, but the image and pod identity must not preclude it.

The two goals reinforce the no-embedded-UI decision from ADR-0040: in the split topology a tester runs the container, visits the daemon URL, is redirected to ui.kaged.dev/connect, registers the daemon, and authenticates — zero source, zero dependencies but Docker.

Constraints that shape the build:

• ESM-only codebase ("type": "module", verbatimModuleSyntax, .ts extensions). bun build --compile handles ESM natively. We pin a Bun floor of ≥ 1.3.9.
• @kaged/natives is a ~93 MB per-triple NAPI cdylib. Only linux-x64-gnu has been built; linux-arm64-gnu is needed for the homelab (Pi) and likely the cluster. The project rule is natives are never committed to git — built in CI, distributed via optionalDependencies.
• Project plugins are bwrap-sandboxed subprocesses, frequently spawned as bun ./dist/index.js; system plugins are operator TypeScript dynamically imported at daemon startup. Both must keep working from a compiled daemon.
• PTYs/terminals use a tmux driver; tmux is a runtime dependency in the image.
• bwrap inside a container/pod requires unprivileged user namespaces or elevated capabilities (CAP_SYS_ADMIN / privileged), which many clusters restrict. The sandbox's CgroupsWrapper already detects and degrades (systemd-run / direct cgroup v2 / degraded-none).
• Multi-arch: bun build --compile --target is per-target and the natives .node is per-triple, so the image must be built per-arch (amd64 + arm64).

Decision

The daemon is distributed as a multi-arch OCI image containing a minified, source-stripped bun build --compile binary plus the matching-arch @kaged/natives .node, with no embedded UI. IP protection is explicitly obfuscation-grade — --compile --minify, no sourcemap — because Bun states bytecode does not obscure source. The only real hardening lever (moving sensitive core behind the Rust/NAPI boundary) is acknowledged and deferred.

1. Build

Per-arch:

bun build packages/daemon/src/main.ts \
  --compile --minify \
  --target=bun-linux-x64        # and bun-linux-arm64
  --outfile kaged-daemon
  # NO --sourcemap

• --minify mangles identifiers and strips comments/whitespace (the cheap obfuscation win).
• --sourcemap is forbidden in release builds — it embeds the zstd-compressed original source and auto-resolves it on error.
• Pin a Bun floor of ≥ 1.3.9 in the build environment.

2. IP-protection posture (stated honestly)

We adopt compile + minify. --bytecode is omitted — it is unstable in cross-compilation and provides no IP protection (Bun docs: does not obscure source). If cold-start benchmarks justify it in the future, it can be re-added. This is proportionate to "prevent casual theft during testing." A determined party can still recover readable-ish logic; string literals (prompts, route paths, messages) remain visible. If the threat model hardens before v2, the native boundary is the lever — that is a separate decision, not this one.

3. Image contents

A multi-stage build producing a slim runtime image:

• Builder stage: bun install (resolves the right-triple @kaged/natives via optionalDependencies), bun build --compile --minify --target=bun-linux-<arch>, and obtain the matching-arch .node.
• Runtime stage (slim Debian or equivalent — not distroless, because plugins need a shell + bun):
  • the compiled kaged-daemon binary,
  • the matching-arch @kaged/natives .node (copied beside the binary — see §4),
  • the Bun runtime on PATH (project plugins and agent code-exec expect bun; this does not expose daemon source, which lives inside the compiled binary),
  • tmux, git, bwrap, bash, ca-certificates,
  • no UI assets (served from ui.kaged.dev),
  • a non-root user; /data (sqlite + avatars) and /config as volumes.
• Tagged ghcr.io/kaged-dev/kaged-daemon per the repo image-naming convention.

4. Natives loader adaptation

bun build --compile bundles the JS module graph into a virtual FS (/$bunfs/...). The @kaged/natives loader currently resolves the .node relative to its module location, which inside a compiled binary points at the virtual FS — where the .node is not. The loader must resolve the .node from a runtime-real path: an explicit KAGED_NATIVES_PATH env override, falling back to a path derived from process.execPath (beside the executable). This is robust across Bun versions regardless of whether --compile learns to embed/extract .node assets, and it keeps the "natives never in git" rule intact — the .node is built in CI and copied into the image, never committed.

5. Multi-arch

docker buildx building both linux/amd64 and linux/arm64, each compiling its own binary and bundling its own-triple .node, published as a manifest list. This forces the long-pending linux-arm64-gnu natives build (STATUS #13 / the reported #1 install blocker) — closing that gap is a deliverable of this ADR, not a precondition.

6. Runtime configuration

Env-driven, no source needed:

• KAGED_BIND, KAGED_INSECURE, KAGED_NO_SANDBOX, KAGED_HOME, config volume (KAGED_CONFIG), data volume (sqlite URL).
• KAGED_UI_URL + KAGED_PUBLIC_URL (from ADR-0040) wire the bootstrap redirect to the hosted UI.
• /healthz and /readyz are the K8s liveness/readiness probe targets (the readyz gate already gates setReady(true)).

7. Sandboxing inside a container/pod

bwrap in a pod needs userns or privilege that clusters often forbid. The posture:

• Detect and degrade explicitly. Extend the existing cgroups degrade detection to bwrap/userns availability at startup. If bwrap cannot establish isolation, the daemon does not silently run subagents uncaged.
• Pod-as-sandbox for the in-cluster dev case. Running with cage degraded (or net-only) inside a pod is acceptable when the pod itself is the blast-radius boundary — which is the in-cluster dev model. Full bwrap isolation requires a privileged/userns-enabled pod and is documented as cluster-dependent opt-in.
• ⛔ invariant: the daemon never silently falls to uncaged execution. A degraded sandbox is surfaced in the startup banner, the warnings list (getMe daemon.warnings), and an audit event; it requires the same explicit posture as --no-sandbox.

8. Kubernetes operation

Shipped as an example, not a hard contract: a Deployment (single replica — the daemon is stateful/long-lived), a Service, a PersistentVolumeClaim for /data, a ConfigMap/Secret for config + provider keys, liveness/readiness probes on /healthz//readyz, and a ServiceAccount (initially unused; reserved for the future Kubernetes plugin's in-cluster API access). The no-embedded-UI decision fits cleanly: the in-cluster daemon serves only API + bootstrap redirect; the operator's browser loads ui.kaged.dev and connects via the registry.

9. CLI surface for the host wrapper

Per ADR-0042 §5, the kaged host wrapper delegates to the containerised daemon for commands the wrapper cannot answer locally. The daemon binary exposes a small CLI surface for this:

• kaged-daemon auth launch-url — prints the current connect/launch URL to stdout (consumed by the wrapper's kaged auth launch to open a browser).

Implementation — phased

Phase 0 — Build pipeline + natives resolution

• bun build --compile --minify --target=bun-linux-<arch>, no sourcemap; Bun floor ≥ 1.3.9 asserted.
• Natives loader resolves the .node via KAGED_NATIVES_PATH → process.execPath-relative fallback.
• Smoke test: compiled binary boots, loads natives (all 14 exports), spawns a subprocess, and runs bwrap-availability detection.
• ⛔ invariant: release builds contain no embedded sourcemap (asserting check on the artifact).

Phase 1 — Dockerfile (amd64 first)

• Multi-stage Dockerfile; runtime stage with binary + .node + bun + tmux/git/bwrap/bash/ca-certs; non-root user; /data + /config volumes; expose port; entrypoint runs kaged-daemon start.
• docker run with KAGED_UI_URL/KAGED_PUBLIC_URL reaches the bootstrap redirect end-to-end against ui.kaged.dev.

Phase 2 — Multi-arch + arm64 natives

• buildx manifest list for linux/amd64 + linux/arm64.
• linux-arm64-gnu natives build wired into CI (closes STATUS #13).
• ⛔ invariant: each arch image carries only its own-triple .node.

Phase 3 — Sandbox degrade detection

• Startup detection of bwrap/userns availability; explicit degrade with banner + daemon.warnings + audit event.
• ⛔ invariant: no silent uncaged execution (asserting test: bwrap unavailable ⇒ either explicit-degrade posture set or refuse to spawn caged subagents).

Phase 4 — Kubernetes example

• examples/deployment/kubernetes/ — Deployment/Service/PVC/ConfigMap/Secret/ServiceAccount + probes; operator walkthrough doc.

Phase 5 — Docs + STATUS

• Spec amendments per the checklist; STATUS infra table updated (first real build/release pipeline; arm64 natives; CI/CD line).

Amendment checklist (specs)

• docs/specs/daemon.md — container runtime knobs; /healthz + /readyz probe contract; natives path resolution; sandbox-availability detection + degrade posture.
• ADR-0009 or the sandbox spec — in-container/userns detection, degrade semantics, the no-silent-uncage invariant.
• @kaged/natives loader README/spec — KAGED_NATIVES_PATH + execPath-relative resolution; reaffirm "never in git, CI-built, image-copied."
• AGENTS.md — record the bun build --compile --minify release target and the natives-copy-not-commit rule for images.
• New examples/deployment/docker/ and examples/deployment/kubernetes/.
• STATUS.md — CI/CD (currently ❌), arm64 natives, image publishing.

Consequences

Easier

• Zero-dependency tester distribution (docker run + browser at ui.kaged.dev).
• The long-pending arm64 natives build gets done (forced by multi-arch).
• UI/daemon release cadence fully decoupled; the in-cluster dev loop becomes possible.

Harder / accepted costs

• IP protection is obfuscation-grade only. Stated plainly so no one mistakes the binary for a moat.
• Image is not small. The compiled binary embeds the Bun runtime (~59 MB), the image also ships Bun for plugins, plus the ~93 MB .node — a sizeable image. Acceptable for self-hosted/cluster distribution; note it.
• Bun present in the image slightly dilutes the "pure binary" intuition, but does not expose daemon source (which is compiled). Required for project plugins and agent code-exec.
• Sandboxing in-cluster is constrained (pod-as-sandbox), with full bwrap requiring privileged/userns.
• CI complexity: per-arch native builds + cross-compiled binaries + manifest lists.

Alternatives considered

Alternative A — ship a plain JS bundle, require Bun installed

Smallest image, no compile step. Rejected: leaks source (defeats goal 1) and is not zero-dependency (defeats goal 2).

Alternative B — rely on --bytecode as the IP-protection mechanism

Rejected: Bun's own docs state bytecode does not obscure source. Keeping it for startup only is fine; treating it as protection would be self-deception.

Alternative C — native-boundary rewrite of sensitive core now

Real protection. Deferred: disproportionate friction for "prevent casual theft during testing"; documented as the lever to pull if the threat model hardens before v2.

Alternative D — distroless / bun-free image

Smaller, tighter. Rejected for the default image because project plugins and agent code-exec expect a shell and bun. A slim "API-only, no-plugins, no-exec" variant could be offered later if there's demand.

Open questions

1. .node embedding vs beside-binary. We choose beside-the-binary + explicit path resolution to be Bun-version-robust. Revisit if bun build --compile gains reliable .node embed/extract.
2. Bytecode + native interop on arm64. Bun's 1.3.9 notes flag testing compiled artifacts for native-module interop; confirm specifically on the under-tested linux-arm64-gnu triple.
3. Sandbox-in-cluster posture. Is privileged/userns acceptable for the dev pod, or is pod-as-sandbox permanent? Default to pod-as-sandbox; document privileged opt-in.
4. Dynamic system-plugin import from a compiled binary. System plugins are operator TypeScript imported at runtime against the public @kaged/plugin-types; confirm dynamic import() of an external TS module resolves correctly from the compiled daemon.
5. Image variants. Single fat image vs a future slim API-only variant — defer until demand.
6. Bun version floor maintenance. Pinned ≥ 1.3.9 today; decide where the floor is recorded and enforced (build script assertion vs CI matrix).