Spec: Sandbox

Purpose

This spec defines the sandbox — the mechanism that enforces the cage contract declared by the project DSL. It compiles DSL cage blocks into concrete kernel-level isolation, runs subagent processes inside that isolation, enforces network and filesystem allowlists at the kernel boundary, applies seccomp filtering and cgroup resource limits, and tears the whole apparatus down cleanly when subagents exit.

This document is normative for:

  • The cage compiler — translating a DSL cage block into bwrap argv plus auxiliary setup.
  • The network gatekeeper — per-cage network namespace, kaged-managed DNS, hostname-aware filtering, nftables policy.
  • The default seccomp profile — which syscalls are blocked, allowed, or logged.
  • The cgroups wrapper — how memory/CPU/PID/walltime limits are enforced.
  • The supervisor's spawn/observe/signal/reap protocol.
  • The cage: disabled path — what happens when the operator opts out.
  • The audit events the sandbox emits.
  • The failure-recovery and cleanup guarantees.

It is not normative for:

  • The DSL's cage syntax (that's project-dsl.md Cage block).
  • The subagent supervisor's higher-level responsibilities (run lifecycle, primary→subagent dispatch — that's daemon.md and session-manager.md).
  • The plugin host's sandbox (plugins are sandboxed using the same compiler — see plugin-host.md — but the policy model is different).
  • macOS/Windows sandboxing (deferred; v0 is Linux only per ADR-0009).

Constraints (from ADRs)

Constraint Source
bwrap is the v0 cage mechanism; no firejail, no podman as cage ADR-0009
Network allowlist enforced by kaged-managed per-cage netns + nftables + userspace proxy ADR-0009 amendment Phase 1
Default seccomp profile blocks dangerous syscalls; relaxed mode opt-in per cage ADR-0009
cgroups via systemd-run when systemd is present; direct cgroup v2 fallback otherwise ADR-0009
cage: disabled runs as daemon UID with full host access, no half-measures ADR-0009 amendment
--no-sandbox daemon flag overrides every cage to disabled ADR-0009 amendment
Cage fs[].path is project-root-relative; sandbox resolves to <project-root>/<path> ADR-0011
Works under both deployment modes (per-user UID or system-wide kaged UID) ADR-0010
Implementation language is TypeScript on Bun ADR-0004

Threat model (recap from ADR-0009)

In scope (P0 if violated):

  • A subagent breaking out of its bwrap mounts and reading host paths not in its allowlist.
  • A subagent reaching a network destination not in its allowlist.
  • A subagent persisting state to host disk when declared ephemeral.
  • A subagent signaling the daemon or another subagent.
  • A subagent escalating to root on the host.
  • A subagent exfiltrating other subagents' memory.
  • The supervisor failing to apply a cage policy at all.

Out of scope:

  • Kernel CVEs in the namespace/seccomp surface.
  • LLM-provider-side data leakage.
  • CPU side-channel attacks (Spectre-class).
  • A misbehaving subagent exhausting its cgroup limits (DoS within budget is allowed).

Component architecture

                ┌────────────────────────────────┐
                │  SubagentSupervisor (daemon)   │
                └──────────────┬─────────────────┘
                               │ spawn(cage_policy, project_root, argv)
                               ▼
                ┌────────────────────────────────┐
                │       CageCompiler             │
                │  DSL cage → bwrap argv +       │
                │  netns plan + cgroup setup     │
                └─┬────────────────────────┬─────┘
                  │                        │
                  ▼                        ▼
        ┌─────────────────┐    ┌─────────────────────┐
        │ NetworkGatekeeper│    │  CgroupsWrapper     │
        │  netns setup    │    │  via systemd-run    │
        │  resolver + proxy│   │  or cgroup v2 direct│
        │  nftables rules │    └─────────────────────┘
        └─────────────────┘
                  │
                  ▼
                ┌────────────────────────────────┐
                │  bwrap (subprocess)            │
                │  └── subagent process          │
                │      seccomp filter applied    │
                │      cgroup limits enforced    │
                └────────────────────────────────┘

Five components in packages/sandbox/:

  • CageCompiler — pure function: takes a parsed cage policy plus the project root and produces an EffectiveCage value (bwrap argv, netns plan, cgroup spec, seccomp profile reference, scratch-dir path).
  • NetworkGatekeeper — daemon-wide service. Sets up and tears down per-cage network namespaces, runs the kaged-managed DNS resolver, owns the hostname-allowlisting SOCKS proxy, manages nftables rules.
  • CgroupsWrapper — applies resource limits via systemd-run when available, falls back to direct cgroup v2 manipulation otherwise.
  • SandboxRunner — orchestrates the spawn: invokes the compiler, asks the gatekeeper to set up net, applies cgroups, executes bwrap, returns a handle.
  • SandboxHandle — runtime representation of a live cage. Streams stdout/stderr, accepts signals, observes exit, triggers cleanup.

The SubagentSupervisor in the daemon calls SandboxRunner.spawn(...) and gets a SandboxHandle back. Everything below the supervisor is the sandbox package's job.


The cage compiler

Inputs

// Conceptual TypeScript shape; actual types in packages/sandbox/types.ts.

type CagePolicy =
  | { kind: "disabled" }
  | {
      kind: "enabled";
      fs: Array<{ mode: "ro" | "rw"; path: string }>;       // project-relative
      net: { allow: string[] };                             // hostnames, CIDRs
      state: "ephemeral" | "scratch";
      seccomp: "default" | "relaxed";
      limits: {
        memory_mb: number;
        cpu_shares: number;
        pids: number;
        walltime_sec: number;
      };
    };

type CompileInput = {
  policy: CagePolicy;
  project_root: string;            // absolute path on host; resolved by daemon
  project_id: string;              // for naming the cage and netns
  invocation_id: string;           // ULID, unique per spawn
  session_id: string;              // for audit correlation
  daemon_mode: "user" | "system";  // per ADR-0010
  daemon_uid: number;              // who the daemon runs as
};

Output

type EffectiveCage = {
  // Bubblewrap argv (built but not yet executed)
  bwrap_argv: string[];

  // Network plan (handed to NetworkGatekeeper)
  netns: {
    name: string;                   // unique per invocation, e.g. "kaged-cage-<inv_id>"
    allow: string[];                // hostnames and CIDRs
    enabled: boolean;               // false when net.allow is []
  };

  // Resource limits (handed to CgroupsWrapper)
  cgroup: {
    name: string;                   // matches bwrap process
    memory_mb: number;
    cpu_shares: number;
    pids: number;
    walltime_sec: number;
  };

  // Seccomp profile (filesystem path; bwrap applies via --seccomp <fd>)
  seccomp_profile: string;          // path to BPF file

  // Scratch directory for state=ephemeral or scratch
  scratch_dir: string;              // mktemp, cleaned per `state`

  // Audit metadata
  audit: {
    summary: string;                // human-readable one-line summary
    fs_mounts: Array<{ mode: string; src_host: string; dst_cage: string }>;
    net_allow: string[];
  };
};

The compiler is pure: same input → same output (modulo invocation_id which is timestamp-derived). This enables straightforward testing — pass fixtures, assert outputs.

Compile rules

Filesystem (fs):

  1. For each { mode, path } entry:
    • Resolve path against project_root: resolved = join(project_root, path).
    • Reject if resolved escapes project_root (defensive — the DSL parser already rejected this, but the compiler defends in depth).
    • For ro: append --ro-bind <resolved> <resolved> to bwrap argv (same path inside and outside the cage — operators and prompts can refer to the same paths).
    • For rw: append --bind <resolved> <resolved>.
  2. Always add a tmpfs at /tmp (--tmpfs /tmp).
  3. Always mount the scratch dir at a known location inside the cage:
    • state: ephemeral--bind <scratch_dir> /scratch. Scratch dir is created fresh, cleaned on exit.
    • state: scratch → same, but the scratch dir persists across invocations within the same session (cleaned at session end).
  4. Mount /proc and /dev with the standard restricted subsets:
    • --proc /proc
    • --dev /dev
  5. Unshare namespaces: --unshare-user --unshare-ipc --unshare-pid --unshare-uts --unshare-cgroup. Network is unshared separately (see Network section).
  6. Hostname: --hostname kaged-<invocation_id> (avoid leaking host hostname).
  7. Set --die-with-parent --new-session.
  8. UID mapping: map the daemon UID to UID 65534 (nobody) inside the cage. This is the conservative default; cages do not need to be root inside.

Network (net.allow):

The net.allow list is not directly converted to bwrap args — bwrap can only "share or unshare" the net namespace. Hostname-level allowlisting requires the network gatekeeper (next section). The compiler emits:

  • --unshare-net for the cage (it starts with no network).
  • A netns plan handed to the gatekeeper. The gatekeeper sets up the cage's interface and proxy before the bwrap process starts.

For empty allow: [], the compiler emits the unshare-net but no gatekeeper plan — no veth, no proxy, no DNS. Cleanest fully-isolated case.

Seccomp:

  • seccomp: default → reference the default profile shipped at ${KAGED_HOME}/runtime/seccomp/default.bpf.
  • seccomp: relaxed → reference relaxed.bpf at the same path.

The compiler does not invent profiles per cage. There are exactly two; both are pre-compiled and shipped with the daemon. See Seccomp profile below.

Cgroups:

  • limits.memory_mb → cgroup memory.max.
  • limits.cpu_shares → cgroup cpu.weight.
  • limits.pids → cgroup pids.max.
  • limits.walltime_sec → enforced by the SandboxRunner via a watchdog (not cgroup; cgroups don't do walltime).

Audit summary:

The compiler produces a one-line human-readable summary used in audit logs and the UI cage-policy panel:

cage[scraper@01HXAB] fs=ro:./data,rw:./scratch net=*.bandcamp.com,*.soundcloud.com state=ephemeral mem=256mb walltime=600s

Defensive checks

The compiler validates input even though the DSL parser already did:

  • project_root must exist and be a directory.
  • All fs[].path resolutions must stay inside project_root.
  • net.allow entries that look like provider:model or IPs-with-trailing-junk are rejected.
  • limits.memory_mb >= 16, pids >= 1, walltime_sec >= 1.

Defensive checks fail loud with a clear error pointing at the DSL parser as the layer that should have caught it. A compiler failure on input the parser accepted is a bug.


Network gatekeeper

The hardest piece. Per ADR-0009 amendment, v0 targets Phase 1: per-cage netns + kaged-managed DNS resolver + userspace SOCKS proxy + nftables rules. v0.x's Phase 2 (kernel-level SNI/Host introspection) is a later optimization.

Daemon-wide setup (once, at daemon startup)

  1. The NetworkGatekeeper starts as part of daemon startup phase 3 (running) — see daemon.md Subsystem dependency order. It comes up AFTER storage but BEFORE the subagent supervisor.
  2. Create a daemon-side bridge interface: kaged-br0 (configurable).
  3. Start the kaged-managed DNS resolver: listens on a UDP socket on kaged-br0. Forwards queries to the host's resolver; only returns answers for hostnames that match a current allowlist (set per-cage).
  4. Start the kaged-managed SOCKS5 proxy: listens on a TCP socket on kaged-br0. Accepts connect requests, inspects the requested hostname, looks up the cage that originated the request (by source IP within the bridge subnet), checks the hostname against that cage's allowlist, opens the upstream connection if allowed, refuses with SOCKS error otherwise.
  5. Establish base nftables rules in the gatekeeper's own (root) namespace: NAT for outbound from cages, FORWARD policy.

The resolver and proxy run as goroutines inside the daemon process (not separate processes). They are governed by the gatekeeper module; they have no network privileges of their own beyond what the daemon has.

Per-cage setup (called by SandboxRunner.spawn)

For each cage with a non-empty net.allow:

  1. Create a network namespace: ip netns add kaged-cage-<invocation_id>.
  2. Create a veth pair: ip link add kaged-veth-host-<id> type veth peer name kaged-veth-cage-<id>.
  3. Move the cage end into the netns: ip link set kaged-veth-cage-<id> netns kaged-cage-<id>.
  4. Plug the host end into the bridge: ip link set kaged-veth-host-<id> master kaged-br0.
  5. Assign IPs from a private range (10.143.<cage>.<endpoint>/30).
  6. Inside the cage's netns:
    • Set the default route to the bridge IP.
    • Set /etc/resolv.conf (inside the cage's filesystem view) to point at the kaged resolver IP.
    • Install nftables rules in the cage's netns:
      • Allow UDP 53 to the resolver.
      • Allow TCP outbound only via SOCKS to the kaged proxy IP:port.
      • DROP everything else.
  7. Register the cage in the gatekeeper's allow-table: <cage_ip> → { allow: [...], invocation_id: ..., project_id: ... }.
  8. Hand the netns name to the SandboxRunner, which adds --ns-pid <pid_of_netns_holder> style args to bwrap so the spawned process inherits the namespace.

The bwrap call itself only needs to NOT unshare net (since we did it for it). The compiler emits --share-net inside a specifically-configured netns.

Per-cage teardown (called when subagent exits)

  1. Unregister the cage from the gatekeeper's allow-table.
  2. Tear down the netns: ip netns delete kaged-cage-<id>. Kernel cleans veth automatically.
  3. The bridge kaged-br0 is daemon-wide and stays up.

DNS resolver semantics

  • Listens for A and AAAA queries on UDP 53.
  • For each query:
    • Identify the source cage by source IP.
    • Look up the cage's net.allow list.
    • Check the queried hostname against the allowlist patterns:
      • example.com → exact match.
      • *.example.com → matches <one-label>.example.com but not example.com itself or deeper subdomains.
      • **.example.com → matches any depth of subdomain.
      • CIDR allowlist entries are skipped (they're IP-only and don't affect DNS).
    • If allowed: forward the query to the host's upstream resolver, return the answer with TTL capped at 60s (kaged caches; we don't trust upstream TTLs for security).
    • If denied: return NXDOMAIN. Audit event gatekeeper.dns_denied.
  • IPv6: AAAA queries are answered NXDOMAIN unconditionally in v0 (simpler; revisit in v0.x if operators actually use v6 to allowlisted hosts).

SOCKS5 proxy semantics

  • Listens on TCP. Accepts SOCKS5 with no auth (the source-IP check IS the auth — only cages on kaged-br0 reach the proxy).
  • For each CONNECT request:
    • Identify the source cage by source IP.
    • Look up the cage's allowlist.
    • The CONNECT target is either a hostname or an IP:
      • Hostname: check against allowlist patterns. If a port was specified in the allowlist (example.com:443), the CONNECT port must match too.
      • IP: check against CIDR allowlist entries.
    • On match: resolve the hostname (re-use the gatekeeper's resolver cache), open the upstream connection, splice the streams.
    • On deny: send SOCKS5 error 0x02 (connection not allowed by ruleset). Audit event gatekeeper.tcp_denied.

Why this is safe (and what isn't)

  • The cage cannot bypass the resolver because non-resolver UDP 53 is dropped by the in-namespace nftables rule.
  • The cage cannot bypass the proxy because non-proxy TCP outbound is dropped by the same rule.
  • The cage cannot reach the host loopback (the netns has its own loopback, disconnected from the host's).
  • The cage cannot reach other cages (each has its own /30 on the bridge; nftables drops cross-cage traffic).

What isn't enforced (documented limitations):

  • TLS hostname spoofing. If a cage is allowed to reach good.example.com:443 and tricks DNS into resolving to an attacker IP, the proxy connects to the attacker. v0.x will add SNI introspection on the proxy side. v0 documents this as a known limitation; the resolver-cap-at-60s mitigates simple DNS poisoning windows.
  • IPv6. AAAA queries fail; pure-v6 hosts are unreachable. Documented.
  • UDP outbound. v0 supports only TCP via SOCKS. UDP applications (QUIC, custom protocols) don't work from cages in v0. v0.x revisits.
  • Raw sockets / ICMP. Blocked. Subagents cannot ping anything.

Seccomp profile

Two BPF profiles are shipped pre-compiled. Both are loaded from ${KAGED_HOME}/runtime/seccomp/{default,relaxed}.bpf (or whatever the daemon resolves at startup).

default (the unwavering default)

Blocks the following syscalls with EPERM:

Syscall Reason
ptrace Process injection / debugger attach
kexec_load, kexec_file_load Boot a new kernel
init_module, finit_module, delete_module Load/unload kernel modules
keyctl, request_key, add_key Kernel keyring access
mount, umount, umount2, pivot_root Mount manipulation
swapon, swapoff Swap manipulation
reboot System reboot
nfsservctl, vmsplice, migrate_pages, move_pages Historically exploit-prone
userfaultfd Userfaultfd-based attacks
bpf Loading new BPF programs from inside the cage
perf_event_open Performance-counter side-channel
setns, unshare (with non-trivial flags) Re-namespacing
Any syscall in CAP_SYS_* capability set Catch-all

Killed with SIGSYS (instead of EPERM) on:

Syscall Reason
__NR_iopl, __NR_ioperm x86 I/O port access
clock_settime, settimeofday Adjust system clock

Logged-only (allowed but recorded in audit log on call):

  • None in v0. v0.x may add chmod/chown with suspicious args to logged-only.

Everything else is allowed.

The profile is derived from Flatpak's bwrap-flatpak-seccomp and Docker's default profile, biased toward Flatpak (matches our bwrap-based posture). The exact list ships as a .json source that's compiled to BPF at daemon-build time and shipped in the binary.

relaxed

Blocks only:

  • reboot
  • kexec_load, kexec_file_load
  • init_module, finit_module, delete_module
  • swapon, swapoff

Everything else is allowed.

Use case: subagents that need to call into low-level APIs the default profile blocks (e.g., a debugging tool, a container-build helper, a benchmark that calls perf_event_open). The operator opts in by setting cage.seccomp: relaxed in the DSL.

There is no per-syscall opt-in. Two profiles is the v0 surface.


Cgroups wrapper

When systemd is available

The wrapper uses systemd-run --scope --slice=kaged.slice --quiet -p MemoryMax=<...> -p CPUWeight=<...> -p TasksMax=<...> to launch bwrap. Each cage becomes a transient systemd scope under kaged.slice.

Benefits:

  • Resource limits enforced by systemd.
  • journalctl -u kaged.slice shows per-cage usage history.
  • Clean teardown when bwrap exits.

The wrapper detects systemd availability by probing for /run/systemd/system (per the standard convention).

When systemd is not available

The wrapper falls back to direct cgroup v2 manipulation:

  1. Create a cgroup at /sys/fs/cgroup/kaged.slice/cage-<invocation_id>/ (requires write access; the daemon's user must have permission per daemon.md systemd-section's ProtectControlGroups=no).
  2. Write memory.max, cpu.weight, pids.max.
  3. Spawn bwrap.
  4. Write the bwrap PID into the cgroup's cgroup.procs.
  5. On exit: remove the cgroup directory.

In per-user mode without systemd, the operator may not have cgroup write access. The wrapper detects this and degrades: limits are not enforced, but the daemon emits a warning at startup (sandbox.limits_not_enforced) and the cage runs without limits. The audit log records this on every spawn (subagent.spawn.limits_skipped). Operators in this state are encouraged to enable cgroup delegation (systemctl --user enable --now app.slice in modern systemd setups, or accept the limitation).

Walltime enforcement

cgroups do not enforce walltime. The SandboxRunner starts a setTimeout(walltime_sec * 1000, killCage) per spawn. On expiry:

  1. Send SIGTERM to the bwrap process.
  2. After 5s grace period, send SIGKILL.
  3. Tear down the cage (network, cgroup, scratch).
  4. Emit audit event subagent.killed { reason: "walltime_exceeded" }.

If the daemon itself is killed mid-cage, the watchdog dies with it. On daemon restart, orphaned bwrap processes are NOT auto-killed (per daemon.md Crash semantics) — they appear in kaged status as untracked.


The cage: disabled path

Per ADR-0009 amendment, a subagent with cage: disabled (or any subagent when the daemon runs --no-sandbox) bypasses the sandbox entirely.

The SandboxRunner short-circuits:

  1. No CageCompiler invocation (or compile a no-op effective cage with empty plans for audit purposes).
  2. No netns setup.
  3. No cgroup wrapping.
  4. No seccomp filter.
  5. No bwrap.
  6. Spawn the subagent process directly via Bun.spawn, inheriting the daemon's UID, env, network, filesystem.

The audit event is subagent.spawn.uncaged (always, regardless of how the bypass was triggered — cage: disabled in DSL or --no-sandbox daemon flag). The event payload distinguishes:

{
  "event_type": "subagent.spawn.uncaged",
  "invocation_id": "01HX...",
  "subagent_name": "deployer",
  "project_id": "...",
  "reason": "dsl_disabled" | "daemon_no_sandbox"
}

Walltime is still enforced (the watchdog applies regardless of cage mode). The subagent is still SIGTERM'd if the session ends or the daemon drains.

The SandboxRunner returns a SandboxHandle with caged: false. Callers (the supervisor, the API surface) use that flag to drive UI badges and audit metadata.


Spawn lifecycle

A complete spawn (caged path):

SubagentSupervisor.spawn(policy, project_root, argv, env)
  │
  ▼
SandboxRunner.spawn(...)
  ├─ CageCompiler.compile(policy, project_root, invocation_id, ...)
  │    → EffectiveCage
  ├─ NetworkGatekeeper.setup(effective_cage.netns)
  │    → netns_name, gatekeeper_handle
  ├─ CgroupsWrapper.create(effective_cage.cgroup)
  │    → cgroup_handle
  ├─ Allocate scratch_dir per effective_cage.scratch_dir
  ├─ Build the bwrap argv (already in EffectiveCage)
  ├─ Spawn via Bun.spawn:
  │    - cmd: ["bwrap", ...effective_cage.bwrap_argv, "--", ...argv]
  │    - env: filtered (only project-safe vars passed in)
  │    - stdio: pipe
  │    - cgroup: handle.attach
  ├─ Start walltime watchdog
  ├─ Audit: subagent.spawn { cage_summary, invocation_id, ... }
  ▼
return SandboxHandle {
  invocation_id,
  pid,
  caged: true,
  effective_cage,
  stdout(), stderr(), signal(), wait(), close()
}

close() (called when supervisor decides the run ends):

  1. Cancel walltime watchdog.
  2. If process still running: SIGTERM, wait grace, SIGKILL.
  3. CgroupsWrapper.destroy(cgroup_handle).
  4. NetworkGatekeeper.teardown(netns_name).
  5. Remove scratch_dir if state == ephemeral. Preserve and re-mount-on-next-spawn if state == scratch (until session end).
  6. Audit: subagent.exit { invocation_id, exit_code, reason }.

Failure modes

Spawn-time failures

Failure Detection Behavior
bwrap not found on PATH At daemon startup (self-check gate per daemon.md) Daemon refuses to start
Compile error (path resolves outside project_root, invalid policy) CageCompiler spawn rejects with SandboxCompileError; supervisor marks run failed
Netns creation fails (kernel doesn't support userns, permission denied) NetworkGatekeeper spawn rejects with SandboxNetworkError; supervisor marks run failed
Cgroup creation fails CgroupsWrapper Logs warning, retries without limits if degraded mode is allowed; otherwise rejects
bwrap process spawn fails (exec error) Bun.spawn error spawn rejects with SandboxSpawnError; supervisor marks run failed
Project path missing (e.g., cage.fs.path: data but <root>/data doesn't exist) CageCompiler defensive check spawn rejects with SandboxPathMissing; supervisor marks run failed and surfaces clearly to UI

In all spawn-time failures, the partially-set-up cage is torn down before the error returns. There are no leaked netns, no orphaned cgroups, no dangling scratch dirs.

Runtime failures

Failure Detection Behavior
Subagent exits with non-zero SandboxHandle.wait() resolves with code Supervisor marks run failed; cage is torn down per close()
Subagent is killed by seccomp (SIGSYS) Exit signal Audit event subagent.killed { reason: "seccomp" } with which syscall (if available via dmesg-style channel); supervisor marks run failed
Subagent hits walltime Watchdog fires subagent.killed { reason: "walltime_exceeded" }; supervisor marks run failed
Subagent hits memory limit OOMKilled (cgroup) subagent.killed { reason: "oom" }; supervisor marks run failed
Network gatekeeper denies a request Gatekeeper logs the deny Audit event gatekeeper.{dns,tcp}_denied; subagent sees the connection failure; supervisor takes no action (denials are part of the policy, not bugs)
Resolver upstream down Gatekeeper detects DNS queries fail with SERVFAIL; subagent sees connection failures; audit logs gatekeeper.upstream_unreachable

Daemon-side failures

Failure Behavior
Daemon SIGTERM All sandboxes drain per daemon.md Phase 4: supervisor SIGTERMs subagents, sandboxes close cleanly
Daemon SIGKILL bwrap processes orphaned, netns dangling, cgroups left behind. Daemon's next startup runs runtime/ cleanup but does NOT kill orphans (per documented policy)
Gatekeeper crashes inside daemon process The daemon itself crashes (gatekeeper is in-process); recovery is daemon restart per Restart=on-failure

Audit events from the sandbox

Event When Carries
subagent.spawn After successful spawn (caged) invocation_id, project_id, session_id, cage_summary, effective_cage_hash
subagent.spawn.uncaged After successful spawn (uncaged) invocation_id, project_id, session_id, reason (dsl_disabled or daemon_no_sandbox)
subagent.exit On normal exit invocation_id, exit_code, duration_ms
subagent.killed On SIGTERM/SIGKILL by supervisor invocation_id, reason (walltime_exceeded, oom, seccomp, session_ended, cancelled)
gatekeeper.dns_allowed Per resolved query invocation_id, hostname, resolved_ip (sampled at audit.log_sample_rate; off by default)
gatekeeper.dns_denied Per denied query invocation_id, hostname
gatekeeper.tcp_allowed Per allowed CONNECT invocation_id, target, port (sampled; off by default)
gatekeeper.tcp_denied Per denied CONNECT invocation_id, target, port
gatekeeper.upstream_unreachable When the daemon's host resolver fails error message
sandbox.compile_error Compile-time failure invocation_id, error_code, details
sandbox.limits_not_enforced At startup if cgroup degradation reason
policy.violation Cage limit hit (memory, pids, walltime, seccomp) invocation_id, limit, value

dns_allowed and tcp_allowed default off because they're high-volume. Operators opt in via local config [audit].log_gatekeeper_allow_rate = 1.0 (sample rate 0-1) to capture them for debugging.


Plugin sandboxing (cross-reference)

Plugins are sandboxed using the same CageCompiler and SandboxRunner machinery. The differences:

  • Plugin manifests declare capabilities (read:fs:<path>, net:<host:port>, etc.) instead of a project DSL cage block. The plugin host translates capabilities to a CagePolicy before calling the sandbox.
  • Plugin sandboxes are typically wider (plugins often need exec:bash:...) but apply the same defense-in-depth.
  • Plugin sandboxes do not have cage: disabled mode — plugins always run sandboxed. The capability allowlist is the operator's lever.

Full plugin sandboxing details in plugin-host.md.


Implementation notes (not normative)

  • Bun.spawn vs node:child_process: use Bun.spawn. It exposes stdio streams as ReadableStreams natively and integrates with Bun's runtime.
  • netns syscalls from TypeScript: Bun.spawn(["ip", "netns", ...]) for the namespace operations. setns directly from JS isn't worth the FFI; shelling out to ip is fine.
  • Seccomp BPF compilation: done at daemon-build time by a small Rust or Go helper (libseccomp bindings); the compiled .bpf files ship in the binary. Runtime just loads them.
  • SOCKS proxy implementation: use Bun's Bun.listen for the TCP side. The SOCKS5 handshake is ~50 lines.
  • Resolver implementation: use Bun's UDP socket (Bun.udpSocket) for the listener; for upstream queries, call out to the host's resolver via dns.resolve from Bun's node compat.
  • Per-cage authorization in resolver/proxy: the source IP on kaged-br0 identifies the cage. The gatekeeper maintains an in-memory Map<IP, CageContext> updated on setup/teardown.
  • Walltime watchdog: setTimeout(walltime_sec * 1000, ...). Clear on normal exit.
  • Audit log writes are async but the spawn function awaits the first audit write before returning success — guarantees the event is durable before the supervisor proceeds.

Testing notes

Per ADR-0003:

  • CageCompiler tests — fixtures: each DSL cage shape from project-dsl.md examples, assert resulting bwrap argv exactly. Snapshot-style.
  • NetworkGatekeeper tests — integration: spin up the gatekeeper inside a test daemon, create a fake cage, assert DNS allow/deny against an allowlist, assert TCP allow/deny.
  • Escape attempts — these are CI-blocking tests. A subagent in a cage with fs: [] and net.allow: [] is given a script that tries to read /etc/shadow, fork, ptrace, kexec, mount, etc. EVERY attempt must fail. Adding new tests when new escape vectors are reported.
  • Walltime test — start a subagent that sleeps 60s with walltime_sec: 5. Assert it's killed within 5-10s with walltime_exceeded.
  • OOM test — start a subagent that mallocs in a loop with memory_mb: 32. Assert it's killed with oom.
  • Seccomp testdefault profile: subagent calls ptrace, asserts SIGSYS. relaxed profile: same call succeeds.
  • cage: disabled test — subagent declared disabled can read host files. The supervisor emits subagent.spawn.uncaged.
  • --no-sandbox test — daemon flag overrides all cages to disabled.
  • Cleanup tests — kill -9 the supervisor, then restart, verify orphaned bwrap processes are reported but not killed; verify the next clean spawn works.

Open questions

  1. IPv6 support. v0 returns NXDOMAIN for AAAA. Operators using v6-only hosts can't reach them. v0.x revisits.
  2. UDP outbound from cages. Not supported in v0. QUIC, custom UDP protocols don't work. Possible v0.x: a UDP-aware proxy alongside SOCKS.
  3. Per-syscall seccomp. Two profiles (default, relaxed) are the v0 surface. If operators consistently need a third, we add it as a built-in. Per-cage syscall allowlists are explicitly deferred — that's a big complexity bump.
  4. Cage telemetry granularity. What metrics does the gatekeeper expose to the daemon? Per-cage bytes-in/out, request-rate, denial-rate. v0 minimum: deny-counts. v0.x: full metrics for ops dashboards.
  5. Hostname allowlist semantics for SNI. v0.x will add SNI introspection in the proxy. v0 documents the TLS hostname spoofing limitation honestly.
  6. macOS support. Not v0. Deferred. The architectural shape (compiler + runner + handle) will translate; the bwrap+netns+nftables substrate will not.

Amendments

(none yet)


References