影 kaged
← back to Architecture Decision Records Accepted

ADR-0030: Log streaming via Server-Sent Events

  • Status: Accepted
  • Date: 2026-06-02
  • Deciders: @karasu
  • Supersedes: —
  • Superseded by: —

Context

ADR-0029 established the structured operational logging pipeline (dual-sink SQLite + files, HTTP query endpoints). The logging spec explicitly deferred real-time log streaming to a future iteration. Currently the UI polls GET /projects/:id/logs every 3 seconds via React Query's refetchInterval. This is wasteful — most polls return no new entries, and when entries do arrive there's up to 3 seconds of latency.

The daemon already has WebSocket infrastructure for session-level streaming (ADR-0016: streaming tokens, run events, compaction events). However, log streaming is a different concern from session streaming:

  1. Logs are project-scoped, not session-scoped. Tying them to the session WS would pump log data through every connected session socket, even when the operator doesn't have the log drawer open.
  2. Logs are one-way (daemon → UI). No client-to-server messages needed.
  3. Log subscriptions are ephemeral — the operator opens the drawer, watches, closes it. No persistence requirement.
  4. The session WS already carries output, events, and pty channels. Adding a logs channel would increase frame traffic on a multiplex that's latency-sensitive for streaming tokens.

Why not extend the session WebSocket

The session WS (ADR-0016) is the wrong vehicle for project-scoped logs:

  • The session WS is tied to a single session. Project logs span all sessions in a project.
  • Adding log frames to the session WS means every frame needs filtering on the client side.
  • The session WS has backpressure limits per channel. Log traffic (especially debug-level during active sessions) could compete with token streaming for buffer space.
  • The operator may have the log drawer open while no session is active. No session WS exists in that state.

Why SSE, not a new WebSocket

SSE is the right fit because:

  1. One-directional. Log streaming is daemon → UI only. SSE is purpose-built for this. WS provides bidirectional capability that's unused here.
  2. No subprotocol negotiation. The session WS requires kaged.v1 subprotocol negotiation. SSE is just Content-Type: text/event-stream — simpler for a fire-and-forget stream.
  3. Auto-reconnect. The browser's EventSource API reconnects automatically on disconnect. No hello/resume_from_seq handshake needed — logs are a live tail, not a gapless replay protocol.
  4. HTTP-compatible. SSE works through the Vite dev proxy without special WebSocket configuration. It's a normal GET request with a long-lived response body.
  5. Lightweight per-connection. No framing overhead (no channel/seq/type wrapper). Each SSE event is a single data: line with a JSON payload.
  6. Complementary to the existing HTTP endpoints. The existing GET /projects/:id/logs returns paginated history. The new SSE endpoint streams only new entries. The UI fetches backlog via HTTP, then subscribes to SSE for live updates.

Decision

The daemon exposes a Server-Sent Events endpoint for live log streaming, scoped to a project. The UI subscribes when the log drawer opens and disconnects when it closes. Historical log entries are fetched via the existing HTTP endpoints; SSE streams only new entries written after subscription.

Endpoint

GET /api/v1/projects/:id/logs/stream

  • Content-Type: text/event-stream
  • Authentication: Same as all other /api/v1/ endpoints.
  • Query parameters: level and source filters (same semantics as the paginated endpoints). No pagination params — SSE is a forward-only live tail.

Event format

Each SSE event carries a single log entry as a JSON payload:

event: log
data: {"id":"01JX...","ts":1748700000000,"level":"error","source":"plugin","message":"...","projectId":"my-project","sessionId":"ses_abc","pluginName":"memory-markdown","context":{...}}
  • Event type: log for operational log entries. The event field is typed so the client can distinguish if we add other event types later (e.g., heartbeat).
  • Data payload: identical to the OperationalLogEntry shape from the logging spec.

Heartbeat

The daemon sends a comment line (: keepalive) every 30 seconds to prevent intermediate proxies from closing idle connections. The browser EventSource API ignores comment lines.

Subscription lifecycle

  1. Client opens EventSource on GET /projects/:id/logs/stream?source=session&level=info.
  2. Daemon validates the project exists and the filters are valid. Returns 404 or 400 on error (before switching to streaming).
  3. Daemon begins streaming new log entries that match the filters.
  4. Client calls eventSource.close() when the drawer closes.
  5. Daemon detects the disconnect and cleans up the subscription.

Backlog gap

SSE does not replay entries that were written before the subscription started. The UI's flow is:

  1. Fetch backlog via GET /projects/:id/logs?limit=100 (existing endpoint).
  2. Open SSE stream.
  3. Merge incoming SSE events into the local list, deduplicating by id against the backlog.

This avoids needing sequence numbers or replay buffers on the server side.

Daemon implementation

The daemon's logger (packages/daemon/src/runtime/logger.ts) gains a subscriber registry:

  • subscribe(projectId, filters, callback) → returns an unsubscribe function.
  • The write() function in logger.ts, after writing to both sinks, iterates active subscribers and calls matching callbacks.
  • The SSE handler creates a subscriber on connection, removes on disconnect.
  • Subscribers are filtered in-memory (level + source match against the subscriber's filter params).

No polling in the daemon. The write() path directly fans out to active SSE subscribers.

UI implementation

  • New hook: useProjectLogStream(projectId, filters, enabled) — opens an EventSource when enabled is true (drawer open), closes when false.
  • Returns { entries: LogEntry[] } — accumulates streamed entries since subscription.
  • LogDrawer combines: backlog from useProjectLogs (existing, without refetchInterval) + live entries from useProjectLogStream.
  • Deduplication by id against the backlog.

Consequences

What this commits us to

  • A new SSE endpoint in the daemon.
  • A subscriber registry in the logger module.
  • A new UI hook for SSE consumption.
  • Removing refetchInterval: 3000 from useProjectLogs.
  • The Vite dev proxy must support SSE streaming (text/event-stream responses). Vite's proxy handles this by default for non-upgrade requests.

What this forecloses

  • Using the session WebSocket for log delivery. Logs have their own transport now.
  • Backpressure-aware streaming for logs. SSE doesn't have WS's per-channel buffer limits. A misbehaving client that can't consume fast enough will have the connection dropped by the OS TCP buffer, which is acceptable — the client reconnects via EventSource auto-reconnect.

What becomes easier

  • Real-time log visibility. No polling delay, no wasted requests.
  • Clean separation between session streaming (WS) and log streaming (SSE).
  • The log drawer only consumes resources when open.

What becomes harder

  • One more transport to maintain. SSE is simpler than WS but still has edge cases (proxy timeouts, connection limits).
  • The subscriber registry adds a small overhead to every log write (iterate subscribers). Negligible for single-operator use; worth monitoring if multi-operator mode ever ships.

Alternatives considered

Alternative A — Extend session WebSocket with a logs channel

Why tempting: Reuses existing WS infrastructure. No new endpoint.

Why rejected: Logs are project-scoped, not session-scoped. Adding a logs channel to the session WS means log data flows through every session connection, even when the drawer is closed. The session WS is latency-sensitive for token streaming; log frames add noise. No session WS exists when the operator views logs without an active session.

Alternative B — WebSocket endpoint dedicated to logs

Why tempting: WS is already in the codebase. Bidirectional in case we need client → server commands later.

Why rejected: We don't need bidirectional for logs. WS requires subprotocol negotiation, a hello handshake, and frame-level multiplexing. SSE is a plain HTTP GET with text/event-stream. Simpler to implement, simpler to consume, auto-reconnect built into the browser API. Adding bidirectional capability "just in case" is premature complexity.

Alternative C — Polling with adaptive interval

Why tempting: Easiest to implement. Tune refetchInterval based on activity.

Why rejected: Still wasteful. An adaptive interval adds heuristics for no benefit — SSE is the correct solution for server-push of one-directional events. ADR-0016 already rejected polling for session state; the same reasoning applies to logs.

References