影 kaged
← back to Architecture Decision Records Proposed

ADR-0052: Message regeneration and two-tier, operator-cancellable retry

  • Status: Proposed
  • Date: 2026-07-01
  • Deciders: @karasu, with colleagues
  • Supersedes: —
  • Superseded by: —

Context

When a primary run fails on a transient provider error (e.g. "The service may be temporarily overloaded, please try again later" — an upstream 429/503), the run transitions to failed, the session returns to idle, and the failing assistant message is persisted with an error_message. The operator sees the error but has no path back to a successful reply short of retyping the prompt.

Two capabilities are wanted:

  1. Regenerate — re-run the agent from a chosen point. Clicking regenerate on an agent reply supersedes that reply and everything after it, then re-runs from the operator message that produced it.
  2. Retry mode — on a transient failure, automatically regenerate on a visible, cancellable backoff until it succeeds or the operator cancels.

The load-bearing question is where retry lives and how many tiers there are. docs/specs/llm.md §Retry policy already specifies a retry middleware in every resolved model (max 3 attempts, base 1 s, cap 30 s, jitter, retries 429/5xx + network errors, honors Retry-After). In code the middleware (wrapWithRetry, packages/llm/src/middleware/retry.ts) is implemented and unit-tested but not wired into resolveModel — the code lags its own spec. Meanwhile the daemon already performs exactly one context_overflow retry (compact then re-run, primary-runner.ts, per ADR-0024).

So there are really two natural retry tiers, and the operator has asked that both be cancellable and operator-legible — including the case where a provider says "no quota for 50 minutes": the operator must be able to cancel and come back rather than have the system hammer the endpoint, and a long Retry-After should inform (not be ignored by) the next attempt.

Decision

kaged will implement two explicit retry tiers with a hard boundary, both operator-cancellable, and will wire the specced retry middleware into resolveModel to make Tier 1 real.

  • Tier 1 — fast, automatic, in-run (transport). The wrapWithRetry middleware, wired into resolveModel per llm.md. It retries the open-call (doStream/doGenerate setup) only — never after the first delta has been published — so live streaming is never replayed. It uses the run's AbortSignal, so cancelling the run aborts an in-flight Tier 1 retry sleep. Bounded (3 attempts, 30 s cap). If a Retry-After exceeds Tier 1's window, Tier 1 does not burn attempts waiting; it stops and hands an absolute next-eligible time to Tier 2.

  • Tier 2 — slow, visible, operator-driven (scheduler). The frontend. After a run fails, the UI arms only from a persisted, normalized retry classification on the failed run (not from a coarse run_failed alone). When armed, it shows a countdown (base 2 s, ×2, cap 300 s, unlimited) — or, when the failed run carries a far-future retry_after_until, a scheduled "retry after HH:MM" with the countdown paused — and on expiry calls POST /api/v1/sessions/:id/messages/:mid/regenerate. The operator can cancel at any point between attempts.

  • Regenerate endpoint — supersedes all messages after the operator message :mid and dispatches a fresh run from that message's content, reusing the ordinary session concurrency/transition logic (it may queue like a posted message). It creates no new message row and does not mutate the preserved operator row (message immutability intact); it returns a receipt.

  • Retry classification (the tier boundary). A failed run persists a normalized { retryable: boolean, retry_class: string, retry_after_until?: number }. Tier 2 auto-arms only when retryable is true for a transient/network/rate-limit class. It must not auto-arm for context_overflow (daemon already compaction-retried), auth_failed, credentials_unresolved, spend_limit_exceeded, or hard-quota failures. The message wire exposes run_id (for correlation) and this classification (for arming).

  • Anchor. Regeneration anchors on the operator message id (stable across regenerations), never run_id (which changes each regeneration).

  • Cancellation semantics. Tier 1 cancel = abort the run (existing POST .../runs/:rid/cancel), which aborts the in-flight retry sleep. Tier 2 cancel = a frontend timer stop (no server run exists during the countdown), modeled as an armed → dispatching → cancelled state machine with a generation token and a fetch AbortController; client-only cancel is best-effort (no server lease), with an immediate run.cancel fallback if a run was already dispatched. Prerequisite fix: a cancelled run must be recorded as cancelled, not failed — today primary-runner.ts can map an aborted finishReason to failed, which would wrongly arm Tier 2 on a cancel.

Consequences

What this commits us to

  • Wiring wrapWithRetry into resolveModel (closing the code/spec gap), with the explicit invariant: no automatic in-run retry once the first delta is published.
  • A normalized retry classification persisted on failed runs and exposed on the message wire, plus run_id.
  • Fixing the cancel-vs-failed outcome mismatch in primary-runner.ts before Tier 2 can trust the failed-run signal.
  • Persisting an absolute retry_after_until when a provider advises a backoff longer than Tier 1's window, so Tier 2 respects it across a fresh run.

What this forecloses

  • Retries that survive a closed browser tab (Tier 2 is interactive-only — accepted).
  • A strict "no stray run on cancel" guarantee, unless a future server-side retry lease is added (explicitly out of scope; best-effort cancel accepted).

What becomes easier

  • Transient blips recover automatically and invisibly (Tier 1); persistent transient failure becomes a visible, cancellable, honest wait (Tier 2).
  • One code path backs both regenerate and Tier 2 retry.

What becomes harder

  • Two retry mechanisms plus compaction-retry coexist; the boundaries are drawn on the persisted retry classification and the first-delta invariant, and must be respected on both sides.

Alternatives considered

Alternative A — single frontend-only tier, remove middleware retry

The original framing. Rejected: it contradicts llm.md (higher authority than a plan), throws away automatic fast recovery, and would require amending ADR-0049's retry stance.

Alternative B — daemon-only retry (wrapWithRetry, no visible tier)

Survives tab close, needs no UI. Rejected: hides the transient error the operator wants to see, gives no countdown/cancel, and cannot express a long cooldown to the operator.

Alternative C — anchor on run_id

Rejected: a reply's run id changes on every regeneration while the operator row persists; run id is not a stable anchor.

Alternative D — arm Tier 2 on run_failed alone (no classification)

Simpler, but retries things it must not (auth, hard quota, context overflow). Rejected in favor of an explicit normalized classification.

References

  • Spec amendments: docs/specs/http-api.md, docs/specs/session-manager.md, docs/specs/llm.md, docs/specs/ui/README.md.
  • ADR-0016 (streaming-first UI), ADR-0024 (context compaction / the context_overflow retry excluded from Tier 2), ADR-0026 (spend limits — a non-retryable class), ADR-0049 (@kaged/llm resolver + middleware; the retry middleware this ADR wires in), ADR-0039 (@kaged/wire).
  • Design consult (colleagues): two-tier boundary, first-delta invariant, retry_after_until handoff, and the cancel-vs-failed prerequisite fix.