影 kaged
← back to Architecture Decision Records Accepted

ADR-0014: All LLM providers route through @kaged/llm; Mastra integrates via a LanguageModelV2 shim

  • Status: Accepted
  • Date: 2026-05-23
  • Deciders: @karasu
  • Supersedes: —
  • Superseded by: —

Context

ADR-0012 committed kaged to Mastra v1.x as the agentic substrate. Mastra's Agent.model field accepts any object implementing Vercel AI SDK's LanguageModelV2 interface — in practice, @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, and friends.

In parallel, @kaged/llm was built (see docs/specs/llm.md) as a pure-fetch, no-SDK provider interface. Four API shapes are implemented end-to-end against the pi-ai reference, with a 123-test suite that exercises Anthropic Messages, OpenAI Chat Completions, OpenAI Responses, and Google Generative AI wire protocols. The provider test endpoint (POST /api/v1/local/providers/:name/test) already exercises the full path under Bun.

When wiring handlePostMessage through Mastra, the question is which provider implementation Mastra's Agent.model field receives:

  1. Mastra's built-in providers — import { anthropic } from '@ai-sdk/anthropic', pass anthropic('claude-sonnet-4-...') as model. @kaged/llm retains a small role for the provider test endpoint and any out-of-loop calls.
  2. An adapter that wraps @kaged/llm — implement LanguageModelV2 once in @kaged/llm, expose it as a factory the harness uses for model. All provider calls — agent loop, test endpoint, anything else — route through the same code path.

Two facts make this load-bearing rather than a style choice:

  • OAuth / subscription routes. A class of providers exists that Vercel and Mastra deliberately don't ship @ai-sdk/* packages for: OAuth-based access to consumer subscriptions (Claude Pro, ChatGPT Plus, etc.). Programmatic use of these subscriptions is in a TOS gray area that Vercel and Mastra (corporate, well-known, lawyer-served) choose not to step into. kaged is operator-owned, self-hosted infrastructure. Whether to use an OAuth-backed personal subscription with kaged is a decision the operator makes about their own account and their own provider relationship — kaged's role is to provide the capability and let the operator own the call. If we route the agent loop through @ai-sdk/* we cannot offer OAuth providers in the loop at all. If we route through @kaged/llm we can.
  • Provider control surface. We already built the wire-protocol implementation against the pi-ai reference. Going through @ai-sdk/* re-introduces a framework layer between kaged and the HTTP request. Inline custom headers, per-call retry policy, OAuth refresh interleaved with the stream, kaged-shaped telemetry, ARM64 dep hygiene — all are easier when our own code is on both sides of the boundary.

A unified provider path is also more aligned with the manifesto's transparency principle: an operator can read @kaged/llm's source and know exactly what's on the wire. A framework-mediated @ai-sdk/anthropic is opaque by comparison.

Decision

All LLM provider calls — agent-loop, provider test, future direct calls — route through @kaged/llm. The package exposes a LanguageModelV2 factory (Vercel AI SDK provider interface v2, the version Mastra v1.x consumes) that the harness uses as the model field on every Mastra Agent. kaged does not depend on any @ai-sdk/<provider> package. The only Vercel-published dependency is @ai-sdk/provider-v5 (or whichever package publishes the LanguageModelV2 interface types Mastra v1.x targets), pinned to the version Mastra targets.

What this commits us to

  • A LanguageModelV2-shaped factory in @kaged/llm (working name: kagedModel(route, options) → LanguageModelV2).
  • A mapping layer between kaged's StreamEvent shape and Vercel's LanguageModelV2StreamPart shape. Bidirectional: doStream consumes Vercel-shaped LanguageModelV2CallOptions and emits Vercel-shaped stream parts.
  • Tracking the LanguageModelV2 spec version that Mastra v1.x consumes. If Mastra moves to a hypothetical V3 in a future minor version, our shim updates once.
  • Tracking provider API changes ourselves (Anthropic ships a new Messages API revision, OpenAI changes Responses-API shapes, etc.). @kaged/llm already does this for its four adapters; the burden is unchanged.

What this forecloses

  • The convenience of import { anthropic } from '@ai-sdk/anthropic' and getting upstream provider-shape changes automatically. We pay attention to provider release notes ourselves.
  • A future contributor casually adding "just one more @ai-sdk/<x>" — the dependency is excluded by policy and the policy lives here.

What becomes easier

  • One provider layer. Every LLM call — agent-loop, provider test, ad-hoc — uses the same code, the same logging, the same retry policy, the same auth resolution.
  • OAuth providers. @kaged/llm is operator-owned; we can ship adapters for OAuth/subscription routes Vercel won't.
  • Inline control. Per-call header injection, OAuth refresh interleaved with streaming, custom telemetry hooks, fallback chains — all in one package we own.
  • Operator transparency. Reading @kaged/llm source tells the operator exactly what bytes hit the provider. No framework opacity.
  • Bun hygiene. @kaged/llm is pure-fetch Bun-native code. @ai-sdk/* packages assume Node and pull in Node polyfills that are friction under Bun (ADR-0004).
  • Smaller dep tree. One Vercel-published type-only package (@ai-sdk/provider-v5) plus @mastra/core, instead of @mastra/core + N provider packages.

What becomes harder

  • We maintain the shim. Mapping LanguageModelV2CallOptions → Context + StreamOptions and StreamEvent → LanguageModelV2StreamPart is ours to keep correct.
  • When Mastra bumps the provider interface version (V2 → V3 in some future release), our shim updates. The blast radius is one file in @kaged/llm.
  • Provider API drift is on us. We monitor upstream changes for Anthropic, OpenAI, Google, etc. This is already true for the four @kaged/llm adapters; this ADR doesn't change the burden, it commits us to keeping it.

Vendor risk

The vendor we'd worry about under the alternative — Vercel, publishing @ai-sdk/* packages — is absent from our dep graph under this decision. The only Vercel-published surface we depend on is the LanguageModelV2 interface type, which is a TS shape, not runtime code. If Vercel changed direction tomorrow, our shim continues to work as long as Mastra continues to consume LanguageModelV2. If Mastra moves to a new interface version, we update one file.

The vendor risk that remains — Mastra itself — is unchanged from ADR-0012. This decision does not deepen it.

OAuth deferral, not OAuth implementation

This ADR establishes the path for OAuth providers. It does not implement them. v0 ships with API-key providers only (as already speced in local-config.md). OAuth provider adapters, the credential-refresh flow, and operator UX for "I understand this uses my personal subscription" land in a follow-up — likely a separate ADR plus a local-config.md amendment, when the work is scheduled.

The operator-owns-TOS-choice stance is normative now, so the architecture doesn't have to be revisited when OAuth lands.

Verification

The core claim — that @kaged/llm's pure-fetch adapters work end-to-end under Bun against real providers — is already verified by:

  • 123 passing tests in packages/llm/__tests__/ covering all four API shapes (anthropic-messages, openai-completions, openai-responses, google-generative-ai).
  • The POST /api/v1/local/providers/:name/test endpoint, which exercises the full streamModel/completeModel path against real providers when the operator configures credentials.

The remaining work this ADR commits us to — the LanguageModelV2 shim — is verifiable by:

  • Unit tests in packages/llm/__tests__/ that construct a LanguageModelV2 instance, feed it canned LanguageModelV2CallOptions, assert the resulting Vercel-shape stream parts match expectations.
  • An integration test in packages/harness/__tests__/ that constructs a Mastra Agent with a kaged-backed model, calls agent.stream(...), and asserts the full output matches what @kaged/llm produced.

Alternatives considered

Alternative A — Mastra's built-in @ai-sdk/* providers for the agent loop; @kaged/llm only for the test endpoint and any OAuth escape

Why tempting: Fewer lines of code in kaged. @ai-sdk/* packages track upstream provider changes for us. Each provider is a well-tested package.

Why rejected: Two provider paths to maintain. Different logging, different retry semantics, different auth resolution between the agent loop and the test endpoint. The OAuth use case introduces a kaged path anyway, so we're carrying that code regardless. Less operator transparency. Pulls in N npm packages where one shim suffices. Vercel's SDK assumes Node; under Bun we keep tripping over the assumptions.

Alternative B — This decision

Why chosen: Unified provider layer. OAuth supported by design, not as a special case. Full inline control over every call. Smaller dep tree. Operator-readable transport. Bun-native throughout. The shim is the only new code, and it's smaller than the test suite that already verifies the adapters underneath it.

Alternative C — Skip Mastra; build the agent loop on top of @kaged/llm directly

Why tempting: Total control. No LanguageModelV2 shim. One less framework to learn.

Why rejected: Contradicts ADR-0012. Mastra is the agentic substrate decision; this ADR is about which provider implementation Mastra uses, not whether to use Mastra. The agent loop — tool dispatch, supervisor/sub-agent topology, suspend/resume checkpoints, Processors — is the part Mastra is genuinely better-engineered to provide than a hand-rolled equivalent would be in any reasonable timeframe. The provider layer is the part we already built and want to keep.

References

Amendments

(none yet)