API Reference

This page describes the public surface of @inbrowser/relay.

Exports

The relay does not export any providers. Import the cloud provider factories (geminiModelClient, openrouterModelClient, anthropicModelClient, ollamaModelClient, claudeCliModelClient, claudeCodeModelClient) from @inbrowser/model.

createRelay

function createRelay(opts: CreateRelayOpts): Relay;

CreateRelayOpts:

Relay:

handleStart returns:

handleStream returns:

NormalizedRequest

NormalizedRequest is the shared ModelRequest (from @inbrowser/model) plus the relay-only transport fields:

type NormalizedRequest = ModelRequest & {
  // ModelRequest carries: messages: ModelMessage[]; tools: ToolSpec[];
  // toolUseEnabled: boolean; temperature?; topP?; topK?; reasoningEffort?
  provider: string; // routing key — looked up in createRelay's providers map
  model: string; // upstream model id, passed to the ModelClientFactory
  apiKey?: string;
  signal?: AbortSignal; // page-direct consumer cancellation only
};

provider is the lookup key in the providers map. model and apiKey are handed to the ModelClientFactory (factory({ apiKey, model })); the per-call settings (messages, tools, sampling) ride the ModelRequest into .chat(). apiKey is not stored in job metadata by the relay. It is optional on the wire because the relay resolves it differently per mode (see below); by the time a provider’s ModelClient runs, the relay has guaranteed a resolved value.

Server-managed API keys

By default the relay is BYOK: the client sends apiKey in the request body and the relay 400s if it is missing. To keep the key on the server instead, list the provider in CreateRelayOpts.apiKeys:

import { geminiModelClient } from '@inbrowser/model/providers/gemini';
import { anthropicModelClient } from '@inbrowser/model/providers/anthropic';
import { ollamaModelClient } from '@inbrowser/model/providers/ollama';

type ApiKeySource =
  | string
  | ((ctx: { req: NormalizedRequest; request: Request }) => string | Promise<string>);

const relay = createRelay({
  store,
  providers: {
    gemini: geminiModelClient,
    anthropic: anthropicModelClient,
    ollama: ollamaModelClient,
  },
  apiKeys: {
    gemini: () => process.env.GEMINI_API_KEY ?? '',
    anthropic: () => process.env.ANTHROPIC_API_KEY ?? '',
    // ollama omitted, so it stays BYOK (the client supplies its base URL)
  },
});

Rules per provider:

• Listed (server-managed): the relay resolves the key and overwrites whatever the client sent. A client that sends a non-empty apiKey anyway gets a 400 so a forgotten BYOK field cannot silently leak to the wire. If the resolver throws, handleStart returns 500 and no job is created.
• Not listed (BYOK): unchanged. The client supplies apiKey; a missing key is a 400.

The function form receives the raw Request, so the key can be derived from an Authorization header, a session cookie, or a per-user store:

apiKeys: {
  anthropic: async ({ request }) => {
    const userId = await getUserIdFromSession(request);
    const key = await db.getUserKey(userId, 'anthropic');
    if (!key) throw new Error('no anthropic key for user');
    return key;
  },
}

ModelEvent

The relay’s event type is ModelEvent from @inbrowser/model, re-exported from @inbrowser/relay:

type ModelEvent =
  | { kind: 'text'; text: string }
  | { kind: 'thinking'; text: string }
  | { kind: 'tool_call'; id: string; name: string; args: unknown; signature?: string }
  | { kind: 'usage'; usage: ModelUsage }
  | { kind: 'error'; message: string };

interface ModelUsage {
  promptTokens: number;
  outputTokens: number;
  cachedTokens?: number;
  reasoningTokens?: number;
  costUsd?: number;
}

The turn ends when the chat() iterable returns. On a normal end a usage event is emitted before the return (it carries the final accounting); there is no separate turn_complete event. An error event is itself terminal: after it the iterable returns with no usage event. Consumers can rely on exactly one of {a usage event, an error event} per turn.

Provider contract: ModelClientFactory

A provider is a ModelClient from @inbrowser/model, registered as a ModelClientFactory:

interface ModelClient {
  readonly id: string;
  readonly supportsTools: boolean;
  chat(req: ModelRequest, signal: AbortSignal): AsyncIterable<ModelEvent>;
}

// The shape createRelay's `providers` map holds (from @inbrowser/model):
type ModelClientFactory = (config: { apiKey?: string; model: string }) => ModelClient;

The relay calls the factory per request with { apiKey, model } (so BYOK per-request keys and routing both work), then drives .chat(req, signal) under @inbrowser/resumable. The ModelClient owns upstream protocol details only.

Built-In Providers

The relay ships none. The cloud provider factories live in @inbrowser/model and are imported from @inbrowser/model. Each is a factory that returns a ModelClient; the cloud ones match ModelClientFactory directly (config { apiKey?, model }).

The two Claude providers authorize from a logged-in subscription instead of an API key, so a client sends an empty apiKey. See How to use a subscription Claude provider.

SSE Wire Format

Relay-to-client events are single-line SSE data events:

data: {"kind":"text","text":"hello"}

data: [DONE]

SSE_STREAM_OPEN is emitted first as an SSE comment:

: stream-open

[DONE] is emitted only when the job reaches terminal state. A connection that closes without [DONE] means the client should reconnect from its current offset.

Client

function createResumableClient(opts: ResumableClientOpts): ResumableClient;

ResumableClientOpts:

client.stream(req) starts a job and yields ModelEvents until the relay emits [DONE], an unrecoverable error occurs, or the caller aborts.

Adapters

createAstroRoutes(relay, opts?) returns { start, stream } APIRoute-like handlers. opts.jobIdParam defaults to id.

createExpressHandlers(relay, opts?) returns { start, stream } Express-compatible handlers. Options: