Tutorial

Tutorial: Create A Relay With A Fake Provider

Build a relay over a memory store and a fake provider, start a job, stream it, and resume from an offset.

This tutorial creates a relay that streams fake inference events through the same start and SSE flow used by real providers. No API keys or external services are required.

1. Create The Relay

import { createMemoryJobStore } from '@inbrowser/resumable';
import { createRelay, type ModelEvent } from '@inbrowser/relay';
import type { ModelClientFactory } from '@inbrowser/model';

// A provider is a `ModelClientFactory`: the relay calls it with
// `{ apiKey, model }` per request to build a `ModelClient`, then drives
// its `.chat(req, signal)`.
const fakeProvider: ModelClientFactory = ({ model }) => ({
  id: `fake:${model}`,
  supportsTools: false,
  async *chat() {
    yield { kind: 'text', text: `hello from fake/${model}` };
    yield { kind: 'thinking', text: 'checking the durable log' };
    yield {
      kind: 'usage',
      usage: { promptTokens: 4, outputTokens: 8 },
    };
  },
});

const relay = createRelay({
  store: createMemoryJobStore<ModelEvent>(),
  providers: {
    fake: fakeProvider,
  },
});

The factory returns a ModelClient whose chat() is an async iterable. The relay constructs one per request, runs it under a resumable job engine, and stores every ModelEvent it yields. The turn ends when the iterable returns; a usage event carries the final accounting (there is no turn_complete event).

2. Start A Job

const startResponse = await relay.handleStart(
  new Request('http://localhost/api/inference/job', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      provider: 'fake',
      model: 'demo-model',
      messages: [{ role: 'user', text: 'Say hello' }],
      tools: [],
      apiKey: 'demo-key',
    }),
  }),
);

const { jobId } = (await startResponse.json()) as { jobId: string };
console.log(jobId);

The response status is 201. The returned jobId identifies the durable event log for this generation.

3. Stream The Job

const streamResponse = await relay.handleStream(
  new Request(`http://localhost/api/inference/job/${jobId}/stream`),
  { jobId },
);

console.log(await streamResponse.text());

The stream is SSE text:

TEXT

: stream-open

data: {"kind":"text","text":"hello from fake/demo-model"}

data: {"kind":"thinking","text":"checking the durable log"}

data: {"kind":"usage","usage":{"promptTokens":4,"outputTokens":8}}

data: [DONE]

[DONE] appears only when the job reaches terminal state.

4. Resume From An Offset

Stream again from event 2:

const resumed = await relay.handleStream(
  new Request(`http://localhost/api/inference/job/${jobId}/stream?from=2`),
  { jobId },
);

console.log(await resumed.text());

The relay skips events 0 and 1, then returns the usage event and [DONE]. That is the same replay rule used by createResumableClient when a browser connection drops.

5. Stop The Relay

await relay.stop();

You now have the full relay shape: start a job, stream the event log, and resume from the next event the client needs.