Engine Reference

This page describes the opt-in @inbrowser/model/local export: the engine factory, the Engine surface, the event vocabulary, and the stream transformers.

For preset data and the shared static types they carry, see ./presets.md. For the worker helpers, see ./adapters-and-worker.md.

This page covers the on-device engine. The same package also owns the cloud providers and the shared ModelClient contract — see the package README and contract source for those.

Exports

Everything on this page is imported from @inbrowser/model/local.

The removed @inbrowser/model/relay and @inbrowser/model/agent adapter subpaths are gone. The engine is now a ModelClient via createEngineModelClient (from @inbrowser/model/local). See ./adapters-and-worker.md.

createEngine

function createEngine(opts: CreateEngineOpts): Engine;

Constructs an Engine bound to a single model. Weight loading is deferred until ensureReady() or the first generate() call. Spread a ModelPreset into the call along with optional EngineHooks.

import { createEngine, gemma4_E2B } from '@inbrowser/model/local';

const engine = createEngine({ ...gemma4_E2B, onLoadProgress: console.log });

CreateEngineOpts is ModelPreset & EngineHooks.

ModelPreset

A fully-specified model configuration.

EngineHooks

Non-preset construction options.

ModelRef

HF Hub repo id with an optional pinned revision.

Dtype

type Dtype = 'q4f16' | 'q8' | 'fp16' | 'fp32';

Backend

type Backend = 'auto' | 'webgpu' | 'wasm';

EngineCapabilities

Static capability declaration carried on ModelPreset.capabilities.

Engine

The runtime engine. One engine per model instance.

EngineState

type EngineState = 'idle' | 'loading' | 'ready' | 'error' | 'disposed';

EngineEventMap

The events engine.on() subscribes to.

LoadProgress

Progress for the observable phases of cold start.

type LoadProgress =
  | { phase: 'fetch'; file: string; loadedBytes: number; totalBytes: number }
  | { phase: 'init'; backend: Backend }
  | { phase: 'warmup'; tokensGenerated: number }
  | { phase: 'ready' };

generate

generate(
  messages: ReadonlyArray<EngineMessage>,
  opts?: GenerateOpts,
): AsyncIterable<EngineEvent>;

Applies the model’s chat template to messages, drives the decode loop, and yields EngineEvents. The terminal event is usage on success, or error on failure.

for await (const evt of engine.generate([{ role: 'user', text: 'Hello' }])) {
  if (evt.kind === 'token') process.stdout.write(evt.text);
}

EngineMessage

Engine-side chat message. The engine is toolless: there is no tool role.

MediaPart

type MediaPart =
  | { kind: 'image'; data: Blob | ArrayBuffer; mimeType: string }
  | { kind: 'audio'; data: Blob | ArrayBuffer; mimeType: string };

GenerateOpts

ToolSpec

Tool declaration matching the OpenAI function-calling format.

interface ToolSpec {
  type: 'function';
  function: {
    name: string;
    description: string;
    parameters: unknown;
  };
}

EngineEvent

The engine’s narrow event vocabulary.

type EngineEvent =
  | { kind: 'token'; text: string }
  | { kind: 'thinking'; text: string }
  | { kind: 'tool_call'; id: string; name: string; args: unknown }
  | { kind: 'usage'; promptTokens: number; outputTokens: number; decodeMs: number }
  | { kind: 'error'; message: string; recoverable: boolean };

definePreset

function definePreset<P extends ModelPreset>(p: P): P;

Compile-time identity helper. At runtime it returns its argument unchanged; its value is the completeness check it enforces on caller-defined presets. Used to author both the bundled presets and community presets.

import { definePreset } from '@inbrowser/model/local';

export const myPreset = definePreset({
  model: { modelId: 'org/model-ONNX' },
  dtype: 'q4f16',
  backend: 'auto',
  capabilities: {
    supportsTools: false,
    supportsVision: false,
    supportsAudio: false,
    contextWindow: 8_192,
    supportsThinking: false,
  },
});

Stream transformers

The engine emits only token, usage, and error events. Two transformers wrap an AsyncIterable<EngineEvent> and re-emit the same shape with additional tool_call or thinking events surfaced. generate() applies parseToolCalls internally when tools are passed to a tools-capable preset; splitThinking is applied by the consumer.

parseToolCalls

function parseToolCalls(
  source: AsyncIterable<EngineEvent>,
  opts?: ToolCallParseOpts,
): AsyncIterable<EngineEvent>;

Detects native tool-call envelopes in the token stream and re-emits them as tool_call events. thinking, usage, and error events forward unchanged. token events outside an envelope forward as token; inside an envelope they are buffered and converted to a single tool_call on close.

ToolCallParseOpts:

splitThinking

function splitThinking(
  source: AsyncIterable<EngineEvent>,
  opts?: ThinkingSplitOpts,
): AsyncIterable<EngineEvent>;

Splits reasoning-tagged content out of the token stream, re-emitting text inside the tags as thinking events. usage and error events forward unchanged.

for await (const evt of splitThinking(engine.generate(msgs))) {
  if (evt.kind === 'thinking') showReasoning(evt.text);
  else if (evt.kind === 'token') showOutput(evt.text);
}

ThinkingSplitOpts:

A preset’s capabilities.thinkingTags is shape-compatible with ThinkingSplitOpts, so it can be spread directly: splitThinking(stream, preset.capabilities.thinkingTags).

Related

• Tutorial: ../tutorials/01-run-a-model-in-the-browser.md
• Design discussion: ../explanation/design.md
• Presets: ./presets.md
• Adapters and worker: ./adapters-and-worker.md