Skip to content
Ask
Overview

inbrowser

The inbrowser monorepo: a resumable job engine, an inference relay, an agent runtime, and an on-device model engine.

Put a language model in a web app without running an inference backend. It runs on the user’s GPU in the browser tab, or proxies to a cloud model, and the token stream survives reloads and dropped connections

@inbrowser is six composable libraries:

  1. @inbrowser/model - Run a language model on the user’s GPU or through a cloud provider, and switch between them with a one-line change.
  2. @inbrowser/agent - Let the model use your tools and take several steps to finish a task, right in the browser.
  3. @inbrowser/resumable - Stream a long answer so a reload, a closed tab, or a dropped connection picks up where it left off instead of starting over.
  4. @inbrowser/relay - Add a thin server when you’d rather keep API keys off the client or share one run across devices.
  5. @inbrowser/workspace - Run files, shell commands, package installs, git, snapshots, and preview compilation in a browser workspace.
  6. @inbrowser/sandbox - Bind workspace capabilities into agent-facing tools, events, checkpoints, and artifacts.

No API Key or BYOK

On-device models don’t need API keys. Most cloud providers use BYOK (bring your own key): each user supplies their own, and it stays on the client instead of on a server you run. Firebase AI Logic instead uses the host app’s configured Firebase project, backend, Authentication, and App Check context.

Run a model in the browser

TS
import { createEngine, createEngineModelClient, smollm2_360m } from '@inbrowser/model/local';

const engine = createEngine({
  ...smollm2_360m,
  onLoadProgress: (p) => {
    if (p.phase === 'fetch') {
      const pct = Math.round((p.loadedBytes / p.totalBytes) * 100);
      console.log(`downloading ${p.file}: ${pct}%`);
    } else {
      console.log(p.phase); // 'init' | 'warmup' | 'ready'
    }
  },
});

await engine.ensureReady(); // downloads ~180 MB once, then caches it

const client = createEngineModelClient(engine);

for await (const ev of client.chat({ 
  messages: [{ 
    role: 'user',
    text: 'Explain quantum tunneling in two sentences.' 
  }], 
  tools: [], 
  toolUseEnabled: false 
}, AbortSignal.timeout(60_000))) {
  if (ev.kind === 'text') console.log(ev.text);
}

The weights download once via Transformers.js and run on WebGPU (WASM fallback when there’s no GPU). createEngineModelClient wraps the engine as a ModelClient — the same interface every cloud provider implements, so swapping it for geminiModelClient({ apiKey, model }), createFirebaseAiLogicModelClient(firebaseModel), or another provider changes that one line and nothing else.

OpenRouter OAuth for BYOK

Cloud models need a key, and there is no server to keep one on, so the user brings it. For OpenRouter that can be a one-click connect instead of a pasted key:

TS
import { beginOpenRouterOAuth, completeOpenRouterOAuth } from '@inbrowser/model/providers/openrouter-oauth';
import { openrouterModelClient } from '@inbrowser/model/providers/openrouter';

// 1. Send the user to OpenRouter to authorize (full-page redirect or popup).
const { authUrl, codeVerifier } = await beginOpenRouterOAuth({ callbackUrl: location.href });
sessionStorage.setItem('openrouter_verifier', codeVerifier);
location.href = authUrl;

// 2. Back on your callback page, exchange the ?code for the user's own key.
const code = new URLSearchParams(location.search).get('code')!;
const { key } = await completeOpenRouterOAuth({
  code,
  codeVerifier: sessionStorage.getItem('openrouter_verifier')!,
});

// The key belongs to the user, provisioned in their browser. Nothing on your server.
const client = openrouterModelClient({ apiKey: key, model: 'anthropic/claude-3.5-sonnet' });

The user clicks Connect OpenRouter and authorizes. Your app receives a key tied to their OpenRouter account, so usage is billed to them and they can revoke it whenever they want. PKCE means there is no client secret, so the exchange runs entirely in the browser.

Give the model tools

TS
import {
  createAgentSession,
  createReactLoopStrategy,
  createToolRegistry,
  createDispatch,
  createMetricsCollector,
} from '@inbrowser/agent';
import { openrouterModelClient } from '@inbrowser/model/providers/openrouter';

// A tool is a plain object: name, description, JSON-schema params, and execute().
const getWeather = {
  name: 'get_weather',
  description: 'Current temperature for a city.',
  parameters: { type: 'object', properties: { city: { type: 'string' } }, required: ['city'] },
  async execute({ city }: { city: string }) {
    const r = await fetch(`https://wttr.in/${encodeURIComponent(city)}?format=%t`);
    return { ok: true, summary: `${city}: ${await r.text()}` };
  },
};

const registry = createToolRegistry();
registry.register(getWeather);

const session = createAgentSession({
  strategy: createReactLoopStrategy(),
  llm: openrouterModelClient({ apiKey: '<BYOK>', model: 'z-ai/glm-5.2' }),
  tools: createDispatch(registry),
  toolList: registry.list(),
  toolContext: () => ({ signal: new AbortController().signal }),
  systemPromptBuilder: () => 'Answer using the tools when they help.',
  metrics: createMetricsCollector(),
  history: [],
});

const events = session.submit('Is it jacket weather in Oslo?', new AbortController().signal);
for await (const event of events) {
  if (event.kind === 'text') console.log(event.chunk);
}

A ReAct loop needs a model that can call tools, so this uses a cloud client. Running on-device? Most small presets can’t drive a tool loop. Pair smollm2_360m with createRetrievalStrategy() to ground answers in your own documents instead, or step up to qwen2_5_0_5b, the smallest preset that advertises tool calling. The presets table below marks which is which.

Keep the stream alive across reloads

TS
import { createJobEngine, createIdbJobStore } from '@inbrowser/resumable';

type Token = { text: string };

// IndexedDB-backed: the event log persists across page reloads.
const engine = createJobEngine<Token>({ 
  store: createIdbJobStore<Token>() 
});

// The producer keeps running even if the tab navigates away.
const { jobId } = await engine.start(async function* () {
  for (const word of ['Durable ', 'by ', 'default.']) yield { text: word };
});

// Reconnect with the last seq you saw — only newer events replay.
for await (const ev of engine.subscribe(jobId, { from: 0 })) {
  if (ev.kind === 'event') console.log(ev.seq, ev.value.text);
  else if (ev.kind === 'terminal') console.log('status:', ev.status);
}

The log is append-only and ordered by seq. A consumer that reconnects after a refresh passes the last seq it saw and gets only what it missed. There are no duplicates, no lost tokens. The store is swappable: createMemoryJobStore() for a single process, createIdbJobStore() for the browser, createRtdbJobStore() to share a job across machines.

Packages

@inbrowser/model

The shared ModelClient contract that relay and agent both consume, cloud provider factories and adapters, and an opt-in on-device engine (Transformers.js + ONNX Runtime Web). Import the contract from @inbrowser/model, providers from @inbrowser/model/providers/<name>, and the on-device runtime from @inbrowser/model/local.

Cloud providers — each returns a ModelClient; most are { apiKey, model, … } factories, while Firebase AI Logic wraps a model constructed by the host app:

FactoryConfigNotes
geminiModelClient(config)GeminiConfigGoogle AI Studio / Vertex
createFirebaseAiLogicModelClient(model, opts?)Constructed Firebase GenerativeModelHost owns Firebase, App Check, backend, and location; no Firebase dependency; not directly relay-registerable
openrouterModelClient(config)OpenRouterConfigUnified API, many models
requestyModelClient(config)RequestyConfigOpenAI-compatible gateway, many models
anthropicModelClient(config)AnthropicConfigAnthropic Claude
openaiCompatModelClient(config)OpenAiCompatConfigAny OpenAI-compatible server
ollamaModelClient(config)OllamaConfigLocal Ollama server
llamaServerModelClient(config)LlamaServerConfigllama.cpp llama-server
claudeCliModelClient(config)ClaudeCliConfigClaude CLI subprocess (Node only)
claudeCodeModelClient(config)ClaudeCodeConfigClaude Code Agent SDK (Node only)

OpenRouter PKCE browser auth lives at @inbrowser/model/providers/openrouter-oauth (beginOpenRouterOAuth / completeOpenRouterOAuth).

On-device engine — the lower-level API under createEngineModelClient. Use it directly when you want raw token events rather than the ModelClient contract:

TS
import { createEngine, smollm2_360m } from '@inbrowser/model/local';

const engine = createEngine({ ...smollm2_360m });
await engine.ensureReady();

for await (const event of engine.generate([{ role: 'user', text: 'Hi' }])) {
  if (event.kind === 'token') console.log(event.text);
}
ExportDescription
createEngine(opts)Creates an Engine that loads ONNX models via Transformers.js
createEngineModelClient(engine)Wraps an Engine as a ModelClient
definePreset(p)Type-safe identity for community presets
parseToolCalls(stream, opts?)Extracts tool calls from an EngineEvent stream
splitThinking(stream, opts?)Separates <think> blocks from output
withRetry(client, opts?)Wraps a ModelClient to retry transient failures
hostEngineInWorker(self, opts?)Hosts an Engine inside a Web Worker
connectWorkerEngine(opts)Connects to a worker-hosted engine

Bundled presets (all q4f16). “Tools” marks presets that advertise native tool calling. The rest can still retrieve and answer, but can’t drive a ReAct loop:

PresetParamsDownloadToolsNotes
smollm2_360m360M~180 MBDefault. Runs on WASM, no GPU required.
qwen2_5_0_5b0.5B~0.5 GBSmallest tool-capable preset.
qwen2_5_coder_1_5b1.5B~1.28 GBCode / fill-in-the-middle. WebGPU only.
qwen3_1_7b1.7B~1.36 GBGeneral, frontier-for-size. WebGPU only.
deepseek_r1_qwen_1_5b1.5B~1.37 GBReasoning model; emits <think> blocks.
gemma4_E2B~2.3B eff.~500 MBAudio-capable. Needs WebGPU.
gemma4_E4B~4.5B eff.~1.5 GBAudio-capable. Needs a discrete GPU.

Author your own with definePreset({ model: { modelId }, dtype, backend, capabilities }).

@inbrowser/resumable

A backend-agnostic resumable streaming-job engine. Producers write typed events into a durable ordered log; subscribers tail it from any offset. Single root entrypoint.

ExportDescription
createJobEngine(opts)Creates a JobEngine<TEvent> with start(), subscribe(), get(), stop()
createMemoryJobStore(opts?)In-process store (ephemeral)
createIdbJobStore(opts?)IndexedDB store (browser-persistent)
createRtdbJobStore(opts)Firebase RTDB store (shared across machines)
connectJobEngine(port)Wraps a MessagePort as a ConnectedJobEngine
hostJobEngine(opts)Hosts a JobEngine in a Worker
sseFromJob(source, opts?)Streams a job subscription as an SSE Response
encodeSseEvent(value)Serializes a value as an SSE data: line
createResumableClient(opts)Environment-agnostic reconnecting HTTP client
installBrowserLifecycle()Returns an abort-on-tab-foreground hook
probeStoreDurability(opts)Verifies events survive engine handoff
probeSweepTtl(opts)Verifies TTL-based cleanup

createJobEngine options:

OptionTypeDefaultDescription
storeJobStore<TEvent>requiredBacking store for events
loggerLoggersilentLoggerDebug/info/warn/error logger
sweepSweepScheduleundefinedExpiry sweep { intervalMs, statusFilter?, onResult? } (store must implement sweepExpired)
now() => numberDate.nowClock for TTL checks

The RTDB store also exports serviceAccountTokenProvider and staticTokenProvider for auth.

@inbrowser/relay

Wires @inbrowser/resumable to the ModelClient contract: routes a provider request, streams the model’s events into the durable log, and serves SSE. Use it when you do want a server in the loop (server-managed keys, shared jobs). Single root entrypoint.

TS
import { createRelay } from '@inbrowser/relay';
import { openrouterModelClient } from '@inbrowser/model/providers/openrouter';
import { createMemoryJobStore } from '@inbrowser/resumable';

const relay = createRelay({
  store: createMemoryJobStore(),
  providers: { openrouter: openrouterModelClient },
});

// Start an inference job.
const res = await relay.handleStart(new Request('http://localhost', {
  method: 'POST',
  body: JSON.stringify({
    provider: 'openrouter',
    model: 'anthropic/claude-3.5-sonnet',
    messages: [{ role: 'user', text: 'Write a haiku' }],
    apiKey: '<BYOK>',
  }),
}));
const { jobId } = await res.json();

// Stream as SSE. Reconnect after a drop by passing the last seq you saw.
const stream = await relay.handleStream(new Request('http://localhost'), { jobId, from: 0 });
ExportDescription
createRelay(opts)Creates a Relay with handleStart, handleStream, engine, stop
createResumableClient(opts)Relay-typed reconnecting client (AsyncIterable<ModelEvent>)
installBrowserLifecycle()Proactive abort on tab-visibility change
createAstroRoutes(relay, opts?){ start, stream } Astro route handlers
createExpressHandlers(relay, opts?){ start, stream } Express-compatible handlers
readSseDataLines(body)SSE line reader (async generator)
encodeSseEvent(event)SSE event serializer

createRelay options:

OptionTypeDescription
storeJobStore<ModelEvent>Backing store for the event log
providersRecord<string, ModelClientFactory>Provider name → factory from @inbrowser/model
loggerLoggerOptional logger
sweepSweepScheduleOptional expiry sweep
apiKeysRecord<string, ApiKeySource>Optional server-managed keys per provider (the browser never carries the key)

@inbrowser/agent

A browser-safe agent runtime plus a Node CLI. The runtime (session, strategies, tools, metrics) is the root entrypoint; Node-only and CLI code live behind /node and /cli subpaths so they never reach the browser bundle. See “Give the model tools” above for a full session.

ExportEntryDescription
createAgentSession(config).Creates an AgentSession with submit(prompt, signal), cancel(), id
createReactLoopStrategy(opts?).ReAct multi-tool loop (needs a tool-capable model)
createRetrievalStrategy(opts?).Retrieve-then-read RAG strategy (works with small on-device models)
createPlannerExecutorStrategy(opts?).Skill-catalog planner-executor strategy
createToolRegistry().In-memory registry: register, replace, unregister, list, has, fork
createDispatch(registry).Stateless dispatch with execute(call, ctx) (call is a ToolCall)
createMemoizedDispatch(dispatch, opts?).Content-addressed memoized dispatch
createMetricsCollector().Token/cost collector: recordTurn, totals, reset
computeTurnMetrics / findPricing.Standalone turn-metric and pricing helpers
createMemoryStorage() / createLocalStorageAdapter() / noopStorage.Storage implementations
noopObserver / combineObservers(...).SandboxObserver helpers
wrapMutating(handler, opts).Wraps a handler so mutations are logged for undo/replay
replayEvents(opts).Replays logged mutations against a dispatch
isWrappedHandler(handler).Checks the WRAPPED_MARKER symbol
SKILL_CATALOG / routeSkill(prompt, options?).Skill catalog and routing (catalog is a field on options)
createSpecRegistry() / evaluateSpec().Eval harness
openEventLog(projectId, opts?)/nodeNDJSON append-only event log
connectMcpTools(opts)/nodeMCP client tools
main(opts?)/cliCLI entry point
CLI_SPEC / parseArgs(argv, cwd)/cliCLI schema and parser

CLI commands (agent):

CommandDescription
runHeadless single session. Prompt via positional arg or --json - stdin.
fleetRun N isolated sessions in parallel
describeMachine-readable descriptions of commands, scenarios, events
schemaDump full CLI schema as JSON
eventsStream the per-project mutation event log with filters
undoReverse a previously-committed mutation via recorded reverseOp
migratePlan forward replay of a project event log
serveInverse-mode MCP server over stdio
versionPrint package version
helpShow usage

Any command emits structured output with the global --output json (-o json) flag, or when piped to a non-TTY.

Session events (SessionEvent kind): turn_started, text, thinking, tool_started, tool_finished, workspace_changed, runtime_changed, turn_completed, error, completed, strategy_event.

Installation

Packages are published independently. Install what you need:

TERMINAL
bun add @inbrowser/resumable    # resumable streaming-job engine
bun add @inbrowser/model        # model contract; providers via /providers/<name>; on-device via /local
bun add @inbrowser/relay        # LLM relay (depends on resumable + model)
bun add @inbrowser/agent        # agent runtime + CLI
bun add @inbrowser/workspace    # browser workspace: files, shell, preview, git
bun add @inbrowser/sandbox      # tools, events, checkpoints, artifacts

The opt-in @inbrowser/model/local engine needs Transformers.js as an optional peer dependency. Install it only in applications that run models on-device:

TERMINAL
bun add @huggingface/transformers

Development (this monorepo)

TERMINAL
bun install
bun run build          # builds all packages in dependency order
bun run typecheck      # type-checks all packages
bun run test           # runs all package tests
bun run check          # biome lint + format

Filter to one workspace:

TERMINAL
bun --filter '@inbrowser/agent' run test

Examples

ExampleWhat it demonstrates
examples/model-basicScript-only model helpers: thinking splitting, tool-call parsing, usage normalization
examples/agent-basicScript-only agent session with a fake model, a real tool registry, ReAct events, and workspace mutation
examples/resumable-basicScript-only resumable job flow: start, subscribe, resume from offset, inspect final snapshot
examples/relay-basicScript-only relay flow: fake provider, memory-backed job, SSE stream, reconnect from offset
examples/workspace-basicScript-only workspace flow: files, shell, snapshots, git, and React preview compilation
examples/sandbox-basicScript-only sandbox flow: standard tools, chronological events, checkpoints, restore
examples/workspace-browserBrowser IDE-style workspace demo for files, preview compilation, terminal, packages, git, snapshots, and events
examples/sandbox-browserBrowser sandbox manager for tools, events, checkpoints, files, shell, and preview
examples/local-llm-pocOn-device model in the browser: preset selection, load progress, WebGPU/WASM
examples/resumable-hono-youtube-briefcastMulti-step media workflow using @inbrowser/resumable’s durable log on a Hono server

Dependency graph

@inbrowser/resumable    no internal deps
@inbrowser/model        no internal deps
@inbrowser/relay        depends on resumable + model
@inbrowser/workspace    no internal deps
@inbrowser/sandbox      depends on workspace
@inbrowser/agent        depends on model + sandbox

Status

Pre-1.0. Versions are coordinated manually:

PackageVersion
@inbrowser/resumable0.4.0
@inbrowser/model0.4.0
@inbrowser/relay0.4.0
@inbrowser/workspace0.4.0
@inbrowser/sandbox0.4.0
@inbrowser/agent0.4.0

Breaking changes are expected until 1.0.