Stream Responses in Your App

Streaming works exactly like OpenAI’s streaming API - set stream: true and read server-sent events. This guide covers the actual UI pattern: showing tokens as they arrive, then revealing which model answered and what it cost, once the answer is complete.

Basic Streaming

const stream = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Write a short poem about routing." }],
  stream: true,
});

for await (const chunk of stream) {
  const token = chunk.choices[0]?.delta?.content;
  if (token) process.stdout.write(token);
}

This is identical to streaming against OpenAI directly - nothing Routor-specific required to get tokens flowing.

Getting the Routing Decision Mid-Stream

A streamed response doesn’t have a routor field on each chunk the way a non-streaming response has one on the final object - the routing decision only becomes final once the full answer is generated (final token usage isn’t known until the stream ends). To get it, opt in with routor_events: true in the request body:

{
  "model": "auto",
  "messages": [{ "role": "user", "content": "..." }],
  "stream": true,
  "routor_events": true
}

With that flag set, Routor writes one extra SSE event after the last content chunk and before [DONE], carrying the full decision:

data: {"choices":[{"delta":{"content":"outing"}}]}

data: {"routor":{"model":"google/gemini-3.5-flash","tier":"SIMPLE","category":"writing","savingsPct":91.2,"qualityPct":-2.4,"promptTokens":12,"completionTokens":34,"totalTokens":46,"finishReason":"stop"}}

data: [DONE]

In your stream-reading loop, check for a routor key instead of choices to catch this event:

const stream = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: prompt }],
  stream: true,
  // @ts-expect-error — routor_events is a Routor-specific extension field
  routor_events: true,
});

let answer = "";
for await (const chunk of stream) {
  const anyChunk = chunk as any;
  if (anyChunk.routor) {
    console.log(`\n[${anyChunk.routor.model} · ${anyChunk.routor.tier} · ${anyChunk.routor.savingsPct.toFixed(1)}% cheaper]`);
    continue;
  }
  const token = anyChunk.choices?.[0]?.delta?.content;
  if (token) { answer += token; process.stdout.write(token); }
}

(The @ts-expect-error/as any casts are only needed because routor_events and the routor event aren’t part of the official OpenAI SDK’s types - the request and response both work fine at runtime without them if you’re not using TypeScript, or if you’re building the request/parsing the stream by hand instead of through the SDK.)

Building the UI Pattern

The natural UI is: show tokens as they stream in, then reveal a small routing badge once the routor event arrives - this is exactly what the dashboard’s Chat page and the Try Routor app both do. Rough shape:

Render the assistant bubble as empty, tokens appended as they arrive
When a routor event lands, stop appending text and render the routing summary (model, tier, savings %) either above or below the finished answer
finishReason on that event tells you whether the answer completed normally ("stop") or was cut off by a token limit ("length") - worth showing a “response truncated” note in the second case

Where to Go Next

Full request/response reference - every request field, non-streaming included
Build your first app - the non-streaming starting point this guide builds on

Getting Started

Core Concepts

Dashboard

Playground

Providers

Guides

Stream Responses in Your App

Basic Streaming

Getting the Routing Decision Mid-Stream

Building the UI Pattern

Where to Go Next

​Basic Streaming

​Getting the Routing Decision Mid-Stream

​Building the UI Pattern

​Where to Go Next

Basic Streaming

Getting the Routing Decision Mid-Stream

Building the UI Pattern

Where to Go Next