Skip to main content

LangChain and LangGraph

Routor works with LangChain because ChatOpenAI accepts a custom baseURL. You get intelligent routing with zero changes to your chains, agents, or tools. This guide walks through a complete agent built with LangGraph that uses Routor for every model call.

Setup

Install the LangChain packages you need:
npm install @langchain/openai @langchain/core langchain @langchain/langgraph
Create a Routor-backed model. The only differences from a normal ChatOpenAI instance are the baseURL and the apiKey:
import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  apiKey: process.env.ROUTOR_API_KEY,   // "openAIApiKey" on @langchain/openai < 0.3
  configuration: { baseURL: "https://api.routor.ai/v1" },
  modelName: "auto",
});
from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    api_key=os.environ["ROUTOR_API_KEY"],
    base_url="https://api.routor.ai/v1",
    model="auto",
)
Routor handles the model selection from here. Every invoke, stream, or tool call goes through the router.

A Complete LangGraph Agent

This example builds a research agent with LangGraph. The agent has two tools: a web search tool and a calculator. Routor detects the tool definitions and automatically routes to a tool-capable model ranked by function-calling accuracy.

Python

import os
from typing import Annotated, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langchain_core.messages import HumanMessage
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode

# ── Routor-backed model ──────────────────────────────────────────
model = ChatOpenAI(
    api_key=os.environ["ROUTOR_API_KEY"],
    base_url="https://api.routor.ai/v1",
    model="auto",
)

# ── Tools ────────────────────────────────────────────────────────
@tool
def search_web(query: str) -> str:
    """Search the web for up-to-date information."""
    # Replace with your search implementation (Tavily, SerpAPI, etc.)
    return f"Search results for: {query}"

@tool
def calculate(expression: str) -> str:
    """Evaluate a math expression and return the result."""
    try:
        return str(eval(expression))
    except Exception as e:
        return f"Error: {e}"

tools = [search_web, calculate]
model_with_tools = model.bind_tools(tools)

# ── Graph state ──────────────────────────────────────────────────
class AgentState(TypedDict):
    messages: Annotated[list, add_messages]

def agent_node(state: AgentState):
    response = model_with_tools.invoke(state["messages"])
    return {"messages": [response]}

def should_continue(state: AgentState):
    last_message = state["messages"][-1]
    if last_message.tool_calls:
        return "tools"
    return END

# ── Build the graph ──────────────────────────────────────────────
graph = StateGraph(AgentState)
graph.add_node("agent", agent_node)
graph.add_node("tools", ToolNode(tools))

graph.add_edge(START, "agent")
graph.add_conditional_edges("agent", should_continue, {"tools": "tools", END: END})
graph.add_edge("tools", "agent")  # loop back after tool execution

app = graph.compile()

# ── Run it ───────────────────────────────────────────────────────
result = app.invoke({
    "messages": [HumanMessage(
        content="Search for the latest Claude Opus pricing, then calculate the monthly cost for 5 million input tokens."
    )]
})

for msg in result["messages"]:
    print(f"[{msg.type}] {msg.content}")

Node.js / TypeScript

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage } from "@langchain/core/messages";
import { tool } from "@langchain/core/tools";
import { StateGraph, START, END, Annotation } from "@langchain/langgraph";
import { ToolNode } from "@langchain/langgraph/prebuilt";
import { z } from "zod";

// ── Routor-backed model ──────────────────────────────────────────
const model = new ChatOpenAI({
  apiKey: process.env.ROUTOR_API_KEY!,
  configuration: { baseURL: "https://api.routor.ai/v1" },
  modelName: "auto",
});

// ── Tools ────────────────────────────────────────────────────────
const searchWeb = tool(
  async ({ query }) => `Search results for: ${query}`,
  {
    name: "search_web",
    description: "Search the web for up-to-date information.",
    schema: z.object({ query: z.string() }),
  }
);

const calculate = tool(
  async ({ expression }) => {
    try { return String(Function(`"use strict"; return (${expression})`)()); }
    catch (e) { return `Error: ${e}`; }
  },
  {
    name: "calculate",
    description: "Evaluate a math expression and return the result.",
    schema: z.object({ expression: z.string() }),
  }
);

const tools = [searchWeb, calculate];
const modelWithTools = model.bindTools(tools);

// ── Graph state ──────────────────────────────────────────────────
const AgentState = Annotation.Root({
  messages: Annotation<any[]>({ reducer: (a, b) => [...a, ...b] }),
});

async function agentNode(state: typeof AgentState.State) {
  const response = await modelWithTools.invoke(state.messages);
  return { messages: [response] };
}

function shouldContinue(state: typeof AgentState.State) {
  const lastMessage = state.messages[state.messages.length - 1];
  return lastMessage.tool_calls?.length ? "tools" : END;
}

// ── Build the graph ──────────────────────────────────────────────
const graph = new StateGraph(AgentState)
  .addNode("agent", agentNode)
  .addNode("tools", new ToolNode(tools))
  .addEdge(START, "agent")
  .addConditionalEdges("agent", shouldContinue, { tools: "tools", END })
  .addEdge("tools", "agent")
  .compile();

// ── Run it ───────────────────────────────────────────────────────
const result = await graph.invoke({
  messages: [new HumanMessage(
    "Search for the latest Claude Opus pricing, then calculate the monthly cost for 5 million input tokens."
  )],
});

for (const msg of result.messages) {
  console.log(`[${msg._getType()}] ${msg.content}`);
}

How Routor Handles the Agent

When the agent sends a request with tools attached, Routor:
  1. Detects the tools array in the request
  2. Classifies the prompt difficulty and task category
  3. Narrows the model pool to only tool-calling models
  4. Ranks them by BFCL function-calling accuracy for the detected scenario (single, parallel, or multi-turn)
  5. Picks the best-value model and falls back silently if it fails
You do not configure any of this. The tools are detected automatically from the request body.

Using a Routing Profile

For a production agent, create a Routing Profile in the dashboard with constraints like a tier floor or cost cap. Each profile gets its own API key, so the profile rules apply with no code changes:
# Production agent key - enforces STANDARD tier minimum
model = ChatOpenAI(
    api_key=os.environ["ROUTOR_AGENT_KEY"],
    base_url="https://api.routor.ai/v1",
    model="auto",
)
// Experimental agent key - allows NANO tier for testing
const model = new ChatOpenAI({
  apiKey: process.env.ROUTOR_TEST_KEY,
  configuration: { baseURL: "https://api.routor.ai/v1" },
  modelName: "auto",
});

Reading Routing Metadata

LangChain only surfaces the standard OpenAI response fields, so the full routor object from the response body is not exposed through response_metadata or additional_kwargs. What you can read in LangChain is the actual model that handled the request:
response = model.invoke([HumanMessage("explain closures")])
print(response.response_metadata["model_name"])  # e.g. "deepseek-v4-flash"
const response = await model.invoke([new HumanMessage("explain closures")]);
console.log(response.response_metadata.model_name);  // e.g. "deepseek-v4-flash"
For the full routing decision (tier, confidence, savings), check the Logs page in the dashboard, call the API directly with the OpenAI SDK (the response body includes a routor object), or use the Debug Endpoint to inspect a decision without spending credits.

Streaming

LangGraph supports streaming token-by-token. Routor passes through the stream in OpenAI SSE format, so it works without changes:
for chunk in model.stream([HumanMessage("write a short poem about routing")]):
    print(chunk.content, end="", flush=True)
const stream = await model.stream([new HumanMessage("write a short poem about routing")]);
for await (const chunk of stream) {
  process.stdout.write(chunk.content as string);
}