Stateful Context & Tool Search — The Responses API

Persistent Conversations

Unlike Chat Completions where you manually manage message history, the Responses API can persist conversations server-side.

// First message
const r1 = await openai.responses.create({
  model: "gpt-6.1-sol",
  store: true,
  input: "My name is Alex and I'm building a SaaS app."
});

// Follow-up - references the previous response
const r2 = await openai.responses.create({
  model: "gpt-6.1-sol",
  store: true,
  previous_response_id: r1.id,
  input: "What tech stack would you recommend for my project?"
});

Tool Search & Deferred Loading

When you have dozens of function tools, namespaces, or MCP servers, loading all parameter schemas upfront wastes tokens. On GPT-5.4+ and GPT-6 models, enabling { type: "tool_search" } alongside defer_loading: true exposes only lightweight tool names/descriptions upfront and dynamically loads matching schemas at the end of context (preserving prompt cache prefixes).

const response = await openai.responses.create({
  model: "gpt-6.1-sol",
  tools: [
    { type: "tool_search" },
    { type: "function", name: "get_weather", description: "Fetch live city weather", parameters: { ... }, defer_loading: true },
    { type: "function", name: "book_flight", description: "Reserve airline seats", parameters: { ... }, defer_loading: true },
    // ... 50 more deferred functions or MCP servers
  ],
  input: "What's the weather in London?"
});